250 Commits

Author SHA1 Message Date
Daniel Harte
e71c78ae84 cmd/syncthing: Remove folder limit on /rest/system/browse
Previously limited to 10 results, now unlimited.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3279
LGTM: calmh, AudriusButkevicius
2016-06-08 07:09:50 +00:00
Jakob Borg
03a8027efc cmd/syncthing: Refactor out staticsServer (prev. embeddedStatic) a bit
The purpose of this operation is to separate the serving of GUI assets a
bit from the serving of the REST API. It's by no means complete. The end
goal is something like a combined server type that embeds a statics
server and an API server and wraps it in authentication and HTTPS and
stuff, plus possibly a named pipe server that only provides the API and
does not wrap in the same authentication etc.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3273
2016-06-07 07:46:45 +00:00
Jakob Borg
5aacfd1639 cmd/syncthing: Make API serve loop more robust (fixes #3136)
This sacrifices the ability to return an error when creating the service
for being more persistent in keeping it running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3270
LGTM: AudriusButkevicius, canton7
2016-06-06 22:12:23 +00:00
Audrius Butkevicius
1612baca92 gui: /rest/system/browse with no arguments returns drives on Windows (ref #3201)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3203
2016-05-31 19:27:07 +00:00
Antony Male
6361172bea cmd/syncthing: Be more explicit about how assets should be cached (fixes #3182)
With the previous setup, browsers were free to use a local cache for any
length of time they pleased: we didn't set an 'Expires' header (or max-age
directive), and Cache-Control just said "you're free to cache this".

Therefore be more explicit: we don't mind if browsers cache things, but they
MUST revalidate everything on every request.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3221
2016-05-30 13:54:55 +00:00
Jakob Borg
da5010d37a cmd/syncthing: Use API to generate API Key and folder ID (fixes #3179)
Expose a random string generator in the API and use it when the GUI
needs random strings for API key and folder ID.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3192
2016-05-26 07:25:34 +00:00
Jakob Borg
e6b78e5d56 lib/rand: Break out random functions into separate package
The intention for this package is to provide a combination of the
security of crypto/rand and the convenience of math/rand. It should be
the first choice of random data unless ultimate performance is required
and the usage is provably irrelevant from a security standpoint.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3186
2016-05-26 07:02:56 +00:00
Jakob Borg
98734375f2 cmd/syncthing: Correctly set, parse and compare modified time HTTP headers (fixes #3165)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3167
2016-05-23 12:16:14 +00:00
Jakob Borg
4bc2b3f369 gui: Set CSRF stuff earlier (fixes #3138)
We need to set these properties *before* Angular starts making requests,
and doing that from the response to a request is too late. The obvious
choice (to me) would be to use the angular $cookies service, but that
service isn't available until after initialization so we can't use it.
Instead, add a special file that is loaded by index.html and includes
the info we need before the JS app even starts running.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3152
2016-05-22 10:26:09 +00:00
Jakob Borg
bf7fcc612d cmd/syncthing: Enforce stricter CSRF policy on /rest GET requests (fixes #3134)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3137
2016-05-21 13:48:55 +00:00
klemens
bd41e21c26 all: Correct spelling in comments
Skip-check: authors pr-build-mac

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3056
2016-05-08 10:54:22 +00:00
Jakob Borg
39899e40bf cmd/syncthing: Use ReadAll + json.Unmarshal in places were we care about consuming the reader
Because json.NewDecoder(r).Decode(&v) doesn't necessarily consume all
data on the reader, that means an HTTP connection can't be reused. We
don't do a lot of HTTP traffic where we read JSON responses, but the
discovery is one such place. The other two are for POSTs from the GUI,
where it's not exactly critical but still nice if the connection still
can be keep-alive'd after the request as well.

Also ensure that we call req.Body.Close() for clarity, even though this
should by all accounts not really be necessary.

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/3050
2016-05-06 22:01:56 +00:00
Audrius Butkevicius
674fc566bb lib/connections: Refactor
1. Removes separate relay lists and relay clients/services, just makes it a listen address
2. Easier plugging-in of other transports
3. Allows "hot" disabling and enabling NAT services
4. Allows "hot" listen address changes
5. Changes listen address list with a preferable "default" value just like for discovery
6. Debounces global discovery announcements as external addresses change (which it might alot upon starting)
7. Stops this whole "pick other peers relay by latency". This information is no longer available,
   but I don't think it matters as most of the time other peer only has one relay.
8. Rename ListenAddress to ListenAddresses, as well as in javascript land.
9. Stop serializing deprecated values to JSON

GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2982
2016-05-04 19:38:12 +00:00
AudriusButkevicius
1a5f524ae4 lib/model, lib/protocol: Implement temporary indexes (fixes #950)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2252
2016-04-15 10:59:41 +00:00
Jakob Borg
52c7804f32 lib/connections: Silence vet and lint warnings
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/2971
2016-04-13 11:50:51 +00:00
Laurent Etiemble
7b4e1e9055 cmd/syncthing: Fix handler ordering so CORS middleware wraps all the others but the debug one 2016-04-03 13:24:55 +02:00
Jakob Borg
4c3cd4c9e3 lib/ignore: Replace lib/fnmatch with github.com/gobwas/glob
Because it's literally ten times faster:



	benchmark                  old ns/op     new ns/op     delta

	BenchmarkMatch-8           13842         1200          -91.33%

	BenchmarkMatchCached-8     139           147           +5.76%



	benchmark                  old allocs     new allocs     delta

	BenchmarkMatch-8           0              0              +0.00%

	BenchmarkMatchCached-8     0              0              +0.00%



	benchmark                  old bytes     new bytes     delta

	BenchmarkMatch-8           12            0             -100.00%

	BenchmarkMatchCached-8     0             0             +0.00%
2016-04-02 20:03:24 +01:00
Audrius Butkevicius
29913dd1e4 lib/connections: Refactor address listing into connection service 2016-03-25 07:35:18 +00:00
Jakob Borg
a492cfba13 cmd/syncthing: Extract interfaces for things the API depends on
Enables testing of the API service, in the long run.
2016-03-21 19:36:08 +00:00
Jakob Borg
9dec6f1324 cmd/syncthing: Return 500 with an error object instead of empty 200 on marshalling failure in REST response 2016-03-20 11:54:53 +01:00
Jakob Borg
ba5231dc89 apiService should not reference global variable 'locations' (hinders testing) 2016-03-13 11:03:00 +01:00
Jakob Borg
032365d57c Fix STGUIASSETS search paths & order (fixes #2827) 2016-03-12 12:17:25 +00:00
Audrius Butkevicius
3b146eda0d Clarify GUI stuff (fixes #2819) 2016-03-06 22:07:15 +00:00
Audrius Butkevicius
bf353a42cd Merge pull request #2780 from letiemble/CORS_Support2
Move CORS middleware to process un-authenticated OPTIONS requests
2016-02-12 21:29:45 +00:00
Laurent Etiemble
d8e19b776e Swap the corsMiddleware and the csrfMiddleware to the unauthenticated OPTIONS requests are first processed. 2016-02-12 22:10:08 +01:00
Jakob Borg
3c7164846d Return "No such object in the index" when /rest/db/file gets called on something that doesn't exist
Better than the confusing result of getting a blank fileinfo that looks
valid apart from being all crap.
2016-02-12 14:55:16 +01:00
Jakob Borg
a7a9d7d85c Return correct content type for /rest/events 2016-02-02 12:40:42 +01:00
Antony Male
5971c00a4f Support multiple API keys (command-line and config) (fixes #2747) 2016-01-30 15:18:09 +00:00
Jakob Borg
fae68a5396 Return status code 307 instead of 302 when redirecting from HTTP to HTTPS 2016-01-29 11:07:51 +01:00
Laurent Etiemble
fed374fcb6 Add a CORS handler to deal with preflight OPTIONS requests 2016-01-26 21:55:51 +01:00
Jakob Borg
11d4986517 Humanize serialization of version vectors (again) 2016-01-20 11:14:08 -08:00
Jakob Borg
97b1c66d4a Improve API/GUI shutdown handling (fixes #2694)
This fixes both a race condition where we could assign s.stop from one
goroutine and then read it from another without locking, and handles the
fact that listener may be nil at shutdown if we've had a bad
CommitConfiguration call in the meantime.
2016-01-14 11:06:36 +01:00
Audrius Butkevicius
cd54186113 Add support for themes (fixes #1925) 2016-01-10 17:57:27 +00:00
Audrius Butkevicius
1827dda0c6 Svc -> Service 2015-12-23 15:31:12 +00:00
Jakob Borg
6f95afdc59 Humanize serialization of version vectors 2015-12-22 21:53:25 +01:00
Anderson Mesquita
316be5ee34 Add helper function to format JSON responses
Every time a JSON object is returned in an HTTP response, the
appropriate header needs to be set and the object itself needs to be
encoded. Doing this in every function is repetitive and error prone
(getDBFile and postDBScan, for instance, never set any headers).

This adds a helper function to centralize the appropriate JSON response
handling.
2015-12-16 09:06:17 +01:00
Antony Male
1340e54327 Only print 'Enabled/Disabled debug data for ...' if it was enabled/disabled 2015-12-06 17:15:55 +00:00
Jakob Borg
a6a573f5dc Fix STTRACE=http (it should use the http debug logger) 2015-12-05 15:59:35 +01:00
Jakob Borg
415415b5b2 Response with Bad Request to bad config posts, not Internal Server Errors
The error is the clients fault, or so we think, so we should not accept
the blame.
2015-12-05 15:56:10 +01:00
Jakob Borg
6d11006b54 Generate ECDSA keys instead of RSA
This replaces the current 3072 bit RSA certificates with 384 bit ECDSA
certificates. The advantage is these certificates are smaller and
essentially instantaneous to generate. According to RFC4492 (ECC Cipher
Suites for TLS), Table 1: Comparable Key Sizes, ECC has comparable
strength to 3072 bit RSA at 283 bits - so we exceed that.

There is no compatibility issue with existing Syncthing code - this is
verified by the integration test ("h2" instance has the new
certificate).

There are browsers out there that don't understand ECC certificates yet,
although I think they're dying out. In the meantime, I've retained the
RSA code for the HTTPS certificate, but pulled it down to 2048 bits. I
don't think a higher security level there is motivated, is this matches
current industry standard for HTTPS certificates.
2015-11-27 09:15:12 +01:00
Jakob Borg
ff8a66d22f Add metrics for HTTP calls 2015-11-21 09:48:57 +01:00
Jakob Borg
f7ad97918a HTTP debug middleware should be behind ShouldDebug() 2015-11-21 09:39:40 +01:00
Jakob Borg
9df701906f Run JSON decoding through the usual setting of defaults and fixing up
I see no reason not to do this, and it gives a unified place (the prepare()
call) to initialize cached attributes and so on.
2015-11-04 20:33:10 +00:00
Jakob Borg
7c79985a29 Clarify listen address 2015-10-13 22:07:22 +09:00
Jakob Borg
953a67bc3a Option -gui-address should accept scheme prefixes (fixes #2371) 2015-10-13 08:26:07 +09:00
Jakob Borg
76af9ba53d Implement facility based logger, debugging via REST API
This implements a new debug/trace infrastructure based on a slightly
hacked up logger. Instead of the traditional "if debug { ... }" I've
rewritten the logger to have no-op Debugln and Debugf, unless debugging
has been enabled for a given "facility". The "facility" is just a
string, typically a package name.

This will be slightly slower than before; but not that much as it's
mostly a function call that returns immediately. For the cases where it
matters (the Debugln takes a hex.Dump() of something for example, and
it's not in a very occasional "if err != nil" branch) there is an
l.ShouldDebug(facility) that is fast enough to be used like the old "if
debug".

The point of all this is that we can now toggle debugging for the
various packages on and off at runtime. There's a new method
/rest/system/debug that can be POSTed a set of facilities to enable and
disable debug for, or GET from to get a list of facilities with
descriptions and their current debug status.

Similarly a /rest/system/log?since=... can grab the latest log entries,
up to 250 of them (hardcoded constant in main.go) plus the initial few.

Not implemented in this commit (but planned) is a simple debug GUI
available on /debug that shows the current log in an easily pasteable
format and has checkboxes to enable the various debug facilities.

The debug instructions to a user then becomes "visit this URL, check
these boxes, reproduce your problem, copy and paste the log". The actual
log viewer on the hypothetical /debug URL can poll regularly for new log
entries and this bypass the 250 line limit.

The existing STTRACE=foo variable is still obeyed and just sets the
start state of the system.
2015-10-03 18:09:53 +02:00
Jakob Borg
15716a0772 Fix STGUIAPIKEY and STGUIADDR overrides (fixes #2335)
Also removes STGUIAUTH and corresponding --gui-authentication as this
seems fundamentally insecure and I'm unsure of the actual use case for
it?
2015-09-30 09:36:11 +02:00
Jakob Borg
36ac757c3a Remove global cfg variable (fixes #2294)
Not necessarily the easiest way to fix just this bug, but the root cause
was using the (at that point uninitialized) cfg variable, so it seemed
sensible to just get rid of it to avoid that kind of crap.
2015-09-29 20:23:15 +02:00
Jakob Borg
4581c57478 Fix import paths 2015-09-22 19:38:46 +02:00
Jakob Borg
b0cd7be39b New global discovery protocol over HTTPS (fixes #628, fixes #1907) 2015-09-20 21:10:53 +02:00