Also attempt to handle this nicer by ignoring the truncate failure when
it doesn't matter, and recover by deleting the temp file when it does.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4594
This keeps the data we need about sequence numbers and object counts
persistently in the database. The sizeTracker is expanded into a
metadataTracker than handled multiple folders, and the Counts struct is
made protobuf serializable. It gains a Sequence field to assist in
tracking that as well, and a collection of Counts become a CountsSet
(for serialization purposes).
The initial database scan is also a consistency check of the global
entries. This shouldn't strictly be necessary. Nonetheless I added a
created timestamp to the metadata and set a variable to compare against
that. When the time since the metadata creation is old enough, we drop
the metadata and rebuild from scratch like we used to, while also
consistency checking.
A new environment variable STCHECKDBEVERY can override this interval,
and for example be set to zero to force the check immediately.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4547
LGTM: imsodin
So STDEADLOCK seems to do the same thing as STDEADLOCKTIMEOUT, except in
the other package. Consolidate?
STDEADLOCKTHRESHOLD is actually called STLOCKTHRESHOLD, correct the help
text.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4598
Two issues since the filesystem migration;
- filepath.Base() doesn't do what the code expected when the path ends
in a slash, and we make sure the path ends in a slash.
- the return should be fully qualified, not just the last component.
With this change it works as before, and passes the new test for it.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4591
LGTM: imsodin
Fix the folder restart behavior (ignore Label), improve the API for that
(imho).
Also removes the tab switch animation in the settings modal, because
annoying.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4577
* release:
lib/connections: Trust the model to tell us if we are connected
build: More signatures, more better (ref #3420)
lib/model: Trigger a pull when ignore patterns change
We need to reset prevSeq so that we force a full check when someone
reconnects - the sequence number may not have changed due to the
reconnect. (This is a regression; we did this before f6ea2a7.)
Also add an optimization: we schedule a pull after scanning, but there
is no need to do so if no changes were detected. This matters now
because the scheduled pull actually traverses the database which is
expensive.
This, however, makes the pull not happen on initial scan if there were
no changes during the initial scan. Compensate by always scheduling a
pull after initial scan in the rwfolder itself.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4508
LGTM: imsodin, AudriusButkevicius
This is step one of a hundred fifty on the path to case insensitivity.
It brings in the basic case folding mechanism and adds it to the
mtimefs, as this is something outside the fileset that touches stuff in
the database based on name. No effort to convert or handle existing
entries when the insensitivity is changed, I don't think we need it...
Useless by itself but includes tests and will reduce the review load
along the way.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4521
This makes it OK to not have any listeners working. Specifically,
- We don't complain about an empty listener address
- We don't complain about not having anything to announce to global
discovery servers
- We don't send local discovery packets when there is nothing to
announce.
The last point also fixes a thing where the list of addresses for local
discovery was set at startup time and never refreshed.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4517
Diff is large due to comment reformatting and indentation but all it
does is wrap the file mtime/size/permissions check in an "if
stat.IsRegular()".
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4507
This removes a significant, complex chunk of database code. The
"replace" operation walked both the old and new in lockstep and made the
relevant changes to make the new situation correct. But since delta
indexes we pretty much never need this - we just used replace to drop
the existing data and start over.
This makes that explicit and removes the complexity.
(This is one of those things that would be annoying to make case
insensitive, while the actual "drop and then insert" that we do is
easier.)
This is fairly well unit tested...
The one change to the tests is to cover the fact that previously replace
with something identical didn't bump the sequence number, while
obviously removing everything and re-inserting does. This is not
behavior we depend on anywhere.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4500
LGTM: imsodin, AudriusButkevicius
With VPNs and stuff we can get a single failure on an interface that
supposedly supports broadcasts without it being fatal.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4415
The folder marker conversion forgot to hide the .stfolder. This adds
that, for those who have not yet been converted.
Also adds Hide() calls to the folder start, to mend historical
unhidedness. (I'm sure this will upset someone who is manually managing
their .stignores in the other direction...)
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4384
When STHASHING is set, don't benchmark as it's already decided. If weak
hashing isn't set to "auto", don't benchmark that either.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4349
These functions were very naive and slow. We haven't done much about
them because they pretty much don't matter at all for Syncthing
performance. They are however called very often in the discovery server
and these optimizations have a huge effect on the CPU load on the
public discovery servers.
The code isn't exactly obvious, but we have good test coverage on all
these functions.
benchmark old ns/op new ns/op delta
BenchmarkLuhnify-8 12458 1045 -91.61%
BenchmarkUnluhnify-8 12598 1074 -91.47%
BenchmarkChunkify-8 10792 104 -99.04%
benchmark old allocs new allocs delta
BenchmarkLuhnify-8 18 1 -94.44%
BenchmarkUnluhnify-8 18 1 -94.44%
BenchmarkChunkify-8 44 2 -95.45%
benchmark old bytes new bytes delta
BenchmarkLuhnify-8 1278 64 -94.99%
BenchmarkUnluhnify-8 1278 64 -94.99%
BenchmarkChunkify-8 42552 128 -99.70%
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4346
Prior to this, the following is possible:
- Create a symlink "foo -> /somewhere", it gets synced
- Delete "foo", it gets versioned
- Create "foo/bar", it gets synced
- Delete "foo/bar", it gets versioned in "/somewhere/bar"
With this change, versioners should never version symlinks.
As of Go 1.8 it's valid to not have a GOPATH set. We ask the Go tool
what the GOPATH seems to be, and if we find ourselves (or a valid copy
of ourselves...) in that location we do not fiddle with the GOPATH.
Jakob Borg