| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`ruff check .` had gone unrun long enough to report 568 errors, which is the
same as having no linter: the next real finding would have been invisible in the
noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the
98 it cannot fix are the next commit.
What actually changed: import sorting (225), imports nobody used (87, none of
them a re-export — no `__init__.py` is touched, which was the one way this could
have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69)
and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12
this project requires, `Optional[X]` to `X | None` (24), and f-strings with
nothing to interpolate (19).
Checked rather than assumed: every module in the three packages still imports,
and the suite is 2893 passed — the same count, test for test, as the merge
before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`IndexProgress` said "scanning, this many bytes of that many" and nothing
more. A group's roots are walked one after another, so a second directory
added during a large scan showed as the bar jumping back to 0 %. It now also
carries the root being walked and its position in the roots table, the kind
of walk (scan, rescan, reconcile, watch), file counts, and the roots
waiting for the scan lock in order: queued by the initial scan, by a
retarget, and by a plug; dropped when a root is removed.
`GET /api/index-status` answers for every group at once, including a group
still in its initial scan, so a client can show indexing on any page. It
names roots: loopback only, like `current_dir`.
`index_progress` and the handshake ack gain the same counters, still naming
nothing (decision D3): the root is a position in the roots table the member
already opened from the sealed index, and the queue is a count. The pusher
keeps speaking while a root only waits for the lock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
|
|
reconcile, and delta sync
Indexer performance work, in four parts:
- Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node
restart no longer re-hashes every file — measured at 23 minutes for a
114 GB library on a slow disk before this, near-instant after. Hashing
is deliberately kept sequential (max_workers=1): it was never actually
concurrent despite the pool size, and two interleaved reads seek-thrash
a spinning disk instead of going faster.
- Byte-based scan progress (IndexProgress), surfaced via the loopback
index-status route, the handshake ack, and a periodic INDEX_PROGRESS
push to connected peers — drives a progress bar in the Create Group
wizard and "add a directory" in Settings, and an animated presence dot.
Guaranteed to settle back to idle via try/finally and a final push on
the scanning->false transition.
- The reconcile backstop's directory walks now run in the executor
instead of blocking the daemon's event loop; its interval defaults to
10 min (was 60s) with adaptive backoff to 2h when nothing changes,
reset on a real change or a peer connecting, and is now a per-group
operator setting (signed op + group Settings UI).
- INDEX_DELTA wired up (protocol support existed, nothing called it):
_on_index_change now sends additions/deletions instead of rebuilding
the full entries list, coalesced over a short window so a burst of
file events produces one push, and the hub swarm registration for
public groups only (re-)registers newly added hashes.
Also fixes several bugs found while testing the above against real
libraries (a 114 GB and a 100+ GB group on a USB HDD):
- /api/reload blocked until the reload — including a brand-new group's
full initial scan — finished, which the Electron bridge's fixed 30s
call timeout turned into a hard failure on any real library. The route
now fires the reload without waiting (ops.start_reload), matching
add_root/remove_root's existing pattern; the wizard's own step order
was fixed to wait for the group to actually appear hosted before the
steps that need it (extra roots, GEK), with retries for the residual
race between that and the daemon's own bookkeeping.
- transport.js's hand-rolled msgpack codec had no case for uint64/int64
(0xcf/0xd3) and crashed decoding any message containing one — hit by
IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB.
Verified against real msgpack-encoded bytes from the Python side.
- chat_hist_resp, and this change's own index_progress and
set_scan_settings_ack pushes, were not routed by message type and
could be handed to an unrelated pending request by the transport's
"oldest pending" fallback, stalling it until its own 30s timeout and
corrupting whatever received the wrong reply in its place.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|