meshbay.git/packages/meshbay-node/src/meshbay_node/indexer/indexer.py, branch 0.20

fix(node): index, serve and write files past Windows' MAX_PATH

2026-10-14T09:17:38Z

With the system's default setting (LongPathsEnabled off) every call given a plain path of 260 characters or more refuses it, so a deep library was partly invisible: the walk could not list such a directory and its files never reached the members. The node now hands each call that touches a group's disk the extended form (paths.long_path), and keeps the plain path in the index, the hash cache and the log. A directory is listed under its long form and its entries named from the plain one (indexer.walk_files, wire._walk_dirs). Covered: the walk, stats and hashing, reconcile and the watcher, the folder list members see, serving over WebRTC and QUIC, deletion, uploads and their recovery, folder creation and removal, audio tags and the cover beside a track, photos, and a show's episode numbering. ffmpeg extends a long path itself and keeps the plain one. long_path now normalises before prefixing (the extended form skips the system's normalisation) and starts at 248 characters, where creating a directory is already refused. test_long_paths.py runs with the setting off: 15 of its 17 tests fail without the change. Checked by hand in the installed app with files at 312 to 380 characters. Co-Authored-By: Claude Opus 5.5

feat(node): skip a file the disk stops answering for, then pause its root

2026-10-14T08:17:49Z

A Windows tester's scan stuck for good on one directory of a large disk: one read that never returned held the indexer's only thread, and with it every root behind that one. - A disk call is timed by its silence, not its length: each directory entry listed and each megabyte read counts as an answer. After 60 s of silence the call is abandoned on its daemon thread, the file is noted unread and the scan goes on. - Three silent files in a row pause the root (not persisted). Resume skips the files already known to give no answer; a restart, a replug or a change to the file retries them. - The walk is per directory with os.scandir: a subdirectory that cannot be listed costs itself, and a listing that stops answering pauses the root at once. A scan no longer stats anything from the event loop. - The log names the file after 10 s of silence, the indexing band shows it, and Group settings shows the paused root, where it stuck and the unread files with their reason, on the node's machine only. - generate_http_api.py writes LF on Windows too. Verified with real kernel-blocked opens (an unacknowledged oplock) in the installed frozen app. Co-Authored-By: Claude Opus 5.5

feat: let the operator pause and resume a root's indexing (MNP 6.2)

2026-10-11T22:17:32Z

A whole-root walk now checks its root before every file: removed or ejected, it stops instead of reading the drive to the end; paused, it finishes from the hash cache alone. A paused root is not watched, not swept by reconcile and survives a restart (roster), its indexed files stay served. Resume runs the reconcile sweep over the root, so entries and their enrichment are kept. Signed root_pause / root_resume over MNP, loopback routes, CLI `root pause|resume`, a pause button on the indexing dock and pause/resume in the roots table of Group settings. Co-Authored-By: Claude Opus 5.5

perf(node): sample 9 MB with the size above 9 MB, keep known ids

2026-10-07T23:27:58Z

hash_version 3: size + first 4 MB + last 4 MB + 1 MB at the middle, 5.5x faster cold on a USB disk than the 45 MB sample. The cache now serves a hit under whatever version it holds, so no existing id moves. Co-Authored-By: Claude Opus 5.5

fix(node): drop a directory moved out of the root on Windows too

2026-10-07T20:22:51Z

On Windows watchdog cannot tell what a deleted path was -- it is gone -- and reports a directory moved out of the root as a file deleted, so the fix in 3d5a168 never ran there and test_the_watcher_reports_a_directory_moved_out failed. A deleted path the index still holds entries below was a directory. Co-Authored-By: Claude Opus 5.5

fix(node): drop a directory moved out of the root from the index

2026-10-07T10:48:34Z

Watchdog reports such a move as one "directory deleted" event and nothing for the files, which the indexer ignored until the next reconcile. The freeze rules still apply: root live, directory gone, parent present. Co-Authored-By: Claude Opus 5.5

fix(node): plug an auto-ejected removable root back once its files return

2026-10-05T09:10:50Z

A node started with the desktop session runs before the session has mounted its USB drives. The safety net then auto-ejected every removable root and persisted it exactly like an operator's eject, so after each reboot those roots stayed ejected until someone plugged them by hand (seen on a node whose /media drives were mounted a minute after it started). An auto-eject is now stored as such ("auto" in roster.db). At startup and at every reconcile, an auto-ejected root whose path is readable again is checked against a few files the hash cache knows under it, at the same path with the same size and mtime; one found and the root is plugged back and rescanned. An empty mount point or another drive in its place is not recognised and stays ejected. An operator's eject is never undone automatically. Co-Authored-By: Claude Opus 5.5

feat(mnp): sharing a folder is decided on the node's machine only (MNP 6.0)

2026-10-02T08:20:09Z

root_add, root_update and group_attach leave MNP: adding a directory and switching writable/removable go through the loopback API (native dialog in the desktop app) or the CLI. The operator's Settings tab still lists the roots from any browser, read-only. The desktop app refuses to sign those ops; a loopback flag change now reaches open pages (publish_roots). Co-Authored-By: Claude Opus 5.5

style: ruff's own fixes, mechanically applied

2026-09-19T12:24:13Z

`ruff check .` had gone unrun long enough to report 568 errors, which is the same as having no linter: the next real finding would have been invisible in the noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the 98 it cannot fix are the next commit. What actually changed: import sorting (225), imports nobody used (87, none of them a re-export — no `__init__.py` is touched, which was the one way this could have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69) and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12 this project requires, `Optional[X]` to `X | None` (24), and f-strings with nothing to interpolate (19). Checked rather than assumed: every module in the three packages still imports, and the suite is 2893 passed — the same count, test for test, as the merge before it. Co-Authored-By: Claude Opus 5

docs: cite MESHBAY_DESIGN.md and a section instead of the merged notes

2026-09-19T00:12:47Z

The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and deleted from the tree on 2026-09-11, but ~230 comments across the three packages still named them — usually written `docs/musicbay.md §3.2`, as though the file were still in docs/. A reader had to know §16 existed to resolve any of them. They now name the section directly. Every mapping comes from §16, the concordance, which already records where each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the Videos app and §6.5 where the subject is derived data, photos -> §9.9, auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 -> §3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys -> §4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section. Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`) are retargeted too: those collide with real section numbers in the design document, so leaving them would have been worse than the named form. Four cases the concordance does not cover, each decided rather than guessed: Sub-item references into documents that no longer exist — mediacenter's `§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and §9.2 do not reproduce. The module-level citation stays; the sub-item pointer is dropped. The V-findings keep their labels but lose the dead `§10.1/` prefix. §13.8 lists V1–V13 as per-application open items, which is not what the labels mean in these comments, so pointing them at §13.8 would have been a false citation. `apps.md §5`'s virtualization requirement has no counterpart anywhere in the design document. The requirement is stated in the comment itself, so the citation is dropped rather than aimed at a section that does not say it. Comments that attributed a *sentence* to an old note — musicbay's "several thousand files" example, its "what got measured" note, its measured ~11%/~26% cover-art figures, the "original no root, whole shared tree" call — state the fact without attribution now. §9.8 does not contain those sentences and citing it for them would have been wrong. CLAUDE.md's "a reference to a document that no longer exists" row now says the concordance is for git history and out-of-tree material; the code cites sections directly. Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox leg of test_sticky_header.py's browser harness, which is broken at the browser level on this machine — headless Firefox (snap) dies with `[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders nothing, and the probe exits `{"error": "no measurement"}` after its full 90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing here can affect it: every changed line in style.css is inside a comment. Also checked: ast.parse on every changed .py, `node --check` on every changed .js, the /* */ balance in style.css, and that no changed line exceeds the width its file already used. Co-Authored-By: Claude Opus 5