meshbay.git/packages/meshbay-node/tests/test_ops.py, branch 0.16

test(node): read ops wherever it is split

2026-09-24T23:17:32Z

The tests that read ops.py for an absence (fastapi, a TOML path without as_posix, the reference app's name) and the one counting operations by module now take every file of ops.py or the ops package. Co-Authored-By: Claude Opus 5.5

fix(node): take the availability poll and every upload write off the loop

2026-09-18T13:59:24Z

The rest of AV9's disk half. Serving a file left the loop in the commit before this one; two paths were still on it. **The availability poll.** `RootSet.refresh_availability` stats every root, and eleven call sites reached it from `async def` — the reconcile loop among them, on a timer. On a sleeping disk that is a stall once per tick, and the stat is also what keeps the disk awake, so a node paid spin-up for a library nobody was reading. All eleven now go through `off_disk`, `Root.is_live` included. **The upload write.** `open`/`write`, and the resolve, the stat, the free-name search, the rename and the unlink around it. This one could not simply be awaited: the handler was synchronous, so nothing could come between the `chunk_index != state.next_index` check and the `advance` that answers it, and that is the whole of the chunk-ordering rule. Awaiting the write opens the gap — chunk 1 arriving while chunk 0 is in the disk thread reads a position that has not moved and is refused as out of order, so an upload would fail on a slow disk and nowhere else. Verified, not assumed: without the lock the new ordering test refuses three chunks of four. So the check, the write and the advance are one critical section again, under a lock held **per group**. Not per session: `partial_uploads` lives in the group context so a reconnecting client finds its upload where it left it, which means two sessions of one member share the position of one `.part` file. Arrival order is preserved by construction — the dispatcher creates one task per message as it arrives, tasks start in creation order, and the lock is the first thing each one waits on, so its waiters queue in arrival order too. `_do_file_upload` is a coroutine now, which is why forty-two test call sites gain an `await`. Their outcomes are unchanged, file by file, against the run before the change. `test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`, imported rather than defined there. It now asks for the ones written in the module, which is what its own docstring means; all forty-three operations are still checked. Co-Authored-By: Claude Opus 5

test(node): the backslash-path test modelled the wrong platform

2026-09-06T15:58:27Z

It asserted `Path(win_dir).as_posix() == "C:/Users/alice/Media"`, which is only true where `Path` is a `WindowsPath`. On every other machine a backslash is an ordinary filename character, `as_posix()` converts nothing, and the test failed against correct code — so it has never passed on this suite's usual host, and never guarded anything there. `PureWindowsPath` names the flavour and makes it the same assertion on all three platforms. The round trip also only ever proved that `as_posix()` produces a parseable string, never that the config writer calls it — which is the defect, and one no Linux machine can reproduce: the file is written, parsed and served correctly here and fails on the operator's Windows box. A second test reads ops.py for every f-string landing on the right of a TOML `path =` and requires `as_posix()` in it. Weak evidence, and the only kind available for a platform the suite does not run on; checked to fail with the call removed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us

fix(groups): finish Phase 1 — MNP root management, upload targets, eject state

2026-09-06T15:48:36Z

Review of the Phase 1 commit found the RO/RW model sound but three paths unfinished, each of which broke the flow the phase exists to deliver. Plus 29 test failures it introduced and no coverage for anything it added. Uploads went to the wrong directory. The node read a `root` field on file_upload that no client ever sent, so every upload landed in the first writable root while the Files toolbar offered its button based on the root being browsed — with two writable roots, uploading from one wrote into the other. Files now names the root it is showing; Chat names one chosen in the shell (an operator-configured directory arrives in Phase 2); the node refuses an unknown name rather than falling back, and refuses read-only and ejected roots by code. Shared directories were unreachable on the web. The table read its roots only from the loopback API, which resolves to "not available" in a browser, so the section rendered for nobody there — while the Uploads controls it replaced had worked — and the transport.updateRoot/ejectRoot/plugRoot methods beside it were dead. MNP is now the path, loopback the fallback for a local node with no live connection, and adding a root over MNP takes a typed path since no web page can browse a remote disk. Ejecting updated nobody's screen. transport.js resolves an admin ack against the pending request and returns, which is right for every op whose caller knows the value it chose; the root acks carry state only the node can compute, so the operator who clicked Eject was the one client that never saw it happen. And the ejected flag reached roster.db but was never read back, so a restart undid it and the next scan read an empty mount point as an erased library. Also: the member-upload endpoint answered 200 and did nothing (removed); the wizard ignored the first root's RW switch; reload compared roots on name and path, so editing writable in node.toml did nothing; the table had no path column, which is the only thing separating two libraries sharing a basename; apps_enabled normalisation differed between the two sides of a signed subject. Tests: eject/plug, per-root upload refusal and the node.toml rewrite had no coverage at all. test_member_upload_policy.py is replaced by test_root_writable_policy.py — it tested a removed feature — and every property worth keeping from it moved rather than being dropped. Docs: draft-v6 structural decision 9 is annotated as superseded (the operator can no longer have a directory only they may write to — a real capability removed, flagged rather than hidden), the man page documents the root verb and the RO/RW fields, and refactor-groups.md §7b records what the plan got wrong. Suite: 41 failures before, 13 after — all 13 pre-existing on main. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us

fix(node): make init, node.toml editing and CLI output work on Windows

2026-09-04T00:35:41Z

Found by running the daemon on Windows for the first time: - `meshbay-node init` wrote `unlock_file = "C:\Users\..."`, and attach_group / add_root write `path = "C:\..."` — a raw Windows path in a TOML basic string is a parse error (`\U`, `\a`, ... are escape sequences), so the config would not load. All now write `Path(...).as_posix()`; pathlib reads the forward-slash form fine on Windows. - any `print()` carrying a `->` arrow or em dash (the CLI help and messages are full of them) raised UnicodeEncodeError on a cp1252 console and took the command down. New `platform.force_utf8_stdio()` reconfigures stdout/stderr to UTF-8, called at the top of `main()`. Verified on Windows: init writes parseable LF node.toml, the keystore Argon2-decrypts, the loopback control API binds 127.0.0.1, and `_update_node_toml` reads a CRLF file and rewrites it LF-only with its standalone comments intact. Two regression tests added in test_ops.py. Co-Authored-By: Claude Sonnet 5

feat(node): persistent index cache, visible scan progress, adaptive reconcile, and delta sync

2026-08-23T19:55:20Z

Indexer performance work, in four parts: - Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node restart no longer re-hashes every file — measured at 23 minutes for a 114 GB library on a slow disk before this, near-instant after. Hashing is deliberately kept sequential (max_workers=1): it was never actually concurrent despite the pool size, and two interleaved reads seek-thrash a spinning disk instead of going faster. - Byte-based scan progress (IndexProgress), surfaced via the loopback index-status route, the handshake ack, and a periodic INDEX_PROGRESS push to connected peers — drives a progress bar in the Create Group wizard and "add a directory" in Settings, and an animated presence dot. Guaranteed to settle back to idle via try/finally and a final push on the scanning->false transition. - The reconcile backstop's directory walks now run in the executor instead of blocking the daemon's event loop; its interval defaults to 10 min (was 60s) with adaptive backoff to 2h when nothing changes, reset on a real change or a peer connecting, and is now a per-group operator setting (signed op + group Settings UI). - INDEX_DELTA wired up (protocol support existed, nothing called it): _on_index_change now sends additions/deletions instead of rebuilding the full entries list, coalesced over a short window so a burst of file events produces one push, and the hub swarm registration for public groups only (re-)registers newly added hashes. Also fixes several bugs found while testing the above against real libraries (a 114 GB and a 100+ GB group on a USB HDD): - /api/reload blocked until the reload — including a brand-new group's full initial scan — finished, which the Electron bridge's fixed 30s call timeout turned into a hard failure on any real library. The route now fires the reload without waiting (ops.start_reload), matching add_root/remove_root's existing pattern; the wizard's own step order was fixed to wait for the group to actually appear hosted before the steps that need it (extra roots, GEK), with retries for the residual race between that and the daemon's own bookkeeping. - transport.js's hand-rolled msgpack codec had no case for uint64/int64 (0xcf/0xd3) and crashed decoding any message containing one — hit by IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB. Verified against real msgpack-encoded bytes from the Python side. - chat_hist_resp, and this change's own index_progress and set_scan_settings_ack pushes, were not routed by message type and could be handed to an unrelated pending request by the transport's "oldest pending" fallback, stalling it until its own 30s timeout and corrupting whatever received the wrong reply in its place. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA

feat: unified group management, public groups, and activity-based sidebar

2026-08-20T20:21:19Z

Create Group wizard (Electron-only) consolidates 6 steps across 4 interfaces into a single multi-step page: group creation on hub, node attachment, root selection via folder picker, GEK initialization, and auto-pairing — all in one flow. Browser SPA keeps its current behavior unchanged. Public group support (Option A — GEK for all groups): - All groups have GEK regardless of visibility; open-join groups auto-admit via TOFU when join_policy is "open" - Key rotation blocked for public groups (API guard + UI hidden) - Hub signaling allows WebRTC offers for nodes hosting open-join groups even when the caller isn't a member yet - attach_group writes join_policy to node.toml - Daemon loads GEK for all groups, not just private ones - Known-device path in join_request now auto-admits to open-join groups Node loopback API bridge (Electron IPC): - node:detect, node:call, node:pairing-code IPC handlers in main process - Renderer never sees tokens, paths, or keys (session token = physical access) - platform.js node namespace for UI consumption - Loopback endpoints: roots CRUD, member-upload toggle, reload Bug fixes: - Root change detection: removed premature ctx["roots"] updates from add_root and remove_root that prevented indexer retarget on reload - Duplicate offline message: global fallback now gated on !group - Signaling membership check: fallback to open-join groups for non-members Sidebar groups sorted by last_activity_at (most recent first): - New Group.last_activity_at column with Alembic migration - POST /v1/groups/{id}/activity endpoint, called on connect and chat send - Client-side sort + throttled hub updates (1/min) Co-Authored-By: Claude Opus 4.6

feat(node): several named roots per group, and one implementation per operation

2026-08-18T00:15:02Z

Stage A — a group's content is a set of named roots --------------------------------------------------- `shared_dir` becomes a list of {name, path, kind}. The name is the directory's basename, derived once at add time and *stored*: recomputing it would re-identify a whole library the day someone renames a folder on disk. Duplicate names are refused case-insensitively and no root may contain another — both compared with NFC folding, because most of these directories live on exFAT or NTFS where `Films` and `films` are one directory. Every index path carries its root name, in a one-root group as much as in a five-root one. One path shape has to be got right once; two have to be kept right for ever. **A root that goes away freezes; it never empties.** Unmounting a volume makes watchdog report every file under it as deleted, or presents an empty directory to the next scan. Acting on either propagates deletions for a whole library to every member, as though the owner had erased it. So a deletion is acted on only once its root is confirmed readable, and availability is tracked per root — one unplugged drive leaves the others serving. 12 tests, verified to fail against an indexer without the check. Events are not trusted to be complete either: ReadDirectoryChangesW drops them under load and inotify on a FUSE mount misses changes made outside it. A periodic reconciliation sweep is the only thing that recovers a missed event. MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content hash, a node id and an endpoint — no paths, no filenames — and private groups register nothing (H7). Stage B — one implementation behind every front door ---------------------------------------------------- C1 and C6 were both "a second path into the node with its own weaker handshake". Two implementations of `revoke` with two authorization checks is that shape one size down. `meshbay_node/ops.py` holds each operation once, takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback API is one `_op(...)` line per endpoint; the MNP handlers call the same functions. test_ops.py asserts the shape rather than trusting it. Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload` (SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP operations: rotation is the half of revocation that revocation cannot do, since the ex-member holds the current key, and the node generates the replacement with its own CSPRNG — no key material crosses the wire, which is what the C5b rule is actually about. Two bugs found by running it rather than by testing it ------------------------------------------------------ GroupIndex is keyed by **content hash**, so the same bytes at two paths are one entry — which is also why a scan reports ten files and indexes nine. Reconciliation compared paths, so it decided the second path was a missed event every 60 s, rewrote the entry and pushed an index update to every connected peer. Seen in a live node's log. `meshbay-node reload` crashed on first use with `subprocess` unimported: the module compiles fine, which is the "syntax, not names" trap already recorded for the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be added to the parser without an entry there. Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the wire carried "0.2" — harmless only because nothing imported it. And _do_dir_create/_do_dir_delete referenced an undefined `filename` on their error path. 740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment. Co-Authored-By: Claude Opus 5