meshbay.git/packages/meshbay-common/tests, branch 0.14

fix: hold every background task, in both codebases

2026-09-13T13:40:50Z

asyncio keeps only a weak reference to a task, so a coroutine started with `asyncio.ensure_future(...)` whose result is discarded can be collected while it is still running: the loop logs "Task was destroyed but it is pending!" and the work simply does not happen. No error reaches the caller, and what is lost is whatever that coroutine was in the middle of. The node already had a guard for this, written after an abandoned stream task lost a transcode slot for good — and it read one file, `webrtc_server.py`, because that is where the defect was found. Outside that file there were nineteen sites: the hub's `chat_notify` (a notification for every member of a group), the indexer's debounce (every real-time index update), eleven in `daemon.py` including the SIGHUP reload and each enrichment pass, two in `ops.py`, and five in the loopback API. `meshbay_common.background.spawn()` is the one door. It holds the task, drops it when it finishes, and logs what it raised under the coroutine's own name — an exception in a task nobody awaits was otherwise reported by asyncio at collection time, out of context or not at all. A peer session's `_spawn` stays as it is: that one can also *cancel* what it holds, which a module-level holder cannot, because a session ends and a process does not. `test_background_tasks.py` walks every package's source and refuses a discarded handle. It parses rather than greps, so an assignment, a comprehension or an await is not mistaken for one, and it was checked against a deliberate reintroduction. A guard that stops at the edge of the file where the bug was found is a guard against that bug, not against its class. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW

fix: an empty group claim is a claim on nothing

2026-09-12T14:36:54Z

A node that hosts no groups sends no `group_ids` on its hub socket, and the hub resolved the claim with `set(claimed_groups or authorized)` — so "I host nothing" arrived as "I host every group this account belongs to", other members' included. Such a node can serve none of them: it holds no GEK, and its own handshake refuses them with "Group not hosted on this node". `/v1/groups/{id}/nodes` answers in registration order and `_node_groups` is in-memory, so which node a client was sent to depended on who reconnected first after a hub restart. GroupPage took `nodes[0]` with no fallback. On 2026-09-11 a hub deploy at 20:14 reshuffled the registry, a second member's unconfigured node won the race, and a group stopped opening for everyone in it with its only real host online throughout. Any member could take one of their groups down, by accident, by leaving an empty node running. Four changes, because no one of them is sufficient: - the hub never widens an absent claim, and `update_groups` goes through the same ceiling as registration — it assigned its list verbatim, so the bound that makes C2 hold at authentication was one message wide - the node states the empty set rather than omitting the field - the refusal carries `not_hosted`, so a client can tell "try the next node" from "you, here, must do something first" - GroupPage walks the list instead of indexing into it The three lines involved date from 13, 20 and 23 August and each is defensible alone. The defect is in the seam, which is where the last two also were: a falsy empty collection must never mean "unspecified". Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT

refactor(common): delete the Double Ratchet implementation nothing uses

2026-09-10T15:32:00Z

311 lines of Signal Double Ratchet and 167 lines of tests for it, with no caller: group chat is a key per group, per epoch, per device, and a ratchet was ruled out for it on the record — a node that serves history to devices which were not present has to hand out each chain's earliest key, which is forward secrecy of zero. Deleted for the same reason `senderkeys.py` was: an implementation kept for a use nobody has reads as an alternative somebody may reach for, and its cost is paid at every refactor that has to keep it compiling. The four comments that mention a ratchet keep doing so — they explain why this is not one, which is the part worth keeping. Common suite 158 passed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3

refactor(common): delete the sender-key implementation nothing uses

2026-09-10T15:30:22Z

`senderkeys.py` and its 13 tests implemented Signal-style sender keys, and production has never called them: chat is a key per group, per epoch, per device, derived by name. The reasoning that ruled the ratchet out stays where it belongs — in `chatbox.py`, at the top of the module that replaced it — because the argument is the useful part, and it now stands on its own instead of pointing at a file to compare against. Kept code that nothing calls is worse than absent code: it reads as an alternative somebody may reach for, and it has to be maintained past every refactor to stay compiling, which is maintenance spent on a decision already made. The three comments naming `GroupSenderKeyStore` are rewritten to say the thing they were illustrating. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3

Merge origin/main into the chat encryption work

2026-09-07T16:03:52Z

Both sides landed a breaking MNP change and both called it 2.0, which is right: the sealed upload, the removal of `stream_seg` and mandatory chat encryption share one flag day. They are recorded as one version in `__init__.py` rather than as a race between two. The resolutions that were decisions rather than mechanics: * **`MNP_MIN_SUPPORTED` moves to "2.0".** The sealed upload alone was a *confined* break — a 1.x peer could still connect, browse, download, stream and chat, with only its uploads refused by `upload_not_sealed` — so the floor deliberately stayed at "1.0". Mandatory chat encryption ends that confinement: a 1.x peer can neither produce a sealed chat message nor read one, so it would connect, look fine, and be unable to say anything. Refusing it at the handshake is the honest form. The per-message `upload_not_sealed` path is untouched and still right if the floor is ever lowered. * **`sendChat` throws on an `error` reply**, from origin, applied to the sealed send. It matters more after this change, not less: the node now refuses a stale epoch, a malformed envelope and a device claim that is not the connection's own, so there are three new ways for a message to be rejected and none of them may look like a message that was sent. * **`req_id` supersedes the per-type routing** this branch added for `chat_keys_resp` and `device_hello_ack`. Both blocks are kept beside the existing `chat_hist_resp` one, for the same stated reason — a node too old to stamp — and their comments no longer claim to be the mechanism that closes the class. `req_id` is. * **`chat_send_probe.py` is rebuilt on origin's structure**, not beside it: two scenarios, a stub that stamps `req_id`, `music_meta_req` as the older pending request. The encrypted path is layered on — a real Ed25519 device key generated in the page, and a `chat_keys_resp` sealed by the shipped Python, because a payload the page built itself would prove only that the page agrees with the page. * **`test_reply_correlation.py` now sends a sealed message.** Its subject is which of the two messages leaving that handler carries the id; plaintext chat was only the fixture, and the node refuses one now. * `groupbox` keeps both new purposes (`upload`, `chat_keys`); `protocol.py` keeps origin's removal of `STREAM_SEGMENT` and this branch's correction of the "Double Ratchet message" comment on `CHAT_MESSAGE`, which was wrong when it was written and is wrong differently now. Full suite on the merged tree: 1993 passed, 11 failed — the same 11 that fail on a pristine checkout (2 Windows service tests, 1 apps-enabled policy, 7 transcode tests that pass in isolation, and the WebRTC invite test that hangs on its own). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr

feat(chat): encrypt group chat under per-device epoch keys (MNP 2.0)

2026-09-07T15:50:28Z

Chat messages are sealed with AES-256-GCM under a key derived per group, per epoch, per *device*, and signed over the ciphertext with the device key the node pinned. The node relays and archives; it cannot read a message. There is no switch. MNP goes to 2.0 and MNP_MIN_SUPPORTED moves with it, so a 1.x peer is refused at the handshake with `version_too_old` rather than admitted and then unable to speak. An opt-in flag was designed and rejected: every node is a test node, so it would have bought nothing and left a plaintext branch reachable — C6's lesson one feature later. A test reads the source and refuses any code that consults a `chat_encrypted` setting. Not Sender Keys, and `senderkeys.py` is now documented as unused. With distribution under the group key and a node that serves history to devices which were not present, the node must retain each chain's earliest key, and a chain key at iteration i yields every message key from i on by pure HKDF — forward secrecy is zero either way. What the ratchet was left buying was stateful client code with silent failure modes, three of them reproduced: any member could sign as any other, a second device dropped the first's chain, and the skipped-key cache grew without bound. The reasoning is in docs/chat-sender-keys.md, which is the specification and the decision record. Epochs, not rotation: the epoch key is wrapped under the group key at delivery and never stored under it, so `gek_rotate` is a re-wrap. A group-key-derived archive key would have made every message ever sent unreadable on the first `member unpin`, which is the documented step after removing a member. A new epoch opens on member revoke/unpin, device revoke and `gek_rotate`; old epochs are kept and still delivered, so history stays readable to everyone who could already read it, and nothing anywhere deletes one. Three prerequisites this needed, each a live defect on its own: * The peer registry was keyed by user_id, so one account's second device evicted the first and the broadcast skipped recipients by account — a person's phone never saw what they typed on their laptop. * The handshake authenticated an account, never a device. `device_hello` (additive, signed, refused unless the key is a live device of this account in the node's own roster) is what lets the node refuse a member claiming somebody else's key. * `_admin_exec_file_delete` authorized against the exact uploading key, so device linking had already broken deleting your own file from your other device. It now authorizes against any non-revoked device of `uploader_id`. Found by driving the real panel over the real transport, not by reading source: `chat_keys_resp` was routed by arrival order and handed to an unanswered `media_meta_req` — the original frozen-tab defect in a message type that did not exist when that probe was written. And `_asText` had been deleted with an unrelated helper beside it; its only caller sits inside a promise the panel catches, so every conversation rendered empty with nothing in the console. Existing node data is migrated by QE/migration/migrate_chat_encryption.py (not versioned, per the QE rule), run with the node stopped. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr

feat(mnp)!: seal the upload under the group key

2026-09-07T15:46:33Z

Downloads have been encrypted under a GEK-derived key since the beginning: `file_chunk` and `stream_data` both go through `chunk_ciphertext`. Uploads never were. `file_upload` carried the filename and the raw bytes in plain msgpack, and `file_upload_ack` carried the name the node stored them under — so the same file was ciphertext leaving a node and plaintext arriving at one. There was no threat model behind that asymmetry. Both halves now travel sealed under a third groupbox purpose, HKDF(GEK, info="meshbay:upload:v1"). The filename, the destination folder and the bytes are all inside the seal; only `upload_id` and `chunk_index` stay in clear, because the node routes and orders on them before it can decrypt. This direction seals *towards* the node — it holds the GEK for its own group — and it opens the payload before it picks a destination or touches the disk. What that forced, and why none of it is optional: - `filename` was the correlation key on both sides. It cannot be: matching an ack to its request by name would hand back exactly what the seal hides. `upload_id` replaces it — client-drawn, opaque to the node, unique within a connection, never an authorization input. The property it guarded (one refusal fails one upload, not every upload in flight) is unchanged. - Refusals can no longer quote what they refused. `No directory named 'X'` becomes `No such directory in this group` plus the `code` that was already there; the client knows what it sent. - No plaintext fallback. A path that still accepts plaintext is not a sealed path, so an unsealed `file_upload` is refused with `upload_not_sealed`. Hardened while here, because what comes out of a seal is authenticated but not validated — a member can seal anything: `filename` and `data` have their types checked before any upload state is created, and `chunk_index`/`total_chunks`, which are outside the seal by necessity, can no longer raise where a refusal was meant. Tests. `test_upload_sealed.py` pins the node half: nothing identifying on the wire, tamper/wrong-key/wrong-group all refused with nothing written, and multi-chunk reassembly unchanged. `test_upload_seal_client.py` drives the shipped `uploadFile` over the shipped `crypto.js` under node and feeds its real frames to the real `_do_file_upload` — the file lands intact, and the ack the node actually produced comes back with the name it chose for a collision, which is the half a source-reading test cannot see. Both upload purposes join the JS/Python groupbox parity vectors. BREAKING CHANGE: MNP 2.0. `file_upload`/`file_upload_ack` change shape on the wire every deployed client speaks, which is MAJOR by the same rule 1.0 was — but the break is confined to uploads. `MNP_MIN_SUPPORTED` stays at "1.0", so a 1.x peer still connects, browses, downloads, streams and chats; only its uploads are refused, with a message saying which side is old. The client checks the node's version before sending a chunk, so neither side meets this as a timeout. This is the version negotiation shipped in 1.0 earning its keep: 1.0 cost a flag day, 2.0 costs a refusal code. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3

feat!: MNP 1.0 — seal index and handshake_ack under the group key

2026-09-03T14:16:55Z

`index_sync`, `index_delta` and the `handshake_ack` config payload now travel sealed under a GEK-derived subkey (`meshbay_common/groupbox.py`, mirrored by `sealGroup`/`openGroup` in `crypto.js`). Only `type`, `v`, `group_id` and the ack's `node_pk`/`proof`/`sig` stay in clear — a receiver must route and authenticate before it would trust a decryption. Verify, then decrypt. The ack line is integrity, not confidentiality: the signed handshake transcript names no ack field, so `is_node_admin`, `enabled_apps`, `video_root` and the rest were authenticated by the DTLS channel alone. The index line is defence in depth against a repeat of C1/C6 — a peer served before the handshake completes now gets ciphertext, not filenames. Nothing against an observer, the hub, or a member; that is the whole claim. `index_progress` stays clear (D3, counters only). Chat is out of scope. Failure is fatal: a payload that does not open ends the session naming the message type — never an empty index or an empty `enabled_apps`, both of which are legitimate states. Version negotiation ships here too (phase 15.6, brought forward): `v` + `v_min` on `handshake` and `handshake_challenge`, refused with `version_too_old` / `version_too_new` / `version_unreadable`. The flag day was already being paid for; the next breaking change now costs a refusal message. BREAKING CHANGE: breaks the WebRTC wire every deployed client speaks. Hub and every node must deploy together; the SPA is served by the hub, so a browser picks up the new client on reload. See MESHBAY_NODE_PROTOCOL.md §11.1a, §13.1. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HkzbhmMmK8PqQBtGz5zCvY

feat(node): several named roots per group, and one implementation per operation

2026-08-18T00:15:02Z

Stage A — a group's content is a set of named roots --------------------------------------------------- `shared_dir` becomes a list of {name, path, kind}. The name is the directory's basename, derived once at add time and *stored*: recomputing it would re-identify a whole library the day someone renames a folder on disk. Duplicate names are refused case-insensitively and no root may contain another — both compared with NFC folding, because most of these directories live on exFAT or NTFS where `Films` and `films` are one directory. Every index path carries its root name, in a one-root group as much as in a five-root one. One path shape has to be got right once; two have to be kept right for ever. **A root that goes away freezes; it never empties.** Unmounting a volume makes watchdog report every file under it as deleted, or presents an empty directory to the next scan. Acting on either propagates deletions for a whole library to every member, as though the owner had erased it. So a deletion is acted on only once its root is confirmed readable, and availability is tracked per root — one unplugged drive leaves the others serving. 12 tests, verified to fail against an indexer without the check. Events are not trusted to be complete either: ReadDirectoryChangesW drops them under load and inotify on a FUSE mount misses changes made outside it. A periodic reconciliation sweep is the only thing that recovers a missed event. MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content hash, a node id and an endpoint — no paths, no filenames — and private groups register nothing (H7). Stage B — one implementation behind every front door ---------------------------------------------------- C1 and C6 were both "a second path into the node with its own weaker handshake". Two implementations of `revoke` with two authorization checks is that shape one size down. `meshbay_node/ops.py` holds each operation once, takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback API is one `_op(...)` line per endpoint; the MNP handlers call the same functions. test_ops.py asserts the shape rather than trusting it. Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload` (SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP operations: rotation is the half of revocation that revocation cannot do, since the ex-member holds the current key, and the node generates the replacement with its own CSPRNG — no key material crosses the wire, which is what the C5b rule is actually about. Two bugs found by running it rather than by testing it ------------------------------------------------------ GroupIndex is keyed by **content hash**, so the same bytes at two paths are one entry — which is also why a scan reports ten files and indexes nine. Reconciliation compared paths, so it decided the second path was a missed event every 60 s, rewrote the entry and pushed an index update to every connected peer. Seen in a live node's log. `meshbay-node reload` crashed on first use with `subprocess` unimported: the module compiles fine, which is the "syntax, not names" trap already recorded for the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be added to the parser without an entry there. Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the wire carried "0.2" — harmless only because nothing imported it. And _do_dir_create/_do_dir_delete referenced an undefined `filename` on their error path. 740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment. Co-Authored-By: Claude Opus 5

fix(client): refresh the token when the node says "not a member"

2026-08-14T01:40:43Z

A member added to a group after they signed in was refused by the node, told "Not a member of this group", and had no way forward but to log out and back in. The hub bakes `groups` into the access token at login and never pushes updates, so the token said they were in nothing while the database said otherwise. This lands on every newly invited member, at their first action, and the message tells them the opposite of the truth — toto2 was a member of newdemo on the hub and read that they were not. The refusal now carries a code the client can act on (`not_a_member`) rather than prose it would have to string-match, and the SPA refreshes the access token once and retries. Refreshing re-reads membership from the database, so the retry succeeds. Once per mount: if a fresh token still says not a member, that is the truth and it gets shown. The SPA had stored a refresh token since Phase 8 and never used it. It does now. Found in a browser, doing the ordinary thing — the automated run never sees it, because e2e.py logs in after being added to the group. Tests: 233 node+common, including a handshake test that the refusal carries the code, and the full e2e run against the live deployment. Co-Authored-By: Claude Opus 5