meshbay.git/packages/meshbay-node/src/meshbay_node, branch 0.14

feat: a group can be left out of Search, and Search tries every node

2026-09-14T19:45:53Z

`search_listed` is a per-group setting on the node, changed by a signed operator op and carried in the sealed handshake ack. Search reads it after the handshake and stops there: no index is fetched, cached or merged, in any of the four views, and the page says how many groups it left out. The switch is a "Search" section in the group's settings, shown to the operator. Absent means listed, at every layer: roster default, ack default, and the client only drops a group on an explicit `false` — so an upgrade or an older node removes nothing from anyone's Search. It is a listing preference and protects nothing: the node serves the same index to Search and to the group page and cannot tell them apart, every member lists the group by opening it, and a client that ignores the flag lists it in Search too. Design §9.11 says so, so it is never described as private. The cost is one handshake per unlisted group, because only the node knows the setting. Search also took `nodes[0]` twice — for the index and for the pooled connection — the defect 4cce50f fixed on the group page only. One `connectToGroup` now walks the list the same way: a refusal about this browser stops, `not_hosted` or a failed connection moves on. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01XuNrwLf5EFWCMHzfoEvnpm

chore: bump version to 0.14.0

2026-09-14T12:45:19Z

Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya

feat(node): progress names the root under way and the roots waiting

2026-09-14T09:08:12Z

`IndexProgress` said "scanning, this many bytes of that many" and nothing more. A group's roots are walked one after another, so a second directory added during a large scan showed as the bar jumping back to 0 %. It now also carries the root being walked and its position in the roots table, the kind of walk (scan, rescan, reconcile, watch), file counts, and the roots waiting for the scan lock in order: queued by the initial scan, by a retarget, and by a plug; dropped when a root is removed. `GET /api/index-status` answers for every group at once, including a group still in its initial scan, so a client can show indexing on any page. It names roots: loopback only, like `current_dir`. `index_progress` and the handshake ack gain the same counters, still naming nothing (decision D3): the root is a position in the roots table the member already opened from the sealed index, and the queue is a count. The pusher keeps speaking while a root only waits for the lock. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya

fix(node): a scan keeps its progress while a burst or a plug runs beside it

2026-09-14T08:56:40Z

A watchdog burst wrote the indexer's single `progress` directly. A file dropped into a folder during a large scan added its size to the scan's total, then cleared `scanning` when its own hash finished, so the progress went blank with hours of hashing left. Bursts now keep their own counters, shown only while no whole-root walk runs. A plug rescan took no scan lock and walked its root beside an added root's scan, both resetting the same counters and reading the drive in turn. It now waits for the lock, and skips the rescan if the root was ejected or removed while it waited, since the rescan drops the entries before it walks. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya

fix(node): a reload and a plug rescan outlive the session that asked

2026-09-14T07:40:23Z

A root added from the client arrives over MNP, and _retarget_indexer started the daemon's reload with the session's own _spawn. When that session closed - a client reconnecting 47 s into the scan of a 900 GB root - shutdown_tasks() cancelled the reload mid-scan, and the reload queued behind it, without a line in the log. The new root was in node.toml and in the indexer's set but never in the group's context; the lock was free and nothing retried, so the node served the old roots table for hours while reconcile hashed the whole drive as missed events. One loopback reload fixed the live node in 9 ms. _reload_config now runs the work in a node-owned task and awaits it through asyncio.shield, so a caller that goes away only stops waiting; a cancelled reload is logged. plug_root does the same for its rescan, which drops the root's entries before walking the disk and so left the root empty when its admin op's session closed. The existing MNP test replaced _spawn with a list and could not cancel anything; the new tests close the session for real and fail without this. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya

fix(node): an added root is served before it is scanned

2026-09-14T00:35:53Z

Adding a large directory to a running group made the reload await the scan of the new root before putting the new RootSet in the group's context, holding _reload_lock the whole time. For the hours a large drive takes to hash, the node served the old set: - a file request under the new root got None from entry_abs_path and the handler died on None.exists() without replying; - a writable/removable toggle answered with the live table, still the old one, so the directory vanished from the operator's settings; - reconcile saw every file the scan had not reached as a missed event and hashed it again on the same executor, rewriting progress under the scan. retarget now applies the set, the roots table and the watcher first, and with wait=False scans the added roots in the background; the daemon swaps ctx["roots"] before calling it. A scan lock shared by the initial scan, added-root scans and reconcile makes the reconcile loop sit out a running scan without backing off. Every transport site that resolves an entry answers ROOT_NOT_SERVED instead of crashing, and a delete keeps the entry. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya

fix(node): chat is bounded in size and in rate

2026-09-13T18:28:52Z

A chat message is the plainest member-supplied write there is: the node stores it in `chat.db`, where nothing expires it — retention is a manual command (§6.6) — relays it to every other connected member, and has the hub write a notification for every member of the group. Nothing bounded any of it. The only ceiling was the DataChannel frame, 64 MB once the handshake is done, so one member in a loop filled the operator's disk and saturated everyone else's connection, and the node's answer to each message was `ack`. Uploads, the other member-supplied write, have carried a filename allowlist, strict chunk ordering, a no-overwrite rule and a 4 GB cap since C5a — because somebody asked what one member costs the others on that path. Nobody had asked it on this one. Two bounds, for the two halves of the question: **64 KB of ciphertext** for what one message may cost, and **60 a minute per account per group** for how often one member may impose it. Both are checked before anything is stored or relayed; a refusal names itself and is audited, so "why is my disk full" has an answer. The rate is keyed by account, not by connection: a second tab does not make anyone type faster, and keying on the session would hand a script one budget per socket it opens. No node-wide ceiling beside it, deliberately. The link-preview limiter has one because a preview spends the *node's* egress and its third-party quota, which is one shared thing; a chat message spends the sender's own group, and a node-wide ceiling would let a busy group silence a quiet one — this same defect one level up. The last test in the new file is that property: a member at their limit has not spent anybody else's. Two things stay open on purpose and are named rather than quietly done: retention still keeps everything, because a default that deletes people's history is not a review's call; and the composer still offers to send an oversized message, so this is §6.4's pattern with only the node half built. §6.6 gains the rule, §13.5b the label — AV20, with AV21–AV23 registering the three fixes this week that closed the same kind of gap elsewhere. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW

fix(node): the node's own controls take no authority from a hub token

2026-09-13T14:29:36Z

`_is_node_admin()` is `self._user_id == node_user_id`, and `_user_id` is the `sub` of a JWT the hub issued. Six node-wide controls were gated on that alone: `node_status` — which lists every group on the machine with each root's **absolute path** — plus `node_settings_set`, `roster_read`, `denylist_read`, `denylist_clear` and `node_reload`. So the answer to "are you the operator of this node" was "the hub says so", which NS4 and M3 rule out in as many words: operator authority comes from the node's roster and from nowhere else, and asking the hub is how the hub installs itself as node administrator. The reach is bounded — a completed handshake also needs the group key — but an active hub obtains one legitimately in an open-join group, which §3.5 concedes, and from there it could read the operator's directory layout or clear the denylist, which is the persisted revocation H4 exists to keep. `_operator_device()` requires both halves now: the account is the one the node belongs to, *and* the device on this connection has proved a key the roster holds as an operator. `device_hello` is signed over a transcript naming the node, the group and this connection's nonce, and `operator_pks()` is rebuilt from the roster on each call, so an unpinned browser and a revoked one are both refused at once. The hub holds no user keys and cannot countersign a device. Keeping the account check as well is deliberate: dropping it would widen these node-wide controls to any paired operator of any group on the machine, which is a separate decision. `_is_node_admin()` stays as what it is in the handshake ack — a hint telling a client whether to offer the Node page — and says so. Nothing changes for a paired operator: `device_hello` runs unconditionally after the ack, and anyone using the Node page's controls is already paired, since `root_add` and every other signed op has always verified against `operator_pks()`. A browser that never paired now reads nothing there, which is the state in which it could already write nothing. test_node_status.py's fixture set the account and not the device, which is how it went on passing; it now wires the device the way `device_hello` leaves it. The adversary itself is in test_security_regressions.py — a token naming the owner's account with no proved device, which the previous source answered with `node_status_ack`. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW

fix(node): a transfer id names a lease, or it names nothing

2026-09-13T14:06:57Z

`_do_file_request` read `tr` as a boolean. Present meant "this is a leased transfer, skip the leaseless ceiling", and nothing asked whether this node had ever granted such a lease — `slots.touch(tr)` was called beside it and its answer, `False` if it is not granted, was discarded. So any non-empty string bought the whole library with no ceiling of any kind: not the per-member cap, not the node-wide one, not the leaseless bound that exists to bound a client claiming to be browsing. The queue held only the clients that chose to wait. `_lease_of` decides it now, and the three answers differ on purpose: - **granted**, and of *this* session — served, and touched so the sweeper does not reclaim a transfer that is plainly moving. The session is checked as well as the id, because touching another connection's lease refreshed its idle timer. - **queued** — refused with `lease_not_granted`, on the upload path too, before anything reaches the operator's disk. A member reading while queued is the cap not applying. - **unknown** — bounded by the leaseless ceiling rather than refused. That is also what a reconnect looks like from here, where the session's leases died with the old connection and the client is re-opening them, and it leaves the residual §5.5 already states: a client that lies gets that bound's worth of files at a time, not the group. Noted once per connection so the residual is visible rather than merely documented. Nothing changes for the shipped client: the transfer store awaits `lease.acquire()` before it reads a byte, so the refused case is one it never enters. §5.5 gains a paragraph saying the node decides which of the two a request is — the document described the accounting without ever saying it was enforced, which is how it came not to be. `test_lease_enforcement.py` drives the real handlers over a real index; six of its nine cases fail against the previous source, each on the property. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW

fix: hold every background task, in both codebases

2026-09-13T13:40:50Z

asyncio keeps only a weak reference to a task, so a coroutine started with `asyncio.ensure_future(...)` whose result is discarded can be collected while it is still running: the loop logs "Task was destroyed but it is pending!" and the work simply does not happen. No error reaches the caller, and what is lost is whatever that coroutine was in the middle of. The node already had a guard for this, written after an abandoned stream task lost a transcode slot for good — and it read one file, `webrtc_server.py`, because that is where the defect was found. Outside that file there were nineteen sites: the hub's `chat_notify` (a notification for every member of a group), the indexer's debounce (every real-time index update), eleven in `daemon.py` including the SIGHUP reload and each enrichment pass, two in `ops.py`, and five in the loopback API. `meshbay_common.background.spawn()` is the one door. It holds the task, drops it when it finishes, and logs what it raised under the coroutine's own name — an exception in a task nobody awaits was otherwise reported by asyncio at collection time, out of context or not at all. A peer session's `_spawn` stays as it is: that one can also *cancel* what it holds, which a module-level holder cannot, because a session ends and a process does not. `test_background_tasks.py` walks every package's source and refuses a discarded handle. It parses rather than greps, so an assignment, a comprehension or an await is not mistaken for one, and it was checked against a deliberate reintroduction. A guard that stops at the edge of the file where the bug was found is a guard against that bug, not against its class. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW