aboutsummaryrefslogtreecommitdiffstats
path: root/packages/meshbay-node/src/meshbay_node/ops.py
Commit message (Collapse)AuthorAgeFilesLines
* refactor(node): move the operator's group controls out of webrtc_serverChristophe Besson31 hours1-1/+1
| | | | | | | | GroupOpsMixin in transport/webrtc/group_ops.py: member revocation and unpinning, the group key rotation, apps and their directories with the allow-list, and Search listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(node): member invite --link and member cancel in the CLIChristophe Besson2 days1-0/+78
| | | | | | | | | The CLI makes both halves itself — the node's code, then the hub's ticket bound to the address — and prints the link; a refused ticket takes the code back, and cancel takes back both. The CLI never asks the hub to mail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: drop the invitation-link plan from the treeChristophe Besson2 days1-1/+1
| | | | | | | | The plan was a working document; what it decided lives in MESHBAY_DESIGN.md and MESHBAY_NODE_PROTOCOL.md. Code and tests now cite those instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(node): invitation links — a code bound to no account until redeemedChristophe Besson2 days1-1/+36
| | | | | | | | | | New invite kind "link": member of one group, once, never operator, not spendable by an active member, capped at 20 per group, cancellable by handle. Signed ops invite_link_create / invite_cancel, loopback routes, and the known-device join path now accepts a link code. Adds the plan, docs/invite-links.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix: removing someone who never redeemed their invitationChristophe Besson5 days1-3/+19
| | | | | | | | | | A member row appears only when a code is consumed, so revoking someone invited to the wrong group was refused for having no row — and the node's refusal aborted the browser's removal before its hub half, leaving them a member everywhere with a live code. Revoking now cancels unredeemed codes for that group, and a node refusal no longer cancels the hub removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* style: ruff's own fixes, mechanically appliedChristophe Besson6 days1-4/+6
| | | | | | | | | | | | | | | | | | | | `ruff check .` had gone unrun long enough to report 568 errors, which is the same as having no linter: the next real finding would have been invisible in the noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the 98 it cannot fix are the next commit. What actually changed: import sorting (225), imports nobody used (87, none of them a re-export — no `__init__.py` is touched, which was the one way this could have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69) and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12 this project requires, `Optional[X]` to `X | None` (24), and f-strings with nothing to interpolate (19). Checked rather than assumed: every module in the three packages still imports, and the suite is 2893 passed — the same count, test for test, as the merge before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Merge origin/main: the operator's upload ceiling beside the disk-thread workChristophe Besson6 days1-35/+50
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One conflict, in §15.3's open list, and it was two changes agreeing rather than disagreeing: this side removed the rows for the third-party search bound and the node-announcement bound because both are now built (AV27, AV28), while the other side kept them and added a new one. Resolved by keeping what is genuinely still open — per-device revocation having no CLI — and leaving the two closed. `webrtc_server.py` merged without conflict but the two sides met inside one function: `_upload_chunk` gained the operator's `max_upload_gb` ceiling from there and the per-group lock and `off_disk` calls from here. Read back rather than trusted: the operator's ceiling now sits inside the critical section that keeps chunk ordering, and the unlink beside it goes to the disk thread with everything else. 2893 passed. The twelve `test_sticky_header.py[firefox]` setup errors are the open Firefox on this machine, as before.
| * fix(node): node.toml's transfer pools reach the transportChristophe Besson7 days1-37/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The daemon built the defaults dict for `Roster.node_settings` by hand and left out `max_concurrent_downloads` and `max_concurrent_uploads`. Absent from the dict, both resolved to None, were assigned back onto the config, and the transport skipped them — so node.toml was parsed, validated, and then replaced by `transfers.py`'s own 8. Invisible to anyone who left the value at 8, which is the value the template suggests. There were three copies of that dict and they all disagreed: node_status' was missing those two and `max_upload_gb` besides. One builder now, `config.node_settings_defaults`, and the resolver's key list is a class attribute the tests hold it to, along with the writer's. 1429 passed against a baseline of 1423; the two new behavioural tests fail with the builder reverted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| * feat(node): per-file upload ceiling is an operator setting, default 8 GBChristophe Besson7 days1-0/+22
| | | | | | | | | | | | | | | | | | | | | | Was a 4 GB constant in webrtc_server.py, the same on a small board and on a machine holding a library. Now max_upload_gb in node.toml, on the Node page and via `meshbay-node transfers max-size`, read from the transport context per chunk so a change reaches an upload already running. MESHBAY_DESIGN.md §6.4; §15.3 records a defect found beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | docs: cite MESHBAY_DESIGN.md and a section instead of the merged notesChristophe Besson7 days1-6/+7
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and deleted from the tree on 2026-09-11, but ~230 comments across the three packages still named them — usually written `docs/musicbay.md §3.2`, as though the file were still in docs/. A reader had to know §16 existed to resolve any of them. They now name the section directly. Every mapping comes from §16, the concordance, which already records where each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the Videos app and §6.5 where the subject is derived data, photos -> §9.9, auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 -> §3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys -> §4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section. Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`) are retargeted too: those collide with real section numbers in the design document, so leaving them would have been worse than the named form. Four cases the concordance does not cover, each decided rather than guessed: Sub-item references into documents that no longer exist — mediacenter's `§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and §9.2 do not reproduce. The module-level citation stays; the sub-item pointer is dropped. The V-findings keep their labels but lose the dead `§10.1/` prefix. §13.8 lists V1–V13 as per-application open items, which is not what the labels mean in these comments, so pointing them at §13.8 would have been a false citation. `apps.md §5`'s virtualization requirement has no counterpart anywhere in the design document. The requirement is stated in the comment itself, so the citation is dropped rather than aimed at a section that does not say it. Comments that attributed a *sentence* to an old note — musicbay's "several thousand files" example, its "what got measured" note, its measured ~11%/~26% cover-art figures, the "original no root, whole shared tree" call — state the fact without attribution now. §9.8 does not contain those sentences and citing it for them would have been wrong. CLAUDE.md's "a reference to a document that no longer exists" row now says the concordance is for git history and out-of-tree material; the code cites sections directly. Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox leg of test_sticky_header.py's browser harness, which is broken at the browser level on this machine — headless Firefox (snap) dies with `[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders nothing, and the probe exits `{"error": "no measurement"}` after its full 90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing here can affect it: every changed line in style.css is inside a comment. Also checked: ast.parse on every changed .py, `node --check` on every changed .js, the /* */ balance in style.css, and that no changed line exceeds the width its file already used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): take the availability poll and every upload write off the loopChristophe Besson7 days1-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rest of AV9's disk half. Serving a file left the loop in the commit before this one; two paths were still on it. **The availability poll.** `RootSet.refresh_availability` stats every root, and eleven call sites reached it from `async def` — the reconcile loop among them, on a timer. On a sleeping disk that is a stall once per tick, and the stat is also what keeps the disk awake, so a node paid spin-up for a library nobody was reading. All eleven now go through `off_disk`, `Root.is_live` included. **The upload write.** `open`/`write`, and the resolve, the stat, the free-name search, the rename and the unlink around it. This one could not simply be awaited: the handler was synchronous, so nothing could come between the `chunk_index != state.next_index` check and the `advance` that answers it, and that is the whole of the chunk-ordering rule. Awaiting the write opens the gap — chunk 1 arriving while chunk 0 is in the disk thread reads a position that has not moved and is refused as out of order, so an upload would fail on a slow disk and nowhere else. Verified, not assumed: without the lock the new ordering test refuses three chunks of four. So the check, the write and the advance are one critical section again, under a lock held **per group**. Not per session: `partial_uploads` lives in the group context so a reconnecting client finds its upload where it left it, which means two sessions of one member share the position of one `.part` file. Arrival order is preserved by construction — the dispatcher creates one task per message as it arrives, tasks start in creation order, and the lock is the first thing each one waits on, so its waiters queue in arrival order too. `_do_file_upload` is a coroutine now, which is why forty-two test call sites gain an `await`. Their outcomes are unchanged, file by file, against the run before the change. `test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`, imported rather than defined there. It now asks for the ones written in the module, which is what its own docstring means; all forty-three operations are still checked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: a group can be left out of Search, and Search tries every nodeChristophe Besson11 days1-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | `search_listed` is a per-group setting on the node, changed by a signed operator op and carried in the sealed handshake ack. Search reads it after the handshake and stops there: no index is fetched, cached or merged, in any of the four views, and the page says how many groups it left out. The switch is a "Search" section in the group's settings, shown to the operator. Absent means listed, at every layer: roster default, ack default, and the client only drops a group on an explicit `false` — so an upgrade or an older node removes nothing from anyone's Search. It is a listing preference and protects nothing: the node serves the same index to Search and to the group page and cannot tell them apart, every member lists the group by opening it, and a client that ignores the flag lists it in Search too. Design §9.11 says so, so it is never described as private. The cost is one handshake per unlisted group, because only the node knows the setting. Search also took `nodes[0]` twice — for the index and for the pooled connection — the defect 4cce50f fixed on the group page only. One `connectToGroup` now walks the list the same way: a refusal about this browser stops, `not_hosted` or a failed connection moves on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XuNrwLf5EFWCMHzfoEvnpm
* fix: hold every background task, in both codebasesChristophe Besson12 days1-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | asyncio keeps only a weak reference to a task, so a coroutine started with `asyncio.ensure_future(...)` whose result is discarded can be collected while it is still running: the loop logs "Task was destroyed but it is pending!" and the work simply does not happen. No error reaches the caller, and what is lost is whatever that coroutine was in the middle of. The node already had a guard for this, written after an abandoned stream task lost a transcode slot for good — and it read one file, `webrtc_server.py`, because that is where the defect was found. Outside that file there were nineteen sites: the hub's `chat_notify` (a notification for every member of a group), the indexer's debounce (every real-time index update), eleven in `daemon.py` including the SIGHUP reload and each enrichment pass, two in `ops.py`, and five in the loopback API. `meshbay_common.background.spawn()` is the one door. It holds the task, drops it when it finishes, and logs what it raised under the coroutine's own name — an exception in a task nobody awaits was otherwise reported by asyncio at collection time, out of context or not at all. A peer session's `_spawn` stays as it is: that one can also *cancel* what it holds, which a module-level holder cannot, because a session ends and a process does not. `test_background_tasks.py` walks every package's source and refuses a discarded handle. It parses rather than greps, so an assignment, a comprehension or an await is not mistaken for one, and it was checked against a deliberate reintroduction. A guard that stops at the edge of the file where the bug was found is a guard against that bug, not against its class. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
* refactor(mnp)!: one operation for an app's folders, not one per appChristophe Besson2026-09-101-27/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `video_root`, `audio_root` and `photo_roots` are gone — the messages, the signed operations, the handlers, the `ops` wrappers, the three scalars on the handshake ack, and the client's handlers for their acks. `app_directories` does the same thing for every application, keyed by the app's own registry name, and it is what the SPA has been sending. The three were the same instruction three times, differing only in the key they wrote and whether they carried a string or a list. That shape is what made adding an application mean adding a message type, an op, a handler and a widget; it also meant three validation paths, and the older ones validated nothing — a typo was stored and then quietly matched no entry, an app showing an empty tab with no way to tell "misconfigured" from "no files yet". **What stays, and why.** `Roster.LEGACY_DIR_KEYS` still reads `video_root` and friends out of `group_settings`: that is a key on an operator's disk, not on the wire, and a node upgraded into this must find its own configuration. The Search page still reads its own older cache keys, for the same reason — the cache outlives a deploy. `CTX_ALIASES` keeps only `chat`, which is the one app whose second name something still reads. The two per-app policy test files go with the messages. What only they held — the real challenge/response path from message to database, which no other test exercises — is retargeted at `app_directories` in `test_app_directories_signed.py`, and the handler's own refusals (unknown app, malformed `directories`, nobody to authorize it) join `test_app_directories.py`. Node and common suites 1368 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* fix(node): push the grants a per-member cap change producesChristophe Besson2026-09-091-2/+38
| | | | | | | | | | | | | | | | | | | | | | | Raising the per-member cap from 2 to 4 left both waiting transfers on "waiting". The pool granted them correctly and nobody told the peers: `ops.set_transfer_limits` computed `granted` and never sent a `transfer_state`, where the node-wide path (`WebRTCTransport.set_capacity`) does. That is the first row of §5.2 of ~/next/improve-downloads.md — "node granted a slot, the push was lost" — reached by writing the decision and forgetting the send. It is the same omission as the missing `touch()` call one layer up, on the same day: a mechanism that is right everywhere except at the seam where it has to reach somebody. The client recovered after its 60-second watchdog re-asked, which is why this looked like a slow queue rather than a lost message. `transfer_probe.py --operator` covers it now, separately from the node-wide hot-swap it already covered — different door, different code path, and only one of them was tested. Verified failing with the push removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* fix(node): a running transfer keeps its slot, and a dead grant lets goChristophe Besson2026-09-081-5/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two defects in the lease machinery, both found in the node's own log, neither reachable from any test on either side. **`touch()` was never called.** The node ignored `tr` on `file_req` entirely, so `used` stayed False for every download ever made and the sweeper revoked each grant thirty seconds in — while the file was transferring at 20 MB/s. The pool was correct and the handlers were correct; the call between them was missing, which is why neither side's tests could see it. **The requeue was a permanent cycle.** A revoked grant went back in the queue, was granted again a millisecond later because there was room, and was revoked again thirty seconds on. The node logged the same two reclaims every thirty seconds for as long as it ran — minutes after the transfers involved had finished. Three chances now, then the lease is closed and the peer told. `test_the_counter_never_drifts` could not have caught it: nothing drifted, the same lease simply never left. A lease that starts being used forgets its earlier misses: a client that took two grants to get going is slow, not abandoned. Also `transfers show` reported the module defaults rather than the operator's values until something had transferred, so `transfers set 2 2` answered "applied now" and the next line said 0/8 — indistinguishable, from outside, from the hot-swap that did nothing for months. The test asserted the defaults and so agreed with the bug; found by typing the command. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat(node): make the transfer caps settable, node-wide and per groupChristophe Besson2026-09-081-0/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Step 3 of ~/next/improve-downloads.md. Step 2 built the pools with constants; this gives them to the operator, in the two scopes they belong to. **The pools are the machine's.** `[node] max_concurrent_downloads` and `max_concurrent_uploads`, default 8, on the §2.11 pattern: node.toml for a fresh install, a roster.db override for immediate effect, editable from the Node page and from `meshbay-node transfers show|set`, applied live through the one `set_capacity` step 1 fixed. **The per-member cap is a group's.** How many transfers one member may run at once here — on the node like every other group setting (not the hub, which would have authority over someone else's disk; not node.toml, which is hand-written and needs a restart), changed by a signed operator instruction (`OP_TRANSFER_LIMITS`, subject "d=2,u=2" so what is signed names the outcome), broadcast to the group, and read live by the pools. That was the one thing step 2's shape could not express: `per_member` was a single node-wide number. `group_limits` and `member_cap(kind, member)` make it a lookup — the group's own value if it has one, the node's default otherwise — and it is deliberately the only dimension that is not node-wide. Three refusals, each with a test: - **absent means the default (2), never "unlimited".** A group that predates the setting coming back unlimited would leave the node-wide pool as the only control, which is the situation slots exist to end; - **zero is not "unlimited"**, and is not "this member may not transfer" either: the floor is one everywhere, and the CLI says to revoke the member instead; - **an unreadable row reads as unset**, not as zero — the same discipline the sealed messages follow, where a payload that does not open must never become a default state on its own. `handshake_ack` carries this member's own caps for this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn — never as "unlimited", which would have the interface contradicting the node. 1164 node, 793 hub, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat(node): transfer leases, pools and a queue for downloads and uploadsChristophe Besson2026-09-081-0/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Step 2 of ~/next/improve-downloads.md. A download is invisible to the node: it is a series of independent `file_req` messages, with nothing saying one started or ended, so there is nothing to count and nothing to cap. The lease is that missing object. `meshbay_node/transfers.py` holds the decisions and has no asyncio and no transport in it, on purpose. The failure modes this has to survive — a slot the node never gets back, a client waiting on a grant the node has forgotten — are races through a DataChannel and unprovable there; here the clock is a parameter and every method returns what changed, so the caller does the I/O and the tests drive the worst case directly. What it decides: - two pools, downloads and uploads, separate from the stream pool: different resources with different costs, and merging them makes both caps meaningless; - per-member cap checked *before* the node-wide one, so a member at their own limit queues behind their own transfers rather than holding a slot a second member has none of. Per account across their devices, or the cap becomes a function of how many tabs somebody opens; - a queue that skips a member at their cap instead of waiting for them — granting strictly in arrival order lets one member's limit stall everyone; - `tr` drawn by the client and idempotent, which is what makes a reconnect safe; - bounded per member, because unbounded queues are how a node runs out of memory politely. Every way a slot comes back, with the session teardown as the one that matters (a closed tab, a quit browser and a dead network all arrive at `shutdown_tasks`, and none of them needs a timer): explicit close, session gone, a grant nobody took up in 30 s passed to the next in line, and a granted transfer silent for 120 s reclaimed with its peer told, so a widget can offer a resume rather than sit on a lie. `GET /api/transfers` is the operator's window: when somebody reports a transfer stuck at waiting, it is the only thing that says whether the node ever had them in a queue — a log cannot, when the symptom is that nothing is happening. It carries no filename and no path, which a test pins, because this is exactly where one would be tempting. Three things found while writing it, two of them mine: - the randomised property test rejected `in_use <= cap` at once, and it was right to: lowering a cap never interrupts a running transfer, so the count legitimately sits above the new value. The invariant is that a *new* grant never happens past the cap; - the sweeper was started with `self._spawn`, which ties a task to one session's set. It died with whichever peer opened the first transfer, and every other peer's abandoned lease then stopped being reclaimed — a node that fills up over days with nothing in the log. It belongs to the node now, with its strong reference on the transport context; - the pools are node-wide while `_peer_registry` is per group (finding H1), so a slot freed in one group can grant one in another and the peer to notify is not in the notifier's registry. Silently wrong in the first version. Nothing enforces a lease yet: `file_req` is untouched, no client asks, and the node grants everything. That is step 4's flag day, and this lands alone. 1148 node, 793 hub, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* fix(node): make max_concurrent_streams take effect without a restartChristophe Besson2026-09-081-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `ops.set_node_settings` hot-swapped the stream pool by assigning `webrtc._stream_sem`. That attribute has never existed on WebRTCTransport — the pool is `ctx["_transcode_sem"]` — so `hasattr(webrtc, '_stream_sem')` was always False and the branch never ran. The setting was accepted, written to roster.db and node.toml, and applied only on the next restart, which is exactly what draft-v6 §2.11 says it does not need. An operator lowering the cap on a struggling machine, or raising it after "Server busy", saw nothing happen and had no way to find out why. `WebRTCTransport.set_capacity()` is the one implementation, on the object that owns the state, so the download and upload caps the transfer-slots plan adds next do not each grow their own copy of the mistake. Resizing has semantics worth stating: the new cap governs new streams and never interrupts one that is running, because a slot is held for the length of a film and lowering a number must not take somebody's film away. The replacement pool is built with the permits that remain (`new - in_flight`, floored at zero) — a full set would briefly allow more concurrent viewers than either the old cap or the new one. That needs a count of slots in use, so `_stream_video` now maintains one instead of the code reading the semaphore's private `_value`: a number this code keeps itself survives the semaphore object being replaced underneath it, and the same counter makes the "N of M in use" log lines mean something. test_stream_capacity.py drives the real transport and the real `_stream_video`; `test_ops_calls_the_real_mechanism` fails if the dead attribute comes back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat(chat): encrypt group chat under per-device epoch keys (MNP 2.0)Christophe Besson2026-09-071-1/+248
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Chat messages are sealed with AES-256-GCM under a key derived per group, per epoch, per *device*, and signed over the ciphertext with the device key the node pinned. The node relays and archives; it cannot read a message. There is no switch. MNP goes to 2.0 and MNP_MIN_SUPPORTED moves with it, so a 1.x peer is refused at the handshake with `version_too_old` rather than admitted and then unable to speak. An opt-in flag was designed and rejected: every node is a test node, so it would have bought nothing and left a plaintext branch reachable — C6's lesson one feature later. A test reads the source and refuses any code that consults a `chat_encrypted` setting. Not Sender Keys, and `senderkeys.py` is now documented as unused. With distribution under the group key and a node that serves history to devices which were not present, the node must retain each chain's earliest key, and a chain key at iteration i yields every message key from i on by pure HKDF — forward secrecy is zero either way. What the ratchet was left buying was stateful client code with silent failure modes, three of them reproduced: any member could sign as any other, a second device dropped the first's chain, and the skipped-key cache grew without bound. The reasoning is in docs/chat-sender-keys.md, which is the specification and the decision record. Epochs, not rotation: the epoch key is wrapped under the group key at delivery and never stored under it, so `gek_rotate` is a re-wrap. A group-key-derived archive key would have made every message ever sent unreadable on the first `member unpin`, which is the documented step after removing a member. A new epoch opens on member revoke/unpin, device revoke and `gek_rotate`; old epochs are kept and still delivered, so history stays readable to everyone who could already read it, and nothing anywhere deletes one. Three prerequisites this needed, each a live defect on its own: * The peer registry was keyed by user_id, so one account's second device evicted the first and the broadcast skipped recipients by account — a person's phone never saw what they typed on their laptop. * The handshake authenticated an account, never a device. `device_hello` (additive, signed, refused unless the key is a live device of this account in the node's own roster) is what lets the node refuse a member claiming somebody else's key. * `_admin_exec_file_delete` authorized against the exact uploading key, so device linking had already broken deleting your own file from your other device. It now authorizes against any non-revoked device of `uploader_id`. Found by driving the real panel over the real transport, not by reading source: `chat_keys_resp` was routed by arrival order and handed to an unanswered `media_meta_req` — the original frozen-tab defect in a message type that did not exist when that probe was written. And `_asText` had been deleted with an unrelated helper beside it; its only caller sits inside a promise the panel catches, so every conversation rendered empty with nothing in the console. Existing node data is migrated by QE/migration/migrate_chat_encryption.py (not versioned, per the QE rule), run with the node stopped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
* fix(node): remove --upload-dir rather than document itChristophe Besson2026-09-071-10/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | Caught in review, and the review was right. The previous commit documented the flag as deprecated so that `--help` and the man page would agree. That solved the wrong problem: the flag contradicts the model this whole refactor exists to establish, and the coherent answer was to delete it. It wrote `upload_dir` into a *brand-new* `[[groups]]` block, and `GroupConfig.__post_init__` reads that key by forcing every other root read-only and appending that path as the one writable one. So `group add --dir X --writable --upload-dir Y` silently made X read-only — two mechanisms deciding which directories accept uploads, one of them invisible, in a group created after the model that replaced it. Gone from the CLI, from `ops.attach_group`, from the loopback API and from the MNP `group_attach` payload, which now carries `writable` instead. The *read* path in `config.py` is deliberately untouched: an existing node.toml using `upload_dir` must keep working, and that is the only legitimate use left. The man page says so under the config key, and no longer lists an option. The test that guarded the deprecation wording now guards its absence — and earned itself immediately by finding a `group add` usage string still offering the flag. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* fix(node): `root list` printed "?" for every pathChristophe Besson2026-09-061-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | `RootSet.describe()` feeds two audiences that want opposite things. The index payload goes to every member and has always deliberately carried no paths — a member is told what exists and whether it is readable, not that the library sits under someone's home directory. The loopback API answers the operator themselves, over a channel that already requires their machine and the run token, and the path is exactly what they asked for. The `root list` CLI I added in Phase 1 read `path` from the member form, so it printed a placeholder for every directory. Nothing caught it: the CLI reads a dict, the API returns a dict, and neither end states which keys it owes. `describe(with_paths=True)` is the operator's view and `list_groups` is the only caller. The shared-directories table has the same hole over MNP — the roots there come from the index payload — so its Path column now appears only when a path is actually present, rather than rendering a column of blanks. The test asserts both halves, because they pull opposite ways: one that only checked the operator sees paths would be satisfied by leaking them to every member. It reads the indexer's source for the member side, and compares the CLI's key reads against what the payload offers for the other — checked to fail in each direction independently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* feat(node): an upload lands in the folder it was sent toChristophe Besson2026-09-061-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is no `uploads/` subdirectory any more, and the client names the folder rather than the root. It was the last of v5's quarantine — the per-user layer went on 2026-08-14 for the same reason — and it goes on the same grounds: a folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. Somebody dropping a file into the folder they are looking at expects it to be in that folder. **What made the quarantine worth having was never the subdirectory.** It is the filename allowlist, the size cap, the chunk ordering and the no-overwrite rule, and all four are untouched: an existing file is never replaced, the second sender of IMG_1234.jpg gets a free name, and the check still sits at the write. Letting the client choose the destination is safe for one reason and only one: it is resolved through `RootSet.resolve()`, which refuses `..`, absolute segments and anything whose resolved form escapes its root, symlinks included. A member answers "which of this group's folders", never "which path on the operator's disk" — and the test that used to assert the node chose now asserts that, with six shapes of escape. `direct` goes with it. Its only job was to say "no subdirectory for this root", which is now every root, and a config flag that does nothing is worse than none. Chat's attachment folder finally does something: the directory the operator picks in the Chat settings pane is where attachments are written, falling back to the first writable root while they have not chosen one, or if the one they chose has since been made read-only or ejected — a stale choice should not become a refusal at send time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* fix(node): the MNP root path never reloaded, and my first repair made it worseChristophe Besson2026-09-061-32/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous commit was the wrong fix. `add_root` did leave the running node unchanged, but editing the live `RootSet` in place — which is what I did — is wrong in the other direction. `DirectoryIndexer.retarget` decides what to scan by diffing the names it already holds against the ones it is handed, and `_retarget_indexer` hands it `groups_ctx[gid]["roots"]`: the very object the op had edited. So the new root sat on both sides of the comparison, nothing was scanned, and the directory would have appeared in the table permanently empty. `_reload_config_inner` diffs the same way and would have concluded nothing changed. `remove_root` had the same shape and would have kept serving a removed directory's files. The real defect is that two front doors did different things. `ui/app.py` has always fired the daemon's `reload_fn` after these ops, which re-reads node.toml and builds a *fresh* set; the MNP path retargeted a stale object instead. That asymmetry is exactly what `ops.py` exists to prevent, and it is why the bug survived until an operator added a directory from a browser — the loopback path worked all along. So: the ops leave the live set alone, `_retarget_indexer` asks the daemon to reload, and `update_root` keeps editing in place because flags change no files and the synchronous upload handler reads that object on the next request. The tests now check the files rather than `describe()`, which proves nothing about whether anything was scanned. One of them demonstrates the failure mode instead of describing it, so the rule is checkable and will say so if `retarget` ever changes. Two more cover the seam itself — that the MNP path reloads, and that a context with no daemon still retargets. Diagnosed by reading the running node's journal rather than the source: the first add logged "Reloading config" and a rescan, the two later ones logged neither. I should have looked there before the first attempt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* fix(node): adding a root reached node.toml but not the running nodeChristophe Besson2026-09-061-1/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `add_root` appended to the config and to node.toml and stopped there. `_retarget_indexer` — the MNP path — then re-points the indexer at `groups_ctx[gid]["roots"]`, an object nobody had touched, so it was retargeted at exactly what it already had. The directory was in the config file and invisible everywhere else until a restart. Worse than invisible: the ack does carry the new set, so the client showed the directory for one paint and the next index_sync took it away again — which reads as a UI bug and is not one. Adding it a second time was then refused as colliding with itself, which is the only reason anyone found out. `remove_root` and `update_root` already updated the live set; this one was missed. The loopback API hid it, because `ui/app.py` fires `reload_fn()` after the op and that re-reads node.toml from disk. The MNP path does not, and the shared-directories table only started offering Add over MNP in this refactor — a latent bug made reachable. The ack now describes the set the node will actually serve rather than one built on the side, so the two cannot disagree. test_root_ops_reach_the_live_set.py holds all three ops to it, including the counter-property that the same directory is still refused twice and that node.toml and the live set stay in step — the two halves drifting is how an operator's next restart silently undoes their last change. Four of its seven fail against the code above. Recovery on a node already in this state is `meshbay-node reload`: node.toml has everything, nothing was lost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* fix(client): a successful root op must never blank the operator's tableChristophe Besson2026-09-061-2/+12
| | | | | | | | | | | | | | | | | | | | | | | `ops.update_root` and `remove_root` returned `[]` when the group context had no live RootSet, and `group-page.js` accepted it: `if (msg.roots)` is true for an empty array. An op that succeeded would have emptied the shared-directories table, and "the node says this group has no directories" is not something the client can tell from "the node could not say". Both ends now refuse it — the node builds from config rather than answering empty, and the client requires a non-empty array. Found while adding `test_spa_imports.py`, which is the other half of this: it resolves every named import across the SPA against what the target actually exports. That failure has a shape nothing else here catches — no build step to fail, so the browser resolves the graph at load, finds a missing binding, and the page renders blank or the component just does not appear. `node --check` parses one file at a time and the source-reading guards look inside a file rather than between two. The settings split moved two shared components into a new module and rewired eight files to import them, which is exactly the change where a rename lands in one file and not the other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* feat(node): Phase 2 server side — one directory setting for every appChristophe Besson2026-09-061-53/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `video_root` (a string), `audio_root` (a string) and `photo_roots` (a list) said the same thing three ways: three roster accessors, three ops, three MNP messages, three admin-op subjects. They become `set_app_directories(app_key, paths)` and its single-directory wrapper, stored under `<app>_directories` and keyed by the app's registry name — so an application can be added without touching this layer, which is the whole claim of the plugin architecture. The three old names still work. Their MNP messages are handled, and the roster falls back to the old key when the new one is unset, so a node upgraded into this keeps working with no migration step — the plan called for a script, and a script nobody runs on the machine where it matters is worse than a fallback. Two things are new rather than moved: The paths are validated. The setters this replaces accepted anything, so a typo — or a path left behind when a root was removed — was stored happily and then matched no entry, leaving an app showing an empty tab with nothing to distinguish "misconfigured" from "no files yet". Deliberately not `RootSet.resolve()`: that also refuses a currently-unavailable root, and an operator must be able to point an app at a library on a drive they ejected. The legacy scalar is derived, never stored. `video_root` still rides on the handshake ack for MNP 1.0 clients; kept as a second stored value it would drift from the list within one run, which reads as "it works after a restart". Also here: chat's own two settings (a directory, which must be on a read-write root because it is a destination rather than a view, and a link-preview switch gating the unfurl path — checked before the cache, or turning it off would still serve every preview already fetched), the `app_directories`, `chat_directory` and `chat_link_preview` MNP messages, the plural `<app>_directories` on the handshake ack, and `music` as the app's one identifier where storage said `audio` and the registry said `music`. The Music enricher now resolves a boundary per configured directory rather than one for the group: with several, a single boundary is wrong for all but one of them, and for Music that is the difference between reading a folder as an artist and reading it as a release. Suite: 11 failures, all pre-existing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* fix(groups): finish Phase 1 — MNP root management, upload targets, eject stateChristophe Besson2026-09-061-23/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Review of the Phase 1 commit found the RO/RW model sound but three paths unfinished, each of which broke the flow the phase exists to deliver. Plus 29 test failures it introduced and no coverage for anything it added. Uploads went to the wrong directory. The node read a `root` field on file_upload that no client ever sent, so every upload landed in the first writable root while the Files toolbar offered its button based on the root being browsed — with two writable roots, uploading from one wrote into the other. Files now names the root it is showing; Chat names one chosen in the shell (an operator-configured directory arrives in Phase 2); the node refuses an unknown name rather than falling back, and refuses read-only and ejected roots by code. Shared directories were unreachable on the web. The table read its roots only from the loopback API, which resolves to "not available" in a browser, so the section rendered for nobody there — while the Uploads controls it replaced had worked — and the transport.updateRoot/ejectRoot/plugRoot methods beside it were dead. MNP is now the path, loopback the fallback for a local node with no live connection, and adding a root over MNP takes a typed path since no web page can browse a remote disk. Ejecting updated nobody's screen. transport.js resolves an admin ack against the pending request and returns, which is right for every op whose caller knows the value it chose; the root acks carry state only the node can compute, so the operator who clicked Eject was the one client that never saw it happen. And the ejected flag reached roster.db but was never read back, so a restart undid it and the next scan read an empty mount point as an erased library. Also: the member-upload endpoint answered 200 and did nothing (removed); the wizard ignored the first root's RW switch; reload compared roots on name and path, so editing writable in node.toml did nothing; the table had no path column, which is the only thing separating two libraries sharing a basename; apps_enabled normalisation differed between the two sides of a signed subject. Tests: eject/plug, per-root upload refusal and the node.toml rewrite had no coverage at all. test_member_upload_policy.py is replaced by test_root_writable_policy.py — it tested a removed feature — and every property worth keeping from it moved rather than being dropped. Docs: draft-v6 structural decision 9 is annotated as superseded (the operator can no longer have a directory only they may write to — a real capability removed, flagged rather than hidden), the man page documents the root verb and the RO/RW fields, and refactor-groups.md §7b records what the plan got wrong. Suite: 41 failures before, 13 after — all 13 pre-existing on main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* feat: groups refactor Phase 1 — root RO/RW model + shared directories UIChristophe Besson2026-09-061-48/+245
| | | | | | | | | | | | Replace the upload boolean with per-root writable/removable/ejected flags. Backend: new ops (update_root, eject_root, plug_root), MNP 1.1 protocol messages, live RootSet updates so API always reflects current state, CLI root subcommand (add/remove/set/list/eject/plug). Frontend: SharedDirectoriesTable with optimistic toggle switches, eject/plug in Files and Settings, upload gated on root.writable, ejected-root filtering in all media apps, updated Create Group wizard, 10-locale i18n. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node): make init, node.toml editing and CLI output work on WindowsChristophe Besson2026-09-041-3/+5
| | | | | | | | | | | | | | | | | | | | | Found by running the daemon on Windows for the first time: - `meshbay-node init` wrote `unlock_file = "C:\Users\..."`, and attach_group / add_root write `path = "C:\..."` — a raw Windows path in a TOML basic string is a parse error (`\U`, `\a`, ... are escape sequences), so the config would not load. All now write `Path(...).as_posix()`; pathlib reads the forward-slash form fine on Windows. - any `print()` carrying a `->` arrow or em dash (the CLI help and messages are full of them) raised UnicodeEncodeError on a cp1252 console and took the command down. New `platform.force_utf8_stdio()` reconfigures stdout/stderr to UTF-8, called at the top of `main()`. Verified on Windows: init writes parseable LF node.toml, the keystore Argon2-decrypts, the loopback control API binds 127.0.0.1, and `_update_node_toml` reads a CRLF file and rewrites it LF-only with its standalone comments intact. Two regression tests added in test_ops.py. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(node): pin utf-8 (and LF) on every text file the node reads or writesChristophe Besson2026-09-041-9/+9
| | | | | | | | | | | | | | | node.toml, the keystore envelope, the unlock key, the loopback UI token, pairing/invite code files and the denylist were all read and written with the platform default encoding and newline translation. On Windows that is cp1252 + CRLF: a node.toml or keystore holding any non-ASCII byte failed to load, and ops.py's line-based node.toml editor round-tripped CRLF in and LF out. Every read is now `encoding="utf-8"`; every write is `encoding="utf-8", newline="\n"` so the files stay LF whatever the OS. No-op where the locale was already UTF-8. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(node): an invitation the hub never registered is a code nobody can useChristophe Besson2026-09-031-7/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `create_invite` wrote the invite to the roster and *then* asked for the hub. An unreachable hub therefore raised "Hub not connected" after the code was already stored: the operator saw an error and no code, and a valid invitation sat in the roster that nobody had been given. Every retry left another. Registering first means a failure costs nothing — no code exists to be orphaned. A membership row without an invite is harmless: without the code there is still no group key. The endpoint is idempotent (`if not mem: db.add(...)`, no 409), so the SPA registering the same membership again right after createInvite costs nothing either. The registration is now fatal rather than swallowed, which is the part that matters. `/v1/groups/mine` joins GroupMember, so someone who was never registered does not see the group at all and can never redeem the code. Tolerating that failure handed the operator a code that cannot work and said nothing — a worse outcome than the error, because it is silent. Skipped only when there is no username to register with: the MNP path allows an empty one and there the SPA is the one that registers. Found by test_invite_then_join_delivers_the_gek, whose fixture had no hub and which passed only because the failure was swallowed. It has one now. And 0443cf8 added this registration to the CLI path without any test asserting it happened, which is how it came to be skipped whenever the hub was merely absent — test_cli_invite_asks_the_hub_for_an_account_ never_a_key checks it now, and test_an_unreachable_hub_leaves_no_invite_behind covers the orphan (verified failing against the previous ordering). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AbwJDbNTkiRUh7HTWEoyss
* fix(node): CLI member invite now registers hub membership and enforces codeChristophe Besson2026-09-011-0/+8
| | | | | | | | | | | | | | | | | | Two bugs fixed: 1. `meshbay-node member invite <user>` created a local roster invite but never told the hub to add the user to group_members, so the group was invisible in the SPA. The node now calls POST /v1/groups/{id}/members/{username} after creating the invite, and the hub endpoint accepts node-scoped tokens (the admin_id check is the real authorization guard). 2. The WebRTC handshake let a previously-pinned user reconnect without a code even when a new invite was pending (e.g. after leave + re-invite). Now any pending invite forces code entry, regardless of existing member/pin status. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node): make WebRTC STUN fallback actually use every configured serverChristophe Besson2026-09-011-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | aiortc's connection_kwargs() keeps only the first STUN URI from RTCConfiguration.iceServers ("only a single STUN server is supported"), and aioice.ice.Connection has a single stun_server field. So the node's four default STUN servers -- and anything added on the Node page or with `meshbay-node stun add` -- collapsed to stun:stun.l.google.com:19302. When that one server was slow or unreachable from the node, ICE gathering (get_component_candidates, timeout=5) burned its full 5 s with no server-reflexive candidate, adding seconds to every browser connection. The multi-server fallback of draft-v6 s2.12 was configuration only. transport/stun_multi patches aioice.ice.server_reflexive_candidate (same monkey-patch technique ice_filter.py uses on get_host_addresses) so a single ICE gather races the STUN binding request against every configured server on the one bound socket and takes the first answer. One reachable server anywhere in the list now yields a reflexive candidate in one RTT. - daemon: install_stun_multi() alongside install_ice_filter() - ops.set_node_settings: push the list to stun_multi.set_servers() so the CLI / Node-page hot-swap takes effect without a restart - webrtc_server.handle_offer: log ICE gather time and srflx count - test_stun_multi.py: fan-out, first-answer-wins, all-fail, empty-list fallback, DNS failure Verified end to end with a real RTCPeerConnection: with a black-hole STUN server first in the list, gathering still completes in ~0.07 s with full srflx candidates (previously a 5 s stall). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BSsQhfxEAhwi4nqc4hASmq
* feat: passphrase change and account recovery (auth-confirm)Christophe Besson2026-09-011-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The passphrase derives two independent client-side values: auth_key (the hub verifier) and bundle_key (AES-GCM key for the per-node identity bundles, which live on nodes and never on the hub). Changing or recovering a passphrase is therefore two operations — swap the hub verifier, and re-wrap every reachable node's identity bundle. Flow A — change a known passphrase (Profile page) - POST /v1/users/password re-proves the current passphrase, swaps pw_hash/salt/version, revokes every refresh token and returns a fresh pair so the tab that made the change stays signed in. - MeshBayTransport.rewrapAllNodes: for every group's online node, connect with the old key, read the identity off the handshake, store it back under the new key. Returns updated / unreachable / failed so the UI can point at the operator-unpin fallback for the gaps. Always-shown confirmation dialog listing reachable and unreachable groups. Recovery key - keyderive.js generateRecoveryKey (32 random bytes, grouped Base32) and deriveRecoveryKey (HKDF-SHA256, domain meshbay:recovery:v1:<username>). - Every per-node identity gets a second copy wrapped under the recovery key: keypair_bundles.bundle_enc_recovery (node-only column, added in _SCHEMA_KEYPAIR and via a PRAGMA-guarded ALTER for existing DBs), carried on keypair_bundle_store / _resp. MNP 0.13 -> 0.14, additive. - session.recoveryKey is persisted in IndexedDB (slot rk) and lazy-loaded on connect, so a group joined in any later session still leaves a recovery copy. - Shown once at registration; optionally folded into the verification e-mail as a pass-through the hub never stores or logs, with an opt-out. - Profile -> Recovery key re-loads R and backfills every reachable node via rewrapAllNodes in bundleKey mode (no passphrase re-entry). Flow B — recover a lost passphrase (#/reset, linked from sign-in) - POST /v1/users/password/reset-request {username, email}: both must be the pair on file, checked against the blind email_hash (never decrypted). A mismatch — wrong e-mail, unknown username, non-active account — takes the identical no-op path (no code, no mail, same 200), so it reveals nothing and cannot be used to spray reset mail from a username alone. 5/min, 1-hour single-use code. - POST /v1/users/password/reset {username, code, new_auth_key}: same expiry / attempts / single-use checks as e-mail verification; revokes every session and deletes every registered device key so a stored one cannot sign back in past the reset. - ResetPasswordPage: request code -> code + optional recovery key + new passphrase -> reset + sign-in -> fan-out. connect() falls back to the recovery-wrapped copy when the passphrase key cannot open bundle_enc. Without a recovery key: sign-in is restored and each group needs the operator-unpin fallback. Supporting fixes (found in live testing) - member unpin now also deletes the keypair bundle; connect() mints a fresh identity when handed a bundle it cannot open (unless _rewrapOnly, set by rewrapAllNodes), so a rejoin completes instead of dead-ending before the invite-code prompt. - A browser with no bundle key gets a passphrase prompt on the group page instead of a "go back to the browser you registered on" message. - RegisterPage / LoginPage / ResetPasswordPage trim the username so every key derivation matches the hub's stored form. Docs: docs/auth-confirm.md. Locale keys across all ten catalogues. Tests: test_password_change, test_password_reset, test_recovery_email, test_recovery_key, test_rewrap_fanout, test_bundle_store_recovery, plus additions to test_admin_ops_mnp and test_webrtc_transport. Hub suite 492 passed; node suite 741 passed (the lone test_packaging_units failure is a pre-existing RPM-spec flake, reproducible on main). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GGkxJW9br8Y9bhT8ywJ3oc
* feat(ui): configurable ICE interfaces in the Node pageChristophe Besson2026-08-301-5/+12
| | | | | | | | The ice_interfaces setting (auto-exclude vs explicit whitelist) is now editable from the Node page, persisted via the settings API and roster, and hot-swapped at runtime by re-installing the aioice filter. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: configurable STUN server fallbacks for WebRTC ICEChristophe Besson2026-08-301-9/+46
| | | | | | | | | | The WebRTC transport relied on a single Google STUN server — if it was unreachable, ICE gathering waited the full 4s timeout. Now four public servers are used by default (Google ×2, Cloudflare, Mozilla), configurable via node.toml, the Node page UI, and the CLI (meshbay-node stun list|add| remove|reset). Changes are hot-swapped on the live transport. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): editable node settings in the Node page (D5)Christophe Besson2026-08-291-1/+114
| | | | | | | | | | | | | Expose invite_ttl_hours, pair_ttl_hours, device_request_ttl_minutes, max_concurrent_streams and transcode_incompatible_video in the Node management panel. Changes are applied immediately via roster.db and written back to node.toml so they survive a DB wipe. On startup, roster overrides take precedence over node.toml defaults. Draft v6 §2.11 documents the design; MNP gains node_settings_set / node_settings_set_ack for the browser path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(node): correct TMDB movie matching, per-file overrides, rematchChristophe Besson2026-08-291-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A batch of wrong poster-grid matches found live on a real library (2026-08-29): a two-volume film's second part matched the first; a numbered sequel matched a same-year making-of documentary; several entries of one franchise matched a single early entry whose localized TMDB title is the franchise name; one matched nothing. One mechanism: _tmdb_search returned the first candidate query whose title-similarity ratio merely cleared 0.6, before alternative_title / the Roman-numeral variant was ever tried. Matching: - title_parse: fold guessit's volume/part number back into display_title so the parts of a multi-part film stay distinct in the query, the card and the override. - _tmdb_search: keep a strong PASS 1 fast path (ratio >= 0.85, one request), otherwise score every candidate query and pick the best. A year-exact rescue lifts a sub-0.6 top hit to the confidence floor only when TMDB's own year-filtered result lands exactly on the filename's year. No local re-ranking of any single result list; no tmdb.py change. Fix match / rematch: - _admin_exec_tmdb_override: a movie override touches its own file only (guessit gives a whole franchise one display_title); a show override still fans out. Corrected files are marked in media_cache.tmdb_override. - media_cache: tmdb_override table; clear_file_tmdb / clear_tmdb_matches drop auto-resolved matches while sparing manual corrections. - ops.rematch_video + `meshbay-node video rematch` (loopback endpoint + CLI verb): re-resolve a group's video matches after a matcher fix. file_tmdb is keyed by content hash and otherwise only pruned on deletion, so nothing dislodged a cached match before. - a rename now drops the stale auto match too (daemon _reenrich_renamed_video_entries). UI: - VideoDetailModal shows the source filename and resolved TMDB id; an unmatched poster gets a badge (3 new video.* i18n keys x 10 locales). So a wrong match can actually be identified before hitting Fix match. docs/mediacenter.md 10.1 records this and the V8-V13 follow-up backlog (show-branch ladder, year-aware _best_match, wider sequel_variants, the 0.6-0.85 extra calls, movie grid merge, per-card rematch). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
* feat: node workflow redesign — wizard auto-config, reset, MusicBrainz contactChristophe Besson2026-08-291-17/+9
| | | | | | | | | | | | | | | | | | | | | | | | Wizard (Electron): - Auto-provisions node config (hub URL + username) from logged-in user - node:start handles both cold start and restart of misconfigured daemon - Waits for daemon to reach 'running', auto-links node key on hub - probeNode accepts intermediate states for wizard progress feedback Reset (meshbay-node reset): - Unlinks node key from hub (DELETE /me/node_key, best-effort) - Stops and disables daemon (systemctl --user disable --now) - Erases ~/.config/meshbay, ~/.local/share/meshbay, ~/.local/state/meshbay MusicBrainz contact: - Resolved from owner's hub email instead of per-node roster config - Removed musicbrainz_contact UI and WebRTC handshake field - Removed set_musicbrainz_contact/musicbrainz_contact from roster Node pairing: - Added operator pairing banner on NodePage - Added operator_paired flag to list_groups Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): share the (path,size,mtime)->hash index cache across every groupChristophe Besson2026-08-261-0/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An operator routinely shares the same physical folder into more than one group (a music library, a Séries drive) — IndexCache used to be opened once per group (data_dir/{group_id}/index_cache.db), so the second group to reference an already-fully-hashed multi-terabyte folder paid the same full content read the first one did. IndexCache itself carried no group_id in its schema; only daemon.py's wiring did. Now one instance, opened once at startup (data_dir/index_cache.db), shared by every group's DirectoryIndexer. Confirmed against a real deployment (2026-08-25/26): a group sharing an already-indexed folder with an existing group indexes it instantly, with zero rehashing. Also fixes a related cross-group correctness gap found during this work: media_cache.db (thumbnails, TMDB/MusicBrainz metadata — already node-wide, untouched by this change) was pruned for a file the moment it left *one* group's index, even if another group's index still held the same content hash — forcing a redundant re-fetch/re-probe/re-thumbnail for a group that never actually lost anything. Prune now runs only once no group's index references the file_id any more. Adds a node admin UI action ("Maintenance" card, prune-index-cache) to drop cache rows that no longer belong to any group's roots — skips anything under a root that is merely temporarily unavailable (indexer.py's "a root that goes away freezes, never empties" rule extends to this cache too, or a reconnected drive would pay a full rehash for no reason). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
* feat: add Photos group appChristophe Besson2026-08-251-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A new group application (docs/apps.md's plug-in mechanism), following the plan in docs/photos.md. Unlike Videos/Music: several photo roots per group instead of one (photo_roots is a set, one signed op replaces it whole), a single album-grid view with no third-party matching step, and per-photo info read from the file's own EXIF at index time — no metadata service, no credential, no outbound network call at all. Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera` on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`. Node: roster.py stores photo_roots as a group_settings entry (JSON list, same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the whole set in one op, same pattern as apps_enabled; a new PhotoEnricher (indexer/enrich_photo.py) runs Pillow in its own small bounded pool, separate from the video/audio pools, producing a resized thumbnail plus the two EXIF fields — never GPS, checked by a grep-based regression test. Client: photos-app.js — one album card per directory containing images, a per-album photo grid, and a lightbox with next/previous (keyboard and buttons), zoom in/out/fit/100% starting from the actual on-screen fit percentage, and a "zip this album" button reusing files-app.js's own zip mechanism (lifted into file-utils.js's downloadDirectory so both call the same implementation). group-settings.js gets an add/remove multi-root picker, distinct from Videos/Music's single-value one. Bugs found and fixed before this ever shipped, worth keeping the story of: - enrich_photo.py read width/height from the raw image *before* applying EXIF orientation correction, and read DateTimeOriginal off the plain 0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict round-trips through Pillow either way, which is exactly what would have hidden both bugs; the regression test builds EXIF with piexif instead, matching what real hardware produces. - photos-app.js's album grouping stripped a trailing path segment from entry.path under the assumption it still carried a filename — it doesn't (files-app.js's own convention: e.path is already the containing directory), so every album collapsed one level into its parent. Found live against a real multi-folder library. - transport.js's ADMIN_OP_TYPES allowlist (already the fix for an identical bug on video_root/apps_enabled, see 4783d81) was missing photo_roots: its admin_challenge matched no pending request and was silently dropped, so saving a photo root just timed out after 30s with no error. - daemon.py pruned a thumbnail when its file left the index (root removed or reconfigured) but never forgot the content hash was "already attempted" — the same bytes reappearing under a renamed/relocated root (an operator's real workflow) were then permanently skipped, forever, with nothing to indicate why. Discarding the attempt alongside the cache entry on prune is what makes pruning actually reversible. - packages/meshbay-client's app:// protocol handler served every file with no Cache-Control header, so Chromium was free to serve a stale cached copy indefinitely — none of several `npm run sync-ui` + reload cycles during development actually picked up the new code until the renderer's disk cache was cleared by hand. Now sends Cache-Control: no-store. - the lightbox's zoomed image used flex centering (align-items/ justify-content: center) combined with overflow: auto — a well-known trap where the browser centers overflowing content by shifting it, and the leading half of that overflow (here, the top of a zoomed photo) sits outside what the scrollport can actually reach. Reported live as "unusable". Fixed by switching to top/left alignment once zoomed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL
* feat(node): add audio_root, gate Music enrichment on it like video_rootChristophe Besson2026-08-241-8/+21
| | | | | | | | | | | | | | | | | | | | | | | musicbay.md's original call — Music needs no root, tag reads are cheap so just cover the whole shared tree — didn't hold up against a real messy library: everything under every shared folder got mixed together with no way to scope Music down to an actual music collection. This adds an audio_root setting, symmetric to video_root in every respect: signed operator op (audio_root/audio_root_ack, MNP bumped to 0.10), validated against a real directory in the group's own roots before a signature is even asked for, gates tag/cover enrichment exactly like video_root gates ffprobe/TMDB (nothing runs until it's set, only files under it once it is), and a set/change fires a one-off sweep of whatever the folder already contains. The old trigger — sweep everything the instant "music" joins enabled_apps — is gone along with the root-less design it belonged to; setting audio_root is now the trigger, mirroring set_video_root's enrich_video_root_fn exactly. Test coverage mirrors the video_root suite: policy (refuse before a signature round trip, accept/store correctly) and the enrichment gate itself (nothing without a root, only files under it, sweep on set).
* feat(node): Music app node-side — indexing, MusicBrainz enrichment, protocolChristophe Besson2026-08-241-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implements the node half of docs/musicbay.md against MNP 0.8: - IndexEntry gains artist/album/track_no (reuses duration/thumb_hash/ display_title, already generic). New musicbrainz_config/_enabled and music_meta_req/_resp message pairs, mirroring the TMDB shape. - title_parse.parse_track_filename: track-number-prefix + title parsing, fallback-only (embedded tags are the primary source, unlike Videos). - indexer.enrich_audio.AudioEnricher: mutagen-based tag/embedded-cover extraction through its own bounded pool (asyncio.to_thread, no subprocess — no ffmpeg-shaped deadlock risk). Gated on "music" in a group's enabled_apps rather than a video_root-style scoped folder. - musicbrainz.py: MusicBrainzClient — no API key (unlike TMDB), just a self-imposed ~1 req/s pace and a configurable, non-default User-Agent contact string; inert (no calls at all) when no contact is configured, never sends an unidentified client. - media_cache.py: file_mbid/mbid_meta tables alongside the existing TMDB ones, cover art reusing the thumbs table via a synthetic musicbrainz:{mbid} id, pruned on file deletion. - roster.py/ops.py/webrtc_server.py: musicbrainz_contact (node-wide) and musicbrainz_enabled (per-group, from the start) as signed operator settings, ALLOWED_APPS gains "music", _do_music_meta_request resolves and caches a release-level MusicBrainz match per (artist, album). - daemon.py: AudioEnricher/MusicBrainzClient wired alongside the video ones; a group's existing library is swept when "music" is newly enabled (no video_root equivalent — see musicbay.md §2.1). 41 new tests (musicbrainz.py against a mocked transport, admin-op policy for both new settings, media_cache round-trip/pruning, enrich_audio end-to-end against real ffmpeg-generated MP3s). Full suite (common + node + hub): 1116 passed, no regressions. Client-side (music-app.js, persistent player bar) not started yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
* fix(node,hub): HEVC transcode fallback, live-add progress, per-group TMDB toggleChristophe Besson2026-08-241-14/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Three bugs found live testing the Videos app against a real HEVC/EAC3 show, plus a design change requested afterward: - Streaming always did "-c:v copy", which faithfully reports a source's real hev1 codec string but is unplayable in a browser with no HEVC decoder (most Chrome/Linux builds). The node now transcodes to H264 whenever the probed codec is browser-incompatible (media_probe.py's new BROWSER_INCOMPATIBLE_VIDEO_CODECS), with a `transcode_incompatible_video` node.toml opt-out for operators who know their viewers already decode it. - Dropping a whole season into an already-watched folder gave no scanning indicator and no progress bar: IndexProgress was only ever updated by the two bulk scan paths, never by the real-time per-file watchdog path (_schedule_update/_debounce/_update_entry). That path now accounts a "burst" the same way, without double-counting a file rewritten mid-debounce. - A stray literal "0" rendered in the video detail modal when there was no TMDB match (`meta.confidence` is 0, and `0 && x` renders "0" in JSX/htm, not nothing) — `confident` is now a real boolean. - Whether TMDB is used at all moves from a node-wide setting to per-group (OP_TMDB_ENABLED/tmdb_enabled/tmdb_enabled_ack, scoped like OP_VIDEO_ROOT): an operator running a real media-library group alongside test/demo groups on one node wants outbound TMDB traffic for the one that needs it, not all of them. The custom API token and query language stay node-wide, one shared credential/cache (tmdb_config/OP_TMDB_CONFIG, unchanged reasoning). MNP_VERSION 0.6 -> 0.7, additive. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
* feat(node,hub): add Videos group app (poster grid, flat list, TMDB metadata)Christophe Besson2026-08-241-0/+62
| | | | | | | | | | | | | | | | | | | | | Implements docs/mediacenter.md: a "Videos" group application built on the existing files index rather than a separate catalogue. On the node side, new indexer enrichment (technical probe, filename/season parsing, thumbnail generation) runs per-file once an operator has chosen a video_root for the group, plus a TMDB client for on-demand poster/metadata lookups (never client-side, thumbnails delivered over the existing chunk path). On the hub side, a new video-app.js renders a lazily-mounted poster grid or a thumbnail-only flat list, with TMDB entirely optional per group. Along the way: the global apps registry now drives Settings' default-tab picker instead of a hardcoded list, and the video_root is configured from group Settings (like uploads) rather than from Files, with the node refusing to run any TMDB/thumbnail work until one is set. Fixes several bugs found via live testing against a real library, notably a race between two effects writing the same "image ready" state that could leave a poster grid spinning forever on a same-tab revisit — see mediacenter.md §5.4 for the full account of each one.
* feat(node): persistent index cache, visible scan progress, adaptive ↵Christophe Besson2026-08-231-1/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | reconcile, and delta sync Indexer performance work, in four parts: - Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node restart no longer re-hashes every file — measured at 23 minutes for a 114 GB library on a slow disk before this, near-instant after. Hashing is deliberately kept sequential (max_workers=1): it was never actually concurrent despite the pool size, and two interleaved reads seek-thrash a spinning disk instead of going faster. - Byte-based scan progress (IndexProgress), surfaced via the loopback index-status route, the handshake ack, and a periodic INDEX_PROGRESS push to connected peers — drives a progress bar in the Create Group wizard and "add a directory" in Settings, and an animated presence dot. Guaranteed to settle back to idle via try/finally and a final push on the scanning->false transition. - The reconcile backstop's directory walks now run in the executor instead of blocking the daemon's event loop; its interval defaults to 10 min (was 60s) with adaptive backoff to 2h when nothing changes, reset on a real change or a peer connecting, and is now a per-group operator setting (signed op + group Settings UI). - INDEX_DELTA wired up (protocol support existed, nothing called it): _on_index_change now sends additions/deletions instead of rebuilding the full entries list, coalesced over a short window so a burst of file events produces one push, and the hub swarm registration for public groups only (re-)registers newly added hashes. Also fixes several bugs found while testing the above against real libraries (a 114 GB and a 100+ GB group on a USB HDD): - /api/reload blocked until the reload — including a brand-new group's full initial scan — finished, which the Electron bridge's fixed 30s call timeout turned into a hard failure on any real library. The route now fires the reload without waiting (ops.start_reload), matching add_root/remove_root's existing pattern; the wizard's own step order was fixed to wait for the group to actually appear hosted before the steps that need it (extra roots, GEK), with retries for the residual race between that and the daemon's own bookkeeping. - transport.js's hand-rolled msgpack codec had no case for uint64/int64 (0xcf/0xd3) and crashed decoding any message containing one — hit by IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB. Verified against real msgpack-encoded bytes from the Python side. - chat_hist_resp, and this change's own index_progress and set_scan_settings_ack pushes, were not routed by message type and could be handed to an unrelated pending request by the transport's "oldest pending" fallback, stalling it until its own 30s timeout and corrupting whatever received the wrong reply in its place. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
* feat(hub): split the group UI into a pluggable "applications" architectureChristophe Besson2026-08-231-0/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GroupPage's 6620-line app.js carried Chat and Files wedged in directly, with no way to add another group-level app without touching the shell itself. It is now app.js (routing, non-group pages) plus nine focused files — apps.js (the registry), chat-app.js, files-app.js, video-player.js, group-page.js (the shell), group-settings.js, hub-client.js, icon.js and file-utils.js — with docs/apps.md as the checklist for adding one (Videos/Music/Photos are sketched there, not built). Node side gained the matching enablement mechanism, mirroring member_upload exactly: a roster setting, a signed apps_enabled op enforced by _has_admin_authority, exposed in the handshake ack. Operators toggle applications per group from Settings, which also gained a small reorder: Invite, Pairing, Applications, Shared directories, Uploads, danger zone, Your devices, Members. Two bugs surfaced during the split, both missing an import across the new file boundary and invisible to node --check or a module-load probe since they only throw when the code path actually runs: - group-page.js called onRefreshAuth on a stale-token handshake rejection, but app.js never imported refreshAccessToken from hub-client.js — so a brand new member (including a group's own creator) hit "Not a member of this group" and the retry silently failed, throwing before it could refresh the token. - chat-app.js called getLocale() for message timestamps without importing it from i18n.js. Opening Chat on a group with real messages threw mid- render; uncaught, that appears to wedge Preact's render scheduler, so every button on the page stopped responding until reload. Caught the second class of bug with a proper no-undef audit across all split files (a temporarily installed ESLint 9, since the system one is too old to parse this codebase's syntax) rather than trusting grep. 827 tests pass; 6 new ones cover the apps_enabled policy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
* feat: unified group management, public groups, and activity-based sidebarChristophe Besson2026-08-201-28/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Create Group wizard (Electron-only) consolidates 6 steps across 4 interfaces into a single multi-step page: group creation on hub, node attachment, root selection via folder picker, GEK initialization, and auto-pairing — all in one flow. Browser SPA keeps its current behavior unchanged. Public group support (Option A — GEK for all groups): - All groups have GEK regardless of visibility; open-join groups auto-admit via TOFU when join_policy is "open" - Key rotation blocked for public groups (API guard + UI hidden) - Hub signaling allows WebRTC offers for nodes hosting open-join groups even when the caller isn't a member yet - attach_group writes join_policy to node.toml - Daemon loads GEK for all groups, not just private ones - Known-device path in join_request now auto-admits to open-join groups Node loopback API bridge (Electron IPC): - node:detect, node:call, node:pairing-code IPC handlers in main process - Renderer never sees tokens, paths, or keys (session token = physical access) - platform.js node namespace for UI consumption - Loopback endpoints: roots CRUD, member-upload toggle, reload Bug fixes: - Root change detection: removed premature ctx["roots"] updates from add_root and remove_root that prevented indexer retarget on reload - Duplicate offline message: global fallback now gated on !group - Signaling membership check: fallback to open-join groups for non-members Sidebar groups sorted by last_activity_at (most recent first): - New Group.last_activity_at column with Alembic migration - POST /v1/groups/{id}/activity endpoint, called on connect and chat send - Client-side sort + throttled hub updates (1/min) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): full Node admin panel — CLI parity, hot-reload, group lifecycleChristophe Besson2026-08-201-16/+201
| | | | | | | | | | | | | | | | | | | | Node admin panel (NodePage) now covers every CLI operation over MNP: group attach/detach, roster, member unpin, GEK rotate, denylist, reload. Daemon hot-loads new groups and tears down removed ones on config reload instead of requiring a full restart. Group attach/detach via MNP or local API triggers an automatic reload so the group is live immediately. Fixed GroupPage hang on first visit to a newly created group: the JWT issued at login didn't include the new group, the node rejected with not_a_member, and the token-refresh path returned without re-triggering the connect effect (Boolean(token) didn't change). Now bumps retryKey after a successful refresh so the effect re-runs with the fresh token. NodePage marks groups hosted by the node but absent from the hub with a "not on hub" badge so stale groups are visible and easy to remove. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>