aboutsummaryrefslogtreecommitdiffstats
path: root/packages/meshbay-common/src
Commit message (Collapse)AuthorAgeFilesLines
* chore: bump version to 0.16.00.16Christophe Besson5 hours1-1/+1
| | | | Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move file browsing and folder/file ops out of webrtc_serverChristophe Besson25 hours1-1/+1
| | | | | | | FilesMixin in transport/webrtc/files.py; the blocking disk helpers join _locate in webrtc/disk.py, and the lease states go to webrtc/limits.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: drop the invitation-link plan from the treeChristophe Besson2 days1-1/+1
| | | | | | | | The plan was a working document; what it decided lives in MESHBAY_DESIGN.md and MESHBAY_NODE_PROTOCOL.md. Code and tests now cite those instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(node): invitation links — a code bound to no account until redeemedChristophe Besson2 days3-0/+15
| | | | | | | | | | New invite kind "link": member of one group, once, never operator, not spendable by an active member, capped at 20 per group, cancellable by handle. Signed ops invite_link_create / invite_cancel, loopback routes, and the known-device join path now accepts a link code. Adds the plan, docs/invite-links.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat: the node signs its handshake challenge (MNP 3.4)Christophe Besson2 days2-2/+41
| | | | | | | | | node_pk in handshake_challenge is now signed over the channel binding and both nonces, so a client can check the node key before a join rather than only at the ack. Both transports; the browser and the QUIC client refuse a wrong signature and treat an absent one as an older node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore: bump version to 0.15.0Christophe Besson5 days1-1/+1
| | | | Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* style: the 98 ruff could not fix, so the linter is a signal againChristophe Besson6 days1-13/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pass before this applied ruff's own fixes. These are the ones needing a decision, and the point of doing them is that `ruff check .` now passes: a linter reporting 98 known-acceptable findings reports nothing, because the next real one arrives invisible. **Lines over 100 (70).** Mostly wrapped where they stood. Two exceptions: the aligned trailing comments in `protocol.py`'s message table were shortened rather than wrapped, because wrapping one row of a table breaks the table; and in `models.py` the column comments moved above their columns for the same reason. **Imports below the first statement (14).** `csam.py` kept its FastAPI imports under a section header halfway down the file; two node tests had a constant and a `pytestmark` wedged between two import blocks. Moved, not suppressed. **Bindings nothing reads (4).** Three in tests, where the call stays and only the name goes — `_user(client, "listener")` is there to create the user, not to return one. The fourth was in `revocation.py` and was not a lint finding at all: `_connect_and_listen` opened an httpx stream to the WebSocket URL, did `pass`, and then opened the real connection through the `websockets` library. One pointless request per connect, left over from before that library was used directly. Removed, and `httpx` with it. **`l` as a name (4)**, **semicolons (6)** in the POC spikes, and the rest. 2893 passed, the same count as the two commits before it. `meshbay_node/revocation.py` is worth a decision separately: 154 lines that nothing imports, superseded by `hub_client.maintain_ws`'s `on_revocation`. This commit only stopped it failing the linter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* style: ruff's own fixes, mechanically appliedChristophe Besson6 days5-12/+14
| | | | | | | | | | | | | | | | | | | | `ruff check .` had gone unrun long enough to report 568 errors, which is the same as having no linter: the next real finding would have been invisible in the noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the 98 it cannot fix are the next commit. What actually changed: import sorting (225), imports nobody used (87, none of them a re-export — no `__init__.py` is touched, which was the one way this could have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69) and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12 this project requires, `Optional[X]` to `X | None` (24), and f-strings with nothing to interpolate (19). Checked rather than assumed: every module in the three packages still imports, and the suite is 2893 passed — the same count, test for test, as the merge before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: cite MESHBAY_DESIGN.md and a section instead of the merged notesChristophe Besson7 days6-25/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and deleted from the tree on 2026-09-11, but ~230 comments across the three packages still named them — usually written `docs/musicbay.md §3.2`, as though the file were still in docs/. A reader had to know §16 existed to resolve any of them. They now name the section directly. Every mapping comes from §16, the concordance, which already records where each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the Videos app and §6.5 where the subject is derived data, photos -> §9.9, auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 -> §3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys -> §4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section. Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`) are retargeted too: those collide with real section numbers in the design document, so leaving them would have been worse than the named form. Four cases the concordance does not cover, each decided rather than guessed: Sub-item references into documents that no longer exist — mediacenter's `§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and §9.2 do not reproduce. The module-level citation stays; the sub-item pointer is dropped. The V-findings keep their labels but lose the dead `§10.1/` prefix. §13.8 lists V1–V13 as per-application open items, which is not what the labels mean in these comments, so pointing them at §13.8 would have been a false citation. `apps.md §5`'s virtualization requirement has no counterpart anywhere in the design document. The requirement is stated in the comment itself, so the citation is dropped rather than aimed at a section that does not say it. Comments that attributed a *sentence* to an old note — musicbay's "several thousand files" example, its "what got measured" note, its measured ~11%/~26% cover-art figures, the "original no root, whole shared tree" call — state the fact without attribution now. §9.8 does not contain those sentences and citing it for them would have been wrong. CLAUDE.md's "a reference to a document that no longer exists" row now says the concordance is for git history and out-of-tree material; the code cites sections directly. Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox leg of test_sticky_header.py's browser harness, which is broken at the browser level on this machine — headless Firefox (snap) dies with `[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders nothing, and the probe exits `{"error": "no measurement"}` after its full 90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing here can affect it: every changed line in style.css is inside a comment. Also checked: ast.parse on every changed .py, `node --check` on every changed .js, the /* */ balance in style.css, and that no changed line exceeds the width its file already used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: embedded subtitles in the video player (MNP 3.3)Christophe Besson8 days2-1/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MSE decodes no in-band text track, so a subtitle cannot ride inside the fragmented MP4 the player is fed. The node extracts one track whole, converts it to WebVTT and caches it under its own hash; the client pulls that blob through the ordinary file_req/chunk path and hangs a <track> on the video element — the same indirection as a TMDB poster or an audio transcode, which is what makes a film's subtitles extracted once in the life of the file rather than once per viewing. Whole-file also makes the cues absolute, so a seek and an audio-language change both leave the track untouched. **The ordinal counts every subtitle stream, including the ones never listed.** Only text codecs are offered: a bitmap track (PGS, VOBSUB — about a fifth of a real library) has no path to WebVTT without OCR, and one extracted anyway yields a header with no cues, which is a menu entry that shows nothing and reports no error. Numbering the survivors of that filter would give a PGS/SRT/SRT file the ordinals 0 and 1 for its text tracks and `-map 0:s:0` would then extract the PGS — the same trap `AudioTrack.ordinal` exists for, one level deeper. A fixture whose first subtitle stream cannot be decoded pins it, and the handler checks membership of the probed list, never a range. Additive and MINOR: the selector is drawn from `subtitle_tracks` in the node's own `stream_init` and from no version number, so `subtitle_req` is never sent to a peer that would not answer it. The floor stays at 3.0. Also here: a failed extraction never touches playback, a superseded reply cannot install its blob over a newer choice, and `_languageName` is shared with the audio labels — lifted by both label harnesses, since a lift that names one function stops covering the rule the moment logic moves out of it. Tests: 9 node (tracks told apart by the words in the extracted cues, not by tags), 10 client. Full suite green: 1545 node/common, 1252 hub. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
* feat: let the viewer pick the audio trackChristophe Besson8 days1-1/+21
| | | | | | | | | | | | | The streaming path mapped 0:a:0 unconditionally, so a dubbed film played in whichever language was muxed first and the others were unreachable. The node now enumerates the tracks in stream_init and honours audio_track in stream_req; switching is the seek path, since one ffmpeg carries one track. MNP 3.2, additive: the player draws its selector from the node's own list and never from a version number, so an older node is never asked for a track it would ignore. MNP_MIN_SUPPORTED does not move. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* mnp 3.1: per-account blobs the node cannot readChristophe Besson9 days2-1/+27
| | | | | | | | | | | | | | | One row per playlist plus a manifest, so starring a track rewrites that playlist rather than the whole collection. blob_enc is a BLOB, not base64 TEXT: these run to hundreds of kilobytes. user_id comes from the session and never from the message; kind is validated against a pattern; every cap refuses with a stated reason rather than truncating. Additive, so MNP_MIN_SUPPORTED does not move — a 3.0 node answers "unknown message type" and the client writes to the next one it reaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: a group can be left out of Search, and Search tries every nodeChristophe Besson11 days2-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | `search_listed` is a per-group setting on the node, changed by a signed operator op and carried in the sealed handshake ack. Search reads it after the handshake and stops there: no index is fetched, cached or merged, in any of the four views, and the page says how many groups it left out. The switch is a "Search" section in the group's settings, shown to the operator. Absent means listed, at every layer: roster default, ack default, and the client only drops a group on an explicit `false` — so an upgrade or an older node removes nothing from anyone's Search. It is a listing preference and protects nothing: the node serves the same index to Search and to the group page and cannot tell them apart, every member lists the group by opening it, and a client that ignores the flag lists it in Search too. Design §9.11 says so, so it is never described as private. The cost is one handshake per unlisted group, because only the node knows the setting. Search also took `nodes[0]` twice — for the index and for the pooled connection — the defect 4cce50f fixed on the group page only. One `connectToGroup` now walks the list the same way: a refusal about this browser stops, `not_hosted` or a failed connection moves on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XuNrwLf5EFWCMHzfoEvnpm
* chore: bump version to 0.14.0Christophe Besson11 days1-1/+1
| | | | | Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
* fix: hold every background task, in both codebasesChristophe Besson12 days1-0/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | asyncio keeps only a weak reference to a task, so a coroutine started with `asyncio.ensure_future(...)` whose result is discarded can be collected while it is still running: the loop logs "Task was destroyed but it is pending!" and the work simply does not happen. No error reaches the caller, and what is lost is whatever that coroutine was in the middle of. The node already had a guard for this, written after an abandoned stream task lost a transcode slot for good — and it read one file, `webrtc_server.py`, because that is where the defect was found. Outside that file there were nineteen sites: the hub's `chat_notify` (a notification for every member of a group), the indexer's debounce (every real-time index update), eleven in `daemon.py` including the SIGHUP reload and each enrichment pass, two in `ops.py`, and five in the loopback API. `meshbay_common.background.spawn()` is the one door. It holds the task, drops it when it finishes, and logs what it raised under the coroutine's own name — an exception in a task nobody awaits was otherwise reported by asyncio at collection time, out of context or not at all. A peer session's `_spawn` stays as it is: that one can also *cancel* what it holds, which a module-level holder cannot, because a session ends and a process does not. `test_background_tasks.py` walks every package's source and refuses a discarded handle. It parses rather than greps, so an assignment, a comprehension or an await is not mistaken for one, and it was checked against a deliberate reintroduction. A guard that stops at the edge of the file where the bug was found is a guard against that bug, not against its class. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
* fix: an empty group claim is a claim on nothingChristophe Besson13 days1-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A node that hosts no groups sends no `group_ids` on its hub socket, and the hub resolved the claim with `set(claimed_groups or authorized)` — so "I host nothing" arrived as "I host every group this account belongs to", other members' included. Such a node can serve none of them: it holds no GEK, and its own handshake refuses them with "Group not hosted on this node". `/v1/groups/{id}/nodes` answers in registration order and `_node_groups` is in-memory, so which node a client was sent to depended on who reconnected first after a hub restart. GroupPage took `nodes[0]` with no fallback. On 2026-09-11 a hub deploy at 20:14 reshuffled the registry, a second member's unconfigured node won the race, and a group stopped opening for everyone in it with its only real host online throughout. Any member could take one of their groups down, by accident, by leaving an empty node running. Four changes, because no one of them is sufficient: - the hub never widens an absent claim, and `update_groups` goes through the same ceiling as registration — it assigned its list verbatim, so the bound that makes C2 hold at authentication was one message wide - the node states the empty set rather than omitting the field - the refusal carries `not_hosted`, so a client can tell "try the next node" from "you, here, must do something first" - GroupPage walks the list instead of indexing into it The three lines involved date from 13, 20 and 23 August and each is defensible alone. The defect is in the seam, which is where the last two also were: a falsy empty collection must never mean "unspecified". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* refactor(mnp)!: one operation for an app's folders, not one per appChristophe Besson2026-09-102-28/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `video_root`, `audio_root` and `photo_roots` are gone — the messages, the signed operations, the handlers, the `ops` wrappers, the three scalars on the handshake ack, and the client's handlers for their acks. `app_directories` does the same thing for every application, keyed by the app's own registry name, and it is what the SPA has been sending. The three were the same instruction three times, differing only in the key they wrote and whether they carried a string or a list. That shape is what made adding an application mean adding a message type, an op, a handler and a widget; it also meant three validation paths, and the older ones validated nothing — a typo was stored and then quietly matched no entry, an app showing an empty tab with no way to tell "misconfigured" from "no files yet". **What stays, and why.** `Roster.LEGACY_DIR_KEYS` still reads `video_root` and friends out of `group_settings`: that is a key on an operator's disk, not on the wire, and a node upgraded into this must find its own configuration. The Search page still reads its own older cache keys, for the same reason — the cache outlives a deploy. `CTX_ALIASES` keeps only `chat`, which is the one app whose second name something still reads. The two per-app policy test files go with the messages. What only they held — the real challenge/response path from message to database, which no other test exercises — is retargeted at `app_directories` in `test_app_directories_signed.py`, and the handler's own refusals (unknown app, malformed `directories`, nobody to authorize it) join `test_app_directories.py`. Node and common suites 1368 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* refactor(common): delete the Double Ratchet implementation nothing usesChristophe Besson2026-09-101-311/+0
| | | | | | | | | | | | | | | | | | | 311 lines of Signal Double Ratchet and 167 lines of tests for it, with no caller: group chat is a key per group, per epoch, per device, and a ratchet was ruled out for it on the record — a node that serves history to devices which were not present has to hand out each chain's earliest key, which is forward secrecy of zero. Deleted for the same reason `senderkeys.py` was: an implementation kept for a use nobody has reads as an alternative somebody may reach for, and its cost is paid at every refactor that has to keep it compiling. The four comments that mention a ratchet keep doing so — they explain why this is not one, which is the part worth keeping. Common suite 158 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* refactor(common): delete the sender-key implementation nothing usesChristophe Besson2026-09-102-322/+9
| | | | | | | | | | | | | | | | | | | | `senderkeys.py` and its 13 tests implemented Signal-style sender keys, and production has never called them: chat is a key per group, per epoch, per device, derived by name. The reasoning that ruled the ratchet out stays where it belongs — in `chatbox.py`, at the top of the module that replaced it — because the argument is the useful part, and it now stands on its own instead of pointing at a file to compare against. Kept code that nothing calls is worse than absent code: it reads as an alternative somebody may reach for, and it has to be maintained past every refactor to stay compiling, which is maintenance spent on a decision already made. The three comments naming `GroupSenderKeyStore` are rewritten to say the thing they were illustrating. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* refactor(mnp)!: one answer to "may this member write", and it is the rootChristophe Besson2026-09-102-8/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | The group-wide `member_upload` switch is gone: the message, the signed operation, the field on the handshake ack, the `upload` alias on every root in the index payload, and the client's fallback path to it. Whether a member may write has been a property of each root for a while, and that is the model that survives: a single flag over the group cannot express "this library is published read-only and that folder is a drop box", which is the ordinary arrangement. What was left of the switch was a handler that logged a deprecation and acted on nothing, and a client that read `ack.member_upload` whenever the roots carried no `writable` — a second source for one question, with whichever the code consulted first deciding it. `roots.describe()` drops `upload` for the same reason: it was `writable` under an older name, and two names for one boolean is one too many. The paperclip now says "nowhere to write" rather than picking a root, in a group that has none writable. That is the honest answer; the fallback picked whatever came first and failed at send time. Node suite 1215 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* feat: MNP 3.0 — a transfer needs a leaseChristophe Besson2026-09-092-3/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stage 4 of ~/next/improve-downloads.md, the flag day. Leases become compulsory and a 2.x peer is refused at the handshake. **The bound on leaseless reads (§3.4.1) did not exist, and it is what makes the rest mean anything.** Browsing a group is never subject to a transfer slot — that is an operator decision and a requirement: a member must be able to browse a group at capacity exactly as they browse an idle one. But "not leased" cannot mean "unbounded", or a client that simply omits `tr` transfers outside every cap and the caps are decoration. A session may now read two distinct files at once without a lease: one because a viewer looks at one file, two so that prefetching the next photo stays possible. A count of files and not a byte budget, because a RAW photo is 60-80 MB and is browsing while a 40 MB archive is a download, and no size threshold separates them. Thumbnails, posters and cover art never reach this check at all — they resolve out of the node's own cache. It is a fairness control among cooperating clients, in the company of `max_concurrent_streams`, and is not a defence against a member determined to saturate a node's disk. That member is a member, and the answer to them is `member revoke`. **MNP_VERSION and MNP_MIN_SUPPORTED both move to 3.0**, on both sides. The messages are additive; the requirement is not. An opt-in switch would leave a leaseless branch reachable on every node, which is finding C6's lesson — a transport that accepted a bare JWT — one feature later. **The desktop client now checks before it connects.** The SPA is served by the hub and picks up a new client on reload; the application ships its own interface, so an un-updated one would sign in, list groups, and fail every connection with `version_too_old` — a refusal in a protocol vocabulary with nothing anyone can act on. It asks `/v1/hub/version` for `client.minimum` and says so plainly instead. An unreachable hub is deliberately *not* "too old": a captive portal or a closed laptop must not make starting the application impossible. **Every package is aligned on 0.13.0.** `meshbay-client/package.json` had drifted to 1.0.0 while the Python packages were on 0.12.0 — invisible until something compared those numbers, and then load-bearing: an installed client announcing 1.0.0 sorts above a 0.13.0 minimum and walks through the gate meant to stop it. That is stated in the code rather than left to be rediscovered; it is acceptable exactly once, because the operator is updating every client, node and hub by hand for this flag day. A new test fails if two packages ever disagree again, and another fails if the hub would refuse the client the tree builds. Node suite 1209 passed, hub suite 861 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat: resume an interrupted upload, and pause oneChristophe Besson2026-09-091-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stage 8 of ~/next/improve-downloads.md, second half, plus the gap it exposed in stage 7. **Asking where to resume.** The node identifies an upload by (member, directory, filename), so a client resuming one has to name the file — and `transfer_open`, the obvious place to ask, travels in clear. Naming it there would undo exactly what sealing this path bought in MNP 2.0: before it, the same file was ciphertext leaving a node and plaintext arriving at one. So the question is asked inside the seal that already exists, as an ordinary `file_upload` with no bytes and `chunk_index: -1`. The node writes nothing, creates no state, reserves no name, and answers with `resume_from` in the sealed ack. A node that predates it refuses the index, which the client reads as "start from the beginning" — the behaviour it had anyway — and the wait is bounded so one that answers neither does not strand an upload. The probe is answered after every check the write path makes, so it cannot ask questions about a directory the caller may not write to, and it answers only about the member who asks: otherwise one member could measure another's progress on a file they never sent, and worse, resume it. **Pausing an upload.** Reported: no pause button on an upload, even in the desktop app. Stage 7 built pause around the download path — a target declares whether it can be stopped — and an upload has no local target to ask. It was also refused by design, since a transfer handed a lease it cannot re-create must not be offered a button that would drop its slot for good. Uploads now ask for their slot rather than being handed one, and say they are pausable outright: a File is seekable and the node keeps the position. Resuming re-probes rather than trusting the client's own memory, so it works across a reconnect too. **And the slot they hold.** `_do_file_upload` never called `slots.touch(tr)`. Chunks are not gated by the lease, so the file arrived — but the node reclaimed a grant nobody appeared to be using after thirty seconds, twice, then abandoned it, and the widget follows the lease. Measured from the journal: a 3.5 GB upload read "waiting, 0 ahead" for a minute and a half while it was transferring. The download twin of this was fixed on 2026-09-08; the same omission was still here, invisible until uploads took a real lease. `test_the_upload_itself_is_sealed` now checks every message `uploadFile` sends rather than the first. Adding the probe put a second one in front of the one it was written for, and it would have kept passing while guarding nothing. Node suite 1202 passed, hub suite 850 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat(node): make the transfer caps settable, node-wide and per groupChristophe Besson2026-09-082-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Step 3 of ~/next/improve-downloads.md. Step 2 built the pools with constants; this gives them to the operator, in the two scopes they belong to. **The pools are the machine's.** `[node] max_concurrent_downloads` and `max_concurrent_uploads`, default 8, on the §2.11 pattern: node.toml for a fresh install, a roster.db override for immediate effect, editable from the Node page and from `meshbay-node transfers show|set`, applied live through the one `set_capacity` step 1 fixed. **The per-member cap is a group's.** How many transfers one member may run at once here — on the node like every other group setting (not the hub, which would have authority over someone else's disk; not node.toml, which is hand-written and needs a restart), changed by a signed operator instruction (`OP_TRANSFER_LIMITS`, subject "d=2,u=2" so what is signed names the outcome), broadcast to the group, and read live by the pools. That was the one thing step 2's shape could not express: `per_member` was a single node-wide number. `group_limits` and `member_cap(kind, member)` make it a lookup — the group's own value if it has one, the node's default otherwise — and it is deliberately the only dimension that is not node-wide. Three refusals, each with a test: - **absent means the default (2), never "unlimited".** A group that predates the setting coming back unlimited would leave the node-wide pool as the only control, which is the situation slots exist to end; - **zero is not "unlimited"**, and is not "this member may not transfer" either: the floor is one everywhere, and the CLI says to revoke the member instead; - **an unreadable row reads as unset**, not as zero — the same discipline the sealed messages follow, where a payload that does not open must never become a default state on its own. `handshake_ack` carries this member's own caps for this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn — never as "unlimited", which would have the interface contradicting the node. 1164 node, 793 hub, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* feat(node): transfer leases, pools and a queue for downloads and uploadsChristophe Besson2026-09-081-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Step 2 of ~/next/improve-downloads.md. A download is invisible to the node: it is a series of independent `file_req` messages, with nothing saying one started or ended, so there is nothing to count and nothing to cap. The lease is that missing object. `meshbay_node/transfers.py` holds the decisions and has no asyncio and no transport in it, on purpose. The failure modes this has to survive — a slot the node never gets back, a client waiting on a grant the node has forgotten — are races through a DataChannel and unprovable there; here the clock is a parameter and every method returns what changed, so the caller does the I/O and the tests drive the worst case directly. What it decides: - two pools, downloads and uploads, separate from the stream pool: different resources with different costs, and merging them makes both caps meaningless; - per-member cap checked *before* the node-wide one, so a member at their own limit queues behind their own transfers rather than holding a slot a second member has none of. Per account across their devices, or the cap becomes a function of how many tabs somebody opens; - a queue that skips a member at their cap instead of waiting for them — granting strictly in arrival order lets one member's limit stall everyone; - `tr` drawn by the client and idempotent, which is what makes a reconnect safe; - bounded per member, because unbounded queues are how a node runs out of memory politely. Every way a slot comes back, with the session teardown as the one that matters (a closed tab, a quit browser and a dead network all arrive at `shutdown_tasks`, and none of them needs a timer): explicit close, session gone, a grant nobody took up in 30 s passed to the next in line, and a granted transfer silent for 120 s reclaimed with its peer told, so a widget can offer a resume rather than sit on a lie. `GET /api/transfers` is the operator's window: when somebody reports a transfer stuck at waiting, it is the only thing that says whether the node ever had them in a queue — a log cannot, when the symptom is that nothing is happening. It carries no filename and no path, which a test pins, because this is exactly where one would be tempting. Three things found while writing it, two of them mine: - the randomised property test rejected `in_use <= cap` at once, and it was right to: lowering a cap never interrupts a running transfer, so the count legitimately sits above the new value. The invariant is that a *new* grant never happens past the cap; - the sweeper was started with `self._spawn`, which ties a task to one session's set. It died with whichever peer opened the first transfer, and every other peer's abandoned lease then stopped being reclaimed — a node that fills up over days with nothing in the log. It belongs to the node now, with its strong reference on the transport context; - the pools are node-wide while `_peer_registry` is per group (finding H1), so a slot freed in one group can grant one in another and the peer to notify is not in the notifier's registry. Silently wrong in the first version. Nothing enforces a lease yet: `file_req` is untouched, no client asks, and the node grants everything. That is step 4's flag day, and this lands alone. 1148 node, 793 hub, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
* chore: bump all packages to 0.12.0Christophe Besson2026-09-081-1/+1
| | | | | Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V8EDjk6pkYZrCbo63m2x87
* feat(chat): Tier 2 — a member verifies another member's device itselfChristophe Besson2026-09-072-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Chat messages have been signed by the sending device since MNP 2.0, but a reader had no way to know that the device belonged to the account the node named: the signature proved *a device*, and `sender_id` was still the node's word. This closes that for any account a client has already seen. **What was blocking it was not effort — the evidence was not being kept.** `_do_device_add` verified the countersignature that admits a second device and stored only `added_by_pk`: *which* key approved, never the proof. And `device_add_transcript` binds `nonce_node`, the approving connection's handshake nonce, so even a stored signature was unverifiable by anyone who had not been on that connection. `identities` gains `add_sig`, `add_nonce` and `add_ts`, added before the migration's early return — which fires on every roster widened since 2026-08-18, i.e. all of them, so putting them inside it would have meant they never arrived. `group_roster_req`/`resp` relays, sealed under a new groupbox purpose and answered to **any member of the group**, every live device of every active member with the evidence that admitted it. The node decides nothing: it hands over evidence and the client walks the chain from each account's root outwards (`_verifyRoster`). That is deliberate — the node is the party the property holds against, so it is not asked to assert trust. Two holes the tests caught while this was being built: - "no signature" was being treated as a trust root, so a node that writes the roster could put any key in an account's row and have it laundered straight into the verified set. A root is a device that names **no** countersigner. - pinning only the verified subset at first sight raised "key changed" on legitimate second devices whose countersignature predates this change. First sight pins everything the node says, because that is what trust-on-first-use means and an alarm that fires on normal events stops being read. The property, and it must not be rounded up: **once a client has seen an account, a node that later substitutes a key for it is detected. Nothing is gained at first sight**, where there is nothing to compare against — the same boundary `per-node-identity-v1.md` draws, unmoved. The cost, stated because it is real: the roster is member-visible, so every member learns how many devices the others hold and their public keys. It stays inside the group, the hub is not involved, and it is scoped per group. A member who cannot see the keys cannot check them. User-visible surface: one notice, "this account is using a key you have not seen before", in ten languages. Nothing else. 16 tests — 7 on the node (the evidence is stored, it verifies from the roster alone, a fabricated device carries none, another group's members are not disclosed), 9 running the shipped `_verifyRoster` under node against rosters built by the shipped Python: a chain of three in any order, a signature by the wrong key, one for another node, one for another account, and two fabricated devices signing each other admitting nothing. Tier 3 (operator-signed roster attestation) stays deferred, with nothing depending on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
* docs(chat): correct the claim that two devices never share a keyChristophe Besson2026-09-071-4/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | `chatbox.py` said per-device subkey derivation means "two devices never share an AES key". That is false in the deployment that exists, and stating it hid the reason the design is actually safe. Two clients of one account on one node normally hold the **same** identity key: a second browser fetches the keypair bundle from the node and recovers the existing key rather than minting a new one, and so does a fresh Electron install. Device linking — a distinct key, countersigned — is the exception, not the rule, which is why nobody is ever asked to pin anything when they open a second browser. So two clients routinely share a device key and therefore its chat subkey. What makes that safe is the nonce, not the derivation: 96 random bits, never a counter. Two independent senders under one key collide only on the birthday bound, unreachable at chat volume; two independent senders advancing one *counter* collide immediately, which is precisely what C1 and §15.0b are about. The true property is "no mutable sending state at all" — the hazard removed rather than partitioned — and the design therefore degrades correctly into the deployment as it is, where a chain-based one would have failed silently on the day someone opened a second tab. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
* Merge origin/main into the chat encryption workChristophe Besson2026-09-073-18/+203
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both sides landed a breaking MNP change and both called it 2.0, which is right: the sealed upload, the removal of `stream_seg` and mandatory chat encryption share one flag day. They are recorded as one version in `__init__.py` rather than as a race between two. The resolutions that were decisions rather than mechanics: * **`MNP_MIN_SUPPORTED` moves to "2.0".** The sealed upload alone was a *confined* break — a 1.x peer could still connect, browse, download, stream and chat, with only its uploads refused by `upload_not_sealed` — so the floor deliberately stayed at "1.0". Mandatory chat encryption ends that confinement: a 1.x peer can neither produce a sealed chat message nor read one, so it would connect, look fine, and be unable to say anything. Refusing it at the handshake is the honest form. The per-message `upload_not_sealed` path is untouched and still right if the floor is ever lowered. * **`sendChat` throws on an `error` reply**, from origin, applied to the sealed send. It matters more after this change, not less: the node now refuses a stale epoch, a malformed envelope and a device claim that is not the connection's own, so there are three new ways for a message to be rejected and none of them may look like a message that was sent. * **`req_id` supersedes the per-type routing** this branch added for `chat_keys_resp` and `device_hello_ack`. Both blocks are kept beside the existing `chat_hist_resp` one, for the same stated reason — a node too old to stamp — and their comments no longer claim to be the mechanism that closes the class. `req_id` is. * **`chat_send_probe.py` is rebuilt on origin's structure**, not beside it: two scenarios, a stub that stamps `req_id`, `music_meta_req` as the older pending request. The encrypted path is layered on — a real Ed25519 device key generated in the page, and a `chat_keys_resp` sealed by the shipped Python, because a payload the page built itself would prove only that the page agrees with the page. * **`test_reply_correlation.py` now sends a sealed message.** Its subject is which of the two messages leaving that handler carries the id; plaintext chat was only the fixture, and the node refuses one now. * `groupbox` keeps both new purposes (`upload`, `chat_keys`); `protocol.py` keeps origin's removal of `STREAM_SEGMENT` and this branch's correction of the "Double Ratchet message" comment on `CHAT_MESSAGE`, which was wrong when it was written and is wrong differently now. Full suite on the merged tree: 1993 passed, 11 failed — the same 11 that fail on a pristine checkout (2 Windows service tests, 1 apps-enabled policy, 7 transcode tests that pass in isolation, and the WebRTC invite test that hangs on its own). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
| * feat(mnp)!: seal the upload under the group keyChristophe Besson2026-09-073-1/+160
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Downloads have been encrypted under a GEK-derived key since the beginning: `file_chunk` and `stream_data` both go through `chunk_ciphertext`. Uploads never were. `file_upload` carried the filename and the raw bytes in plain msgpack, and `file_upload_ack` carried the name the node stored them under — so the same file was ciphertext leaving a node and plaintext arriving at one. There was no threat model behind that asymmetry. Both halves now travel sealed under a third groupbox purpose, HKDF(GEK, info="meshbay:upload:v1"). The filename, the destination folder and the bytes are all inside the seal; only `upload_id` and `chunk_index` stay in clear, because the node routes and orders on them before it can decrypt. This direction seals *towards* the node — it holds the GEK for its own group — and it opens the payload before it picks a destination or touches the disk. What that forced, and why none of it is optional: - `filename` was the correlation key on both sides. It cannot be: matching an ack to its request by name would hand back exactly what the seal hides. `upload_id` replaces it — client-drawn, opaque to the node, unique within a connection, never an authorization input. The property it guarded (one refusal fails one upload, not every upload in flight) is unchanged. - Refusals can no longer quote what they refused. `No directory named 'X'` becomes `No such directory in this group` plus the `code` that was already there; the client knows what it sent. - No plaintext fallback. A path that still accepts plaintext is not a sealed path, so an unsealed `file_upload` is refused with `upload_not_sealed`. Hardened while here, because what comes out of a seal is authenticated but not validated — a member can seal anything: `filename` and `data` have their types checked before any upload state is created, and `chunk_index`/`total_chunks`, which are outside the seal by necessity, can no longer raise where a refusal was meant. Tests. `test_upload_sealed.py` pins the node half: nothing identifying on the wire, tamper/wrong-key/wrong-group all refused with nothing written, and multi-chunk reassembly unchanged. `test_upload_seal_client.py` drives the shipped `uploadFile` over the shipped `crypto.js` under node and feeds its real frames to the real `_do_file_upload` — the file lands intact, and the ack the node actually produced comes back with the name it chose for a collision, which is the half a source-reading test cannot see. Both upload purposes join the JS/Python groupbox parity vectors. BREAKING CHANGE: MNP 2.0. `file_upload`/`file_upload_ack` change shape on the wire every deployed client speaks, which is MAJOR by the same rule 1.0 was — but the break is confined to uploads. `MNP_MIN_SUPPORTED` stays at "1.0", so a 1.x peer still connects, browses, downloads, streams and chats; only its uploads are refused, with a message saying which side is old. The client checks the node's version before sending a chunk, so neither side meets this as a timeout. This is the version negotiation shipped in 1.0 earning its keep: 1.0 cost a flag day, 2.0 costs a refusal code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
| * refactor(mnp)!: remove stream_seg, the last unencrypted content messageChristophe Besson2026-09-071-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `stream_seg` answered with an MPEG-TS segment as base64 with no encryption at all — the one message on the content plane that never went through a GEK-derived key. Live on both transports, answering any authenticated member. It predates `stream_data`, which does the same job properly (`chunk_ciphertext`, keyed per segment, AES-256-GCM) and has since Phase 12. Its only browser caller, `fetchStreamSegment`, was defined and never once invoked — a plaintext media endpoint with no client. Removed rather than repaired. Gone with it: `_extract_segment` and the ffmpeg semaphore in quic_server, the `fetch_stream_segment` QUIC client method, and `_b64decode` in transport.js, which had no other caller. The H6 regression test lived on this handler — it pinned `_do_stream_segment_async` to a coroutine so `subprocess.run` could not stall the event loop for thirty seconds per request. It is replaced by the property that outlives the handler: no transport carries media outside an AEAD, asserted on `stream_seg` and `data_b64` across all three transport modules. The half of H6 that survives — the live streaming path still spawns ffmpeg — keeps its own test. BREAKING CHANGE: `stream_seg` is no longer answered on either transport. No shipping client sends it. Recorded as part of MNP 2.0, whose other half — the sealed upload — carries the version bump. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
| * fix(mnp): give a reply an id, so it stops being routed by luckChristophe Besson2026-09-071-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MNP carried no correlation id. A reply named its own type and nothing else, so a client with more than one request in flight worked out which one a message answered from the message itself — and for the replies that name nothing it could not. `_dispatch` fell through to matching by arrival order, which is a guess. `_sendAndWait` had the right value all along: it keys `_pending` by `this._seqId++` and never put it on the wire. The guess fails asymmetrically, which is why it hid. The victim is not the request that was answered wrongly — it is the unrelated one that now waits out its own 30s timeout for a reply already delivered elsewhere. Live on 2026-09-06: five `music_meta_req` sat pending for over 100 seconds behind a failing MusicBrainz, and a `device_list_result` was handed to one of them. The composer is disabled while a send is in flight, so a chat message whose reply went astray the same way left the Chat tab looking frozen for thirty seconds, then unfroze on its own. The `ack` half of this was fixed on 2026-08-30 by matching on request type. That closed the instance and left the class open: a refusal has no type to match on either, and `_dispatch_message`'s catch-all answers every unforeseen failure with `{"type": "error", "detail": "Request failed"}` — 238 of this module's 240 error sends name nothing at all. `req_id` now rides on the request and comes back on the reply. On the node it is published for the whole handler in a ContextVar and stamped by `_send`: a parameter would have meant threading an argument through all 240 send sites, and asyncio copies the context into a task, so a handler that `_spawn`s its real work still answers under the right id. It is never stamped on a broadcast — those answer nothing, and the owner check in `_send` is what keeps a chat broadcast or an index push from reaching another peer looking like a reply. On the client, `_dispatch` resolves on `req_id` first and the arrival-order fallback is gone the moment a node proves it stamps (`_correlates`, armed by the handshake's own reply). The fallback stays for an MNP 1.0 node, unchanged and no wider: there it is the only thing there is, and removing it would leave device_list_result, join_result and the handshake replies reaching nobody. Two things fall out. `sendChat` refuses an `error` reply like every other request in the file — it returned it as success, which did not matter while a refusal reached the wrong caller anyway and would now show a rejected message as sent. And `_group_ctx` uses `.get`: a reload pops a removed group while sessions connected to it are open, and every request they had left raised KeyError into that same catch-all. Sealed index messages are the one exception to the fast path. They cannot be handed over until they are opened, which is asynchronous while `_dispatch` is not — resolving on the id alone gave `fetchIndex` the envelope and skipped `onIndexSync` entirely. Caught by extending `index_seal_probe.mjs` to stamp a reply the way a current node does, after the hub suite passed over it: the probe built its own frames and had never seen one. Tests, all failing before and passing after: `test_chat_send.py` drives the real ChatPanel over the real transport for both shapes of reply with an older request pending (3 of its 6 are new, and the 3 for `ack` pass either way, so it discriminates); `test_reply_correlation.py` pins the node's half — the refusals that name nothing else, the broadcast that must not be stamped, and a late reply from a spawned task answering under its own id rather than the most recent request's. Full suite: 1897 passed, same 11 pre-existing failures as before. QUIC keeps its own dispatch and is not stamped. It is disabled by default and no browser request reaches it, but the asymmetry is real. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dn1xYx9uT69mCB6UDvyKAN
* | feat(chat): encrypt group chat under per-device epoch keys (MNP 2.0)Christophe Besson2026-09-078-22/+326
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Chat messages are sealed with AES-256-GCM under a key derived per group, per epoch, per *device*, and signed over the ciphertext with the device key the node pinned. The node relays and archives; it cannot read a message. There is no switch. MNP goes to 2.0 and MNP_MIN_SUPPORTED moves with it, so a 1.x peer is refused at the handshake with `version_too_old` rather than admitted and then unable to speak. An opt-in flag was designed and rejected: every node is a test node, so it would have bought nothing and left a plaintext branch reachable — C6's lesson one feature later. A test reads the source and refuses any code that consults a `chat_encrypted` setting. Not Sender Keys, and `senderkeys.py` is now documented as unused. With distribution under the group key and a node that serves history to devices which were not present, the node must retain each chain's earliest key, and a chain key at iteration i yields every message key from i on by pure HKDF — forward secrecy is zero either way. What the ratchet was left buying was stateful client code with silent failure modes, three of them reproduced: any member could sign as any other, a second device dropped the first's chain, and the skipped-key cache grew without bound. The reasoning is in docs/chat-sender-keys.md, which is the specification and the decision record. Epochs, not rotation: the epoch key is wrapped under the group key at delivery and never stored under it, so `gek_rotate` is a re-wrap. A group-key-derived archive key would have made every message ever sent unreadable on the first `member unpin`, which is the documented step after removing a member. A new epoch opens on member revoke/unpin, device revoke and `gek_rotate`; old epochs are kept and still delivered, so history stays readable to everyone who could already read it, and nothing anywhere deletes one. Three prerequisites this needed, each a live defect on its own: * The peer registry was keyed by user_id, so one account's second device evicted the first and the broadcast skipped recipients by account — a person's phone never saw what they typed on their laptop. * The handshake authenticated an account, never a device. `device_hello` (additive, signed, refused unless the key is a live device of this account in the node's own roster) is what lets the node refuse a member claiming somebody else's key. * `_admin_exec_file_delete` authorized against the exact uploading key, so device linking had already broken deleting your own file from your other device. It now authorizes against any non-revoked device of `uploader_id`. Found by driving the real panel over the real transport, not by reading source: `chat_keys_resp` was routed by arrival order and handed to an unanswered `media_meta_req` — the original frozen-tab defect in a message type that did not exist when that probe was written. And `_asText` had been deleted with an unrelated helper beside it; its only caller sits inside a promise the panel catches, so every conversation rendered empty with nothing in the console. Existing node data is migrated by QE/migration/migrate_chat_encryption.py (not versioned, per the QE rule), run with the node stopped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
* feat(node): Phase 2 server side — one directory setting for every appChristophe Besson2026-09-062-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `video_root` (a string), `audio_root` (a string) and `photo_roots` (a list) said the same thing three ways: three roster accessors, three ops, three MNP messages, three admin-op subjects. They become `set_app_directories(app_key, paths)` and its single-directory wrapper, stored under `<app>_directories` and keyed by the app's registry name — so an application can be added without touching this layer, which is the whole claim of the plugin architecture. The three old names still work. Their MNP messages are handled, and the roster falls back to the old key when the new one is unset, so a node upgraded into this keeps working with no migration step — the plan called for a script, and a script nobody runs on the machine where it matters is worse than a fallback. Two things are new rather than moved: The paths are validated. The setters this replaces accepted anything, so a typo — or a path left behind when a root was removed — was stored happily and then matched no entry, leaving an app showing an empty tab with nothing to distinguish "misconfigured" from "no files yet". Deliberately not `RootSet.resolve()`: that also refuses a currently-unavailable root, and an operator must be able to point an app at a library on a drive they ejected. The legacy scalar is derived, never stored. `video_root` still rides on the handshake ack for MNP 1.0 clients; kept as a second stored value it would drift from the list within one run, which reads as "it works after a restart". Also here: chat's own two settings (a directory, which must be on a read-write root because it is a destination rather than a view, and a link-preview switch gating the unfurl path — checked before the cache, or turning it off would still serve every preview already fetched), the `app_directories`, `chat_directory` and `chat_link_preview` MNP messages, the plural `<app>_directories` on the handshake ack, and `music` as the app's one identifier where storage said `audio` and the registry said `music`. The Music enricher now resolves a boundary per configured directory rather than one for the group: with several, a single boundary is wrong for all but one of them, and for Music that is the difference between reading a folder as an artist and reading it as a release. Suite: 11 failures, all pre-existing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
* feat: groups refactor Phase 1 — root RO/RW model + shared directories UIChristophe Besson2026-09-063-1/+10
| | | | | | | | | | | | Replace the upload boolean with per-root writable/removable/ejected flags. Backend: new ops (update_root, eject_root, plug_root), MNP 1.1 protocol messages, live RootSet updates so API always reflects current state, CLI root subcommand (add/remove/set/list/eject/plug). Frontend: SharedDirectoriesTable with optimistic toggle switches, eject/plug in Files and Settings, upload gated on root.writable, ejected-root filtering in all media apps, updated Create Group wizard, 10-locale i18n. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): indexing v2 — partial-read hashing for files above 40 MBChristophe Besson2026-09-061-0/+2
| | | | | | | | | | | | | | | | | Files above 40 MB are no longer read in full. Instead, blake3 hashes 45 MB of samples (first 20 MB + last 20 MB + 5 MB at 50% offset). Files at or below 40 MB are unchanged (full read, hash_version 1). A new `hash_version` field on IndexEntry (default 1) travels on the wire and through the cache so both versions coexist without breaking existing nodes or clients. The IndexCache auto-migrates its schema on open (ALTER TABLE), so no manual step is required on upgrade. A standalone migration script is available in QE/migration/ for operators who want to preview or force a full re-hash. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: bump all packages to 0.11.00.11Christophe Besson2026-09-051-1/+1
| | | | Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: tolerate clock skew when verifying JWTs (leeway 60s)Christophe Besson2026-09-041-1/+9
| | | | | | | | | | | | | | | A client whose clock is a little fast could not connect at all: the MNP handshake verified the hub-issued token with no leeway, so a token whose `iat` was a few seconds ahead of the node's clock failed with "the token is not yet valid (iat)". Seen against a freshly-resumed VM guest. `meshbay_common.handshake.JWT_LEEWAY_SECONDS = 60` is the shared value; applied to the handshake, the node's own hub-token decode, revocation-token verification, and the hub's access-token decode. 60s absorbs NTP-level skew without meaningfully widening the window on a stolen token (they already carry a jti and an exp). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* feat!: MNP 1.0 — seal index and handshake_ack under the group keyChristophe Besson2026-09-033-4/+216
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `index_sync`, `index_delta` and the `handshake_ack` config payload now travel sealed under a GEK-derived subkey (`meshbay_common/groupbox.py`, mirrored by `sealGroup`/`openGroup` in `crypto.js`). Only `type`, `v`, `group_id` and the ack's `node_pk`/`proof`/`sig` stay in clear — a receiver must route and authenticate before it would trust a decryption. Verify, then decrypt. The ack line is integrity, not confidentiality: the signed handshake transcript names no ack field, so `is_node_admin`, `enabled_apps`, `video_root` and the rest were authenticated by the DTLS channel alone. The index line is defence in depth against a repeat of C1/C6 — a peer served before the handshake completes now gets ciphertext, not filenames. Nothing against an observer, the hub, or a member; that is the whole claim. `index_progress` stays clear (D3, counters only). Chat is out of scope. Failure is fatal: a payload that does not open ends the session naming the message type — never an empty index or an empty `enabled_apps`, both of which are legitimate states. Version negotiation ships here too (phase 15.6, brought forward): `v` + `v_min` on `handshake` and `handshake_challenge`, refused with `version_too_old` / `version_too_new` / `version_unreadable`. The flag day was already being paid for; the next breaking change now costs a refusal message. BREAKING CHANGE: breaks the WebRTC wire every deployed client speaks. Hub and every node must deploy together; the SPA is served by the hub, so a browser picks up the new client on reload. See MESHBAY_NODE_PROTOCOL.md §11.1a, §13.1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HkzbhmMmK8PqQBtGz5zCvY
* refactor!: one file_chunk and index_sync encoder for every transportChristophe Besson2026-09-033-18/+102
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `file_chunk` and `index_sync` were each built twice, once per transport, and the two copies did not agree. WebRTC sent binary, unsigned chunks carrying a `file_id`; QUIC sent base64 fields, two BLAKE3 hashes, a per-chunk Ed25519 signature and no `file_id`. `index_sync` was plain entries on one transport and a `GroupIndex.serialize()` envelope on the other. One message type, two shapes, one consumer each, and nothing that failed when they drifted — finding C6 one size down, in the two places the handshake unification did not reach. Phase 9.15 moved WebRTC to the binary format and dropped the per-chunk signature; the QUIC encoder was never brought along. It is dropped here rather than reintroduced: the AES-GCM tag authenticates the ciphertext under a GEK-derived key, and since C3 the node authenticates itself once in the handshake instead of once per megabyte. `meshbay_common.protocol` now owns the chunk codec (`chunk_ciphertext`, `file_chunk_wire`, `file_chunk_plaintext`) and `meshbay_node/transport/wire.py` the index builder, which also absorbs the delta the daemon used to hand-build. `test_transport_wire_parity.py` fails if either server grows its own copy back. `ChunkRequest`/`ChunkResponse` are deleted. `ChunkResponse` described the QUIC half while reading like the contract for both, which is what made the fork hard to see at all. BREAKING CHANGE: MNP 0.15 changes the encoding of `file_chunk` and `index_sync` on the QUIC transport. The WebRTC shapes are byte for byte unchanged and no QUIC client ships, which is why this is a MINOR bump; a deployed QUIC peer would have made it MAJOR. Also fixes a test fixture that put a `Path` where the daemon puts a `RootSet`. Nothing caught it: the old QUIC index handler never touched `roots`, and `entry_abs_path` fell through `Path.resolve(strict=...)`, reading the virtual path as a truthy flag and returning the right file by accident. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
* chore: release 0.10.00.10Christophe Besson2026-09-011-1/+1
| | | | | | | | | | | | All three packages (common, hub, node) bump 0.9.0 -> 0.10.0 together. Highlights since v0.9.0: email verification for registration, email change and invitations; passphrase change and account recovery; reCAPTCHA v2 on Register and Password Reset; node JSON-only control API with the Node page absorbing the admin dashboard; WebRTC STUN fallback fix; assorted hub UI fixes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QNPfgH6VWcRzJDZGuzy1jJ
* feat: passphrase change and account recovery (auth-confirm)Christophe Besson2026-09-012-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The passphrase derives two independent client-side values: auth_key (the hub verifier) and bundle_key (AES-GCM key for the per-node identity bundles, which live on nodes and never on the hub). Changing or recovering a passphrase is therefore two operations — swap the hub verifier, and re-wrap every reachable node's identity bundle. Flow A — change a known passphrase (Profile page) - POST /v1/users/password re-proves the current passphrase, swaps pw_hash/salt/version, revokes every refresh token and returns a fresh pair so the tab that made the change stays signed in. - MeshBayTransport.rewrapAllNodes: for every group's online node, connect with the old key, read the identity off the handshake, store it back under the new key. Returns updated / unreachable / failed so the UI can point at the operator-unpin fallback for the gaps. Always-shown confirmation dialog listing reachable and unreachable groups. Recovery key - keyderive.js generateRecoveryKey (32 random bytes, grouped Base32) and deriveRecoveryKey (HKDF-SHA256, domain meshbay:recovery:v1:<username>). - Every per-node identity gets a second copy wrapped under the recovery key: keypair_bundles.bundle_enc_recovery (node-only column, added in _SCHEMA_KEYPAIR and via a PRAGMA-guarded ALTER for existing DBs), carried on keypair_bundle_store / _resp. MNP 0.13 -> 0.14, additive. - session.recoveryKey is persisted in IndexedDB (slot rk) and lazy-loaded on connect, so a group joined in any later session still leaves a recovery copy. - Shown once at registration; optionally folded into the verification e-mail as a pass-through the hub never stores or logs, with an opt-out. - Profile -> Recovery key re-loads R and backfills every reachable node via rewrapAllNodes in bundleKey mode (no passphrase re-entry). Flow B — recover a lost passphrase (#/reset, linked from sign-in) - POST /v1/users/password/reset-request {username, email}: both must be the pair on file, checked against the blind email_hash (never decrypted). A mismatch — wrong e-mail, unknown username, non-active account — takes the identical no-op path (no code, no mail, same 200), so it reveals nothing and cannot be used to spray reset mail from a username alone. 5/min, 1-hour single-use code. - POST /v1/users/password/reset {username, code, new_auth_key}: same expiry / attempts / single-use checks as e-mail verification; revokes every session and deletes every registered device key so a stored one cannot sign back in past the reset. - ResetPasswordPage: request code -> code + optional recovery key + new passphrase -> reset + sign-in -> fan-out. connect() falls back to the recovery-wrapped copy when the passphrase key cannot open bundle_enc. Without a recovery key: sign-in is restored and each group needs the operator-unpin fallback. Supporting fixes (found in live testing) - member unpin now also deletes the keypair bundle; connect() mints a fresh identity when handed a bundle it cannot open (unless _rewrapOnly, set by rewrapAllNodes), so a rejoin completes instead of dead-ending before the invite-code prompt. - A browser with no bundle key gets a passphrase prompt on the group page instead of a "go back to the browser you registered on" message. - RegisterPage / LoginPage / ResetPasswordPage trim the username so every key derivation matches the hub's stored form. Docs: docs/auth-confirm.md. Locale keys across all ten catalogues. Tests: test_password_change, test_password_reset, test_recovery_email, test_recovery_key, test_rewrap_fanout, test_bundle_store_recovery, plus additions to test_admin_ops_mnp and test_webrtc_transport. Hub suite 492 passed; node suite 741 passed (the lone test_packaging_units failure is a pre-existing RPM-spec flake, reproducible on main). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GGkxJW9br8Y9bhT8ywJ3oc
* chore: bump version to 0.9.0v0.9.0Christophe Besson2026-08-311-1/+1
| | | | | | Packaging system complete and verified on Ubuntu 26.04 and Fedora 44. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): editable node settings in the Node page (D5)Christophe Besson2026-08-291-0/+2
| | | | | | | | | | | | | Expose invite_ttl_hours, pair_ttl_hours, device_request_ttl_minutes, max_concurrent_streams and transcode_incompatible_video in the Node management panel. Changes are applied immediately via roster.db and written back to node.toml so they survive a DB wipe. On startup, roster overrides take precedence over node.toml defaults. Draft v6 §2.11 documents the design; MNP gains node_settings_set / node_settings_set_ack for the browser path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: V13 — per-card "re-match this one file" (MNP 0.13)Christophe Besson2026-08-293-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | A one-click alternative to the full "Fix match" search-and-pick flow, and reachable without SSH (`meshbay-node video rematch` clears a whole group). - MNP 0.13: tmdb_rematch / tmdb_rematch_ack (additive — an older node logs "unknown type", the button just does nothing). OP_TMDB_REMATCH, signed like tmdb_override (media_cache is shared node-wide). - media_cache.drop_tmdb_match(file_id): forgets the match AND the override marker — deliberately stronger than clear_file_tmdb, since the operator is explicitly asking for a fresh resolution. - webrtc_server: _do_tmdb_rematch / _admin_exec_tmdb_rematch, dispatch + admin-response routing, broadcasts tmdb_rematch_ack. - transport.js: rematchTmdbMatch(fileId, signFn); 'tmdb_rematch' in the admin-op allowlist; tmdb_rematch_ack handled like tmdb_override_ack. - video-app.js: a "Re-match" button beside "Fix match" in the detail modal (isNodeAdmin), then bumpMediaMetaGeneration(). video.rematch_one key in all ten locales. docs/mediacenter.md §10.1: V8–V13 marked done. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
* feat: node workflow redesign — wizard auto-config, reset, MusicBrainz contactChristophe Besson2026-08-292-9/+5
| | | | | | | | | | | | | | | | | | | | | | | | Wizard (Electron): - Auto-provisions node config (hub URL + username) from logged-in user - node:start handles both cold start and restart of misconfigured daemon - Waits for daemon to reach 'running', auto-links node key on hub - probeNode accepts intermediate states for wizard progress feedback Reset (meshbay-node reset): - Unlinks node key from hub (DELETE /me/node_key, best-effort) - Stops and disables daemon (systemctl --user disable --now) - Erases ~/.config/meshbay, ~/.local/share/meshbay, ~/.local/state/meshbay MusicBrainz contact: - Resolved from owner's hub email instead of per-node roster config - Removed musicbrainz_contact UI and WebRTC handshake field - Removed set_musicbrainz_contact/musicbrainz_contact from roster Node pairing: - Added operator pairing banner on NodePage - Added operator_paired flag to list_groups Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: release 0.8.00.8Christophe Besson2026-08-281-1/+1
| | | | | | | | | | | | | | | | | Bump the three packages to 0.8.0 (released together) and realign meshbay-common's __version__, which had drifted to 0.7.0 while the pyproject stayed at 0.6.0. Dependency pins updated to meshbay-common>=0.8.0. Protocol versions are independent and unchanged: MNP 0.12, MHP 0.1. The Electron client stays on its own 0.1.0 track (hub MIN_CLIENT_VERSION). Since 0.6/0.7: public-groups admin switch with full server-side enforcement and a group Revoke action; chat link previews (node-side URL unfurl, SSRF- gated); chat composer focus + scroll-to-bottom on tab entry; whole-group "Filter files" search. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018gKJ85aZyvEwarXMFzFEwi
* feat(chat): link previews for pasted URLsChristophe Besson2026-08-282-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Paste an http(s) link in a group's chat and it unfurls into an OpenGraph card — title, description, site name, and image — the way WhatsApp/Signal/ Slack do it. The fetch is the node's, never the browser's or the hub's. The browser cannot: a strict img-src/connect-src and CORS block it, and a direct fetch would leak every reader's IP to the linked host on each render. The hub must not touch group content (draft-v6 §2.5). The node already fetches third-party metadata for the Videos and Music apps, over the same authorised path. Flow mirrors media_meta_req: the client sends `link_preview_req {url}`, the node replies `link_preview_resp` with the card fields (or `ok: false`), and any OG image is stored under its blake3 in the existing media_cache thumb store — the client then fetches it via the normal file_req path, exactly like a poster. Nothing durable is added: the card text lives in a bounded in-memory TTL cache on the node (draft-v6 §2.7 — enrichment on demand, the asking device caches), and MNP goes 0.11 → 0.12 (additive: an older node logs "unknown type" and the client shows the bare link). Because the URL is chosen by a *member* and triggers an outbound request from the operator's machine, `linkpreview.safe_url` is an SSRF gate: http(s) only, no credentials, and every resolved address must be globally routable — no loopback, private, link-local, multicast or reserved range, cloud-metadata included. Redirects are followed by hand so each hop is re-checked. Residual, documented in the module: DNS rebinding between the check and connect, closed properly by pinning the checked IP — a follow-up. Also fixes a long-standing chat annoyance the preview cards made worse: opening the Chat tab landed a screen or two above the newest message because the scroll-to-bottom ran before attachment thumbnails and (now) preview cards had loaded and grown the content. A ResizeObserver keeps the view pinned to the bottom through late content growth, and does nothing once the reader scrolls up. Tests: test_linkpreview.py (the SSRF gate and the OpenGraph parse, incl. redirect re-validation and image downscaling) and test_link_preview_request.py (reply shape, the media_cache image round-trip, the result cache). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018gKJ85aZyvEwarXMFzFEwi
* feat: add Photos group appChristophe Besson2026-08-253-3/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A new group application (docs/apps.md's plug-in mechanism), following the plan in docs/photos.md. Unlike Videos/Music: several photo roots per group instead of one (photo_roots is a set, one signed op replaces it whole), a single album-grid view with no third-party matching step, and per-photo info read from the file's own EXIF at index time — no metadata service, no credential, no outbound network call at all. Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera` on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`. Node: roster.py stores photo_roots as a group_settings entry (JSON list, same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the whole set in one op, same pattern as apps_enabled; a new PhotoEnricher (indexer/enrich_photo.py) runs Pillow in its own small bounded pool, separate from the video/audio pools, producing a resized thumbnail plus the two EXIF fields — never GPS, checked by a grep-based regression test. Client: photos-app.js — one album card per directory containing images, a per-album photo grid, and a lightbox with next/previous (keyboard and buttons), zoom in/out/fit/100% starting from the actual on-screen fit percentage, and a "zip this album" button reusing files-app.js's own zip mechanism (lifted into file-utils.js's downloadDirectory so both call the same implementation). group-settings.js gets an add/remove multi-root picker, distinct from Videos/Music's single-value one. Bugs found and fixed before this ever shipped, worth keeping the story of: - enrich_photo.py read width/height from the raw image *before* applying EXIF orientation correction, and read DateTimeOriginal off the plain 0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict round-trips through Pillow either way, which is exactly what would have hidden both bugs; the regression test builds EXIF with piexif instead, matching what real hardware produces. - photos-app.js's album grouping stripped a trailing path segment from entry.path under the assumption it still carried a filename — it doesn't (files-app.js's own convention: e.path is already the containing directory), so every album collapsed one level into its parent. Found live against a real multi-folder library. - transport.js's ADMIN_OP_TYPES allowlist (already the fix for an identical bug on video_root/apps_enabled, see 4783d81) was missing photo_roots: its admin_challenge matched no pending request and was silently dropped, so saving a photo root just timed out after 30s with no error. - daemon.py pruned a thumbnail when its file left the index (root removed or reconfigured) but never forgot the content hash was "already attempted" — the same bytes reappearing under a renamed/relocated root (an operator's real workflow) were then permanently skipped, forever, with nothing to indicate why. Discarding the attempt alongside the cache entry on prune is what makes pruning actually reversible. - packages/meshbay-client's app:// protocol handler served every file with no Cache-Control header, so Chromium was free to serve a stale cached copy indefinitely — none of several `npm run sync-ui` + reload cycles during development actually picked up the new code until the renderer's disk cache was cleared by hand. Now sends Cache-Control: no-store. - the lightbox's zoomed image used flex centering (align-items/ justify-content: center) combined with overflow: auto — a well-known trap where the browser centers overflowing content by shifting it, and the leading half of that overflow (here, the top of a zoomed photo) sits outside what the scrollport can actually reach. Reported live as "unusable". Fixed by switching to top/left alignment once zoomed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL
* feat(node): add audio_root, gate Music enrichment on it like video_rootChristophe Besson2026-08-243-1/+15
| | | | | | | | | | | | | | | | | | | | | | | musicbay.md's original call — Music needs no root, tag reads are cheap so just cover the whole shared tree — didn't hold up against a real messy library: everything under every shared folder got mixed together with no way to scope Music down to an actual music collection. This adds an audio_root setting, symmetric to video_root in every respect: signed operator op (audio_root/audio_root_ack, MNP bumped to 0.10), validated against a real directory in the group's own roots before a signature is even asked for, gates tag/cover enrichment exactly like video_root gates ffprobe/TMDB (nothing runs until it's set, only files under it once it is), and a set/change fires a one-off sweep of whatever the folder already contains. The old trigger — sweep everything the instant "music" joins enabled_apps — is gone along with the root-less design it belonged to; setting audio_root is now the trigger, mirroring set_video_root's enrich_video_root_fn exactly. Test coverage mirrors the video_root suite: policy (refuse before a signature round trip, accept/store correctly) and the enrichment gate itself (nothing without a root, only files under it, sweep on set).
* feat(music): transcode WMA/Musepack to AAC so they actually playChristophe Besson2026-08-242-1/+13
| | | | | | | | | | | | | | | | | | | | | Tagging and covers for these two formats landed already, but neither one decodes in any mainstream browser's <audio> element at all — a real library scan turned up 273 such files that would show up correctly in the Music app and then simply fail on click. This closes that gap: the node transcodes to AAC/M4A on request (a one-shot whole-file conversion, not live-piped like video's fMP4 segments — an audio file is small enough that streaming it buys nothing), caches the result under its own content hash the same way a TMDB poster or a MusicBrainz cover is cached, and serves it back through the ordinary file_req/chunk path. That path used to assume anything in the media cache was thumbnail-sized (single chunk, always); generalized it to slice a cached blob the same way a real file on disk gets sliced, since a transcoded track can be several MB. New MNP pair (`audio_transcode_req`/`_resp`, version bump to 0.9), shares its concurrency cap with video's transcode pool rather than getting its own — both are real ffmpeg processes on the same node. Every other audio format is untouched: this only fires for .wma/.mpc, the two extensions that need it.