diff options
| -rw-r--r-- | docs/MESHBAY_NODE_PROTOCOL.md | 2197 | ||||
| -rw-r--r-- | docs/playlists.md | 516 |
2 files changed, 2713 insertions, 0 deletions
diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md new file mode 100644 index 0000000..f7e8c48 --- /dev/null +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -0,0 +1,2197 @@ +# MeshBay Node Protocol (MNP) + +**Wire version:** `3.0` — `meshbay_common/__init__.py` (`MNP_VERSION`) +**Oldest peer accepted:** `3.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) +**Normative implementation:** `meshbay-common` (`protocol.py`, `handshake.py`, +`groupbox.py`, `chatbox.py`, `adminop.py`, `join.py`, `device.py`, `crypto.py`, +`webcrypto.py`), `meshbay-node` (`transport/wire.py`, `transport/webrtc_server.py`, +`transport/quic_server.py`, `transfers.py`, `uploads.py`), browser client +(`meshbay-hub/static/transport.js`, `static/crypto.js`). +**Document status:** descriptive specification of the protocol as implemented on +2026-09-10. It describes the protocol as it stands. Where the code and this document +disagree, the code is authoritative and this document is the thing to fix. + +**It is self-contained.** Every rule below is given with the reason it exists, in +place — a reader should never have to open a second file to find out what a rule is +protecting. The only outward references are to source files, which are the authority +for details this document rounds off. + +--- + +## 1. Scope + +MNP is the protocol spoken between a **client** (browser SPA, desktop client, CLI) +and a **node** (the daemon that holds a group's files and the group key). It covers: + +* mutual authentication of client and node, bound to the concrete transport channel; +* delivery of the group key (GEK) to an identity the node has pinned; +* the content plane — index, file chunks, uploads, video streaming, chat; +* the transfer slots a download or an upload runs under, and the caps on them; +* operator-authorized administration of the node, signed with the operator's key. + +MNP is **not**: + +* the protocol between node and hub — that is MHP (`0.1`), which carries signaling, + revocation push and presence, and is out of scope except where a step of MNP + depends on it (§4); +* a discovery mechanism. Presence comes from the hub's socket registry; `ping`/`pong` + exists only for liveness on an *already open* channel, because opening a connection + costs a full ICE/DTLS handshake (measured 0.6–7 s). + +### 1.1 Design invariants + +These hold for every exchange described below. They are the reason the protocol has +the shape it has, and each is argued where it is applied. + +| # | Invariant | +|---|---| +| I1 | **The hub is not trusted with content or keys.** It issues JWTs and relays SDP. A valid JWT is necessary but never sufficient: every session must additionally prove possession of the group key. | +| I2 | **Nothing arriving over MNP contributes key material.** The node generates the group key itself, and each chat epoch key too, and wraps them for keys peers have proved they hold. No message exists by which a member hands the node key material, and none may be added. | +| I3 | **Authority over the node comes from the node's own roster**, never from a token claim. Privileged operations are authorized by an Ed25519 signature over a structured transcript (§10). | +| I4 | **Every signed or MAC'd transcript is domain-separated and length-prefixed.** Bare concatenation is forbidden: without the lengths, two different field splits produce the same bytes, and a signature over one is a signature over the other. | +| I5 | **Every proof is bound to the channel it was made on.** An absent channel binding is a refusal, never a degraded proof (§6.4). | +| I6 | **Authentication is mutual.** The node proves group-key possession over a client-chosen nonce and signs the transcript with its long-term key; the client pins that key per node (§6.1, step 11). | +| I7 | **Per-group isolation.** Peer registry, chat store, chat epoch keys and index are resolved per group on a multi-group node. | +| I8 | **A message that must open under the group key, and does not, ends the session.** Never a default, never an empty result: an unopenable `enabled_apps` would read as "the operator disabled every app" and an unopenable index as "the group is empty", both indistinguishable from legitimate states (§11.1, §6.6). | +| I9 | **Both peers declare the protocol range they speak, and check the other's.** A mismatch is a refusal with a code, not a field that turns up missing (§13). | +| I10 | **A requirement only a newer peer can meet is enforced at the handshake, not per message.** The alternative — an opt-in switch, "enforce it for peers that speak the new version" — leaves the permissive branch reachable on every node, and the branch left open is the one that gets used (§13). | + +--- + +## 2. Notation + +``` +C the client (browser SPA, desktop client) +N the node daemon +H the hub (signaling and token issuance only) +X -> Y msg X sends message type `msg` to Y +[...] optional / conditional +|| byte concatenation +LP(x) len(x) as 4-byte big-endian, followed by x +b64(x) standard base64 of x, as an ASCII string +``` + +Field names are given exactly as they appear on the wire. A message is a msgpack map; +`type` and `v` are present on every message the node emits, and `type` on every message +it accepts. + +--- + +## 3. Framing and encoding + +### 3.1 Frame format + +Identical on every transport: + +``` ++--------------------------------+------------------------------------------+ +| length : uint32, big-endian | payload : msgpack map (use_bin_type=true) | ++--------------------------------+------------------------------------------+ + 4 bytes `length` bytes +``` + +* WebRTC: frames are written to a single ordered, reliable `DataChannel` named by the + client; the receiver accumulates bytes and extracts complete frames + (`_DataChannelBuffer`). A frame may span several DataChannel messages, and one + DataChannel message may carry several frames. +* QUIC: each **bidirectional stream** carries one request/response exchange; the + handshake runs on the first stream (`_StreamBuffer`, same extraction logic). + +### 3.2 Size limits + +| Bound | Value | Where | +|---|---|---| +| Max frame **before** the client's group-key proof | 64 KiB | `PRE_HANDSHAKE_MAX_MSG` | +| Max frame **after** the proof | 64 MiB | `MAX_MSG` | +| File chunk (plaintext) | 1 MiB | `CHUNK_SIZE` | +| Video segment (plaintext, before encryption) | 256 KiB | `STREAM_SEGMENT_SIZE` | +| Upload chunk sent by the browser | 48 KiB | fits the aiortc SCTP limit after msgpack overhead | +| Upload total per file | 4 GiB | `MAX_UPLOAD_BYTES` | +| Files one session may read at once **without a transfer lease** | 12 | `MAX_LEASELESS_IN_FLIGHT` (§11.2) | + +The two-tier frame limit is not tidiness. A flat 64 MiB budget applied before +authentication let an unauthenticated peer announce a large frame and dribble bytes +into it, holding that much memory per connection for as long as it liked; a hundred +such connections is the node's memory, from peers that have proved nothing. Exceeding +the limit is a hard protocol error and the buffer raises rather than truncating — +truncating would hand a parser a valid-looking prefix of something it never received. + +### 3.3 Versioning field + +Every message carries `v`. It is **not** what decides compatibility: the version each +side speaks and the oldest it accepts are exchanged and checked once, in the first +message each peer sends, before anything else is decided (§13.1). A `v` on any later +message is informational — no handler branches on it — and a peer whose range was +refused never gets to send one. + +### 3.4 Errors + +A refusal is a frame of type `error`: + +``` +{ "type": "error", "detail": <human-readable string>, ["code": <machine code>], + ["req_id": <the request being refused>], + ["upload_id" | "tr" | "file_id": <what it is about>] } +``` + +* `detail` is authored to be safe to show a peer. Raw exception text, ffmpeg stderr and + stack traces never reach the wire — they name paths on the operator's disk and + versions of the operator's software, to somebody who asked for a file. +* `code` is the same refusal in a form the client can act on. Matching on `detail` is a + string comparison that breaks the day someone improves the wording. +* `req_id` is stamped on every reply sent while answering a request, refusals included + (§3.5). It is what makes a refusal reach the caller that earned it: `error` is the one + reply with no field of its own to be recognised by, so without the id it cannot be + routed at all. +* An upload refusal names the **`upload_id`**, never the file: the filename is inside + the seal, and quoting it back in clear would hand over exactly what sealing the upload + path is for. A transfer refusal names the `tr`; a refusal to serve an unleased read + names the `file_id`, which the caller sent in clear anyway. + +Codes in use: + +| Family | Codes | +|---|---| +| Handshake | `not_a_member` (§6.3); `version_too_old`, `version_too_new`, `version_unreadable` (§13.1) | +| Transfers | `transfer_required`, `bad_transfer_id`, `bad_transfer_size`, `bad_kind`, `not_your_transfer`, `too_many_queued` (§11.2) | +| Upload | `upload_not_sealed`, `no_group_key`, `upload_incomplete`, `bad_chunk_encoding`, `bad_chunk_index`, `invalid_filename`, `no_roots`, `no_such_root`, `no_writable_root`, `root_read_only`, `root_unavailable`, `no_such_directory`, `already_exists`, `not_started`, `too_large` (§11.4) | +| Directories | `root_read_only`, `root_unavailable` (§11.5) | + +Everything else refuses with `detail` alone. A code is added when a client has a +different thing to *do* about the refusal — retry, re-authenticate, offer an update — +and not merely to enumerate. + +### 3.5 Request/response correlation + +**A request may carry `req_id`, and the node stamps it on every reply it sends while +answering that request.** The client draws it, the node never interprets it, and the +match is exact. + +The alternative is matching a reply to a request by arrival order, which is a guess: it +is wrong whenever two replies reorder, and it has no chance at all for the one reply +that names nothing of its own — `error`. A refusal that reaches no caller leaves the +request it belonged to waiting out its timeout while some unrelated request is resolved +with the refusal instead. + +Node-side the id lives in a task-local (`contextvars`), not threaded through every send +site, and it is stamped **only on messages going back to the session that asked**. A +handler that also pushes to other peers — a chat broadcast, an index delta — reaches +them through their own connection, where nothing is stamped, because those messages +answer no request. An explicit `req_id` already on a message wins over the ambient one. + +Routing, in order. A message carrying no id is either a push or a broadcast, and each +has a key of its own: + +| Message | Matched by | +|---|---| +| any reply carrying `req_id` | that id, exactly | +| `file_chunk` | `file_id` + `chunk_index` | +| `index_sync`, `index_delta` | queued, opened, then resolved by their id — see below | +| `file_upload_ack`, upload `error` | `upload_id`, against the uploader that drew it | +| `transfer_state` | `tr`, against the lease that drew it | +| `pong` | the echoed `token` | +| `media_meta_resp`, `music_meta_resp`, `audio_transcode_resp` | the `file_id` asked about | +| `season_meta_resp` | `tmdb_id` + `season` | +| `link_preview_resp` | the `url` | +| `admin_challenge` | the `op` field, against the pending request that named that op | +| `*_ack` from a signed op | `type` minus the `_ack` suffix, against the same key | +| `chat_msg`, `stream_*`, `*_ack` broadcasts | dedicated handlers; they are unsolicited | + +Two of those keys are worth their line. Chunks are the one request that runs several at +a time interleaved with everything else, so nothing but a key of their own can identify +them. A `pong` is sharper still: it is sent *while* other traffic is in flight, so +anything less than an exact match would hand it to whatever was waiting — resolving a +history request with a message that has no messages in it, and emptying the conversation +on screen. + +The index messages are the one case where the id is not enough on its own, and it is the +shape any future sealed reply will have: they carry an id like everything else, but +cannot be handed to their caller until they are **opened**, which the synchronous +dispatcher cannot do. They are queued, opened, and resolved afterwards under the same +id. Resolving them on arrival would give the caller an envelope — a nonce and a +ciphertext — and skip the handler that decrypts. + +Anything that arrives naming no request and matching no key is unsolicited and is +dropped rather than handed to a waiting caller. New request/response pairs that can +overlap in flight **must** therefore be distinguishable: `req_id` is the general +answer, and a discriminator of the message's own (`file_id`, `url`, `upload_id`, `tr`, +`op`) is what keeps a reply matchable without it. + +--- + +## 4. Session model + +``` + +-------------------------+ + | channel established | no MNP state yet + +-----------+-------------+ + | handshake + v + +-------------------------+ + refuse <-----+ version range checked | too old / too new / unreadable + | token authorized | JWT decoded, NOT authenticated + +-----------+-------------+ + | handshake_challenge + v + +-------------------------+ + | PRE-PROOF WINDOW | bounded: 4 fetches, 5 join attempts, + | bundles, join, device | 64 KiB frames, every event audited + +-----------+-------------+ + | handshake_response (valid client proof) + v + +-------------------------+ + | AUTHENTICATED | `_user_id` / `_group_id` set, + | full message set | frame limit raised to 64 MiB + +-----------+-------------+ + | channel closes + v + +-------------------------+ + | TORN DOWN | transfer leases released, peer + | | unregistered, tasks cancelled + +-------------------------+ +``` + +**States** + +1. *Unauthenticated.* Only `handshake` is accepted. Anything else is answered + `Handshake required`. +2. *Pre-proof.* Entered when the version ranges agree, the JWT authorizes, and the node + has a group key for the group. The identity is **decoded but not authenticated**. + The only messages accepted are the ones a peer provably needs before it can compute + a proof: `keypair_bundle_fetch`, `gek_bundle_fetch` and `join_request`. This window + is a disclosure surface a hub that forges a JWT can reach, so it is bounded and + audited (§7). +3. *Authenticated.* The client's HMAC over the handshake transcript verified. The + full message set opens, and the node has answered with its own proof and signature. +4. *Torn down.* Everything the connection held is given back, and the important word is + *deterministic*: a transfer lease is scoped to the connection precisely so that a + closed tab, a quit browser and a dropped network all arrive here and none of them + needs a timer (§11.2). + +A session is per (connection, group). `group_id` is mandatory in the handshake, so one +connection serves exactly one group; a client in two groups on one node opens two +connections. A session may additionally identify **which device** of the account it is, +with `device_hello` (§9.4) — the handshake proves the account and the group, and never +proved the device. + +--- + +## 5. Transport establishment + +MNP is transport-agnostic above the frame layer. **WebRTC is the transport**: it is +what the browser SPA and the desktop client speak, and it implements the whole protocol. +A QUIC transport is in development — see §5.2. + +Every transport shares one handshake module, and a parity test fails if one grows a copy +of its own. A second implementation of an authentication step is a second place for the +group-key proof to be skipped. + +### 5.1 WebRTC (browser and desktop client) + +The hub relays SDP only; it never sees a DataChannel byte. Non-trickle ICE: the offer +carries its candidates, with a 4 s gathering deadline after which the client offers +whatever it has (host candidates are enough on a LAN). + +``` + C (browser) H (hub) N (node) + | | | + | |<===== MHP WebSocket ====>| persistent, authenticated + | | /v1/nodes/ws | Ed25519 node auth + | | | + | create offer, gather ICE (<= 4 s) | + | | | + |--- POST /v1/nodes/{node_id}/webrtc/offer --------->| + | {sdp, ice_candidates}| | + | | authorize: shared active group, + | | or an open-join group when public + | | groups are enabled; <=16 KiB SDP; + | | <=3 pending per user; 30/min + | | | + | |--- ws {webrtc_offer, | + | | peer_id, user_id, | + | | sdp} ------------>| + | | | RTCPeerConnection, + | | | answer + ICE + | |<-- ws {webrtc_answer, | + | | peer_id, sdp} ----| + |<-- 200 {sdp, ice_candidates, peer_id} -------------| + | | | + | setRemoteDescription; DTLS; SCTP; DataChannel open + | | | + |======================= MNP frames =================| hub is out of the loop +``` + +Notes that matter to MNP: + +* The **raw answer SDP is retained before `setRemoteDescription`** — Chrome strips + `sha-256` from a multi-hash SDP, and the fingerprint is needed for the channel + binding (§6.4). +* The node answers offers **off its WebSocket read loop**: awaiting negotiation inline + would stop it reading the socket for the length of one slow ICE run — running, but + invisible to the hub. +* A 15 s timeout on the hub's side turns a silent node into `504`, not a hung request. + +### 5.2 QUIC (in development) + +A QUIC transport is being built, for the LAN, port-forwarded and hub-less cases where +signaling through the hub is unnecessary or unavailable. + +**It is not functional and is not a shipped feature.** No client speaks it, it +implements only part of the message set, and nothing in this document should be read as +a statement about what it does today. What is settled is the framing and the identity +model, and both are recorded here because they constrain the design of everything else: + +``` + C (native) N (node) + | | + |---- QUIC connect, ALPN "meshbay-mnp" ------->| TLS 1.3, self-signed node cert + |<---------------------------------------------| the certificate is the identity + | | + |==== stream 0 : MNP handshake =============== | + |==== stream n : one request/response each === | +``` + +The node's TLS certificate is not verified as a PKI chain — identity is established at +the MNP layer, and the certificate hash is the channel binding (§6.4). + +--- + +## 6. The handshake + +One implementation for every transport: `meshbay_common/handshake.py`. A parity test +fails if a transport skips a step. + +### 6.1 Full exchange + +``` + C N + | | + | 1. handshake | + | {v, v_min, token, group_id, nonce: b64(nonce_c)} | + |-------------------------------------------------------------->| + | 2. check_version() | + | - v >= our v_min | + | - v_min <= our v | + | else error{code} | + | authorize_token() | + | - EdDSA verify vs hub | + | - scope == "user" | + | - sub non-empty | + | - group_id non-empty | + | - denylist(user,jti,gp)| + | - group_id in groups[] | + | - group hosted here | + | 3. |nonce_c| >= 32 | + | 4. GEK exists for group | + | | + | 5. handshake_challenge | + | {v, v_min, nonce: b64(nonce_s), node_pk} | + |<--------------------------------------------------------------| + | | + | 5b. client checks the node's range the same way | + | | + | ....... pre-proof window (section 7) ....................... | + | keypair_bundle_fetch / gek_bundle_fetch / join_request | + | -- the client obtains a GEK to prove with | + | ............................................................ | + | | + | 6. binding = webrtc_binding(offer_fp, answer_fp) | + | proof_c = HMAC-SHA256(GEK, T("client")) | + | | + | 7. handshake_response {v, proof: b64(proof_c)} | + |-------------------------------------------------------------->| + | 8. rebuild binding; | + | refuse if empty; | + | compare_digest(proof) | + | 9. session authenticated: | + | frame limit -> 64 MiB, | + | peer registry, audit | + | | + | 10. handshake_ack | + | {v, node_pk, proof: b64(proof_n), sig: b64(Ed25519(T)), | + | nonce, ct} | + | ct = seal(GEK, "ack", "handshake_ack", group_id, | + | {is_node_admin, enabled_apps, | + | <app>_directories, ...}) | + |<--------------------------------------------------------------| + | | + | 11. verify proof_n == HMAC(GEK, T("node")) -> else refuse | + | verify Ed25519(ack.node_pk, ack.sig, T("node")) -> refuse | + | verify ack.node_pk == challenge.node_pk -> else refuse| + | pin/compare node_pk for this node_id (TOFU) -> else refuse| + | 12. THEN open ct -> else refuse (never a default config) | + | | + |========================= session open ========================| +``` + +Step 11 is not optional politeness. A client that accepts a bare `handshake_ack` +without a preceding challenge, or that skips any of these checks, reopens the hole this +step exists to close: a peer that had hijacked signaling could accept the client's +proof, ignore it, and serve a forged index, forged chat history and a forged +`is_node_admin` flag — the last of which offers the person an administration panel on +somebody else's node. + +**Step 12 comes after step 11, and the order is the point.** Everything in step 11 +decides whether this peer is worth trusting at all; opening the payload first would +mean acting on data from a peer not yet authenticated. And a payload that does not +open is a refusal, not an empty configuration — see §6.6. + +### 6.2 Transcript + +``` +T(role) = "meshbay:mnp:handshake:v1" + || LP(role) "client" | "node" + || LP(group_id) + || LP(nonce_c) >= 32 bytes, client CSPRNG + || LP(nonce_s) 32 bytes, node CSPRNG + || LP(binding) transport channel binding, MUST be non-empty + +proof = HMAC-SHA256(GEK, T(role)) +``` + +The role is inside the transcript, so a client proof can never be replayed as a node +proof. `nonce_c` is what makes the **node's** proof fresh: without it a recorded +`handshake_ack` is replayable by an impersonating peer. + +`make_proof` raises rather than returning a value when `binding` is empty or the GEK is +absent. `verify_proof` compares with `hmac.compare_digest`. + +### 6.3 Authorization rules (`authorize_token`) + +| Rule | Refusal | Rationale | +|---|---|---| +| JWT verifies under the hub's Ed25519 public key (`EdDSA`) | `Invalid JWT: ...` | | +| `scope == "user"` | `Wrong token scope` | a node-scoped daemon token must not be usable as a client token | +| `sub` non-empty | `Token has no subject` | | +| `group_id` non-empty | `group_id is required` | an absent group means no membership check to make; there is no default group, and a node's first group is not one | +| not on the denylist for `user_id`, `jti` **or** `group_id` | `Token revoked` | all three targets, and persisted to disk: a revocation that a restart forgets is not one | +| `group_id ∈ token.groups` | `Not a member of this group`, code `not_a_member` | the membership check itself — a token is proof of an account, never of a group | +| `group_id ∈ node.hosted_groups` | `Group not hosted on this node` | | + +`AuthorizedPeer` carries `user_id`, `group_id`, `username`, `jti` — and deliberately +**no user public key**. A key arriving in a token would be a key the hub chose, and the +node records the uploader's key in order to decide who may later delete a file: that +would let whoever issues tokens decide it instead. Identity keys are pinned by the +node's roster. The hub certifies accounts, not keys. + +`not_a_member` is almost always a token minted before the person was added to the +group (`groups` is baked in at login and the hub pushes no updates), so the client +refreshes once and retries on that code rather than telling a member they are not one. + +### 6.4 Channel binding + +| Transport | Anchor | Construction | +|---|---|---| +| WebRTC | both DTLS certificate fingerprints | `LP(offer_fp) \|\| LP(answer_fp)`, each the raw 32 bytes of the `a=fingerprint:sha-256` line | +| QUIC | node certificate | `LP(SHA-256(server_cert_der))` | + +An empty binding is refused on both sides (`Channel binding unavailable`) — an absent +binding is never a degraded proof, because a proof that is not bound to a channel is a +proof somebody can relay. The QUIC anchor is weaker than an RFC 5705 exporter, which +aioquic does not expose: it names the server's certificate rather than the concrete +session, so on a resumed session the anchor travels with the session ticket. Stated +here and in §14.2 rather than left to be inferred. + +### 6.5 `node_pk` in the challenge + +The node announces its public key in `handshake_challenge`, before anything is proved. +This is deliberate and safe: + +* a first-time joiner needs it *before* the ack — `join_request` signs a transcript + naming this node (§8.2), and someone who has never held the GEK cannot complete the + handshake that would prove the key; +* it is **unverified at that point**. The ack proves possession and signs the + transcript; the client refuses if `ack.node_pk` differs from the announced value; +* a wrong value only makes the node's own verification fail. + +### 6.6 `handshake_ack` fields + +Three fields are in clear, and the rest travel **sealed under a group-key-derived +subkey** (§11.1a). The split is not aesthetic: the three below *are* the +authentication, and a client verifies them in order to decide whether to trust anything +at all — including a decryption. + +| Field | Meaning | +|---|---| +| `node_pk` | node's long-term Ed25519 public key, base64 raw 32 bytes | +| `proof` | `HMAC(GEK, T("node"))` | +| `sig` | `Ed25519(sk_node, T("node"))` | +| `nonce`, `ct` | the sealed payload; everything below is inside it | +| `is_node_admin` | whether this peer is the node's operator — computed from the node's own record (`node_user_id`), never from a hub claim | +| `node_user_id` | the operator's account id, when known | +| `node_pk_x25519` | the node's X25519 public key, when configured | +| `enabled_apps` | which group applications to show. Empty/absent means "all registered ones" client-side | +| `<app>_directories` | each application's entry-point folders, keyed by the app's own registry name (`video`, `music`, `photo`, `chat`), **always a list**. This is the only form. The scalar `video_root` / `audio_root` / `photo_roots` fields that used to sit beside it are gone: one folder was never the general case, and two shapes for one answer meant whichever the reader consulted first decided it | +| `chat_directory` | where chat attachments are written. Singular because Chat genuinely has one destination; `""` means the operator has not chosen | +| `chat_link_preview` | whether the node unfurls links posted here. Absent means on | +| `chat_epoch` | the chat epoch a client must seal under right now (§11.7). There is no `chat_encrypted` beside it, because there is no switch | +| `transfer_limits` | `{download, upload}` — this member's own caps in this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn; never as "unlimited", which would have the interface contradicting the node (§11.2) | +| `tmdb_enabled`, `musicbrainz_enabled` | per-group metadata lookups | +| `tmdb_token_customized`, `tmdb_language` | node-wide TMDB config; the token itself is never sent | +| `indexing` | `{scanning, scanned_bytes, total_bytes}` so a client connecting mid-scan shows progress immediately. Never a path or filename | +| `scan_settings` | `{reconcile_interval_secs, debounce_secs}` — displayed, not enforced from here | + +Everything after `is_node_admin` is presentation state. It rides on the ack so a client +that connects after the operator configured something does not have to wait for a live +change notice to discover it. + +**Why this payload is sealed, and it is integrity rather than confidentiality.** The +node signs `T("node")`, which names `role`, `group_id`, both nonces and the channel +binding — and **no ack field at all**. Without the seal, every value in the table above +would be authenticated by the DTLS/TLS channel and nothing else. Sealing gives them an +AEAD tag from a key the hub does not hold, which is a stronger statement than any +amount of confidentiality on the index. `chat_epoch` is the sharpest example: a forged +one would have a client sealing its messages under a key the group has retired. + +On QUIC (§5.2) the payload is sealed but **empty**: it carries none of these fields, +because it serves no browser. The seal is there regardless, so that one message has one +shape on every transport — a field added later then has somewhere authenticated to go, +rather than arriving in clear beside a sealed one. + +A payload that does not open **ends the session**. It is not an empty configuration: an +`enabled_apps` that failed to open would reach the client's documented fallback — show +every registered app — which is a confident wrong answer, indistinguishable from an +operator's real choice (I8). + +--- + +## 7. The pre-proof window + +Between `handshake_challenge` and a valid `handshake_response` the peer is +*authorized* but not *authenticated*. Three message families are served there, each +because the peer provably cannot compute a proof without it. **Everything else is +answered `Handshake required`**: the dispatcher's authenticated branch begins +immediately after these three, so the table below is the window, exhaustively. + +| Message | Why it must precede the proof | Bound | +|---|---|---| +| `keypair_bundle_fetch` | the client's own identity keys for this node live in an encrypted bundle stored on it | counts against `MAX_PRE_PROOF_FETCHES` = 4; audited | +| `gek_bundle_fetch` | the wrapped group key is what the proof is computed with | same counter | +| `join_request` | a first-time member holds no group key at all. Accepted after the proof as well — an operator pairing a browser is already connected — because its authority comes from the pairing code and the signature, never from the session state | 5 attempts per connection, 20 failures per 600 s node-wide | + +Device linking (§9) is **not** in this window. `device_add_request` and every message +after it are answered only on an authenticated session, and the device budget of 5 +attempts per connection applies there. + +Exceeding the fetch budget is audited as `pre-proof fetch flood` and answered +`Too many requests`. Every fetch in this window is written to the audit log with the +message type, because this is a disclosure surface a hub that forges a JWT can reach: +the hub mints the tokens, so it can present one for any account, and what it can then +ask for is that account's *encrypted* keypair bundle. The bundle is useless without the +account passphrase, which is why the window is bounded and audited rather than closed +— and it closes for good when clients stop storing keypair bundles on other people's +nodes. + +### 7.1 Identity bundles + +``` + C N + |-- keypair_bundle_fetch {v} ----------------------->| + |<- keypair_bundle_resp {v, found, | + | [bundle_enc], [bundle_enc_recovery]} ----------| + | | + | decrypt bundle_enc with the passphrase-derived bundle key, + | or bundle_enc_recovery with the recovery key + | | + |-- keypair_bundle_store {v, bundle_enc, | after minting or re-wrapping + | [bundle_enc_recovery]} ----------------------->| + |<- ack {v, detail: "keypair_bundle_stored"} --------| + | | + |-- keypair_bundle_delete {v} ---------------------->| withdraw the backup +``` + +* The bundle is opaque to the node: it is encrypted client-side under a key derived + from the account passphrase (`keyderive.js`), and optionally a second copy under the + account recovery key. The node stores bytes and serves them back to the same + `user_id`. +* `keypair_bundle_store` is accepted **after** authentication (it is not in the + pre-proof list); the fetch is what happens before. +* A `store` omitting `bundle_enc_recovery` leaves any existing recovery copy in place. +* Identity keys are **per node**. There is nothing to carry between nodes, and an + operator who cracks the copy on their own disk gets a key that opens nothing + anywhere else. + +### 7.2 Wrapped group key + +``` + C N + |-- gek_bundle_fetch {v} ------------------------->| + |<- gek_bundle_resp {v, found, | + | [pk_eph_b64, nonce_b64, wrapped_b64]} -------| +``` + +The bundle is an ECIES wrap of the GEK for the caller's X25519 key: + +``` +sk_eph, pk_eph <- fresh X25519 keypair (node side) +shared = X25519(sk_eph, pk_recipient) +wrap_key = HKDF-SHA256(shared, salt = pk_eph, info = "meshbay:gek_wrap:v1:aes", len 32) +wrapped = AES-256-GCM(wrap_key).encrypt(nonce_96, GEK, aad = pk_recipient) +``` + +AES-GCM because WebCrypto has no ChaCha20-Poly1305; a `chacha20-poly1305` variant with +`info = "meshbay:gek_wrap:v1"` exists for native clients. The recipient's public key is +the AEAD's associated data, so a bundle cannot be re-addressed. + +`found: false` is the normal answer: per-member bundles are not stored, and the key is +produced on demand by the join path (§8). The node keeps one stored bundle of its own +(`_node_{user_id}`), which is how the daemon reloads its GEK across restarts. + +--- + +## 8. Pairing and join + +The substitution this section exists to prevent: an invite flow that fetched the +invitee's public key **from the hub** and wrapped the group key for whatever came back +would hand the group key to a hub that answered with its own — handed over by an honest +inviter following the protocol exactly, with nothing anywhere looking wrong. So the key +comes from its owner over an authenticated channel, and is bound to an identity by a +one-time code the hub never sees. + +### 8.1 Exchange + +``` + operator (paired) N invitee C + | | | + |-- invite_create ---->| (signed admin op, section 9)| + | {user_id, | | + | group_id, | | + | username} | | + |<- invite_result -----| | + | {code, expires_at, | | + | user_id, username}| | + | | | + |=== code delivered out of band, not via the hub ====>| + | | | + | |<-- handshake / challenge ----| nonce_s, node_pk known + | | | + | |<-- join_request -------------| + | | {group_id, pk_ed25519, | + | | pk_x25519, code, ts, sig}| + | | | + | | verify: attempts, node-wide | + | | failure window, key format,| + | | |ts - now| <= 120 s, | + | | group_id == session group, | + | | Ed25519(sig) over J, | + | | roster device lookup, | + | | consume_invite(code) | + | | | + | |-- join_result -------------->| + | | {ok, recognised, role, | + | | gek: true, | + | | pk_eph_b64, nonce_b64, | + | | wrapped_b64} | + | | | + | | client unwraps the GEK, then computes + | | the handshake proof and completes (section 6) +``` + +### 8.2 Join transcript + +``` +J = "meshbay:join:v1" + || LP(node_pk_b64) the key announced in handshake_challenge + || LP(group_id) "" for operator pairing, which is node-wide + || LP(user_id) + || LP(pk_ed25519_b64) the caller's own identity key + || LP(pk_x25519_b64) the encryption key the GEK will be wrapped for + || LP(nonce_s) the node's handshake nonce + || LP(ts) unix seconds, decimal ASCII + +sig = Ed25519(sk_ed, J) +``` + +Two properties carry the design: + +* **the identity key vouches for the encryption key.** Both are inside one signature, + which is what makes "wrap the GEK for the key the peer presented" safe; +* **`nonce_s` binds the join to this connection**, so a signed join cannot be lifted + onto another. + +The **code is never signed and never echoed.** It is a bearer secret: compared against +a stored `sha256(code)` and destroyed on use. + +### 8.3 Node-side decision table + +Evaluated in order (`_do_join_request`): + +| Condition | Outcome | +|---|---| +| `join_attempts >= 5` on this connection | `error: Too many attempts` | +| `>= 20` node-wide failures in 600 s | `error: Pairing temporarily locked`, audited `join_throttled` | +| key not 32 raw bytes, or bad base64 | `join_result{ok:false, reason:"invalid_keys"}` | +| `\|ts - now\| > 120` | `stale_request` | +| `group_id` non-empty and != session group | `group_mismatch` | +| signature does not verify over `J` | `signature_invalid` | +| Ed25519 key is a pinned device but the presented X25519 differs | `key_changed` | +| account has devices here, this key is not one | `unknown_device` — the way in is a device-add (§9), not a new invite | +| device known, no member row, group policy `open` | member row created (`approved_by: "open-join"`) | +| device known, a **pending invite** exists for this user | code required even for a known device; `code_required` / `code_invalid` on failure | +| device known, member row resolved | `join_result{ok, recognised:true, role}` + wrapped GEK | +| unknown device, no code, policy `open` | pin TOFU, admit, wrap (`via: "tofu"`, audited) | +| unknown device, no code, policy `invite` | `code_required` | +| unknown device, code invalid or spent | `code_invalid` | +| unknown device, code valid | pin identity, set member row from the invite, wrap | + +The member row is resolved as: this group's row, then the row for the join message's +`group_id`, then the node-wide (`""`) row — which is where an operator opening any +group finds their authority. + +`join_ok` refuses to produce a key when `roster.is_authorized(group_id, user_id)` is +false: `join_result{ok:true, gek:false, reason:"not_authorized_for_group"}`. Hub +membership alone must not produce a key. + +### 8.4 Pairing codes + +* 8 characters, Crockford base32 (no `I`, `L`, `O`, `U`), rendered `XXXX-XXXX` — 40 bits. + Reading one back is case-insensitive, dashes and spaces are decoration, and the + excluded letters fold onto the digits they resemble: a code read out over the phone + should not be able to fail in a way the node could have absorbed. +* Single use, stored only as `sha256(code)`. A password KDF over 40 uniformly random + bits would buy nothing. +* Three lifetimes, each matched to the conversation the code crosses, and all three + settable by the operator: + + | Code | Default | Why | + |---|---|---| + | member invitation | 7 days | it is sent by mail or message and answered whenever the other person next looks; a day dies over a weekend, and reissuing needs the inviter at a browser with the node online | + | operator pairing (§8.5) | 24 h | it crosses an SSH session — printed, then typed minutes later | + | device-add request (§9) | 1 h | read off one screen and typed into another, in one sitting | + + The longer window costs little: a code is single use, bound to one account, never + seen by the hub, and 40 bits do not fall to guessing in a week against the node-wide + lockout below. +* Valid for exactly one `user_id` in one group. +* Guessing is bounded per connection and node-wide, and every failure is an audit event + rather than a silent grind. +* `join_policy` is read from the **node's own configuration**, never from the hub: a + hub that could declare a group open would be handing itself the key to it. + +### 8.5 Operator pairing + +The same message with `group_id: ""`. Authority is node-wide, and the code comes from +`meshbay-node operator pair` over SSH — the hub never sees it. The reason it cannot is +worth stating plainly: the node cannot ask the hub which key belongs to its operator +without letting the hub answer with its own, which is the same substitution as §8's +invite flow, one level up, and it would make the hub node administrator everywhere. +There is **one** source of operator authority and it is the roster: a node.toml naming +an `admin_pk_ed25519` is warned about at startup and never obeyed, because a second +source of authority is a second thing to get wrong. + +--- + +## 9. Device linking + +Identity keys are per node, so one person using a browser and a desktop client holds +two keys on the same node. A second device is admitted by **a key the node already +pinned** — never by the hub, which stores no user keys and therefore cannot countersign +anything. + +Every message in this section is answered on an **authenticated** session only (§7): +both the device filing a request and the device approving it have completed a handshake +on their own connection. + +### 9.1 Exchange + +``` + new device D N approver A (already pinned) + | | | + | code <- random, 40 bits, displayed on D's screen | + | code_hash = sha256(code "\x1f" pk_ed "\x1f" pk_x) | + | | | + |-- device_add_request ->| | + | {pk_ed25519, | checks: account known here, | + | pk_x25519, | < 5 devices, |ts| <= 120s,| + | code_hash, ts, sig} | Ed25519(sig) over D_req | + |<- device_add_request_ack | + | {expires_at} | filed as pending, inert | + | | | + |=== the code is read off D's screen, typed into A ====>| + | | | + | |<--- device_lookup {} --------| + | |---- device_lookup_result --->| + | | {requests: [{pk_ed25519, | + | | pk_x25519, code_hash, | + | | created_at}, ...]} | + | | | + | | A recomputes sha256(code||keys) + | | for each candidate and keeps the match. + | | No match -> refuse before signing. + | | | + | |<--- device_add --------------| + | | {pk_ed25519, pk_x25519, | + | | code_hash, label, ts, | + | | sig over D_add} | + | | verify sig against EVERY | + | | live device of the account; | + | | take_device_request(hash) | + | |---- device_add_ack --------->| + | | {pk_ed25519} | +``` + +### 9.2 Transcripts + +``` +D_req = "meshbay:device_req:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) || + LP(pk_x) || LP(code_hash) || LP(nonce_s) || LP(ts) signed by the NEW device + +D_add = "meshbay:device_add:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) || + LP(pk_x) || LP(nonce_s) || LP(ts) signed by a PINNED device + +code_hash = sha256( code "\x1f" pk_ed25519_b64 "\x1f" pk_x25519_b64 ) +``` + +* `D_req` is **proof of possession only**. It says the caller holds the keys, never + that they belong to this account. The countersignature is what establishes that. +* `D_add` deliberately **omits the code**: the code is a bearer secret used to find the + request, never signed, never echoed. What is signed is the key pair being admitted, + so a signature collected for one device cannot admit another. +* Because both keys go into `code_hash`, a node cannot answer the approver with a + substituted key: the approver recomputes the hash from what it typed and what it was + given. Nothing here rests on a human comparing digits. +* `device_lookup` takes no hash argument. Taking one from the client was circular — the + client cannot compute the hash without already knowing the keys it is asking about. + +### 9.3 Listing and revocation + +``` + C -> N device_list {} + N -> C device_list_result {pending, devices: [{pk_ed25519, label, pinned_at, + pinned_via, added_by_pk, is_this_one}]} + + C -> N device_revoke {pk_ed25519, ts, sig over D_add for the victim's keys} + N -> C device_add_ack {revoked: pk_ed25519} +``` + +Anyone may read their own devices and nobody else's. Revocation is countersigned like +an addition, and **the last device cannot be removed** — an account with no device on +a node can only return through an operator's invitation code. A revoked device is +marked, not deleted, so a lost laptop stops being able to admit its replacement. + +Per-connection attempt budget for every device message: 5, audited on exhaustion. +`MAX_DEVICES_PER_USER` is 5. + +### 9.4 Which device is on this connection (`device_hello`) + +``` + C -> N device_hello {pk_ed25519, ts, sig over D_hello} + N -> C device_hello_ack {pk_ed25519} +``` + +``` +D_hello = "meshbay:device_hello:v1" || LP(node_pk) || LP(group_id) || LP(user_id) + || LP(pk_ed25519) || LP(nonce_s) || LP(ts) +``` + +The handshake authenticates a **group membership** (the group-key HMAC) and an +**account** (the hub's token). It does not authenticate a *device*, and an account may +hold several (§9). Without this message the node can only guess which one is talking — +and it records the uploader of every file and the author of every chat message, so a +guess there is an attribution the person cannot correct. + +What is checked, in order: the key is a live device **of this account in the node's own +roster** (never a token claim), the timestamp is fresh, and the signature verifies over +a transcript naming this node, this group and this connection's nonce. A key that is +merely well-formed proves nothing. + +Idempotent for the same key and refused for a different one: a connection does not get +to change device half way through, which would let one session's uploads and messages +be attributed to two. Sending a chat message **requires** this to have happened +(§11.7), because the `device` field a receiver verifies a signature against is checked +against the connection rather than believed. + +--- + +## 10. Operator-authorized operations + +### 10.1 Why a signature and not a token + +The hub issues JWTs, so a JWT can never establish node-level authority. Every +destructive or privileged operation is authorized by an Ed25519 signature over a +structured transcript, verified against the keys the node's roster records as holding +operator authority — read fresh on every call, so revoking a paired browser takes +effect immediately. + +### 10.2 Two-hop exchange + +``` + C (operator) N + | | + |-- <op message> {op-specific fields} --------->| + | | cheap pre-check: + | | is there any key that + | | could authorize this? + | | (_has_admin_authority) + |<- admin_challenge ----------------------------| + | {op_id, op, subject, nonce, ts, | node keeps the authoritative + | node_pk, group_id} | copy in _admin_ops[op_id] + | | + | the client REBUILDS the transcript from the announced FIELDS + | and refuses to sign if `op`/`subject` are not what the user asked for + | | + |-- admin_response {op_id, signature, op} ----->| + | | pop(op_id) - single use + | | now - ts <= 120 s + | | rebuild A from STORED state + | | verify vs roster operator keys + | | (file_delete also accepts the + | | uploader's recorded key) + | | execute via ops.py + |<- <op>_ack {op-specific fields} --------------| + | | + | some acks are ALSO broadcast to every peer in the group +``` + +### 10.3 Transcript + +``` +A = "meshbay:admin:v1" + || LP(op) e.g. "file_delete" + || LP(node_pk_b64) so a signature for node A is invalid on node B + || LP(group_id) so authority does not leak across groups on a multi-group node + || LP(subject) what is being acted on + || LP(nonce) 32 bytes, node CSPRNG, single use + || LP(ts) unix seconds, TTL 120 s + +sig = Ed25519(sk_operator, A) +``` + +The structure is the whole point. A challenge of 32 raw random bytes, signed blind, +would be an unbound signing oracle: the signed message would name no operation, no +subject, no node and no time, so a signature obtained for one purpose would be +structurally valid for any other, on any node, for ever. + +**The node never takes a signed value off the wire.** It rebuilds `A` from +`_admin_ops[op_id]`; the client rebuilds it from the announced fields. They agree by +producing the same bytes. + +### 10.4 Operation catalogue + +`subject` is what the client must display and match before signing. Where the ack is +broadcast, every connected peer in the group learns the change without reconnecting. + +| Op | Subject | Authority | Ack | Broadcast | +|---|---|---|---|---| +| `file_delete` | `file_id` | operator **or** the file's recorded `uploader_pk` | `file_delete_ack{file_id}` | no | +| `dir_delete` | path relative to the root | operator | `dir_delete_ack{dir}` | no | +| `invite_create` | invitee `user_id` | operator only (delegation designed, deferred) | `invite_result{code, expires_at, user_id, username}` | no — the code is shown once | +| `member_revoke` | `user_id` | operator | `member_revoke_ack` | no | +| `member_unpin` | `user_id` | operator | `member_unpin_ack{user_id}` | no | +| `gek_rotate` | `group_id` | operator | `gek_rotate_ack{group_id, authorized_members, note}` | no | +| `apps_enabled` | the app set | operator | `apps_enabled_ack{apps}` | yes | +| `set_scan_settings` | the interval/debounce pair | operator | `set_scan_settings_ack{...}` | yes | +| `tmdb_config` | `custom_token=yes\|no,language=...` | operator | `tmdb_config_ack{token_customized, language}` | yes (never the token) | +| `tmdb_enabled` | `enabled` | operator | `tmdb_enabled_ack{enabled}` | yes | +| `tmdb_override` | `file_id=..,tmdb_id=..,media_type=..` | operator | `tmdb_override_ack{file_id, tmdb_id, media_type}` | yes | +| `tmdb_rematch` | `file_id=..` | operator | `tmdb_rematch_ack{file_id}` | yes | +| `musicbrainz_enabled` | `enabled` | operator | `musicbrainz_enabled_ack{enabled}` | yes | +| `root_add`, `root_remove` | the root | operator | `root_add_ack` / `root_remove_ack` | no | +| `root_update` | `<root>:rw=on\|off,rem=on\|off` | operator | `root_update_ack` | yes | +| `root_eject`, `root_plug` | the root name | operator | `root_eject_ack` / `root_plug_ack` | yes | +| `app_directories` | `<app>:<dir>,<dir>,...` | operator | `app_directories_ack{app, dirs}` | yes | +| `chat_directory` | the path | operator | `chat_directory_ack{path}` | yes | +| `chat_link_preview` | `on\|off` | operator | `chat_link_preview_ack{enabled}` | yes | +| `transfer_limits` | `d=<n>,u=<n>` | operator | `transfer_limits_ack{limits}` | yes | +| `chat_epoch` | `group_id` | operator | `chat_epoch_ack{epoch}` | yes | +| `group_attach`, `group_detach` | `group_id` | operator | `group_attach_ack` / `group_detach_ack` | no | + +**Upload policy is not in this table**, and that is the design: whether a member may +write is a property of each root (`root_update`), not a switch over the group. A single +group-wide flag cannot express "this library is published read-only and that folder is a +drop box", which is the ordinary arrangement. + +`app_directories` is the **only** way an application's folders are set: one message +for every application, keyed by the app's own registry name, so adding an application +adds no message type, no signed op and no handler. + +The three narrower ops it replaced — `video_root`, `audio_root`, `photo_roots` — are +gone from the catalogue. They were the same instruction three times, differing only in +the key they wrote and whether they carried a string or a list, and that shape is what +made adding an application mean adding a message type, an op, a handler and a widget. +It also meant three validation paths, and the older ones validated nothing: a typo was +stored, matched no entry, and the application showed an empty tab with no way to tell +"misconfigured" from "no files yet". One op has one validation path, and an unknown +application name is refused rather than stored. + +Their *storage* keys survive on the node — `Roster.LEGACY_DIR_KEYS` still reads +`video_root` and friends out of `group_settings` — because that is a key on an +operator's disk rather than on the wire, and a node upgraded into this has to find its +own configuration. + +A second family of operator messages is **not** signed: `node_status`, `roster_read`, +`denylist_read`, `denylist_clear`, `node_settings_set`, `node_reload`. These are gated +by `is_node_admin()` — the authenticated session's `user_id` equals the account the +node records as its own operator (`node_user_id`), computed from the node's own state +and never from a hub claim. Three of them only read; the other three run through the +same `ops.py` entry points as the CLI and the loopback admin API. The distinction from +the signed table above is deliberate but worth stating plainly: a signed op proves +possession of an operator *key*, while these prove only that the session belongs to the +operator's *account*, which the handshake already established. + +Rules that hold across the table: + +* **No MNP message can activate a group key.** The rule (I2) targets key material + arriving from outside, not the instruction: `gek_rotate` and `chat_epoch` are allowed + precisely because the node generates the new key itself with its own CSPRNG. Initial + `gek-init` stays local — with no group key there is no completed session to carry a + signed op anyway. +* **There is no operation by which key material reaches the node** (§12). The node + wraps for a key the recipient has proved possession of, so no such message is needed — + and a path that does not exist cannot be mis-authorized, which is I10 applied to a + message instead of a version. +* An operator cannot revoke or unpin **themselves** over the connection their pin + authorizes. +* Rotation is what actually removes a revoked member's access. Revocation stops the + node serving the *next* key; the ex-member still holds the current one, and content + they already downloaded stays readable. The ack says so in words. + +### 10.5 One implementation, several front doors + +The loopback admin API, the CLI and the signed MNP handlers are three thin adapters +over the same functions in `meshbay_node/ops.py`. Those functions take the daemon +state, raise `OpError`, and know nothing about HTTP. + +One operation with two implementations means two authorization checks, and the weaker +one is the one that decides. A front door is allowed to differ in how it *authenticates* +— a signature here, a run token on loopback, an operator's shell for the CLI — and never +in what it *does*. + +--- + +## 11. Content plane + +Everything in this section requires an authenticated session (§4). All content is +encrypted under keys derived from the GEK, so a node that serves a chunk to a session +that never proved GEK possession serves ciphertext nobody can open — but the +authorization check comes first regardless. + +### 11.1 Index + +The Mesh Group Index is the list of files the node shares for one group. Entries are +content-addressed: `id` is the BLAKE3 hash of the file. + +``` + C N + |-- index_sync {v} ------------------------->| + |<- index_sync {v, group_id, nonce, ct} -----| ct = seal(..., "index_sync", ...) + | payload: {version, entries[], | + | dirs[], roots[]} | + | + | ... operator drops files into a watched folder ... + | + |<- index_delta {v, group_id, nonce, ct} ----| pushed, unsolicited, to every + | payload: {base_version, version, | peer in this group + | additions[], deletions[], | + | updates[], roots[]} | + | + |<- index_progress {v, group_id, scanning, | every ~2 s while scanning, + | scanned_bytes, total_bytes} -----------| plus once on the return to idle + NOT sealed — see below +``` + +`IndexEntry` wire fields (`index_entry_wire`): + +| Field | Meaning | +|---|---| +| `id` | BLAKE3 of the file content, hex | +| `name`, `path` | filename, and the virtual directory it lives in (`<root>/<subpath>`) | +| `size`, `type`, `added_at` | bytes; `video\|audio\|image\|document\|archive\|other`; unix seconds | +| `duration`, `width`, `height`, `thumb_hash` | filled asynchronously by enrichment | +| `uploader_id` | who uploaded it; `null` for content pre-existing on disk | +| `display_title`, `season`, `episode` | Videos app, parsed from the name/folder | +| `artist`, `album`, `track_no` | Music app, from tags or parsed | +| `taken_at`, `camera` | Photos app, best-effort from EXIF | + +`uploader_pk` exists on the dataclass but is **not** in the wire dict: it is the key +the node recorded at upload time, used server-side to authorize `file_delete`. + +Three lists, not two: `updates` carries entries whose `id` is unchanged (same content) +but whose fields changed — enrichment filling in `duration`/`thumb_hash` after the file +was first indexed with hash and size only. `diff()` only places an id there once it has +appeared unchanged in a prior snapshot. + +`dirs` and `roots` exist because directories are not index entries. Without them a +folder just created, or one emptied, does not exist as far as the UI is concerned, and +a member cannot tell "the drive is unplugged" from "it is all still there" — an +unavailable root is listed, with its content frozen rather than hidden. + +Each root is described as `{name, kind, available, writable, removable, ejected}` — and +never a path: a member is told what exists and whether it is readable, never where on +the operator's disk it lives. `roots` rides on `index_delta` as well as `index_sync`, +because a full index is only ever sent on request: without it, a root added, removed, +ejected or plugged would leave every connected client's directory table stale until +somebody reloaded the page, and the delta that tells them something changed would be the +one message unable to say what. + +`index_progress` carries counters only, never a path or filename. + +### 11.1a The sealed envelope + +A family of messages travels sealed under a subkey derived from the GEK +(`meshbay_common/groupbox.py`, mirrored by `sealGroup`/`openGroup` in `crypto.js`). +There is one subkey per **purpose**, and five purposes: + +``` +key(purpose) = HKDF-SHA256(GEK, salt = <none>, info = <purpose info>, 32 bytes) + +nonce = 12 random bytes, per message +ct = AES-256-GCM(key).encrypt(nonce, msgpack(payload), aad) +aad = "<msg_type>|<group_id>" UTF-8 +``` + +| Purpose | `info` | Seals | +|---|---|---| +| `index` | `meshbay:index:v1` | `index_sync`, `index_delta` | +| `ack` | `meshbay:ack:v1` | the `handshake_ack` configuration payload | +| `upload` | `meshbay:upload:v1` | `file_upload`, `file_upload_ack` (§11.4) | +| `chat_keys` | `meshbay:chat_keys:v1` | `chat_keys_resp` — the group's chat epoch keys (§11.7) | +| `roster` | `meshbay:roster:v1` | `group_roster_resp` — members, device keys and the evidence that admitted each (§11.7) | + +`chat_keys` is the one whose payload *is* key material: a peer that has completed the +handshake holds the group key and can open it, and anything short of that gets a +ciphertext. That is the same statement the index makes, one step stronger. + +`salt = <none>` is Python's `salt=None` and WebCrypto's `salt: new Uint8Array(0)`; +RFC 5869 extracts with a zero key either way. The subkeys are purpose-separated +rather than borrowed from a file's key space — `GroupIndex.serialize()` reuses +`chunk_key_aes` with a pseudo-file ("the index as chunk 0 of a virtual index file"), +which is a hack this deliberately does not repeat. + +**What stays in clear, and why each one has to:** + +| Field | Why | +|---|---| +| `type`, `v` | the receiver must route and version-check before it can decrypt | +| `group_id` | already in clear in the handshake; it is the AAD and selects the key | +| `node_pk`, `proof`, `sig` (ack) | they *are* the authentication — verified before a decryption is trusted (§6.1 step 12) | +| `index_progress`, in full | counters only, never a path or a filename, pushed every ~2 s for the whole length of a scan. Sealing it would buy a rough library size and cost a decrypt per push | +| `upload_id`, `chunk_index` (upload) | the node routes and orders on them before it can decrypt; `upload_id` is client-drawn, opaque, and never an authorization input | +| `transfer_open` / `_close` / `_state`, in full, and `tr` wherever it rides (§11.2) | `tr` is opaque and client-drawn, `bytes` and `chunks` are numbers, and there is no filename and no path anywhere in them. Adding one to make a log line prettier is exactly the trade this envelope exists to refuse | + +`version` and `base_version` are **inside** the payload: there is no reason to act on +a version number carried by a message that has not been authenticated. + +The AAD binds a ciphertext to its message type and its group, so an `index_sync` +body cannot be replayed as an `index_delta`, nor moved between two groups on one node. + +**What this buys, and what it does not.** It buys integrity for the ack (§6.6), and, +for the index, defence in depth against one specific class of bug: a peer that has not +completed the handshake being served data anyway. That bug is an authorization mistake +in one branch of one handler, it is easy to write, and it is invisible until somebody +reads that branch. Sealed, it leaks ciphertext instead of filenames and folder names. + +The upload is the same argument in the other direction. It seals *towards* the node, +which holds the group key for its own group and opens the payload before it decides a +destination or touches the disk — so the write path is covered by the same key as the +read path, and a file is never plaintext on one leg of its journey and ciphertext on the +other. + +It buys nothing against a network observer — DTLS/TLS already covers that — nothing +against the hub, which never sees channel traffic, nothing against a member, who holds +the group key, and nothing at rest: the index stays plain in the node's memory and the +files stay plain on the operator's disk, which is the design. Chat messages are **not** +sealed by this envelope: they have their own key hierarchy, per epoch and per device, +because the node must be able to relay and archive a message it cannot read (§11.7). +The control plane is not covered either — see §14.2. + +**Nonce collision, since `upload` is the first purpose with volume.** One subkey per +purpose and a fresh 96-bit random nonce per message: at one message per 48 KiB chunk, +2³² chunks is 200 TB uploaded under a single GEK before the collision probability +reaches 2⁻³², and `gek_rotate` exists. Deriving the nonce from the payload instead +would be worse, not better — two chunks of identical bytes are ordinary in a file. + +`GroupIndex.serialize()` is not a candidate for reuse here: it compresses with zstd, +which no browser can decompress (`DecompressionStream` offers gzip and deflate only), +so reusing it would mean shipping a WASM decoder to every client for no gain. + +**Failure is fatal, never degraded** (I8). A client that cannot open an index message +ends the session naming the message type; it never reports an empty index, because "the +group has no files" is a state a real group can be in. + +### 11.2 Transfer leases + +A download is not a message. `file_req` asks for one chunk; a client fetching a 4 GB +film sends four thousand of them, eight in flight at a time, and nothing in that says a +transfer started or that it ended. Without an object standing for the transfer itself +there is nothing to count, and therefore nothing an operator can cap. + +The **lease** is that object, and every transfer runs under one. + +``` + C N + |-- transfer_open {v, tr, kind, bytes, chunks} ----->| kind: "download" | "upload" + | | per-member cap first, + | | then the node-wide pool + |<- transfer_state {v, tr, state, kind, | + | used, cap, node_used, node_cap, | + | [ahead], [reason]} ----------------------------| state: granted | queued + | | + |-- file_req {..., tr} ----------------------------->| every chunk under this lease + |-- file_upload {..., tr} -------------------------->| says the transfer is alive + | | + |<- transfer_state {tr, state: "granted"} -----------| pushed when a queued lease + | | reaches the head + | | + |-- transfer_close {v, tr, reason} ----------------->| done | cancelled | paused + |<- transfer_state {tr, state: "closed", reason} ----| +``` + +**Six properties carry the design, and each is a decision:** + +* **`tr` is drawn by the client**, 16 random bytes, exactly like `upload_id`. Re-opening + with the same `tr` is **idempotent**, so a reconnect cannot charge a member twice for + one transfer — and re-asking is how a client recovers a grant whose push was lost. +* **A lease is scoped to the connection, never to the account.** It dies with the + session, which is what makes the reclaim deterministic: a closed tab, a quit browser + and a dropped network all arrive at the same teardown, and none of them needs a timer. +* **A lease covers a job, not a file.** A directory downloaded as a zip is dozens of + files and *one* lease. One per file would deadlock against the member's own cap: the + job cannot finish until it holds them all, and it can never hold more than its cap — + two, by default. +* **Nothing is persisted.** A restart drops every session anyway, and a lease that + outlived the process would be a slot nothing can release. +* **Leases are counted, bytes are not.** What a slot protects is concurrency — open file + handles, disk seeks, the channel buffer each transfer keeps full. +* **Per-member first, then node-wide.** A member at their own cap queues behind their + own transfers and never holds a node-wide slot a second member has none of. Reversed, + whoever arrives first takes everything. + +**Caps.** Node-wide, 8 concurrent per kind by default; per member per group, 2 by +default. A group with no value of its own gets the default, never "unlimited": reading +an absent setting as no limit would leave the node-wide cap as the only control, which +is the situation leases exist to end. The per-group value is a signed operator +operation (`transfer_limits`, §10.4, bounded to 1–32; zero is refused, because a member +who may not transfer at all is a member the operator revokes). The node-wide values are +daemon settings. A member's own caps ride on the handshake ack so the interface can say +"2 of your 2 slots are busy" rather than draw a spinner that explains nothing. + +**Queueing.** One FIFO per kind. `_pump` walks it in arrival order and **skips** a +member who is at their own cap rather than stopping at them — granting strictly in +order lets one member's limit stall every other member behind them. A queued lease is +told how many are `ahead` of it. Beyond 32 queued per member the answer is +`too_many_queued`, because an unbounded queue is how a node runs out of memory politely. + +`used` and `cap` on `transfer_state` are this member's own count and this member's own +limit **in this group** — the same value `_has_room` enforces and the same one the +handshake ack announces. Three readings of one number, and an interface that draws a +different one from the node's is an interface that offers a slot the node will queue. + +**Reclaim.** The session teardown is the primary path and it is immediate. A sweeper +runs every 15 s for whatever the teardown cannot see, and tells two failures apart: + +| Situation | Timer | What happens | +|---|---|---| +| Granted, never taken up | 30 s | Back to the tail of the queue, `reason: "not_taken_up"` — the client died between asking and starting | +| Same, three times | — | Closed, `reason: "abandoned"`. Without the bound the requeue is a permanent cycle: revoked, put back, granted again because there is room, revoked 30 s later, for ever | +| Granted, used, then silent | 120 s | Closed, `reason: "idle"`, and the peer is told, so its widget can offer a resume rather than sit on a lie | +| Connection gone | none | Everything it held, at once | + +The sweeper belongs to the **node**, not to the session that opened the first transfer. +Tying it to a session would kill it when that peer left, and every other peer's +abandoned lease would then never be reclaimed. + +**A chunk request is what "alive" looks like.** `tr` rides on `file_req` and on +`file_upload` for exactly this: it is the only signal the node has that a granted lease +is being used. Without it the sweeper cannot tell a transfer running at 20 MB/s from a +client that asked for a slot and vanished, and it revokes both. + +**Pausing is releasing.** A paused transfer holds nothing: the client closes the lease +with `reason: "paused"` and keeps its own position, and resuming asks for a *new* lease +and queues behind whatever is waiting now. The alternative — holding a slot while +paused — is a member who pauses three downloads and blocks the group. + +**Refusals.** `bad_transfer_id` (no `tr`), `bad_transfer_size` (`bytes`/`chunks` not +numbers), `bad_kind`, `too_many_queued`, and `not_your_transfer` — the last for opening +or closing a `tr` another connection holds, which would otherwise be a denial of +service one random id away. + +#### Reads that carry no lease + +Browsing a group is **never** subject to a transfer slot: not the poster grid, not the +covers, not opening a photo or a PDF to look at it. A member must be able to browse a +group that is at capacity exactly as they browse an idle one. Thumbnails, posters and +cover art never reach the check at all — they resolve out of the node's own media +cache. + +But "not leased" cannot mean "unbounded", or a client that simply omits `tr` transfers +outside every cap and the caps are decoration. So a session may read **12 distinct +files at once** without a lease; the thirteenth is refused with `transfer_required` and +a sentence telling the person to download the file rather than preview it. An entry +already being read is always admitted, whatever the count — refusing a chunk halfway +through a photo because the limit moved is worse than never having admitted it. An +entry is released when its last chunk goes out, or after 60 s of silence, because a +viewer closed mid-file simply stops asking and says nothing. + +**The number is derived from what the client legitimately does**, and it has to be: +the music player warms a read-ahead window of 5 tracks on Wi-Fi, so playing an album has +six files in flight before anyone has done anything unusual. 12 is those six at their +widest, two for a photo viewer and its own prefetch in the same session, and the rest as +headroom for the next feature that reads ahead. A test derives the floor from the +player's own constant, so raising the client's prefetch without raising this fails in CI +rather than in front of a person. Generosity is cheap here and refusal is not: the cost +of being too high is a client that could have been queued and was not, and the cost of +being too low is a member told to download a track they are trying to play. + +Deliberately a count of files and not a byte budget: a RAW photo out of a camera is +60–80 MB and is browsing, a 40 MB archive is a download, and no size threshold +separates them. What separates them is which function asked. + +#### What leases are not + +**A fairness control among cooperating clients**, in the company of +`max_concurrent_streams` — not a defence against a member determined to saturate a +node's disk. A client that lies, labelling a bulk download as a view, gets 12 files at a +time instead of its member cap. That is the residual, it is bounded, it is audited, and +the answer to the member behind it is `member revoke`, not a protocol rule. Stating it +is the point: a control described as a security boundary will eventually be relied on +as one. + +### 11.3 File download + +``` + C N + |-- file_req {v, file_id, chunk_index, [tr]} ------->| + | | `tr` present: mark that lease + | | alive (§11.2) + | | index lookup; if the id is + | | not a file, try the media + | | cache (thumbnail/poster/ + | | cover/transcode), sliced the + | | same way — never leased + | | `tr` absent: admit against the + | | leaseless ceiling, else + | | `transfer_required` + | | backpressure: wait while + | | bufferedAmount > 2 MiB + |<- file_chunk {v, file_id, chunk_index, | + | plaintext_size, nonce, ct} --------------------| +``` + +The client pipelines 8 chunk requests at a time and reassembles in order; a chunk that +fails is retried 6 times, 1.5 s apart, because a DataChannel that hiccups mid-film +should cost a pause and not the whole transfer. + +Chunk encryption: + +``` +chunk_key = HKDF-SHA256(GEK, salt = none, len 32, + info = "file:" || file_hash || ":chunk:" || uint32be(chunk_index) || ":aes") +nonce = 12 random bytes +ct = AES-256-GCM(chunk_key).encrypt(nonce, plaintext) no AAD +``` + +* The `:aes` suffix keeps AES keys distinct from the ChaCha20 variant + (`chunk_key`/`encrypt_chunk`, `info` without the suffix) derived from the same GEK. +* `nonce` and `ct` are msgpack **binary**, not base64. Every message that carries + content carries it this way, and none carries it outside an AEAD. +* The key is a pure function of (GEK, file hash, index), so chunks are cacheable, + resumable and requestable out of order. This is what makes a download resumable at + all: a client that stopped at chunk 900 asks for 900 next time, and no state on the + node was keeping its place. `file_id` and `chunk_index` ride on the response because a + client running several downloads at once cannot otherwise tell whose reply arrived. +* Requests are handled off the message loop: the reply may wait for room on the + channel, and blocking the loop for that would stall the very uploads whose acks free + the buffer being waited on. +* **One encoder, every transport** (`protocol.file_chunk_wire` / `file_chunk_plaintext`). + A message type with one encoder per transport is a message type free to drift, and its + name then says nothing about which shape will arrive. +* **There is no per-chunk signature, and none is needed**: the AEAD tag authenticates + the ciphertext under a key only group members hold, and the node authenticates itself + once, in the handshake, rather than once per megabyte. +* **A chunk that is not this shape aborts the download**, with an error. There is no + fallback that decodes it some other way: a client that guesses at a chunk it does not + recognise writes its guess into the file the person is saving. + +### 11.4 Upload + +``` + C N + |-- transfer_open {tr, kind: "upload", ...} -------->| a slot, like a download + |<- transfer_state {tr, state: "granted"} -----------| (§11.2) + | | + |-- file_upload {v, upload_id, chunk_index: -1, | the probe: "where am I?" + | total_chunks, tr, nonce, ct} ----------------->| writes nothing, reserves + | ct = seal(upload, {filename, dir, root, | nothing + | data: b""}) | + |<- file_upload_ack {v, upload_id, chunk_index: -1, | + | nonce, ct} ------------------------------------| + | ct = seal(upload, {filename, stored_as, dir, | + | resume_from}) | + | | + |-- file_upload {v, upload_id, chunk_index, | 48 KiB chunks, window 32 + | total_chunks, tr, nonce, ct} ----------------->| + | ct = seal(upload, {filename, data, | open under the group key, + | dir, root}) | or refuse (upload_not_sealed) + | | filename allowlist + | | root writable and available + | | destination resolves in-group + | | chunk_index == next expected + | | running total <= 4 GiB + | | append to <stored_name>.part + |<- file_upload_ack {v, upload_id, chunk_index, | + | nonce, ct} ------------------------------------| + | ct = seal(upload, {filename, stored_as, dir}) | + | ... repeat ... | + | | last chunk: rename .part -> + | | final, tag the index entry + | | with uploader_id/uploader_pk +``` + +* **Sealed, both halves.** The filename, the destination and the bytes travel inside + the seal; only `upload_id`, `chunk_index`, `total_chunks` and `tr` stay in clear, + because the node routes, orders and accounts on them before it can decrypt anything. + This direction seals *towards* the node — it holds the group key for its own group — + which is the mirror image of the index, and it means a refusal cannot quote back what + it just refused. +* **`filename`, `dir` and `root` are repeated on every chunk**, not sent once in a + header. A hundred bytes against a 48 KiB chunk, against the alternative: a header that + arrives once is state the node has to carry, and upload state that can disagree with + the chunk in hand is the thing the chunk-ordering rule and the free-name rule exist to + prevent. +* **`upload_id` replaces `filename` as the correlation key.** It has to: matching an ack + to a request by name would hand back exactly what the seal is for. It is client-drawn, + opaque to the node, unique within one connection, and never an authorization input. +* **`group_id` is deliberately not on the message.** The session already decided which + group it is on, and the node uses that as the AAD. A client naming its own group here + would be choosing which key its bytes are checked against. +* **A chunk that does not open is refused** with `upload_not_sealed`, and nothing is + written. One answer covers "not sealed at all" and "sealed wrong": distinguishing them + tells a peer which of the two it got right. There is no plaintext fallback — a path + that still accepts plaintext is not a sealed path. + +**Resuming, and the probe chunk.** The node identifies an upload by +`(member, directory, filename)`, so a client resuming one has to name the file. +`transfer_open` is the obvious place to ask and it travels in clear, which would undo +precisely what sealing this path bought. So the question is asked **inside the seal +that already exists**: an ordinary `file_upload` with no bytes and `chunk_index = -1` +(`UPLOAD_PROBE_INDEX`). Every check below has already run by then, so it cannot be used +to ask questions about a directory the caller may not write to; the node writes nothing, +reserves no name, and answers `resume_from` — how many chunks of this file it already +holds — inside the seal, because that is a fact about the operator's disk. `resume_from` +is absent from an ordinary ack, so the two are told apart without looking at +`chunk_index`. `stored_as` on a probe answer is only what is *really* on disk, and empty +when there is nothing: reporting the free name the node would pick would promise a +destination that the real chunk 0 may not choose. A node that does not understand the +index refuses it, which a client reads as "start from the beginning"; the client also +bounds its wait at 5 s, so a node that answers neither the probe nor its refusal costs +one restart rather than a stuck upload. Starting over is always safe, which is what +makes both fallbacks available. + +**The position outlives the connection.** Upload state is held by the **group**, keyed +by `(user_id, rel_dir, filename)`, not by the session: state on the session dies with +it, and an upload interrupted at 99% would then have to start again from zero — on a +connection flaky enough to have interrupted it once. Keyed by member as well as by +name, because a shared directory means two people can be sending `IMG_1234.jpg` at the +same moment and neither may inherit the other's position. + +**Pausing an upload is the same shape as pausing a download** (§11.2): the lease goes +back, and the position does not have to be remembered accurately, because the node holds +it and the probe asks for it on the way back in. A pause is taken **between two chunks, +never inside one** — the node refuses a chunk that is not the one it expects, so a chunk +boundary is the only position worth having. The client also throttles on its own send +buffer (1 MiB), or the whole file lands in it in seconds and the progress bar becomes a +work of fiction. + +**An abandoned `.part` is reaped.** It is otherwise a gigabyte of somebody else's disk +that nothing will ever finish, delete or look at again — invisible in the index, because +a `.part` is not an index entry. One with no upload behind it is deleted after 24 h. +Generous on purpose: the cost of waiting is disk, and the cost of being wrong is +deleting an upload somebody is still making, which is unrecoverable and looks to them +like a transfer that failed for no reason. A day covers a laptop closed +overnight, a phone in a tunnel, and a client that resumes on its next launch. + +* **Confinement.** `filename` must match the name allowlist (`SAFE_UPLOAD_NAME`); the + resolved path must sit under a shared root. This is not hypothetical tidiness — a + filename rendered into the interface is stored XSS if it is allowed to contain markup, + and it reaches every member of the group. +* **No overwrite, ever.** A colliding name is given a free one and the uploader is told + what it became in `stored_as`; the client must use that value when referencing the + file (a chat attachment, for example). Without the rule any member could replace any + shared file by uploading one with the same name — silently, and with the index still + pointing at what that name meant before. +* **Ordering.** Out-of-order or replayed chunks are refused — otherwise a chunk with + index > 0 appends blindly to whatever `.part` is on disk. +* **Types are checked after the seal opens.** What comes out of an AEAD is + *authenticated*, not *validated*: it is msgpack a member wrote, and a `filename` that + is a number raises where a refusal was meant. `data` must be binary: no sealed message + can carry a string there, so a string is a peer doing something else entirely. +* **Destination.** The client names the folder it is browsing, never a path: the node + resolves it against the group's own roots, which refuses `..`, absolute segments and + anything escaping its root, symlinks included. With several roots, the node picking one + would send a member's file to a disk the operator did not intend, and that is + discovered weeks later. An unknown root name is refused rather than falling back to a + writable one, for the same reason. An unavailable or read-only root is a refusal, not + a fallback. +* **No quarantine subdirectory.** The file lands in the folder the sender is looking at, + not in an `uploads/` folder of the node's invention: a shared directory nobody can + organise is not a shared directory, and a folder appearing beside the operator's + library because somebody sent a file is the node deciding how their disk is arranged. + What confines an upload is the allowlist, the size cap, the chunk ordering and the + no-overwrite rule — never a subdirectory. +* **Policy.** Writability is a property of the root, and it binds the operator too — + "read-only for everyone" is what makes a published library one. Enforced here rather + than by hiding a button: the button is a courtesy to people who are not trying. +* The uploader key recorded is the one **the node pinned** for the device on this + connection (§9.4), not one the token carried — which is what makes `file_delete` + authorizable by the uploader without letting the hub delete anyone's files. + +### 11.5 Directories and deletion + +``` + C -> N dir_create {name, dir} any member; allowlisted, confined, audited + N -> C dir_create_ack {dir} + + C -> N dir_delete {dir} operator, signed (section 10) + C -> N file_delete {file_id} operator or uploader, signed +``` + +Creating a directory is not privileged — a member who can add a file may organise where +it goes — but it writes to the operator's disk, so it is audited like one and it obeys +the same per-root policy as an upload: the parent's root must be writable +(`root_read_only`) and available (`root_unavailable`). Read-only means read-only, and a +member who cannot add a file to a published library must not be able to leave empty +folders in it either. + +A directory cannot be created at the virtual root: that would be adding a root, which is +operator configuration rather than a file operation. Deleting a root by name through +`dir_delete` is likewise refused. + +`dir_delete` removes an **empty** directory and nothing else: it is never recursive, so +whatever the caller intended it cannot destroy content. The operator deletes the files +first, and sees what they are losing. The emptiness check runs before the challenge is +issued, so a non-empty directory never produces a signable transcript. + +### 11.6 Video streaming (MSE) + +Segments are produced by ffmpeg as fragmented MP4 and pushed under client-granted +credit. Without the credit scheme the node hands ffmpeg's whole output to the channel +as fast as it is produced, and the browser holds a multi-gigabyte film in a JavaScript +array while MediaSource consumes it a segment at a time. + +``` + C N + |-- stream_req {v, file_id, start, credits} -------->| + | | retire this session's previous + | | stream (a second request + | | means the first is over) + | | acquire a transcode slot (8) + | | ffprobe: codec, duration, audio + | | spawn ffmpeg + | | -ss before -i (index seek) + | | video: copy, or libx264 when + | | the browser cannot decode + | | audio: always AAC, 2 ch + | | frag_keyframe+empty_moov + |<- stream_init {v, file_id, codec, duration, start}-| + | | + | check MediaSource.isTypeSupported(codec) | + | | + |<- stream_data {v, file_id, segment_index, | 256 KiB, encrypted with the + | nonce, ct, plaintext_size} --------------------| same per-chunk derivation as + | ... x credits ... | a file chunk, index = segment + | | + |-- stream_more {v, n} ----------------------------->| n > 0 grants; n == 0 is a + | ... continues ... | keepalive, not a no-op + | | + |-- stream_stop {v} -------------------------------->| viewer closed + |<- stream_end {v, file_id} ------------------------ | natural end of file +``` + +| Rule | Value / behaviour | +|---|---| +| Credit granted per `stream_more` | clamped to `[0, 256]` | +| Client default window | 24 segments (6 MiB) | +| A client that sends no `credits` | unpaced — the node streams as fast as it can, and the client is responsible for what it buffers | +| Silence timeout | 120 s since the peer last said anything, polled every 3 s | +| `n == 0` | keepalive: a viewer buffered 90 s ahead grants nothing and must still be able to say it is there | +| Concurrent transcodes | 8 node-wide, semaphore on the transport context | +| Seeking | a new `stream_req` with `start`; the previous stream is retired first, ffmpeg respawned with `-ss` | +| `start` in `stream_init` | the value actually used — seeking lands on the keyframe at or before the request, and the client adds it back as `SourceBuffer.timestampOffset` | + +An ffmpeg failure before any output produces `error: Could not stream this file`; +stderr stays server-side, where it belongs — it names paths on the operator's disk and +the operator's ffmpeg build, to somebody who asked to watch a film. + +Segments are encrypted with the same per-chunk derivation as a file chunk, the segment +index standing in for the chunk index. There is no unencrypted streaming path: a segment +of a film is content, and content does not leave a node outside an AEAD. + +### 11.7 Chat + +Chat is encrypted on the wire and at rest, under a key hierarchy of its own — not the +sealed envelope of §11.1a, because the node has to relay and archive a message it cannot +read, and receivers include devices that were not connected when it was sent. + +``` + C N other peers + |-- device_hello {...} ----------------------------->| §9.4 — required before a send + | | + |-- chat_keys_req {v} ------------------------------>| + |<- chat_keys_resp {v, group_id, nonce, ct} ---------| sealed, purpose "chat_keys" + | payload: {epochs: [{epoch, key}], current} | EVERY live epoch, not just now + | | + |-- chat_msg {v, format: 1, epoch, device, | + | nonce, ct, sig, sender_name, | + | [thread_id], [iteration]} -------------------->| + | | envelope checks (below) + | | persist to THIS group's store + | | broadcast ------------------->| + | | {chat_msg, sender_id, + | | sender_name, format, epoch, + | | device, nonce, ct, sig, + | | thread_id, timestamp} + | | notify the hub: group id and + | | sender id only + |<- ack {v} -----------------------------------------| + | + |-- chat_hist {v, [before], [limit <= 200]} -------->| + |<- chat_hist_resp {v, has_more, messages: [{id, | + | sender_id, sender_name, timestamp, thread_id, | + | format, epoch, device, nonce, ct, sig}]} ------| + | + |<= chat_epoch_ack {v, epoch} =======================| pushed: a new epoch is open +``` + +**The key hierarchy.** + +``` +epoch_key 32 bytes, generated by the NODE, one per group per epoch +device_key = HKDF-SHA256(epoch_key, salt = <none>, len 32, + info = "meshbay:chat:dev:v1|<group_id>|<device_b64>") +nonce = 12 random bytes, per message +ct = AES-256-GCM(device_key).encrypt(nonce, msgpack(payload), aad) +aad = "chat_msg|<group_id>|<epoch>" +sig = Ed25519(sk_device, "meshbay:chat:v1" + || LP(group_id) || LP(epoch) + || LP(device) || LP(nonce) || LP(ct)) +``` + +**Why a key per device and not a ratchet.** A ratchet cannot deliver forward secrecy in +this setting, and it is better to say so than to appear to have it. The node serves +history to devices that were not present when a message was written, so it must retain +and hand out each chain's *earliest* key — and a chain key at iteration *i* yields every +message key from *i* onward by pure HKDF. Forward secrecy is then zero, and the ratchet +is computing HKDF over a value every member already holds. A Signal-style sender-key +implementation and a Double Ratchet were both written for this and never called by +production; both have been **deleted**, because code nothing calls reads as an +alternative somebody may reach for and its passing tests read as evidence of a +protection that is not in the product. + +Deriving per device by name buys what the ratchet was there for and one thing more: +**there is no mutable sending state at all**, so nothing can be advanced twice. Two +devices advancing one chain produce key and nonce reuse, which is the failure this +design cannot have. Be precise about what that does *not* say: two clients of one +account normally hold the **same** identity key — a second browser recovers it from the +keypair bundle rather than minting a new one — so they share a device key and therefore +this subkey. That is safe here only because the nonce is 96 random bits and not a +counter: two independent senders under one key collide on the birthday bound, which at +chat volume is unreachable, whereas two independent senders advancing one counter +collide immediately. The design degrades correctly into the deployment that exists; a +chain-based one would not have. + +**Epochs.** A new epoch is opened when the set of devices that may read *future* +messages shrinks — member revoke, member unpin, device revoke, `gek_rotate` — and by +hand with the signed `chat_epoch` op (§10.4). Old epochs are kept and still delivered to +current members, which is what keeps history readable to the people who could already +read it. The epoch key is wrapped under the group key **at delivery**, never stored +under it, so rotating the group key is a re-wrap and costs nothing. + +**Signing is separate from encryption, and it is what establishes who said something.** +The signature is over the *ciphertext*, so it can be checked before decryption and by +anyone holding the roster — including on a stored row, without the epoch key. It names +the device key **the node pinned**, never a key the sender presents alongside the +message: a signature verified against a key from the same message proves only that its +sender owns some key, which any member can arrange. + +One thing is deliberately **not** in the signing transcript: this connection's nonce. +Every other transcript in this protocol binds one; this one cannot, because a receiver +reading history has no access to the connection a message arrived on. Replay is +therefore refused by storage instead, on the unique `(device, nonce)` pair. A replayed +message is a validly signed copy of a real one, so nothing about the signature refuses +it; it is dropped and logged rather than raised at the sender, because the message it +duplicates is already stored and there is nothing for anyone to retry. + +**What the node checks before it stores or relays anything:** + +| Rule | Why | +|---|---| +| `format` is the sealed one | Plaintext is refused, always — not "accepted and marked", and not "unless a switch says otherwise". A member who can post in clear into a group whose members believe their chat is encrypted is a downgrade, and every peer that reaches this point is able to seal | +| `device`, `nonce`, `sig` present, right lengths, `ct` non-empty | a malformed envelope is refused before storage, not stored and puzzled over later | +| the connection has identified its device (§9.4) | the `device` field is what receivers verify against | +| `device` == this connection's pinned device | **a device may only send as itself.** A member free to name another member's key could *be* that member to everyone, and the signature would verify | +| `sender_id` is taken from the authenticated session | never read from the message; it never was | + +`format` is a *storage* state as well as a wire one, so the store can distinguish rows +the wire would not accept. Nothing changes what is accepted from a peer: the sealed +form, or a refusal. + +**Who can verify what.** `group_roster_req` answers **any member** — not only the +operator — with each device key in the group, which already-pinned key countersigned +it, and the signature, nonce and timestamp needed to rebuild what was signed. That is +what lets a member check for themselves that a message came from a device belonging to +the account it claims, instead of taking the node's `sender_id` on trust. The reply is +sealed under the `roster` purpose (§11.1a): it is the group's membership, and a peer +that has not completed the handshake has no business reading it. The node hands over +evidence and decides nothing; a node that lies here is caught by a client that has seen +the account before. + +**Other rules.** + +* The store, the peer registry and the epoch keys are resolved **per group**. Reading + them off the shared transport context sent every group's messages to the first + group's database and served them back to anyone on the node. +* The broadcast excludes **this connection**, not this account. The sender's other + devices are ordinary recipients: they did not compose the message and have no local + echo of it, so skipping them by `user_id` left a person's second device silently + missing everything they said from the first. +* The hub is told a message exists — group id and sender id, nothing else. No display + name: the body is unreadable to the hub, and shipping the author's name beside it + would leave the hub a per-message record of who spoke where, which is the metadata + this is otherwise about not producing. The sender id stays because the hub needs it in + order not to notify the author of their own message. +* `before` pages backwards from the newest, which is the direction a chat is read; + `has_more` is asked about the oldest row returned, so an empty page correctly says no. +* A message that does not open is shown as unreadable, never as blank. Rendering it + empty would make a message nobody can read indistinguishable from a message nobody + wrote. +* The boundary, stated as everywhere else: the node operator and every current member + hold the group key and therefore the epoch keys. This is the same boundary as file + access, by design. What it protects against is someone who obtains the node's storage + without the keystore password. + +### 11.8 Link unfurl + +``` + C -> N link_preview_req {v, url} + N -> C link_preview_resp {v, url, ok, + [title, description, site_name, image_thumb_hash]} +``` + +The node fetches the URL because the browser cannot (CSP and CORS) and doing so would +leak every reader's IP to whatever was pasted. `ok: false` means "no preview" — +blocked, unreachable, or not HTML — and the client shows the bare link. Any image is +cached in the node's thumb store, so `image_thumb_hash` is fetched over the ordinary +`file_req` path. Rate limits: 15 per connection and 60 node-wide per 60 s; results +cached 1 h, 256 entries. + +### 11.9 Metadata applications + +All of these are read-only lookups against third-party services, cached node-side and +keyed by content hash. None is signed: they change nothing in the node's own state. +The corresponding *settings* are signed operations (§10.4). + +| Request | Response | Notes | +|---|---|---| +| `media_meta_req {file_id}` | `media_meta_resp {file_id, tmdb_id, title, original_title, overview, poster_thumb_hash, backdrop_thumb_hash, release_date, first_air_date, genres, vote_average, runtime, cast, director, confidence, [season, episode]}` | `confidence: 0` means no confident match — the client falls back to a thumbnail-only card, it is not an error | +| `season_meta_req {tmdb_id, season}` | `season_meta_resp {...}` | a show's single overview does not describe every season alike | +| `tmdb_search_req {media_type, query}` | `tmdb_search_resp {results: [{id, title, year, poster}]}` | candidates for a human to pick from; never collapsed to one guess | +| `music_meta_req {file_id}` | `music_meta_resp {file_id, ...}` | MusicBrainz; cover art cached like a poster | +| `audio_transcode_req {file_id}` | `audio_transcode_resp {file_id, hash, size, mime}` | WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by `hash` over `file_req` | + +Every one of these is keyed by the entry's **`file_id`**, never by a path: a path names +the folder a file is in, so two files in one folder — any multi-episode season — would +resolve to whichever entry the index returned first. + +Posters, covers, thumbnails and transcode results all live in the media cache and are +served through `file_req` by their hash, sliced into chunks exactly like a real file. + +### 11.10 Liveness + +``` + C -> N ping {v, token} + N -> C pong {v, token} the caller's token echoed back +``` + +Echoed rather than bare, so a client can match the answer to the probe it sent and +measure a round trip instead of being reassured by a reply to an earlier one. This is +**not** discovery: opening a connection in order to ping costs a full ICE/DTLS +handshake, so presence in the group list comes from the hub's socket registry. + +--- + +## 12. Message reference + +Direction is `C→N` (client to node), `N→C` (node to client, solicited) or `N⇒C` +(node to client, unsolicited push/broadcast). "Stage" is the earliest session state in +which the node accepts the message: **pre** = pre-proof window, **auth** = after the +client's group-key proof, **signed** = after an `admin_response` verified against an +operator key. + +Any request may carry `req_id`; every reply the node sends while answering one carries +it back (§3.5). + +| Type | Dir | Stage | Purpose | +|---|---|---|---| +| `handshake` | C→N | — | open the session; version range, token, group, client nonce | +| `handshake_challenge` | N→C | — | node nonce, node's version range, announced `node_pk` | +| `handshake_response` | C→N | — | client's `HMAC(GEK, T("client"))` | +| `handshake_ack` | N→C | — | node proof, node signature, session parameters **sealed** (§11.1a) | +| `keypair_bundle_fetch` / `_resp` | C→N / N→C | pre | the caller's encrypted identity bundle | +| `keypair_bundle_store` | C→N | auth | back up (or re-wrap) that bundle | +| `keypair_bundle_delete` | C→N | auth | withdraw the backup | +| `gek_bundle_fetch` / `_resp` | C→N / N→C | pre | the caller's wrapped GEK | +| `join_request` / `join_result` | C→N / N→C | pre | pin or recognise an identity; wrap the GEK | +| `invite_create` / `invite_result` | C→N / N→C | signed | issue a one-time pairing code | +| `device_add_request` / `_ack` | C→N / N→C | auth | file a new device as pending | +| `device_lookup` / `device_lookup_result` | C→N / N→C | auth | candidates for the approver to hash-match | +| `device_add` / `device_add_ack` | C→N / N→C | auth | admit a device, countersigned | +| `device_list` / `device_list_result` | C→N / N→C | auth | this account's devices | +| `device_revoke` | C→N | auth | retire a device, countersigned (answers `device_add_ack`) | +| `index_sync` | C→N, N⇒C | auth | full index — requested, or pushed on first change | +| `index_delta` | N⇒C | auth | additions / deletions / updates against a base version | +| `index_progress` | N⇒C | auth | scan counters, no paths | +| `file_req` / `file_chunk` | C→N / N→C | auth | one encrypted chunk of a file, thumbnail or cache blob; optional `tr` names the lease it runs under | +| `file_upload` / `file_upload_ack` | C→N / N→C | auth | push a chunk; **both halves sealed** — name, destination and bytes inside, `upload_id`, `chunk_index` and `tr` outside. `chunk_index = -1` is the resume probe (§11.4) | +| `transfer_open` / `transfer_close` | C→N | auth | ask for a transfer slot / give it back | +| `transfer_state` | N→C, N⇒C | auth | granted, queued (with `ahead`) or closed (with `reason`) | +| `device_hello` / `_ack` | C→N / N→C | auth | which device of this account is on this connection (§9.4) | +| `chat_keys_req` / `_resp` | C→N / N→C | auth | every live chat epoch key, **sealed** | +| `group_roster_req` / `_resp` | C→N / N→C | auth | this group's members and device keys, with the evidence that admitted each, **sealed** | +| `dir_create` / `dir_create_ack` | C→N / N→C | auth | create a folder | +| `dir_delete` / `dir_delete_ack` | C→N / N→C | signed | remove an empty folder | +| `file_delete` / `file_delete_ack` | C→N / N→C | signed | delete a file (operator or uploader) | +| `stream_req` | C→N | auth | start or seek an MSE stream | +| `stream_init` / `stream_data` / `stream_end` | N→C | auth | codec header, encrypted fMP4 segments, end | +| `stream_more` / `stream_stop` | C→N | auth | grant credit / abandon the stream | +| `chat_msg` | C→N, N⇒C | auth | send and fan out a message | +| `chat_hist` / `chat_hist_resp` | C→N / N→C | auth | paged history | +| `chat_attach` | — | auth | attachment metadata (declared, unused on the wire) | +| `link_preview_req` / `_resp` | C→N / N→C | auth | OpenGraph unfurl | +| `media_meta_req` / `_resp` | C→N / N→C | auth | TMDB metadata for one file | +| `season_meta_req` / `_resp` | C→N / N→C | auth | per-season TMDB fields | +| `tmdb_search_req` / `_resp` | C→N / N→C | auth | candidate matches for an operator | +| `music_meta_req` / `_resp` | C→N / N→C | auth | MusicBrainz metadata for one file | +| `audio_transcode_req` / `_resp` | C→N / N→C | auth | browser-playable copy of a WMA/MPC file | +| `ping` / `pong` | C→N / N→C | auth | liveness on an open channel | +| `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | +| `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | +| `transfer_limits` / `_ack` | C→N / N⇒C | signed | per-member transfer caps for this group | +| `chat_epoch` / `_ack` | C→N / N⇒C | signed | open a new chat epoch by hand | +| `app_directories` / `_ack` | C→N / N⇒C | signed | one application's folders, keyed by app name | +| `chat_directory` / `_ack` | C→N / N⇒C | signed | where chat attachments are written | +| `chat_link_preview` / `_ack` | C→N / N⇒C | signed | whether the node unfurls posted links | +| `root_update` / `_ack` | C→N / N⇒C | signed | a root's `writable` / `removable` flags | +| `root_eject` / `_ack`, `root_plug` / `_ack` | C→N / N⇒C | signed | take a removable root offline, put it back | +| `gek_rotate` / `_ack` | C→N / N→C | signed | node generates a new group key | +| `apps_enabled` / `_ack` | C→N / N⇒C | signed | which group apps are shown | +| `set_scan_settings` / `_ack` | C→N / N⇒C | signed | reconcile interval and debounce | +| `tmdb_config` / `_ack` | C→N / N⇒C | signed | node-wide TMDB token and language | +| `tmdb_enabled` / `_ack` | C→N / N⇒C | signed | per-group TMDB on/off | +| `tmdb_override` / `_ack` | C→N / N⇒C | signed | correct a wrong automatic match | +| `tmdb_rematch` / `_ack` | C→N / N⇒C | signed | drop one file's cached match | +| `musicbrainz_enabled` / `_ack` | C→N / N⇒C | signed | per-group MusicBrainz on/off | +| `root_add` / `_ack`, `root_remove` / `_ack` | C→N / N→C | signed | add or remove a shared directory | +| `group_attach` / `_ack`, `group_detach` / `_ack` | C→N / N→C | signed | start or stop hosting a group | +| `node_status` / `_ack` | C→N / N→C | auth (operator) | all groups, roots, daemon state | +| `roster_read` / `_ack` | C→N / N→C | auth (operator) | pinned identities and members | +| `denylist_read` / `_ack` | C→N / N→C | auth (operator) | current refusals | +| `denylist_clear` / `_ack` | C→N / N→C | auth (operator) | remove entries | +| `node_settings_set` / `_ack` | C→N / N→C | auth (operator) | change daemon settings | +| `node_reload` / `_ack` | C→N / N→C | auth (operator) | re-read `node.toml` | +| `ephemeral_stream` | — | — | reserved, mobile live push | +| `error` | N→C | any | refusal, with `detail` and optionally `code`, `req_id`, and the `upload_id` / `tr` / `file_id` it is about | +| `ack` | N→C | auth | generic acknowledgement (chat, keypair bundle store) | + +**Three messages do not exist, and their absence is a rule rather than an omission:** + +| Not a message | Why there is none | +|---|---| +| any request that returns the group key in plaintext | members obtain it by unwrapping their own ECIES bundle (§7.2); a node that can be asked for the key in clear is a node one authorization mistake away from handing it over | +| any message by which a member stores key material on the node | the node wraps for a key the recipient has proved possession of (I2). A member-supplied bundle is a key of somebody else's choosing | +| any content message outside an AEAD | a segment of a film and a chunk of a file are the same thing to everyone but the codec | + +A node that receives an unknown type logs it and does nothing. That is what makes the +additive rule of §13 work. + +### 12.1 Transport coverage + +**WebRTC implements this document.** It is what the browser SPA and the desktop client +speak, and every message above is available on it. + +**QUIC is in development** (§5.2): a partial message set, no client, and not a shipped +feature. Nothing about it is a compatibility commitment yet. + +Two rules hold across transports, and both are about there being exactly one of each +message: + +* **One encoder per message type, shared by every transport.** `file_chunk` comes from + `meshbay_common.protocol`, `index_sync` and `index_delta` from + `meshbay_node/transport/wire.py`, and a parity test fails if a server grows a copy of + its own. Two encoders for one type is a type free to drift, with a name that no longer + says which shape will arrive — and, when one of them is a sealed envelope, a second + construction site that goes on sending cleartext. +* **`GroupIndex.serialize()` / `deserialize()` describes no MNP message.** It is a + signed, compressed, encrypted at-rest and interchange format, and reading it as a wire + contract is a mistake worth naming: the sealed envelope of §11.1a is what index + messages travel under, and it is deliberately not this, because zstd decompresses in + no browser. + +--- + +## 13. Versioning and compatibility + +MNP versions independently of the package version. Current: **`3.0`**; oldest peer +accepted: **`3.0`**. + +The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; +`MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about +whether an older peer can still do anything useful: + +* **Additive change → MINOR.** A change is additive only if all of these hold: no + existing field changes meaning, type or encoding; a peer that ignores the new field or + message still behaves correctly; and the new message is only sent to a peer that + advertised support, or is harmless to drop. The floor does not move. +* **Breaking change → MAJOR**, and then the question is *where to refuse*. A break + confined to one exchange can be refused per message, with a code, leaving the rest of + the protocol working — a peer that cannot upload can still browse, download, stream and + chat. A break that touches something every session depends on cannot be confined, and + the honest form is to refuse at the handshake: **a stated refusal is a bug report, a + feature that quietly does not work is a support case.** +* **A requirement is breaking even when its messages are additive** (I10). New message + types and a new optional field are additive on the wire; *requiring* them is not, and + an opt-in switch that enforces the requirement only for peers that speak the new + version leaves the permissive branch reachable on every node. That is the branch that + ends up being used. + +### 13.1 Negotiation + +Both peers declare two values in the first message each sends — `handshake` from the +client, `handshake_challenge` from the node: + +``` +v the version this build speaks MNP_VERSION +v_min the oldest peer it will talk to MNP_MIN_SUPPORTED +``` + +Each side then checks the other, before anything else is decided: + +| Condition | Refusal `code` | +|---|---| +| peer's `v` < our `v_min` | `version_too_old` | +| peer's `v_min` > our `v` | `version_too_new` | +| `v` unparseable | `version_unreadable` | + +The refusal is shaped like `not_a_member`: a peer-safe sentence for a human and a `code` +the client matches on, because matching on the text is a string comparison that breaks +the day someone improves the wording. A peer that declares no `v_min` is read as +accepting only what it speaks. + +Versions compare **numerically**, as `(major, minor)`: as strings `"0.9" > "0.15"`. + +**Refusing at the handshake is not enough on its own, for a client that ships its own +interface.** The browser SPA is served by the hub and is therefore never out of step +with it. An installed desktop client can be, and `version_too_old` is a refusal in a +protocol vocabulary with nothing a person can act on. So the client asks +`GET /v1/hub/version` for `client.minimum` **before it connects**, and says "this +version can no longer connect" instead. An unreachable hub is deliberately *not* treated +as too old: a captive portal or a closed laptop must not make starting the application +impossible. + +**The version a peer announces is only as good as the number it ships with.** Every +package in the tree carries one version, and a test fails if two disagree — a client +announcing a number from a different scheme sorts wherever that scheme puts it, and +walks through the gate meant to stop it. + +--- + +## 14. Security properties and stated limits + +### 14.1 What the protocol establishes + +| Property | Mechanism | +|---|---| +| A peer holds the group key | `HMAC(GEK, T("client"))` over a node-chosen nonce, bound to the channel | +| The node holds the group key and is the one previously seen | `HMAC(GEK, T("node"))` over a client-chosen nonce, plus `Ed25519(sk_node, T)` and a per-node pin | +| No MitM on the signaling path | both DTLS fingerprints (or the QUIC certificate hash) inside every transcript; empty binding is a refusal | +| The hub cannot read content | the group key never reaches it; chunk keys and chat epoch keys derive from or are wrapped under it | +| The hub cannot substitute a key at invite | the node wraps for a key its owner presented and signed, bound to an identity by a code the hub never sees | +| The hub cannot administer a node | privileged ops need an Ed25519 signature from a roster-pinned operator key | +| The hub cannot impersonate a device owner | device linking is countersigned by a key the node pinned; the hub stores no user keys | +| A revoked token cannot connect | denylist on user, `jti` and group, persisted across restarts | +| A signature cannot be repurposed | domain-separated, length-prefixed transcripts naming op, subject, node, group, nonce and time | +| One group cannot read another on the same node | per-group index, chat store, epoch keys and peer registry; the sealed envelope's AAD names the group | +| The ack's configuration is authenticated by a key the hub does not hold | sealed under `ack_key`; the signed handshake transcript names no ack field, so this is the only thing that authenticates them | +| A peer served before the handshake completes gets ciphertext, not filenames | `index_sync`/`index_delta` sealed under `index_key` — defence in depth against a serve-before-authentication mistake (§11.1a) | +| An upload's filename, destination and content never appear on the wire in clear | `file_upload`/`file_upload_ack` sealed under `upload_key`; `upload_id` replaces the filename as the correlation key | +| No message carries content outside an AEAD | file chunks and stream segments alike, under keys derived per file and per chunk | +| A body cannot be replayed as another message or into another group | AAD = `"<msg_type>\|<group_id>"`, and `"chat_msg\|<group_id>\|<epoch>"` for chat | +| A chat message names the device that wrote it, checkably by any member | signature over the ciphertext with a device key the node pinned; `group_roster_resp` hands over the evidence to verify it without trusting the node | +| A device may only send as itself | `device` on `chat_msg` is compared with the device this connection proved (§9.4), not believed | +| A chat message cannot be replayed into the archive | unique `(device, nonce)` at rest; the signing transcript cannot bind a connection nonce, because history readers have no connection | +| An old chat message cannot be re-presented under a later key | the epoch is inside the AAD | +| A member cannot transfer outside the caps the operator set | a lease per job, per-member cap before node-wide pool, and the leaseless path bounded to 12 files per session (§11.2) | +| A reconnect cannot charge a member twice for one transfer | `tr` is drawn by the client and `transfer_open` is idempotent on it | +| A refusal reaches the request it refuses | `req_id` stamped on every reply, including `error` (§3.5) | + +### 14.2 What it deliberately does not establish + +* **The browser SPA is served by the hub.** A hub that ships malicious client code can + read a pairing code out of the page, or the group key out of memory. This is accepted + permanently for the browser client and is what the native client removes. Keep the two + attacks apart: the pairing code defeats a hub that *lies in its directory* — silent, + undetectable, per-request — not one that *rewrites the client*, which is an artifact + that can be inspected and compared. +* **The pre-proof window is a disclosure surface.** A hub that forges a JWT can fetch a + member's *encrypted* keypair bundle. It is bounded, audited, and closes when clients + stop storing bundles on other people's nodes (§7). +* **Transfer leases are fairness, not security.** They bound cooperating clients. A + client that lies — labelling a bulk download as a view — gets 12 files at a time + instead of its member cap; that is the residual, it is bounded and audited, and the + answer to a member determined to saturate a node's disk is `member revoke` (§11.2). +* **The QUIC transport is in development** (§5.2) and establishes nothing yet. When it + does, its channel binding is a certificate hash rather than an RFC 5705 exporter, + which is weaker: on a resumed session the anchor travels with the session ticket. +* **Nothing is confidential from a member, or at rest on the operator's disk.** The + sealed envelope (§11.1a) is defence in depth against our own next + serve-before-authentication bug; it is not a claim against anyone who holds the group + key. Chat is the one thing encrypted at rest as well, which protects against someone + who obtains the node's storage without the keystore password — and against nobody + who holds the keys. +* **The control plane is still in clear.** Sealing covers content: the index, the ack, + file chunks, stream segments, uploads, the chat keys and the group roster. It does not + cover the admin and configuration acks (`app_directories_ack`, `root_*_ack` and the + rest), which carry the same folder names the sealed index carries; the media-metadata replies (`media_meta_resp`, `music_meta_resp`, + `link_preview_resp`), which carry titles, artists and synopses; `node_status_ack`, + which carries absolute paths on the operator's disk to an operator session; the + identity replies (`roster_read_ack`, `device_list_result`); or `invite_result`, which + carries a pairing code. All are inside DTLS/TLS and none reaches the hub, but none is + behind the group key. +* **Transfer messages are in clear on purpose**, and that is a deliberate line rather + than an omission: `tr` is opaque, `bytes` and `chunks` are numbers, and there is no + filename and no path anywhere in them (§11.1a). +* **Rotation is the only thing that removes access.** Revoking a member stops the node + serving the next key; the current key and anything already downloaded stay readable. + A chat epoch is opened at the same time, which stops them reading what is said next — + not what was said before, which they could already read. + +--- + +## Appendix A — Transcripts at a glance + +``` +handshake "meshbay:mnp:handshake:v1" LP(role) LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding) + -> HMAC-SHA256 under the GEK; role in {"client","node"} + +admin op "meshbay:admin:v1" LP(op) LP(node_pk) LP(group_id) LP(subject) LP(nonce) LP(ts) + -> Ed25519 by an operator key from the roster + +join "meshbay:join:v1" LP(node_pk) LP(group_id) LP(user_id) + LP(pk_ed25519) LP(pk_x25519) LP(nonce_s) LP(ts) + -> Ed25519 by the joining identity + +device req "meshbay:device_req:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519) + LP(code_hash) LP(nonce_s) LP(ts) + -> Ed25519 by the NEW device (possession only) + +device add "meshbay:device_add:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519) + LP(nonce_s) LP(ts) + -> Ed25519 by an ALREADY-PINNED device of the same account + +device hello "meshbay:device_hello:v1" LP(node_pk) LP(group_id) LP(user_id) + LP(pk_ed25519) LP(nonce_s) LP(ts) + -> Ed25519 by the device claiming this connection + +chat msg "meshbay:chat:v1" LP(group_id) LP(epoch) LP(device) LP(nonce) LP(ct) + -> Ed25519 by the SENDING device, over the CIPHERTEXT + (the one transcript that binds no connection nonce: a reader of + history has no connection. Replay is refused at rest instead, + on the unique (device, nonce) pair) + +sealed msg aad = "<msg_type>|<group_id>" UTF-8, NOT length-prefixed + key = HKDF-SHA256(GEK, salt=<none>, + info="meshbay:{index,ack,upload,chat_keys,roster}:v1", 32) + -> AES-256-GCM, 12-byte random nonce per message + +chat seal aad = "chat_msg|<group_id>|<epoch>" UTF-8, NOT length-prefixed + key = HKDF-SHA256(epoch_key, salt=<none>, + info="meshbay:chat:dev:v1|<group_id>|<device_b64>", 32) + -> AES-256-GCM, 12-byte random nonce per message + (the two AADs above are the one place bare concatenation is used: + every half is fixed-vocabulary or numeric, and the separator + cannot occur in a msg_type, a group id or an epoch) + +channel binding + WebRTC LP(offer_fp) LP(answer_fp) raw 32-byte SHA-256 fingerprints + QUIC LP(SHA-256(server_cert_der)) + +LP(x) = uint32be(len(x)) || x every field, no exceptions +``` + +## Appendix B — Constants + +| Constant | Value | Source | +|---|---|---| +| `MNP_VERSION` | `3.0` | `meshbay_common/__init__.py` | +| `MNP_MIN_SUPPORTED` | `3.0` | `handshake.py` | +| `NONCE_LEN` | 32 bytes (both handshake nonces) | `handshake.py` | +| `ADMIN_CHALLENGE_TTL` | 120 s | `adminop.py` | +| `JOIN_TTL`, `DEVICE_TTL` | 120 s | `join.py`, `device.py` | +| Pairing / device code | 40 bits, Crockford base32, single use | `roster.py`, `device.py` | +| Code lifetimes (default, settable) | invitation 7 d, operator pairing 24 h, device request 1 h | `roster.py` | +| `MAX_PRE_PROOF_FETCHES` | 4 per connection | `webrtc_server.py` | +| `MAX_JOIN_ATTEMPTS` | 5 per connection | ” | +| `MAX_JOIN_FAILURES_WINDOW` / `JOIN_FAILURE_WINDOW` | 20 / 600 s, node-wide | ” | +| Device attempts | 5 per connection | ” | +| `MAX_DEVICES_PER_USER` | 5 | `roster.py` | +| `PRE_HANDSHAKE_MAX_MSG` / `MAX_MSG` | 64 KiB / 64 MiB | `webrtc_server.py` | +| `CHUNK_SIZE` | 1 MiB | ” | +| `DOWNLOAD_BUFFER_HIGH` | 2 MiB | ” | +| `MAX_UPLOAD_BYTES` | 4 GiB | ” | +| Download pipeline / chunk retry (client) | 8 in flight; 6 attempts, 1.5 s apart | `file-utils.js` | +| Upload chunk / window / send-buffer high water (client) | 48 KiB / 32 / 1 MiB | `transport.js` | +| `UPLOAD_ID_LEN` | 16 bytes, hex on the wire | `protocol.py` | +| `UPLOAD_PROBE_INDEX` / probe timeout | `-1` / 5 s | `protocol.py`, `transport.js` | +| `PART_SUFFIX` / `ORPHAN_AFTER_SECS` | `.part` / 24 h | `uploads.py` | +| `DEFAULT_MAX_CONCURRENT` (node-wide, per kind) | 8 | `transfers.py` | +| `DEFAULT_MAX_PER_MEMBER` (per group, per kind) | 2, settable 1–32 | `transfers.py`, `webrtc_server.py` | +| `GRANT_DEADLINE_SECS` / `IDLE_TIMEOUT_SECS` | 30 s / 120 s | `transfers.py` | +| `MAX_QUEUED_PER_MEMBER` / `MAX_MISSED_GRANTS` | 32 / 3 | ” | +| `MAX_LEASELESS_IN_FLIGHT` / `LEASELESS_IDLE_SECS` | 12 files / 60 s | ” | +| `TRANSFER_SWEEP_SECS` | 15 s | `webrtc_server.py` | +| Lease watchdog (client) | 60 s, then re-ask | `transport.js` | +| `STREAM_SEGMENT_SIZE` | 256 KiB | `webrtc_server.py` | +| `STREAM_MAX_CREDIT` | 256 | ” | +| `STREAM_CREDIT_TIMEOUT` / `_POLL` | 120 s / 3 s | ” | +| Client stream credits | 24 | `transport.js` | +| `MAX_CONCURRENT_TRANSCODES` | 8 | `webrtc_server.py` | +| Link preview rate | 15/conn, 60/node per 60 s; cache 1 h × 256 | ” | +| ICE gathering deadline | 4 s | `transport.js` | +| Signaling: max SDP, pending per user, rate | 16 KiB, 3, 30/min, 15 s answer timeout | `api/signaling.py` | +| GEK | 256-bit, node CSPRNG | `crypto.py` | +| Chat epoch key | 256-bit, node CSPRNG, one per group per epoch | `chatbox.py` | +| Chunk cipher | AES-256-GCM, 96-bit nonce (ChaCha20-Poly1305 variant for native) | `webcrypto.py`, `crypto.py` | +| GEK wrap | X25519 + HKDF-SHA256 + AES-256-GCM, AAD = recipient public key | `crypto.py` | +| Sealed message envelope | HKDF-SHA256 subkey per purpose, AES-256-GCM, 96-bit random nonce | `groupbox.py`, `static/crypto.js` | +| Chat envelope | HKDF-SHA256 subkey per device per epoch, AES-256-GCM, 96-bit random nonce, Ed25519 over the ciphertext | `chatbox.py` | +| File id | BLAKE3, hex | `crypto.py` | +| Minimum installed client | `GET /v1/hub/version` → `client.minimum` | `meshbay-hub/api/hub.py` | + +## Appendix C — Where the authority lives + +This document is descriptive: where it and the code disagree, the code is right and this +is a bug. Nothing here depends on another document. + +``` +meshbay-common/ protocol.py message types, chunk and upload codecs + handshake.py transcript, proofs, version negotiation, token rules + groupbox.py the sealed envelope and its purposes + chatbox.py chat epoch keys, per-device subkeys, signing + adminop.py the admin transcript and the operation catalogue + join.py join transcript and pairing codes + device.py device request / add / hello transcripts + crypto.py GEK, chunk keys, ECIES wrap, BLAKE3 ids + webcrypto.py the AES variants the browser can also compute + +meshbay-node/ transport/webrtc_server.py the reference implementation of MNP + transport/quic_server.py the QUIC transport, in development (§5.2) + transport/wire.py the one index encoder + transfers.py leases, queues, caps, leaseless reads + uploads.py partial uploads and orphaned .part files + roster.py, ops.py, daemon.py roster, operations, group contexts + +meshbay-hub/ api/signaling.py SDP relay limits + static/transport.js the client half of every exchange above + static/crypto.js the browser mirror of groupbox/chatbox + api/hub.py the minimum client version gate +``` diff --git a/docs/playlists.md b/docs/playlists.md new file mode 100644 index 0000000..aef2366 --- /dev/null +++ b/docs/playlists.md @@ -0,0 +1,516 @@ +# MeshBay — Playlists (design) + +> Status: **proposal**, not implemented. This is M4 in `docs/musicbay.md` §9, +> which deferred it for the right reason: *"a genuinely new category of +> per-account node state, not covered by anything E9 already enumerates — +> needs its own design pass (ownership, sync across devices, whether it's +> node-local or something else)"*. This document is that pass. +> +> Read `docs/musicbay.md` first — Music is built, and this adds nothing to +> its playback path. Read `docs/refactoring-search.md` second: the +> cross-group consolidation this feature needs already exists there, and +> most of the work is recognising that. +> +> **Scope, settled before writing this:** a playlist belongs to **one +> account** and is never shared with other group members. That answer is what +> keeps §5 small; see §10 for what changes if it is ever reversed. +> +> Follows the project convention: every claim names the adversary it holds +> against (§9). + +--- + +## 0. What was asked, in one paragraph + +Somewhere to keep a user's playlists, so that they survive a cache clear and +turn up on that person's other devices — and so that one playlist may hold +albums from **several different groups on several different nodes**, the way +the Search page already searches a consolidated view. With the constraint, +stated up front, that nodes go offline for an evening or for a month and that +this must not corrupt anything. + +--- + +## 1. What this design does not reopen + +- **Views over the index, never a catalogue** (`desktop-client-v1.md` §6.10, + draft-v6 §2.7). A playlist is a list of *references*; it creates no second + identity for a file and no server-side database of content. +- **Nothing about content reaches the hub** (H7, draft-v6 §2.5). §3.1. +- **No new streaming path.** `musicbay.md` §2.2 stands untouched: a track is + fetched through `pipelinedDownload` and handed to `<audio>`. Playlists add + no node-side playback code whatsoever. +- **Node-side derived data lives in the node's own `data_dir`**, never beside + a shared file (`mediacenter.md` §2). The blob in §3.3 goes in `bundles.db`. +- **Identity keys are per node** (`per-node-identity-v1.md`). §3.4 exists + entirely because of this, and getting it wrong is the fastest way to ship a + playlist that cannot be read from the second device it was invented for. + +--- + +## 2. Three problems, and only one of them is hard + +| | Problem | Where it is solved | +|---|---|---| +| A | Somewhere durable to put a few KB per account | §3 — several acceptable answers, one clearly best | +| B | How to **name** a track that lives in one group on one node, inside a list that spans several | §4.2 — where the silent failures live | +| C | Reconciling N copies when nodes are ON and OFF | §5 — the stated fear, and it is well founded *only* for the wrong granularity | + +The request reads as a storage problem. Storage is the easy third of it. + +--- + +## 3. Where the state lives + +### 3.1 Not the hub, and the rule is narrower than "nothing on the hub" + +The hub *does* hold small per-account state: `user_preferences` +(`db/models.py:303`), behind an allowlist of keys that already includes +`music_keep_screen_on` and `default_tab:<group_id>`. So the operative rule is +not "the hub stores nothing about a user". It is: + +> **No content metadata on the hub.** + +A playlist is, literally, a list of blake3 hashes of private-group files — +plus, if it is to render while nodes are offline (§4.2), their titles and +artists. That is the exact object H7 removed from the hub, and it is what +`CLAUDE.md` already pins one size smaller for the video resume position: +*"nothing new learns what you watch"*. + +**An encrypted blob on the hub** is technically trivial and is still refused. +The precedent is exact and recent: keypair bundles *were* hub-stored, and T3 +reduction phases 1 and 2 deliberately moved them onto nodes. Putting a +different per-account blob back would undo a migration that has already been +paid for. The residual cost is real even sealed — the hub learns the +cardinality (how many playlists), the size (roughly how many tracks) and the +timing of every write (when this account listens and edits), and it retains +ciphertext indefinitely under a passphrase-derived key, which is C4's own +argument. + +If a future need makes hub involvement genuinely attractive, the only +acceptable shape is a **version vector** — `playlists_rev: 7` in +`user_preferences`, telling a client that some node is stale — never the +payload. Even that is probably not worth its own migration; §6 gets the same +result with no hub change at all. + +### 3.2 Not node-to-node + +Refused, and not on cost grounds. Nodes do not know each other, share no +authenticated channel, and `replication.py` is legacy public-content code +that has nothing to do with this. Beyond the protocol that would have to be +invented, it leaks the thing this architecture is most careful about: +node A would learn that this account also uses node B — that two unrelated +operators host the same person. Per-node identity exists precisely so that +this correlation is unavailable (`per-node-identity-v1.md`). + +### 3.3 The node, as an opaque per-account blob — a shape already built + +The node already stores an encrypted per-account object it cannot read: the +keypair bundle (`bundle_store.py`, table `keypair_bundles`, MNP ops +`KEYPAIR_BUNDLE_STORE` / `_FETCH` / `_DELETE`). Playlists are the same shape +with a different payload, so this introduces **no new trust boundary**: the +node is not being asked to hold anything of a kind it does not already hold +for this same account. + +A new table in the same `bundles.db`: + +```sql +CREATE TABLE IF NOT EXISTS user_blobs ( + user_id TEXT NOT NULL, + kind TEXT NOT NULL, -- "playlists" today; the column is the + -- reason a second one needs no migration + rev INTEGER NOT NULL, + blob_enc TEXT NOT NULL, + blob_enc_recovery TEXT, -- reserved, §10 O2 — see below + stored_at TEXT NOT NULL DEFAULT (datetime('now')), + PRIMARY KEY (user_id, kind) +); +``` + +`blob_enc_recovery` is declared now and left NULL. `bundle_store.py` already +carries `_migrate_keypair_recovery` — a `PRAGMA table_info` dance — for +exactly this column added to exactly this kind of table one release late. +`CREATE TABLE IF NOT EXISTS` never adds a column, which is the node-local +twin of the `create_all()` lesson in `CLAUDE.md`. Declaring the slot costs +nothing today and removes the migration entirely. + +### 3.4 The key — the one thing that must not be got wrong + +Identity keys are **per node**. A blob encrypted under one is unreadable from +every other node, which is the precise opposite of the requirement. + +The only secret this account holds *everywhere* is the **bundle key**: +Argon2id over the passphrase, rederived at every sign-in on every device +(`keyderive.js:145`), and already the key that opens the per-node identity +bundles. So: + +``` +playlist_key = HKDF-SHA256(bundle_key_v2, info = "meshbay:playlists:v1") +``` + +Three consequences, each of which is a line of code somewhere: + +- **`deriveEncryptionKey` must return an HKDF handle as well as the AES-GCM + one.** Today it imports the 32 Argon2 bytes non-extractably as `AES-GCM` + with `['encrypt','decrypt']`, from which nothing can be derived. Import the + *same* `out.hash` a second time as `HKDF` with `['deriveKey']`. One Argon2 + run, two handles, no extra cost on the 650 ms sign-in path. Do **not** add a + second Argon2 call, and do **not** encrypt playlists directly under the + bundle key with a different AAD: `groupbox.py` already sets the convention + ("*Never reuse `chunk_key_aes` with a pseudo-file for this*") and + purpose-separated subkeys are what it looks like here. +- **v2 only.** `bundleKey` is `{v2, v1}` so that a bundle written before the + KDF changed can still be opened. Playlists are new; there is no legacy blob + and therefore no v1 branch. One less thing that can silently take the wrong + fork. +- **The nonce is 96 random bits, never a counter.** Two devices of one account + derive the *same* playlist key — that is the whole point — so a counter + would repeat. This is the same reasoning already recorded for chat subkeys + in `CLAUDE.md`'s module table, and it is safe for the same reason. + +A passphrase change re-derives the key and the blob must be re-encrypted on +the next write, exactly as the keypair bundle already is ("*re-encrypted on +the next backup*"). + +**AAD** = `"user_blob|playlists|<user_id>"`, mirroring +`groupbox.associated_data`. It binds the ciphertext to its owner and its +kind. It does not, and cannot, prevent rollback — §5.4. + +--- + +## 4. The data model + +### 4.1 The blob + +Plaintext-before-sealing, msgpack (same encoding as everything else on MNP): + +``` +{ + "v": 1, + "playlists": { + "<playlist_id>": { + "name": "Evening", + "rev": 7, + "device": "<device_pk_ed25519, first 16 hex>", + "updated_at": 1757000000, -- display only, never a merge input + "deleted": false, + "tracks": [ <entry>, ... ] + }, + ... + } +} +``` + +`playlist_id` is a client-generated UUID, with one reserved value: +`"favorites"`. **Favourites is a playlist**, not a second mechanism — deciding +that now is what stops a parallel store being built next to this one in three +months. + +### 4.2 A track reference, and why each field is there + +``` +{ "group_id": ..., "file_id": <blake3 hex>, "hash_version": 1|2, + "path": "Some Artist/An Album/03 - A Track.flac", + "title": ..., "artist": ..., "album": ..., "duration": 214 } +``` + +A bare hash is not enough, and each field prevents a specific failure: + +- **`group_id`** — the player resolves its connection per track from + `entry.groupId` (`music-player.js:288`). Without it there is nothing to + dial, and a `file_id` alone has no meaning outside a group. +- **`title` / `artist` / `album` / `duration`, denormalised** — this is not + redundancy, it is the core of the design. With them a playlist renders + **completely** with every node offline, unplayable entries greyed out, in + the same spirit as the Search page reporting its `unreachable` list rather + than failing. Without them, an offline playlist is a column of hex strings — + and *that* is the incoherence the request is worried about. It is a + display problem, and it is solved by copying four small strings. +- **`hash_version`** — the index already has two hashing schemes + (`protocol.py:292`: 1 = whole file, 2 = 45 MB sample). A re-hash would + orphan every entry in every playlist, silently and all at once. +- **`path`** — content addressing survives a move; a path survives a + re-encode. Keeping both means either can repair the other: on a sight of + the live index, an entry whose `file_id` is absent but whose `path` matches + has its id rewritten in place (and vice versa), once, on the client. + +Rehydration is by design cheap: a playlist entry is a subset of the +`IndexEntry`-plus-`groupId` shape the player already consumes, so +`onPlayQueue(tracks, startIndex)` takes it unchanged. + +**Availability bonus, close to free.** `source-merge.js` exists because the +same content appears in more than one group. At play time, if the entry's own +`group_id` has no reachable node but the same `file_id` appears in another +cached index whose node is up, play it from there. That is a global playlist +that heals itself when one operator's machine is off, reusing the +fold-on-content-hash logic already written for Search. Default on; see §9 for +the one thing it changes. + +### 4.3 Size + +A thousand tracks at ~200 bytes each is ~200 KB before compression. That is +small, and it is also an unbounded write primitive pointed at someone else's +disk, so: + +- The node **caps and refuses**, never truncates. 256 KB of `blob_enc` is + generous for the shape above. (Noted in passing: `_do_keypair_bundle_store` + (`webrtc_server.py:1056`) has no cap at all today. Out of scope here, worth + its own line somewhere.) +- The client **pads the plaintext up to the next 4 KB** before sealing. The + ciphertext length otherwise tells the operator roughly how many tracks this + account has collected. Cheap, and it is the only metadata this design leaks + to a node that the node cannot already see. + +--- + +## 5. Merge — the part that has to be right + +The stated fear is correct **for a single blob under last-writer-wins**: node +A is off while an edit is made, node B is off while the next one is, and one +edit disappears with nothing to show for it. Four rules remove it. + +### 5.1 The unit is a playlist, not the collection + +The blob is a map keyed by `playlist_id`, and merging is per key. Two +playlists edited on two devices never collide, which is the overwhelmingly +common case for one person with two or three devices. + +### 5.2 `rev`, never the wall clock + +Each playlist carries a monotonic `rev` and the `device` that last wrote it. +Merge takes the higher `rev`; a tie is broken by the lexicographically +smaller `device`, so every device reaches the same answer without talking to +any other. `updated_at` is carried for display and is **never** read by the +merge — clocks across devices are not trustworthy, and a clock-based merge +fails roughly one time in twenty, which is the frequency at which this +codebase's history says a defect ships. + +### 5.3 A deletion is a tombstone, never an absence + +`deleted: true`, kept. Absence must mean "this copy is older than the one +that created it". Otherwise a node rehomed after three weeks **resurrects +every deleted playlist** — this is the single most likely defect in the whole +design, it looks like a sync working correctly right up until it doesn't, and +it deserves its own named test. Tombstones are collected only when every +known node reports a `rev` at or above the deleting one, which for a +single-node account is immediate and for a multi-node one is eventual; a +tombstone is ~40 bytes, so there is no hurry. + +### 5.4 Rollback, and why local-first answers it + +AEAD authenticates a blob; it does not stop a node handing back an older one +it still has (or a fresh one it never received). The defence is that **the +client is the authority**: the merged state lives in the client's own +IndexedDB, and merge takes the maximum `rev` across *local plus every node +answering*. A stale or lying node can only lose the tie. It can never lower +the merged state, because the local copy is one of the inputs. + +This is what makes an offline node a non-event rather than a hazard. Nodes +are backups and a transport. They are not the source of truth, and no node +being reachable at all still leaves every playlist correct and, thanks to +§4.2, fully legible. + +### 5.5 What is genuinely lost, stated plainly + +Device 1 makes an edit, reaches no node, and is then lost or cleared: that +edit is gone. This is the exposure of any offline-first application, it is +not fixable without a durable always-reachable writer (which is §3.1, and +refused), and it is still enormously better than today, where the same edit +is lost on a cache clear regardless of what was online. + +--- + +## 6. Sync — and the point is that it adds no dialing + +A sweep of every group's node costs 10 s per unreachable one; the Search page +does it deliberately, batched three at a time, because the user asked it to. +Playlists must not do that at sign-in. + +**Sync rides on connections that were happening anyway.** Whenever a +transport to any node is open for any other reason — opening a group, the +Search page's sweep, the music pool dialing to play a track — the client +piggybacks a `user_blob_fetch`, merges, and sends `user_blob_store` back if +that node's copy is behind. `ConnectionPool` (`search-page.js:36`) already +holds up to `MAX_POOL_SIZE` (12) live connections and hands them out by +group, so this is a hook, not a new mechanism. + +Two additions on top: + +- **An explicit "Sync now"** on the Playlists page, which does the Search-style + sweep and reports which nodes it could not reach — the same honest + reporting `fetchAllIndexes` already does. +- **On sign-in, nothing.** The local copy is authoritative and complete + (§5.4); the first group opened will reconcile. + +A device that only ever opens one group therefore only ever converges with +one node. That is correct and not a defect: convergence is eventual, and the +copy the user is looking at is right the whole time. + +--- + +## 7. Protocol and node-side implementation + +### 7.1 MNP — additive, MINOR bump (2.1) + +``` +user_blob_store { kind, rev, blob_enc } client → node +user_blob_fetch { kind } client → node +user_blob_resp { kind, rev, blob_enc|null } node → client +``` + +Modelled on `KEYPAIR_BUNDLE_*` in every respect, including that the node +stores and returns an opaque string. `kind` is validated against a small +allowlist (`{"playlists"}`) so the table does not become an arbitrary +key/value store for whatever a client feels like writing. + +Every reply carries `req_id` through the ordinary `_send` path. This is not +optional and does not need re-arguing: `CLAUDE.md` records at length what +arrival-order matching costs, and the victim is never the request that was +answered wrongly. + +### 7.2 Node side + +| Piece | Where | What | +|---|---|---| +| Storage | `meshbay_node/bundle_store.py` | `user_blobs` table (§3.3), `store_user_blob` / `fetch_user_blob`, same shape as `store_keypair` / `fetch_keypair` | +| Handlers | `transport/webrtc_server.py` | `_do_user_blob_store` / `_do_user_blob_fetch`, `self._user_id` from the authenticated session (NS6), never from the message | +| Cap | same | Refuse over 256 KB with a stated reason; refuse an unknown `kind` | +| Audit | same | `user_blob_store` / `user_blob_fetch` events, same as `keypair_bundle_store` already logs | + +`user_id` comes from the session, exactly as `_do_keypair_bundle_store` takes +it — a `user_id` in the message body would let any member read or overwrite +any other member's blob, which is the C5 shape one size down. + +**No hub change. No change to indexing, streaming, transcoding, or the GEK.** + +### 7.3 What the node can and cannot do with it + +It can delete it, lose it with its disk, or serve a stale copy (§5.4 covers +the last). Durability is "the local copy, plus N node copies", with no +guarantee from any single one — which for an account on one node means the +local copy matters. Worth one line in the UI, not a warning dialog. + +--- + +## 8. Client side + +### 8.1 A pure module, deliberately separate from the UI + +`static/playlists.js` holds the data layer and **no UI**: load/merge/save, +the tombstone rules, the id/path repair, seal and open. This is not tidiness. +Source-reading tests are weak evidence and are most of what this repo can do +for the SPA — but a pure function over two objects can be *executed*, the way +`tests/harness/mse_harness.mjs` lifts the real player functions and runs +them. The merge is the one part of this feature that can be properly tested, +so it must not be entangled with a component. + +`tests/harness/playlist_merge.mjs` runs the real `merge()` over scripted +divergences: two devices, one node offline for each in turn, a delete on one +side and an edit on the other, a resurrect attempt, a `rev` tie. Model the +environment, never the code under test. + +### 8.2 Where it appears + +Playlists cross groups, so they do not belong to a group's Music tab: + +- **New route `/playlists`** and `static/playlists-page.js`, a sibling of + `/search`, plus a sidebar entry. It renders from the **cached** indexes + (`getAllCachedIndexes`), so it opens instantly and works with everything + offline; it dials only to play, or on "Sync now". +- **"Add to playlist"** in `music-app.js` (album and track level) and in + `search-page.js`'s music results — both already have the entry in hand, + with `groupId` attached. +- **The player bar needs nothing.** It is already at shell level in `app.js`, + already resolves a connection per track from `entry.groupId`, and already + crosses groups within one queue. This is worth stating loudly, because the + cross-group requirement reads like the hard part and is in fact already + built (`musicbay.md` §9b: *"The player needed no change"*). + +### 8.3 The IndexedDB detail that will otherwise be missed + +`hub-client.js` opens `meshbay` at `IDB_VERSION = 1` with a single store, +`group_indexes`. Adding a `playlists` store means **bumping to 2** and +handling it in the existing `onupgradeneeded`, which currently creates one +store and would otherwise never run again. A store that is never created +throws on first access, at a point far from the version constant. + +### 8.4 Checklist, per `apps.md` + +1. `playlists.js` (data) and `playlists-page.js` (UI). +2. Route + sidebar entry in `app.js`. +3. `webapp.py`'s `_ASSETS` tuple — both new files. +4. `test_hook_ordering.py` (`STATIC_FILES`) and + `test_transport_contracts.py` (`SPLIT_FILES`) — both new files. +5. i18n keys in all ten `static/locales/*.js`; `test_locales.py` holds them + to `en.js`'s key set. +6. `npm run sync-ui` in `meshbay-client`, confirmed reported. + +No `apps.js` registry entry and no `ALLOWED_APPS` change: this is not a group +application. It is a page, like Search. + +--- + +## 9. Security — per adversary + +| Claim | Passive hub | Active hub | Malicious node operator | Another member | +|---|---|---|---|---| +| Playlist contents (which tracks, which groups) | ✅ never transmitted to the hub | ✅ never transmitted to the hub | sealed under a key derived from the passphrase; the node holds ciphertext only | ✅ never served to anyone but the authenticated owner (`user_id` from the session) | +| Existence of playlists / how many | ✅ | ✅ | visible — one row, `stored_at`, and a padded length (§4.3) | ✅ | +| Editing timing | ✅ | ✅ | visible for writes reaching *that* node | ✅ | +| Integrity of the merged state | — | — | can serve stale or nothing; cannot lower the merged `rev` (§5.4) | — | +| Deleting a playlist | — | — | can delete its own copy; other nodes and the local copy survive it | — | +| Which tracks are actually played | — | — | already visible — the node serves the bytes | — | + +**The claim this design supports:** playlists add **no new authorization +boundary and no new key hierarchy**. The node already stores an opaque +per-account object for this same account under this same key material; this +is a second payload of an existing kind, and the hub is not involved at all. + +**The claim it must not make:** that a node cannot lose or withhold a +playlist. It can. The property is convergence with a local authority +(§5.4/§5.5), not durability guaranteed by any node. + +**One thing §4.2's fallback changes.** Playing a track from group B because +group A's node is offline means operator B, not operator A, sees that play. +Both already host that file for this account and already see its other plays, +so nothing new is learned by anyone — but it is a substitution of *observer*, +it is not obvious from the UI, and so it is written down here rather than +discovered later. + +--- + +## 10. Open items — deliberately deferred + +| # | Item | Why not now | +|---|---|---| +| O1 | Sharing a playlist with other group members | **Explicitly out of scope** (settled before this document). It is not an extension: a shared playlist is group state, sealed under the GEK, with concurrent writers — which makes §5's per-playlist LWW insufficient and an OR-Set over tracks mandatory. If it is ever wanted, reopen §5, not §3 | +| O2 | A second copy wrapped under the account recovery key | `bundle_enc_recovery` is the exact precedent and the column is reserved in §3.3, so this is a client-side change alone when wanted. Not built now because a forgotten passphrase already strands more than playlists | +| O3 | Smart/auto playlists (by artist, by year, recently added) | These are queries over the cached index and need no storage at all. Genuinely a separate feature, and cheaper than this one | +| O4 | Ordering conflicts resolved better than LWW | Reordering is rare and losing a reorder is survivable; losing an added track is not, and per-playlist `rev` already prevents that for one user. Revisit only with O1 | +| O5 | Export / import a playlist as a file | Trivial once §8.1 exists (it is `JSON.stringify` of a merge unit) and worth doing, but it is not what makes playlists work across devices | +| O6 | Tombstone collection driven by an explicit per-node acknowledged `rev` | §5.3's rule is adequate at this scale; a real garbage collector matters at thousands of deletions, which is not a real state | + +--- + +## 11. Acceptance before shipping + +1. `tests/harness/playlist_merge.mjs` runs the real `merge()` through the + five divergences in §8.1, including the resurrect attempt, and **fails + with the tombstone rule removed**. Check that, or the test is decoration. +2. A round trip through a real node: seal, `user_blob_store`, restart the + daemon, `user_blob_fetch`, open. Confirms the blob survives the process, + not just the test. +3. Read `bundles.db` back and confirm the playlist plaintext is not in it — + the same check `test_chat_key_storage.py` already makes for epoch keys, + for the same reason. A plaintext table beside it is the obvious thing to + write and would collapse the whole claim silently. +4. Two browsers, one account, one node: edit in each, converge, and confirm + both agree. Then repeat with the node stopped between the two edits, and + confirm neither edit is lost when it comes back. +5. Open `/playlists` with **every** node offline and confirm the page renders + in full — names, artists, durations, entries greyed — because that is the + requirement §4.2 exists for and it cannot be unit-tested meaningfully. +6. Confirm the size cap refuses rather than truncates, and that the refusal + reaches the client as a stated reason rather than a bare `error`. +7. Confirm sign-in still runs Argon2id exactly **once** after §3.4's change + (measure it; the budget is the 650 ms already recorded in draft-v5 §7.1). |