# MeshBay Node Protocol (MNP) **Wire version:** `3.4` — `meshbay_common/__init__.py` (`MNP_VERSION`) **Oldest peer accepted:** `3.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) **Normative implementation:** `meshbay-common` (`protocol.py`, `handshake.py`, `groupbox.py`, `chatbox.py`, `adminop.py`, `join.py`, `device.py`, `crypto.py`, `webcrypto.py`), `meshbay-node` (`transport/wire.py`, `transport/webrtc_server.py` and `transport/webrtc/`, `transport/quic_server.py`, `transfers.py`, `uploads.py`), browser client (`meshbay-hub/static/transport.js`, `static/crypto.js`). **Document status:** descriptive specification of the protocol as implemented on 2026-09-10. It describes the protocol as it stands. Where the code and this document disagree, the code is authoritative and this document is the thing to fix. **It is self-contained.** Every rule below is given with the reason it exists, in place — a reader should never have to open a second file to find out what a rule is protecting. The only outward references are to source files, which are the authority for details this document rounds off. --- ## 1. Scope MNP is the protocol spoken between a **client** (browser SPA, desktop client, CLI) and a **node** (the daemon that holds a group's files and the group key). It covers: * mutual authentication of client and node, bound to the concrete transport channel; * delivery of the group key (GEK) to an identity the node has pinned; * the content plane — index, file chunks, uploads, video streaming, chat; * the transfer slots a download or an upload runs under, and the caps on them; * operator-authorized administration of the node, signed with the operator's key. MNP is **not**: * the protocol between node and hub — that is MHP (`0.1`), which carries signaling, revocation push and presence, and is out of scope except where a step of MNP depends on it (§4); * a discovery mechanism. Presence comes from the hub's socket registry; `ping`/`pong` exists only for liveness on an *already open* channel, because opening a connection costs a full ICE/DTLS handshake (measured 0.6–7 s). ### 1.1 Design invariants These hold for every exchange described below. They are the reason the protocol has the shape it has, and each is argued where it is applied. | # | Invariant | |---|---| | I1 | **The hub is not trusted with content or keys.** It issues JWTs and relays SDP. A valid JWT is necessary but never sufficient: every session must additionally prove possession of the group key. | | I2 | **Nothing arriving over MNP contributes key material.** The node generates the group key itself, and each chat epoch key too, and wraps them for keys peers have proved they hold. No message exists by which a member hands the node key material, and none may be added. | | I3 | **Authority over the node comes from the node's own roster**, never from a token claim. Privileged operations are authorized by an Ed25519 signature over a structured transcript (§10). | | I4 | **Every signed or MAC'd transcript is domain-separated and length-prefixed.** Bare concatenation is forbidden: without the lengths, two different field splits produce the same bytes, and a signature over one is a signature over the other. | | I5 | **Every proof is bound to the channel it was made on.** An absent channel binding is a refusal, never a degraded proof (§6.4). | | I6 | **Authentication is mutual.** The node proves group-key possession over a client-chosen nonce and signs the transcript with its long-term key; the client pins that key per node (§6.1, step 11). | | I7 | **Per-group isolation.** Peer registry, chat store, chat epoch keys and index are resolved per group on a multi-group node. | | I8 | **A message that must open under the group key, and does not, ends the session.** Never a default, never an empty result: an unopenable `enabled_apps` would read as "the operator disabled every app" and an unopenable index as "the group is empty", both indistinguishable from legitimate states (§11.1, §6.6). | | I9 | **Both peers declare the protocol range they speak, and check the other's.** A mismatch is a refusal with a code, not a field that turns up missing (§13). | | I10 | **A requirement only a newer peer can meet is enforced at the handshake, not per message.** The alternative — an opt-in switch, "enforce it for peers that speak the new version" — leaves the permissive branch reachable on every node, and the branch left open is the one that gets used (§13). | --- ## 2. Notation ``` C the client (browser SPA, desktop client) N the node daemon H the hub (signaling and token issuance only) X -> Y msg X sends message type `msg` to Y [...] optional / conditional || byte concatenation LP(x) len(x) as 4-byte big-endian, followed by x b64(x) standard base64 of x, as an ASCII string ``` Field names are given exactly as they appear on the wire. A message is a msgpack map; `type` and `v` are present on every message the node emits, and `type` on every message it accepts. --- ## 3. Framing and encoding ### 3.1 Frame format Identical on every transport: ``` +--------------------------------+------------------------------------------+ | length : uint32, big-endian | payload : msgpack map (use_bin_type=true) | +--------------------------------+------------------------------------------+ 4 bytes `length` bytes ``` * WebRTC: frames are written to a single ordered, reliable `DataChannel` named by the client; the receiver accumulates bytes and extracts complete frames (`_DataChannelBuffer`). A frame may span several DataChannel messages, and one DataChannel message may carry several frames. * QUIC: each **bidirectional stream** carries one request/response exchange; the handshake runs on the first stream (`_StreamBuffer`, same extraction logic). ### 3.2 Size limits | Bound | Value | Where | |---|---|---| | Max frame **before** the client's group-key proof | 64 KiB | `PRE_HANDSHAKE_MAX_MSG` | | Max frame **after** the proof | 64 MiB | `MAX_MSG` | | File chunk (plaintext) | 1 MiB | `CHUNK_SIZE` | | Video segment (plaintext, before encryption) | 256 KiB | `STREAM_SEGMENT_SIZE` | | Upload chunk sent by the browser | 48 KiB | fits the aiortc SCTP limit after msgpack overhead | | Upload total per file | 8 GiB, operator-settable | `MAX_UPLOAD_BYTES`, `max_upload_gb` | | Files one session may read at once **without a transfer lease** | 12 | `MAX_LEASELESS_IN_FLIGHT` (§11.2) | The two-tier frame limit is not tidiness. A flat 64 MiB budget applied before authentication let an unauthenticated peer announce a large frame and dribble bytes into it, holding that much memory per connection for as long as it liked; a hundred such connections is the node's memory, from peers that have proved nothing. Exceeding the limit is a hard protocol error and the buffer raises rather than truncating — truncating would hand a parser a valid-looking prefix of something it never received. ### 3.3 Versioning field Every message carries `v`. It is **not** what decides compatibility: the version each side speaks and the oldest it accepts are exchanged and checked once, in the first message each peer sends, before anything else is decided (§13.1). A `v` on any later message is informational — no handler branches on it — and a peer whose range was refused never gets to send one. ### 3.4 Errors A refusal is a frame of type `error`: ``` { "type": "error", "detail": , ["code": ], ["req_id": ], ["upload_id" | "tr" | "file_id": ] } ``` * `detail` is authored to be safe to show a peer. Raw exception text, ffmpeg stderr and stack traces never reach the wire — they name paths on the operator's disk and versions of the operator's software, to somebody who asked for a file. * `code` is the same refusal in a form the client can act on. Matching on `detail` is a string comparison that breaks the day someone improves the wording. * `req_id` is stamped on every reply sent while answering a request, refusals included (§3.5). It is what makes a refusal reach the caller that earned it: `error` is the one reply with no field of its own to be recognised by, so without the id it cannot be routed at all. * An upload refusal names the **`upload_id`**, never the file: the filename is inside the seal, and quoting it back in clear would hand over exactly what sealing the upload path is for. A transfer refusal names the `tr`; a refusal to serve an unleased read names the `file_id`, which the caller sent in clear anyway. Codes in use: | Family | Codes | |---|---| | Handshake | `not_a_member` (§6.3); `version_too_old`, `version_too_new`, `version_unreadable` (§13.1) | | Transfers | `transfer_required`, `bad_transfer_id`, `bad_transfer_size`, `bad_kind`, `not_your_transfer`, `too_many_queued` (§11.2) | | Upload | `upload_not_sealed`, `no_group_key`, `upload_incomplete`, `bad_chunk_encoding`, `bad_chunk_index`, `invalid_filename`, `no_roots`, `no_such_root`, `no_writable_root`, `root_read_only`, `root_unavailable`, `no_such_directory`, `already_exists`, `not_started`, `too_large` (§11.4) | | Directories | `root_read_only`, `root_unavailable` (§11.5) | Everything else refuses with `detail` alone. A code is added when a client has a different thing to *do* about the refusal — retry, re-authenticate, offer an update — and not merely to enumerate. ### 3.5 Request/response correlation **A request may carry `req_id`, and the node stamps it on every reply it sends while answering that request.** The client draws it, the node never interprets it, and the match is exact. The alternative is matching a reply to a request by arrival order, which is a guess: it is wrong whenever two replies reorder, and it has no chance at all for the one reply that names nothing of its own — `error`. A refusal that reaches no caller leaves the request it belonged to waiting out its timeout while some unrelated request is resolved with the refusal instead. Node-side the id lives in a task-local (`contextvars`), not threaded through every send site, and it is stamped **only on messages going back to the session that asked**. A handler that also pushes to other peers — a chat broadcast, an index delta — reaches them through their own connection, where nothing is stamped, because those messages answer no request. An explicit `req_id` already on a message wins over the ambient one. Routing, in order. A message carrying no id is either a push or a broadcast, and each has a key of its own: | Message | Matched by | |---|---| | any reply carrying `req_id` | that id, exactly | | `file_chunk` | `file_id` + `chunk_index` | | `index_sync`, `index_delta` | queued, opened, then resolved by their id — see below | | `file_upload_ack`, upload `error` | `upload_id`, against the uploader that drew it | | `transfer_state` | `tr`, against the lease that drew it | | `pong` | the echoed `token` | | `media_meta_resp`, `music_meta_resp`, `audio_transcode_resp` | the `file_id` asked about | | `season_meta_resp` | `tmdb_id` + `season` | | `link_preview_resp` | the `url` | | `admin_challenge` | the `op` field, against the pending request that named that op | | `*_ack` from a signed op | `type` minus the `_ack` suffix, against the same key | | `chat_msg`, `stream_*`, `*_ack` broadcasts | dedicated handlers; they are unsolicited | Two of those keys are worth their line. Chunks are the one request that runs several at a time interleaved with everything else, so nothing but a key of their own can identify them. A `pong` is sharper still: it is sent *while* other traffic is in flight, so anything less than an exact match would hand it to whatever was waiting — resolving a history request with a message that has no messages in it, and emptying the conversation on screen. The index messages are the one case where the id is not enough on its own, and it is the shape any future sealed reply will have: they carry an id like everything else, but cannot be handed to their caller until they are **opened**, which the synchronous dispatcher cannot do. They are queued, opened, and resolved afterwards under the same id. Resolving them on arrival would give the caller an envelope — a nonce and a ciphertext — and skip the handler that decrypts. Anything that arrives naming no request and matching no key is unsolicited and is dropped rather than handed to a waiting caller. New request/response pairs that can overlap in flight **must** therefore be distinguishable: `req_id` is the general answer, and a discriminator of the message's own (`file_id`, `url`, `upload_id`, `tr`, `op`) is what keeps a reply matchable without it. --- ## 4. Session model ``` +-------------------------+ | channel established | no MNP state yet +-----------+-------------+ | handshake v +-------------------------+ refuse <-----+ version range checked | too old / too new / unreadable | token authorized | JWT decoded, NOT authenticated +-----------+-------------+ | handshake_challenge v +-------------------------+ | PRE-PROOF WINDOW | bounded: 4 fetches, 5 join attempts, | bundles, join, device | 64 KiB frames, every event audited +-----------+-------------+ | handshake_response (valid client proof) v +-------------------------+ | AUTHENTICATED | `_user_id` / `_group_id` set, | full message set | frame limit raised to 64 MiB +-----------+-------------+ | channel closes v +-------------------------+ | TORN DOWN | transfer leases released, peer | | unregistered, tasks cancelled +-------------------------+ ``` **States** 1. *Unauthenticated.* Only `handshake` is accepted. Anything else is answered `Handshake required`. 2. *Pre-proof.* Entered when the version ranges agree, the JWT authorizes, and the node has a group key for the group. The identity is **decoded but not authenticated**. The only messages accepted are the ones a peer provably needs before it can compute a proof: `keypair_bundle_fetch`, `gek_bundle_fetch` and `join_request`. This window is a disclosure surface a hub that forges a JWT can reach, so it is bounded and audited (§7). 3. *Authenticated.* The client's HMAC over the handshake transcript verified. The full message set opens, and the node has answered with its own proof and signature. 4. *Torn down.* Everything the connection held is given back, and the important word is *deterministic*: a transfer lease is scoped to the connection precisely so that a closed tab, a quit browser and a dropped network all arrive here and none of them needs a timer (§11.2). A session is per (connection, group). `group_id` is mandatory in the handshake, so one connection serves exactly one group; a client in two groups on one node opens two connections. A session may additionally identify **which device** of the account it is, with `device_hello` (§9.4) — the handshake proves the account and the group, and never proved the device. --- ## 5. Transport establishment MNP is transport-agnostic above the frame layer. **WebRTC is the transport**: it is what the browser SPA and the desktop client speak, and it implements the whole protocol. A QUIC transport is in development — see §5.2. Every transport shares one handshake module, and a parity test fails if one grows a copy of its own. A second implementation of an authentication step is a second place for the group-key proof to be skipped. ### 5.1 WebRTC (browser and desktop client) The hub relays SDP only; it never sees a DataChannel byte. Non-trickle ICE: the offer carries its candidates, with a 4 s gathering deadline after which the client offers whatever it has (host candidates are enough on a LAN). ``` C (browser) H (hub) N (node) | | | | |<===== MHP WebSocket ====>| persistent, authenticated | | /v1/nodes/ws | Ed25519 node auth | | | | create offer, gather ICE (<= 4 s) | | | | |--- POST /v1/nodes/{node_id}/webrtc/offer --------->| | {sdp, ice_candidates}| | | | authorize: shared active group, | | or an open-join group when public | | groups are enabled; <=16 KiB SDP; | | <=3 pending per user; 30/min | | | | |--- ws {webrtc_offer, | | | peer_id, user_id, | | | sdp} ------------>| | | | RTCPeerConnection, | | | answer + ICE | |<-- ws {webrtc_answer, | | | peer_id, sdp} ----| |<-- 200 {sdp, ice_candidates, peer_id} -------------| | | | | setRemoteDescription; DTLS; SCTP; DataChannel open | | | |======================= MNP frames =================| hub is out of the loop ``` Notes that matter to MNP: * The **raw answer SDP is retained before `setRemoteDescription`** — Chrome strips `sha-256` from a multi-hash SDP, and the fingerprint is needed for the channel binding (§6.4). * The node answers offers **off its WebSocket read loop**: awaiting negotiation inline would stop it reading the socket for the length of one slow ICE run — running, but invisible to the hub. * A 15 s timeout on the hub's side turns a silent node into `504`, not a hung request. ### 5.2 QUIC (in development) A QUIC transport is being built, for the LAN, port-forwarded and hub-less cases where signaling through the hub is unnecessary or unavailable. **It is not functional and is not a shipped feature.** No client speaks it, it implements only part of the message set, and nothing in this document should be read as a statement about what it does today. What is settled is the framing and the identity model, and both are recorded here because they constrain the design of everything else: ``` C (native) N (node) | | |---- QUIC connect, ALPN "meshbay-mnp" ------->| TLS 1.3, self-signed node cert |<---------------------------------------------| the certificate is the identity | | |==== stream 0 : MNP handshake =============== | |==== stream n : one request/response each === | ``` The node's TLS certificate is not verified as a PKI chain — identity is established at the MNP layer, and the certificate hash is the channel binding (§6.4). --- ## 6. The handshake One implementation for every transport: `meshbay_common/handshake.py`. A parity test fails if a transport skips a step. ### 6.1 Full exchange ``` C N | | | 1. handshake | | {v, v_min, token, group_id, nonce: b64(nonce_c)} | |-------------------------------------------------------------->| | 2. check_version() | | - v >= our v_min | | - v_min <= our v | | else error{code} | | authorize_token() | | - EdDSA verify vs hub | | - scope == "user" | | - sub non-empty | | - group_id non-empty | | - denylist(user,jti,gp)| | - group_id in groups[] | | - group hosted here | | 3. |nonce_c| >= 32 | | 4. GEK exists for group | | | | 5. handshake_challenge | | {v, v_min, nonce: b64(nonce_s), node_pk, | | sig: b64(Ed25519(C))} (3.4, section 6.5) | |<--------------------------------------------------------------| | | | 5b. client checks the node's range the same way | | | | ....... pre-proof window (section 7) ....................... | | keypair_bundle_fetch / gek_bundle_fetch / join_request | | -- the client obtains a GEK to prove with | | ............................................................ | | | | 6. binding = webrtc_binding(offer_fp, answer_fp) | | proof_c = HMAC-SHA256(GEK, T("client")) | | | | 7. handshake_response {v, proof: b64(proof_c)} | |-------------------------------------------------------------->| | 8. rebuild binding; | | refuse if empty; | | compare_digest(proof) | | 9. session authenticated: | | frame limit -> 64 MiB, | | peer registry, audit | | | | 10. handshake_ack | | {v, node_pk, proof: b64(proof_n), sig: b64(Ed25519(T)), | | nonce, ct} | | ct = seal(GEK, "ack", "handshake_ack", group_id, | | {is_node_admin, enabled_apps, | | _directories, ...}) | |<--------------------------------------------------------------| | | | 11. verify proof_n == HMAC(GEK, T("node")) -> else refuse | | verify Ed25519(ack.node_pk, ack.sig, T("node")) -> refuse | | verify ack.node_pk == challenge.node_pk -> else refuse| | pin/compare node_pk for this node_id (TOFU) -> else refuse| | 12. THEN open ct -> else refuse (never a default config) | | | |========================= session open ========================| ``` Step 11 is not optional politeness. A client that accepts a bare `handshake_ack` without a preceding challenge, or that skips any of these checks, reopens the hole this step exists to close: a peer that had hijacked signaling could accept the client's proof, ignore it, and serve a forged index, forged chat history and a forged `is_node_admin` flag — the last of which offers the person an administration panel on somebody else's node. **Step 12 comes after step 11, and the order is the point.** Everything in step 11 decides whether this peer is worth trusting at all; opening the payload first would mean acting on data from a peer not yet authenticated. And a payload that does not open is a refusal, not an empty configuration — see §6.6. ### 6.2 Transcript ``` T(role) = "meshbay:mnp:handshake:v1" || LP(role) "client" | "node" || LP(group_id) || LP(nonce_c) >= 32 bytes, client CSPRNG || LP(nonce_s) 32 bytes, node CSPRNG || LP(binding) transport channel binding, MUST be non-empty proof = HMAC-SHA256(GEK, T(role)) ``` The role is inside the transcript, so a client proof can never be replayed as a node proof. `nonce_c` is what makes the **node's** proof fresh: without it a recorded `handshake_ack` is replayable by an impersonating peer. `make_proof` raises rather than returning a value when `binding` is empty or the GEK is absent. `verify_proof` compares with `hmac.compare_digest`. ### 6.3 Authorization rules (`authorize_token`) | Rule | Refusal | Rationale | |---|---|---| | JWT verifies under the hub's Ed25519 public key (`EdDSA`) | `Invalid JWT: ...` | | | `scope == "user"` | `Wrong token scope` | a node-scoped daemon token must not be usable as a client token | | `sub` non-empty | `Token has no subject` | | | `group_id` non-empty | `group_id is required` | an absent group means no membership check to make; there is no default group, and a node's first group is not one | | not on the denylist for `user_id`, `jti` **or** `group_id` | `Token revoked` | all three targets, and persisted to disk: a revocation that a restart forgets is not one | | `group_id ∈ token.groups` | `Not a member of this group`, code `not_a_member` | the membership check itself — a token is proof of an account, never of a group | | `group_id ∈ node.hosted_groups` | `Group not hosted on this node`, code `not_hosted` | the hub may hand a client several nodes for one group, and only some of them host it | `AuthorizedPeer` carries `user_id`, `group_id`, `username`, `jti` — and deliberately **no user public key**. A key arriving in a token would be a key the hub chose, and the node records the uploader's key in order to decide who may later delete a file: that would let whoever issues tokens decide it instead. Identity keys are pinned by the node's roster. The hub certifies accounts, not keys. `not_a_member` is almost always a token minted before the person was added to the group (`groups` is baked in at login and the hub pushes no updates), so the client refreshes once and retries on that code rather than telling a member they are not one. `not_hosted` is the client's signal to try the **next** node the hub offered for the group rather than to report a failure. `/v1/groups/{id}/nodes` returns every node registered for the group, in hub registration order, and that order is not a ranking: a node listed first is not necessarily one that holds the group's files. Refusing without a code made this indistinguishable from a refusal the reader has to act on, and a client that stopped at the first node let one wrongly registered peer make a group unopenable for all of its members (2026-09-11). ### 6.4 Channel binding | Transport | Anchor | Construction | |---|---|---| | WebRTC | both DTLS certificate fingerprints | `LP(offer_fp) \|\| LP(answer_fp)`, each the raw 32 bytes of the `a=fingerprint:sha-256` line | | QUIC | node certificate | `LP(SHA-256(server_cert_der))` | An empty binding is refused on both sides (`Channel binding unavailable`) — an absent binding is never a degraded proof, because a proof that is not bound to a channel is a proof somebody can relay. The QUIC anchor is weaker than an RFC 5705 exporter, which aioquic does not expose: it names the server's certificate rather than the concrete session, so on a resumed session the anchor travels with the session ticket. Stated here and in §14.2 rather than left to be inferred. ### 6.5 `node_pk` in the challenge The node announces its public key in `handshake_challenge`, before anything is proved. This is deliberate and safe: * a first-time joiner needs it *before* the ack — `join_request` signs a transcript naming this node (§8.2), and someone who has never held the GEK cannot complete the handshake that would prove the key; * the ack proves possession and signs the transcript; the client refuses if `ack.node_pk` differs from the announced value; * a wrong value only makes the node's own verification fail. **Since 3.4 it is also signed, in the challenge itself:** ``` C = "meshbay:mnp:challenge:v1" || LP(group_id) || LP(nonce_c) || LP(nonce_s) || LP(binding) sig = Ed25519(sk_node, C) ``` `binding` is the transport's channel binding (§6.4), already known when the challenge is sent; `nonce_c` makes the signature fresh. So it cannot be recorded and replayed, nor relayed through a peer with different fingerprints. The prefix is its own, so it is never interchangeable with the ack's signature over `T("node")`. Why it exists: a join (§8) is sent **in the pre-proof window**, before the ack — so before 3.4 an invitation code went to whichever peer answered signaling, and in a group with two hosts, to whichever host answered first. With the signature, a client that knows which node key to expect can refuse to send a code anywhere else. What it does not do: it proves that the peer holds *the key it announces*, not that this is the key the client wanted. A client with no expectation learns nothing more than before, and the ack remains the proof of GEK possession. Client rule, from the node's answer and never from its version (§13): a `sig` that does not verify is a refusal (`Node challenge signature invalid`); no `sig` is an older node, whose key is proved only at the ack. A node signs whenever it has a binding, and a node with none sends no signature rather than an unbound one — the proof would be refused on that connection anyway. ### 6.6 `handshake_ack` fields Three fields are in clear, and the rest travel **sealed under a group-key-derived subkey** (§11.1a). The split is not aesthetic: the three below *are* the authentication, and a client verifies them in order to decide whether to trust anything at all — including a decryption. | Field | Meaning | |---|---| | `node_pk` | node's long-term Ed25519 public key, base64 raw 32 bytes | | `proof` | `HMAC(GEK, T("node"))` | | `sig` | `Ed25519(sk_node, T("node"))` | | `nonce`, `ct` | the sealed payload; everything below is inside it | | `is_node_admin` | whether this peer is the node's operator — computed from the node's own record (`node_user_id`), never from a hub claim | | `node_user_id` | the operator's account id, when known | | `node_pk_x25519` | the node's X25519 public key, when configured | | `enabled_apps` | which group applications to show. Empty/absent means "all registered ones" client-side | | `_directories` | each application's entry-point folders, keyed by the app's own registry name (`video`, `music`, `photo`, `chat`), **always a list**. This is the only form. The scalar `video_root` / `audio_root` / `photo_roots` fields that used to sit beside it are gone: one folder was never the general case, and two shapes for one answer meant whichever the reader consulted first decided it | | `chat_directory` | where chat attachments are written. Singular because Chat genuinely has one destination; `""` means the operator has not chosen | | `chat_link_preview` | whether the node unfurls links posted here. Absent means on | | `search_listed` | whether the reader's cross-group Search lists this group. Presentation only — the index is served identically either way. Absent means listed | | `chat_epoch` | the chat epoch a client must seal under right now (§11.7). There is no `chat_encrypted` beside it, because there is no switch | | `transfer_limits` | `{download, upload}` — this member's own caps in this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn; never as "unlimited", which would have the interface contradicting the node (§11.2) | | `tmdb_enabled`, `musicbrainz_enabled` | per-group metadata lookups | | `tmdb_token_customized`, `tmdb_language` | node-wide TMDB config; the token itself is never sent | | `indexing` | `{scanning, scanned_bytes, total_bytes}` so a client connecting mid-scan shows progress immediately. Never a path or filename | | `scan_settings` | `{reconcile_interval_secs, debounce_secs}` — displayed, not enforced from here | Everything after `is_node_admin` is presentation state. It rides on the ack so a client that connects after the operator configured something does not have to wait for a live change notice to discover it. **Why this payload is sealed, and it is integrity rather than confidentiality.** The node signs `T("node")`, which names `role`, `group_id`, both nonces and the channel binding — and **no ack field at all**. Without the seal, every value in the table above would be authenticated by the DTLS/TLS channel and nothing else. Sealing gives them an AEAD tag from a key the hub does not hold, which is a stronger statement than any amount of confidentiality on the index. `chat_epoch` is the sharpest example: a forged one would have a client sealing its messages under a key the group has retired. On QUIC (§5.2) the payload is sealed but **empty**: it carries none of these fields, because it serves no browser. The seal is there regardless, so that one message has one shape on every transport — a field added later then has somewhere authenticated to go, rather than arriving in clear beside a sealed one. A payload that does not open **ends the session**. It is not an empty configuration: an `enabled_apps` that failed to open would reach the client's documented fallback — show every registered app — which is a confident wrong answer, indistinguishable from an operator's real choice (I8). --- ## 7. The pre-proof window Between `handshake_challenge` and a valid `handshake_response` the peer is *authorized* but not *authenticated*. Three message families are served there, each because the peer provably cannot compute a proof without it. **Everything else is answered `Handshake required`**: the dispatcher's authenticated branch begins immediately after these three, so the table below is the window, exhaustively. | Message | Why it must precede the proof | Bound | |---|---|---| | `keypair_bundle_fetch` | the client's own identity keys for this node live in an encrypted bundle stored on it | counts against `MAX_PRE_PROOF_FETCHES` = 4; audited | | `gek_bundle_fetch` | the wrapped group key is what the proof is computed with | same counter | | `join_request` | a first-time member holds no group key at all. Accepted after the proof as well — an operator pairing a browser is already connected — because its authority comes from the pairing code and the signature, never from the session state | 5 attempts per connection, 20 failures per 600 s node-wide | Device linking (§9) is **not** in this window. `device_add_request` and every message after it are answered only on an authenticated session, and the device budget of 5 attempts per connection applies there. Exceeding the fetch budget is audited as `pre-proof fetch flood` and answered `Too many requests`. Every fetch in this window is written to the audit log with the message type, because this is a disclosure surface a hub that forges a JWT can reach: the hub mints the tokens, so it can present one for any account, and what it can then ask for is that account's *encrypted* keypair bundle. The bundle is useless without the account passphrase, which is why the window is bounded and audited rather than closed — and it closes for good when clients stop storing keypair bundles on other people's nodes. ### 7.1 Identity bundles ``` C N |-- keypair_bundle_fetch {v} ----------------------->| |<- keypair_bundle_resp {v, found, | | [bundle_enc], [bundle_enc_recovery]} ----------| | | | decrypt bundle_enc with the passphrase-derived bundle key, | or bundle_enc_recovery with the recovery key | | |-- keypair_bundle_store {v, bundle_enc, | after minting or re-wrapping | [bundle_enc_recovery]} ----------------------->| |<- ack {v, detail: "keypair_bundle_stored"} --------| | | |-- keypair_bundle_delete {v} ---------------------->| withdraw the backup ``` * The bundle is opaque to the node: it is encrypted client-side under a key derived from the account passphrase (`keyderive.js`), and optionally a second copy under the account recovery key. The node stores bytes and serves them back to the same `user_id`. * `keypair_bundle_store` is accepted **after** authentication (it is not in the pre-proof list); the fetch is what happens before. * A `store` omitting `bundle_enc_recovery` leaves any existing recovery copy in place. * Identity keys are **per node**. There is nothing to carry between nodes, and an operator who cracks the copy on their own disk gets a key that opens nothing anywhere else. ### 7.1a Per-account blobs (MNP 3.1) The same shape as a keypair bundle with a different payload — playlists today (`docs/playlists.md` §8). The node stores bytes it cannot read for an account it already holds a bundle for, so this adds **no new trust boundary**. ``` C N |-- user_blob_list {v} ---------------------------->| which kinds exist here |<- user_blob_list_resp {v, blobs: [{kind, rev}]} --| revisions only, no payload | | |-- user_blob_fetch {v, kind} ---------------------->| |<- user_blob_resp {v, kind, rev|null, | | blob_enc|null} -------------------------------| | | |-- user_blob_store {v, kind, rev, blob_enc} ------->| |<- ack {v, detail: "user_blob_stored"} ------------| | | |-- user_blob_delete {v, kind} --------------------->| ``` * **`kind` is a namespace, validated against a pattern**: `playlists` is the manifest, `playlist:` is one playlist's tracks. That is what lets one playlist be rewritten without re-uploading the whole collection, and it is a pattern rather than "anything" so the table does not become a key/value store for whatever a client feels like writing. * **`blob_enc` is msgpack `bin`, not base64.** These run to hundreds of kilobytes, where base64 is a third of every write. * **`user_id` comes from the authenticated session, never from the message.** A `user_id` in the body would let any member read or overwrite any other member's blob. * **Caps refuse, never truncate**: 64 KB for the manifest, 1 MB for one body, 8 MB per account per node, each with a stated reason. A truncating cap loses tracks silently, which is the failure the design exists to prevent. * The 1 MB body cap is **not the binding one**. A browser cannot send a frame above the negotiated `max-message-size`, which aiortc fixes at **65 536**, so a client tops out near 64 KB per write however high this cap is set — the same constraint that keeps uploads chunking at 48 KB. Reads are not limited that way: the node answers with a whole body of up to 1 MB. See `docs/playlists.md` §15.3. * A `fetch` for a kind never written answers `null`, not an error: that is the ordinary state of a node the reader has just joined. * The node keeps **no history**. The client is the authority on which revision is current and holds its own copy; a node keeping older revisions would mean the node deciding, which is exactly what it must not do. ### 7.2 Wrapped group key ``` C N |-- gek_bundle_fetch {v} ------------------------->| |<- gek_bundle_resp {v, found, | | [pk_eph_b64, nonce_b64, wrapped_b64]} -------| ``` The bundle is an ECIES wrap of the GEK for the caller's X25519 key: ``` sk_eph, pk_eph <- fresh X25519 keypair (node side) shared = X25519(sk_eph, pk_recipient) wrap_key = HKDF-SHA256(shared, salt = pk_eph, info = "meshbay:gek_wrap:v1:aes", len 32) wrapped = AES-256-GCM(wrap_key).encrypt(nonce_96, GEK, aad = pk_recipient) ``` AES-GCM because WebCrypto has no ChaCha20-Poly1305; a `chacha20-poly1305` variant with `info = "meshbay:gek_wrap:v1"` exists for native clients. The recipient's public key is the AEAD's associated data, so a bundle cannot be re-addressed. `found: false` is the normal answer: per-member bundles are not stored, and the key is produced on demand by the join path (§8). The node keeps one stored bundle of its own (`_node_{user_id}`), which is how the daemon reloads its GEK across restarts. --- ## 8. Pairing and join The substitution this section exists to prevent: an invite flow that fetched the invitee's public key **from the hub** and wrapped the group key for whatever came back would hand the group key to a hub that answered with its own — handed over by an honest inviter following the protocol exactly, with nothing anywhere looking wrong. So the key comes from its owner over an authenticated channel, and is bound to an identity by a one-time code the hub never sees. ### 8.1 Exchange ``` operator (paired) N invitee C | | | |-- invite_create ---->| (signed admin op, section 9)| | {user_id, | | | group_id, | | | username} | | |<- invite_result -----| | | {code, expires_at, | | | user_id, username}| | | | | |=== code delivered out of band, not via the hub ====>| | | | | |<-- handshake / challenge ----| nonce_s, node_pk known | | | | |<-- join_request -------------| | | {group_id, pk_ed25519, | | | pk_x25519, code, ts, sig}| | | | | | verify: attempts, node-wide | | | failure window, key format,| | | |ts - now| <= 120 s, | | | group_id == session group, | | | Ed25519(sig) over J, | | | roster device lookup, | | | consume_invite(code) | | | | | |-- join_result -------------->| | | {ok, recognised, role, | | | gek: true, | | | pk_eph_b64, nonce_b64, | | | wrapped_b64} | | | | | | client unwraps the GEK, then computes | | the handshake proof and completes (section 6) ``` ### 8.2 Join transcript ``` J = "meshbay:join:v1" || LP(node_pk_b64) the key announced in handshake_challenge || LP(group_id) "" for operator pairing, which is node-wide || LP(user_id) || LP(pk_ed25519_b64) the caller's own identity key || LP(pk_x25519_b64) the encryption key the GEK will be wrapped for || LP(nonce_s) the node's handshake nonce || LP(ts) unix seconds, decimal ASCII sig = Ed25519(sk_ed, J) ``` Two properties carry the design: * **the identity key vouches for the encryption key.** Both are inside one signature, which is what makes "wrap the GEK for the key the peer presented" safe; * **`nonce_s` binds the join to this connection**, so a signed join cannot be lifted onto another. The **code is never signed and never echoed.** It is a bearer secret: compared against a stored `sha256(code)` and destroyed on use. ### 8.3 Node-side decision table Evaluated in order (`_do_join_request`): | Condition | Outcome | |---|---| | `join_attempts >= 5` on this connection | `error: Too many attempts` | | `>= 20` node-wide failures in 600 s | `error: Pairing temporarily locked`, audited `join_throttled` | | key not 32 raw bytes, or bad base64 | `join_result{ok:false, reason:"invalid_keys"}` | | `\|ts - now\| > 120` | `stale_request` | | `group_id` non-empty and != session group | `group_mismatch` | | signature does not verify over `J` | `signature_invalid` | | Ed25519 key is a pinned device but the presented X25519 differs | `key_changed` | | account has devices here, this key is not one | `unknown_device` — the way in is a device-add (§9), not a new invite | | device known, no member row, group policy `open` | member row created (`approved_by: "open-join"`) | | device known, a **pending invite** exists for this user | code required even for a known device; `code_required` / `code_invalid` on failure | | device known, not an active member of the session group, a code offered | redeemed like any code — an invitation **link** reaches here from someone pinned through another group, or removed and invited back; `code_invalid` on failure | | device known, member row resolved | `join_result{ok, recognised:true, role}` + wrapped GEK | | unknown device, no code, policy `open` | pin TOFU, admit, wrap (`via: "tofu"`, audited) | | unknown device, no code, policy `invite` | `code_required` | | unknown device, code invalid or spent | `code_invalid` | | unknown device, code valid | pin identity, set member row from the invite, wrap (`via: "code"`, or `"link"`) | The member row is resolved as: this group's row, then the row for the join message's `group_id`, then the node-wide (`""`) row — which is where an operator opening any group finds their authority. `join_ok` refuses to produce a key when `roster.is_authorized(group_id, user_id)` is false: `join_result{ok:true, gek:false, reason:"not_authorized_for_group"}`. Hub membership alone must not produce a key. ### 8.4 Pairing codes * 8 characters, Crockford base32 (no `I`, `L`, `O`, `U`), rendered `XXXX-XXXX` — 40 bits. Reading one back is case-insensitive, dashes and spaces are decoration, and the excluded letters fold onto the digits they resemble: a code read out over the phone should not be able to fail in a way the node could have absorbed. * Single use, stored only as `sha256(code)`. A password KDF over 40 uniformly random bits would buy nothing. * Three lifetimes, each matched to the conversation the code crosses, and all three settable by the operator: | Code | Default | Why | |---|---|---| | member invitation | 7 days | it is sent by mail or message and answered whenever the other person next looks; a day dies over a weekend, and reissuing needs the inviter at a browser with the node online | | operator pairing (§8.5) | 24 h | it crosses an SSH session — printed, then typed minutes later | | device-add request (§9) | 1 h | read off one screen and typed into another, in one sitting | The longer window costs little: a code is single use, bound to one account, never seen by the hub, and 40 bits do not fall to guessing in a week against the node-wide lockout below. * Valid for exactly one `user_id` in one group — **except an invitation link** (§8.6). * Guessing is bounded per connection and node-wide, and every failure is an audit event rather than a silent grind. * `join_policy` is read from the **node's own configuration**, never from the hub: a hub that could declare a group open would be handing itself the key to it. ### 8.5 Operator pairing The same message with `group_id: ""`. Authority is node-wide, and the code comes from `meshbay-node operator pair` over SSH — the hub never sees it. The reason it cannot is worth stating plainly: the node cannot ask the hub which key belongs to its operator without letting the hub answer with its own, which is the same substitution as §8's invite flow, one level up, and it would make the hub node administrator everywhere. There is **one** source of operator authority and it is the roster: a node.toml naming an `admin_pk_ed25519` is warned about at startup and never obeyed, because a second source of authority is a second thing to get wrong. ### 8.6 Invitation links (`kind = "link"`) A code for someone who may have no account yet, so it names none: it is bound to the first account that redeems it (`invite_link_create`, §10.4). What stops a stranger holding it is not the node but the hub, which lets only the account whose verified address the inviter named reach the node at all (`MESHBAY_DESIGN.md` §3.4). At the node it is therefore a **bearer code**, and everything else about it is fixed: * role `member`, never `operator`; a row that says otherwise is refused, not honoured; * one named group, never node-wide, and only redeemed on a connection authenticated to that group — the account codes keep redeeming anywhere, as before; * not by someone already an **active** member of that group, who would otherwise spend it for the person it was meant for; a member row that is revoked does not count; * single use: redemption sets `used_at` **and** `user_id` in one guarded `UPDATE`; * at most **20 unredeemed per group** (`MAX_LINK_INVITES_PER_GROUP`); * cancellable before use by `invite_cancel`, by an `invite_id` handle unrelated to the code; a redeemed one stays as the record of the join. `kind` is a column, not an empty `user_id`: every query that matches invitations by account filters on `kind = "account"`, so a row bound to nobody can never be read as bound to anybody (AV1). A roster from before this gains `kind` and `invite_id` by `ALTER TABLE`, and its codes stay account codes. --- ## 9. Device linking Identity keys are per node, so one person using a browser and a desktop client holds two keys on the same node. A second device is admitted by **a key the node already pinned** — never by the hub, which stores no user keys and therefore cannot countersign anything. Every message in this section is answered on an **authenticated** session only (§7): both the device filing a request and the device approving it have completed a handshake on their own connection. ### 9.1 Exchange ``` new device D N approver A (already pinned) | | | | code <- random, 40 bits, displayed on D's screen | | code_hash = sha256(code "\x1f" pk_ed "\x1f" pk_x) | | | | |-- device_add_request ->| | | {pk_ed25519, | checks: account known here, | | pk_x25519, | < 5 devices, |ts| <= 120s,| | code_hash, ts, sig} | Ed25519(sig) over D_req | |<- device_add_request_ack | | {expires_at} | filed as pending, inert | | | | |=== the code is read off D's screen, typed into A ====>| | | | | |<--- device_lookup {} --------| | |---- device_lookup_result --->| | | {requests: [{pk_ed25519, | | | pk_x25519, code_hash, | | | created_at}, ...]} | | | | | | A recomputes sha256(code||keys) | | for each candidate and keeps the match. | | No match -> refuse before signing. | | | | |<--- device_add --------------| | | {pk_ed25519, pk_x25519, | | | code_hash, label, ts, | | | sig over D_add} | | | verify sig against EVERY | | | live device of the account; | | | take_device_request(hash) | | |---- device_add_ack --------->| | | {pk_ed25519} | ``` ### 9.2 Transcripts ``` D_req = "meshbay:device_req:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) || LP(pk_x) || LP(code_hash) || LP(nonce_s) || LP(ts) signed by the NEW device D_add = "meshbay:device_add:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) || LP(pk_x) || LP(nonce_s) || LP(ts) signed by a PINNED device code_hash = sha256( code "\x1f" pk_ed25519_b64 "\x1f" pk_x25519_b64 ) ``` * `D_req` is **proof of possession only**. It says the caller holds the keys, never that they belong to this account. The countersignature is what establishes that. * `D_add` deliberately **omits the code**: the code is a bearer secret used to find the request, never signed, never echoed. What is signed is the key pair being admitted, so a signature collected for one device cannot admit another. * Because both keys go into `code_hash`, a node cannot answer the approver with a substituted key: the approver recomputes the hash from what it typed and what it was given. Nothing here rests on a human comparing digits. * `device_lookup` takes no hash argument. Taking one from the client was circular — the client cannot compute the hash without already knowing the keys it is asking about. ### 9.3 Listing and revocation ``` C -> N device_list {} N -> C device_list_result {pending, devices: [{pk_ed25519, label, pinned_at, pinned_via, added_by_pk, is_this_one}]} C -> N device_revoke {pk_ed25519, ts, sig over D_add for the victim's keys} N -> C device_add_ack {revoked: pk_ed25519} ``` Anyone may read their own devices and nobody else's. Revocation is countersigned like an addition, and **the last device cannot be removed** — an account with no device on a node can only return through an operator's invitation code. A revoked device is marked, not deleted, so a lost laptop stops being able to admit its replacement. Per-connection attempt budget for every device message: 5, audited on exhaustion. `MAX_DEVICES_PER_USER` is 5. ### 9.4 Which device is on this connection (`device_hello`) ``` C -> N device_hello {pk_ed25519, ts, sig over D_hello} N -> C device_hello_ack {pk_ed25519} ``` ``` D_hello = "meshbay:device_hello:v1" || LP(node_pk) || LP(group_id) || LP(user_id) || LP(pk_ed25519) || LP(nonce_s) || LP(ts) ``` The handshake authenticates a **group membership** (the group-key HMAC) and an **account** (the hub's token). It does not authenticate a *device*, and an account may hold several (§9). Without this message the node can only guess which one is talking — and it records the uploader of every file and the author of every chat message, so a guess there is an attribution the person cannot correct. What is checked, in order: the key is a live device **of this account in the node's own roster** (never a token claim), the timestamp is fresh, and the signature verifies over a transcript naming this node, this group and this connection's nonce. A key that is merely well-formed proves nothing. Idempotent for the same key and refused for a different one: a connection does not get to change device half way through, which would let one session's uploads and messages be attributed to two. Sending a chat message **requires** this to have happened (§11.7), because the `device` field a receiver verifies a signature against is checked against the connection rather than believed. --- ## 10. Operator-authorized operations ### 10.1 Why a signature and not a token The hub issues JWTs, so a JWT can never establish node-level authority. Every destructive or privileged operation is authorized by an Ed25519 signature over a structured transcript, verified against the keys the node's roster records as holding operator authority — read fresh on every call, so revoking a paired browser takes effect immediately. ### 10.2 Two-hop exchange ``` C (operator) N | | |-- {op-specific fields} --------->| | | cheap pre-check: | | is there any key that | | could authorize this? | | (_has_admin_authority) |<- admin_challenge ----------------------------| | {op_id, op, subject, nonce, ts, | node keeps the authoritative | node_pk, group_id} | copy in _admin_ops[op_id] | | | the client REBUILDS the transcript from the announced FIELDS | and refuses to sign if `op`/`subject` are not what the user asked for | | |-- admin_response {op_id, signature, op} ----->| | | pop(op_id) - single use | | now - ts <= 120 s | | rebuild A from STORED state | | verify vs roster operator keys | | (file_delete also accepts the | | uploader's recorded key) | | execute via ops.py |<- _ack {op-specific fields} --------------| | | | some acks are ALSO broadcast to every peer in the group ``` ### 10.3 Transcript ``` A = "meshbay:admin:v1" || LP(op) e.g. "file_delete" || LP(node_pk_b64) so a signature for node A is invalid on node B || LP(group_id) so authority does not leak across groups on a multi-group node || LP(subject) what is being acted on || LP(nonce) 32 bytes, node CSPRNG, single use || LP(ts) unix seconds, TTL 120 s sig = Ed25519(sk_operator, A) ``` The structure is the whole point. A challenge of 32 raw random bytes, signed blind, would be an unbound signing oracle: the signed message would name no operation, no subject, no node and no time, so a signature obtained for one purpose would be structurally valid for any other, on any node, for ever. **The node never takes a signed value off the wire.** It rebuilds `A` from `_admin_ops[op_id]`; the client rebuilds it from the announced fields. They agree by producing the same bytes. ### 10.4 Operation catalogue `subject` is what the client must display and match before signing. Where the ack is broadcast, every connected peer in the group learns the change without reconnecting. | Op | Subject | Authority | Ack | Broadcast | |---|---|---|---|---| | `file_delete` | `file_id` | operator **or** the file's recorded `uploader_pk` | `file_delete_ack{file_id}` | no | | `dir_delete` | path relative to the root | operator | `dir_delete_ack{dir}` | no | | `invite_create` | invitee `user_id` | operator only (delegation designed, deferred) | `invite_result{code, expires_at, user_id, username}` | no — the code is shown once | | `invite_link_create` | `link:`, the session's group | operator only | `invite_link_result{code, invite_id, expires_at, group_id}` | no — the code is shown once | | `invite_cancel` | `invite_id` (32 hex) | operator only | `ack{detail: "invite_cancelled", invite_id}` | no | | `member_revoke` | `user_id` | operator | `member_revoke_ack` | no | | `member_unpin` | `user_id` | operator | `member_unpin_ack{user_id}` | no | | `gek_rotate` | `group_id` | operator | `gek_rotate_ack{group_id, authorized_members, note}` | no | | `apps_enabled` | the app set | operator | `apps_enabled_ack{apps}` | yes | | `set_scan_settings` | the interval/debounce pair | operator | `set_scan_settings_ack{...}` | yes | | `tmdb_config` | `custom_token=yes\|no,language=...` | operator | `tmdb_config_ack{token_customized, language}` | yes (never the token) | | `tmdb_enabled` | `enabled` | operator | `tmdb_enabled_ack{enabled}` | yes | | `tmdb_override` | `file_id=..,tmdb_id=..,media_type=..` | operator | `tmdb_override_ack{file_id, tmdb_id, media_type}` | yes | | `tmdb_rematch` | `file_id=..` | operator | `tmdb_rematch_ack{file_id}` | yes | | `musicbrainz_enabled` | `enabled` | operator | `musicbrainz_enabled_ack{enabled}` | yes | | `root_add`, `root_remove` | the root | operator | `root_add_ack` / `root_remove_ack` | no | | `root_update` | `:rw=on\|off,rem=on\|off` | operator | `root_update_ack` | yes | | `root_eject`, `root_plug` | the root name | operator | `root_eject_ack` / `root_plug_ack` | yes | | `app_directories` | `:,,...` | operator | `app_directories_ack{app, dirs}` | yes | | `chat_directory` | the path | operator | `chat_directory_ack{path}` | yes | | `chat_link_preview` | `on\|off` | operator | `chat_link_preview_ack{enabled}` | yes | | `search_listed` | `on\|off` | operator | `search_listed_ack{listed}` | yes | | `transfer_limits` | `d=,u=` | operator | `transfer_limits_ack{limits}` | yes | | `chat_epoch` | `group_id` | operator | `chat_epoch_ack{epoch}` | yes | | `group_attach`, `group_detach` | `group_id` | operator | `group_attach_ack` / `group_detach_ack` | no | **Upload policy is not in this table**, and that is the design: whether a member may write is a property of each root (`root_update`), not a switch over the group. A single group-wide flag cannot express "this library is published read-only and that folder is a drop box", which is the ordinary arrangement. `app_directories` is the **only** way an application's folders are set: one message for every application, keyed by the app's own registry name, so adding an application adds no message type, no signed op and no handler. The three narrower ops it replaced — `video_root`, `audio_root`, `photo_roots` — are gone from the catalogue. They were the same instruction three times, differing only in the key they wrote and whether they carried a string or a list, and that shape is what made adding an application mean adding a message type, an op, a handler and a widget. It also meant three validation paths, and the older ones validated nothing: a typo was stored, matched no entry, and the application showed an empty tab with no way to tell "misconfigured" from "no files yet". One op has one validation path, and an unknown application name is refused rather than stored. Their *storage* keys survive on the node — `Roster.LEGACY_DIR_KEYS` still reads `video_root` and friends out of `group_settings` — because that is a key on an operator's disk rather than on the wire, and a node upgraded into this has to find its own configuration. A second family of operator messages is **not** signed: `node_status`, `roster_read`, `denylist_read`, `denylist_clear`, `node_settings_set`, `node_reload`. These are gated by `is_node_admin()` — the authenticated session's `user_id` equals the account the node records as its own operator (`node_user_id`), computed from the node's own state and never from a hub claim. Three of them only read; the other three run through the same `ops.py` entry points as the CLI and the loopback admin API. The distinction from the signed table above is deliberate but worth stating plainly: a signed op proves possession of an operator *key*, while these prove only that the session belongs to the operator's *account*, which the handshake already established. Rules that hold across the table: * **No MNP message can activate a group key.** The rule (I2) targets key material arriving from outside, not the instruction: `gek_rotate` and `chat_epoch` are allowed precisely because the node generates the new key itself with its own CSPRNG. Initial `gek-init` stays local — with no group key there is no completed session to carry a signed op anyway. * **There is no operation by which key material reaches the node** (§12). The node wraps for a key the recipient has proved possession of, so no such message is needed — and a path that does not exist cannot be mis-authorized, which is I10 applied to a message instead of a version. * An operator cannot revoke or unpin **themselves** over the connection their pin authorizes. * Rotation is what actually removes a revoked member's access. Revocation stops the node serving the *next* key; the ex-member still holds the current one, and content they already downloaded stays readable. The ack says so in words. ### 10.5 One implementation, several front doors The loopback admin API, the CLI and the signed MNP handlers are three thin adapters over the same functions in `meshbay_node/ops.py`. Those functions take the daemon state, raise `OpError`, and know nothing about HTTP. One operation with two implementations means two authorization checks, and the weaker one is the one that decides. A front door is allowed to differ in how it *authenticates* — a signature here, a run token on loopback, an operator's shell for the CLI — and never in what it *does*. --- ## 11. Content plane Everything in this section requires an authenticated session (§4). All content is encrypted under keys derived from the GEK, so a node that serves a chunk to a session that never proved GEK possession serves ciphertext nobody can open — but the authorization check comes first regardless. ### 11.1 Index The Mesh Group Index is the list of files the node shares for one group. Entries are content-addressed: `id` is the BLAKE3 hash of the file. ``` C N |-- index_sync {v} ------------------------->| |<- index_sync {v, group_id, nonce, ct} -----| ct = seal(..., "index_sync", ...) | payload: {version, entries[], | | dirs[], roots[]} | | | ... operator drops files into a watched folder ... | |<- index_delta {v, group_id, nonce, ct} ----| pushed, unsolicited, to every | payload: {base_version, version, | peer in this group | additions[], deletions[], | | updates[], roots[]} | | |<- index_progress {v, group_id, scanning, | every ~2 s while scanning, | scanned_bytes, total_bytes} -----------| plus once on the return to idle NOT sealed — see below ``` `IndexEntry` wire fields (`index_entry_wire`): | Field | Meaning | |---|---| | `id` | BLAKE3 of the file content, hex | | `name`, `path` | filename, and the virtual directory it lives in (`/`) | | `size`, `type`, `added_at` | bytes; `video\|audio\|image\|document\|archive\|other`; unix seconds | | `duration`, `width`, `height`, `thumb_hash` | filled asynchronously by enrichment | | `uploader_id` | who uploaded it; `null` for content pre-existing on disk | | `display_title`, `season`, `episode` | Videos app, parsed from the name/folder | | `artist`, `album`, `track_no` | Music app, from tags or parsed | | `taken_at`, `camera` | Photos app, best-effort from EXIF | `uploader_pk` exists on the dataclass but is **not** in the wire dict: it is the key the node recorded at upload time, used server-side to authorize `file_delete`. Three lists, not two: `updates` carries entries whose `id` is unchanged (same content) but whose fields changed — enrichment filling in `duration`/`thumb_hash` after the file was first indexed with hash and size only. `diff()` only places an id there once it has appeared unchanged in a prior snapshot. `dirs` and `roots` exist because directories are not index entries. Without them a folder just created, or one emptied, does not exist as far as the UI is concerned, and a member cannot tell "the drive is unplugged" from "it is all still there" — an unavailable root is listed, with its content frozen rather than hidden. Each root is described as `{name, kind, available, writable, removable, ejected}` — and never a path: a member is told what exists and whether it is readable, never where on the operator's disk it lives. `roots` rides on `index_delta` as well as `index_sync`, because a full index is only ever sent on request: without it, a root added, removed, ejected or plugged would leave every connected client's directory table stale until somebody reloaded the page, and the delta that tells them something changed would be the one message unable to say what. `index_progress` carries counters only, never a path or filename. ### 11.1a The sealed envelope A family of messages travels sealed under a subkey derived from the GEK (`meshbay_common/groupbox.py`, mirrored by `sealGroup`/`openGroup` in `crypto.js`). There is one subkey per **purpose**, and five purposes: ``` key(purpose) = HKDF-SHA256(GEK, salt = , info = , 32 bytes) nonce = 12 random bytes, per message ct = AES-256-GCM(key).encrypt(nonce, msgpack(payload), aad) aad = "|" UTF-8 ``` | Purpose | `info` | Seals | |---|---|---| | `index` | `meshbay:index:v1` | `index_sync`, `index_delta` | | `ack` | `meshbay:ack:v1` | the `handshake_ack` configuration payload | | `upload` | `meshbay:upload:v1` | `file_upload`, `file_upload_ack` (§11.4) | | `chat_keys` | `meshbay:chat_keys:v1` | `chat_keys_resp` — the group's chat epoch keys (§11.7) | | `roster` | `meshbay:roster:v1` | `group_roster_resp` — members, device keys and the evidence that admitted each (§11.7) | `chat_keys` is the one whose payload *is* key material: a peer that has completed the handshake holds the group key and can open it, and anything short of that gets a ciphertext. That is the same statement the index makes, one step stronger. `salt = ` is Python's `salt=None` and WebCrypto's `salt: new Uint8Array(0)`; RFC 5869 extracts with a zero key either way. The subkeys are purpose-separated rather than borrowed from a file's key space — `GroupIndex.serialize()` reuses `chunk_key_aes` with a pseudo-file ("the index as chunk 0 of a virtual index file"), which is a hack this deliberately does not repeat. **What stays in clear, and why each one has to:** | Field | Why | |---|---| | `type`, `v` | the receiver must route and version-check before it can decrypt | | `group_id` | already in clear in the handshake; it is the AAD and selects the key | | `node_pk`, `proof`, `sig` (ack) | they *are* the authentication — verified before a decryption is trusted (§6.1 step 12) | | `index_progress`, in full | counters only, never a path or a filename, pushed every ~2 s for the whole length of a scan. Sealing it would buy a rough library size and cost a decrypt per push | | `upload_id`, `chunk_index` (upload) | the node routes and orders on them before it can decrypt; `upload_id` is client-drawn, opaque, and never an authorization input | | `transfer_open` / `_close` / `_state`, in full, and `tr` wherever it rides (§11.2) | `tr` is opaque and client-drawn, `bytes` and `chunks` are numbers, and there is no filename and no path anywhere in them. Adding one to make a log line prettier is exactly the trade this envelope exists to refuse | `version` and `base_version` are **inside** the payload: there is no reason to act on a version number carried by a message that has not been authenticated. The AAD binds a ciphertext to its message type and its group, so an `index_sync` body cannot be replayed as an `index_delta`, nor moved between two groups on one node. **What this buys, and what it does not.** It buys integrity for the ack (§6.6), and, for the index, defence in depth against one specific class of bug: a peer that has not completed the handshake being served data anyway. That bug is an authorization mistake in one branch of one handler, it is easy to write, and it is invisible until somebody reads that branch. Sealed, it leaks ciphertext instead of filenames and folder names. The upload is the same argument in the other direction. It seals *towards* the node, which holds the group key for its own group and opens the payload before it decides a destination or touches the disk — so the write path is covered by the same key as the read path, and a file is never plaintext on one leg of its journey and ciphertext on the other. It buys nothing against a network observer — DTLS/TLS already covers that — nothing against the hub, which never sees channel traffic, nothing against a member, who holds the group key, and nothing at rest: the index stays plain in the node's memory and the files stay plain on the operator's disk, which is the design. Chat messages are **not** sealed by this envelope: they have their own key hierarchy, per epoch and per device, because the node must be able to relay and archive a message it cannot read (§11.7). The control plane is not covered either — see §14.2. **Nonce collision, since `upload` is the first purpose with volume.** One subkey per purpose and a fresh 96-bit random nonce per message: at one message per 48 KiB chunk, 2³² chunks is 200 TB uploaded under a single GEK before the collision probability reaches 2⁻³², and `gek_rotate` exists. Deriving the nonce from the payload instead would be worse, not better — two chunks of identical bytes are ordinary in a file. `GroupIndex.serialize()` is not a candidate for reuse here: it compresses with zstd, which no browser can decompress (`DecompressionStream` offers gzip and deflate only), so reusing it would mean shipping a WASM decoder to every client for no gain. **Failure is fatal, never degraded** (I8). A client that cannot open an index message ends the session naming the message type; it never reports an empty index, because "the group has no files" is a state a real group can be in. ### 11.2 Transfer leases A download is not a message. `file_req` asks for one chunk; a client fetching a 4 GB film sends four thousand of them, eight in flight at a time, and nothing in that says a transfer started or that it ended. Without an object standing for the transfer itself there is nothing to count, and therefore nothing an operator can cap. The **lease** is that object, and every transfer runs under one. ``` C N |-- transfer_open {v, tr, kind, bytes, chunks} ----->| kind: "download" | "upload" | | per-member cap first, | | then the node-wide pool |<- transfer_state {v, tr, state, kind, | | used, cap, node_used, node_cap, | | [ahead], [reason]} ----------------------------| state: granted | queued | | |-- file_req {..., tr} ----------------------------->| every chunk under this lease |-- file_upload {..., tr} -------------------------->| says the transfer is alive | | |<- transfer_state {tr, state: "granted"} -----------| pushed when a queued lease | | reaches the head | | |-- transfer_close {v, tr, reason} ----------------->| done | cancelled | paused |<- transfer_state {tr, state: "closed", reason} ----| ``` **Six properties carry the design, and each is a decision:** * **`tr` is drawn by the client**, 16 random bytes, exactly like `upload_id`. Re-opening with the same `tr` is **idempotent**, so a reconnect cannot charge a member twice for one transfer — and re-asking is how a client recovers a grant whose push was lost. * **A lease is scoped to the connection, never to the account.** It dies with the session, which is what makes the reclaim deterministic: a closed tab, a quit browser and a dropped network all arrive at the same teardown, and none of them needs a timer. * **A lease covers a job, not a file.** A directory downloaded as a zip is dozens of files and *one* lease. One per file would deadlock against the member's own cap: the job cannot finish until it holds them all, and it can never hold more than its cap — two, by default. * **Nothing is persisted.** A restart drops every session anyway, and a lease that outlived the process would be a slot nothing can release. * **Leases are counted, bytes are not.** What a slot protects is concurrency — open file handles, disk seeks, the channel buffer each transfer keeps full. * **Per-member first, then node-wide.** A member at their own cap queues behind their own transfers and never holds a node-wide slot a second member has none of. Reversed, whoever arrives first takes everything. **Caps.** Node-wide, 8 concurrent per kind by default; per member per group, 2 by default. A group with no value of its own gets the default, never "unlimited": reading an absent setting as no limit would leave the node-wide cap as the only control, which is the situation leases exist to end. The per-group value is a signed operator operation (`transfer_limits`, §10.4, bounded to 1–32; zero is refused, because a member who may not transfer at all is a member the operator revokes). The node-wide values are daemon settings. A member's own caps ride on the handshake ack so the interface can say "2 of your 2 slots are busy" rather than draw a spinner that explains nothing. **Queueing.** One FIFO per kind. `_pump` walks it in arrival order and **skips** a member who is at their own cap rather than stopping at them — granting strictly in order lets one member's limit stall every other member behind them. A queued lease is told how many are `ahead` of it. Beyond 32 queued per member the answer is `too_many_queued`, because an unbounded queue is how a node runs out of memory politely. `used` and `cap` on `transfer_state` are this member's own count and this member's own limit **in this group** — the same value `_has_room` enforces and the same one the handshake ack announces. Three readings of one number, and an interface that draws a different one from the node's is an interface that offers a slot the node will queue. **Reclaim.** The session teardown is the primary path and it is immediate. A sweeper runs every 15 s for whatever the teardown cannot see, and tells two failures apart: | Situation | Timer | What happens | |---|---|---| | Granted, never taken up | 30 s | Back to the tail of the queue, `reason: "not_taken_up"` — the client died between asking and starting | | Same, three times | — | Closed, `reason: "abandoned"`. Without the bound the requeue is a permanent cycle: revoked, put back, granted again because there is room, revoked 30 s later, for ever | | Granted, used, then silent | 120 s | Closed, `reason: "idle"`, and the peer is told, so its widget can offer a resume rather than sit on a lie | | Connection gone | none | Everything it held, at once | The sweeper belongs to the **node**, not to the session that opened the first transfer. Tying it to a session would kill it when that peer left, and every other peer's abandoned lease would then never be reclaimed. **A chunk request is what "alive" looks like.** `tr` rides on `file_req` and on `file_upload` for exactly this: it is the only signal the node has that a granted lease is being used. Without it the sweeper cannot tell a transfer running at 20 MB/s from a client that asked for a slot and vanished, and it revokes both. **Pausing is releasing.** A paused transfer holds nothing: the client closes the lease with `reason: "paused"` and keeps its own position, and resuming asks for a *new* lease and queues behind whatever is waiting now. The alternative — holding a slot while paused — is a member who pauses three downloads and blocks the group. **Refusals.** `bad_transfer_id` (no `tr`), `bad_transfer_size` (`bytes`/`chunks` not numbers), `bad_kind`, `too_many_queued`, and `not_your_transfer` — the last for opening or closing a `tr` another connection holds, which would otherwise be a denial of service one random id away. #### Reads that carry no lease Browsing a group is **never** subject to a transfer slot: not the poster grid, not the covers, not opening a photo or a PDF to look at it. A member must be able to browse a group that is at capacity exactly as they browse an idle one. Thumbnails, posters and cover art never reach the check at all — they resolve out of the node's own media cache. But "not leased" cannot mean "unbounded", or a client that simply omits `tr` transfers outside every cap and the caps are decoration. So a session may read **12 distinct files at once** without a lease; the thirteenth is refused with `transfer_required` and a sentence telling the person to download the file rather than preview it. An entry already being read is always admitted, whatever the count — refusing a chunk halfway through a photo because the limit moved is worse than never having admitted it. An entry is released when its last chunk goes out, or after 60 s of silence, because a viewer closed mid-file simply stops asking and says nothing. **The number is derived from what the client legitimately does**, and it has to be: the music player warms a read-ahead window of 5 tracks on Wi-Fi, so playing an album has six files in flight before anyone has done anything unusual. 12 is those six at their widest, two for a photo viewer and its own prefetch in the same session, and the rest as headroom for the next feature that reads ahead. A test derives the floor from the player's own constant, so raising the client's prefetch without raising this fails in CI rather than in front of a person. Generosity is cheap here and refusal is not: the cost of being too high is a client that could have been queued and was not, and the cost of being too low is a member told to download a track they are trying to play. Deliberately a count of files and not a byte budget: a RAW photo out of a camera is 60–80 MB and is browsing, a 40 MB archive is a download, and no size threshold separates them. What separates them is which function asked. #### What leases are not **A fairness control among cooperating clients**, in the company of `max_concurrent_streams` — not a defence against a member determined to saturate a node's disk. A client that lies, labelling a bulk download as a view, gets 12 files at a time instead of its member cap. That is the residual, it is bounded, it is audited, and the answer to the member behind it is `member revoke`, not a protocol rule. Stating it is the point: a control described as a security boundary will eventually be relied on as one. ### 11.3 File download ``` C N |-- file_req {v, file_id, chunk_index, [tr]} ------->| | | `tr` present: mark that lease | | alive (§11.2) | | index lookup; if the id is | | not a file, try the media | | cache (thumbnail/poster/ | | cover/transcode), sliced the | | same way — never leased | | `tr` absent: admit against the | | leaseless ceiling, else | | `transfer_required` | | backpressure: wait while | | bufferedAmount > 2 MiB |<- file_chunk {v, file_id, chunk_index, | | plaintext_size, nonce, ct} --------------------| ``` The client pipelines 8 chunk requests at a time and reassembles in order; a chunk that fails is retried 6 times, 1.5 s apart, because a DataChannel that hiccups mid-film should cost a pause and not the whole transfer. Chunk encryption: ``` chunk_key = HKDF-SHA256(GEK, salt = none, len 32, info = "file:" || file_hash || ":chunk:" || uint32be(chunk_index) || ":aes") nonce = 12 random bytes ct = AES-256-GCM(chunk_key).encrypt(nonce, plaintext) no AAD ``` * The `:aes` suffix keeps AES keys distinct from the ChaCha20 variant (`chunk_key`/`encrypt_chunk`, `info` without the suffix) derived from the same GEK. * `nonce` and `ct` are msgpack **binary**, not base64. Every message that carries content carries it this way, and none carries it outside an AEAD. * The key is a pure function of (GEK, file hash, index), so chunks are cacheable, resumable and requestable out of order. This is what makes a download resumable at all: a client that stopped at chunk 900 asks for 900 next time, and no state on the node was keeping its place. `file_id` and `chunk_index` ride on the response because a client running several downloads at once cannot otherwise tell whose reply arrived. * Requests are handled off the message loop: the reply may wait for room on the channel, and blocking the loop for that would stall the very uploads whose acks free the buffer being waited on. * **One encoder, every transport** (`protocol.file_chunk_wire` / `file_chunk_plaintext`). A message type with one encoder per transport is a message type free to drift, and its name then says nothing about which shape will arrive. * **There is no per-chunk signature, and none is needed**: the AEAD tag authenticates the ciphertext under a key only group members hold, and the node authenticates itself once, in the handshake, rather than once per megabyte. * **A chunk that is not this shape aborts the download**, with an error. There is no fallback that decodes it some other way: a client that guesses at a chunk it does not recognise writes its guess into the file the person is saving. ### 11.4 Upload ``` C N |-- transfer_open {tr, kind: "upload", ...} -------->| a slot, like a download |<- transfer_state {tr, state: "granted"} -----------| (§11.2) | | |-- file_upload {v, upload_id, chunk_index: -1, | the probe: "where am I?" | total_chunks, tr, nonce, ct} ----------------->| writes nothing, reserves | ct = seal(upload, {filename, dir, root, | nothing | data: b""}) | |<- file_upload_ack {v, upload_id, chunk_index: -1, | | nonce, ct} ------------------------------------| | ct = seal(upload, {filename, stored_as, dir, | | resume_from}) | | | |-- file_upload {v, upload_id, chunk_index, | 48 KiB chunks, window 32 | total_chunks, tr, nonce, ct} ----------------->| | ct = seal(upload, {filename, data, | open under the group key, | dir, root}) | or refuse (upload_not_sealed) | | filename allowlist | | root writable and available | | destination resolves in-group | | chunk_index == next expected | | running total <= max_upload_gb | | append to .part |<- file_upload_ack {v, upload_id, chunk_index, | | nonce, ct} ------------------------------------| | ct = seal(upload, {filename, stored_as, dir}) | | ... repeat ... | | | last chunk: rename .part -> | | final, tag the index entry | | with uploader_id/uploader_pk ``` * **Sealed, both halves.** The filename, the destination and the bytes travel inside the seal; only `upload_id`, `chunk_index`, `total_chunks` and `tr` stay in clear, because the node routes, orders and accounts on them before it can decrypt anything. This direction seals *towards* the node — it holds the group key for its own group — which is the mirror image of the index, and it means a refusal cannot quote back what it just refused. * **`filename`, `dir` and `root` are repeated on every chunk**, not sent once in a header. A hundred bytes against a 48 KiB chunk, against the alternative: a header that arrives once is state the node has to carry, and upload state that can disagree with the chunk in hand is the thing the chunk-ordering rule and the free-name rule exist to prevent. * **`upload_id` replaces `filename` as the correlation key.** It has to: matching an ack to a request by name would hand back exactly what the seal is for. It is client-drawn, opaque to the node, unique within one connection, and never an authorization input. * **`group_id` is deliberately not on the message.** The session already decided which group it is on, and the node uses that as the AAD. A client naming its own group here would be choosing which key its bytes are checked against. * **A chunk that does not open is refused** with `upload_not_sealed`, and nothing is written. One answer covers "not sealed at all" and "sealed wrong": distinguishing them tells a peer which of the two it got right. There is no plaintext fallback — a path that still accepts plaintext is not a sealed path. **Resuming, and the probe chunk.** The node identifies an upload by `(member, directory, filename)`, so a client resuming one has to name the file. `transfer_open` is the obvious place to ask and it travels in clear, which would undo precisely what sealing this path bought. So the question is asked **inside the seal that already exists**: an ordinary `file_upload` with no bytes and `chunk_index = -1` (`UPLOAD_PROBE_INDEX`). Every check below has already run by then, so it cannot be used to ask questions about a directory the caller may not write to; the node writes nothing, reserves no name, and answers `resume_from` — how many chunks of this file it already holds — inside the seal, because that is a fact about the operator's disk. `resume_from` is absent from an ordinary ack, so the two are told apart without looking at `chunk_index`. `stored_as` on a probe answer is only what is *really* on disk, and empty when there is nothing: reporting the free name the node would pick would promise a destination that the real chunk 0 may not choose. A node that does not understand the index refuses it, which a client reads as "start from the beginning"; the client also bounds its wait at 5 s, so a node that answers neither the probe nor its refusal costs one restart rather than a stuck upload. Starting over is always safe, which is what makes both fallbacks available. **The position outlives the connection.** Upload state is held by the **group**, keyed by `(user_id, rel_dir, filename)`, not by the session: state on the session dies with it, and an upload interrupted at 99% would then have to start again from zero — on a connection flaky enough to have interrupted it once. Keyed by member as well as by name, because a shared directory means two people can be sending `IMG_1234.jpg` at the same moment and neither may inherit the other's position. **Pausing an upload is the same shape as pausing a download** (§11.2): the lease goes back, and the position does not have to be remembered accurately, because the node holds it and the probe asks for it on the way back in. A pause is taken **between two chunks, never inside one** — the node refuses a chunk that is not the one it expects, so a chunk boundary is the only position worth having. The client also throttles on its own send buffer (1 MiB), or the whole file lands in it in seconds and the progress bar becomes a work of fiction. **An abandoned `.part` is reaped.** It is otherwise a gigabyte of somebody else's disk that nothing will ever finish, delete or look at again — invisible in the index, because a `.part` is not an index entry. One with no upload behind it is deleted after 24 h. Generous on purpose: the cost of waiting is disk, and the cost of being wrong is deleting an upload somebody is still making, which is unrecoverable and looks to them like a transfer that failed for no reason. A day covers a laptop closed overnight, a phone in a tunnel, and a client that resumes on its next launch. * **Confinement.** `filename` must match the name allowlist (`SAFE_UPLOAD_NAME`); the resolved path must sit under a shared root. This is not hypothetical tidiness — a filename rendered into the interface is stored XSS if it is allowed to contain markup, and it reaches every member of the group. * **No overwrite, ever.** A colliding name is given a free one and the uploader is told what it became in `stored_as`; the client must use that value when referencing the file (a chat attachment, for example). Without the rule any member could replace any shared file by uploading one with the same name — silently, and with the index still pointing at what that name meant before. * **Ordering.** Out-of-order or replayed chunks are refused — otherwise a chunk with index > 0 appends blindly to whatever `.part` is on disk. * **Types are checked after the seal opens.** What comes out of an AEAD is *authenticated*, not *validated*: it is msgpack a member wrote, and a `filename` that is a number raises where a refusal was meant. `data` must be binary: no sealed message can carry a string there, so a string is a peer doing something else entirely. * **Destination.** The client names the folder it is browsing, never a path: the node resolves it against the group's own roots, which refuses `..`, absolute segments and anything escaping its root, symlinks included. With several roots, the node picking one would send a member's file to a disk the operator did not intend, and that is discovered weeks later. An unknown root name is refused rather than falling back to a writable one, for the same reason. An unavailable or read-only root is a refusal, not a fallback. * **No quarantine subdirectory.** The file lands in the folder the sender is looking at, not in an `uploads/` folder of the node's invention: a shared directory nobody can organise is not a shared directory, and a folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. What confines an upload is the allowlist, the size cap, the chunk ordering and the no-overwrite rule — never a subdirectory. * **Policy.** Writability is a property of the root, and it binds the operator too — "read-only for everyone" is what makes a published library one. Enforced here rather than by hiding a button: the button is a courtesy to people who are not trying. * The uploader key recorded is the one **the node pinned** for the device on this connection (§9.4), not one the token carried — which is what makes `file_delete` authorizable by the uploader without letting the hub delete anyone's files. ### 11.5 Directories and deletion ``` C -> N dir_create {name, dir} any member; allowlisted, confined, audited N -> C dir_create_ack {dir} C -> N dir_delete {dir} operator, signed (section 10) C -> N file_delete {file_id} operator or uploader, signed ``` Creating a directory is not privileged — a member who can add a file may organise where it goes — but it writes to the operator's disk, so it is audited like one and it obeys the same per-root policy as an upload: the parent's root must be writable (`root_read_only`) and available (`root_unavailable`). Read-only means read-only, and a member who cannot add a file to a published library must not be able to leave empty folders in it either. A directory cannot be created at the virtual root: that would be adding a root, which is operator configuration rather than a file operation. Deleting a root by name through `dir_delete` is likewise refused. `dir_delete` removes an **empty** directory and nothing else: it is never recursive, so whatever the caller intended it cannot destroy content. The operator deletes the files first, and sees what they are losing. The emptiness check runs before the challenge is issued, so a non-empty directory never produces a signable transcript. ### 11.6 Video streaming (MSE) Segments are produced by ffmpeg as fragmented MP4 and pushed under client-granted credit. Without the credit scheme the node hands ffmpeg's whole output to the channel as fast as it is produced, and the browser holds a multi-gigabyte film in a JavaScript array while MediaSource consumes it a segment at a time. ``` C N |-- stream_req {v, file_id, start, credits, | | audio_track?} -------------------------------->| | | retire this session's previous | | stream (a second request | | means the first is over) | | acquire a transcode slot (8) | | ffprobe: codec, duration, the | | audio tracks | | spawn ffmpeg | | -ss before -i (index seek), | | -noaccurate_seek when the | | video is copied | | video: copy, or libx264 when | | the browser cannot decode | | audio: always AAC, 2 ch, | | -map 0:a: | | frag_keyframe+empty_moov |<- stream_init {v, file_id, codec, duration, start, | | audio_tracks[], audio_track, | | subtitle_tracks[]} ----------------------------| | | | check MediaSource.isTypeSupported(codec) | | | |<- stream_data {v, file_id, segment_index, | 256 KiB, encrypted with the | nonce, ct, plaintext_size} --------------------| same per-chunk derivation as | ... x credits ... | a file chunk, index = segment | | |-- stream_more {v, n} ----------------------------->| n > 0 grants; n == 0 is a | ... continues ... | keepalive, not a no-op | | |-- stream_stop {v} -------------------------------->| viewer closed |<- stream_end {v, file_id} ------------------------ | natural end of file ``` | Rule | Value / behaviour | |---|---| | Credit granted per `stream_more` | clamped to `[0, 256]` | | Client default window | 24 segments (6 MiB) | | A client that sends no `credits` | unpaced — the node streams as fast as it can, and the client is responsible for what it buffers | | Silence timeout | 120 s since the peer last said anything, polled every 3 s | | `n == 0` | keepalive: a viewer buffered 90 s ahead grants nothing and must still be able to say it is there | | Concurrent transcodes | 8 node-wide, semaphore on the transport context | | Seeking | a new `stream_req` with `start`; the previous stream is retired first, ffmpeg respawned with `-ss` | | Accurate seek | **off when the video is copied, on when it is re-encoded.** Copied video has to begin on a keyframe and cannot be trimmed to the request; re-encoded audio can, and is. Leaving both at the default put a whole GOP of silence at the head of every seek and left sound and picture a GOP apart — with correct timestamps throughout, so nothing downstream could detect it | | `start` in `stream_init` | the value actually used. On the copy path it is **measured** before the stream is served — the same seek with the same stream mapping, one frame copied under `-copyts`, 0.06–0.07 s — because an index seek lands on an indexed keyframe that the frame list cannot predict and that moves with which audio track is mapped. The client adds it back as `SourceBuffer.timestampOffset`; a subtitle cue carries the source's absolute time, so reporting anything else puts every line on screen away from the voice | | `audio_tracks` in `stream_init` | every audio track: `i` (the **audio ordinal**, what `-map 0:a:` takes, never the container stream index), `lang`, `title`, `codec`, `ch`. Empty for a file with no audio | | `audio_track` in `stream_req` | which ordinal to map. Absent, out of range or malformed is the first track | | `audio_track` in `stream_init` | the ordinal actually used, for the same reason `start` is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. `null` when the file has no audio | | Changing track | a new `stream_req` at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream | | Capability discovery | **the list, not the version.** A client draws its selector from `audio_tracks` and sends `audio_track` only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language | | `subtitle_tracks` in `stream_init` | the subtitle tracks that can be shown: `i` (the **subtitle ordinal**, what `-map 0:s:` takes, counted over *every* subtitle stream including the ones absent from this list), `lang`, `title`, `codec`. Empty for a file with no convertible subtitles | | Which subtitle tracks are listed | text codecs only (subrip, ass, mov_text, …). Bitmap streams (PGS, VOBSUB — about a fifth of a real library) have no WebVTT without OCR, and one extracted anyway yields a header with no cues: a track that appears in the menu and shows nothing. A file whose subtitles are all bitmap reports none, exactly like a file with none | | Why the ordinal is not the list position | the two differ whenever a bitmap stream precedes a text one. Renumbering the survivors would map `0:s:0` to the stream that cannot be decoded — which is an empty WebVTT, not an error | | Fetching a track | `subtitle_req {file_id, track}` → `subtitle_resp {file_id, track, hash, size, mime}`; the blob is pulled by `hash` over `file_req`, the same indirection as a poster or an audio transcode. Extracted whole-file, converted to WebVTT, cached under the file's own id — so a film is extracted once, not once per viewing | | Subtitles and seeking | nothing. The cues carry the source's absolute timestamps, so a seek and an audio-language change both leave the client's `` untouched | An ffmpeg failure before any output produces `error: Could not stream this file`; stderr stays server-side, where it belongs — it names paths on the operator's disk and the operator's ffmpeg build, to somebody who asked to watch a film. Segments are encrypted with the same per-chunk derivation as a file chunk, the segment index standing in for the chunk index. There is no unencrypted streaming path: a segment of a film is content, and content does not leave a node outside an AEAD. ### 11.7 Chat Chat is encrypted on the wire and at rest, under a key hierarchy of its own — not the sealed envelope of §11.1a, because the node has to relay and archive a message it cannot read, and receivers include devices that were not connected when it was sent. ``` C N other peers |-- device_hello {...} ----------------------------->| §9.4 — required before a send | | |-- chat_keys_req {v} ------------------------------>| |<- chat_keys_resp {v, group_id, nonce, ct} ---------| sealed, purpose "chat_keys" | payload: {epochs: [{epoch, key}], current} | EVERY live epoch, not just now | | |-- chat_msg {v, format: 1, epoch, device, | | nonce, ct, sig, sender_name, | | [thread_id], [iteration]} -------------------->| | | envelope checks (below) | | persist to THIS group's store | | broadcast ------------------->| | | {chat_msg, sender_id, | | sender_name, format, epoch, | | device, nonce, ct, sig, | | thread_id, timestamp} | | notify the hub: group id and | | sender id only |<- ack {v} -----------------------------------------| | |-- chat_hist {v, [before], [limit <= 200]} -------->| |<- chat_hist_resp {v, has_more, messages: [{id, | | sender_id, sender_name, timestamp, thread_id, | | format, epoch, device, nonce, ct, sig}]} ------| | |<= chat_epoch_ack {v, epoch} =======================| pushed: a new epoch is open ``` **The key hierarchy.** ``` epoch_key 32 bytes, generated by the NODE, one per group per epoch device_key = HKDF-SHA256(epoch_key, salt = , len 32, info = "meshbay:chat:dev:v1||") nonce = 12 random bytes, per message ct = AES-256-GCM(device_key).encrypt(nonce, msgpack(payload), aad) aad = "chat_msg||" sig = Ed25519(sk_device, "meshbay:chat:v1" || LP(group_id) || LP(epoch) || LP(device) || LP(nonce) || LP(ct)) ``` **Why a key per device and not a ratchet.** A ratchet cannot deliver forward secrecy in this setting, and it is better to say so than to appear to have it. The node serves history to devices that were not present when a message was written, so it must retain and hand out each chain's *earliest* key — and a chain key at iteration *i* yields every message key from *i* onward by pure HKDF. Forward secrecy is then zero, and the ratchet is computing HKDF over a value every member already holds. A Signal-style sender-key implementation and a Double Ratchet were both written for this and never called by production; both have been **deleted**, because code nothing calls reads as an alternative somebody may reach for and its passing tests read as evidence of a protection that is not in the product. Deriving per device by name buys what the ratchet was there for and one thing more: **there is no mutable sending state at all**, so nothing can be advanced twice. Two devices advancing one chain produce key and nonce reuse, which is the failure this design cannot have. Be precise about what that does *not* say: two clients of one account normally hold the **same** identity key — a second browser recovers it from the keypair bundle rather than minting a new one — so they share a device key and therefore this subkey. That is safe here only because the nonce is 96 random bits and not a counter: two independent senders under one key collide on the birthday bound, which at chat volume is unreachable, whereas two independent senders advancing one counter collide immediately. The design degrades correctly into the deployment that exists; a chain-based one would not have. **Epochs.** A new epoch is opened when the set of devices that may read *future* messages shrinks — member revoke, member unpin, device revoke, `gek_rotate` — and by hand with the signed `chat_epoch` op (§10.4). Old epochs are kept and still delivered to current members, which is what keeps history readable to the people who could already read it. The epoch key is wrapped under the group key **at delivery**, never stored under it, so rotating the group key is a re-wrap and costs nothing. **Signing is separate from encryption, and it is what establishes who said something.** The signature is over the *ciphertext*, so it can be checked before decryption and by anyone holding the roster — including on a stored row, without the epoch key. It names the device key **the node pinned**, never a key the sender presents alongside the message: a signature verified against a key from the same message proves only that its sender owns some key, which any member can arrange. One thing is deliberately **not** in the signing transcript: this connection's nonce. Every other transcript in this protocol binds one; this one cannot, because a receiver reading history has no access to the connection a message arrived on. Replay is therefore refused by storage instead, on the unique `(device, nonce)` pair. A replayed message is a validly signed copy of a real one, so nothing about the signature refuses it; it is dropped and logged rather than raised at the sender, because the message it duplicates is already stored and there is nothing for anyone to retry. **What the node checks before it stores or relays anything:** | Rule | Why | |---|---| | `format` is the sealed one | Plaintext is refused, always — not "accepted and marked", and not "unless a switch says otherwise". A member who can post in clear into a group whose members believe their chat is encrypted is a downgrade, and every peer that reaches this point is able to seal | | `device`, `nonce`, `sig` present, right lengths, `ct` non-empty | a malformed envelope is refused before storage, not stored and puzzled over later | | the connection has identified its device (§9.4) | the `device` field is what receivers verify against | | `device` == this connection's pinned device | **a device may only send as itself.** A member free to name another member's key could *be* that member to everyone, and the signature would verify | | `sender_id` is taken from the authenticated session | never read from the message; it never was | `format` is a *storage* state as well as a wire one, so the store can distinguish rows the wire would not accept. Nothing changes what is accepted from a peer: the sealed form, or a refusal. **Who can verify what.** `group_roster_req` answers **any member** — not only the operator — with each device key in the group, which already-pinned key countersigned it, and the signature, nonce and timestamp needed to rebuild what was signed. That is what lets a member check for themselves that a message came from a device belonging to the account it claims, instead of taking the node's `sender_id` on trust. The reply is sealed under the `roster` purpose (§11.1a): it is the group's membership, and a peer that has not completed the handshake has no business reading it. The node hands over evidence and decides nothing; a node that lies here is caught by a client that has seen the account before. **Other rules.** * The store, the peer registry and the epoch keys are resolved **per group**. Reading them off the shared transport context sent every group's messages to the first group's database and served them back to anyone on the node. * The broadcast excludes **this connection**, not this account. The sender's other devices are ordinary recipients: they did not compose the message and have no local echo of it, so skipping them by `user_id` left a person's second device silently missing everything they said from the first. * The hub is told a message exists — group id and sender id, nothing else. No display name: the body is unreadable to the hub, and shipping the author's name beside it would leave the hub a per-message record of who spoke where, which is the metadata this is otherwise about not producing. The sender id stays because the hub needs it in order not to notify the author of their own message. * `before` pages backwards from the newest, which is the direction a chat is read; `has_more` is asked about the oldest row returned, so an empty page correctly says no. * A message that does not open is shown as unreadable, never as blank. Rendering it empty would make a message nobody can read indistinguishable from a message nobody wrote. * The boundary, stated as everywhere else: the node operator and every current member hold the group key and therefore the epoch keys. This is the same boundary as file access, by design. What it protects against is someone who obtains the node's storage without the keystore password. ### 11.8 Link unfurl ``` C -> N link_preview_req {v, url} N -> C link_preview_resp {v, url, ok, [title, description, site_name, image_thumb_hash]} ``` The node fetches the URL because the browser cannot (CSP and CORS) and doing so would leak every reader's IP to whatever was pasted. `ok: false` means "no preview" — blocked, unreachable, or not HTML — and the client shows the bare link. Any image is cached in the node's thumb store, so `image_thumb_hash` is fetched over the ordinary `file_req` path. Rate limits: 15 per connection and 60 node-wide per 60 s; results cached 1 h, 256 entries. ### 11.9 Metadata applications All of these are read-only lookups against third-party services, cached node-side and keyed by content hash. None is signed: they change nothing in the node's own state. The corresponding *settings* are signed operations (§10.4). | Request | Response | Notes | |---|---|---| | `media_meta_req {file_id}` | `media_meta_resp {file_id, tmdb_id, title, original_title, overview, poster_thumb_hash, backdrop_thumb_hash, release_date, first_air_date, genres, vote_average, runtime, cast, director, confidence, [season, episode]}` | `confidence: 0` means no confident match — the client falls back to a thumbnail-only card, it is not an error | | `season_meta_req {tmdb_id, season}` | `season_meta_resp {...}` | a show's single overview does not describe every season alike | | `tmdb_search_req {media_type, query}` | `tmdb_search_resp {results: [{id, title, year, poster}]}` | candidates for a human to pick from; never collapsed to one guess | | `music_meta_req {file_id}` | `music_meta_resp {file_id, ...}` | MusicBrainz; cover art cached like a poster | | `audio_transcode_req {file_id}` | `audio_transcode_resp {file_id, hash, size, mime}` | WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by `hash` over `file_req` | | `subtitle_req {file_id, track}` | `subtitle_resp {file_id, track, hash, size, mime}` | MSE decodes no in-band text track, so a subtitle travels beside the stream. `track` is the ordinal from `stream_init.subtitle_tracks` and is echoed back, because one film's two tracks are exactly the pair that can be in flight together. Fetch the result by `hash` over `file_req` | Every one of these is keyed by the entry's **`file_id`**, never by a path: a path names the folder a file is in, so two files in one folder — any multi-episode season — would resolve to whichever entry the index returned first. Posters, covers, thumbnails and transcode results all live in the media cache and are served through `file_req` by their hash, sliced into chunks exactly like a real file. ### 11.10 Liveness ``` C -> N ping {v, token} N -> C pong {v, token} the caller's token echoed back ``` Echoed rather than bare, so a client can match the answer to the probe it sent and measure a round trip instead of being reassured by a reply to an earlier one. This is **not** discovery: opening a connection in order to ping costs a full ICE/DTLS handshake, so presence in the group list comes from the hub's socket registry. --- ## 12. Message reference Direction is `C→N` (client to node), `N→C` (node to client, solicited) or `N⇒C` (node to client, unsolicited push/broadcast). "Stage" is the earliest session state in which the node accepts the message: **pre** = pre-proof window, **auth** = after the client's group-key proof, **signed** = after an `admin_response` verified against an operator key. Any request may carry `req_id`; every reply the node sends while answering one carries it back (§3.5). | Type | Dir | Stage | Purpose | |---|---|---|---| | `handshake` | C→N | — | open the session; version range, token, group, client nonce | | `handshake_challenge` | N→C | — | node nonce, node's version range, announced `node_pk`, and since 3.4 `sig` over the challenge transcript | | `handshake_response` | C→N | — | client's `HMAC(GEK, T("client"))` | | `handshake_ack` | N→C | — | node proof, node signature, session parameters **sealed** (§11.1a) | | `keypair_bundle_fetch` / `_resp` | C→N / N→C | pre | the caller's encrypted identity bundle | | `keypair_bundle_store` | C→N | auth | back up (or re-wrap) that bundle | | `keypair_bundle_delete` | C→N | auth | withdraw the backup | | `user_blob_store` | C→N | auth | write one per-account blob (playlists) | | `user_blob_fetch` / `user_blob_resp` | C→N / N→C | auth | read one, or `null` | | `user_blob_list` / `user_blob_list_resp` | C→N / N→C | auth | which kinds, at what revision — never a payload | | `user_blob_delete` | C→N | auth | drop one | | `gek_bundle_fetch` / `_resp` | C→N / N→C | pre | the caller's wrapped GEK | | `join_request` / `join_result` | C→N / N→C | pre | pin or recognise an identity; wrap the GEK | | `invite_create` / `invite_result` | C→N / N→C | signed | issue a one-time pairing code | | `invite_link_create` / `invite_link_result` | C→N / N→C | signed | issue a code bound to no account, for an invitation link (§8.6) | | `invite_cancel` | C→N | signed | take back an unredeemed invitation link | | `device_add_request` / `_ack` | C→N / N→C | auth | file a new device as pending | | `device_lookup` / `device_lookup_result` | C→N / N→C | auth | candidates for the approver to hash-match | | `device_add` / `device_add_ack` | C→N / N→C | auth | admit a device, countersigned | | `device_list` / `device_list_result` | C→N / N→C | auth | this account's devices | | `device_revoke` | C→N | auth | retire a device, countersigned (answers `device_add_ack`) | | `index_sync` | C→N, N⇒C | auth | full index — requested, or pushed on first change | | `index_delta` | N⇒C | auth | additions / deletions / updates against a base version | | `index_progress` | N⇒C | auth | scan counters, no paths | | `file_req` / `file_chunk` | C→N / N→C | auth | one encrypted chunk of a file, thumbnail or cache blob; optional `tr` names the lease it runs under | | `file_upload` / `file_upload_ack` | C→N / N→C | auth | push a chunk; **both halves sealed** — name, destination and bytes inside, `upload_id`, `chunk_index` and `tr` outside. `chunk_index = -1` is the resume probe (§11.4) | | `transfer_open` / `transfer_close` | C→N | auth | ask for a transfer slot / give it back | | `transfer_state` | N→C, N⇒C | auth | granted, queued (with `ahead`) or closed (with `reason`) | | `device_hello` / `_ack` | C→N / N→C | auth | which device of this account is on this connection (§9.4) | | `chat_keys_req` / `_resp` | C→N / N→C | auth | every live chat epoch key, **sealed** | | `group_roster_req` / `_resp` | C→N / N→C | auth | this group's members and device keys, with the evidence that admitted each, **sealed** | | `dir_create` / `dir_create_ack` | C→N / N→C | auth | create a folder | | `dir_delete` / `dir_delete_ack` | C→N / N→C | signed | remove an empty folder | | `file_delete` / `file_delete_ack` | C→N / N→C | signed | delete a file (operator or uploader) | | `stream_req` | C→N | auth | start or seek an MSE stream | | `stream_init` / `stream_data` / `stream_end` | N→C | auth | codec header, encrypted fMP4 segments, end | | `stream_more` / `stream_stop` | C→N | auth | grant credit / abandon the stream | | `chat_msg` | C→N, N⇒C | auth | send and fan out a message | | `chat_hist` / `chat_hist_resp` | C→N / N→C | auth | paged history | | `chat_attach` | — | auth | attachment metadata (declared, unused on the wire) | | `link_preview_req` / `_resp` | C→N / N→C | auth | OpenGraph unfurl | | `media_meta_req` / `_resp` | C→N / N→C | auth | TMDB metadata for one file | | `season_meta_req` / `_resp` | C→N / N→C | auth | per-season TMDB fields | | `tmdb_search_req` / `_resp` | C→N / N→C | auth | candidate matches for an operator | | `music_meta_req` / `_resp` | C→N / N→C | auth | MusicBrainz metadata for one file | | `audio_transcode_req` / `_resp` | C→N / N→C | auth | browser-playable copy of a WMA/MPC file | | `subtitle_req` / `_resp` | C→N / N→C | auth | one embedded subtitle track as WebVTT, by cache hash | | `ping` / `pong` | C→N / N→C | auth | liveness on an open channel | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | | `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | | `transfer_limits` / `_ack` | C→N / N⇒C | signed | per-member transfer caps for this group | | `chat_epoch` / `_ack` | C→N / N⇒C | signed | open a new chat epoch by hand | | `app_directories` / `_ack` | C→N / N⇒C | signed | one application's folders, keyed by app name | | `chat_directory` / `_ack` | C→N / N⇒C | signed | where chat attachments are written | | `chat_link_preview` / `_ack` | C→N / N⇒C | signed | whether the node unfurls posted links | | `search_listed` / `_ack` | C→N / N⇒C | signed | whether members' cross-group Search lists this group | | `root_update` / `_ack` | C→N / N⇒C | signed | a root's `writable` / `removable` flags | | `root_eject` / `_ack`, `root_plug` / `_ack` | C→N / N⇒C | signed | take a removable root offline, put it back | | `gek_rotate` / `_ack` | C→N / N→C | signed | node generates a new group key | | `apps_enabled` / `_ack` | C→N / N⇒C | signed | which group apps are shown | | `set_scan_settings` / `_ack` | C→N / N⇒C | signed | reconcile interval and debounce | | `tmdb_config` / `_ack` | C→N / N⇒C | signed | node-wide TMDB token and language | | `tmdb_enabled` / `_ack` | C→N / N⇒C | signed | per-group TMDB on/off | | `tmdb_override` / `_ack` | C→N / N⇒C | signed | correct a wrong automatic match | | `tmdb_rematch` / `_ack` | C→N / N⇒C | signed | drop one file's cached match | | `musicbrainz_enabled` / `_ack` | C→N / N⇒C | signed | per-group MusicBrainz on/off | | `root_add` / `_ack`, `root_remove` / `_ack` | C→N / N→C | signed | add or remove a shared directory | | `group_attach` / `_ack`, `group_detach` / `_ack` | C→N / N→C | signed | start or stop hosting a group | | `node_status` / `_ack` | C→N / N→C | auth (operator) | all groups, roots, daemon state | | `roster_read` / `_ack` | C→N / N→C | auth (operator) | pinned identities and members | | `denylist_read` / `_ack` | C→N / N→C | auth (operator) | current refusals | | `denylist_clear` / `_ack` | C→N / N→C | auth (operator) | remove entries | | `node_settings_set` / `_ack` | C→N / N→C | auth (operator) | change daemon settings | | `node_reload` / `_ack` | C→N / N→C | auth (operator) | re-read `node.toml` | | `ephemeral_stream` | — | — | reserved, mobile live push | | `error` | N→C | any | refusal, with `detail` and optionally `code`, `req_id`, and the `upload_id` / `tr` / `file_id` it is about | | `ack` | N→C | auth | generic acknowledgement (chat, keypair bundle store) | **Three messages do not exist, and their absence is a rule rather than an omission:** | Not a message | Why there is none | |---|---| | any request that returns the group key in plaintext | members obtain it by unwrapping their own ECIES bundle (§7.2); a node that can be asked for the key in clear is a node one authorization mistake away from handing it over | | any message by which a member stores key material on the node | the node wraps for a key the recipient has proved possession of (I2). A member-supplied bundle is a key of somebody else's choosing | | any content message outside an AEAD | a segment of a film and a chunk of a file are the same thing to everyone but the codec | A node that receives an unknown type logs it and does nothing. That is what makes the additive rule of §13 work. ### 12.1 Transport coverage **WebRTC implements this document.** It is what the browser SPA and the desktop client speak, and every message above is available on it. **QUIC is in development** (§5.2): a partial message set, no client, and not a shipped feature. Nothing about it is a compatibility commitment yet. Two rules hold across transports, and both are about there being exactly one of each message: * **One encoder per message type, shared by every transport.** `file_chunk` comes from `meshbay_common.protocol`, `index_sync` and `index_delta` from `meshbay_node/transport/wire.py`, and a parity test fails if a server grows a copy of its own. Two encoders for one type is a type free to drift, with a name that no longer says which shape will arrive — and, when one of them is a sealed envelope, a second construction site that goes on sending cleartext. * **`GroupIndex.serialize()` / `deserialize()` describes no MNP message.** It is a signed, compressed, encrypted at-rest and interchange format, and reading it as a wire contract is a mistake worth naming: the sealed envelope of §11.1a is what index messages travel under, and it is deliberately not this, because zstd decompresses in no browser. --- ## 13. Versioning and compatibility MNP versions independently of the package version. Current: **`3.4`**; oldest peer accepted: **`3.0`** — 3.1, 3.2, 3.3 and 3.4 are all additive, so the floor does not move with them. 3.4 adds the challenge signature (section 6.5) and invitation links (section 8.6): `invite_link_create`, `invite_link_result`, `invite_cancel`, which an older node answers as unknown messages. The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; `MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about whether an older peer can still do anything useful: * **Additive change → MINOR.** A change is additive only if all of these hold: no existing field changes meaning, type or encoding; a peer that ignores the new field or message still behaves correctly; and the new message is only sent to a peer that advertised support, or is harmless to drop. The floor does not move. * **Breaking change → MAJOR**, and then the question is *where to refuse*. A break confined to one exchange can be refused per message, with a code, leaving the rest of the protocol working — a peer that cannot upload can still browse, download, stream and chat. A break that touches something every session depends on cannot be confined, and the honest form is to refuse at the handshake: **a stated refusal is a bug report, a feature that quietly does not work is a support case.** * **Discovery from the answer, not from the version number.** 3.2's audio-track selection is the shape to copy: the node lists the tracks in `stream_init`, and the client sends `audio_track` only when it was given a list. A peer that ignores that field would not degrade — it would serve a different language in silence, which is a wrong answer and not a missing feature — and what keeps the change additive is that no client can ever put an old node in that position. This is not the opt-in switch I10 refuses: there is no second branch on the node, which always enumerates, always honours the request and always reports the track it used. * **A requirement is breaking even when its messages are additive** (I10). New message types and a new optional field are additive on the wire; *requiring* them is not, and an opt-in switch that enforces the requirement only for peers that speak the new version leaves the permissive branch reachable on every node. That is the branch that ends up being used. ### 13.1 Negotiation Both peers declare two values in the first message each sends — `handshake` from the client, `handshake_challenge` from the node: ``` v the version this build speaks MNP_VERSION v_min the oldest peer it will talk to MNP_MIN_SUPPORTED ``` Each side then checks the other, before anything else is decided: | Condition | Refusal `code` | |---|---| | peer's `v` < our `v_min` | `version_too_old` | | peer's `v_min` > our `v` | `version_too_new` | | `v` unparseable | `version_unreadable` | The refusal is shaped like `not_a_member`: a peer-safe sentence for a human and a `code` the client matches on, because matching on the text is a string comparison that breaks the day someone improves the wording. A peer that declares no `v_min` is read as accepting only what it speaks. Versions compare **numerically**, as `(major, minor)`: as strings `"0.9" > "0.15"`. **Refusing at the handshake is not enough on its own, for a client that ships its own interface.** The browser SPA is served by the hub and is therefore never out of step with it. An installed desktop client can be, and `version_too_old` is a refusal in a protocol vocabulary with nothing a person can act on. So the client asks `GET /v1/hub/version` for `client.minimum` **before it connects**, and says "this version can no longer connect" instead. An unreachable hub is deliberately *not* treated as too old: a captive portal or a closed laptop must not make starting the application impossible. **The version a peer announces is only as good as the number it ships with.** Every package in the tree carries one version, and a test fails if two disagree — a client announcing a number from a different scheme sorts wherever that scheme puts it, and walks through the gate meant to stop it. --- ## 14. Security properties and stated limits ### 14.1 What the protocol establishes | Property | Mechanism | |---|---| | A peer holds the group key | `HMAC(GEK, T("client"))` over a node-chosen nonce, bound to the channel | | The node holds the group key and is the one previously seen | `HMAC(GEK, T("node"))` over a client-chosen nonce, plus `Ed25519(sk_node, T)` and a per-node pin | | No MitM on the signaling path | both DTLS fingerprints (or the QUIC certificate hash) inside every transcript; empty binding is a refusal | | The hub cannot read content | the group key never reaches it; chunk keys and chat epoch keys derive from or are wrapped under it | | The hub cannot substitute a key at invite | the node wraps for a key its owner presented and signed, bound to an identity by a code the hub never sees | | The hub cannot administer a node | privileged ops need an Ed25519 signature from a roster-pinned operator key | | The hub cannot impersonate a device owner | device linking is countersigned by a key the node pinned; the hub stores no user keys | | A revoked token cannot connect | denylist on user, `jti` and group, persisted across restarts | | A signature cannot be repurposed | domain-separated, length-prefixed transcripts naming op, subject, node, group, nonce and time | | One group cannot read another on the same node | per-group index, chat store, epoch keys and peer registry; the sealed envelope's AAD names the group | | The ack's configuration is authenticated by a key the hub does not hold | sealed under `ack_key`; the signed handshake transcript names no ack field, so this is the only thing that authenticates them | | A peer served before the handshake completes gets ciphertext, not filenames | `index_sync`/`index_delta` sealed under `index_key` — defence in depth against a serve-before-authentication mistake (§11.1a) | | An upload's filename, destination and content never appear on the wire in clear | `file_upload`/`file_upload_ack` sealed under `upload_key`; `upload_id` replaces the filename as the correlation key | | No message carries content outside an AEAD | file chunks and stream segments alike, under keys derived per file and per chunk | | A body cannot be replayed as another message or into another group | AAD = `"\|"`, and `"chat_msg\|\|"` for chat | | A chat message names the device that wrote it, checkably by any member | signature over the ciphertext with a device key the node pinned; `group_roster_resp` hands over the evidence to verify it without trusting the node | | A device may only send as itself | `device` on `chat_msg` is compared with the device this connection proved (§9.4), not believed | | A chat message cannot be replayed into the archive | unique `(device, nonce)` at rest; the signing transcript cannot bind a connection nonce, because history readers have no connection | | An old chat message cannot be re-presented under a later key | the epoch is inside the AAD | | A member cannot transfer outside the caps the operator set | a lease per job, per-member cap before node-wide pool, and the leaseless path bounded to 12 files per session (§11.2) | | A reconnect cannot charge a member twice for one transfer | `tr` is drawn by the client and `transfer_open` is idempotent on it | | A refusal reaches the request it refuses | `req_id` stamped on every reply, including `error` (§3.5) | ### 14.2 What it deliberately does not establish * **The browser SPA is served by the hub.** A hub that ships malicious client code can read a pairing code out of the page, or the group key out of memory. This is accepted permanently for the browser client and is what the native client removes. Keep the two attacks apart: the pairing code defeats a hub that *lies in its directory* — silent, undetectable, per-request — not one that *rewrites the client*, which is an artifact that can be inspected and compared. * **An invitation link's code is a bearer code** (§8.6). The node admits whoever brings it first; what restricts who can bring it is the hub, which lets only the addressed account reach the node — a rule an active hub does not have to keep. The challenge signature (§6.5) keeps the code from reaching any node but the issuer; it does not keep it from a hub the inviter asked to mail it, which then holds it. * **The pre-proof window is a disclosure surface.** A hub that forges a JWT can fetch a member's *encrypted* keypair bundle. It is bounded, audited, and closes when clients stop storing bundles on other people's nodes (§7). * **Transfer leases are fairness, not security.** They bound cooperating clients. A client that lies — labelling a bulk download as a view — gets 12 files at a time instead of its member cap; that is the residual, it is bounded and audited, and the answer to a member determined to saturate a node's disk is `member revoke` (§11.2). * **The QUIC transport is in development** (§5.2) and establishes nothing yet. When it does, its channel binding is a certificate hash rather than an RFC 5705 exporter, which is weaker: on a resumed session the anchor travels with the session ticket. * **Nothing is confidential from a member, or at rest on the operator's disk.** The sealed envelope (§11.1a) is defence in depth against our own next serve-before-authentication bug; it is not a claim against anyone who holds the group key. Chat is the one thing encrypted at rest as well, which protects against someone who obtains the node's storage without the keystore password — and against nobody who holds the keys. * **The control plane is still in clear.** Sealing covers content: the index, the ack, file chunks, stream segments, uploads, the chat keys and the group roster. It does not cover the admin and configuration acks (`app_directories_ack`, `root_*_ack` and the rest), which carry the same folder names the sealed index carries; the media-metadata replies (`media_meta_resp`, `music_meta_resp`, `link_preview_resp`), which carry titles, artists and synopses; `node_status_ack`, which carries absolute paths on the operator's disk to an operator session; the identity replies (`roster_read_ack`, `device_list_result`); or `invite_result` and `invite_link_result`, which carry a pairing code. All are inside DTLS/TLS and none reaches the hub, but none is behind the group key. * **Transfer messages are in clear on purpose**, and that is a deliberate line rather than an omission: `tr` is opaque, `bytes` and `chunks` are numbers, and there is no filename and no path anywhere in them (§11.1a). * **Rotation is the only thing that removes access.** Revoking a member stops the node serving the next key; the current key and anything already downloaded stay readable. A chat epoch is opened at the same time, which stops them reading what is said next — not what was said before, which they could already read. --- ## Appendix A — Transcripts at a glance ``` handshake "meshbay:mnp:handshake:v1" LP(role) LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding) -> HMAC-SHA256 under the GEK; role in {"client","node"} challenge "meshbay:mnp:challenge:v1" LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding) -> Ed25519 by the node key, in handshake_challenge (3.4) admin op "meshbay:admin:v1" LP(op) LP(node_pk) LP(group_id) LP(subject) LP(nonce) LP(ts) -> Ed25519 by an operator key from the roster join "meshbay:join:v1" LP(node_pk) LP(group_id) LP(user_id) LP(pk_ed25519) LP(pk_x25519) LP(nonce_s) LP(ts) -> Ed25519 by the joining identity device req "meshbay:device_req:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519) LP(code_hash) LP(nonce_s) LP(ts) -> Ed25519 by the NEW device (possession only) device add "meshbay:device_add:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519) LP(nonce_s) LP(ts) -> Ed25519 by an ALREADY-PINNED device of the same account device hello "meshbay:device_hello:v1" LP(node_pk) LP(group_id) LP(user_id) LP(pk_ed25519) LP(nonce_s) LP(ts) -> Ed25519 by the device claiming this connection chat msg "meshbay:chat:v1" LP(group_id) LP(epoch) LP(device) LP(nonce) LP(ct) -> Ed25519 by the SENDING device, over the CIPHERTEXT (the one transcript that binds no connection nonce: a reader of history has no connection. Replay is refused at rest instead, on the unique (device, nonce) pair) sealed msg aad = "|" UTF-8, NOT length-prefixed key = HKDF-SHA256(GEK, salt=, info="meshbay:{index,ack,upload,chat_keys,roster}:v1", 32) -> AES-256-GCM, 12-byte random nonce per message chat seal aad = "chat_msg||" UTF-8, NOT length-prefixed key = HKDF-SHA256(epoch_key, salt=, info="meshbay:chat:dev:v1||", 32) -> AES-256-GCM, 12-byte random nonce per message (the two AADs above are the one place bare concatenation is used: every half is fixed-vocabulary or numeric, and the separator cannot occur in a msg_type, a group id or an epoch) channel binding WebRTC LP(offer_fp) LP(answer_fp) raw 32-byte SHA-256 fingerprints QUIC LP(SHA-256(server_cert_der)) LP(x) = uint32be(len(x)) || x every field, no exceptions ``` ## Appendix B — Constants | Constant | Value | Source | |---|---|---| | `MNP_VERSION` | `3.4` | `meshbay_common/__init__.py` | | `MNP_MIN_SUPPORTED` | `3.0` | `handshake.py` | | `NONCE_LEN` | 32 bytes (both handshake nonces) | `handshake.py` | | `ADMIN_CHALLENGE_TTL` | 120 s | `adminop.py` | | `JOIN_TTL`, `DEVICE_TTL` | 120 s | `join.py`, `device.py` | | Pairing / device code | 40 bits, Crockford base32, single use | `roster.py`, `device.py` | | Code lifetimes (default, settable) | invitation 7 d, operator pairing 24 h, device request 1 h | `roster.py` | | `MAX_PRE_PROOF_FETCHES` | 4 per connection | `webrtc/dispatch.py` | | `MAX_JOIN_ATTEMPTS` | 5 per connection | `webrtc/admission.py` | | `MAX_JOIN_FAILURES_WINDOW` / `JOIN_FAILURE_WINDOW` | 20 / 600 s, node-wide | ” | | Device attempts | 5 per connection | ” | | `MAX_DEVICES_PER_USER` | 5 | `roster.py` | | `MAX_LINK_INVITES_PER_GROUP` | 20 unredeemed invitation links | `roster.py` | | `PRE_HANDSHAKE_MAX_MSG` / `MAX_MSG` | 64 KiB / 64 MiB | `webrtc/core.py` / `webrtc/limits.py` | | `CHUNK_SIZE` | 1 MiB | `webrtc/limits.py` | | `DOWNLOAD_BUFFER_HIGH` | 2 MiB | `webrtc/files.py` | | `MAX_UPLOAD_BYTES` | 8 GiB, default only — `max_upload_gb` overrides it per node | `webrtc/upload_handlers.py` | | Download pipeline / chunk retry (client) | 8 in flight; 6 attempts, 1.5 s apart | `file-utils.js` | | Upload chunk / window / send-buffer high water (client) | 48 KiB / 32 / 1 MiB | `transport.js` | | `UPLOAD_ID_LEN` | 16 bytes, hex on the wire | `protocol.py` | | `UPLOAD_PROBE_INDEX` / probe timeout | `-1` / 5 s | `protocol.py`, `transport.js` | | `PART_SUFFIX` / `ORPHAN_AFTER_SECS` | `.part` / 24 h | `uploads.py` | | `DEFAULT_MAX_CONCURRENT` (node-wide, per kind) | 8 | `transfers.py` | | `DEFAULT_MAX_PER_MEMBER` (per group, per kind) | 2, settable 1–32 | `transfers.py`, `webrtc/node_ops.py` | | `GRANT_DEADLINE_SECS` / `IDLE_TIMEOUT_SECS` | 30 s / 120 s | `transfers.py` | | `MAX_QUEUED_PER_MEMBER` / `MAX_MISSED_GRANTS` | 32 / 3 | ” | | `MAX_LEASELESS_IN_FLIGHT` / `LEASELESS_IDLE_SECS` | 12 files / 60 s | ” | | `TRANSFER_SWEEP_SECS` | 15 s | `webrtc/transfer_handlers.py` | | Lease watchdog (client) | 60 s, then re-ask | `transport.js` | | `STREAM_SEGMENT_SIZE` | 256 KiB | `webrtc/apps/streaming.py` | | `STREAM_MAX_CREDIT` | 256 | ” | | `STREAM_CREDIT_TIMEOUT` / `_POLL` | 120 s / 3 s | ” | | Client stream credits | 24 | `transport.js` | | `MAX_CONCURRENT_TRANSCODES` | 8 | `webrtc/apps/streaming.py` | | Link preview rate | 15/conn, 60/node per 60 s; cache 1 h × 256 | `webrtc/chat.py` | | ICE gathering deadline | 4 s | `transport.js` | | Signaling: max SDP, pending per user, rate | 16 KiB, 3, 30/min, 15 s answer timeout | `api/signaling.py` | | GEK | 256-bit, node CSPRNG | `crypto.py` | | Chat epoch key | 256-bit, node CSPRNG, one per group per epoch | `chatbox.py` | | Chunk cipher | AES-256-GCM, 96-bit nonce (ChaCha20-Poly1305 variant for native) | `webcrypto.py`, `crypto.py` | | GEK wrap | X25519 + HKDF-SHA256 + AES-256-GCM, AAD = recipient public key | `crypto.py` | | Sealed message envelope | HKDF-SHA256 subkey per purpose, AES-256-GCM, 96-bit random nonce | `groupbox.py`, `static/crypto.js` | | Chat envelope | HKDF-SHA256 subkey per device per epoch, AES-256-GCM, 96-bit random nonce, Ed25519 over the ciphertext | `chatbox.py` | | File id | BLAKE3, hex | `crypto.py` | | Minimum installed client | `GET /v1/hub/version` → `client.minimum` | `meshbay-hub/api/hub.py` | ## Appendix C — Where the authority lives This document is descriptive: where it and the code disagree, the code is right and this is a bug. Nothing here depends on another document. ``` meshbay-common/ protocol.py message types, chunk and upload codecs handshake.py transcript, proofs, version negotiation, token rules groupbox.py the sealed envelope and its purposes chatbox.py chat epoch keys, per-device subkeys, signing adminop.py the admin transcript and the operation catalogue join.py join transcript and pairing codes device.py device request / add / hello transcripts crypto.py GEK, chunk keys, ECIES wrap, BLAKE3 ids webcrypto.py the AES variants the browser can also compute meshbay-node/ transport/webrtc_server.py the reference implementation of MNP, transport/webrtc/ assembled from these modules transport/quic_server.py the QUIC transport, in development (§5.2) transport/wire.py the one index encoder transfers.py leases, queues, caps, leaseless reads uploads.py partial uploads and orphaned .part files roster.py, ops.py, daemon.py roster, operations, group contexts meshbay-hub/ api/signaling.py SDP relay limits static/transport.js the client half of every exchange above static/crypto.js the browser mirror of groupbox/chatbox api/hub.py the minimum client version gate ```