aboutsummaryrefslogtreecommitdiffstats

MeshBay Node Protocol (MNP)

Wire version: 3.4 — meshbay_common/__init__.py (MNP_VERSION) Oldest peer accepted: 3.0 — handshake.py (MNP_MIN_SUPPORTED) Normative implementation: meshbay-common (protocol.py, handshake.py, groupbox.py, chatbox.py, adminop.py, join.py, device.py, crypto.py, webcrypto.py), meshbay-node (transport/wire.py, transport/webrtc_server.py and transport/webrtc/, transport/quic_server.py, transfers.py, uploads.py), browser client (meshbay-hub/static/transport.js, static/crypto.js). Document status: descriptive specification of the protocol as implemented on 2026-09-10. It describes the protocol as it stands. Where the code and this document disagree, the code is authoritative and this document is the thing to fix.

It is self-contained. Every rule below is given with the reason it exists, in place — a reader should never have to open a second file to find out what a rule is protecting. The only outward references are to source files, which are the authority for details this document rounds off.


1. Scope

MNP is the protocol spoken between a client (browser SPA, desktop client, CLI) and a node (the daemon that holds a group's files and the group key). It covers:

  • mutual authentication of client and node, bound to the concrete transport channel;
  • delivery of the group key (GEK) to an identity the node has pinned;
  • the content plane — index, file chunks, uploads, video streaming, chat;
  • the transfer slots a download or an upload runs under, and the caps on them;
  • operator-authorized administration of the node, signed with the operator's key.

MNP is not:

  • the protocol between node and hub — that is MHP (0.1), which carries signaling, revocation push and presence, and is out of scope except where a step of MNP depends on it (§4);
  • a discovery mechanism. Presence comes from the hub's socket registry; ping/pong exists only for liveness on an already open channel, because opening a connection costs a full ICE/DTLS handshake (measured 0.6–7 s).

1.1 Design invariants

These hold for every exchange described below. They are the reason the protocol has the shape it has, and each is argued where it is applied.

# Invariant
I1 The hub is not trusted with content or keys. It issues JWTs and relays SDP. A valid JWT is necessary but never sufficient: every session must additionally prove possession of the group key.
I2 Nothing arriving over MNP contributes key material. The node generates the group key itself, and each chat epoch key too, and wraps them for keys peers have proved they hold. No message exists by which a member hands the node key material, and none may be added.
I3 Authority over the node comes from the node's own roster, never from a token claim. Privileged operations are authorized by an Ed25519 signature over a structured transcript (§10).
I4 Every signed or MAC'd transcript is domain-separated and length-prefixed. Bare concatenation is forbidden: without the lengths, two different field splits produce the same bytes, and a signature over one is a signature over the other.
I5 Every proof is bound to the channel it was made on. An absent channel binding is a refusal, never a degraded proof (§6.4).
I6 Authentication is mutual. The node proves group-key possession over a client-chosen nonce and signs the transcript with its long-term key; the client pins that key per node (§6.1, step 11).
I7 Per-group isolation. Peer registry, chat store, chat epoch keys and index are resolved per group on a multi-group node.
I8 A message that must open under the group key, and does not, ends the session. Never a default, never an empty result: an unopenable enabled_apps would read as "the operator disabled every app" and an unopenable index as "the group is empty", both indistinguishable from legitimate states (§11.1, §6.6).
I9 Both peers declare the protocol range they speak, and check the other's. A mismatch is a refusal with a code, not a field that turns up missing (§13).
I10 A requirement only a newer peer can meet is enforced at the handshake, not per message. The alternative — an opt-in switch, "enforce it for peers that speak the new version" — leaves the permissive branch reachable on every node, and the branch left open is the one that gets used (§13).

2. Notation

C            the client (browser SPA, desktop client)
N            the node daemon
H            the hub (signaling and token issuance only)
X -> Y  msg  X sends message type `msg` to Y
[...]        optional / conditional
||           byte concatenation
LP(x)        len(x) as 4-byte big-endian, followed by x
b64(x)       standard base64 of x, as an ASCII string

Field names are given exactly as they appear on the wire. A message is a msgpack map; type and v are present on every message the node emits, and type on every message it accepts.


3. Framing and encoding

3.1 Frame format

Identical on every transport:

+--------------------------------+------------------------------------------+
| length : uint32, big-endian    | payload : msgpack map (use_bin_type=true) |
+--------------------------------+------------------------------------------+
        4 bytes                           `length` bytes
  • WebRTC: frames are written to a single ordered, reliable DataChannel named by the client; the receiver accumulates bytes and extracts complete frames (_DataChannelBuffer). A frame may span several DataChannel messages, and one DataChannel message may carry several frames.
  • QUIC: each bidirectional stream carries one request/response exchange; the handshake runs on the first stream (_StreamBuffer, same extraction logic).

3.2 Size limits

Bound Value Where
Max frame before the client's group-key proof 64 KiB PRE_HANDSHAKE_MAX_MSG
Max frame after the proof 64 MiB MAX_MSG
File chunk (plaintext) 1 MiB CHUNK_SIZE
Video segment (plaintext, before encryption) 256 KiB STREAM_SEGMENT_SIZE
Upload chunk sent by the browser 48 KiB fits the aiortc SCTP limit after msgpack overhead
Upload total per file 8 GiB, operator-settable MAX_UPLOAD_BYTES, max_upload_gb
Files one session may read at once without a transfer lease 12 MAX_LEASELESS_IN_FLIGHT (§11.2)

The two-tier frame limit is not tidiness. A flat 64 MiB budget applied before authentication let an unauthenticated peer announce a large frame and dribble bytes into it, holding that much memory per connection for as long as it liked; a hundred such connections is the node's memory, from peers that have proved nothing. Exceeding the limit is a hard protocol error and the buffer raises rather than truncating — truncating would hand a parser a valid-looking prefix of something it never received.

3.3 Versioning field

Every message carries v. It is not what decides compatibility: the version each side speaks and the oldest it accepts are exchanged and checked once, in the first message each peer sends, before anything else is decided (§13.1). A v on any later message is informational — no handler branches on it — and a peer whose range was refused never gets to send one.

3.4 Errors

A refusal is a frame of type error:

{ "type": "error", "detail": <human-readable string>, ["code": <machine code>],
  ["req_id": <the request being refused>],
  ["upload_id" | "tr" | "file_id": <what it is about>] }
  • detail is authored to be safe to show a peer. Raw exception text, ffmpeg stderr and stack traces never reach the wire — they name paths on the operator's disk and versions of the operator's software, to somebody who asked for a file.
  • code is the same refusal in a form the client can act on. Matching on detail is a string comparison that breaks the day someone improves the wording.
  • req_id is stamped on every reply sent while answering a request, refusals included (§3.5). It is what makes a refusal reach the caller that earned it: error is the one reply with no field of its own to be recognised by, so without the id it cannot be routed at all.
  • An upload refusal names the upload_id, never the file: the filename is inside the seal, and quoting it back in clear would hand over exactly what sealing the upload path is for. A transfer refusal names the tr; a refusal to serve an unleased read names the file_id, which the caller sent in clear anyway.

Codes in use:

Family Codes
Handshake not_a_member (§6.3); version_too_old, version_too_new, version_unreadable (§13.1)
Transfers transfer_required, bad_transfer_id, bad_transfer_size, bad_kind, not_your_transfer, too_many_queued (§11.2)
Upload upload_not_sealed, no_group_key, upload_incomplete, bad_chunk_encoding, bad_chunk_index, invalid_filename, no_roots, no_such_root, no_writable_root, root_read_only, root_unavailable, no_such_directory, already_exists, not_started, too_large (§11.4)
Directories root_read_only, root_unavailable (§11.5)

Everything else refuses with detail alone. A code is added when a client has a different thing to do about the refusal — retry, re-authenticate, offer an update — and not merely to enumerate.

3.5 Request/response correlation

A request may carry req_id, and the node stamps it on every reply it sends while answering that request. The client draws it, the node never interprets it, and the match is exact.

The alternative is matching a reply to a request by arrival order, which is a guess: it is wrong whenever two replies reorder, and it has no chance at all for the one reply that names nothing of its own — error. A refusal that reaches no caller leaves the request it belonged to waiting out its timeout while some unrelated request is resolved with the refusal instead.

Node-side the id lives in a task-local (contextvars), not threaded through every send site, and it is stamped only on messages going back to the session that asked. A handler that also pushes to other peers — a chat broadcast, an index delta — reaches them through their own connection, where nothing is stamped, because those messages answer no request. An explicit req_id already on a message wins over the ambient one.

Routing, in order. A message carrying no id is either a push or a broadcast, and each has a key of its own:

Message Matched by
any reply carrying req_id that id, exactly
file_chunk file_id + chunk_index
index_sync, index_delta queued, opened, then resolved by their id — see below
file_upload_ack, upload error upload_id, against the uploader that drew it
transfer_state tr, against the lease that drew it
pong the echoed token
media_meta_resp, music_meta_resp, audio_transcode_resp the file_id asked about
season_meta_resp tmdb_id + season
link_preview_resp the url
admin_challenge the op field, against the pending request that named that op
*_ack from a signed op type minus the _ack suffix, against the same key
chat_msg, stream_*, *_ack broadcasts dedicated handlers; they are unsolicited

Two of those keys are worth their line. Chunks are the one request that runs several at a time interleaved with everything else, so nothing but a key of their own can identify them. A pong is sharper still: it is sent while other traffic is in flight, so anything less than an exact match would hand it to whatever was waiting — resolving a history request with a message that has no messages in it, and emptying the conversation on screen.

The index messages are the one case where the id is not enough on its own, and it is the shape any future sealed reply will have: they carry an id like everything else, but cannot be handed to their caller until they are opened, which the synchronous dispatcher cannot do. They are queued, opened, and resolved afterwards under the same id. Resolving them on arrival would give the caller an envelope — a nonce and a ciphertext — and skip the handler that decrypts.

Anything that arrives naming no request and matching no key is unsolicited and is dropped rather than handed to a waiting caller. New request/response pairs that can overlap in flight must therefore be distinguishable: req_id is the general answer, and a discriminator of the message's own (file_id, url, upload_id, tr, op) is what keeps a reply matchable without it.


4. Session model

                +-------------------------+
                |  channel established    |   no MNP state yet
                +-----------+-------------+
                            | handshake
                            v
                +-------------------------+
   refuse <-----+  version range checked  |   too old / too new / unreadable
                |  token authorized       |   JWT decoded, NOT authenticated
                +-----------+-------------+
                            | handshake_challenge
                            v
                +-------------------------+
                |  PRE-PROOF WINDOW       |   bounded: 4 fetches, 5 join attempts,
                |  bundles, join, device  |   64 KiB frames, every event audited
                +-----------+-------------+
                            | handshake_response (valid client proof)
                            v
                +-------------------------+
                |  AUTHENTICATED          |   `_user_id` / `_group_id` set,
                |  full message set       |   frame limit raised to 64 MiB
                +-----------+-------------+
                            | channel closes
                            v
                +-------------------------+
                |  TORN DOWN              |   transfer leases released, peer
                |                         |   unregistered, tasks cancelled
                +-------------------------+

States

  1. Unauthenticated. Only handshake is accepted. Anything else is answered Handshake required.
  2. Pre-proof. Entered when the version ranges agree, the JWT authorizes, and the node has a group key for the group. The identity is decoded but not authenticated. The only messages accepted are the ones a peer provably needs before it can compute a proof: keypair_bundle_fetch, gek_bundle_fetch and join_request. This window is a disclosure surface a hub that forges a JWT can reach, so it is bounded and audited (§7).
  3. Authenticated. The client's HMAC over the handshake transcript verified. The full message set opens, and the node has answered with its own proof and signature.
  4. Torn down. Everything the connection held is given back, and the important word is deterministic: a transfer lease is scoped to the connection precisely so that a closed tab, a quit browser and a dropped network all arrive here and none of them needs a timer (§11.2).

A session is per (connection, group). group_id is mandatory in the handshake, so one connection serves exactly one group; a client in two groups on one node opens two connections. A session may additionally identify which device of the account it is, with device_hello (§9.4) — the handshake proves the account and the group, and never proved the device.


5. Transport establishment

MNP is transport-agnostic above the frame layer. WebRTC is the transport: it is what the browser SPA and the desktop client speak, and it implements the whole protocol. A QUIC transport is in development — see §5.2.

Every transport shares one handshake module, and a parity test fails if one grows a copy of its own. A second implementation of an authentication step is a second place for the group-key proof to be skipped.

5.1 WebRTC (browser and desktop client)

The hub relays SDP only; it never sees a DataChannel byte. Non-trickle ICE: the offer carries its candidates, with a 4 s gathering deadline after which the client offers whatever it has (host candidates are enough on a LAN).

  C (browser)                H (hub)                    N (node)
      |                         |                          |
      |                         |<===== MHP WebSocket ====>|  persistent, authenticated
      |                         |        /v1/nodes/ws      |  Ed25519 node auth
      |                         |                          |
      | create offer, gather ICE (<= 4 s)                  |
      |                         |                          |
      |--- POST /v1/nodes/{node_id}/webrtc/offer --------->|
      |    {sdp, ice_candidates}|                          |
      |                         |  authorize: shared active group,
      |                         |  or an open-join group when public
      |                         |  groups are enabled; <=16 KiB SDP;
      |                         |  <=3 pending per user; 30/min
      |                         |                          |
      |                         |--- ws {webrtc_offer,     |
      |                         |        peer_id, user_id, |
      |                         |        sdp} ------------>|
      |                         |                          | RTCPeerConnection,
      |                         |                          | answer + ICE
      |                         |<-- ws {webrtc_answer,    |
      |                         |        peer_id, sdp} ----|
      |<-- 200 {sdp, ice_candidates, peer_id} -------------|
      |                         |                          |
      |   setRemoteDescription; DTLS; SCTP; DataChannel open
      |                         |                          |
      |======================= MNP frames =================|   hub is out of the loop

Notes that matter to MNP:

  • The raw answer SDP is retained before setRemoteDescription — Chrome strips sha-256 from a multi-hash SDP, and the fingerprint is needed for the channel binding (§6.4).
  • The node answers offers off its WebSocket read loop: awaiting negotiation inline would stop it reading the socket for the length of one slow ICE run — running, but invisible to the hub.
  • A 15 s timeout on the hub's side turns a silent node into 504, not a hung request.

5.2 QUIC (in development)

A QUIC transport is being built, for the LAN, port-forwarded and hub-less cases where signaling through the hub is unnecessary or unavailable.

It is not functional and is not a shipped feature. No client speaks it, it implements only part of the message set, and nothing in this document should be read as a statement about what it does today. What is settled is the framing and the identity model, and both are recorded here because they constrain the design of everything else:

  C (native)                                     N (node)
      |                                              |
      |---- QUIC connect, ALPN "meshbay-mnp" ------->|  TLS 1.3, self-signed node cert
      |<---------------------------------------------|  the certificate is the identity
      |                                              |
      |==== stream 0 : MNP handshake =============== |
      |==== stream n : one request/response each === |

The node's TLS certificate is not verified as a PKI chain — identity is established at the MNP layer, and the certificate hash is the channel binding (§6.4).


6. The handshake

One implementation for every transport: meshbay_common/handshake.py. A parity test fails if a transport skips a step.

6.1 Full exchange

  C                                                              N
  |                                                               |
  |  1. handshake                                                 |
  |      {v, v_min, token, group_id, nonce: b64(nonce_c)}         |
  |-------------------------------------------------------------->|
  |                                     2. check_version()        |
  |                                        - v   >= our v_min     |
  |                                        - v_min <= our v       |
  |                                        else error{code}       |
  |                                        authorize_token()      |
  |                                        - EdDSA verify vs hub  |
  |                                        - scope == "user"      |
  |                                        - sub non-empty        |
  |                                        - group_id non-empty   |
  |                                        - denylist(user,jti,gp)|
  |                                        - group_id in groups[] |
  |                                        - group hosted here    |
  |                                     3. |nonce_c| >= 32        |
  |                                     4. GEK exists for group   |
  |                                                               |
  |  5. handshake_challenge                                       |
  |      {v, v_min, nonce: b64(nonce_s), node_pk,                 |
  |       sig: b64(Ed25519(C))}           (3.4, section 6.5)      |
  |<--------------------------------------------------------------|
  |                                                               |
  |  5b. client checks the node's range the same way              |
  |                                                               |
  |  ....... pre-proof window (section 7) ....................... |
  |  keypair_bundle_fetch / gek_bundle_fetch / join_request       |
  |     -- the client obtains a GEK to prove with                 |
  |  ............................................................ |
  |                                                               |
  |  6. binding = webrtc_binding(offer_fp, answer_fp)             |
  |     proof_c = HMAC-SHA256(GEK, T("client"))                   |
  |                                                               |
  |  7. handshake_response {v, proof: b64(proof_c)}               |
  |-------------------------------------------------------------->|
  |                                     8. rebuild binding;       |
  |                                        refuse if empty;       |
  |                                        compare_digest(proof)  |
  |                                     9. session authenticated: |
  |                                        frame limit -> 64 MiB, |
  |                                        peer registry, audit   |
  |                                                               |
  | 10. handshake_ack                                             |
  |      {v, node_pk, proof: b64(proof_n), sig: b64(Ed25519(T)),  |
  |       nonce, ct}                                              |
  |      ct = seal(GEK, "ack", "handshake_ack", group_id,         |
  |                {is_node_admin, enabled_apps,                  |
  |                 <app>_directories, ...})                      |
  |<--------------------------------------------------------------|
  |                                                               |
  | 11. verify proof_n == HMAC(GEK, T("node"))     -> else refuse |
  |     verify Ed25519(ack.node_pk, ack.sig, T("node")) -> refuse |
  |     verify ack.node_pk == challenge.node_pk     -> else refuse|
  |     pin/compare node_pk for this node_id (TOFU) -> else refuse|
  | 12. THEN open ct  -> else refuse (never a default config)     |
  |                                                               |
  |========================= session open ========================|

Step 11 is not optional politeness. A client that accepts a bare handshake_ack without a preceding challenge, or that skips any of these checks, reopens the hole this step exists to close: a peer that had hijacked signaling could accept the client's proof, ignore it, and serve a forged index, forged chat history and a forged is_node_admin flag — the last of which offers the person an administration panel on somebody else's node.

Step 12 comes after step 11, and the order is the point. Everything in step 11 decides whether this peer is worth trusting at all; opening the payload first would mean acting on data from a peer not yet authenticated. And a payload that does not open is a refusal, not an empty configuration — see §6.6.

6.2 Transcript

T(role) = "meshbay:mnp:handshake:v1"
          || LP(role)         "client" | "node"
          || LP(group_id)
          || LP(nonce_c)      >= 32 bytes, client CSPRNG
          || LP(nonce_s)      32 bytes, node CSPRNG
          || LP(binding)      transport channel binding, MUST be non-empty

proof = HMAC-SHA256(GEK, T(role))

The role is inside the transcript, so a client proof can never be replayed as a node proof. nonce_c is what makes the node's proof fresh: without it a recorded handshake_ack is replayable by an impersonating peer.

make_proof raises rather than returning a value when binding is empty or the GEK is absent. verify_proof compares with hmac.compare_digest.

6.3 Authorization rules (authorize_token)

Rule Refusal Rationale
JWT verifies under the hub's Ed25519 public key (EdDSA) Invalid JWT: ...
scope == "user" Wrong token scope a node-scoped daemon token must not be usable as a client token
sub non-empty Token has no subject
group_id non-empty group_id is required an absent group means no membership check to make; there is no default group, and a node's first group is not one
not on the denylist for user_id, jti or group_id Token revoked all three targets, and persisted to disk: a revocation that a restart forgets is not one
group_id ∈ token.groups Not a member of this group, code not_a_member the membership check itself — a token is proof of an account, never of a group
group_id ∈ node.hosted_groups Group not hosted on this node, code not_hosted the hub may hand a client several nodes for one group, and only some of them host it

AuthorizedPeer carries user_id, group_id, username, jti — and deliberately no user public key. A key arriving in a token would be a key the hub chose, and the node records the uploader's key in order to decide who may later delete a file: that would let whoever issues tokens decide it instead. Identity keys are pinned by the node's roster. The hub certifies accounts, not keys.

not_a_member is almost always a token minted before the person was added to the group (groups is baked in at login and the hub pushes no updates), so the client refreshes once and retries on that code rather than telling a member they are not one.

not_hosted is the client's signal to try the next node the hub offered for the group rather than to report a failure. /v1/groups/{id}/nodes returns every node registered for the group, in hub registration order, and that order is not a ranking: a node listed first is not necessarily one that holds the group's files. Refusing without a code made this indistinguishable from a refusal the reader has to act on, and a client that stopped at the first node let one wrongly registered peer make a group unopenable for all of its members (2026-09-11).

6.4 Channel binding

Transport Anchor Construction
WebRTC both DTLS certificate fingerprints LP(offer_fp) \|\| LP(answer_fp), each the raw 32 bytes of the a=fingerprint:sha-256 line
QUIC node certificate LP(SHA-256(server_cert_der))

An empty binding is refused on both sides (Channel binding unavailable) — an absent binding is never a degraded proof, because a proof that is not bound to a channel is a proof somebody can relay. The QUIC anchor is weaker than an RFC 5705 exporter, which aioquic does not expose: it names the server's certificate rather than the concrete session, so on a resumed session the anchor travels with the session ticket. Stated here and in §14.2 rather than left to be inferred.

6.5 node_pk in the challenge

The node announces its public key in handshake_challenge, before anything is proved. This is deliberate and safe:

  • a first-time joiner needs it before the ack — join_request signs a transcript naming this node (§8.2), and someone who has never held the GEK cannot complete the handshake that would prove the key;
  • the ack proves possession and signs the transcript; the client refuses if ack.node_pk differs from the announced value;
  • a wrong value only makes the node's own verification fail.

Since 3.4 it is also signed, in the challenge itself:

C   = "meshbay:mnp:challenge:v1" || LP(group_id) || LP(nonce_c) || LP(nonce_s) || LP(binding)
sig = Ed25519(sk_node, C)

binding is the transport's channel binding (§6.4), already known when the challenge is sent; nonce_c makes the signature fresh. So it cannot be recorded and replayed, nor relayed through a peer with different fingerprints. The prefix is its own, so it is never interchangeable with the ack's signature over T("node").

Why it exists: a join (§8) is sent in the pre-proof window, before the ack — so before 3.4 an invitation code went to whichever peer answered signaling, and in a group with two hosts, to whichever host answered first. With the signature, a client that knows which node key to expect can refuse to send a code anywhere else.

What it does not do: it proves that the peer holds the key it announces, not that this is the key the client wanted. A client with no expectation learns nothing more than before, and the ack remains the proof of GEK possession.

Client rule, from the node's answer and never from its version (§13): a sig that does not verify is a refusal (Node challenge signature invalid); no sig is an older node, whose key is proved only at the ack. A node signs whenever it has a binding, and a node with none sends no signature rather than an unbound one — the proof would be refused on that connection anyway.

6.6 handshake_ack fields

Three fields are in clear, and the rest travel sealed under a group-key-derived subkey (§11.1a). The split is not aesthetic: the three below are the authentication, and a client verifies them in order to decide whether to trust anything at all — including a decryption.

Field Meaning
node_pk node's long-term Ed25519 public key, base64 raw 32 bytes
proof HMAC(GEK, T("node"))
sig Ed25519(sk_node, T("node"))
nonce, ct the sealed payload; everything below is inside it
is_node_admin whether this peer is the node's operator — computed from the node's own record (node_user_id), never from a hub claim
node_user_id the operator's account id, when known
node_pk_x25519 the node's X25519 public key, when configured
enabled_apps which group applications to show. Empty/absent means "all registered ones" client-side
<app>_directories each application's entry-point folders, keyed by the app's own registry name (video, music, photo, chat), always a list. This is the only form. The scalar video_root / audio_root / photo_roots fields that used to sit beside it are gone: one folder was never the general case, and two shapes for one answer meant whichever the reader consulted first decided it
chat_directory where chat attachments are written. Singular because Chat genuinely has one destination; "" means the operator has not chosen
chat_link_preview whether the node unfurls links posted here. Absent means on
search_listed whether the reader's cross-group Search lists this group. Presentation only — the index is served identically either way. Absent means listed
chat_epoch the chat epoch a client must seal under right now (§11.7). There is no chat_encrypted beside it, because there is no switch
transfer_limits {download, upload} — this member's own caps in this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn; never as "unlimited", which would have the interface contradicting the node (§11.2)
tmdb_enabled, musicbrainz_enabled per-group metadata lookups
tmdb_token_customized, tmdb_language node-wide TMDB config; the token itself is never sent
indexing {scanning, scanned_bytes, total_bytes} so a client connecting mid-scan shows progress immediately. Never a path or filename
scan_settings {reconcile_interval_secs, debounce_secs} — displayed, not enforced from here

Everything after is_node_admin is presentation state. It rides on the ack so a client that connects after the operator configured something does not have to wait for a live change notice to discover it.

Why this payload is sealed, and it is integrity rather than confidentiality. The node signs T("node"), which names role, group_id, both nonces and the channel binding — and no ack field at all. Without the seal, every value in the table above would be authenticated by the DTLS/TLS channel and nothing else. Sealing gives them an AEAD tag from a key the hub does not hold, which is a stronger statement than any amount of confidentiality on the index. chat_epoch is the sharpest example: a forged one would have a client sealing its messages under a key the group has retired.

On QUIC (§5.2) the payload is sealed but empty: it carries none of these fields, because it serves no browser. The seal is there regardless, so that one message has one shape on every transport — a field added later then has somewhere authenticated to go, rather than arriving in clear beside a sealed one.

A payload that does not open ends the session. It is not an empty configuration: an enabled_apps that failed to open would reach the client's documented fallback — show every registered app — which is a confident wrong answer, indistinguishable from an operator's real choice (I8).


7. The pre-proof window

Between handshake_challenge and a valid handshake_response the peer is authorized but not authenticated. Three message families are served there, each because the peer provably cannot compute a proof without it. Everything else is answered Handshake required: the dispatcher's authenticated branch begins immediately after these three, so the table below is the window, exhaustively.

Message Why it must precede the proof Bound
keypair_bundle_fetch the client's own identity keys for this node live in an encrypted bundle stored on it counts against MAX_PRE_PROOF_FETCHES = 4; audited
gek_bundle_fetch the wrapped group key is what the proof is computed with same counter
join_request a first-time member holds no group key at all. Accepted after the proof as well — an operator pairing a browser is already connected — because its authority comes from the pairing code and the signature, never from the session state 5 attempts per connection, 20 failures per 600 s node-wide

Device linking (§9) is not in this window. device_add_request and every message after it are answered only on an authenticated session, and the device budget of 5 attempts per connection applies there.

Exceeding the fetch budget is audited as pre-proof fetch flood and answered Too many requests. Every fetch in this window is written to the audit log with the message type, because this is a disclosure surface a hub that forges a JWT can reach: the hub mints the tokens, so it can present one for any account, and what it can then ask for is that account's encrypted keypair bundle. The bundle is useless without the account passphrase, which is why the window is bounded and audited rather than closed — and it closes for good when clients stop storing keypair bundles on other people's nodes.

7.1 Identity bundles

  C                                                    N
  |-- keypair_bundle_fetch {v} ----------------------->|
  |<- keypair_bundle_resp {v, found,                   |
  |     [bundle_enc], [bundle_enc_recovery]} ----------|
  |                                                    |
  |   decrypt bundle_enc with the passphrase-derived bundle key,
  |   or bundle_enc_recovery with the recovery key
  |                                                    |
  |-- keypair_bundle_store {v, bundle_enc,             |  after minting or re-wrapping
  |     [bundle_enc_recovery]} ----------------------->|
  |<- ack {v, detail: "keypair_bundle_stored"} --------|
  |                                                    |
  |-- keypair_bundle_delete {v} ---------------------->|  withdraw the backup
  • The bundle is opaque to the node: it is encrypted client-side under a key derived from the account passphrase (keyderive.js), and optionally a second copy under the account recovery key. The node stores bytes and serves them back to the same user_id.
  • keypair_bundle_store is accepted after authentication (it is not in the pre-proof list); the fetch is what happens before.
  • A store omitting bundle_enc_recovery leaves any existing recovery copy in place.
  • Identity keys are per node. There is nothing to carry between nodes, and an operator who cracks the copy on their own disk gets a key that opens nothing anywhere else.

7.1a Per-account blobs (MNP 3.1)

The same shape as a keypair bundle with a different payload — playlists today (docs/playlists.md §8). The node stores bytes it cannot read for an account it already holds a bundle for, so this adds no new trust boundary.

  C                                                    N
  |-- user_blob_list {v} ---------------------------->|  which kinds exist here
  |<- user_blob_list_resp {v, blobs: [{kind, rev}]} --|  revisions only, no payload
  |                                                    |
  |-- user_blob_fetch {v, kind} ---------------------->|
  |<- user_blob_resp {v, kind, rev|null,               |
  |     blob_enc|null} -------------------------------|
  |                                                    |
  |-- user_blob_store {v, kind, rev, blob_enc} ------->|
  |<- ack {v, detail: "user_blob_stored"} ------------|
  |                                                    |
  |-- user_blob_delete {v, kind} --------------------->|
  • kind is a namespace, validated against a pattern: playlists is the manifest, playlist:<id> is one playlist's tracks. That is what lets one playlist be rewritten without re-uploading the whole collection, and it is a pattern rather than "anything" so the table does not become a key/value store for whatever a client feels like writing.
  • blob_enc is msgpack bin, not base64. These run to hundreds of kilobytes, where base64 is a third of every write.
  • user_id comes from the authenticated session, never from the message. A user_id in the body would let any member read or overwrite any other member's blob.
  • Caps refuse, never truncate: 64 KB for the manifest, 1 MB for one body, 8 MB per account per node, each with a stated reason. A truncating cap loses tracks silently, which is the failure the design exists to prevent.
  • The 1 MB body cap is not the binding one. A browser cannot send a frame above the negotiated max-message-size, which aiortc fixes at 65 536, so a client tops out near 64 KB per write however high this cap is set — the same constraint that keeps uploads chunking at 48 KB. Reads are not limited that way: the node answers with a whole body of up to 1 MB. See docs/playlists.md §15.3.
  • A fetch for a kind never written answers null, not an error: that is the ordinary state of a node the reader has just joined.
  • The node keeps no history. The client is the authority on which revision is current and holds its own copy; a node keeping older revisions would mean the node deciding, which is exactly what it must not do.

7.2 Wrapped group key

  C                                                   N
  |-- gek_bundle_fetch {v} ------------------------->|
  |<- gek_bundle_resp {v, found,                     |
  |     [pk_eph_b64, nonce_b64, wrapped_b64]} -------|

The bundle is an ECIES wrap of the GEK for the caller's X25519 key:

sk_eph, pk_eph  <- fresh X25519 keypair (node side)
shared          =  X25519(sk_eph, pk_recipient)
wrap_key        =  HKDF-SHA256(shared, salt = pk_eph, info = "meshbay:gek_wrap:v1:aes", len 32)
wrapped         =  AES-256-GCM(wrap_key).encrypt(nonce_96, GEK, aad = pk_recipient)

AES-GCM because WebCrypto has no ChaCha20-Poly1305; a chacha20-poly1305 variant with info = "meshbay:gek_wrap:v1" exists for native clients. The recipient's public key is the AEAD's associated data, so a bundle cannot be re-addressed.

found: false is the normal answer: per-member bundles are not stored, and the key is produced on demand by the join path (§8). The node keeps one stored bundle of its own (_node_{user_id}), which is how the daemon reloads its GEK across restarts.


8. Pairing and join

The substitution this section exists to prevent: an invite flow that fetched the invitee's public key from the hub and wrapped the group key for whatever came back would hand the group key to a hub that answered with its own — handed over by an honest inviter following the protocol exactly, with nothing anywhere looking wrong. So the key comes from its owner over an authenticated channel, and is bound to an identity by a one-time code the hub never sees.

8.1 Exchange

  operator (paired)          N                          invitee C
      |                      |                              |
      |-- invite_create ---->|  (signed admin op, section 9)|
      |     {user_id,        |                              |
      |      group_id,       |                              |
      |      username}       |                              |
      |<- invite_result -----|                              |
      |   {code, expires_at, |                              |
      |    user_id, username}|                              |
      |                      |                              |
      |=== code delivered out of band, not via the hub ====>|
      |                      |                              |
      |                      |<-- handshake / challenge ----|   nonce_s, node_pk known
      |                      |                              |
      |                      |<-- join_request -------------|
      |                      |    {group_id, pk_ed25519,    |
      |                      |     pk_x25519, code, ts, sig}|
      |                      |                              |
      |                      | verify: attempts, node-wide  |
      |                      |   failure window, key format,|
      |                      |   |ts - now| <= 120 s,       |
      |                      |   group_id == session group, |
      |                      |   Ed25519(sig) over J,       |
      |                      |   roster device lookup,      |
      |                      |   consume_invite(code)       |
      |                      |                              |
      |                      |-- join_result -------------->|
      |                      |   {ok, recognised, role,     |
      |                      |    gek: true,                |
      |                      |    pk_eph_b64, nonce_b64,    |
      |                      |    wrapped_b64}              |
      |                      |                              |
      |                      |          client unwraps the GEK, then computes
      |                      |          the handshake proof and completes (section 6)

8.2 Join transcript

J = "meshbay:join:v1"
    || LP(node_pk_b64)    the key announced in handshake_challenge
    || LP(group_id)       "" for operator pairing, which is node-wide
    || LP(user_id)
    || LP(pk_ed25519_b64) the caller's own identity key
    || LP(pk_x25519_b64)  the encryption key the GEK will be wrapped for
    || LP(nonce_s)        the node's handshake nonce
    || LP(ts)             unix seconds, decimal ASCII

sig = Ed25519(sk_ed, J)

Two properties carry the design:

  • the identity key vouches for the encryption key. Both are inside one signature, which is what makes "wrap the GEK for the key the peer presented" safe;
  • nonce_s binds the join to this connection, so a signed join cannot be lifted onto another.

The code is never signed and never echoed. It is a bearer secret: compared against a stored sha256(code) and destroyed on use.

8.3 Node-side decision table

Evaluated in order (_do_join_request):

Condition Outcome
join_attempts >= 5 on this connection error: Too many attempts
>= 20 node-wide failures in 600 s error: Pairing temporarily locked, audited join_throttled
key not 32 raw bytes, or bad base64 join_result{ok:false, reason:"invalid_keys"}
\|ts - now\| > 120 stale_request
group_id non-empty and != session group group_mismatch
signature does not verify over J signature_invalid
Ed25519 key is a pinned device but the presented X25519 differs key_changed
account has devices here, this key is not one unknown_device — the way in is a device-add (§9), not a new invite
device known, no member row, group policy open member row created (approved_by: "open-join")
device known, a pending invite exists for this user code required even for a known device; code_required / code_invalid on failure
device known, not an active member of the session group, a code offered redeemed like any code — an invitation link reaches here from someone pinned through another group, or removed and invited back; code_invalid on failure
device known, member row resolved join_result{ok, recognised:true, role} + wrapped GEK
unknown device, no code, policy open pin TOFU, admit, wrap (via: "tofu", audited)
unknown device, no code, policy invite code_required
unknown device, code invalid or spent code_invalid
unknown device, code valid pin identity, set member row from the invite, wrap (via: "code", or "link")

The member row is resolved as: this group's row, then the row for the join message's group_id, then the node-wide ("") row — which is where an operator opening any group finds their authority.

join_ok refuses to produce a key when roster.is_authorized(group_id, user_id) is false: join_result{ok:true, gek:false, reason:"not_authorized_for_group"}. Hub membership alone must not produce a key.

8.4 Pairing codes

  • 8 characters, Crockford base32 (no I, L, O, U), rendered XXXX-XXXX — 40 bits. Reading one back is case-insensitive, dashes and spaces are decoration, and the excluded letters fold onto the digits they resemble: a code read out over the phone should not be able to fail in a way the node could have absorbed.
  • Single use, stored only as sha256(code). A password KDF over 40 uniformly random bits would buy nothing.
  • Three lifetimes, each matched to the conversation the code crosses, and all three settable by the operator:
Code Default Why
member invitation 7 days it is sent by mail or message and answered whenever the other person next looks; a day dies over a weekend, and reissuing needs the inviter at a browser with the node online
operator pairing (§8.5) 24 h it crosses an SSH session — printed, then typed minutes later
device-add request (§9) 1 h read off one screen and typed into another, in one sitting

The longer window costs little: a code is single use, bound to one account, never seen by the hub, and 40 bits do not fall to guessing in a week against the node-wide lockout below. * Valid for exactly one user_id in one group — except an invitation link (§8.6). * Guessing is bounded per connection and node-wide, and every failure is an audit event rather than a silent grind. * join_policy is read from the node's own configuration, never from the hub: a hub that could declare a group open would be handing itself the key to it.

8.5 Operator pairing

The same message with group_id: "". Authority is node-wide, and the code comes from meshbay-node operator pair over SSH — the hub never sees it. The reason it cannot is worth stating plainly: the node cannot ask the hub which key belongs to its operator without letting the hub answer with its own, which is the same substitution as §8's invite flow, one level up, and it would make the hub node administrator everywhere. There is one source of operator authority and it is the roster: a node.toml naming an admin_pk_ed25519 is warned about at startup and never obeyed, because a second source of authority is a second thing to get wrong.

A code for someone who may have no account yet, so it names none: it is bound to the first account that redeems it (invite_link_create, §10.4). What stops a stranger holding it is not the node but the hub, which lets only the account whose verified address the inviter named reach the node at all (MESHBAY_DESIGN.md §3.4). At the node it is therefore a bearer code, and everything else about it is fixed:

  • role member, never operator; a row that says otherwise is refused, not honoured;
  • one named group, never node-wide, and only redeemed on a connection authenticated to that group — the account codes keep redeeming anywhere, as before;
  • not by someone already an active member of that group, who would otherwise spend it for the person it was meant for; a member row that is revoked does not count;
  • single use: redemption sets used_at and user_id in one guarded UPDATE;
  • at most 20 unredeemed per group (MAX_LINK_INVITES_PER_GROUP);
  • cancellable before use by invite_cancel, by an invite_id handle unrelated to the code; a redeemed one stays as the record of the join.

kind is a column, not an empty user_id: every query that matches invitations by account filters on kind = "account", so a row bound to nobody can never be read as bound to anybody (AV1). A roster from before this gains kind and invite_id by ALTER TABLE, and its codes stay account codes.


9. Device linking

Identity keys are per node, so one person using a browser and a desktop client holds two keys on the same node. A second device is admitted by a key the node already pinned — never by the hub, which stores no user keys and therefore cannot countersign anything.

Every message in this section is answered on an authenticated session only (§7): both the device filing a request and the device approving it have completed a handshake on their own connection.

9.1 Exchange

  new device D                 N                    approver A (already pinned)
      |                        |                              |
      | code <- random, 40 bits, displayed on D's screen      |
      | code_hash = sha256(code "\x1f" pk_ed "\x1f" pk_x)     |
      |                        |                              |
      |-- device_add_request ->|                              |
      |   {pk_ed25519,         |  checks: account known here, |
      |    pk_x25519,          |    < 5 devices, |ts| <= 120s,|
      |    code_hash, ts, sig} |    Ed25519(sig) over D_req   |
      |<- device_add_request_ack                              |
      |   {expires_at}         |   filed as pending, inert    |
      |                        |                              |
      |=== the code is read off D's screen, typed into A ====>|
      |                        |                              |
      |                        |<--- device_lookup {} --------|
      |                        |---- device_lookup_result --->|
      |                        |     {requests: [{pk_ed25519, |
      |                        |       pk_x25519, code_hash,  |
      |                        |       created_at}, ...]}     |
      |                        |                              |
      |                        |     A recomputes sha256(code||keys)
      |                        |     for each candidate and keeps the match.
      |                        |     No match -> refuse before signing.
      |                        |                              |
      |                        |<--- device_add --------------|
      |                        |     {pk_ed25519, pk_x25519,  |
      |                        |      code_hash, label, ts,   |
      |                        |      sig over D_add}         |
      |                        |  verify sig against EVERY    |
      |                        |  live device of the account; |
      |                        |  take_device_request(hash)   |
      |                        |---- device_add_ack --------->|
      |                        |     {pk_ed25519}             |

9.2 Transcripts

D_req = "meshbay:device_req:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) ||
        LP(pk_x) || LP(code_hash) || LP(nonce_s) || LP(ts)        signed by the NEW device

D_add = "meshbay:device_add:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) ||
        LP(pk_x) || LP(nonce_s) || LP(ts)                          signed by a PINNED device

code_hash = sha256( code "\x1f" pk_ed25519_b64 "\x1f" pk_x25519_b64 )
  • D_req is proof of possession only. It says the caller holds the keys, never that they belong to this account. The countersignature is what establishes that.
  • D_add deliberately omits the code: the code is a bearer secret used to find the request, never signed, never echoed. What is signed is the key pair being admitted, so a signature collected for one device cannot admit another.
  • Because both keys go into code_hash, a node cannot answer the approver with a substituted key: the approver recomputes the hash from what it typed and what it was given. Nothing here rests on a human comparing digits.
  • device_lookup takes no hash argument. Taking one from the client was circular — the client cannot compute the hash without already knowing the keys it is asking about.

9.3 Listing and revocation

  C -> N   device_list {}
  N -> C   device_list_result {pending, devices: [{pk_ed25519, label, pinned_at,
                                                   pinned_via, added_by_pk, is_this_one}]}

  C -> N   device_revoke {pk_ed25519, ts, sig over D_add for the victim's keys}
  N -> C   device_add_ack {revoked: pk_ed25519}

Anyone may read their own devices and nobody else's. Revocation is countersigned like an addition, and the last device cannot be removed — an account with no device on a node can only return through an operator's invitation code. A revoked device is marked, not deleted, so a lost laptop stops being able to admit its replacement.

Per-connection attempt budget for every device message: 5, audited on exhaustion. MAX_DEVICES_PER_USER is 5.

9.4 Which device is on this connection (device_hello)

  C -> N   device_hello {pk_ed25519, ts, sig over D_hello}
  N -> C   device_hello_ack {pk_ed25519}
D_hello = "meshbay:device_hello:v1" || LP(node_pk) || LP(group_id) || LP(user_id)
          || LP(pk_ed25519) || LP(nonce_s) || LP(ts)

The handshake authenticates a group membership (the group-key HMAC) and an account (the hub's token). It does not authenticate a device, and an account may hold several (§9). Without this message the node can only guess which one is talking — and it records the uploader of every file and the author of every chat message, so a guess there is an attribution the person cannot correct.

What is checked, in order: the key is a live device of this account in the node's own roster (never a token claim), the timestamp is fresh, and the signature verifies over a transcript naming this node, this group and this connection's nonce. A key that is merely well-formed proves nothing.

Idempotent for the same key and refused for a different one: a connection does not get to change device half way through, which would let one session's uploads and messages be attributed to two. Sending a chat message requires this to have happened (§11.7), because the device field a receiver verifies a signature against is checked against the connection rather than believed.


10. Operator-authorized operations

10.1 Why a signature and not a token

The hub issues JWTs, so a JWT can never establish node-level authority. Every destructive or privileged operation is authorized by an Ed25519 signature over a structured transcript, verified against the keys the node's roster records as holding operator authority — read fresh on every call, so revoking a paired browser takes effect immediately.

10.2 Two-hop exchange

  C (operator)                                        N
      |                                               |
      |-- <op message> {op-specific fields} --------->|
      |                                               | cheap pre-check:
      |                                               |   is there any key that
      |                                               |   could authorize this?
      |                                               |   (_has_admin_authority)
      |<- admin_challenge ----------------------------|
      |   {op_id, op, subject, nonce, ts,             |  node keeps the authoritative
      |    node_pk, group_id}                         |  copy in _admin_ops[op_id]
      |                                               |
      |  the client REBUILDS the transcript from the announced FIELDS
      |  and refuses to sign if `op`/`subject` are not what the user asked for
      |                                               |
      |-- admin_response {op_id, signature, op} ----->|
      |                                               | pop(op_id) - single use
      |                                               | now - ts <= 120 s
      |                                               | rebuild A from STORED state
      |                                               | verify vs roster operator keys
      |                                               | (file_delete also accepts the
      |                                               |  uploader's recorded key)
      |                                               | execute via ops
      |<- <op>_ack {op-specific fields} --------------|
      |                                               |
      |         some acks are ALSO broadcast to every peer in the group

10.3 Transcript

A = "meshbay:admin:v1"
    || LP(op)          e.g. "file_delete"
    || LP(node_pk_b64) so a signature for node A is invalid on node B
    || LP(group_id)    so authority does not leak across groups on a multi-group node
    || LP(subject)     what is being acted on
    || LP(nonce)       32 bytes, node CSPRNG, single use
    || LP(ts)          unix seconds, TTL 120 s

sig = Ed25519(sk_operator, A)

The structure is the whole point. A challenge of 32 raw random bytes, signed blind, would be an unbound signing oracle: the signed message would name no operation, no subject, no node and no time, so a signature obtained for one purpose would be structurally valid for any other, on any node, for ever.

The node never takes a signed value off the wire. It rebuilds A from _admin_ops[op_id]; the client rebuilds it from the announced fields. They agree by producing the same bytes.

10.4 Operation catalogue

subject is what the client must display and match before signing. Where the ack is broadcast, every connected peer in the group learns the change without reconnecting.

Op Subject Authority Ack Broadcast
file_delete file_id operator or the file's recorded uploader_pk file_delete_ack{file_id} no
dir_delete path relative to the root operator dir_delete_ack{dir} no
invite_create invitee user_id operator only (delegation designed, deferred) invite_result{code, expires_at, user_id, username} no — the code is shown once
invite_link_create link:<group_id>, the session's group operator only invite_link_result{code, invite_id, expires_at, group_id} no — the code is shown once
invite_cancel invite_id (32 hex) operator only ack{detail: "invite_cancelled", invite_id} no
member_revoke user_id operator member_revoke_ack no
member_unpin user_id operator member_unpin_ack{user_id} no
gek_rotate group_id operator gek_rotate_ack{group_id, authorized_members, note} no
apps_enabled the app set operator apps_enabled_ack{apps} yes
set_scan_settings the interval/debounce pair operator set_scan_settings_ack{...} yes
tmdb_config custom_token=yes\|no,language=... operator tmdb_config_ack{token_customized, language} yes (never the token)
tmdb_enabled enabled operator tmdb_enabled_ack{enabled} yes
tmdb_override file_id=..,tmdb_id=..,media_type=.. operator tmdb_override_ack{file_id, tmdb_id, media_type} yes
tmdb_rematch file_id=.. operator tmdb_rematch_ack{file_id} yes
musicbrainz_enabled enabled operator musicbrainz_enabled_ack{enabled} yes
root_add, root_remove the root operator root_add_ack / root_remove_ack no
root_update <root>:rw=on\|off,rem=on\|off operator root_update_ack yes
root_eject, root_plug the root name operator root_eject_ack / root_plug_ack yes
app_directories <app>:<dir>,<dir>,... operator app_directories_ack{app, dirs} yes
chat_directory the path operator chat_directory_ack{path} yes
chat_link_preview on\|off operator chat_link_preview_ack{enabled} yes
search_listed on\|off operator search_listed_ack{listed} yes
transfer_limits d=<n>,u=<n> operator transfer_limits_ack{limits} yes
chat_epoch group_id operator chat_epoch_ack{epoch} yes
group_attach, group_detach group_id operator group_attach_ack / group_detach_ack no

Upload policy is not in this table, and that is the design: whether a member may write is a property of each root (root_update), not a switch over the group. A single group-wide flag cannot express "this library is published read-only and that folder is a drop box", which is the ordinary arrangement.

app_directories is the only way an application's folders are set: one message for every application, keyed by the app's own registry name, so adding an application adds no message type, no signed op and no handler.

The three narrower ops it replaced — video_root, audio_root, photo_roots — are gone from the catalogue. They were the same instruction three times, differing only in the key they wrote and whether they carried a string or a list, and that shape is what made adding an application mean adding a message type, an op, a handler and a widget. It also meant three validation paths, and the older ones validated nothing: a typo was stored, matched no entry, and the application showed an empty tab with no way to tell "misconfigured" from "no files yet". One op has one validation path, and an unknown application name is refused rather than stored.

Their storage keys survive on the node — Roster.LEGACY_DIR_KEYS still reads video_root and friends out of group_settings — because that is a key on an operator's disk rather than on the wire, and a node upgraded into this has to find its own configuration.

A second family of operator messages is not signed: node_status, roster_read, denylist_read, denylist_clear, node_settings_set, node_reload. These are gated by is_node_admin() — the authenticated session's user_id equals the account the node records as its own operator (node_user_id), computed from the node's own state and never from a hub claim. Three of them only read; the other three run through the same ops entry points as the CLI and the loopback admin API. The distinction from the signed table above is deliberate but worth stating plainly: a signed op proves possession of an operator key, while these prove only that the session belongs to the operator's account, which the handshake already established.

Rules that hold across the table:

  • No MNP message can activate a group key. The rule (I2) targets key material arriving from outside, not the instruction: gek_rotate and chat_epoch are allowed precisely because the node generates the new key itself with its own CSPRNG. Initial gek-init stays local — with no group key there is no completed session to carry a signed op anyway.
  • There is no operation by which key material reaches the node (§12). The node wraps for a key the recipient has proved possession of, so no such message is needed — and a path that does not exist cannot be mis-authorized, which is I10 applied to a message instead of a version.
  • An operator cannot revoke or unpin themselves over the connection their pin authorizes.
  • Rotation is what actually removes a revoked member's access. Revocation stops the node serving the next key; the ex-member still holds the current one, and content they already downloaded stays readable. The ack says so in words.

10.5 One implementation, several front doors

The loopback admin API, the CLI and the signed MNP handlers are three thin adapters over the same functions in meshbay_node/ops/. Those functions take the daemon state, raise OpError, and know nothing about HTTP.

One operation with two implementations means two authorization checks, and the weaker one is the one that decides. A front door is allowed to differ in how it authenticates — a signature here, a run token on loopback, an operator's shell for the CLI — and never in what it does.


11. Content plane

Everything in this section requires an authenticated session (§4). All content is encrypted under keys derived from the GEK, so a node that serves a chunk to a session that never proved GEK possession serves ciphertext nobody can open — but the authorization check comes first regardless.

11.1 Index

The Mesh Group Index is the list of files the node shares for one group. Entries are content-addressed: id is the BLAKE3 hash of the file.

  C                                            N
  |-- index_sync {v} ------------------------->|
  |<- index_sync {v, group_id, nonce, ct} -----|   ct = seal(..., "index_sync", ...)
  |     payload: {version, entries[],          |
  |               dirs[], roots[]}             |
  |
  |            ... operator drops files into a watched folder ...
  |
  |<- index_delta {v, group_id, nonce, ct} ----|   pushed, unsolicited, to every
  |     payload: {base_version, version,       |   peer in this group
  |               additions[], deletions[],    |
  |               updates[], roots[]}          |
  |
  |<- index_progress {v, group_id, scanning,   |   every ~2 s while scanning,
  |     scanned_bytes, total_bytes} -----------|   plus once on the return to idle
                                                   NOT sealed — see below

IndexEntry wire fields (index_entry_wire):

Field Meaning
id BLAKE3 of the file content, hex
name, path filename, and the virtual directory it lives in (<root>/<subpath>)
size, type, added_at bytes; video\|audio\|image\|document\|archive\|other; unix seconds
duration, width, height, thumb_hash filled asynchronously by enrichment
uploader_id who uploaded it; null for content pre-existing on disk
display_title, season, episode Videos app, parsed from the name/folder
artist, album, track_no Music app, from tags or parsed
taken_at, camera Photos app, best-effort from EXIF

uploader_pk exists on the dataclass but is not in the wire dict: it is the key the node recorded at upload time, used server-side to authorize file_delete.

Three lists, not two: updates carries entries whose id is unchanged (same content) but whose fields changed — enrichment filling in duration/thumb_hash after the file was first indexed with hash and size only. diff() only places an id there once it has appeared unchanged in a prior snapshot.

dirs and roots exist because directories are not index entries. Without them a folder just created, or one emptied, does not exist as far as the UI is concerned, and a member cannot tell "the drive is unplugged" from "it is all still there" — an unavailable root is listed, with its content frozen rather than hidden.

Each root is described as {name, kind, available, writable, removable, ejected} — and never a path: a member is told what exists and whether it is readable, never where on the operator's disk it lives. roots rides on index_delta as well as index_sync, because a full index is only ever sent on request: without it, a root added, removed, ejected or plugged would leave every connected client's directory table stale until somebody reloaded the page, and the delta that tells them something changed would be the one message unable to say what.

index_progress carries counters only, never a path or filename.

11.1a The sealed envelope

A family of messages travels sealed under a subkey derived from the GEK (meshbay_common/groupbox.py, mirrored by sealGroup/openGroup in crypto.js). There is one subkey per purpose, and five purposes:

key(purpose) = HKDF-SHA256(GEK, salt = <none>, info = <purpose info>, 32 bytes)

nonce        = 12 random bytes, per message
ct           = AES-256-GCM(key).encrypt(nonce, msgpack(payload), aad)
aad          = "<msg_type>|<group_id>"        UTF-8
Purpose info Seals
index meshbay:index:v1 index_sync, index_delta
ack meshbay:ack:v1 the handshake_ack configuration payload
upload meshbay:upload:v1 file_upload, file_upload_ack (§11.4)
chat_keys meshbay:chat_keys:v1 chat_keys_resp — the group's chat epoch keys (§11.7)
roster meshbay:roster:v1 group_roster_resp — members, device keys and the evidence that admitted each (§11.7)

chat_keys is the one whose payload is key material: a peer that has completed the handshake holds the group key and can open it, and anything short of that gets a ciphertext. That is the same statement the index makes, one step stronger.

salt = <none> is Python's salt=None and WebCrypto's salt: new Uint8Array(0); RFC 5869 extracts with a zero key either way. The subkeys are purpose-separated rather than borrowed from a file's key space — GroupIndex.serialize() reuses chunk_key_aes with a pseudo-file ("the index as chunk 0 of a virtual index file"), which is a hack this deliberately does not repeat.

What stays in clear, and why each one has to:

Field Why
type, v the receiver must route and version-check before it can decrypt
group_id already in clear in the handshake; it is the AAD and selects the key
node_pk, proof, sig (ack) they are the authentication — verified before a decryption is trusted (§6.1 step 12)
index_progress, in full counters only, never a path or a filename, pushed every ~2 s for the whole length of a scan. Sealing it would buy a rough library size and cost a decrypt per push
upload_id, chunk_index (upload) the node routes and orders on them before it can decrypt; upload_id is client-drawn, opaque, and never an authorization input
transfer_open / _close / _state, in full, and tr wherever it rides (§11.2) tr is opaque and client-drawn, bytes and chunks are numbers, and there is no filename and no path anywhere in them. Adding one to make a log line prettier is exactly the trade this envelope exists to refuse

version and base_version are inside the payload: there is no reason to act on a version number carried by a message that has not been authenticated.

The AAD binds a ciphertext to its message type and its group, so an index_sync body cannot be replayed as an index_delta, nor moved between two groups on one node.

What this buys, and what it does not. It buys integrity for the ack (§6.6), and, for the index, defence in depth against one specific class of bug: a peer that has not completed the handshake being served data anyway. That bug is an authorization mistake in one branch of one handler, it is easy to write, and it is invisible until somebody reads that branch. Sealed, it leaks ciphertext instead of filenames and folder names.

The upload is the same argument in the other direction. It seals towards the node, which holds the group key for its own group and opens the payload before it decides a destination or touches the disk — so the write path is covered by the same key as the read path, and a file is never plaintext on one leg of its journey and ciphertext on the other.

It buys nothing against a network observer — DTLS/TLS already covers that — nothing against the hub, which never sees channel traffic, nothing against a member, who holds the group key, and nothing at rest: the index stays plain in the node's memory and the files stay plain on the operator's disk, which is the design. Chat messages are not sealed by this envelope: they have their own key hierarchy, per epoch and per device, because the node must be able to relay and archive a message it cannot read (§11.7). The control plane is not covered either — see §14.2.

Nonce collision, since upload is the first purpose with volume. One subkey per purpose and a fresh 96-bit random nonce per message: at one message per 48 KiB chunk, 2³² chunks is 200 TB uploaded under a single GEK before the collision probability reaches 2⁻³², and gek_rotate exists. Deriving the nonce from the payload instead would be worse, not better — two chunks of identical bytes are ordinary in a file.

GroupIndex.serialize() is not a candidate for reuse here: it compresses with zstd, which no browser can decompress (DecompressionStream offers gzip and deflate only), so reusing it would mean shipping a WASM decoder to every client for no gain.

Failure is fatal, never degraded (I8). A client that cannot open an index message ends the session naming the message type; it never reports an empty index, because "the group has no files" is a state a real group can be in.

11.2 Transfer leases

A download is not a message. file_req asks for one chunk; a client fetching a 4 GB film sends four thousand of them, eight in flight at a time, and nothing in that says a transfer started or that it ended. Without an object standing for the transfer itself there is nothing to count, and therefore nothing an operator can cap.

The lease is that object, and every transfer runs under one.

  C                                                    N
  |-- transfer_open {v, tr, kind, bytes, chunks} ----->|  kind: "download" | "upload"
  |                                                    | per-member cap first,
  |                                                    | then the node-wide pool
  |<- transfer_state {v, tr, state, kind,              |
  |     used, cap, node_used, node_cap,                |
  |     [ahead], [reason]} ----------------------------|  state: granted | queued
  |                                                    |
  |-- file_req {..., tr} ----------------------------->|  every chunk under this lease
  |-- file_upload {..., tr} -------------------------->|  says the transfer is alive
  |                                                    |
  |<- transfer_state {tr, state: "granted"} -----------|  pushed when a queued lease
  |                                                    |  reaches the head
  |                                                    |
  |-- transfer_close {v, tr, reason} ----------------->|  done | cancelled | paused
  |<- transfer_state {tr, state: "closed", reason} ----|

Six properties carry the design, and each is a decision:

  • tr is drawn by the client, 16 random bytes, exactly like upload_id. Re-opening with the same tr is idempotent, so a reconnect cannot charge a member twice for one transfer — and re-asking is how a client recovers a grant whose push was lost.
  • A lease is scoped to the connection, never to the account. It dies with the session, which is what makes the reclaim deterministic: a closed tab, a quit browser and a dropped network all arrive at the same teardown, and none of them needs a timer.
  • A lease covers a job, not a file. A directory downloaded as a zip is dozens of files and one lease. One per file would deadlock against the member's own cap: the job cannot finish until it holds them all, and it can never hold more than its cap — two, by default.
  • Nothing is persisted. A restart drops every session anyway, and a lease that outlived the process would be a slot nothing can release.
  • Leases are counted, bytes are not. What a slot protects is concurrency — open file handles, disk seeks, the channel buffer each transfer keeps full.
  • Per-member first, then node-wide. A member at their own cap queues behind their own transfers and never holds a node-wide slot a second member has none of. Reversed, whoever arrives first takes everything.

Caps. Node-wide, 8 concurrent per kind by default; per member per group, 2 by default. A group with no value of its own gets the default, never "unlimited": reading an absent setting as no limit would leave the node-wide cap as the only control, which is the situation leases exist to end. The per-group value is a signed operator operation (transfer_limits, §10.4, bounded to 1–32; zero is refused, because a member who may not transfer at all is a member the operator revokes). The node-wide values are daemon settings. A member's own caps ride on the handshake ack so the interface can say "2 of your 2 slots are busy" rather than draw a spinner that explains nothing.

Queueing. One FIFO per kind. _pump walks it in arrival order and skips a member who is at their own cap rather than stopping at them — granting strictly in order lets one member's limit stall every other member behind them. A queued lease is told how many are ahead of it. Beyond 32 queued per member the answer is too_many_queued, because an unbounded queue is how a node runs out of memory politely.

used and cap on transfer_state are this member's own count and this member's own limit in this group — the same value _has_room enforces and the same one the handshake ack announces. Three readings of one number, and an interface that draws a different one from the node's is an interface that offers a slot the node will queue.

Reclaim. The session teardown is the primary path and it is immediate. A sweeper runs every 15 s for whatever the teardown cannot see, and tells two failures apart:

Situation Timer What happens
Granted, never taken up 30 s Back to the tail of the queue, reason: "not_taken_up" — the client died between asking and starting
Same, three times — Closed, reason: "abandoned". Without the bound the requeue is a permanent cycle: revoked, put back, granted again because there is room, revoked 30 s later, for ever
Granted, used, then silent 120 s Closed, reason: "idle", and the peer is told, so its widget can offer a resume rather than sit on a lie
Connection gone none Everything it held, at once

The sweeper belongs to the node, not to the session that opened the first transfer. Tying it to a session would kill it when that peer left, and every other peer's abandoned lease would then never be reclaimed.

A chunk request is what "alive" looks like. tr rides on file_req and on file_upload for exactly this: it is the only signal the node has that a granted lease is being used. Without it the sweeper cannot tell a transfer running at 20 MB/s from a client that asked for a slot and vanished, and it revokes both.

Pausing is releasing. A paused transfer holds nothing: the client closes the lease with reason: "paused" and keeps its own position, and resuming asks for a new lease and queues behind whatever is waiting now. The alternative — holding a slot while paused — is a member who pauses three downloads and blocks the group.

Refusals. bad_transfer_id (no tr), bad_transfer_size (bytes/chunks not numbers), bad_kind, too_many_queued, and not_your_transfer — the last for opening or closing a tr another connection holds, which would otherwise be a denial of service one random id away.

Reads that carry no lease

Browsing a group is never subject to a transfer slot: not the poster grid, not the covers, not opening a photo or a PDF to look at it. A member must be able to browse a group that is at capacity exactly as they browse an idle one. Thumbnails, posters and cover art never reach the check at all — they resolve out of the node's own media cache.

But "not leased" cannot mean "unbounded", or a client that simply omits tr transfers outside every cap and the caps are decoration. So a session may read 12 distinct files at once without a lease; the thirteenth is refused with transfer_required and a sentence telling the person to download the file rather than preview it. An entry already being read is always admitted, whatever the count — refusing a chunk halfway through a photo because the limit moved is worse than never having admitted it. An entry is released when its last chunk goes out, or after 60 s of silence, because a viewer closed mid-file simply stops asking and says nothing.

The number is derived from what the client legitimately does, and it has to be: the music player warms a read-ahead window of 5 tracks on Wi-Fi, so playing an album has six files in flight before anyone has done anything unusual. 12 is those six at their widest, two for a photo viewer and its own prefetch in the same session, and the rest as headroom for the next feature that reads ahead. A test derives the floor from the player's own constant, so raising the client's prefetch without raising this fails in CI rather than in front of a person. Generosity is cheap here and refusal is not: the cost of being too high is a client that could have been queued and was not, and the cost of being too low is a member told to download a track they are trying to play.

Deliberately a count of files and not a byte budget: a RAW photo out of a camera is 60–80 MB and is browsing, a 40 MB archive is a download, and no size threshold separates them. What separates them is which function asked.

What leases are not

A fairness control among cooperating clients, in the company of max_concurrent_streams — not a defence against a member determined to saturate a node's disk. A client that lies, labelling a bulk download as a view, gets 12 files at a time instead of its member cap. That is the residual, it is bounded, it is audited, and the answer to the member behind it is member revoke, not a protocol rule. Stating it is the point: a control described as a security boundary will eventually be relied on as one.

11.3 File download

  C                                                    N
  |-- file_req {v, file_id, chunk_index, [tr]} ------->|
  |                                                    | `tr` present: mark that lease
  |                                                    |   alive (§11.2)
  |                                                    | index lookup; if the id is
  |                                                    | not a file, try the media
  |                                                    | cache (thumbnail/poster/
  |                                                    | cover/transcode), sliced the
  |                                                    | same way — never leased
  |                                                    | `tr` absent: admit against the
  |                                                    |   leaseless ceiling, else
  |                                                    |   `transfer_required`
  |                                                    | backpressure: wait while
  |                                                    | bufferedAmount > 2 MiB
  |<- file_chunk {v, file_id, chunk_index,             |
  |     plaintext_size, nonce, ct} --------------------|

The client pipelines 8 chunk requests at a time and reassembles in order; a chunk that fails is retried 6 times, 1.5 s apart, because a DataChannel that hiccups mid-film should cost a pause and not the whole transfer.

Chunk encryption:

chunk_key = HKDF-SHA256(GEK, salt = none, len 32,
                        info = "file:" || file_hash || ":chunk:" || uint32be(chunk_index) || ":aes")
nonce     = 12 random bytes
ct        = AES-256-GCM(chunk_key).encrypt(nonce, plaintext)      no AAD
  • The :aes suffix keeps AES keys distinct from the ChaCha20 variant (chunk_key/encrypt_chunk, info without the suffix) derived from the same GEK.
  • nonce and ct are msgpack binary, not base64. Every message that carries content carries it this way, and none carries it outside an AEAD.
  • The key is a pure function of (GEK, file hash, index), so chunks are cacheable, resumable and requestable out of order. This is what makes a download resumable at all: a client that stopped at chunk 900 asks for 900 next time, and no state on the node was keeping its place. file_id and chunk_index ride on the response because a client running several downloads at once cannot otherwise tell whose reply arrived.
  • Requests are handled off the message loop: the reply may wait for room on the channel, and blocking the loop for that would stall the very uploads whose acks free the buffer being waited on.
  • One encoder, every transport (protocol.file_chunk_wire / file_chunk_plaintext). A message type with one encoder per transport is a message type free to drift, and its name then says nothing about which shape will arrive.
  • There is no per-chunk signature, and none is needed: the AEAD tag authenticates the ciphertext under a key only group members hold, and the node authenticates itself once, in the handshake, rather than once per megabyte.
  • A chunk that is not this shape aborts the download, with an error. There is no fallback that decodes it some other way: a client that guesses at a chunk it does not recognise writes its guess into the file the person is saving.

11.4 Upload

  C                                                    N
  |-- transfer_open {tr, kind: "upload", ...} -------->|  a slot, like a download
  |<- transfer_state {tr, state: "granted"} -----------|  (§11.2)
  |                                                    |
  |-- file_upload {v, upload_id, chunk_index: -1,      |  the probe: "where am I?"
  |     total_chunks, tr, nonce, ct} ----------------->|  writes nothing, reserves
  |     ct = seal(upload, {filename, dir, root,        |  nothing
  |                        data: b""})                 |
  |<- file_upload_ack {v, upload_id, chunk_index: -1,  |
  |     nonce, ct} ------------------------------------|
  |     ct = seal(upload, {filename, stored_as, dir,   |
  |                        resume_from})               |
  |                                                    |
  |-- file_upload {v, upload_id, chunk_index,          |  48 KiB chunks, window 32
  |     total_chunks, tr, nonce, ct} ----------------->|
  |     ct = seal(upload, {filename, data,             | open under the group key,
  |                        dir, root})                 |   or refuse (upload_not_sealed)
  |                                                    | filename allowlist
  |                                                    | root writable and available
  |                                                    | destination resolves in-group
  |                                                    | chunk_index == next expected
  |                                                    | running total <= max_upload_gb
  |                                                    | append to <stored_name>.part
  |<- file_upload_ack {v, upload_id, chunk_index,      |
  |     nonce, ct} ------------------------------------|
  |     ct = seal(upload, {filename, stored_as, dir})  |
  |            ... repeat ...                          |
  |                                                    | last chunk: rename .part ->
  |                                                    | final, tag the index entry
  |                                                    | with uploader_id/uploader_pk
  • Sealed, both halves. The filename, the destination and the bytes travel inside the seal; only upload_id, chunk_index, total_chunks and tr stay in clear, because the node routes, orders and accounts on them before it can decrypt anything. This direction seals towards the node — it holds the group key for its own group — which is the mirror image of the index, and it means a refusal cannot quote back what it just refused.
  • filename, dir and root are repeated on every chunk, not sent once in a header. A hundred bytes against a 48 KiB chunk, against the alternative: a header that arrives once is state the node has to carry, and upload state that can disagree with the chunk in hand is the thing the chunk-ordering rule and the free-name rule exist to prevent.
  • upload_id replaces filename as the correlation key. It has to: matching an ack to a request by name would hand back exactly what the seal is for. It is client-drawn, opaque to the node, unique within one connection, and never an authorization input.
  • group_id is deliberately not on the message. The session already decided which group it is on, and the node uses that as the AAD. A client naming its own group here would be choosing which key its bytes are checked against.
  • A chunk that does not open is refused with upload_not_sealed, and nothing is written. One answer covers "not sealed at all" and "sealed wrong": distinguishing them tells a peer which of the two it got right. There is no plaintext fallback — a path that still accepts plaintext is not a sealed path.

Resuming, and the probe chunk. The node identifies an upload by (member, directory, filename), so a client resuming one has to name the file. transfer_open is the obvious place to ask and it travels in clear, which would undo precisely what sealing this path bought. So the question is asked inside the seal that already exists: an ordinary file_upload with no bytes and chunk_index = -1 (UPLOAD_PROBE_INDEX). Every check below has already run by then, so it cannot be used to ask questions about a directory the caller may not write to; the node writes nothing, reserves no name, and answers resume_from — how many chunks of this file it already holds — inside the seal, because that is a fact about the operator's disk. resume_from is absent from an ordinary ack, so the two are told apart without looking at chunk_index. stored_as on a probe answer is only what is really on disk, and empty when there is nothing: reporting the free name the node would pick would promise a destination that the real chunk 0 may not choose. A node that does not understand the index refuses it, which a client reads as "start from the beginning"; the client also bounds its wait at 5 s, so a node that answers neither the probe nor its refusal costs one restart rather than a stuck upload. Starting over is always safe, which is what makes both fallbacks available.

The position outlives the connection. Upload state is held by the group, keyed by (user_id, rel_dir, filename), not by the session: state on the session dies with it, and an upload interrupted at 99% would then have to start again from zero — on a connection flaky enough to have interrupted it once. Keyed by member as well as by name, because a shared directory means two people can be sending IMG_1234.jpg at the same moment and neither may inherit the other's position.

Pausing an upload is the same shape as pausing a download (§11.2): the lease goes back, and the position does not have to be remembered accurately, because the node holds it and the probe asks for it on the way back in. A pause is taken between two chunks, never inside one — the node refuses a chunk that is not the one it expects, so a chunk boundary is the only position worth having. The client also throttles on its own send buffer (1 MiB), or the whole file lands in it in seconds and the progress bar becomes a work of fiction.

An abandoned .part is reaped. It is otherwise a gigabyte of somebody else's disk that nothing will ever finish, delete or look at again — invisible in the index, because a .part is not an index entry. One with no upload behind it is deleted after 24 h. Generous on purpose: the cost of waiting is disk, and the cost of being wrong is deleting an upload somebody is still making, which is unrecoverable and looks to them like a transfer that failed for no reason. A day covers a laptop closed overnight, a phone in a tunnel, and a client that resumes on its next launch.

  • Confinement. filename must match the name allowlist (SAFE_UPLOAD_NAME); the resolved path must sit under a shared root. This is not hypothetical tidiness — a filename rendered into the interface is stored XSS if it is allowed to contain markup, and it reaches every member of the group.
  • No overwrite, ever. A colliding name is given a free one and the uploader is told what it became in stored_as; the client must use that value when referencing the file (a chat attachment, for example). Without the rule any member could replace any shared file by uploading one with the same name — silently, and with the index still pointing at what that name meant before.
  • Ordering. Out-of-order or replayed chunks are refused — otherwise a chunk with index > 0 appends blindly to whatever .part is on disk.
  • Types are checked after the seal opens. What comes out of an AEAD is authenticated, not validated: it is msgpack a member wrote, and a filename that is a number raises where a refusal was meant. data must be binary: no sealed message can carry a string there, so a string is a peer doing something else entirely.
  • Destination. The client names the folder it is browsing, never a path: the node resolves it against the group's own roots, which refuses .., absolute segments and anything escaping its root, symlinks included. With several roots, the node picking one would send a member's file to a disk the operator did not intend, and that is discovered weeks later. An unknown root name is refused rather than falling back to a writable one, for the same reason. An unavailable or read-only root is a refusal, not a fallback.
  • No quarantine subdirectory. The file lands in the folder the sender is looking at, not in an uploads/ folder of the node's invention: a shared directory nobody can organise is not a shared directory, and a folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. What confines an upload is the allowlist, the size cap, the chunk ordering and the no-overwrite rule — never a subdirectory.
  • Policy. Writability is a property of the root, and it binds the operator too — "read-only for everyone" is what makes a published library one. Enforced here rather than by hiding a button: the button is a courtesy to people who are not trying.
  • The uploader key recorded is the one the node pinned for the device on this connection (§9.4), not one the token carried — which is what makes file_delete authorizable by the uploader without letting the hub delete anyone's files.

11.5 Directories and deletion

  C -> N   dir_create {name, dir}         any member; allowlisted, confined, audited
  N -> C   dir_create_ack {dir}

  C -> N   dir_delete {dir}               operator, signed (section 10)
  C -> N   file_delete {file_id}          operator or uploader, signed

Creating a directory is not privileged — a member who can add a file may organise where it goes — but it writes to the operator's disk, so it is audited like one and it obeys the same per-root policy as an upload: the parent's root must be writable (root_read_only) and available (root_unavailable). Read-only means read-only, and a member who cannot add a file to a published library must not be able to leave empty folders in it either.

A directory cannot be created at the virtual root: that would be adding a root, which is operator configuration rather than a file operation. Deleting a root by name through dir_delete is likewise refused.

dir_delete removes an empty directory and nothing else: it is never recursive, so whatever the caller intended it cannot destroy content. The operator deletes the files first, and sees what they are losing. The emptiness check runs before the challenge is issued, so a non-empty directory never produces a signable transcript.

11.6 Video streaming (MSE)

Segments are produced by ffmpeg as fragmented MP4 and pushed under client-granted credit. Without the credit scheme the node hands ffmpeg's whole output to the channel as fast as it is produced, and the browser holds a multi-gigabyte film in a JavaScript array while MediaSource consumes it a segment at a time.

  C                                                    N
  |-- stream_req {v, file_id, start, credits,          |
  |     audio_track?} -------------------------------->|
  |                                                    | retire this session's previous
  |                                                    |   stream (a second request
  |                                                    |   means the first is over)
  |                                                    | acquire a transcode slot (8)
  |                                                    | ffprobe: codec, duration, the
  |                                                    |   audio tracks
  |                                                    | spawn ffmpeg
  |                                                    |   -ss before -i (index seek),
  |                                                    |     -noaccurate_seek when the
  |                                                    |     video is copied
  |                                                    |   video: copy, or libx264 when
  |                                                    |     the browser cannot decode
  |                                                    |   audio: always AAC, 2 ch,
  |                                                    |     -map 0:a:<audio_track>
  |                                                    |   frag_keyframe+empty_moov
  |<- stream_init {v, file_id, codec, duration, start, |
  |     audio_tracks[], audio_track,                   |
  |     subtitle_tracks[]} ----------------------------|
  |                                                    |
  |   check MediaSource.isTypeSupported(codec)         |
  |                                                    |
  |<- stream_data {v, file_id, segment_index,          |  256 KiB, encrypted with the
  |     nonce, ct, plaintext_size} --------------------|  same per-chunk derivation as
  |            ... x credits ...                       |  a file chunk, index = segment
  |                                                    |
  |-- stream_more {v, n} ----------------------------->|  n > 0 grants; n == 0 is a
  |            ... continues ...                       |  keepalive, not a no-op
  |                                                    |
  |-- stream_stop {v} -------------------------------->|  viewer closed
  |<- stream_end {v, file_id} ------------------------ |  natural end of file
Rule Value / behaviour
Credit granted per stream_more clamped to [0, 256]
Client default window 24 segments (6 MiB)
A client that sends no credits unpaced — the node streams as fast as it can, and the client is responsible for what it buffers
Silence timeout 120 s since the peer last said anything, polled every 3 s
n == 0 keepalive: a viewer buffered 90 s ahead grants nothing and must still be able to say it is there
Concurrent transcodes 8 node-wide, semaphore on the transport context
Seeking a new stream_req with start; the previous stream is retired first, ffmpeg respawned with -ss
Accurate seek off when the video is copied, on when it is re-encoded. Copied video has to begin on a keyframe and cannot be trimmed to the request; re-encoded audio can, and is. Leaving both at the default put a whole GOP of silence at the head of every seek and left sound and picture a GOP apart — with correct timestamps throughout, so nothing downstream could detect it
start in stream_init the value actually used. On the copy path it is measured before the stream is served — the same seek with the same stream mapping, one frame copied under -copyts, 0.06–0.07 s — because an index seek lands on an indexed keyframe that the frame list cannot predict and that moves with which audio track is mapped. The client adds it back as SourceBuffer.timestampOffset; a subtitle cue carries the source's absolute time, so reporting anything else puts every line on screen away from the voice
audio_tracks in stream_init every audio track: i (the audio ordinal, what -map 0:a:<n> takes, never the container stream index), lang, title, codec, ch. Empty for a file with no audio
audio_track in stream_req which ordinal to map. Absent, out of range or malformed is the first track
audio_track in stream_init the ordinal actually used, for the same reason start is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. null when the file has no audio
Changing track a new stream_req at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream
Capability discovery the list, not the version. A client draws its selector from audio_tracks and sends audio_track only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language
subtitle_tracks in stream_init the subtitle tracks that can be shown: i (the subtitle ordinal, what -map 0:s:<n> takes, counted over every subtitle stream including the ones absent from this list), lang, title, codec. Empty for a file with no convertible subtitles
Which subtitle tracks are listed text codecs only (subrip, ass, mov_text, …). Bitmap streams (PGS, VOBSUB — about a fifth of a real library) have no WebVTT without OCR, and one extracted anyway yields a header with no cues: a track that appears in the menu and shows nothing. A file whose subtitles are all bitmap reports none, exactly like a file with none
Why the ordinal is not the list position the two differ whenever a bitmap stream precedes a text one. Renumbering the survivors would map 0:s:0 to the stream that cannot be decoded — which is an empty WebVTT, not an error
Fetching a track subtitle_req {file_id, track} → subtitle_resp {file_id, track, hash, size, mime}; the blob is pulled by hash over file_req, the same indirection as a poster or an audio transcode. Extracted whole-file, converted to WebVTT, cached under the file's own id — so a film is extracted once, not once per viewing
Subtitles and seeking nothing. The cues carry the source's absolute timestamps, so a seek and an audio-language change both leave the client's <track> untouched

An ffmpeg failure before any output produces error: Could not stream this file; stderr stays server-side, where it belongs — it names paths on the operator's disk and the operator's ffmpeg build, to somebody who asked to watch a film.

Segments are encrypted with the same per-chunk derivation as a file chunk, the segment index standing in for the chunk index. There is no unencrypted streaming path: a segment of a film is content, and content does not leave a node outside an AEAD.

11.7 Chat

Chat is encrypted on the wire and at rest, under a key hierarchy of its own — not the sealed envelope of §11.1a, because the node has to relay and archive a message it cannot read, and receivers include devices that were not connected when it was sent.

  C                                                    N                     other peers
  |-- device_hello {...} ----------------------------->|  §9.4 — required before a send
  |                                                    |
  |-- chat_keys_req {v} ------------------------------>|
  |<- chat_keys_resp {v, group_id, nonce, ct} ---------|  sealed, purpose "chat_keys"
  |     payload: {epochs: [{epoch, key}], current}     |  EVERY live epoch, not just now
  |                                                    |
  |-- chat_msg {v, format: 1, epoch, device,           |
  |     nonce, ct, sig, sender_name,                   |
  |     [thread_id], [iteration]} -------------------->|
  |                                                    | envelope checks (below)
  |                                                    | persist to THIS group's store
  |                                                    | broadcast ------------------->|
  |                                                    |   {chat_msg, sender_id,
  |                                                    |    sender_name, format, epoch,
  |                                                    |    device, nonce, ct, sig,
  |                                                    |    thread_id, timestamp}
  |                                                    | notify the hub: group id and
  |                                                    |   sender id only
  |<- ack {v} -----------------------------------------|
  |
  |-- chat_hist {v, [before], [limit <= 200]} -------->|
  |<- chat_hist_resp {v, has_more, messages: [{id,     |
  |     sender_id, sender_name, timestamp, thread_id,  |
  |     format, epoch, device, nonce, ct, sig}]} ------|
  |
  |<= chat_epoch_ack {v, epoch} =======================|  pushed: a new epoch is open

The key hierarchy.

epoch_key            32 bytes, generated by the NODE, one per group per epoch
device_key           = HKDF-SHA256(epoch_key, salt = <none>, len 32,
                         info = "meshbay:chat:dev:v1|<group_id>|<device_b64>")
nonce                = 12 random bytes, per message
ct                   = AES-256-GCM(device_key).encrypt(nonce, msgpack(payload), aad)
aad                  = "chat_msg|<group_id>|<epoch>"
sig                  = Ed25519(sk_device, "meshbay:chat:v1"
                                          || LP(group_id) || LP(epoch)
                                          || LP(device) || LP(nonce) || LP(ct))

Why a key per device and not a ratchet. A ratchet cannot deliver forward secrecy in this setting, and it is better to say so than to appear to have it. The node serves history to devices that were not present when a message was written, so it must retain and hand out each chain's earliest key — and a chain key at iteration i yields every message key from i onward by pure HKDF. Forward secrecy is then zero, and the ratchet is computing HKDF over a value every member already holds. A Signal-style sender-key implementation and a Double Ratchet were both written for this and never called by production; both have been deleted, because code nothing calls reads as an alternative somebody may reach for and its passing tests read as evidence of a protection that is not in the product.

Deriving per device by name buys what the ratchet was there for and one thing more: there is no mutable sending state at all, so nothing can be advanced twice. Two devices advancing one chain produce key and nonce reuse, which is the failure this design cannot have. Be precise about what that does not say: two clients of one account normally hold the same identity key — a second browser recovers it from the keypair bundle rather than minting a new one — so they share a device key and therefore this subkey. That is safe here only because the nonce is 96 random bits and not a counter: two independent senders under one key collide on the birthday bound, which at chat volume is unreachable, whereas two independent senders advancing one counter collide immediately. The design degrades correctly into the deployment that exists; a chain-based one would not have.

Epochs. A new epoch is opened when the set of devices that may read future messages shrinks — member revoke, member unpin, device revoke, gek_rotate — and by hand with the signed chat_epoch op (§10.4). Old epochs are kept and still delivered to current members, which is what keeps history readable to the people who could already read it. The epoch key is wrapped under the group key at delivery, never stored under it, so rotating the group key is a re-wrap and costs nothing.

Signing is separate from encryption, and it is what establishes who said something. The signature is over the ciphertext, so it can be checked before decryption and by anyone holding the roster — including on a stored row, without the epoch key. It names the device key the node pinned, never a key the sender presents alongside the message: a signature verified against a key from the same message proves only that its sender owns some key, which any member can arrange.

One thing is deliberately not in the signing transcript: this connection's nonce. Every other transcript in this protocol binds one; this one cannot, because a receiver reading history has no access to the connection a message arrived on. Replay is therefore refused by storage instead, on the unique (device, nonce) pair. A replayed message is a validly signed copy of a real one, so nothing about the signature refuses it; it is dropped and logged rather than raised at the sender, because the message it duplicates is already stored and there is nothing for anyone to retry.

What the node checks before it stores or relays anything:

Rule Why
format is the sealed one Plaintext is refused, always — not "accepted and marked", and not "unless a switch says otherwise". A member who can post in clear into a group whose members believe their chat is encrypted is a downgrade, and every peer that reaches this point is able to seal
device, nonce, sig present, right lengths, ct non-empty a malformed envelope is refused before storage, not stored and puzzled over later
the connection has identified its device (§9.4) the device field is what receivers verify against
device == this connection's pinned device a device may only send as itself. A member free to name another member's key could be that member to everyone, and the signature would verify
sender_id is taken from the authenticated session never read from the message; it never was

format is a storage state as well as a wire one, so the store can distinguish rows the wire would not accept. Nothing changes what is accepted from a peer: the sealed form, or a refusal.

Who can verify what. group_roster_req answers any member — not only the operator — with each device key in the group, which already-pinned key countersigned it, and the signature, nonce and timestamp needed to rebuild what was signed. That is what lets a member check for themselves that a message came from a device belonging to the account it claims, instead of taking the node's sender_id on trust. The reply is sealed under the roster purpose (§11.1a): it is the group's membership, and a peer that has not completed the handshake has no business reading it. The node hands over evidence and decides nothing; a node that lies here is caught by a client that has seen the account before.

Other rules.

  • The store, the peer registry and the epoch keys are resolved per group. Reading them off the shared transport context sent every group's messages to the first group's database and served them back to anyone on the node.
  • The broadcast excludes this connection, not this account. The sender's other devices are ordinary recipients: they did not compose the message and have no local echo of it, so skipping them by user_id left a person's second device silently missing everything they said from the first.
  • The hub is told a message exists — group id and sender id, nothing else. No display name: the body is unreadable to the hub, and shipping the author's name beside it would leave the hub a per-message record of who spoke where, which is the metadata this is otherwise about not producing. The sender id stays because the hub needs it in order not to notify the author of their own message.
  • before pages backwards from the newest, which is the direction a chat is read; has_more is asked about the oldest row returned, so an empty page correctly says no.
  • A message that does not open is shown as unreadable, never as blank. Rendering it empty would make a message nobody can read indistinguishable from a message nobody wrote.
  • The boundary, stated as everywhere else: the node operator and every current member hold the group key and therefore the epoch keys. This is the same boundary as file access, by design. What it protects against is someone who obtains the node's storage without the keystore password.
  C -> N   link_preview_req {v, url}
  N -> C   link_preview_resp {v, url, ok,
                              [title, description, site_name, image_thumb_hash]}

The node fetches the URL because the browser cannot (CSP and CORS) and doing so would leak every reader's IP to whatever was pasted. ok: false means "no preview" — blocked, unreachable, or not HTML — and the client shows the bare link. Any image is cached in the node's thumb store, so image_thumb_hash is fetched over the ordinary file_req path. Rate limits: 15 per connection and 60 node-wide per 60 s; results cached 1 h, 256 entries.

11.9 Metadata applications

All of these are read-only lookups against third-party services, cached node-side and keyed by content hash. None is signed: they change nothing in the node's own state. The corresponding settings are signed operations (§10.4).

Request Response Notes
media_meta_req {file_id} media_meta_resp {file_id, tmdb_id, title, original_title, overview, poster_thumb_hash, backdrop_thumb_hash, release_date, first_air_date, genres, vote_average, runtime, cast, director, confidence, [season, episode]} confidence: 0 means no confident match — the client falls back to a thumbnail-only card, it is not an error
season_meta_req {tmdb_id, season} season_meta_resp {...} a show's single overview does not describe every season alike
tmdb_search_req {media_type, query} tmdb_search_resp {results: [{id, title, year, poster}]} candidates for a human to pick from; never collapsed to one guess
music_meta_req {file_id} music_meta_resp {file_id, ...} MusicBrainz; cover art cached like a poster
audio_transcode_req {file_id} audio_transcode_resp {file_id, hash, size, mime} WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by hash over file_req
subtitle_req {file_id, track} subtitle_resp {file_id, track, hash, size, mime} MSE decodes no in-band text track, so a subtitle travels beside the stream. track is the ordinal from stream_init.subtitle_tracks and is echoed back, because one film's two tracks are exactly the pair that can be in flight together. Fetch the result by hash over file_req

Every one of these is keyed by the entry's file_id, never by a path: a path names the folder a file is in, so two files in one folder — any multi-episode season — would resolve to whichever entry the index returned first.

Posters, covers, thumbnails and transcode results all live in the media cache and are served through file_req by their hash, sliced into chunks exactly like a real file.

11.10 Liveness

  C -> N   ping {v, token}
  N -> C   pong {v, token}       the caller's token echoed back

Echoed rather than bare, so a client can match the answer to the probe it sent and measure a round trip instead of being reassured by a reply to an earlier one. This is not discovery: opening a connection in order to ping costs a full ICE/DTLS handshake, so presence in the group list comes from the hub's socket registry.


12. Message reference

Direction is C→N (client to node), N→C (node to client, solicited) or N⇒C (node to client, unsolicited push/broadcast). "Stage" is the earliest session state in which the node accepts the message: pre = pre-proof window, auth = after the client's group-key proof, signed = after an admin_response verified against an operator key.

Any request may carry req_id; every reply the node sends while answering one carries it back (§3.5).

Type Dir Stage Purpose
handshake C→N — open the session; version range, token, group, client nonce
handshake_challenge N→C — node nonce, node's version range, announced node_pk, and since 3.4 sig over the challenge transcript
handshake_response C→N — client's HMAC(GEK, T("client"))
handshake_ack N→C — node proof, node signature, session parameters sealed (§11.1a)
keypair_bundle_fetch / _resp C→N / N→C pre the caller's encrypted identity bundle
keypair_bundle_store C→N auth back up (or re-wrap) that bundle
keypair_bundle_delete C→N auth withdraw the backup
user_blob_store C→N auth write one per-account blob (playlists)
user_blob_fetch / user_blob_resp C→N / N→C auth read one, or null
user_blob_list / user_blob_list_resp C→N / N→C auth which kinds, at what revision — never a payload
user_blob_delete C→N auth drop one
gek_bundle_fetch / _resp C→N / N→C pre the caller's wrapped GEK
join_request / join_result C→N / N→C pre pin or recognise an identity; wrap the GEK
invite_create / invite_result C→N / N→C signed issue a one-time pairing code
invite_link_create / invite_link_result C→N / N→C signed issue a code bound to no account, for an invitation link (§8.6)
invite_cancel C→N signed take back an unredeemed invitation link
device_add_request / _ack C→N / N→C auth file a new device as pending
device_lookup / device_lookup_result C→N / N→C auth candidates for the approver to hash-match
device_add / device_add_ack C→N / N→C auth admit a device, countersigned
device_list / device_list_result C→N / N→C auth this account's devices
device_revoke C→N auth retire a device, countersigned (answers device_add_ack)
index_sync C→N, N⇒C auth full index — requested, or pushed on first change
index_delta N⇒C auth additions / deletions / updates against a base version
index_progress N⇒C auth scan counters, no paths
file_req / file_chunk C→N / N→C auth one encrypted chunk of a file, thumbnail or cache blob; optional tr names the lease it runs under
file_upload / file_upload_ack C→N / N→C auth push a chunk; both halves sealed — name, destination and bytes inside, upload_id, chunk_index and tr outside. chunk_index = -1 is the resume probe (§11.4)
transfer_open / transfer_close C→N auth ask for a transfer slot / give it back
transfer_state N→C, N⇒C auth granted, queued (with ahead) or closed (with reason)
device_hello / _ack C→N / N→C auth which device of this account is on this connection (§9.4)
chat_keys_req / _resp C→N / N→C auth every live chat epoch key, sealed
group_roster_req / _resp C→N / N→C auth this group's members and device keys, with the evidence that admitted each, sealed
dir_create / dir_create_ack C→N / N→C auth create a folder
dir_delete / dir_delete_ack C→N / N→C signed remove an empty folder
file_delete / file_delete_ack C→N / N→C signed delete a file (operator or uploader)
stream_req C→N auth start or seek an MSE stream
stream_init / stream_data / stream_end N→C auth codec header, encrypted fMP4 segments, end
stream_more / stream_stop C→N auth grant credit / abandon the stream
chat_msg C→N, N⇒C auth send and fan out a message
chat_hist / chat_hist_resp C→N / N→C auth paged history
chat_attach — auth attachment metadata (declared, unused on the wire)
link_preview_req / _resp C→N / N→C auth OpenGraph unfurl
media_meta_req / _resp C→N / N→C auth TMDB metadata for one file
season_meta_req / _resp C→N / N→C auth per-season TMDB fields
tmdb_search_req / _resp C→N / N→C auth candidate matches for an operator
music_meta_req / _resp C→N / N→C auth MusicBrainz metadata for one file
audio_transcode_req / _resp C→N / N→C auth browser-playable copy of a WMA/MPC file
subtitle_req / _resp C→N / N→C auth one embedded subtitle track as WebVTT, by cache hash
ping / pong C→N / N→C auth liveness on an open channel
member_revoke / _ack C→N / N→C signed stop serving the key to someone
member_unpin / _ack C→N / N→C signed forget a pinned identity
transfer_limits / _ack C→N / N⇒C signed per-member transfer caps for this group
chat_epoch / _ack C→N / N⇒C signed open a new chat epoch by hand
app_directories / _ack C→N / N⇒C signed one application's folders, keyed by app name
chat_directory / _ack C→N / N⇒C signed where chat attachments are written
chat_link_preview / _ack C→N / N⇒C signed whether the node unfurls posted links
search_listed / _ack C→N / N⇒C signed whether members' cross-group Search lists this group
root_update / _ack C→N / N⇒C signed a root's writable / removable flags
root_eject / _ack, root_plug / _ack C→N / N⇒C signed take a removable root offline, put it back
gek_rotate / _ack C→N / N→C signed node generates a new group key
apps_enabled / _ack C→N / N⇒C signed which group apps are shown
set_scan_settings / _ack C→N / N⇒C signed reconcile interval and debounce
tmdb_config / _ack C→N / N⇒C signed node-wide TMDB token and language
tmdb_enabled / _ack C→N / N⇒C signed per-group TMDB on/off
tmdb_override / _ack C→N / N⇒C signed correct a wrong automatic match
tmdb_rematch / _ack C→N / N⇒C signed drop one file's cached match
musicbrainz_enabled / _ack C→N / N⇒C signed per-group MusicBrainz on/off
root_add / _ack, root_remove / _ack C→N / N→C signed add or remove a shared directory
group_attach / _ack, group_detach / _ack C→N / N→C signed start or stop hosting a group
node_status / _ack C→N / N→C auth (operator) all groups, roots, daemon state
roster_read / _ack C→N / N→C auth (operator) pinned identities and members
denylist_read / _ack C→N / N→C auth (operator) current refusals
denylist_clear / _ack C→N / N→C auth (operator) remove entries
node_settings_set / _ack C→N / N→C auth (operator) change daemon settings
node_reload / _ack C→N / N→C auth (operator) re-read node.toml
ephemeral_stream — — reserved, mobile live push
error N→C any refusal, with detail and optionally code, req_id, and the upload_id / tr / file_id it is about
ack N→C auth generic acknowledgement (chat, keypair bundle store)

Three messages do not exist, and their absence is a rule rather than an omission:

Not a message Why there is none
any request that returns the group key in plaintext members obtain it by unwrapping their own ECIES bundle (§7.2); a node that can be asked for the key in clear is a node one authorization mistake away from handing it over
any message by which a member stores key material on the node the node wraps for a key the recipient has proved possession of (I2). A member-supplied bundle is a key of somebody else's choosing
any content message outside an AEAD a segment of a film and a chunk of a file are the same thing to everyone but the codec

A node that receives an unknown type logs it and does nothing. That is what makes the additive rule of §13 work.

12.1 Transport coverage

WebRTC implements this document. It is what the browser SPA and the desktop client speak, and every message above is available on it.

QUIC is in development (§5.2): a partial message set, no client, and not a shipped feature. Nothing about it is a compatibility commitment yet.

Two rules hold across transports, and both are about there being exactly one of each message:

  • One encoder per message type, shared by every transport. file_chunk comes from meshbay_common.protocol, index_sync and index_delta from meshbay_node/transport/wire.py, and a parity test fails if a server grows a copy of its own. Two encoders for one type is a type free to drift, with a name that no longer says which shape will arrive — and, when one of them is a sealed envelope, a second construction site that goes on sending cleartext.
  • GroupIndex.serialize() / deserialize() describes no MNP message. It is a signed, compressed, encrypted at-rest and interchange format, and reading it as a wire contract is a mistake worth naming: the sealed envelope of §11.1a is what index messages travel under, and it is deliberately not this, because zstd decompresses in no browser.

13. Versioning and compatibility

MNP versions independently of the package version. Current: 3.4; oldest peer accepted: 3.0 — 3.1, 3.2, 3.3 and 3.4 are all additive, so the floor does not move with them. 3.4 adds the challenge signature (section 6.5) and invitation links (section 8.6): invite_link_create, invite_link_result, invite_cancel, which an older node answers as unknown messages.

The two numbers are separate on purpose. MNP_VERSION says what this build speaks; MNP_MIN_SUPPORTED says what it will talk to, and moving the second is a decision about whether an older peer can still do anything useful:

  • Additive change → MINOR. A change is additive only if all of these hold: no existing field changes meaning, type or encoding; a peer that ignores the new field or message still behaves correctly; and the new message is only sent to a peer that advertised support, or is harmless to drop. The floor does not move.
  • Breaking change → MAJOR, and then the question is where to refuse. A break confined to one exchange can be refused per message, with a code, leaving the rest of the protocol working — a peer that cannot upload can still browse, download, stream and chat. A break that touches something every session depends on cannot be confined, and the honest form is to refuse at the handshake: a stated refusal is a bug report, a feature that quietly does not work is a support case.
  • Discovery from the answer, not from the version number. 3.2's audio-track selection is the shape to copy: the node lists the tracks in stream_init, and the client sends audio_track only when it was given a list. A peer that ignores that field would not degrade — it would serve a different language in silence, which is a wrong answer and not a missing feature — and what keeps the change additive is that no client can ever put an old node in that position. This is not the opt-in switch I10 refuses: there is no second branch on the node, which always enumerates, always honours the request and always reports the track it used.
  • A requirement is breaking even when its messages are additive (I10). New message types and a new optional field are additive on the wire; requiring them is not, and an opt-in switch that enforces the requirement only for peers that speak the new version leaves the permissive branch reachable on every node. That is the branch that ends up being used.

13.1 Negotiation

Both peers declare two values in the first message each sends — handshake from the client, handshake_challenge from the node:

v      the version this build speaks           MNP_VERSION
v_min  the oldest peer it will talk to         MNP_MIN_SUPPORTED

Each side then checks the other, before anything else is decided:

Condition Refusal code
peer's v < our v_min version_too_old
peer's v_min > our v version_too_new
v unparseable version_unreadable

The refusal is shaped like not_a_member: a peer-safe sentence for a human and a code the client matches on, because matching on the text is a string comparison that breaks the day someone improves the wording. A peer that declares no v_min is read as accepting only what it speaks.

Versions compare numerically, as (major, minor): as strings "0.9" > "0.15".

Refusing at the handshake is not enough on its own, for a client that ships its own interface. The browser SPA is served by the hub and is therefore never out of step with it. An installed desktop client can be, and version_too_old is a refusal in a protocol vocabulary with nothing a person can act on. So the client asks GET /v1/hub/version for client.minimum before it connects, and says "this version can no longer connect" instead. An unreachable hub is deliberately not treated as too old: a captive portal or a closed laptop must not make starting the application impossible.

The version a peer announces is only as good as the number it ships with. Every package in the tree carries one version, and a test fails if two disagree — a client announcing a number from a different scheme sorts wherever that scheme puts it, and walks through the gate meant to stop it.


14. Security properties and stated limits

14.1 What the protocol establishes

Property Mechanism
A peer holds the group key HMAC(GEK, T("client")) over a node-chosen nonce, bound to the channel
The node holds the group key and is the one previously seen HMAC(GEK, T("node")) over a client-chosen nonce, plus Ed25519(sk_node, T) and a per-node pin
No MitM on the signaling path both DTLS fingerprints (or the QUIC certificate hash) inside every transcript; empty binding is a refusal
The hub cannot read content the group key never reaches it; chunk keys and chat epoch keys derive from or are wrapped under it
The hub cannot substitute a key at invite the node wraps for a key its owner presented and signed, bound to an identity by a code the hub never sees
The hub cannot administer a node privileged ops need an Ed25519 signature from a roster-pinned operator key
The hub cannot impersonate a device owner device linking is countersigned by a key the node pinned; the hub stores no user keys
A revoked token cannot connect denylist on user, jti and group, persisted across restarts
A signature cannot be repurposed domain-separated, length-prefixed transcripts naming op, subject, node, group, nonce and time
One group cannot read another on the same node per-group index, chat store, epoch keys and peer registry; the sealed envelope's AAD names the group
The ack's configuration is authenticated by a key the hub does not hold sealed under ack_key; the signed handshake transcript names no ack field, so this is the only thing that authenticates them
A peer served before the handshake completes gets ciphertext, not filenames index_sync/index_delta sealed under index_key — defence in depth against a serve-before-authentication mistake (§11.1a)
An upload's filename, destination and content never appear on the wire in clear file_upload/file_upload_ack sealed under upload_key; upload_id replaces the filename as the correlation key
No message carries content outside an AEAD file chunks and stream segments alike, under keys derived per file and per chunk
A body cannot be replayed as another message or into another group AAD = "<msg_type>\|<group_id>", and "chat_msg\|<group_id>\|<epoch>" for chat
A chat message names the device that wrote it, checkably by any member signature over the ciphertext with a device key the node pinned; group_roster_resp hands over the evidence to verify it without trusting the node
A device may only send as itself device on chat_msg is compared with the device this connection proved (§9.4), not believed
A chat message cannot be replayed into the archive unique (device, nonce) at rest; the signing transcript cannot bind a connection nonce, because history readers have no connection
An old chat message cannot be re-presented under a later key the epoch is inside the AAD
A member cannot transfer outside the caps the operator set a lease per job, per-member cap before node-wide pool, and the leaseless path bounded to 12 files per session (§11.2)
A reconnect cannot charge a member twice for one transfer tr is drawn by the client and transfer_open is idempotent on it
A refusal reaches the request it refuses req_id stamped on every reply, including error (§3.5)

14.2 What it deliberately does not establish

  • The browser SPA is served by the hub. A hub that ships malicious client code can read a pairing code out of the page, or the group key out of memory. This is accepted permanently for the browser client and is what the native client removes. Keep the two attacks apart: the pairing code defeats a hub that lies in its directory — silent, undetectable, per-request — not one that rewrites the client, which is an artifact that can be inspected and compared.
  • An invitation link's code is a bearer code (§8.6). The node admits whoever brings it first; what restricts who can bring it is the hub, which lets only the addressed account reach the node — a rule an active hub does not have to keep. The challenge signature (§6.5) keeps the code from reaching any node but the issuer; it does not keep it from a hub the inviter asked to mail it, which then holds it.
  • The pre-proof window is a disclosure surface. A hub that forges a JWT can fetch a member's encrypted keypair bundle. It is bounded, audited, and closes when clients stop storing bundles on other people's nodes (§7).
  • Transfer leases are fairness, not security. They bound cooperating clients. A client that lies — labelling a bulk download as a view — gets 12 files at a time instead of its member cap; that is the residual, it is bounded and audited, and the answer to a member determined to saturate a node's disk is member revoke (§11.2).
  • The QUIC transport is in development (§5.2) and establishes nothing yet. When it does, its channel binding is a certificate hash rather than an RFC 5705 exporter, which is weaker: on a resumed session the anchor travels with the session ticket.
  • Nothing is confidential from a member, or at rest on the operator's disk. The sealed envelope (§11.1a) is defence in depth against our own next serve-before-authentication bug; it is not a claim against anyone who holds the group key. Chat is the one thing encrypted at rest as well, which protects against someone who obtains the node's storage without the keystore password — and against nobody who holds the keys.
  • The control plane is still in clear. Sealing covers content: the index, the ack, file chunks, stream segments, uploads, the chat keys and the group roster. It does not cover the admin and configuration acks (app_directories_ack, root_*_ack and the rest), which carry the same folder names the sealed index carries; the media-metadata replies (media_meta_resp, music_meta_resp, link_preview_resp), which carry titles, artists and synopses; node_status_ack, which carries absolute paths on the operator's disk to an operator session; the identity replies (roster_read_ack, device_list_result); or invite_result and invite_link_result, which carry a pairing code. All are inside DTLS/TLS and none reaches the hub, but none is behind the group key.
  • Transfer messages are in clear on purpose, and that is a deliberate line rather than an omission: tr is opaque, bytes and chunks are numbers, and there is no filename and no path anywhere in them (§11.1a).
  • Rotation is the only thing that removes access. Revoking a member stops the node serving the next key; the current key and anything already downloaded stay readable. A chat epoch is opened at the same time, which stops them reading what is said next — not what was said before, which they could already read.

Appendix A — Transcripts at a glance

handshake   "meshbay:mnp:handshake:v1" LP(role) LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding)
            -> HMAC-SHA256 under the GEK; role in {"client","node"}

challenge   "meshbay:mnp:challenge:v1" LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding)
            -> Ed25519 by the node key, in handshake_challenge (3.4)

admin op    "meshbay:admin:v1"         LP(op) LP(node_pk) LP(group_id) LP(subject) LP(nonce) LP(ts)
            -> Ed25519 by an operator key from the roster

join        "meshbay:join:v1"          LP(node_pk) LP(group_id) LP(user_id)
                                       LP(pk_ed25519) LP(pk_x25519) LP(nonce_s) LP(ts)
            -> Ed25519 by the joining identity

device req  "meshbay:device_req:v1"    LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519)
                                       LP(code_hash) LP(nonce_s) LP(ts)
            -> Ed25519 by the NEW device (possession only)

device add  "meshbay:device_add:v1"    LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519)
                                       LP(nonce_s) LP(ts)
            -> Ed25519 by an ALREADY-PINNED device of the same account

device hello "meshbay:device_hello:v1" LP(node_pk) LP(group_id) LP(user_id)
                                       LP(pk_ed25519) LP(nonce_s) LP(ts)
            -> Ed25519 by the device claiming this connection

chat msg    "meshbay:chat:v1"          LP(group_id) LP(epoch) LP(device) LP(nonce) LP(ct)
            -> Ed25519 by the SENDING device, over the CIPHERTEXT
            (the one transcript that binds no connection nonce: a reader of
             history has no connection. Replay is refused at rest instead,
             on the unique (device, nonce) pair)

sealed msg  aad = "<msg_type>|<group_id>"                   UTF-8, NOT length-prefixed
            key = HKDF-SHA256(GEK, salt=<none>,
                              info="meshbay:{index,ack,upload,chat_keys,roster}:v1", 32)
            -> AES-256-GCM, 12-byte random nonce per message

chat seal   aad = "chat_msg|<group_id>|<epoch>"             UTF-8, NOT length-prefixed
            key = HKDF-SHA256(epoch_key, salt=<none>,
                              info="meshbay:chat:dev:v1|<group_id>|<device_b64>", 32)
            -> AES-256-GCM, 12-byte random nonce per message
            (the two AADs above are the one place bare concatenation is used:
             every half is fixed-vocabulary or numeric, and the separator
             cannot occur in a msg_type, a group id or an epoch)

channel binding
  WebRTC    LP(offer_fp) LP(answer_fp)                    raw 32-byte SHA-256 fingerprints
  QUIC      LP(SHA-256(server_cert_der))

LP(x) = uint32be(len(x)) || x        every field, no exceptions

Appendix B — Constants

Constant Value Source
MNP_VERSION 3.4 meshbay_common/__init__.py
MNP_MIN_SUPPORTED 3.0 handshake.py
NONCE_LEN 32 bytes (both handshake nonces) handshake.py
ADMIN_CHALLENGE_TTL 120 s adminop.py
JOIN_TTL, DEVICE_TTL 120 s join.py, device.py
Pairing / device code 40 bits, Crockford base32, single use roster.py, device.py
Code lifetimes (default, settable) invitation 7 d, operator pairing 24 h, device request 1 h roster.py
MAX_PRE_PROOF_FETCHES 4 per connection webrtc/dispatch.py
MAX_JOIN_ATTEMPTS 5 per connection webrtc/admission.py
MAX_JOIN_FAILURES_WINDOW / JOIN_FAILURE_WINDOW 20 / 600 s, node-wide ”
Device attempts 5 per connection ”
MAX_DEVICES_PER_USER 5 roster.py
MAX_LINK_INVITES_PER_GROUP 20 unredeemed invitation links roster.py
PRE_HANDSHAKE_MAX_MSG / MAX_MSG 64 KiB / 64 MiB webrtc/core.py / webrtc/limits.py
CHUNK_SIZE 1 MiB webrtc/limits.py
DOWNLOAD_BUFFER_HIGH 2 MiB webrtc/files.py
MAX_UPLOAD_BYTES 8 GiB, default only — max_upload_gb overrides it per node webrtc/upload_handlers.py
Download pipeline / chunk retry (client) 8 in flight; 6 attempts, 1.5 s apart file-utils.js
Upload chunk / window / send-buffer high water (client) 48 KiB / 32 / 1 MiB transport.js
UPLOAD_ID_LEN 16 bytes, hex on the wire protocol.py
UPLOAD_PROBE_INDEX / probe timeout -1 / 5 s protocol.py, transport.js
PART_SUFFIX / ORPHAN_AFTER_SECS .part / 24 h uploads.py
DEFAULT_MAX_CONCURRENT (node-wide, per kind) 8 transfers.py
DEFAULT_MAX_PER_MEMBER (per group, per kind) 2, settable 1–32 transfers.py, webrtc/node_ops.py
GRANT_DEADLINE_SECS / IDLE_TIMEOUT_SECS 30 s / 120 s transfers.py
MAX_QUEUED_PER_MEMBER / MAX_MISSED_GRANTS 32 / 3 ”
MAX_LEASELESS_IN_FLIGHT / LEASELESS_IDLE_SECS 12 files / 60 s ”
TRANSFER_SWEEP_SECS 15 s webrtc/transfer_handlers.py
Lease watchdog (client) 60 s, then re-ask transport.js
STREAM_SEGMENT_SIZE 256 KiB webrtc/apps/streaming.py
STREAM_MAX_CREDIT 256 ”
STREAM_CREDIT_TIMEOUT / _POLL 120 s / 3 s ”
Client stream credits 24 transport.js
MAX_CONCURRENT_TRANSCODES 8 webrtc/apps/streaming.py
Link preview rate 15/conn, 60/node per 60 s; cache 1 h × 256 webrtc/chat.py
ICE gathering deadline 4 s transport.js
Signaling: max SDP, pending per user, rate 16 KiB, 3, 30/min, 15 s answer timeout api/signaling.py
GEK 256-bit, node CSPRNG crypto.py
Chat epoch key 256-bit, node CSPRNG, one per group per epoch chatbox.py
Chunk cipher AES-256-GCM, 96-bit nonce (ChaCha20-Poly1305 variant for native) webcrypto.py, crypto.py
GEK wrap X25519 + HKDF-SHA256 + AES-256-GCM, AAD = recipient public key crypto.py
Sealed message envelope HKDF-SHA256 subkey per purpose, AES-256-GCM, 96-bit random nonce groupbox.py, static/crypto.js
Chat envelope HKDF-SHA256 subkey per device per epoch, AES-256-GCM, 96-bit random nonce, Ed25519 over the ciphertext chatbox.py
File id BLAKE3, hex crypto.py
Minimum installed client GET /v1/hub/version → client.minimum meshbay-hub/api/hub.py

Appendix C — Where the authority lives

This document is descriptive: where it and the code disagree, the code is right and this is a bug. Nothing here depends on another document.

meshbay-common/   protocol.py      message types, chunk and upload codecs
                  handshake.py     transcript, proofs, version negotiation, token rules
                  groupbox.py      the sealed envelope and its purposes
                  chatbox.py       chat epoch keys, per-device subkeys, signing
                  adminop.py       the admin transcript and the operation catalogue
                  join.py          join transcript and pairing codes
                  device.py        device request / add / hello transcripts
                  crypto.py        GEK, chunk keys, ECIES wrap, BLAKE3 ids
                  webcrypto.py     the AES variants the browser can also compute

meshbay-node/     transport/webrtc_server.py   the reference implementation of MNP,
                  transport/webrtc/            assembled from these modules
                  transport/quic_server.py     the QUIC transport, in development (§5.2)
                  transport/wire.py            the one index encoder
                  transfers.py                 leases, queues, caps, leaseless reads
                  uploads.py                   partial uploads and orphaned .part files
                  roster.py, ops/, daemon.py   roster, operations, group contexts

meshbay-hub/      api/signaling.py             SDP relay limits
                  static/transport.js          the client half of every exchange above
                  static/crypto.js             the browser mirror of groupbox/chatbox
                  api/hub.py                   the minimum client version gate