MeshBay Node Protocol (MNP)
Wire version: 3.4 — meshbay_common/__init__.py (MNP_VERSION)
Oldest peer accepted: 3.0 — handshake.py (MNP_MIN_SUPPORTED)
Normative implementation: meshbay-common (protocol.py, handshake.py,
groupbox.py, chatbox.py, adminop.py, join.py, device.py, crypto.py,
webcrypto.py), meshbay-node (transport/wire.py, transport/webrtc_server.py
and transport/webrtc/, transport/quic_server.py, transfers.py, uploads.py), browser client
(meshbay-hub/static/transport.js, static/crypto.js).
Document status: descriptive specification of the protocol as implemented on
2026-09-10. It describes the protocol as it stands. Where the code and this document
disagree, the code is authoritative and this document is the thing to fix.
It is self-contained. Every rule below is given with the reason it exists, in place — a reader should never have to open a second file to find out what a rule is protecting. The only outward references are to source files, which are the authority for details this document rounds off.
1. Scope
MNP is the protocol spoken between a client (browser SPA, desktop client, CLI) and a node (the daemon that holds a group's files and the group key). It covers:
- mutual authentication of client and node, bound to the concrete transport channel;
- delivery of the group key (GEK) to an identity the node has pinned;
- the content plane — index, file chunks, uploads, video streaming, chat;
- the transfer slots a download or an upload runs under, and the caps on them;
- operator-authorized administration of the node, signed with the operator's key.
MNP is not:
- the protocol between node and hub — that is MHP (
0.1), which carries signaling, revocation push and presence, and is out of scope except where a step of MNP depends on it (§4); - a discovery mechanism. Presence comes from the hub's socket registry;
ping/pongexists only for liveness on an already open channel, because opening a connection costs a full ICE/DTLS handshake (measured 0.6–7 s).
1.1 Design invariants
These hold for every exchange described below. They are the reason the protocol has the shape it has, and each is argued where it is applied.
| # | Invariant |
|---|---|
| I1 | The hub is not trusted with content or keys. It issues JWTs and relays SDP. A valid JWT is necessary but never sufficient: every session must additionally prove possession of the group key. |
| I2 | Nothing arriving over MNP contributes key material. The node generates the group key itself, and each chat epoch key too, and wraps them for keys peers have proved they hold. No message exists by which a member hands the node key material, and none may be added. |
| I3 | Authority over the node comes from the node's own roster, never from a token claim. Privileged operations are authorized by an Ed25519 signature over a structured transcript (§10). |
| I4 | Every signed or MAC'd transcript is domain-separated and length-prefixed. Bare concatenation is forbidden: without the lengths, two different field splits produce the same bytes, and a signature over one is a signature over the other. |
| I5 | Every proof is bound to the channel it was made on. An absent channel binding is a refusal, never a degraded proof (§6.4). |
| I6 | Authentication is mutual. The node proves group-key possession over a client-chosen nonce and signs the transcript with its long-term key; the client pins that key per node (§6.1, step 11). |
| I7 | Per-group isolation. Peer registry, chat store, chat epoch keys and index are resolved per group on a multi-group node. |
| I8 | A message that must open under the group key, and does not, ends the session. Never a default, never an empty result: an unopenable enabled_apps would read as "the operator disabled every app" and an unopenable index as "the group is empty", both indistinguishable from legitimate states (§11.1, §6.6). |
| I9 | Both peers declare the protocol range they speak, and check the other's. A mismatch is a refusal with a code, not a field that turns up missing (§13). |
| I10 | A requirement only a newer peer can meet is enforced at the handshake, not per message. The alternative — an opt-in switch, "enforce it for peers that speak the new version" — leaves the permissive branch reachable on every node, and the branch left open is the one that gets used (§13). |
2. Notation
C the client (browser SPA, desktop client)
N the node daemon
H the hub (signaling and token issuance only)
X -> Y msg X sends message type `msg` to Y
[...] optional / conditional
|| byte concatenation
LP(x) len(x) as 4-byte big-endian, followed by x
b64(x) standard base64 of x, as an ASCII string
Field names are given exactly as they appear on the wire. A message is a msgpack map;
type and v are present on every message the node emits, and type on every message
it accepts.
3. Framing and encoding
3.1 Frame format
Identical on every transport:
+--------------------------------+------------------------------------------+
| length : uint32, big-endian | payload : msgpack map (use_bin_type=true) |
+--------------------------------+------------------------------------------+
4 bytes `length` bytes
- WebRTC: frames are written to a single ordered, reliable
DataChannelnamed by the client; the receiver accumulates bytes and extracts complete frames (_DataChannelBuffer). A frame may span several DataChannel messages, and one DataChannel message may carry several frames. - QUIC: each bidirectional stream carries one request/response exchange; the
handshake runs on the first stream (
_StreamBuffer, same extraction logic).
3.2 Size limits
| Bound | Value | Where |
|---|---|---|
| Max frame before the client's group-key proof | 64 KiB | PRE_HANDSHAKE_MAX_MSG |
| Max frame after the proof | 64 MiB | MAX_MSG |
| File chunk (plaintext) | 1 MiB | CHUNK_SIZE |
| Video segment (plaintext, before encryption) | 256 KiB | STREAM_SEGMENT_SIZE |
| Upload chunk sent by the browser | 48 KiB | fits the aiortc SCTP limit after msgpack overhead |
| Upload total per file | 8 GiB, operator-settable | MAX_UPLOAD_BYTES, max_upload_gb |
| Files one session may read at once without a transfer lease | 12 | MAX_LEASELESS_IN_FLIGHT (§11.2) |
The two-tier frame limit is not tidiness. A flat 64 MiB budget applied before authentication let an unauthenticated peer announce a large frame and dribble bytes into it, holding that much memory per connection for as long as it liked; a hundred such connections is the node's memory, from peers that have proved nothing. Exceeding the limit is a hard protocol error and the buffer raises rather than truncating — truncating would hand a parser a valid-looking prefix of something it never received.
3.3 Versioning field
Every message carries v. It is not what decides compatibility: the version each
side speaks and the oldest it accepts are exchanged and checked once, in the first
message each peer sends, before anything else is decided (§13.1). A v on any later
message is informational — no handler branches on it — and a peer whose range was
refused never gets to send one.
3.4 Errors
A refusal is a frame of type error:
{ "type": "error", "detail": <human-readable string>, ["code": <machine code>],
["req_id": <the request being refused>],
["upload_id" | "tr" | "file_id": <what it is about>] }
detailis authored to be safe to show a peer. Raw exception text, ffmpeg stderr and stack traces never reach the wire — they name paths on the operator's disk and versions of the operator's software, to somebody who asked for a file.codeis the same refusal in a form the client can act on. Matching ondetailis a string comparison that breaks the day someone improves the wording.req_idis stamped on every reply sent while answering a request, refusals included (§3.5). It is what makes a refusal reach the caller that earned it:erroris the one reply with no field of its own to be recognised by, so without the id it cannot be routed at all.- An upload refusal names the
upload_id, never the file: the filename is inside the seal, and quoting it back in clear would hand over exactly what sealing the upload path is for. A transfer refusal names thetr; a refusal to serve an unleased read names thefile_id, which the caller sent in clear anyway.
Codes in use:
| Family | Codes |
|---|---|
| Handshake | not_a_member (§6.3); version_too_old, version_too_new, version_unreadable (§13.1) |
| Transfers | transfer_required, bad_transfer_id, bad_transfer_size, bad_kind, not_your_transfer, too_many_queued (§11.2) |
| Upload | upload_not_sealed, no_group_key, upload_incomplete, bad_chunk_encoding, bad_chunk_index, invalid_filename, no_roots, no_such_root, no_writable_root, root_read_only, root_unavailable, no_such_directory, already_exists, not_started, too_large (§11.4) |
| Directories | root_read_only, root_unavailable (§11.5) |
Everything else refuses with detail alone. A code is added when a client has a
different thing to do about the refusal — retry, re-authenticate, offer an update —
and not merely to enumerate.
3.5 Request/response correlation
A request may carry req_id, and the node stamps it on every reply it sends while
answering that request. The client draws it, the node never interprets it, and the
match is exact.
The alternative is matching a reply to a request by arrival order, which is a guess: it
is wrong whenever two replies reorder, and it has no chance at all for the one reply
that names nothing of its own — error. A refusal that reaches no caller leaves the
request it belonged to waiting out its timeout while some unrelated request is resolved
with the refusal instead.
Node-side the id lives in a task-local (contextvars), not threaded through every send
site, and it is stamped only on messages going back to the session that asked. A
handler that also pushes to other peers — a chat broadcast, an index delta — reaches
them through their own connection, where nothing is stamped, because those messages
answer no request. An explicit req_id already on a message wins over the ambient one.
Routing, in order. A message carrying no id is either a push or a broadcast, and each has a key of its own:
| Message | Matched by |
|---|---|
any reply carrying req_id |
that id, exactly |
file_chunk |
file_id + chunk_index |
index_sync, index_delta |
queued, opened, then resolved by their id — see below |
file_upload_ack, upload error |
upload_id, against the uploader that drew it |
transfer_state |
tr, against the lease that drew it |
pong |
the echoed token |
media_meta_resp, music_meta_resp, audio_transcode_resp |
the file_id asked about |
season_meta_resp |
tmdb_id + season |
link_preview_resp |
the url |
admin_challenge |
the op field, against the pending request that named that op |
*_ack from a signed op |
type minus the _ack suffix, against the same key |
chat_msg, stream_*, *_ack broadcasts |
dedicated handlers; they are unsolicited |
Two of those keys are worth their line. Chunks are the one request that runs several at
a time interleaved with everything else, so nothing but a key of their own can identify
them. A pong is sharper still: it is sent while other traffic is in flight, so
anything less than an exact match would hand it to whatever was waiting — resolving a
history request with a message that has no messages in it, and emptying the conversation
on screen.
The index messages are the one case where the id is not enough on its own, and it is the shape any future sealed reply will have: they carry an id like everything else, but cannot be handed to their caller until they are opened, which the synchronous dispatcher cannot do. They are queued, opened, and resolved afterwards under the same id. Resolving them on arrival would give the caller an envelope — a nonce and a ciphertext — and skip the handler that decrypts.
Anything that arrives naming no request and matching no key is unsolicited and is
dropped rather than handed to a waiting caller. New request/response pairs that can
overlap in flight must therefore be distinguishable: req_id is the general
answer, and a discriminator of the message's own (file_id, url, upload_id, tr,
op) is what keeps a reply matchable without it.
4. Session model
+-------------------------+
| channel established | no MNP state yet
+-----------+-------------+
| handshake
v
+-------------------------+
refuse <-----+ version range checked | too old / too new / unreadable
| token authorized | JWT decoded, NOT authenticated
+-----------+-------------+
| handshake_challenge
v
+-------------------------+
| PRE-PROOF WINDOW | bounded: 4 fetches, 5 join attempts,
| bundles, join, device | 64 KiB frames, every event audited
+-----------+-------------+
| handshake_response (valid client proof)
v
+-------------------------+
| AUTHENTICATED | `_user_id` / `_group_id` set,
| full message set | frame limit raised to 64 MiB
+-----------+-------------+
| channel closes
v
+-------------------------+
| TORN DOWN | transfer leases released, peer
| | unregistered, tasks cancelled
+-------------------------+
States
- Unauthenticated. Only
handshakeis accepted. Anything else is answeredHandshake required. - Pre-proof. Entered when the version ranges agree, the JWT authorizes, and the node
has a group key for the group. The identity is decoded but not authenticated.
The only messages accepted are the ones a peer provably needs before it can compute
a proof:
keypair_bundle_fetch,gek_bundle_fetchandjoin_request. This window is a disclosure surface a hub that forges a JWT can reach, so it is bounded and audited (§7). - Authenticated. The client's HMAC over the handshake transcript verified. The full message set opens, and the node has answered with its own proof and signature.
- Torn down. Everything the connection held is given back, and the important word is deterministic: a transfer lease is scoped to the connection precisely so that a closed tab, a quit browser and a dropped network all arrive here and none of them needs a timer (§11.2).
A session is per (connection, group). group_id is mandatory in the handshake, so one
connection serves exactly one group; a client in two groups on one node opens two
connections. A session may additionally identify which device of the account it is,
with device_hello (§9.4) — the handshake proves the account and the group, and never
proved the device.
5. Transport establishment
MNP is transport-agnostic above the frame layer. WebRTC is the transport: it is what the browser SPA and the desktop client speak, and it implements the whole protocol. A QUIC transport is in development — see §5.2.
Every transport shares one handshake module, and a parity test fails if one grows a copy of its own. A second implementation of an authentication step is a second place for the group-key proof to be skipped.
5.1 WebRTC (browser and desktop client)
The hub relays SDP only; it never sees a DataChannel byte. Non-trickle ICE: the offer carries its candidates, with a 4 s gathering deadline after which the client offers whatever it has (host candidates are enough on a LAN).
C (browser) H (hub) N (node)
| | |
| |<===== MHP WebSocket ====>| persistent, authenticated
| | /v1/nodes/ws | Ed25519 node auth
| | |
| create offer, gather ICE (<= 4 s) |
| | |
|--- POST /v1/nodes/{node_id}/webrtc/offer --------->|
| {sdp, ice_candidates}| |
| | authorize: shared active group,
| | or an open-join group when public
| | groups are enabled; <=16 KiB SDP;
| | <=3 pending per user; 30/min
| | |
| |--- ws {webrtc_offer, |
| | peer_id, user_id, |
| | sdp} ------------>|
| | | RTCPeerConnection,
| | | answer + ICE
| |<-- ws {webrtc_answer, |
| | peer_id, sdp} ----|
|<-- 200 {sdp, ice_candidates, peer_id} -------------|
| | |
| setRemoteDescription; DTLS; SCTP; DataChannel open
| | |
|======================= MNP frames =================| hub is out of the loop
Notes that matter to MNP:
- The raw answer SDP is retained before
setRemoteDescription— Chrome stripssha-256from a multi-hash SDP, and the fingerprint is needed for the channel binding (§6.4). - The node answers offers off its WebSocket read loop: awaiting negotiation inline would stop it reading the socket for the length of one slow ICE run — running, but invisible to the hub.
- A 15 s timeout on the hub's side turns a silent node into
504, not a hung request.
5.2 QUIC (in development)
A QUIC transport is being built, for the LAN, port-forwarded and hub-less cases where signaling through the hub is unnecessary or unavailable.
It is not functional and is not a shipped feature. No client speaks it, it implements only part of the message set, and nothing in this document should be read as a statement about what it does today. What is settled is the framing and the identity model, and both are recorded here because they constrain the design of everything else:
C (native) N (node)
| |
|---- QUIC connect, ALPN "meshbay-mnp" ------->| TLS 1.3, self-signed node cert
|<---------------------------------------------| the certificate is the identity
| |
|==== stream 0 : MNP handshake =============== |
|==== stream n : one request/response each === |
The node's TLS certificate is not verified as a PKI chain — identity is established at the MNP layer, and the certificate hash is the channel binding (§6.4).
6. The handshake
One implementation for every transport: meshbay_common/handshake.py. A parity test
fails if a transport skips a step.
6.1 Full exchange
C N
| |
| 1. handshake |
| {v, v_min, token, group_id, nonce: b64(nonce_c)} |
|-------------------------------------------------------------->|
| 2. check_version() |
| - v >= our v_min |
| - v_min <= our v |
| else error{code} |
| authorize_token() |
| - EdDSA verify vs hub |
| - scope == "user" |
| - sub non-empty |
| - group_id non-empty |
| - denylist(user,jti,gp)|
| - group_id in groups[] |
| - group hosted here |
| 3. |nonce_c| >= 32 |
| 4. GEK exists for group |
| |
| 5. handshake_challenge |
| {v, v_min, nonce: b64(nonce_s), node_pk, |
| sig: b64(Ed25519(C))} (3.4, section 6.5) |
|<--------------------------------------------------------------|
| |
| 5b. client checks the node's range the same way |
| |
| ....... pre-proof window (section 7) ....................... |
| keypair_bundle_fetch / gek_bundle_fetch / join_request |
| -- the client obtains a GEK to prove with |
| ............................................................ |
| |
| 6. binding = webrtc_binding(offer_fp, answer_fp) |
| proof_c = HMAC-SHA256(GEK, T("client")) |
| |
| 7. handshake_response {v, proof: b64(proof_c)} |
|-------------------------------------------------------------->|
| 8. rebuild binding; |
| refuse if empty; |
| compare_digest(proof) |
| 9. session authenticated: |
| frame limit -> 64 MiB, |
| peer registry, audit |
| |
| 10. handshake_ack |
| {v, node_pk, proof: b64(proof_n), sig: b64(Ed25519(T)), |
| nonce, ct} |
| ct = seal(GEK, "ack", "handshake_ack", group_id, |
| {is_node_admin, enabled_apps, |
| <app>_directories, ...}) |
|<--------------------------------------------------------------|
| |
| 11. verify proof_n == HMAC(GEK, T("node")) -> else refuse |
| verify Ed25519(ack.node_pk, ack.sig, T("node")) -> refuse |
| verify ack.node_pk == challenge.node_pk -> else refuse|
| pin/compare node_pk for this node_id (TOFU) -> else refuse|
| 12. THEN open ct -> else refuse (never a default config) |
| |
|========================= session open ========================|
Step 11 is not optional politeness. A client that accepts a bare handshake_ack
without a preceding challenge, or that skips any of these checks, reopens the hole this
step exists to close: a peer that had hijacked signaling could accept the client's
proof, ignore it, and serve a forged index, forged chat history and a forged
is_node_admin flag — the last of which offers the person an administration panel on
somebody else's node.
Step 12 comes after step 11, and the order is the point. Everything in step 11 decides whether this peer is worth trusting at all; opening the payload first would mean acting on data from a peer not yet authenticated. And a payload that does not open is a refusal, not an empty configuration — see §6.6.
6.2 Transcript
T(role) = "meshbay:mnp:handshake:v1"
|| LP(role) "client" | "node"
|| LP(group_id)
|| LP(nonce_c) >= 32 bytes, client CSPRNG
|| LP(nonce_s) 32 bytes, node CSPRNG
|| LP(binding) transport channel binding, MUST be non-empty
proof = HMAC-SHA256(GEK, T(role))
The role is inside the transcript, so a client proof can never be replayed as a node
proof. nonce_c is what makes the node's proof fresh: without it a recorded
handshake_ack is replayable by an impersonating peer.
make_proof raises rather than returning a value when binding is empty or the GEK is
absent. verify_proof compares with hmac.compare_digest.
6.3 Authorization rules (authorize_token)
| Rule | Refusal | Rationale |
|---|---|---|
JWT verifies under the hub's Ed25519 public key (EdDSA) |
Invalid JWT: ... |
|
scope == "user" |
Wrong token scope |
a node-scoped daemon token must not be usable as a client token |
sub non-empty |
Token has no subject |
|
group_id non-empty |
group_id is required |
an absent group means no membership check to make; there is no default group, and a node's first group is not one |
not on the denylist for user_id, jti or group_id |
Token revoked |
all three targets, and persisted to disk: a revocation that a restart forgets is not one |
group_id ∈ token.groups |
Not a member of this group, code not_a_member |
the membership check itself — a token is proof of an account, never of a group |
group_id ∈ node.hosted_groups |
Group not hosted on this node, code not_hosted |
the hub may hand a client several nodes for one group, and only some of them host it |
AuthorizedPeer carries user_id, group_id, username, jti — and deliberately
no user public key. A key arriving in a token would be a key the hub chose, and the
node records the uploader's key in order to decide who may later delete a file: that
would let whoever issues tokens decide it instead. Identity keys are pinned by the
node's roster. The hub certifies accounts, not keys.
not_a_member is almost always a token minted before the person was added to the
group (groups is baked in at login and the hub pushes no updates), so the client
refreshes once and retries on that code rather than telling a member they are not one.
not_hosted is the client's signal to try the next node the hub offered for the
group rather than to report a failure. /v1/groups/{id}/nodes returns every node
registered for the group, in hub registration order, and that order is not a ranking:
a node listed first is not necessarily one that holds the group's files. Refusing
without a code made this indistinguishable from a refusal the reader has to act on,
and a client that stopped at the first node let one wrongly registered peer make a
group unopenable for all of its members (2026-09-11).
6.4 Channel binding
| Transport | Anchor | Construction |
|---|---|---|
| WebRTC | both DTLS certificate fingerprints | LP(offer_fp) \|\| LP(answer_fp), each the raw 32 bytes of the a=fingerprint:sha-256 line |
| QUIC | node certificate | LP(SHA-256(server_cert_der)) |
An empty binding is refused on both sides (Channel binding unavailable) — an absent
binding is never a degraded proof, because a proof that is not bound to a channel is a
proof somebody can relay. The QUIC anchor is weaker than an RFC 5705 exporter, which
aioquic does not expose: it names the server's certificate rather than the concrete
session, so on a resumed session the anchor travels with the session ticket. Stated
here and in §14.2 rather than left to be inferred.
6.5 node_pk in the challenge
The node announces its public key in handshake_challenge, before anything is proved.
This is deliberate and safe:
- a first-time joiner needs it before the ack —
join_requestsigns a transcript naming this node (§8.2), and someone who has never held the GEK cannot complete the handshake that would prove the key; - the ack proves possession and signs the transcript; the client refuses if
ack.node_pkdiffers from the announced value; - a wrong value only makes the node's own verification fail.
Since 3.4 it is also signed, in the challenge itself:
C = "meshbay:mnp:challenge:v1" || LP(group_id) || LP(nonce_c) || LP(nonce_s) || LP(binding)
sig = Ed25519(sk_node, C)
binding is the transport's channel binding (§6.4), already known when the challenge
is sent; nonce_c makes the signature fresh. So it cannot be recorded and replayed,
nor relayed through a peer with different fingerprints. The prefix is its own, so it
is never interchangeable with the ack's signature over T("node").
Why it exists: a join (§8) is sent in the pre-proof window, before the ack — so before 3.4 an invitation code went to whichever peer answered signaling, and in a group with two hosts, to whichever host answered first. With the signature, a client that knows which node key to expect can refuse to send a code anywhere else.
What it does not do: it proves that the peer holds the key it announces, not that this is the key the client wanted. A client with no expectation learns nothing more than before, and the ack remains the proof of GEK possession.
Client rule, from the node's answer and never from its version (§13): a sig that
does not verify is a refusal (Node challenge signature invalid); no sig is an
older node, whose key is proved only at the ack. A node signs whenever it has a
binding, and a node with none sends no signature rather than an unbound one — the
proof would be refused on that connection anyway.
6.6 handshake_ack fields
Three fields are in clear, and the rest travel sealed under a group-key-derived subkey (§11.1a). The split is not aesthetic: the three below are the authentication, and a client verifies them in order to decide whether to trust anything at all — including a decryption.
| Field | Meaning |
|---|---|
node_pk |
node's long-term Ed25519 public key, base64 raw 32 bytes |
proof |
HMAC(GEK, T("node")) |
sig |
Ed25519(sk_node, T("node")) |
nonce, ct |
the sealed payload; everything below is inside it |
is_node_admin |
whether this peer is the node's operator — computed from the node's own record (node_user_id), never from a hub claim |
node_user_id |
the operator's account id, when known |
node_pk_x25519 |
the node's X25519 public key, when configured |
enabled_apps |
which group applications to show. Empty/absent means "all registered ones" client-side |
<app>_directories |
each application's entry-point folders, keyed by the app's own registry name (video, music, photo, chat), always a list. This is the only form. The scalar video_root / audio_root / photo_roots fields that used to sit beside it are gone: one folder was never the general case, and two shapes for one answer meant whichever the reader consulted first decided it |
chat_directory |
where chat attachments are written. Singular because Chat genuinely has one destination; "" means the operator has not chosen |
chat_link_preview |
whether the node unfurls links posted here. Absent means on |
search_listed |
whether the reader's cross-group Search lists this group. Presentation only — the index is served identically either way. Absent means listed |
chat_epoch |
the chat epoch a client must seal under right now (§11.7). There is no chat_encrypted beside it, because there is no switch |
transfer_limits |
{download, upload} — this member's own caps in this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn; never as "unlimited", which would have the interface contradicting the node (§11.2) |
tmdb_enabled, musicbrainz_enabled |
per-group metadata lookups |
tmdb_token_customized, tmdb_language |
node-wide TMDB config; the token itself is never sent |
indexing |
{scanning, scanned_bytes, total_bytes} so a client connecting mid-scan shows progress immediately. Never a path or filename |
scan_settings |
{reconcile_interval_secs, debounce_secs} — displayed, not enforced from here |
Everything after is_node_admin is presentation state. It rides on the ack so a client
that connects after the operator configured something does not have to wait for a live
change notice to discover it.
Why this payload is sealed, and it is integrity rather than confidentiality. The
node signs T("node"), which names role, group_id, both nonces and the channel
binding — and no ack field at all. Without the seal, every value in the table above
would be authenticated by the DTLS/TLS channel and nothing else. Sealing gives them an
AEAD tag from a key the hub does not hold, which is a stronger statement than any
amount of confidentiality on the index. chat_epoch is the sharpest example: a forged
one would have a client sealing its messages under a key the group has retired.
On QUIC (§5.2) the payload is sealed but empty: it carries none of these fields, because it serves no browser. The seal is there regardless, so that one message has one shape on every transport — a field added later then has somewhere authenticated to go, rather than arriving in clear beside a sealed one.
A payload that does not open ends the session. It is not an empty configuration: an
enabled_apps that failed to open would reach the client's documented fallback — show
every registered app — which is a confident wrong answer, indistinguishable from an
operator's real choice (I8).
7. The pre-proof window
Between handshake_challenge and a valid handshake_response the peer is
authorized but not authenticated. Three message families are served there, each
because the peer provably cannot compute a proof without it. Everything else is
answered Handshake required: the dispatcher's authenticated branch begins
immediately after these three, so the table below is the window, exhaustively.
| Message | Why it must precede the proof | Bound |
|---|---|---|
keypair_bundle_fetch |
the client's own identity keys for this node live in an encrypted bundle stored on it | counts against MAX_PRE_PROOF_FETCHES = 4; audited |
gek_bundle_fetch |
the wrapped group key is what the proof is computed with | same counter |
join_request |
a first-time member holds no group key at all. Accepted after the proof as well — an operator pairing a browser is already connected — because its authority comes from the pairing code and the signature, never from the session state | 5 attempts per connection, 20 failures per 600 s node-wide |
Device linking (§9) is not in this window. device_add_request and every message
after it are answered only on an authenticated session, and the device budget of 5
attempts per connection applies there.
Exceeding the fetch budget is audited as pre-proof fetch flood and answered
Too many requests. Every fetch in this window is written to the audit log with the
message type, because this is a disclosure surface a hub that forges a JWT can reach:
the hub mints the tokens, so it can present one for any account, and what it can then
ask for is that account's encrypted keypair bundle. The bundle is useless without the
account passphrase, which is why the window is bounded and audited rather than closed
— and it closes for good when clients stop storing keypair bundles on other people's
nodes.
7.1 Identity bundles
C N
|-- keypair_bundle_fetch {v} ----------------------->|
|<- keypair_bundle_resp {v, found, |
| [bundle_enc], [bundle_enc_recovery]} ----------|
| |
| decrypt bundle_enc with the passphrase-derived bundle key,
| or bundle_enc_recovery with the recovery key
| |
|-- keypair_bundle_store {v, bundle_enc, | after minting or re-wrapping
| [bundle_enc_recovery]} ----------------------->|
|<- ack {v, detail: "keypair_bundle_stored"} --------|
| |
|-- keypair_bundle_delete {v} ---------------------->| withdraw the backup
- The bundle is opaque to the node: it is encrypted client-side under a key derived
from the account passphrase (
keyderive.js), and optionally a second copy under the account recovery key. The node stores bytes and serves them back to the sameuser_id. keypair_bundle_storeis accepted after authentication (it is not in the pre-proof list); the fetch is what happens before.- A
storeomittingbundle_enc_recoveryleaves any existing recovery copy in place. - Identity keys are per node. There is nothing to carry between nodes, and an operator who cracks the copy on their own disk gets a key that opens nothing anywhere else.
7.1a Per-account blobs (MNP 3.1)
The same shape as a keypair bundle with a different payload — playlists today
(docs/playlists.md §8). The node stores bytes it cannot read for an account it
already holds a bundle for, so this adds no new trust boundary.
C N
|-- user_blob_list {v} ---------------------------->| which kinds exist here
|<- user_blob_list_resp {v, blobs: [{kind, rev}]} --| revisions only, no payload
| |
|-- user_blob_fetch {v, kind} ---------------------->|
|<- user_blob_resp {v, kind, rev|null, |
| blob_enc|null} -------------------------------|
| |
|-- user_blob_store {v, kind, rev, blob_enc} ------->|
|<- ack {v, detail: "user_blob_stored"} ------------|
| |
|-- user_blob_delete {v, kind} --------------------->|
kindis a namespace, validated against a pattern:playlistsis the manifest,playlist:<id>is one playlist's tracks. That is what lets one playlist be rewritten without re-uploading the whole collection, and it is a pattern rather than "anything" so the table does not become a key/value store for whatever a client feels like writing.blob_encis msgpackbin, not base64. These run to hundreds of kilobytes, where base64 is a third of every write.user_idcomes from the authenticated session, never from the message. Auser_idin the body would let any member read or overwrite any other member's blob.- Caps refuse, never truncate: 64 KB for the manifest, 1 MB for one body, 8 MB per account per node, each with a stated reason. A truncating cap loses tracks silently, which is the failure the design exists to prevent.
- The 1 MB body cap is not the binding one. A browser cannot send a frame
above the negotiated
max-message-size, which aiortc fixes at 65 536, so a client tops out near 64 KB per write however high this cap is set — the same constraint that keeps uploads chunking at 48 KB. Reads are not limited that way: the node answers with a whole body of up to 1 MB. Seedocs/playlists.md§15.3. - A
fetchfor a kind never written answersnull, not an error: that is the ordinary state of a node the reader has just joined. - The node keeps no history. The client is the authority on which revision is current and holds its own copy; a node keeping older revisions would mean the node deciding, which is exactly what it must not do.
7.2 Wrapped group key
C N
|-- gek_bundle_fetch {v} ------------------------->|
|<- gek_bundle_resp {v, found, |
| [pk_eph_b64, nonce_b64, wrapped_b64]} -------|
The bundle is an ECIES wrap of the GEK for the caller's X25519 key:
sk_eph, pk_eph <- fresh X25519 keypair (node side)
shared = X25519(sk_eph, pk_recipient)
wrap_key = HKDF-SHA256(shared, salt = pk_eph, info = "meshbay:gek_wrap:v1:aes", len 32)
wrapped = AES-256-GCM(wrap_key).encrypt(nonce_96, GEK, aad = pk_recipient)
AES-GCM because WebCrypto has no ChaCha20-Poly1305; a chacha20-poly1305 variant with
info = "meshbay:gek_wrap:v1" exists for native clients. The recipient's public key is
the AEAD's associated data, so a bundle cannot be re-addressed.
found: false is the normal answer: per-member bundles are not stored, and the key is
produced on demand by the join path (§8). The node keeps one stored bundle of its own
(_node_{user_id}), which is how the daemon reloads its GEK across restarts.
8. Pairing and join
The substitution this section exists to prevent: an invite flow that fetched the invitee's public key from the hub and wrapped the group key for whatever came back would hand the group key to a hub that answered with its own — handed over by an honest inviter following the protocol exactly, with nothing anywhere looking wrong. So the key comes from its owner over an authenticated channel, and is bound to an identity by a one-time code the hub never sees.
8.1 Exchange
operator (paired) N invitee C
| | |
|-- invite_create ---->| (signed admin op, section 9)|
| {user_id, | |
| group_id, | |
| username} | |
|<- invite_result -----| |
| {code, expires_at, | |
| user_id, username}| |
| | |
|=== code delivered out of band, not via the hub ====>|
| | |
| |<-- handshake / challenge ----| nonce_s, node_pk known
| | |
| |<-- join_request -------------|
| | {group_id, pk_ed25519, |
| | pk_x25519, code, ts, sig}|
| | |
| | verify: attempts, node-wide |
| | failure window, key format,|
| | |ts - now| <= 120 s, |
| | group_id == session group, |
| | Ed25519(sig) over J, |
| | roster device lookup, |
| | consume_invite(code) |
| | |
| |-- join_result -------------->|
| | {ok, recognised, role, |
| | gek: true, |
| | pk_eph_b64, nonce_b64, |
| | wrapped_b64} |
| | |
| | client unwraps the GEK, then computes
| | the handshake proof and completes (section 6)
8.2 Join transcript
J = "meshbay:join:v1"
|| LP(node_pk_b64) the key announced in handshake_challenge
|| LP(group_id) "" for operator pairing, which is node-wide
|| LP(user_id)
|| LP(pk_ed25519_b64) the caller's own identity key
|| LP(pk_x25519_b64) the encryption key the GEK will be wrapped for
|| LP(nonce_s) the node's handshake nonce
|| LP(ts) unix seconds, decimal ASCII
sig = Ed25519(sk_ed, J)
Two properties carry the design:
- the identity key vouches for the encryption key. Both are inside one signature, which is what makes "wrap the GEK for the key the peer presented" safe;
nonce_sbinds the join to this connection, so a signed join cannot be lifted onto another.
The code is never signed and never echoed. It is a bearer secret: compared against
a stored sha256(code) and destroyed on use.
8.3 Node-side decision table
Evaluated in order (_do_join_request):
| Condition | Outcome |
|---|---|
join_attempts >= 5 on this connection |
error: Too many attempts |
>= 20 node-wide failures in 600 s |
error: Pairing temporarily locked, audited join_throttled |
| key not 32 raw bytes, or bad base64 | join_result{ok:false, reason:"invalid_keys"} |
\|ts - now\| > 120 |
stale_request |
group_id non-empty and != session group |
group_mismatch |
signature does not verify over J |
signature_invalid |
| Ed25519 key is a pinned device but the presented X25519 differs | key_changed |
| account has devices here, this key is not one | unknown_device — the way in is a device-add (§9), not a new invite |
device known, no member row, group policy open |
member row created (approved_by: "open-join") |
| device known, a pending invite exists for this user | code required even for a known device; code_required / code_invalid on failure |
| device known, not an active member of the session group, a code offered | redeemed like any code — an invitation link reaches here from someone pinned through another group, or removed and invited back; code_invalid on failure |
| device known, member row resolved | join_result{ok, recognised:true, role} + wrapped GEK |
unknown device, no code, policy open |
pin TOFU, admit, wrap (via: "tofu", audited) |
unknown device, no code, policy invite |
code_required |
| unknown device, code invalid or spent | code_invalid |
| unknown device, code valid | pin identity, set member row from the invite, wrap (via: "code", or "link") |
The member row is resolved as: this group's row, then the row for the join message's
group_id, then the node-wide ("") row — which is where an operator opening any
group finds their authority.
join_ok refuses to produce a key when roster.is_authorized(group_id, user_id) is
false: join_result{ok:true, gek:false, reason:"not_authorized_for_group"}. Hub
membership alone must not produce a key.
8.4 Pairing codes
- 8 characters, Crockford base32 (no
I,L,O,U), renderedXXXX-XXXX— 40 bits. Reading one back is case-insensitive, dashes and spaces are decoration, and the excluded letters fold onto the digits they resemble: a code read out over the phone should not be able to fail in a way the node could have absorbed. - Single use, stored only as
sha256(code). A password KDF over 40 uniformly random bits would buy nothing. - Three lifetimes, each matched to the conversation the code crosses, and all three settable by the operator:
| Code | Default | Why |
|---|---|---|
| member invitation | 7 days | it is sent by mail or message and answered whenever the other person next looks; a day dies over a weekend, and reissuing needs the inviter at a browser with the node online |
| operator pairing (§8.5) | 24 h | it crosses an SSH session — printed, then typed minutes later |
| device-add request (§9) | 1 h | read off one screen and typed into another, in one sitting |
The longer window costs little: a code is single use, bound to one account, never
seen by the hub, and 40 bits do not fall to guessing in a week against the node-wide
lockout below.
* Valid for exactly one user_id in one group — except an invitation link (§8.6).
* Guessing is bounded per connection and node-wide, and every failure is an audit event
rather than a silent grind.
* join_policy is read from the node's own configuration, never from the hub: a
hub that could declare a group open would be handing itself the key to it.
8.5 Operator pairing
The same message with group_id: "". Authority is node-wide, and the code comes from
meshbay-node operator pair over SSH — the hub never sees it. The reason it cannot is
worth stating plainly: the node cannot ask the hub which key belongs to its operator
without letting the hub answer with its own, which is the same substitution as §8's
invite flow, one level up, and it would make the hub node administrator everywhere.
There is one source of operator authority and it is the roster: a node.toml naming
an admin_pk_ed25519 is warned about at startup and never obeyed, because a second
source of authority is a second thing to get wrong.
8.6 Invitation links (kind = "link")
A code for someone who may have no account yet, so it names none: it is bound to the
first account that redeems it (invite_link_create, §10.4). What stops a stranger
holding it is not the node but the hub, which lets only the account whose verified
address the inviter named reach the node at all (MESHBAY_DESIGN.md §3.4). At the
node it is therefore a bearer code, and everything else about it is fixed:
- role
member, neveroperator; a row that says otherwise is refused, not honoured; - one named group, never node-wide, and only redeemed on a connection authenticated to that group — the account codes keep redeeming anywhere, as before;
- not by someone already an active member of that group, who would otherwise spend it for the person it was meant for; a member row that is revoked does not count;
- single use: redemption sets
used_atanduser_idin one guardedUPDATE; - at most 20 unredeemed per group (
MAX_LINK_INVITES_PER_GROUP); - cancellable before use by
invite_cancel, by aninvite_idhandle unrelated to the code; a redeemed one stays as the record of the join.
kind is a column, not an empty user_id: every query that matches invitations by
account filters on kind = "account", so a row bound to nobody can never be read as
bound to anybody (AV1). A roster from before this gains kind and invite_id by
ALTER TABLE, and its codes stay account codes.
9. Device linking
Identity keys are per node, so one person using a browser and a desktop client holds two keys on the same node. A second device is admitted by a key the node already pinned — never by the hub, which stores no user keys and therefore cannot countersign anything.
Every message in this section is answered on an authenticated session only (§7): both the device filing a request and the device approving it have completed a handshake on their own connection.
9.1 Exchange
new device D N approver A (already pinned)
| | |
| code <- random, 40 bits, displayed on D's screen |
| code_hash = sha256(code "\x1f" pk_ed "\x1f" pk_x) |
| | |
|-- device_add_request ->| |
| {pk_ed25519, | checks: account known here, |
| pk_x25519, | < 5 devices, |ts| <= 120s,|
| code_hash, ts, sig} | Ed25519(sig) over D_req |
|<- device_add_request_ack |
| {expires_at} | filed as pending, inert |
| | |
|=== the code is read off D's screen, typed into A ====>|
| | |
| |<--- device_lookup {} --------|
| |---- device_lookup_result --->|
| | {requests: [{pk_ed25519, |
| | pk_x25519, code_hash, |
| | created_at}, ...]} |
| | |
| | A recomputes sha256(code||keys)
| | for each candidate and keeps the match.
| | No match -> refuse before signing.
| | |
| |<--- device_add --------------|
| | {pk_ed25519, pk_x25519, |
| | code_hash, label, ts, |
| | sig over D_add} |
| | verify sig against EVERY |
| | live device of the account; |
| | take_device_request(hash) |
| |---- device_add_ack --------->|
| | {pk_ed25519} |
9.2 Transcripts
D_req = "meshbay:device_req:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) ||
LP(pk_x) || LP(code_hash) || LP(nonce_s) || LP(ts) signed by the NEW device
D_add = "meshbay:device_add:v1" || LP(node_pk) || LP(user_id) || LP(pk_ed) ||
LP(pk_x) || LP(nonce_s) || LP(ts) signed by a PINNED device
code_hash = sha256( code "\x1f" pk_ed25519_b64 "\x1f" pk_x25519_b64 )
D_reqis proof of possession only. It says the caller holds the keys, never that they belong to this account. The countersignature is what establishes that.D_adddeliberately omits the code: the code is a bearer secret used to find the request, never signed, never echoed. What is signed is the key pair being admitted, so a signature collected for one device cannot admit another.- Because both keys go into
code_hash, a node cannot answer the approver with a substituted key: the approver recomputes the hash from what it typed and what it was given. Nothing here rests on a human comparing digits. device_lookuptakes no hash argument. Taking one from the client was circular — the client cannot compute the hash without already knowing the keys it is asking about.
9.3 Listing and revocation
C -> N device_list {}
N -> C device_list_result {pending, devices: [{pk_ed25519, label, pinned_at,
pinned_via, added_by_pk, is_this_one}]}
C -> N device_revoke {pk_ed25519, ts, sig over D_add for the victim's keys}
N -> C device_add_ack {revoked: pk_ed25519}
Anyone may read their own devices and nobody else's. Revocation is countersigned like an addition, and the last device cannot be removed — an account with no device on a node can only return through an operator's invitation code. A revoked device is marked, not deleted, so a lost laptop stops being able to admit its replacement.
Per-connection attempt budget for every device message: 5, audited on exhaustion.
MAX_DEVICES_PER_USER is 5.
9.4 Which device is on this connection (device_hello)
C -> N device_hello {pk_ed25519, ts, sig over D_hello}
N -> C device_hello_ack {pk_ed25519}
D_hello = "meshbay:device_hello:v1" || LP(node_pk) || LP(group_id) || LP(user_id)
|| LP(pk_ed25519) || LP(nonce_s) || LP(ts)
The handshake authenticates a group membership (the group-key HMAC) and an account (the hub's token). It does not authenticate a device, and an account may hold several (§9). Without this message the node can only guess which one is talking — and it records the uploader of every file and the author of every chat message, so a guess there is an attribution the person cannot correct.
What is checked, in order: the key is a live device of this account in the node's own roster (never a token claim), the timestamp is fresh, and the signature verifies over a transcript naming this node, this group and this connection's nonce. A key that is merely well-formed proves nothing.
Idempotent for the same key and refused for a different one: a connection does not get
to change device half way through, which would let one session's uploads and messages
be attributed to two. Sending a chat message requires this to have happened
(§11.7), because the device field a receiver verifies a signature against is checked
against the connection rather than believed.
10. Operator-authorized operations
10.1 Why a signature and not a token
The hub issues JWTs, so a JWT can never establish node-level authority. Every destructive or privileged operation is authorized by an Ed25519 signature over a structured transcript, verified against the keys the node's roster records as holding operator authority — read fresh on every call, so revoking a paired browser takes effect immediately.
10.2 Two-hop exchange
C (operator) N
| |
|-- <op message> {op-specific fields} --------->|
| | cheap pre-check:
| | is there any key that
| | could authorize this?
| | (_has_admin_authority)
|<- admin_challenge ----------------------------|
| {op_id, op, subject, nonce, ts, | node keeps the authoritative
| node_pk, group_id} | copy in _admin_ops[op_id]
| |
| the client REBUILDS the transcript from the announced FIELDS
| and refuses to sign if `op`/`subject` are not what the user asked for
| |
|-- admin_response {op_id, signature, op} ----->|
| | pop(op_id) - single use
| | now - ts <= 120 s
| | rebuild A from STORED state
| | verify vs roster operator keys
| | (file_delete also accepts the
| | uploader's recorded key)
| | execute via ops
|<- <op>_ack {op-specific fields} --------------|
| |
| some acks are ALSO broadcast to every peer in the group
10.3 Transcript
A = "meshbay:admin:v1"
|| LP(op) e.g. "file_delete"
|| LP(node_pk_b64) so a signature for node A is invalid on node B
|| LP(group_id) so authority does not leak across groups on a multi-group node
|| LP(subject) what is being acted on
|| LP(nonce) 32 bytes, node CSPRNG, single use
|| LP(ts) unix seconds, TTL 120 s
sig = Ed25519(sk_operator, A)
The structure is the whole point. A challenge of 32 raw random bytes, signed blind, would be an unbound signing oracle: the signed message would name no operation, no subject, no node and no time, so a signature obtained for one purpose would be structurally valid for any other, on any node, for ever.
The node never takes a signed value off the wire. It rebuilds A from
_admin_ops[op_id]; the client rebuilds it from the announced fields. They agree by
producing the same bytes.
10.4 Operation catalogue
subject is what the client must display and match before signing. Where the ack is
broadcast, every connected peer in the group learns the change without reconnecting.
| Op | Subject | Authority | Ack | Broadcast |
|---|---|---|---|---|
file_delete |
file_id |
operator or the file's recorded uploader_pk |
file_delete_ack{file_id} |
no |
dir_delete |
path relative to the root | operator | dir_delete_ack{dir} |
no |
invite_create |
invitee user_id |
operator only (delegation designed, deferred) | invite_result{code, expires_at, user_id, username} |
no — the code is shown once |
invite_link_create |
link:<group_id>, the session's group |
operator only | invite_link_result{code, invite_id, expires_at, group_id} |
no — the code is shown once |
invite_cancel |
invite_id (32 hex) |
operator only | ack{detail: "invite_cancelled", invite_id} |
no |
member_revoke |
user_id |
operator | member_revoke_ack |
no |
member_unpin |
user_id |
operator | member_unpin_ack{user_id} |
no |
gek_rotate |
group_id |
operator | gek_rotate_ack{group_id, authorized_members, note} |
no |
apps_enabled |
the app set | operator | apps_enabled_ack{apps} |
yes |
set_scan_settings |
the interval/debounce pair | operator | set_scan_settings_ack{...} |
yes |
tmdb_config |
custom_token=yes\|no,language=... |
operator | tmdb_config_ack{token_customized, language} |
yes (never the token) |
tmdb_enabled |
enabled |
operator | tmdb_enabled_ack{enabled} |
yes |
tmdb_override |
file_id=..,tmdb_id=..,media_type=.. |
operator | tmdb_override_ack{file_id, tmdb_id, media_type} |
yes |
tmdb_rematch |
file_id=.. |
operator | tmdb_rematch_ack{file_id} |
yes |
musicbrainz_enabled |
enabled |
operator | musicbrainz_enabled_ack{enabled} |
yes |
root_add, root_remove |
the root | operator | root_add_ack / root_remove_ack |
no |
root_update |
<root>:rw=on\|off,rem=on\|off |
operator | root_update_ack |
yes |
root_eject, root_plug |
the root name | operator | root_eject_ack / root_plug_ack |
yes |
app_directories |
<app>:<dir>,<dir>,... |
operator | app_directories_ack{app, dirs} |
yes |
chat_directory |
the path | operator | chat_directory_ack{path} |
yes |
chat_link_preview |
on\|off |
operator | chat_link_preview_ack{enabled} |
yes |
search_listed |
on\|off |
operator | search_listed_ack{listed} |
yes |
transfer_limits |
d=<n>,u=<n> |
operator | transfer_limits_ack{limits} |
yes |
chat_epoch |
group_id |
operator | chat_epoch_ack{epoch} |
yes |
group_attach, group_detach |
group_id |
operator | group_attach_ack / group_detach_ack |
no |
Upload policy is not in this table, and that is the design: whether a member may
write is a property of each root (root_update), not a switch over the group. A single
group-wide flag cannot express "this library is published read-only and that folder is a
drop box", which is the ordinary arrangement.
app_directories is the only way an application's folders are set: one message
for every application, keyed by the app's own registry name, so adding an application
adds no message type, no signed op and no handler.
The three narrower ops it replaced — video_root, audio_root, photo_roots — are
gone from the catalogue. They were the same instruction three times, differing only in
the key they wrote and whether they carried a string or a list, and that shape is what
made adding an application mean adding a message type, an op, a handler and a widget.
It also meant three validation paths, and the older ones validated nothing: a typo was
stored, matched no entry, and the application showed an empty tab with no way to tell
"misconfigured" from "no files yet". One op has one validation path, and an unknown
application name is refused rather than stored.
Their storage keys survive on the node — Roster.LEGACY_DIR_KEYS still reads
video_root and friends out of group_settings — because that is a key on an
operator's disk rather than on the wire, and a node upgraded into this has to find its
own configuration.
A second family of operator messages is not signed: node_status, roster_read,
denylist_read, denylist_clear, node_settings_set, node_reload. These are gated
by is_node_admin() — the authenticated session's user_id equals the account the
node records as its own operator (node_user_id), computed from the node's own state
and never from a hub claim. Three of them only read; the other three run through the
same ops entry points as the CLI and the loopback admin API. The distinction from
the signed table above is deliberate but worth stating plainly: a signed op proves
possession of an operator key, while these prove only that the session belongs to the
operator's account, which the handshake already established.
Rules that hold across the table:
- No MNP message can activate a group key. The rule (I2) targets key material
arriving from outside, not the instruction:
gek_rotateandchat_epochare allowed precisely because the node generates the new key itself with its own CSPRNG. Initialgek-initstays local — with no group key there is no completed session to carry a signed op anyway. - There is no operation by which key material reaches the node (§12). The node wraps for a key the recipient has proved possession of, so no such message is needed — and a path that does not exist cannot be mis-authorized, which is I10 applied to a message instead of a version.
- An operator cannot revoke or unpin themselves over the connection their pin authorizes.
- Rotation is what actually removes a revoked member's access. Revocation stops the node serving the next key; the ex-member still holds the current one, and content they already downloaded stays readable. The ack says so in words.
10.5 One implementation, several front doors
The loopback admin API, the CLI and the signed MNP handlers are three thin adapters
over the same functions in meshbay_node/ops/. Those functions take the daemon
state, raise OpError, and know nothing about HTTP.
One operation with two implementations means two authorization checks, and the weaker one is the one that decides. A front door is allowed to differ in how it authenticates — a signature here, a run token on loopback, an operator's shell for the CLI — and never in what it does.
11. Content plane
Everything in this section requires an authenticated session (§4). All content is encrypted under keys derived from the GEK, so a node that serves a chunk to a session that never proved GEK possession serves ciphertext nobody can open — but the authorization check comes first regardless.
11.1 Index
The Mesh Group Index is the list of files the node shares for one group. Entries are
content-addressed: id is the BLAKE3 hash of the file.
C N
|-- index_sync {v} ------------------------->|
|<- index_sync {v, group_id, nonce, ct} -----| ct = seal(..., "index_sync", ...)
| payload: {version, entries[], |
| dirs[], roots[]} |
|
| ... operator drops files into a watched folder ...
|
|<- index_delta {v, group_id, nonce, ct} ----| pushed, unsolicited, to every
| payload: {base_version, version, | peer in this group
| additions[], deletions[], |
| updates[], roots[]} |
|
|<- index_progress {v, group_id, scanning, | every ~2 s while scanning,
| scanned_bytes, total_bytes} -----------| plus once on the return to idle
NOT sealed — see below
IndexEntry wire fields (index_entry_wire):
| Field | Meaning |
|---|---|
id |
BLAKE3 of the file content, hex |
name, path |
filename, and the virtual directory it lives in (<root>/<subpath>) |
size, type, added_at |
bytes; video\|audio\|image\|document\|archive\|other; unix seconds |
duration, width, height, thumb_hash |
filled asynchronously by enrichment |
uploader_id |
who uploaded it; null for content pre-existing on disk |
display_title, season, episode |
Videos app, parsed from the name/folder |
artist, album, track_no |
Music app, from tags or parsed |
taken_at, camera |
Photos app, best-effort from EXIF |
uploader_pk exists on the dataclass but is not in the wire dict: it is the key
the node recorded at upload time, used server-side to authorize file_delete.
Three lists, not two: updates carries entries whose id is unchanged (same content)
but whose fields changed — enrichment filling in duration/thumb_hash after the file
was first indexed with hash and size only. diff() only places an id there once it has
appeared unchanged in a prior snapshot.
dirs and roots exist because directories are not index entries. Without them a
folder just created, or one emptied, does not exist as far as the UI is concerned, and
a member cannot tell "the drive is unplugged" from "it is all still there" — an
unavailable root is listed, with its content frozen rather than hidden.
Each root is described as {name, kind, available, writable, removable, ejected} — and
never a path: a member is told what exists and whether it is readable, never where on
the operator's disk it lives. roots rides on index_delta as well as index_sync,
because a full index is only ever sent on request: without it, a root added, removed,
ejected or plugged would leave every connected client's directory table stale until
somebody reloaded the page, and the delta that tells them something changed would be the
one message unable to say what.
index_progress carries counters only, never a path or filename.
11.1a The sealed envelope
A family of messages travels sealed under a subkey derived from the GEK
(meshbay_common/groupbox.py, mirrored by sealGroup/openGroup in crypto.js).
There is one subkey per purpose, and five purposes:
key(purpose) = HKDF-SHA256(GEK, salt = <none>, info = <purpose info>, 32 bytes)
nonce = 12 random bytes, per message
ct = AES-256-GCM(key).encrypt(nonce, msgpack(payload), aad)
aad = "<msg_type>|<group_id>" UTF-8
| Purpose | info |
Seals |
|---|---|---|
index |
meshbay:index:v1 |
index_sync, index_delta |
ack |
meshbay:ack:v1 |
the handshake_ack configuration payload |
upload |
meshbay:upload:v1 |
file_upload, file_upload_ack (§11.4) |
chat_keys |
meshbay:chat_keys:v1 |
chat_keys_resp — the group's chat epoch keys (§11.7) |
roster |
meshbay:roster:v1 |
group_roster_resp — members, device keys and the evidence that admitted each (§11.7) |
chat_keys is the one whose payload is key material: a peer that has completed the
handshake holds the group key and can open it, and anything short of that gets a
ciphertext. That is the same statement the index makes, one step stronger.
salt = <none> is Python's salt=None and WebCrypto's salt: new Uint8Array(0);
RFC 5869 extracts with a zero key either way. The subkeys are purpose-separated
rather than borrowed from a file's key space — GroupIndex.serialize() reuses
chunk_key_aes with a pseudo-file ("the index as chunk 0 of a virtual index file"),
which is a hack this deliberately does not repeat.
What stays in clear, and why each one has to:
| Field | Why |
|---|---|
type, v |
the receiver must route and version-check before it can decrypt |
group_id |
already in clear in the handshake; it is the AAD and selects the key |
node_pk, proof, sig (ack) |
they are the authentication — verified before a decryption is trusted (§6.1 step 12) |
index_progress, in full |
counters only, never a path or a filename, pushed every ~2 s for the whole length of a scan. Sealing it would buy a rough library size and cost a decrypt per push |
upload_id, chunk_index (upload) |
the node routes and orders on them before it can decrypt; upload_id is client-drawn, opaque, and never an authorization input |
transfer_open / _close / _state, in full, and tr wherever it rides (§11.2) |
tr is opaque and client-drawn, bytes and chunks are numbers, and there is no filename and no path anywhere in them. Adding one to make a log line prettier is exactly the trade this envelope exists to refuse |
version and base_version are inside the payload: there is no reason to act on
a version number carried by a message that has not been authenticated.
The AAD binds a ciphertext to its message type and its group, so an index_sync
body cannot be replayed as an index_delta, nor moved between two groups on one node.
What this buys, and what it does not. It buys integrity for the ack (§6.6), and, for the index, defence in depth against one specific class of bug: a peer that has not completed the handshake being served data anyway. That bug is an authorization mistake in one branch of one handler, it is easy to write, and it is invisible until somebody reads that branch. Sealed, it leaks ciphertext instead of filenames and folder names.
The upload is the same argument in the other direction. It seals towards the node, which holds the group key for its own group and opens the payload before it decides a destination or touches the disk — so the write path is covered by the same key as the read path, and a file is never plaintext on one leg of its journey and ciphertext on the other.
It buys nothing against a network observer — DTLS/TLS already covers that — nothing against the hub, which never sees channel traffic, nothing against a member, who holds the group key, and nothing at rest: the index stays plain in the node's memory and the files stay plain on the operator's disk, which is the design. Chat messages are not sealed by this envelope: they have their own key hierarchy, per epoch and per device, because the node must be able to relay and archive a message it cannot read (§11.7). The control plane is not covered either — see §14.2.
Nonce collision, since upload is the first purpose with volume. One subkey per
purpose and a fresh 96-bit random nonce per message: at one message per 48 KiB chunk,
2³² chunks is 200 TB uploaded under a single GEK before the collision probability
reaches 2⁻³², and gek_rotate exists. Deriving the nonce from the payload instead
would be worse, not better — two chunks of identical bytes are ordinary in a file.
GroupIndex.serialize() is not a candidate for reuse here: it compresses with zstd,
which no browser can decompress (DecompressionStream offers gzip and deflate only),
so reusing it would mean shipping a WASM decoder to every client for no gain.
Failure is fatal, never degraded (I8). A client that cannot open an index message ends the session naming the message type; it never reports an empty index, because "the group has no files" is a state a real group can be in.
11.2 Transfer leases
A download is not a message. file_req asks for one chunk; a client fetching a 4 GB
film sends four thousand of them, eight in flight at a time, and nothing in that says a
transfer started or that it ended. Without an object standing for the transfer itself
there is nothing to count, and therefore nothing an operator can cap.
The lease is that object, and every transfer runs under one.
C N
|-- transfer_open {v, tr, kind, bytes, chunks} ----->| kind: "download" | "upload"
| | per-member cap first,
| | then the node-wide pool
|<- transfer_state {v, tr, state, kind, |
| used, cap, node_used, node_cap, |
| [ahead], [reason]} ----------------------------| state: granted | queued
| |
|-- file_req {..., tr} ----------------------------->| every chunk under this lease
|-- file_upload {..., tr} -------------------------->| says the transfer is alive
| |
|<- transfer_state {tr, state: "granted"} -----------| pushed when a queued lease
| | reaches the head
| |
|-- transfer_close {v, tr, reason} ----------------->| done | cancelled | paused
|<- transfer_state {tr, state: "closed", reason} ----|
Six properties carry the design, and each is a decision:
tris drawn by the client, 16 random bytes, exactly likeupload_id. Re-opening with the sametris idempotent, so a reconnect cannot charge a member twice for one transfer — and re-asking is how a client recovers a grant whose push was lost.- A lease is scoped to the connection, never to the account. It dies with the session, which is what makes the reclaim deterministic: a closed tab, a quit browser and a dropped network all arrive at the same teardown, and none of them needs a timer.
- A lease covers a job, not a file. A directory downloaded as a zip is dozens of files and one lease. One per file would deadlock against the member's own cap: the job cannot finish until it holds them all, and it can never hold more than its cap — two, by default.
- Nothing is persisted. A restart drops every session anyway, and a lease that outlived the process would be a slot nothing can release.
- Leases are counted, bytes are not. What a slot protects is concurrency — open file handles, disk seeks, the channel buffer each transfer keeps full.
- Per-member first, then node-wide. A member at their own cap queues behind their own transfers and never holds a node-wide slot a second member has none of. Reversed, whoever arrives first takes everything.
Caps. Node-wide, 8 concurrent per kind by default; per member per group, 2 by
default. A group with no value of its own gets the default, never "unlimited": reading
an absent setting as no limit would leave the node-wide cap as the only control, which
is the situation leases exist to end. The per-group value is a signed operator
operation (transfer_limits, §10.4, bounded to 1–32; zero is refused, because a member
who may not transfer at all is a member the operator revokes). The node-wide values are
daemon settings. A member's own caps ride on the handshake ack so the interface can say
"2 of your 2 slots are busy" rather than draw a spinner that explains nothing.
Queueing. One FIFO per kind. _pump walks it in arrival order and skips a
member who is at their own cap rather than stopping at them — granting strictly in
order lets one member's limit stall every other member behind them. A queued lease is
told how many are ahead of it. Beyond 32 queued per member the answer is
too_many_queued, because an unbounded queue is how a node runs out of memory politely.
used and cap on transfer_state are this member's own count and this member's own
limit in this group — the same value _has_room enforces and the same one the
handshake ack announces. Three readings of one number, and an interface that draws a
different one from the node's is an interface that offers a slot the node will queue.
Reclaim. The session teardown is the primary path and it is immediate. A sweeper runs every 15 s for whatever the teardown cannot see, and tells two failures apart:
| Situation | Timer | What happens |
|---|---|---|
| Granted, never taken up | 30 s | Back to the tail of the queue, reason: "not_taken_up" — the client died between asking and starting |
| Same, three times | — | Closed, reason: "abandoned". Without the bound the requeue is a permanent cycle: revoked, put back, granted again because there is room, revoked 30 s later, for ever |
| Granted, used, then silent | 120 s | Closed, reason: "idle", and the peer is told, so its widget can offer a resume rather than sit on a lie |
| Connection gone | none | Everything it held, at once |
The sweeper belongs to the node, not to the session that opened the first transfer. Tying it to a session would kill it when that peer left, and every other peer's abandoned lease would then never be reclaimed.
A chunk request is what "alive" looks like. tr rides on file_req and on
file_upload for exactly this: it is the only signal the node has that a granted lease
is being used. Without it the sweeper cannot tell a transfer running at 20 MB/s from a
client that asked for a slot and vanished, and it revokes both.
Pausing is releasing. A paused transfer holds nothing: the client closes the lease
with reason: "paused" and keeps its own position, and resuming asks for a new lease
and queues behind whatever is waiting now. The alternative — holding a slot while
paused — is a member who pauses three downloads and blocks the group.
Refusals. bad_transfer_id (no tr), bad_transfer_size (bytes/chunks not
numbers), bad_kind, too_many_queued, and not_your_transfer — the last for opening
or closing a tr another connection holds, which would otherwise be a denial of
service one random id away.
Reads that carry no lease
Browsing a group is never subject to a transfer slot: not the poster grid, not the covers, not opening a photo or a PDF to look at it. A member must be able to browse a group that is at capacity exactly as they browse an idle one. Thumbnails, posters and cover art never reach the check at all — they resolve out of the node's own media cache.
But "not leased" cannot mean "unbounded", or a client that simply omits tr transfers
outside every cap and the caps are decoration. So a session may read 12 distinct
files at once without a lease; the thirteenth is refused with transfer_required and
a sentence telling the person to download the file rather than preview it. An entry
already being read is always admitted, whatever the count — refusing a chunk halfway
through a photo because the limit moved is worse than never having admitted it. An
entry is released when its last chunk goes out, or after 60 s of silence, because a
viewer closed mid-file simply stops asking and says nothing.
The number is derived from what the client legitimately does, and it has to be: the music player warms a read-ahead window of 5 tracks on Wi-Fi, so playing an album has six files in flight before anyone has done anything unusual. 12 is those six at their widest, two for a photo viewer and its own prefetch in the same session, and the rest as headroom for the next feature that reads ahead. A test derives the floor from the player's own constant, so raising the client's prefetch without raising this fails in CI rather than in front of a person. Generosity is cheap here and refusal is not: the cost of being too high is a client that could have been queued and was not, and the cost of being too low is a member told to download a track they are trying to play.
Deliberately a count of files and not a byte budget: a RAW photo out of a camera is 60–80 MB and is browsing, a 40 MB archive is a download, and no size threshold separates them. What separates them is which function asked.
What leases are not
A fairness control among cooperating clients, in the company of
max_concurrent_streams — not a defence against a member determined to saturate a
node's disk. A client that lies, labelling a bulk download as a view, gets 12 files at a
time instead of its member cap. That is the residual, it is bounded, it is audited, and
the answer to the member behind it is member revoke, not a protocol rule. Stating it
is the point: a control described as a security boundary will eventually be relied on
as one.
11.3 File download
C N
|-- file_req {v, file_id, chunk_index, [tr]} ------->|
| | `tr` present: mark that lease
| | alive (§11.2)
| | index lookup; if the id is
| | not a file, try the media
| | cache (thumbnail/poster/
| | cover/transcode), sliced the
| | same way — never leased
| | `tr` absent: admit against the
| | leaseless ceiling, else
| | `transfer_required`
| | backpressure: wait while
| | bufferedAmount > 2 MiB
|<- file_chunk {v, file_id, chunk_index, |
| plaintext_size, nonce, ct} --------------------|
The client pipelines 8 chunk requests at a time and reassembles in order; a chunk that fails is retried 6 times, 1.5 s apart, because a DataChannel that hiccups mid-film should cost a pause and not the whole transfer.
Chunk encryption:
chunk_key = HKDF-SHA256(GEK, salt = none, len 32,
info = "file:" || file_hash || ":chunk:" || uint32be(chunk_index) || ":aes")
nonce = 12 random bytes
ct = AES-256-GCM(chunk_key).encrypt(nonce, plaintext) no AAD
- The
:aessuffix keeps AES keys distinct from the ChaCha20 variant (chunk_key/encrypt_chunk,infowithout the suffix) derived from the same GEK. nonceandctare msgpack binary, not base64. Every message that carries content carries it this way, and none carries it outside an AEAD.- The key is a pure function of (GEK, file hash, index), so chunks are cacheable,
resumable and requestable out of order. This is what makes a download resumable at
all: a client that stopped at chunk 900 asks for 900 next time, and no state on the
node was keeping its place.
file_idandchunk_indexride on the response because a client running several downloads at once cannot otherwise tell whose reply arrived. - Requests are handled off the message loop: the reply may wait for room on the channel, and blocking the loop for that would stall the very uploads whose acks free the buffer being waited on.
- One encoder, every transport (
protocol.file_chunk_wire/file_chunk_plaintext). A message type with one encoder per transport is a message type free to drift, and its name then says nothing about which shape will arrive. - There is no per-chunk signature, and none is needed: the AEAD tag authenticates the ciphertext under a key only group members hold, and the node authenticates itself once, in the handshake, rather than once per megabyte.
- A chunk that is not this shape aborts the download, with an error. There is no fallback that decodes it some other way: a client that guesses at a chunk it does not recognise writes its guess into the file the person is saving.
11.4 Upload
C N
|-- transfer_open {tr, kind: "upload", ...} -------->| a slot, like a download
|<- transfer_state {tr, state: "granted"} -----------| (§11.2)
| |
|-- file_upload {v, upload_id, chunk_index: -1, | the probe: "where am I?"
| total_chunks, tr, nonce, ct} ----------------->| writes nothing, reserves
| ct = seal(upload, {filename, dir, root, | nothing
| data: b""}) |
|<- file_upload_ack {v, upload_id, chunk_index: -1, |
| nonce, ct} ------------------------------------|
| ct = seal(upload, {filename, stored_as, dir, |
| resume_from}) |
| |
|-- file_upload {v, upload_id, chunk_index, | 48 KiB chunks, window 32
| total_chunks, tr, nonce, ct} ----------------->|
| ct = seal(upload, {filename, data, | open under the group key,
| dir, root}) | or refuse (upload_not_sealed)
| | filename allowlist
| | root writable and available
| | destination resolves in-group
| | chunk_index == next expected
| | running total <= max_upload_gb
| | append to <stored_name>.part
|<- file_upload_ack {v, upload_id, chunk_index, |
| nonce, ct} ------------------------------------|
| ct = seal(upload, {filename, stored_as, dir}) |
| ... repeat ... |
| | last chunk: rename .part ->
| | final, tag the index entry
| | with uploader_id/uploader_pk
- Sealed, both halves. The filename, the destination and the bytes travel inside
the seal; only
upload_id,chunk_index,total_chunksandtrstay in clear, because the node routes, orders and accounts on them before it can decrypt anything. This direction seals towards the node — it holds the group key for its own group — which is the mirror image of the index, and it means a refusal cannot quote back what it just refused. filename,dirandrootare repeated on every chunk, not sent once in a header. A hundred bytes against a 48 KiB chunk, against the alternative: a header that arrives once is state the node has to carry, and upload state that can disagree with the chunk in hand is the thing the chunk-ordering rule and the free-name rule exist to prevent.upload_idreplacesfilenameas the correlation key. It has to: matching an ack to a request by name would hand back exactly what the seal is for. It is client-drawn, opaque to the node, unique within one connection, and never an authorization input.group_idis deliberately not on the message. The session already decided which group it is on, and the node uses that as the AAD. A client naming its own group here would be choosing which key its bytes are checked against.- A chunk that does not open is refused with
upload_not_sealed, and nothing is written. One answer covers "not sealed at all" and "sealed wrong": distinguishing them tells a peer which of the two it got right. There is no plaintext fallback — a path that still accepts plaintext is not a sealed path.
Resuming, and the probe chunk. The node identifies an upload by
(member, directory, filename), so a client resuming one has to name the file.
transfer_open is the obvious place to ask and it travels in clear, which would undo
precisely what sealing this path bought. So the question is asked inside the seal
that already exists: an ordinary file_upload with no bytes and chunk_index = -1
(UPLOAD_PROBE_INDEX). Every check below has already run by then, so it cannot be used
to ask questions about a directory the caller may not write to; the node writes nothing,
reserves no name, and answers resume_from — how many chunks of this file it already
holds — inside the seal, because that is a fact about the operator's disk. resume_from
is absent from an ordinary ack, so the two are told apart without looking at
chunk_index. stored_as on a probe answer is only what is really on disk, and empty
when there is nothing: reporting the free name the node would pick would promise a
destination that the real chunk 0 may not choose. A node that does not understand the
index refuses it, which a client reads as "start from the beginning"; the client also
bounds its wait at 5 s, so a node that answers neither the probe nor its refusal costs
one restart rather than a stuck upload. Starting over is always safe, which is what
makes both fallbacks available.
The position outlives the connection. Upload state is held by the group, keyed
by (user_id, rel_dir, filename), not by the session: state on the session dies with
it, and an upload interrupted at 99% would then have to start again from zero — on a
connection flaky enough to have interrupted it once. Keyed by member as well as by
name, because a shared directory means two people can be sending IMG_1234.jpg at the
same moment and neither may inherit the other's position.
Pausing an upload is the same shape as pausing a download (§11.2): the lease goes back, and the position does not have to be remembered accurately, because the node holds it and the probe asks for it on the way back in. A pause is taken between two chunks, never inside one — the node refuses a chunk that is not the one it expects, so a chunk boundary is the only position worth having. The client also throttles on its own send buffer (1 MiB), or the whole file lands in it in seconds and the progress bar becomes a work of fiction.
An abandoned .part is reaped. It is otherwise a gigabyte of somebody else's disk
that nothing will ever finish, delete or look at again — invisible in the index, because
a .part is not an index entry. One with no upload behind it is deleted after 24 h.
Generous on purpose: the cost of waiting is disk, and the cost of being wrong is
deleting an upload somebody is still making, which is unrecoverable and looks to them
like a transfer that failed for no reason. A day covers a laptop closed
overnight, a phone in a tunnel, and a client that resumes on its next launch.
- Confinement.
filenamemust match the name allowlist (SAFE_UPLOAD_NAME); the resolved path must sit under a shared root. This is not hypothetical tidiness — a filename rendered into the interface is stored XSS if it is allowed to contain markup, and it reaches every member of the group. - No overwrite, ever. A colliding name is given a free one and the uploader is told
what it became in
stored_as; the client must use that value when referencing the file (a chat attachment, for example). Without the rule any member could replace any shared file by uploading one with the same name — silently, and with the index still pointing at what that name meant before. - Ordering. Out-of-order or replayed chunks are refused — otherwise a chunk with
index > 0 appends blindly to whatever
.partis on disk. - Types are checked after the seal opens. What comes out of an AEAD is
authenticated, not validated: it is msgpack a member wrote, and a
filenamethat is a number raises where a refusal was meant.datamust be binary: no sealed message can carry a string there, so a string is a peer doing something else entirely. - Destination. The client names the folder it is browsing, never a path: the node
resolves it against the group's own roots, which refuses
.., absolute segments and anything escaping its root, symlinks included. With several roots, the node picking one would send a member's file to a disk the operator did not intend, and that is discovered weeks later. An unknown root name is refused rather than falling back to a writable one, for the same reason. An unavailable or read-only root is a refusal, not a fallback. - No quarantine subdirectory. The file lands in the folder the sender is looking at,
not in an
uploads/folder of the node's invention: a shared directory nobody can organise is not a shared directory, and a folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. What confines an upload is the allowlist, the size cap, the chunk ordering and the no-overwrite rule — never a subdirectory. - Policy. Writability is a property of the root, and it binds the operator too — "read-only for everyone" is what makes a published library one. Enforced here rather than by hiding a button: the button is a courtesy to people who are not trying.
- The uploader key recorded is the one the node pinned for the device on this
connection (§9.4), not one the token carried — which is what makes
file_deleteauthorizable by the uploader without letting the hub delete anyone's files.
11.5 Directories and deletion
C -> N dir_create {name, dir} any member; allowlisted, confined, audited
N -> C dir_create_ack {dir}
C -> N dir_delete {dir} operator, signed (section 10)
C -> N file_delete {file_id} operator or uploader, signed
Creating a directory is not privileged — a member who can add a file may organise where
it goes — but it writes to the operator's disk, so it is audited like one and it obeys
the same per-root policy as an upload: the parent's root must be writable
(root_read_only) and available (root_unavailable). Read-only means read-only, and a
member who cannot add a file to a published library must not be able to leave empty
folders in it either.
A directory cannot be created at the virtual root: that would be adding a root, which is
operator configuration rather than a file operation. Deleting a root by name through
dir_delete is likewise refused.
dir_delete removes an empty directory and nothing else: it is never recursive, so
whatever the caller intended it cannot destroy content. The operator deletes the files
first, and sees what they are losing. The emptiness check runs before the challenge is
issued, so a non-empty directory never produces a signable transcript.
11.6 Video streaming (MSE)
Segments are produced by ffmpeg as fragmented MP4 and pushed under client-granted credit. Without the credit scheme the node hands ffmpeg's whole output to the channel as fast as it is produced, and the browser holds a multi-gigabyte film in a JavaScript array while MediaSource consumes it a segment at a time.
C N
|-- stream_req {v, file_id, start, credits, |
| audio_track?} -------------------------------->|
| | retire this session's previous
| | stream (a second request
| | means the first is over)
| | acquire a transcode slot (8)
| | ffprobe: codec, duration, the
| | audio tracks
| | spawn ffmpeg
| | -ss before -i (index seek),
| | -noaccurate_seek when the
| | video is copied
| | video: copy, or libx264 when
| | the browser cannot decode
| | audio: always AAC, 2 ch,
| | -map 0:a:<audio_track>
| | frag_keyframe+empty_moov
|<- stream_init {v, file_id, codec, duration, start, |
| audio_tracks[], audio_track, |
| subtitle_tracks[]} ----------------------------|
| |
| check MediaSource.isTypeSupported(codec) |
| |
|<- stream_data {v, file_id, segment_index, | 256 KiB, encrypted with the
| nonce, ct, plaintext_size} --------------------| same per-chunk derivation as
| ... x credits ... | a file chunk, index = segment
| |
|-- stream_more {v, n} ----------------------------->| n > 0 grants; n == 0 is a
| ... continues ... | keepalive, not a no-op
| |
|-- stream_stop {v} -------------------------------->| viewer closed
|<- stream_end {v, file_id} ------------------------ | natural end of file
| Rule | Value / behaviour |
|---|---|
Credit granted per stream_more |
clamped to [0, 256] |
| Client default window | 24 segments (6 MiB) |
A client that sends no credits |
unpaced — the node streams as fast as it can, and the client is responsible for what it buffers |
| Silence timeout | 120 s since the peer last said anything, polled every 3 s |
n == 0 |
keepalive: a viewer buffered 90 s ahead grants nothing and must still be able to say it is there |
| Concurrent transcodes | 8 node-wide, semaphore on the transport context |
| Seeking | a new stream_req with start; the previous stream is retired first, ffmpeg respawned with -ss |
| Accurate seek | off when the video is copied, on when it is re-encoded. Copied video has to begin on a keyframe and cannot be trimmed to the request; re-encoded audio can, and is. Leaving both at the default put a whole GOP of silence at the head of every seek and left sound and picture a GOP apart — with correct timestamps throughout, so nothing downstream could detect it |
start in stream_init |
the value actually used. On the copy path it is measured before the stream is served — the same seek with the same stream mapping, one frame copied under -copyts, 0.06–0.07 s — because an index seek lands on an indexed keyframe that the frame list cannot predict and that moves with which audio track is mapped. The client adds it back as SourceBuffer.timestampOffset; a subtitle cue carries the source's absolute time, so reporting anything else puts every line on screen away from the voice |
audio_tracks in stream_init |
every audio track: i (the audio ordinal, what -map 0:a:<n> takes, never the container stream index), lang, title, codec, ch. Empty for a file with no audio |
audio_track in stream_req |
which ordinal to map. Absent, out of range or malformed is the first track |
audio_track in stream_init |
the ordinal actually used, for the same reason start is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. null when the file has no audio |
| Changing track | a new stream_req at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream |
| Capability discovery | the list, not the version. A client draws its selector from audio_tracks and sends audio_track only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language |
subtitle_tracks in stream_init |
the subtitle tracks that can be shown: i (the subtitle ordinal, what -map 0:s:<n> takes, counted over every subtitle stream including the ones absent from this list), lang, title, codec. Empty for a file with no convertible subtitles |
| Which subtitle tracks are listed | text codecs only (subrip, ass, mov_text, …). Bitmap streams (PGS, VOBSUB — about a fifth of a real library) have no WebVTT without OCR, and one extracted anyway yields a header with no cues: a track that appears in the menu and shows nothing. A file whose subtitles are all bitmap reports none, exactly like a file with none |
| Why the ordinal is not the list position | the two differ whenever a bitmap stream precedes a text one. Renumbering the survivors would map 0:s:0 to the stream that cannot be decoded — which is an empty WebVTT, not an error |
| Fetching a track | subtitle_req {file_id, track} → subtitle_resp {file_id, track, hash, size, mime}; the blob is pulled by hash over file_req, the same indirection as a poster or an audio transcode. Extracted whole-file, converted to WebVTT, cached under the file's own id — so a film is extracted once, not once per viewing |
| Subtitles and seeking | nothing. The cues carry the source's absolute timestamps, so a seek and an audio-language change both leave the client's <track> untouched |
An ffmpeg failure before any output produces error: Could not stream this file;
stderr stays server-side, where it belongs — it names paths on the operator's disk and
the operator's ffmpeg build, to somebody who asked to watch a film.
Segments are encrypted with the same per-chunk derivation as a file chunk, the segment index standing in for the chunk index. There is no unencrypted streaming path: a segment of a film is content, and content does not leave a node outside an AEAD.
11.7 Chat
Chat is encrypted on the wire and at rest, under a key hierarchy of its own — not the sealed envelope of §11.1a, because the node has to relay and archive a message it cannot read, and receivers include devices that were not connected when it was sent.
C N other peers
|-- device_hello {...} ----------------------------->| §9.4 — required before a send
| |
|-- chat_keys_req {v} ------------------------------>|
|<- chat_keys_resp {v, group_id, nonce, ct} ---------| sealed, purpose "chat_keys"
| payload: {epochs: [{epoch, key}], current} | EVERY live epoch, not just now
| |
|-- chat_msg {v, format: 1, epoch, device, |
| nonce, ct, sig, sender_name, |
| [thread_id], [iteration]} -------------------->|
| | envelope checks (below)
| | persist to THIS group's store
| | broadcast ------------------->|
| | {chat_msg, sender_id,
| | sender_name, format, epoch,
| | device, nonce, ct, sig,
| | thread_id, timestamp}
| | notify the hub: group id and
| | sender id only
|<- ack {v} -----------------------------------------|
|
|-- chat_hist {v, [before], [limit <= 200]} -------->|
|<- chat_hist_resp {v, has_more, messages: [{id, |
| sender_id, sender_name, timestamp, thread_id, |
| format, epoch, device, nonce, ct, sig}]} ------|
|
|<= chat_epoch_ack {v, epoch} =======================| pushed: a new epoch is open
The key hierarchy.
epoch_key 32 bytes, generated by the NODE, one per group per epoch
device_key = HKDF-SHA256(epoch_key, salt = <none>, len 32,
info = "meshbay:chat:dev:v1|<group_id>|<device_b64>")
nonce = 12 random bytes, per message
ct = AES-256-GCM(device_key).encrypt(nonce, msgpack(payload), aad)
aad = "chat_msg|<group_id>|<epoch>"
sig = Ed25519(sk_device, "meshbay:chat:v1"
|| LP(group_id) || LP(epoch)
|| LP(device) || LP(nonce) || LP(ct))
Why a key per device and not a ratchet. A ratchet cannot deliver forward secrecy in this setting, and it is better to say so than to appear to have it. The node serves history to devices that were not present when a message was written, so it must retain and hand out each chain's earliest key — and a chain key at iteration i yields every message key from i onward by pure HKDF. Forward secrecy is then zero, and the ratchet is computing HKDF over a value every member already holds. A Signal-style sender-key implementation and a Double Ratchet were both written for this and never called by production; both have been deleted, because code nothing calls reads as an alternative somebody may reach for and its passing tests read as evidence of a protection that is not in the product.
Deriving per device by name buys what the ratchet was there for and one thing more: there is no mutable sending state at all, so nothing can be advanced twice. Two devices advancing one chain produce key and nonce reuse, which is the failure this design cannot have. Be precise about what that does not say: two clients of one account normally hold the same identity key — a second browser recovers it from the keypair bundle rather than minting a new one — so they share a device key and therefore this subkey. That is safe here only because the nonce is 96 random bits and not a counter: two independent senders under one key collide on the birthday bound, which at chat volume is unreachable, whereas two independent senders advancing one counter collide immediately. The design degrades correctly into the deployment that exists; a chain-based one would not have.
Epochs. A new epoch is opened when the set of devices that may read future
messages shrinks — member revoke, member unpin, device revoke, gek_rotate — and by
hand with the signed chat_epoch op (§10.4). Old epochs are kept and still delivered to
current members, which is what keeps history readable to the people who could already
read it. The epoch key is wrapped under the group key at delivery, never stored
under it, so rotating the group key is a re-wrap and costs nothing.
Signing is separate from encryption, and it is what establishes who said something. The signature is over the ciphertext, so it can be checked before decryption and by anyone holding the roster — including on a stored row, without the epoch key. It names the device key the node pinned, never a key the sender presents alongside the message: a signature verified against a key from the same message proves only that its sender owns some key, which any member can arrange.
One thing is deliberately not in the signing transcript: this connection's nonce.
Every other transcript in this protocol binds one; this one cannot, because a receiver
reading history has no access to the connection a message arrived on. Replay is
therefore refused by storage instead, on the unique (device, nonce) pair. A replayed
message is a validly signed copy of a real one, so nothing about the signature refuses
it; it is dropped and logged rather than raised at the sender, because the message it
duplicates is already stored and there is nothing for anyone to retry.
What the node checks before it stores or relays anything:
| Rule | Why |
|---|---|
format is the sealed one |
Plaintext is refused, always — not "accepted and marked", and not "unless a switch says otherwise". A member who can post in clear into a group whose members believe their chat is encrypted is a downgrade, and every peer that reaches this point is able to seal |
device, nonce, sig present, right lengths, ct non-empty |
a malformed envelope is refused before storage, not stored and puzzled over later |
| the connection has identified its device (§9.4) | the device field is what receivers verify against |
device == this connection's pinned device |
a device may only send as itself. A member free to name another member's key could be that member to everyone, and the signature would verify |
sender_id is taken from the authenticated session |
never read from the message; it never was |
format is a storage state as well as a wire one, so the store can distinguish rows
the wire would not accept. Nothing changes what is accepted from a peer: the sealed
form, or a refusal.
Who can verify what. group_roster_req answers any member — not only the
operator — with each device key in the group, which already-pinned key countersigned
it, and the signature, nonce and timestamp needed to rebuild what was signed. That is
what lets a member check for themselves that a message came from a device belonging to
the account it claims, instead of taking the node's sender_id on trust. The reply is
sealed under the roster purpose (§11.1a): it is the group's membership, and a peer
that has not completed the handshake has no business reading it. The node hands over
evidence and decides nothing; a node that lies here is caught by a client that has seen
the account before.
Other rules.
- The store, the peer registry and the epoch keys are resolved per group. Reading them off the shared transport context sent every group's messages to the first group's database and served them back to anyone on the node.
- The broadcast excludes this connection, not this account. The sender's other
devices are ordinary recipients: they did not compose the message and have no local
echo of it, so skipping them by
user_idleft a person's second device silently missing everything they said from the first. - The hub is told a message exists — group id and sender id, nothing else. No display name: the body is unreadable to the hub, and shipping the author's name beside it would leave the hub a per-message record of who spoke where, which is the metadata this is otherwise about not producing. The sender id stays because the hub needs it in order not to notify the author of their own message.
beforepages backwards from the newest, which is the direction a chat is read;has_moreis asked about the oldest row returned, so an empty page correctly says no.- A message that does not open is shown as unreadable, never as blank. Rendering it empty would make a message nobody can read indistinguishable from a message nobody wrote.
- The boundary, stated as everywhere else: the node operator and every current member hold the group key and therefore the epoch keys. This is the same boundary as file access, by design. What it protects against is someone who obtains the node's storage without the keystore password.
11.8 Link unfurl
C -> N link_preview_req {v, url}
N -> C link_preview_resp {v, url, ok,
[title, description, site_name, image_thumb_hash]}
The node fetches the URL because the browser cannot (CSP and CORS) and doing so would
leak every reader's IP to whatever was pasted. ok: false means "no preview" —
blocked, unreachable, or not HTML — and the client shows the bare link. Any image is
cached in the node's thumb store, so image_thumb_hash is fetched over the ordinary
file_req path. Rate limits: 15 per connection and 60 node-wide per 60 s; results
cached 1 h, 256 entries.
11.9 Metadata applications
All of these are read-only lookups against third-party services, cached node-side and keyed by content hash. None is signed: they change nothing in the node's own state. The corresponding settings are signed operations (§10.4).
| Request | Response | Notes |
|---|---|---|
media_meta_req {file_id} |
media_meta_resp {file_id, tmdb_id, title, original_title, overview, poster_thumb_hash, backdrop_thumb_hash, release_date, first_air_date, genres, vote_average, runtime, cast, director, confidence, [season, episode]} |
confidence: 0 means no confident match — the client falls back to a thumbnail-only card, it is not an error |
season_meta_req {tmdb_id, season} |
season_meta_resp {...} |
a show's single overview does not describe every season alike |
tmdb_search_req {media_type, query} |
tmdb_search_resp {results: [{id, title, year, poster}]} |
candidates for a human to pick from; never collapsed to one guess |
music_meta_req {file_id} |
music_meta_resp {file_id, ...} |
MusicBrainz; cover art cached like a poster |
audio_transcode_req {file_id} |
audio_transcode_resp {file_id, hash, size, mime} |
WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by hash over file_req |
subtitle_req {file_id, track} |
subtitle_resp {file_id, track, hash, size, mime} |
MSE decodes no in-band text track, so a subtitle travels beside the stream. track is the ordinal from stream_init.subtitle_tracks and is echoed back, because one film's two tracks are exactly the pair that can be in flight together. Fetch the result by hash over file_req |
Every one of these is keyed by the entry's file_id, never by a path: a path names
the folder a file is in, so two files in one folder — any multi-episode season — would
resolve to whichever entry the index returned first.
Posters, covers, thumbnails and transcode results all live in the media cache and are
served through file_req by their hash, sliced into chunks exactly like a real file.
11.10 Liveness
C -> N ping {v, token}
N -> C pong {v, token} the caller's token echoed back
Echoed rather than bare, so a client can match the answer to the probe it sent and measure a round trip instead of being reassured by a reply to an earlier one. This is not discovery: opening a connection in order to ping costs a full ICE/DTLS handshake, so presence in the group list comes from the hub's socket registry.
12. Message reference
Direction is C→N (client to node), N→C (node to client, solicited) or N⇒C
(node to client, unsolicited push/broadcast). "Stage" is the earliest session state in
which the node accepts the message: pre = pre-proof window, auth = after the
client's group-key proof, signed = after an admin_response verified against an
operator key.
Any request may carry req_id; every reply the node sends while answering one carries
it back (§3.5).
| Type | Dir | Stage | Purpose |
|---|---|---|---|
handshake |
C→N | — | open the session; version range, token, group, client nonce |
handshake_challenge |
N→C | — | node nonce, node's version range, announced node_pk, and since 3.4 sig over the challenge transcript |
handshake_response |
C→N | — | client's HMAC(GEK, T("client")) |
handshake_ack |
N→C | — | node proof, node signature, session parameters sealed (§11.1a) |
keypair_bundle_fetch / _resp |
C→N / N→C | pre | the caller's encrypted identity bundle |
keypair_bundle_store |
C→N | auth | back up (or re-wrap) that bundle |
keypair_bundle_delete |
C→N | auth | withdraw the backup |
user_blob_store |
C→N | auth | write one per-account blob (playlists) |
user_blob_fetch / user_blob_resp |
C→N / N→C | auth | read one, or null |
user_blob_list / user_blob_list_resp |
C→N / N→C | auth | which kinds, at what revision — never a payload |
user_blob_delete |
C→N | auth | drop one |
gek_bundle_fetch / _resp |
C→N / N→C | pre | the caller's wrapped GEK |
join_request / join_result |
C→N / N→C | pre | pin or recognise an identity; wrap the GEK |
invite_create / invite_result |
C→N / N→C | signed | issue a one-time pairing code |
invite_link_create / invite_link_result |
C→N / N→C | signed | issue a code bound to no account, for an invitation link (§8.6) |
invite_cancel |
C→N | signed | take back an unredeemed invitation link |
device_add_request / _ack |
C→N / N→C | auth | file a new device as pending |
device_lookup / device_lookup_result |
C→N / N→C | auth | candidates for the approver to hash-match |
device_add / device_add_ack |
C→N / N→C | auth | admit a device, countersigned |
device_list / device_list_result |
C→N / N→C | auth | this account's devices |
device_revoke |
C→N | auth | retire a device, countersigned (answers device_add_ack) |
index_sync |
C→N, N⇒C | auth | full index — requested, or pushed on first change |
index_delta |
N⇒C | auth | additions / deletions / updates against a base version |
index_progress |
N⇒C | auth | scan counters, no paths |
file_req / file_chunk |
C→N / N→C | auth | one encrypted chunk of a file, thumbnail or cache blob; optional tr names the lease it runs under |
file_upload / file_upload_ack |
C→N / N→C | auth | push a chunk; both halves sealed — name, destination and bytes inside, upload_id, chunk_index and tr outside. chunk_index = -1 is the resume probe (§11.4) |
transfer_open / transfer_close |
C→N | auth | ask for a transfer slot / give it back |
transfer_state |
N→C, N⇒C | auth | granted, queued (with ahead) or closed (with reason) |
device_hello / _ack |
C→N / N→C | auth | which device of this account is on this connection (§9.4) |
chat_keys_req / _resp |
C→N / N→C | auth | every live chat epoch key, sealed |
group_roster_req / _resp |
C→N / N→C | auth | this group's members and device keys, with the evidence that admitted each, sealed |
dir_create / dir_create_ack |
C→N / N→C | auth | create a folder |
dir_delete / dir_delete_ack |
C→N / N→C | signed | remove an empty folder |
file_delete / file_delete_ack |
C→N / N→C | signed | delete a file (operator or uploader) |
stream_req |
C→N | auth | start or seek an MSE stream |
stream_init / stream_data / stream_end |
N→C | auth | codec header, encrypted fMP4 segments, end |
stream_more / stream_stop |
C→N | auth | grant credit / abandon the stream |
chat_msg |
C→N, N⇒C | auth | send and fan out a message |
chat_hist / chat_hist_resp |
C→N / N→C | auth | paged history |
chat_attach |
— | auth | attachment metadata (declared, unused on the wire) |
link_preview_req / _resp |
C→N / N→C | auth | OpenGraph unfurl |
media_meta_req / _resp |
C→N / N→C | auth | TMDB metadata for one file |
season_meta_req / _resp |
C→N / N→C | auth | per-season TMDB fields |
tmdb_search_req / _resp |
C→N / N→C | auth | candidate matches for an operator |
music_meta_req / _resp |
C→N / N→C | auth | MusicBrainz metadata for one file |
audio_transcode_req / _resp |
C→N / N→C | auth | browser-playable copy of a WMA/MPC file |
subtitle_req / _resp |
C→N / N→C | auth | one embedded subtitle track as WebVTT, by cache hash |
ping / pong |
C→N / N→C | auth | liveness on an open channel |
member_revoke / _ack |
C→N / N→C | signed | stop serving the key to someone |
member_unpin / _ack |
C→N / N→C | signed | forget a pinned identity |
transfer_limits / _ack |
C→N / N⇒C | signed | per-member transfer caps for this group |
chat_epoch / _ack |
C→N / N⇒C | signed | open a new chat epoch by hand |
app_directories / _ack |
C→N / N⇒C | signed | one application's folders, keyed by app name |
chat_directory / _ack |
C→N / N⇒C | signed | where chat attachments are written |
chat_link_preview / _ack |
C→N / N⇒C | signed | whether the node unfurls posted links |
search_listed / _ack |
C→N / N⇒C | signed | whether members' cross-group Search lists this group |
root_update / _ack |
C→N / N⇒C | signed | a root's writable / removable flags |
root_eject / _ack, root_plug / _ack |
C→N / N⇒C | signed | take a removable root offline, put it back |
gek_rotate / _ack |
C→N / N→C | signed | node generates a new group key |
apps_enabled / _ack |
C→N / N⇒C | signed | which group apps are shown |
set_scan_settings / _ack |
C→N / N⇒C | signed | reconcile interval and debounce |
tmdb_config / _ack |
C→N / N⇒C | signed | node-wide TMDB token and language |
tmdb_enabled / _ack |
C→N / N⇒C | signed | per-group TMDB on/off |
tmdb_override / _ack |
C→N / N⇒C | signed | correct a wrong automatic match |
tmdb_rematch / _ack |
C→N / N⇒C | signed | drop one file's cached match |
musicbrainz_enabled / _ack |
C→N / N⇒C | signed | per-group MusicBrainz on/off |
root_add / _ack, root_remove / _ack |
C→N / N→C | signed | add or remove a shared directory |
group_attach / _ack, group_detach / _ack |
C→N / N→C | signed | start or stop hosting a group |
node_status / _ack |
C→N / N→C | auth (operator) | all groups, roots, daemon state |
roster_read / _ack |
C→N / N→C | auth (operator) | pinned identities and members |
denylist_read / _ack |
C→N / N→C | auth (operator) | current refusals |
denylist_clear / _ack |
C→N / N→C | auth (operator) | remove entries |
node_settings_set / _ack |
C→N / N→C | auth (operator) | change daemon settings |
node_reload / _ack |
C→N / N→C | auth (operator) | re-read node.toml |
ephemeral_stream |
— | — | reserved, mobile live push |
error |
N→C | any | refusal, with detail and optionally code, req_id, and the upload_id / tr / file_id it is about |
ack |
N→C | auth | generic acknowledgement (chat, keypair bundle store) |
Three messages do not exist, and their absence is a rule rather than an omission:
| Not a message | Why there is none |
|---|---|
| any request that returns the group key in plaintext | members obtain it by unwrapping their own ECIES bundle (§7.2); a node that can be asked for the key in clear is a node one authorization mistake away from handing it over |
| any message by which a member stores key material on the node | the node wraps for a key the recipient has proved possession of (I2). A member-supplied bundle is a key of somebody else's choosing |
| any content message outside an AEAD | a segment of a film and a chunk of a file are the same thing to everyone but the codec |
A node that receives an unknown type logs it and does nothing. That is what makes the additive rule of §13 work.
12.1 Transport coverage
WebRTC implements this document. It is what the browser SPA and the desktop client speak, and every message above is available on it.
QUIC is in development (§5.2): a partial message set, no client, and not a shipped feature. Nothing about it is a compatibility commitment yet.
Two rules hold across transports, and both are about there being exactly one of each message:
- One encoder per message type, shared by every transport.
file_chunkcomes frommeshbay_common.protocol,index_syncandindex_deltafrommeshbay_node/transport/wire.py, and a parity test fails if a server grows a copy of its own. Two encoders for one type is a type free to drift, with a name that no longer says which shape will arrive — and, when one of them is a sealed envelope, a second construction site that goes on sending cleartext. GroupIndex.serialize()/deserialize()describes no MNP message. It is a signed, compressed, encrypted at-rest and interchange format, and reading it as a wire contract is a mistake worth naming: the sealed envelope of §11.1a is what index messages travel under, and it is deliberately not this, because zstd decompresses in no browser.
13. Versioning and compatibility
MNP versions independently of the package version. Current: 3.4; oldest peer
accepted: 3.0 — 3.1, 3.2, 3.3 and 3.4 are all additive, so the floor does not move
with them. 3.4 adds the challenge signature (section 6.5) and invitation links
(section 8.6): invite_link_create, invite_link_result, invite_cancel, which an older
node answers as unknown messages.
The two numbers are separate on purpose. MNP_VERSION says what this build speaks;
MNP_MIN_SUPPORTED says what it will talk to, and moving the second is a decision about
whether an older peer can still do anything useful:
- Additive change → MINOR. A change is additive only if all of these hold: no existing field changes meaning, type or encoding; a peer that ignores the new field or message still behaves correctly; and the new message is only sent to a peer that advertised support, or is harmless to drop. The floor does not move.
- Breaking change → MAJOR, and then the question is where to refuse. A break confined to one exchange can be refused per message, with a code, leaving the rest of the protocol working — a peer that cannot upload can still browse, download, stream and chat. A break that touches something every session depends on cannot be confined, and the honest form is to refuse at the handshake: a stated refusal is a bug report, a feature that quietly does not work is a support case.
- Discovery from the answer, not from the version number. 3.2's audio-track
selection is the shape to copy: the node lists the tracks in
stream_init, and the client sendsaudio_trackonly when it was given a list. A peer that ignores that field would not degrade — it would serve a different language in silence, which is a wrong answer and not a missing feature — and what keeps the change additive is that no client can ever put an old node in that position. This is not the opt-in switch I10 refuses: there is no second branch on the node, which always enumerates, always honours the request and always reports the track it used. - A requirement is breaking even when its messages are additive (I10). New message types and a new optional field are additive on the wire; requiring them is not, and an opt-in switch that enforces the requirement only for peers that speak the new version leaves the permissive branch reachable on every node. That is the branch that ends up being used.
13.1 Negotiation
Both peers declare two values in the first message each sends — handshake from the
client, handshake_challenge from the node:
v the version this build speaks MNP_VERSION
v_min the oldest peer it will talk to MNP_MIN_SUPPORTED
Each side then checks the other, before anything else is decided:
| Condition | Refusal code |
|---|---|
peer's v < our v_min |
version_too_old |
peer's v_min > our v |
version_too_new |
v unparseable |
version_unreadable |
The refusal is shaped like not_a_member: a peer-safe sentence for a human and a code
the client matches on, because matching on the text is a string comparison that breaks
the day someone improves the wording. A peer that declares no v_min is read as
accepting only what it speaks.
Versions compare numerically, as (major, minor): as strings "0.9" > "0.15".
Refusing at the handshake is not enough on its own, for a client that ships its own
interface. The browser SPA is served by the hub and is therefore never out of step
with it. An installed desktop client can be, and version_too_old is a refusal in a
protocol vocabulary with nothing a person can act on. So the client asks
GET /v1/hub/version for client.minimum before it connects, and says "this
version can no longer connect" instead. An unreachable hub is deliberately not treated
as too old: a captive portal or a closed laptop must not make starting the application
impossible.
The version a peer announces is only as good as the number it ships with. Every package in the tree carries one version, and a test fails if two disagree — a client announcing a number from a different scheme sorts wherever that scheme puts it, and walks through the gate meant to stop it.
14. Security properties and stated limits
14.1 What the protocol establishes
| Property | Mechanism |
|---|---|
| A peer holds the group key | HMAC(GEK, T("client")) over a node-chosen nonce, bound to the channel |
| The node holds the group key and is the one previously seen | HMAC(GEK, T("node")) over a client-chosen nonce, plus Ed25519(sk_node, T) and a per-node pin |
| No MitM on the signaling path | both DTLS fingerprints (or the QUIC certificate hash) inside every transcript; empty binding is a refusal |
| The hub cannot read content | the group key never reaches it; chunk keys and chat epoch keys derive from or are wrapped under it |
| The hub cannot substitute a key at invite | the node wraps for a key its owner presented and signed, bound to an identity by a code the hub never sees |
| The hub cannot administer a node | privileged ops need an Ed25519 signature from a roster-pinned operator key |
| The hub cannot impersonate a device owner | device linking is countersigned by a key the node pinned; the hub stores no user keys |
| A revoked token cannot connect | denylist on user, jti and group, persisted across restarts |
| A signature cannot be repurposed | domain-separated, length-prefixed transcripts naming op, subject, node, group, nonce and time |
| One group cannot read another on the same node | per-group index, chat store, epoch keys and peer registry; the sealed envelope's AAD names the group |
| The ack's configuration is authenticated by a key the hub does not hold | sealed under ack_key; the signed handshake transcript names no ack field, so this is the only thing that authenticates them |
| A peer served before the handshake completes gets ciphertext, not filenames | index_sync/index_delta sealed under index_key — defence in depth against a serve-before-authentication mistake (§11.1a) |
| An upload's filename, destination and content never appear on the wire in clear | file_upload/file_upload_ack sealed under upload_key; upload_id replaces the filename as the correlation key |
| No message carries content outside an AEAD | file chunks and stream segments alike, under keys derived per file and per chunk |
| A body cannot be replayed as another message or into another group | AAD = "<msg_type>\|<group_id>", and "chat_msg\|<group_id>\|<epoch>" for chat |
| A chat message names the device that wrote it, checkably by any member | signature over the ciphertext with a device key the node pinned; group_roster_resp hands over the evidence to verify it without trusting the node |
| A device may only send as itself | device on chat_msg is compared with the device this connection proved (§9.4), not believed |
| A chat message cannot be replayed into the archive | unique (device, nonce) at rest; the signing transcript cannot bind a connection nonce, because history readers have no connection |
| An old chat message cannot be re-presented under a later key | the epoch is inside the AAD |
| A member cannot transfer outside the caps the operator set | a lease per job, per-member cap before node-wide pool, and the leaseless path bounded to 12 files per session (§11.2) |
| A reconnect cannot charge a member twice for one transfer | tr is drawn by the client and transfer_open is idempotent on it |
| A refusal reaches the request it refuses | req_id stamped on every reply, including error (§3.5) |
14.2 What it deliberately does not establish
- The browser SPA is served by the hub. A hub that ships malicious client code can read a pairing code out of the page, or the group key out of memory. This is accepted permanently for the browser client and is what the native client removes. Keep the two attacks apart: the pairing code defeats a hub that lies in its directory — silent, undetectable, per-request — not one that rewrites the client, which is an artifact that can be inspected and compared.
- An invitation link's code is a bearer code (§8.6). The node admits whoever brings it first; what restricts who can bring it is the hub, which lets only the addressed account reach the node — a rule an active hub does not have to keep. The challenge signature (§6.5) keeps the code from reaching any node but the issuer; it does not keep it from a hub the inviter asked to mail it, which then holds it.
- The pre-proof window is a disclosure surface. A hub that forges a JWT can fetch a member's encrypted keypair bundle. It is bounded, audited, and closes when clients stop storing bundles on other people's nodes (§7).
- Transfer leases are fairness, not security. They bound cooperating clients. A
client that lies — labelling a bulk download as a view — gets 12 files at a time
instead of its member cap; that is the residual, it is bounded and audited, and the
answer to a member determined to saturate a node's disk is
member revoke(§11.2). - The QUIC transport is in development (§5.2) and establishes nothing yet. When it does, its channel binding is a certificate hash rather than an RFC 5705 exporter, which is weaker: on a resumed session the anchor travels with the session ticket.
- Nothing is confidential from a member, or at rest on the operator's disk. The sealed envelope (§11.1a) is defence in depth against our own next serve-before-authentication bug; it is not a claim against anyone who holds the group key. Chat is the one thing encrypted at rest as well, which protects against someone who obtains the node's storage without the keystore password — and against nobody who holds the keys.
- The control plane is still in clear. Sealing covers content: the index, the ack,
file chunks, stream segments, uploads, the chat keys and the group roster. It does not
cover the admin and configuration acks (
app_directories_ack,root_*_ackand the rest), which carry the same folder names the sealed index carries; the media-metadata replies (media_meta_resp,music_meta_resp,link_preview_resp), which carry titles, artists and synopses;node_status_ack, which carries absolute paths on the operator's disk to an operator session; the identity replies (roster_read_ack,device_list_result); orinvite_resultandinvite_link_result, which carry a pairing code. All are inside DTLS/TLS and none reaches the hub, but none is behind the group key. - Transfer messages are in clear on purpose, and that is a deliberate line rather
than an omission:
tris opaque,bytesandchunksare numbers, and there is no filename and no path anywhere in them (§11.1a). - Rotation is the only thing that removes access. Revoking a member stops the node serving the next key; the current key and anything already downloaded stay readable. A chat epoch is opened at the same time, which stops them reading what is said next — not what was said before, which they could already read.
Appendix A — Transcripts at a glance
handshake "meshbay:mnp:handshake:v1" LP(role) LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding)
-> HMAC-SHA256 under the GEK; role in {"client","node"}
challenge "meshbay:mnp:challenge:v1" LP(group_id) LP(nonce_c) LP(nonce_s) LP(binding)
-> Ed25519 by the node key, in handshake_challenge (3.4)
admin op "meshbay:admin:v1" LP(op) LP(node_pk) LP(group_id) LP(subject) LP(nonce) LP(ts)
-> Ed25519 by an operator key from the roster
join "meshbay:join:v1" LP(node_pk) LP(group_id) LP(user_id)
LP(pk_ed25519) LP(pk_x25519) LP(nonce_s) LP(ts)
-> Ed25519 by the joining identity
device req "meshbay:device_req:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519)
LP(code_hash) LP(nonce_s) LP(ts)
-> Ed25519 by the NEW device (possession only)
device add "meshbay:device_add:v1" LP(node_pk) LP(user_id) LP(pk_ed25519) LP(pk_x25519)
LP(nonce_s) LP(ts)
-> Ed25519 by an ALREADY-PINNED device of the same account
device hello "meshbay:device_hello:v1" LP(node_pk) LP(group_id) LP(user_id)
LP(pk_ed25519) LP(nonce_s) LP(ts)
-> Ed25519 by the device claiming this connection
chat msg "meshbay:chat:v1" LP(group_id) LP(epoch) LP(device) LP(nonce) LP(ct)
-> Ed25519 by the SENDING device, over the CIPHERTEXT
(the one transcript that binds no connection nonce: a reader of
history has no connection. Replay is refused at rest instead,
on the unique (device, nonce) pair)
sealed msg aad = "<msg_type>|<group_id>" UTF-8, NOT length-prefixed
key = HKDF-SHA256(GEK, salt=<none>,
info="meshbay:{index,ack,upload,chat_keys,roster}:v1", 32)
-> AES-256-GCM, 12-byte random nonce per message
chat seal aad = "chat_msg|<group_id>|<epoch>" UTF-8, NOT length-prefixed
key = HKDF-SHA256(epoch_key, salt=<none>,
info="meshbay:chat:dev:v1|<group_id>|<device_b64>", 32)
-> AES-256-GCM, 12-byte random nonce per message
(the two AADs above are the one place bare concatenation is used:
every half is fixed-vocabulary or numeric, and the separator
cannot occur in a msg_type, a group id or an epoch)
channel binding
WebRTC LP(offer_fp) LP(answer_fp) raw 32-byte SHA-256 fingerprints
QUIC LP(SHA-256(server_cert_der))
LP(x) = uint32be(len(x)) || x every field, no exceptions
Appendix B — Constants
| Constant | Value | Source |
|---|---|---|
MNP_VERSION |
3.4 |
meshbay_common/__init__.py |
MNP_MIN_SUPPORTED |
3.0 |
handshake.py |
NONCE_LEN |
32 bytes (both handshake nonces) | handshake.py |
ADMIN_CHALLENGE_TTL |
120 s | adminop.py |
JOIN_TTL, DEVICE_TTL |
120 s | join.py, device.py |
| Pairing / device code | 40 bits, Crockford base32, single use | roster.py, device.py |
| Code lifetimes (default, settable) | invitation 7 d, operator pairing 24 h, device request 1 h | roster.py |
MAX_PRE_PROOF_FETCHES |
4 per connection | webrtc/dispatch.py |
MAX_JOIN_ATTEMPTS |
5 per connection | webrtc/admission.py |
MAX_JOIN_FAILURES_WINDOW / JOIN_FAILURE_WINDOW |
20 / 600 s, node-wide | ” |
| Device attempts | 5 per connection | ” |
MAX_DEVICES_PER_USER |
5 | roster.py |
MAX_LINK_INVITES_PER_GROUP |
20 unredeemed invitation links | roster.py |
PRE_HANDSHAKE_MAX_MSG / MAX_MSG |
64 KiB / 64 MiB | webrtc/core.py / webrtc/limits.py |
CHUNK_SIZE |
1 MiB | webrtc/limits.py |
DOWNLOAD_BUFFER_HIGH |
2 MiB | webrtc/files.py |
MAX_UPLOAD_BYTES |
8 GiB, default only — max_upload_gb overrides it per node |
webrtc/upload_handlers.py |
| Download pipeline / chunk retry (client) | 8 in flight; 6 attempts, 1.5 s apart | file-utils.js |
| Upload chunk / window / send-buffer high water (client) | 48 KiB / 32 / 1 MiB | transport.js |
UPLOAD_ID_LEN |
16 bytes, hex on the wire | protocol.py |
UPLOAD_PROBE_INDEX / probe timeout |
-1 / 5 s |
protocol.py, transport.js |
PART_SUFFIX / ORPHAN_AFTER_SECS |
.part / 24 h |
uploads.py |
DEFAULT_MAX_CONCURRENT (node-wide, per kind) |
8 | transfers.py |
DEFAULT_MAX_PER_MEMBER (per group, per kind) |
2, settable 1–32 | transfers.py, webrtc/node_ops.py |
GRANT_DEADLINE_SECS / IDLE_TIMEOUT_SECS |
30 s / 120 s | transfers.py |
MAX_QUEUED_PER_MEMBER / MAX_MISSED_GRANTS |
32 / 3 | ” |
MAX_LEASELESS_IN_FLIGHT / LEASELESS_IDLE_SECS |
12 files / 60 s | ” |
TRANSFER_SWEEP_SECS |
15 s | webrtc/transfer_handlers.py |
| Lease watchdog (client) | 60 s, then re-ask | transport.js |
STREAM_SEGMENT_SIZE |
256 KiB | webrtc/apps/streaming.py |
STREAM_MAX_CREDIT |
256 | ” |
STREAM_CREDIT_TIMEOUT / _POLL |
120 s / 3 s | ” |
| Client stream credits | 24 | transport.js |
MAX_CONCURRENT_TRANSCODES |
8 | webrtc/apps/streaming.py |
| Link preview rate | 15/conn, 60/node per 60 s; cache 1 h × 256 | webrtc/chat.py |
| ICE gathering deadline | 4 s | transport.js |
| Signaling: max SDP, pending per user, rate | 16 KiB, 3, 30/min, 15 s answer timeout | api/signaling.py |
| GEK | 256-bit, node CSPRNG | crypto.py |
| Chat epoch key | 256-bit, node CSPRNG, one per group per epoch | chatbox.py |
| Chunk cipher | AES-256-GCM, 96-bit nonce (ChaCha20-Poly1305 variant for native) | webcrypto.py, crypto.py |
| GEK wrap | X25519 + HKDF-SHA256 + AES-256-GCM, AAD = recipient public key | crypto.py |
| Sealed message envelope | HKDF-SHA256 subkey per purpose, AES-256-GCM, 96-bit random nonce | groupbox.py, static/crypto.js |
| Chat envelope | HKDF-SHA256 subkey per device per epoch, AES-256-GCM, 96-bit random nonce, Ed25519 over the ciphertext | chatbox.py |
| File id | BLAKE3, hex | crypto.py |
| Minimum installed client | GET /v1/hub/version → client.minimum |
meshbay-hub/api/hub.py |
Appendix C — Where the authority lives
This document is descriptive: where it and the code disagree, the code is right and this is a bug. Nothing here depends on another document.
meshbay-common/ protocol.py message types, chunk and upload codecs
handshake.py transcript, proofs, version negotiation, token rules
groupbox.py the sealed envelope and its purposes
chatbox.py chat epoch keys, per-device subkeys, signing
adminop.py the admin transcript and the operation catalogue
join.py join transcript and pairing codes
device.py device request / add / hello transcripts
crypto.py GEK, chunk keys, ECIES wrap, BLAKE3 ids
webcrypto.py the AES variants the browser can also compute
meshbay-node/ transport/webrtc_server.py the reference implementation of MNP,
transport/webrtc/ assembled from these modules
transport/quic_server.py the QUIC transport, in development (§5.2)
transport/wire.py the one index encoder
transfers.py leases, queues, caps, leaseless reads
uploads.py partial uploads and orphaned .part files
roster.py, ops/, daemon.py roster, operations, group contexts
meshbay-hub/ api/signaling.py SDP relay limits
static/transport.js the client half of every exchange above
static/crypto.js the browser mirror of groupbox/chatbox
api/hub.py the minimum client version gate