diff options
Diffstat (limited to 'docs/MESHBAY_NODE_PROTOCOL.md')
| -rw-r--r-- | docs/MESHBAY_NODE_PROTOCOL.md | 143 |
1 files changed, 98 insertions, 45 deletions
diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index 8de731e..34e2464 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1,14 +1,14 @@ # MeshBay Node Protocol (MNP) **Wire version:** `4.0` — `meshbay_common/__init__.py` (`MNP_VERSION`) -**Oldest peer accepted:** `3.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) +**Oldest peer accepted:** `4.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) **Normative implementation:** `meshbay-common` (`protocol.py`, `handshake.py`, `groupbox.py`, `chatbox.py`, `adminop.py`, `join.py`, `device.py`, `crypto.py`, -`webcrypto.py`), `meshbay-node` (`transport/wire.py`, `transport/webrtc_server.py` +`webcrypto.py`, `tokens.py`), `meshbay-node` (`transport/wire.py`, `transport/webrtc_server.py` and `transport/webrtc/`, `transport/quic_server.py`, `transfers.py`, `uploads.py`), browser client (`meshbay-hub/static/transport.js`, `static/crypto.js`). **Document status:** descriptive specification of the protocol as implemented on -2026-09-10. It describes the protocol as it stands. Where the code and this document +2026-09-28. It describes the protocol as it stands. Where the code and this document disagree, the code is authoritative and this document is the thing to fix. **It is self-contained.** Every rule below is given with the reason it exists, in @@ -152,10 +152,13 @@ Codes in use: | Family | Codes | |---|---| -| Handshake | `not_a_member` (§6.3); `version_too_old`, `version_too_new`, `version_unreadable` (§13.1) | -| Transfers | `transfer_required`, `bad_transfer_id`, `bad_transfer_size`, `bad_kind`, `not_your_transfer`, `too_many_queued` (§11.2) | -| Upload | `upload_not_sealed`, `no_group_key`, `upload_incomplete`, `bad_chunk_encoding`, `bad_chunk_index`, `invalid_filename`, `no_roots`, `no_such_root`, `no_writable_root`, `root_read_only`, `root_unavailable`, `no_such_directory`, `already_exists`, `not_started`, `too_large` (§11.4) | +| Handshake | `not_a_member`, `not_hosted`, `wrong_node` (§6.3); `version_too_old`, `version_too_new`, `version_unreadable` (§13.1) | +| Transfers | `transfer_required`, `lease_not_granted`, `bad_transfer_id`, `bad_transfer_size`, `bad_kind`, `not_your_transfer`, `too_many_queued` (§11.2) | +| Upload | `upload_not_sealed`, `no_group_key`, `lease_not_granted`, `upload_incomplete`, `bad_chunk_encoding`, `bad_chunk_index`, `invalid_filename`, `no_roots`, `no_such_root`, `no_writable_root`, `root_read_only`, `root_unavailable`, `no_such_directory`, `already_exists`, `not_started`, `too_large` (§11.4) | | Directories | `root_read_only`, `root_unavailable` (§11.5) | +| Chat | `chat_too_large`, `chat_rate_limited` (§11.7) | +| Operator controls | `not_operator` (§10.4) | +| Metadata | `transcode_not_applicable`, `tmdb_search_rate_limited` (§11.9) | Everything else refuses with `detail` alone. A code is added when a client has a different thing to *do* about the refusal — retry, re-authenticate, offer an update — @@ -305,7 +308,9 @@ whatever it has (host candidates are enough on a LAN). | | authorize: shared active group, | | or an open-join group when public | | groups are enabled; <=16 KiB SDP; - | | <=3 pending per user; 30/min + | | <=32 pending per account; per + | | account and node: burst 120, + | | refill 2/s; 600/min per address | | | | |--- ws {webrtc_offer, | | | peer_id, user_id, | @@ -375,7 +380,11 @@ fails if a transport skips a step. | else error{code} | | authorize_token() | | - EdDSA verify vs hub | + | - aud == MNP_AUD, | + | exp/sub/scope present| | - scope == "user" | + | - node claim, if set, | + | == this node's key | | - sub non-empty | | - group_id non-empty | | - denylist(user,jti,gp)| @@ -463,8 +472,8 @@ absent. `verify_proof` compares with `hmac.compare_digest`. |---|---|---| | JWT verifies under the hub's Ed25519 public key (`EdDSA`) | `Invalid JWT: ...` | | | `aud == MNP_AUD`, and `exp`/`sub`/`scope` present (MNP 4.0) | `Invalid JWT: ...` | the member presents a short-lived **node-audience** token (`POST /v1/nodes/mnp-token`), not its hub session token — the operator holds whatever is presented, and the session token opens the hub API. The two audience strings are in `meshbay_common/tokens.py` | -| `node` claim, when set, equals this node's key (MNP 4.0) | `Token is not for this node`, code `wrong_node` | the token names the node it was minted for, so one captured by node A's operator cannot be replayed to node B (E10). A token naming no node is accepted — the hub mints an unbound one only for the requester | | `scope == "user"` | `Wrong token scope` | a node-scoped daemon token must not be usable as a client token | +| `node` claim, when set, equals this node's key (MNP 4.0) | `Token is not for this node`, code `wrong_node` | the token names the node it was minted for, so one captured by node A's operator cannot be replayed to node B (E10). A token naming no node is accepted — the hub mints an unbound one only for the requester | | `sub` non-empty | `Token has no subject` | | | `group_id` non-empty | `group_id is required` | an absent group means no membership check to make; there is no default group, and a node's first group is not one | | not on the denylist for `user_id`, `jti` **or** `group_id` | `Token revoked` | all three targets, and persisted to disk: a revocation that a restart forgets is not one | @@ -472,14 +481,17 @@ absent. `verify_proof` compares with `hmac.compare_digest`. | `group_id ∈ node.hosted_groups` | `Group not hosted on this node`, code `not_hosted` | the hub may hand a client several nodes for one group, and only some of them host it | `AuthorizedPeer` carries `user_id`, `group_id`, `username`, `jti` — and deliberately -**no user public key**. A key arriving in a token would be a key the hub chose, and the +**no user public key**. `username` is read from a `username` claim that neither the MNP +token nor the hub session token carries, so it is empty in practice. A key arriving in a token would be a key the hub chose, and the node records the uploader's key in order to decide who may later delete a file: that would let whoever issues tokens decide it instead. Identity keys are pinned by the node's roster. The hub certifies accounts, not keys. -`not_a_member` is almost always a token minted before the person was added to the -group (`groups` is baked in at login and the hub pushes no updates), so the client -refreshes once and retries on that code rather than telling a member they are not one. +`not_a_member` means the hub did not count this account a member of the group when it +minted the token. The MNP token is minted for each connection, from the membership the +hub holds at that moment, so a stale `groups` claim is no longer the usual cause; the +client still refreshes its session once and retries on that code before telling someone +who was just invited that they are not a member. `not_hosted` is the client's signal to try the **next** node the hub offered for the group rather than to report a failure. `/v1/groups/{id}/nodes` returns every node @@ -537,10 +549,13 @@ this is the key the client wanted. A client with no expectation learns nothing m than before, and the ack remains the proof of GEK possession. Client rule, from the node's answer and never from its version (§13): a `sig` that -does not verify is a refusal (`Node challenge signature invalid`); no `sig` is an -older node, whose key is proved only at the ack. A node signs whenever it has a -binding, and a node with none sends no signature rather than an unbound one — the -proof would be refused on that connection anyway. +does not verify is a refusal (`Node challenge signature invalid`); no `sig` leaves the +key unproved until the ack, and a client holding an invitation-link code then refuses +to send it. A node signs whenever it has a binding, and a node with none sends no +signature rather than an unbound one — the proof would be refused on that connection +anyway. With the floor at 4.0 every peer a client can reach signs, so an absent `sig` +no longer means an older node: the client's handling of it is a branch only a lowered +floor could make reachable again. ### 6.6 `handshake_ack` fields @@ -567,7 +582,7 @@ at all — including a decryption. | `transfer_limits` | `{download, upload}` — this member's own caps in this group, so the interface can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent reads as "no limit known" and the hint is not drawn; never as "unlimited", which would have the interface contradicting the node (§11.2) | | `tmdb_enabled`, `musicbrainz_enabled` | per-group metadata lookups | | `tmdb_token_customized`, `tmdb_language` | node-wide TMDB config; the token itself is never sent | -| `indexing` | `{scanning, scanned_bytes, total_bytes}` so a client connecting mid-scan shows progress immediately. Never a path or filename | +| `indexing` | the `index_progress` counters (§11.1) so a client connecting mid-scan shows progress immediately. Never a path, a filename or a root name | | `scan_settings` | `{reconcile_interval_secs, debounce_secs}` — displayed, not enforced from here | Everything after `is_node_admin` is presentation state. It rides on the ack so a client @@ -1045,8 +1060,8 @@ effect immediately. | | now - ts <= 120 s | | rebuild A from STORED state | | verify vs roster operator keys - | | (file_delete also accepts the - | | uploader's recorded key) + | | (file_delete also accepts any + | | live device of the uploader) | | execute via ops |<- <op>_ack {op-specific fields} --------------| | | @@ -1083,7 +1098,7 @@ broadcast, every connected peer in the group learns the change without reconnect | Op | Subject | Authority | Ack | Broadcast | |---|---|---|---|---| -| `file_delete` | `file_id` | operator **or** the file's recorded `uploader_pk` | `file_delete_ack{file_id}` | no | +| `file_delete` | `file_id` | operator **or** any non-revoked device of the uploading account (`uploader_id`, looked up in the roster); the recorded `uploader_pk` only when the roster cannot answer | `file_delete_ack{file_id}` | no | | `dir_delete` | path relative to the root | operator | `dir_delete_ack{dir}` | no | | `invite_create` | invitee `user_id` | operator only (delegation designed, deferred) | `invite_result{code, expires_at, user_id, username}` | no — the code is shown once | | `invite_link_create` | `link:<group_id>`, the session's group | operator only | `invite_link_result{code, invite_id, expires_at, group_id}` | no — the code is shown once | @@ -1134,13 +1149,16 @@ own configuration. A second family of operator messages is **not** signed: `node_status`, `roster_read`, `denylist_read`, `denylist_clear`, `node_settings_set`, `node_reload`. These are gated -by `is_node_admin()` — the authenticated session's `user_id` equals the account the -node records as its own operator (`node_user_id`), computed from the node's own state -and never from a hub claim. Three of them only read; the other three run through the -same `ops` entry points as the CLI and the loopback admin API. The distinction from -the signed table above is deliberate but worth stating plainly: a signed op proves -possession of an operator *key*, while these prove only that the session belongs to the -operator's *account*, which the handshake already established. +by `_operator_device()`, which asks two things: the authenticated session's `user_id` +is the account the node records as its own (`node_user_id`, `is_node_admin()`), **and** +the device on this connection has proved, with `device_hello` (§9.4), a key the roster +holds as an operator. Anything else is refused with code `not_operator`. The first +alone would be a claim in a token the hub issued, and a hub that can name the operator +is a hub that can be one; the second is what it cannot forge, since it holds no user +keys. Three of them only read; the other three run through the same `ops` entry points +as the CLI and the loopback admin API. The distinction from the signed table above: +a signed op proves possession of an operator key for *this* operation, while these +prove it once per connection, through the device the connection identified. Rules that hold across the table: @@ -1199,7 +1217,9 @@ content-addressed: `id` is the BLAKE3 hash of the file. | updates[], roots[]} | | |<- index_progress {v, group_id, scanning, | every ~2 s while scanning, - | scanned_bytes, total_bytes} -----------| plus once on the return to idle + | scanned_bytes, total_bytes, | plus once on the return to idle + | files_done, files_total, kind, | + | root_pos, queued} ---------------------| NOT sealed — see below ``` @@ -1237,7 +1257,11 @@ ejected or plugged would leave every connected client's directory table stale un somebody reloaded the page, and the delta that tells them something changed would be the one message unable to say what. -`index_progress` carries counters only, never a path or filename. +`index_progress` carries counters only, never a path, a filename or a root name: +`kind` is what the indexer is doing (`scan`, `rescan`, `reconcile`, `watch`, or `""` +when idle), `root_pos` is a position in the `roots` table the member already opened +from the sealed index (`-1` when none), and `queued` is the number of roots waiting +their turn, not their names. ### 11.1a The sealed envelope @@ -1419,6 +1443,19 @@ numbers), `bad_kind`, `too_many_queued`, and `not_your_transfer` — the last fo or closing a `tr` another connection holds, which would otherwise be a denial of service one random id away. +**A lease is what the node granted, not what the client called it.** `tr` is drawn by +the client, so on `file_req` and `file_upload` it is a claim, resolved against the +node's own record for this connection: + +| `tr` names | `file_req` | `file_upload` | +|---|---|---| +| a lease of this connection, **granted** | served as a leased transfer; marks the lease alive | accepted; marks the lease alive | +| a lease of this connection, still **queued** | refused, `lease_not_granted` — reading while queued is the cap not applying | refused, `lease_not_granted` | +| nothing this connection holds, or no `tr` | a leaseless read, counted against the ceiling below | accepted, bounded by the upload protections (§11.4) — this is what a reconnect looks like, with the old connection's leases gone and the client re-opening them | + +Read as a bare presence check, the field would let any non-empty string skip the +leaseless ceiling and every cap behind it. + #### Reads that carry no lease Browsing a group is **never** subject to a transfer slot: not the poster grid, not the @@ -1465,15 +1502,16 @@ as one. ``` C N |-- file_req {v, file_id, chunk_index, [tr]} ------->| - | | `tr` present: mark that lease - | | alive (§11.2) + | | `tr` of a granted lease: mark + | | it alive; of a queued one: + | | `lease_not_granted` (§11.2) | | index lookup; if the id is | | not a file, try the media | | cache (thumbnail/poster/ | | cover/transcode), sliced the | | same way — never leased - | | `tr` absent: admit against the - | | leaseless ceiling, else + | | no granted lease: admit against + | | the leaseless ceiling, else | | `transfer_required` | | backpressure: wait while | | bufferedAmount > 2 MiB @@ -1852,6 +1890,8 @@ duplicates is already stored and there is nothing for anyone to retry. | the connection has identified its device (§9.4) | the `device` field is what receivers verify against | | `device` == this connection's pinned device | **a device may only send as itself.** A member free to name another member's key could *be* that member to everyone, and the signature would verify | | `sender_id` is taken from the authenticated session | never read from the message; it never was | +| ciphertext at most 64 KiB (`chat_too_large`) | a message is a row on the operator's disk that nothing expires, a relayed copy for every connected member and a notification for every member | +| at most 60 messages per 60 s per account per group (`chat_rate_limited`) | keyed by account, not connection, so a second tab does not buy a second budget. There is no node-wide chat ceiling: it would let a busy group silence a quiet one | `format` is a *storage* state as well as a wire one, so the store can distinguish rows the wire would not accept. Nothing changes what is accepted from a peer: the sealed @@ -2003,6 +2043,9 @@ it back (§3.5). | `audio_transcode_req` / `_resp` | C→N / N→C | auth | browser-playable copy of a WMA/MPC file | | `subtitle_req` / `_resp` | C→N / N→C | auth | one embedded subtitle track as WebVTT, by cache hash | | `ping` / `pong` | C→N / N→C | auth | liveness on an open channel | +| `admin_challenge` | N→C | auth | the transcript fields of a signed op, to rebuild and sign (§10.2) | +| `admin_response` | C→N | auth | the operator's signature over that transcript | +| `client_diag` | C→N | auth | the video player's own view of a stream, written to the node's log beside its own (a stream event at INFO, the periodic state at DEBUG); the node acts on none of it and sends no reply | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | | `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | | `transfer_limits` / `_ack` | C→N / N⇒C | signed | per-member transfer caps for this group | @@ -2071,11 +2114,14 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`3.4`**; oldest peer -accepted: **`3.0`** — 3.1, 3.2, 3.3 and 3.4 are all additive, so the floor does not move -with them. 3.4 adds the challenge signature (section 6.5) and invitation links -(section 8.6): `invite_link_create`, `invite_link_result`, `invite_cancel`, which an older -node answers as unknown messages. +MNP versions independently of the package version. Current: **`4.0`**; oldest peer +accepted: **`4.0`**. 4.0 is a MAJOR: a member presents a short-lived node-audience token +bound to one node (§6.3) instead of its hub session token, which is a change to what a +peer must *present*, so a pre-4.0 client is refused at the handshake and the floor moved +with the version. It carries everything 3.x added — the challenge signature (§6.5), +invitation links (§8.6), audio-track and subtitle selection, per-account blobs — so +nothing is currently above the floor, and every capability this document describes is +one every reachable peer has. The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; `MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about @@ -2157,7 +2203,9 @@ walks through the gate meant to stop it. | No MitM on the signaling path | both DTLS fingerprints (or the QUIC certificate hash) inside every transcript; empty binding is a refusal | | The hub cannot read content | the group key never reaches it; chunk keys and chat epoch keys derive from or are wrapped under it | | The hub cannot substitute a key at invite | the node wraps for a key its owner presented and signed, bound to an identity by a code the hub never sees | -| The hub cannot administer a node | privileged ops need an Ed25519 signature from a roster-pinned operator key | +| The hub cannot administer a node | privileged ops need an Ed25519 signature from a roster-pinned operator key; the unsigned operator controls need a device that proved such a key (§10.4) | +| The token a member hands a node opens nothing at the hub | the handshake takes only `aud = MNP_AUD` tokens, and the hub API only its own audience (§6.3) | +| A token captured by one node's operator is useless at another node | the MNP token names the node it was minted for, and a node refuses one naming a different key (`wrong_node`) — before the pre-proof window can serve anything | | The hub cannot impersonate a device owner | device linking is countersigned by a key the node pinned; the hub stores no user keys | | A revoked token cannot connect | denylist on user, `jti` and group, persisted across restarts | | A signature cannot be repurposed | domain-separated, length-prefixed transcripts naming op, subject, node, group, nonce and time | @@ -2183,9 +2231,10 @@ walks through the gate meant to stop it. attacks apart: the pairing code defeats a hub that *lies in its directory* — silent, undetectable, per-request — not one that *rewrites the client*, which is an artifact that can be inspected and compared. -* **An invitation link's code is a bearer code** (§8.6). The node admits whoever brings - it first; what restricts who can bring it is the hub, which lets only the addressed - account reach the node — a rule an active hub does not have to keep. The challenge +* **An invitation link is a bearer secret on both halves** (§8.6). The hub admits the + first account that redeems its ticket, and the node admits whoever brings the code + first; the link is bound to no address, so whoever holds it first joins. What bounds + it is that it works once, for seven days, and can be cancelled. The challenge signature (§6.5) keeps the code from reaching any node but the issuer; it does not keep it from a hub the inviter asked to mail it, which then holds it. * **The pre-proof window is a disclosure surface.** A hub that forges a JWT can fetch a @@ -2281,8 +2330,10 @@ LP(x) = uint32be(len(x)) || x every field, no exceptions | Constant | Value | Source | |---|---|---| -| `MNP_VERSION` | `3.4` | `meshbay_common/__init__.py` | -| `MNP_MIN_SUPPORTED` | `3.0` | `handshake.py` | +| `MNP_VERSION` | `4.0` | `meshbay_common/__init__.py` | +| `MNP_MIN_SUPPORTED` | `4.0` | `handshake.py` | +| `MNP_AUD` / `HUB_API_AUD` | `meshbay:mnp` / `meshbay:hub-api` | `tokens.py` | +| MNP token lifetime | 900 s | `meshbay-hub/auth.py` (`issue_mnp_token`) | | `NONCE_LEN` | 32 bytes (both handshake nonces) | `handshake.py` | | `ADMIN_CHALLENGE_TTL` | 120 s | `adminop.py` | | `JOIN_TTL`, `DEVICE_TTL` | 120 s | `join.py`, `device.py` | @@ -2317,7 +2368,8 @@ LP(x) = uint32be(len(x)) || x every field, no exceptions | `MAX_CONCURRENT_TRANSCODES` | 8 | `webrtc/apps/streaming.py` | | Link preview rate | 15/conn, 60/node per 60 s; cache 1 h × 256 | `webrtc/chat.py` | | ICE gathering deadline | 4 s | `transport.js` | -| Signaling: max SDP, pending per user, rate | 16 KiB, 3, 30/min, 15 s answer timeout | `api/signaling.py` | +| Signaling: max SDP, pending per account, per-account-per-node budget, per-address rate | 16 KiB, 32, burst 120 refilled at 2/s, 600/min per node, 15 s answer timeout | `api/signaling.py` | +| `MAX_CHAT_CIPHERTEXT` / chat rate | 64 KiB / 60 per 60 s per account per group | `webrtc/chat.py` | | GEK | 256-bit, node CSPRNG | `crypto.py` | | Chat epoch key | 256-bit, node CSPRNG, one per group per epoch | `chatbox.py` | | Chunk cipher | AES-256-GCM, 96-bit nonce (ChaCha20-Poly1305 variant for native) | `webcrypto.py`, `crypto.py` | @@ -2342,6 +2394,7 @@ meshbay-common/ protocol.py message types, chunk and upload codecs device.py device request / add / hello transcripts crypto.py GEK, chunk keys, ECIES wrap, BLAKE3 ids webcrypto.py the AES variants the browser can also compute + tokens.py the two token audiences (node vs hub API) meshbay-node/ transport/webrtc_server.py the reference implementation of MNP, transport/webrtc/ assembled from these modules |