aboutsummaryrefslogtreecommitdiffstats

MeshBay — Design

Status: the specification. This document states what MeshBay is, how it is built, and why each part has the shape it has. It replaces the draft series (meshbay-draft-v5.md, meshbay-draft-v6.md, old-draft.md) and the design notes that grew around it; §16 maps every reference those documents and the code make onto a section here.

The convention that is not negotiable: a claim in this document names the adversary it holds against. A property that holds against a passive hub and not an active one is written that way. A section that says "this buys nothing" is as load-bearing as one that says it buys something.

The second convention: this document states design, not history. Where a label like C1, NS6 or T3 appears — and hundreds of code comments cite them — it names the invariant that holds today, not the incident that produced it. §13 is the register of those labels.

Wire versions at the time of writing: MNP 3.4 (oldest peer accepted 3.0), MHP 0.1, packages 0.16.0. The normative source for the wire format is MESHBAY_NODE_PROTOCOL.md; this document states the design the protocol serves, not its byte layout.


0. How to read this

0.1 Scope

Read For
this document the architecture, the trust model, and the reason each decision is what it is
QUICKSTART.md one machine to a working group, for somebody who has installed nothing
USERGUIDE.md using a group and running a node, for the person who does either
MESHBAY_NODE_PROTOCOL.md the MNP wire format, message by message
transfers-v1.md the transfer system's failure-mode analysis, kept because a synthesis cannot carry "every way a slot can be lost"
playlists.md the playlist design and its interface in full, with what building it corrected (§9.10)
cast-smart-tv.md the DLNA/UPnP device backend — designed, not built (§11.4)
WINDOWS-PORT.md the Windows port's audit and packaging detail (§11.2)
PACKAGING-GUIDE.md, HTTPS.md, MAIL-SERVER.md, windows-build.md installation and server operations
CLAUDE.md project conventions and the engineering lessons that govern how changes are made

The user documentation is QUICKSTART.md and USERGUIDE.md (written 2026-09-18). The guide that preceded them described the system before per-node identity, named roots and the sealed wire, and was retired rather than repaired — a document a reader cannot tell the sound parts of is worse than none. The replacements are written from this document and from the code, address a reader who is not a developer, and cover Linux and the browser; Windows stays in PACKAGING-GUIDE.md. They state what is not built as readily as what is (§15.2, §15.3), which is what keeps them from drifting into the same condition as the one they replace.

0.2 Reference labels

Code comments, tests and commit messages cite short labels — C1, H3, NS6, T3, C5b, W2, E9, F1. Every one of them is defined in §13, stated as the rule it names today.

Three label families collide, and the register keeps them apart:

  • C* / H* / M* / L* with no qualifier mean the second review's numbering (§13.3). That is the numbering the code uses. The first review's C1/C2 and the third review's H1/H2/M1–M6/L1–L11 are always written with the review named (§13.1, §13.4).
  • M2a, M2b, M2c and C5a, C5b are sub-items and are unique across the set.
  • D1–D4 are the client-architecture decisions (§14.2); D1–D12 written as "Stage D*" are desktop-client build stages (§15.2).

0.3 The one-sentence version

The hub cannot read your content unless it ships you malicious client code — and against a native client it cannot do that undetectably.

Everything below either supports that sentence or states precisely where it stops.


1. What MeshBay is

1.1 Positioning

MeshBay is a platform for private, encrypted, self-hosted groups with an application store. A group is a set of people, a set of directories on somebody's machine, and a set of applications over them — chat, a file explorer, a video library, a music player, a photo album.

It is not a public file-sharing network. Public groups exist as an optional hub feature and are switched off on the reference deployment (§7.4). The design optimises for the private case throughout: the hub keeps no file names for private groups, registers no content hashes for them, and holds no key that opens them.

1.2 The three parties

        ┌─────────┐   accounts, group registry, signaling relay,      ┌─────────┐
        │   hub   │   notifications, moderation, instance policy      │  peer   │
        └────┬────┘   — never content, never a group key              │  hubs   │
             │                                                        └─────────┘
   MHP 0.1   │  signalling (SDP/ICE, <1 KB), presence, revocation push      MHP
             │
        ┌────┴────┐                    MNP 3.0                    ┌──────────┐
        │  node   │◄──────── WebRTC DataChannel / QUIC ──────────►│  client  │
        └─────────┘   index, file chunks, streams, chat, admin    └──────────┘
     holds the files                                        browser SPA or desktop
     holds the group key                                    client; both are the
     is the content authority                               same UI source
  • The hub is a registrar and a signaling relay. It is in the trusted path by choice, not by necessity (§14.2 D4), and it is never in the data path.
  • The node is the daemon that holds a group's directories and its group key. It is the sole content authority: it decides who is served, what is served, and who may change anything about the group's content.
  • The client is the browser SPA or the desktop application. Both are built from one source tree (§8.4).

1.3 The rule that settles arguments

Group-related server state lives on the node. Always.

Files, indexes, members' devices, pending device requests, invitations, chat, chat epoch keys, per-root availability, application settings, transfer limits — all on the node. The hub holds accounts, the group registry and membership, signaling, notifications, the moderation surface and instance policy, and nothing else about content.

This is decision E9, and it is the rule any new feature is measured against. A feature that wants a row on the hub about a group's content is a feature that has misunderstood the model. It has been re-verified at each content-model change: SwarmSource carries a content hash, a node id and an endpoint — no paths, no filenames — and private groups register nothing at all (H7).


2. Trust model

2.1 Adversaries

Every claim below is written against one of these, and they are the only ones the document uses:

Adversary What they can do
Passive hub Read everything the hub legitimately stores and relays
Active hub Also lie: forge tokens, invent accounts, substitute values it publishes, ship modified client code to a browser
Malicious node operator Read and alter everything on their own machine, including the plaintext files they host
Malicious group member Everything a member may do, plus anything the protocol fails to refuse
Network attacker Observe and tamper with traffic between any two parties
Local attacker Reach loopback services and files on a client or node machine
Registered hub user with no membership Reach every hub endpoint that does not check membership
Federated peer hub Push directory rows and revocations over MHP

2.2 Security claims

Claim Passive hub Active hub Malicious node operator Malicious member Network attacker
Data never transits the hub ✅ ✅ — — ✅
File content is unreadable ✅ ❌ T3 (browser) · ✅ native ❌ by design — the operator hosts the files ❌ members share the group key ✅
The file index is unreadable ✅ ❌ T3 · ✅ native ❌ ❌ ✅
Chat content is unreadable ✅ ❌ T3 · ✅ native ❌ — the operator is a member ❌ ✅
Chat is unreadable off a stolen disk ✅ ✅ ✅ without the keystore passphrase ✅ ✅
Content cannot be modified ✅ ✅ ❌ by design ✅ ✅
The node cannot be impersonated ✅ ✅ — ✅ ✅
Client code integrity ❌ T3, accepted (browser) · ✅ ships in the package (native) ❌ T3 · ⚠️ native: detectable, not prevented ✅ ✅ ✅
The hub cannot obtain the group key ✅ ✅ except in an open-join group, where it can join legitimately (§7.3), and for an invitation the inviter asked the hub to mail, whose code it then holds (§3.4) — — ✅
Node content authority ✅ ✅ ✅ sovereign ✅ ✅
Devices cannot be added by the hub ✅ ✅ — the hub holds no user key and cannot countersign ⚠️ a node adds a device only to itself, where it already reads everything ✅ ✅
Chat senders are authenticated to each other ✅ ✅ ⚠️ only for accounts the reader has already seen (§3.3) ✅ ✅
Keypair bundles (C4) closed for native devices closed for native devices ⚠️ open for any account that also signs in from a browser — —
Deleting your account erases you ✅ hub-side ✅ hub-side ❌ files, pinned identity and bundle stay on the node (§7.7) — —
Your identity keys stay yours ✅ ✅ ⚠️ offline attack on the bundle they hold — succeeds against a weak passphrase, and yields the identity used on that node only ✅ ✅

2.3 What the project must not claim

Three sentences are forbidden, each for a deliberate reason:

  • "Everything is encrypted and unreadable by other parties, even the hub." A hub that ships the code can lift keys from the page regardless of protocol design (T3). That is an artifact-level attack, not a silent directory lie, and it is removed for native clients — not for browsers.
  • "A native client makes the hub untrusted." It converts an undetectable, per-request, per-user attack into a persistent artifact that can be hashed and compared. That value is realised by reproducible builds and published hashes, not by the packaging format. A build signed with a key the hub operator holds relocates trust; it does not remove it.
  • "C4 is closed." It is closed for a native device unconditionally, and open for any account that also uses a browser. An account is only as strong as its weakest client.

"End-to-end" here describes client ↔ node, never client ↔ client. Members and the operator read everything in their group; that is what a group is.

2.4 One boundary worth naming

An operator hosts your content by design. They should not be able to become you. They can still try — a keypair bundle sits on their disk and a weak passphrase gives it up — but what it gives up is the identity you use with them, which unlocks nothing they did not already hold. Reading what they host is by design; reading what other operators host is not, and does not follow (§3.2).


3. Identity

3.1 Accounts

An account lives on the hub: a username, an encrypted email address, a status and a role. A username is 8 to 64 characters, checked at registration only — accounts created under the older 3-character floor keep signing in. The passphrase never leaves the client. It derives two independent values, both salted by the trimmed username:

Value Derivation Consumer
auth_key PBKDF2-SHA512, 600 000 iterations, domain meshbay:auth:v1:<user> hub authentication — the hub stores an Argon2id hash of it
bundle_key Argon2id 128 MB / t=3 / p=1, domain meshbay:bundle:v2:<user> AES-GCM key for the per-node identity bundle, held on each node

This split is T1, and its consequence is structural: the hub never sees a passphrase, so the passphrase floor — 12 characters and roughly 60 estimated bits — can only be enforced client-side, and is.

The two values have different fates. The hub holds a verifier for auth_key and can reset it from an email code. Nobody can reset bundle_key: the hub has held no key material since the invite redesign, and cannot reach a keypair_bundles row, which is served only over MNP to authenticated members. That asymmetry is why passphrase change and passphrase recovery are two features and not one (§3.6).

3.2 Identity keys are per node

A person's identity keypair is created at first contact with a node, encrypted under bundle_key, and left on that node. It is never reused elsewhere.

The reason is blast radius. The adversary is concrete: an operator holding their own node's disk, attacking a bundle offline at their leisure. What cracking one yields is the identity that person uses on that node — where the operator already holds the content, the index and every byte they serve. It is not a key anywhere else: each node gets its own, and a key one node pinned is a stranger to the next, which asks for a code like any first contact.

Two consequences fall out and both are wanted. Two operators cannot tell they host the same person by comparing keys. And the hub stores and publishes no user keys at all — users.pk_ed25519 / pk_x25519 are dropped, PUT /me/keys does not exist, and /pubkeys returns an account id and the node's linking key. There is no directory to substitute from, which is what H3 was.

Two further rules follow directly:

  • Tokens carry no pk_user claim. A key chosen by the hub must never become the identity a node records for an upload, or whoever issues tokens decides who may delete a file. Attribution uses the roster pin.
  • There is no key rotation endpoint. Rotation is per node: member unpin plus a fresh code, which already exists.

Registration therefore generates nothing, which has a pleasant side effect: a scripted signup produces a real account, and a wiped hub and node can be taken to a working demo without a browser.

3.3 Devices

One person may hold several devices on one node — a browser and a desktop client, two laptops. identities is keyed by (user_id, pk_ed25519) with label, added_at, added_by_pk, revoked_at and the evidence columns below. Pinning is never INSERT OR REPLACE: a silent overwrite is a hole the moment a second key is legitimate.

A new device is admitted by a key the node already pinned. The authority is therefore a key the node established locally, exactly as for operator authorisation — and the hub cannot produce it, because it holds no user keys. Device linking adds no hub-reachable authority.

The binding is a one-time code the new device generates and displays, hashed together with the new keys: code_hash = sha256(code ‖ new_pk_ed25519 ‖ new_pk_x25519). The approving device asks the node for this account's pending requests with their stored hashes and recomputes the hash for each until one matches.

The code never reaches the node. That is what makes a substituted key impossible rather than merely detectable: a node offering fabricated keys would have to produce a hash over a code it has never seen.

The approval is deliberately not a human comparing digits. Safety numbers were evaluated and refused, permanently, under decision 19: no new code exchanges between people. A device-linking code is a code between a person's own devices, which is a different thing and was already accepted.

Bounds, all of them anti-abuse rather than the security boundary — the security comes from the hash binding:

Bound Value
Request TTL 1 h, [node] device_request_ttl_minutes
Devices per account per node 5
Attempts per connection 5, then a node-wide lockout
Filing a request requires the account to have at least one pinned identity already

member unpin <user> removes every device of an account, and is the only form the CLI has (§6.7). Revoking one device is revoke_device over MNP, driven from a device the node has already pinned, and it marks the row rather than deleting it — a deleted row is a key the node would happily pin again.

Which device is on a connection is proved separately from which account. The handshake proves the account; device_hello — signed over a transcript naming the node, the group and this connection's nonce — proves the device, and is refused unless the key is a live device of that account in the node's own roster. Without it the node would fall back to the account's oldest key and record it as the author of everything.

Members can verify each other's device keys (Tier 2). group_roster_req / group_roster_resp, sealed under the group key, answers any member with, for each live device of each active member: the key, the key that countersigned it, and the signature, nonce and timestamp that prove it. The client walks the chain itself — the node decides nothing, because the node is the party the property holds against. A device the node lists but cannot evidence never enters the verified set, so a fabricated key is not laundered in by being mentioned.

Three rules keep that honest:

  • A root is a device that names no countersigner, not one that fails to produce a signature. Treating "no proof" as "root" would admit anything a node chose to write.
  • First sight pins everything the node says, not the verified subset — an alarm that fires on legitimate second devices stops being read, and the budget for this whole feature is exactly one notice: "this account's key changed".
  • A device pinned before the evidence columns existed is unevidenced and reads as such.

The property, stated exactly:

Once a member's client has seen an account, a node that later substitutes a key for it is detected. Nothing is gained at first sight, where the client has nothing to compare against.

That second sentence is not a caveat to be dropped. Closing first sight needs an attestation rooted outside the node — an operator-signed roster (Tier 3), which is deferred with nothing depending on it, and worth building only where the operator is not the machine.

The cost, which is real: the roster is member-visible, so every member of a group learns how many devices every other member holds and what their public keys are. It stays inside the group, the hub is not involved, and it is scoped to one group. A member who cannot see the keys cannot check them, so this is not avoidable.

Where device linking does not hold: an approval performed in a browser inherits T3 — the hub serves that browser its code and can read the typed code. The first browser-to-native link is the moment of highest exposure for an account, and it happens once. An account created natively does that first link in the safe direction.

3.4 Admission: invitations and pairing codes

The node wraps the group key, for a key the recipient proved possession of, over an authenticated channel, bound to an identity the operator admitted with a one-time code the hub never sees.

operator (SSH)  meshbay-node member invite bob        → CODE  R3H8-TB6V
                (or the same from the group's Settings tab, signed by the paired browser)
operator        sends the code to bob out of band
bob             opens the group; the client holds no group key
bob → node      join_request {pk_ed25519, pk_x25519, code, sig}   ← pre-proof window
node            code valid for this account → pin the identity, admit to the group
node → bob      the group key, wrapped for the X25519 key bob just proved he holds

Four properties, each load-bearing:

  1. No public key is ever fetched from a directory. The joiner's keys arrive from the joiner, both signed together in one transcript (meshbay:join:v1), so the identity key vouches for the encryption key.
  2. The code binds a key to an account, and the hub never sees it — unless the inviter ticks Send the invitation by e-mail (below). 40 bits, Crockford base32 rendered XXXX-XXXX, single use, valid for exactly one account in one group — except an invitation link's, which names its account when it is redeemed (below) — stored only as sha256(code). A password KDF over 40 uniformly random bits would buy nothing. Guessing is bounded by 5 attempts per connection and a node-wide lockout, and every attempt is an audit event.
  3. The node's roster is the authority, not hub membership. A hub that invents an account, adds it to a group and mints it a token gets not_authorized_for_group.
  4. Wrapping happens on every connection. Nothing is stored per member, so key rotation propagates by itself and revocation actually takes effect. (Rotating the key after a revocation is still required — the ex-member holds the current one, and no protocol can take that back.)
  5. Revoking somebody cancels the code they have not redeemed yet, for that group and no other. There is no membership until a code is consumed, so an invitation sent to the wrong person is the whole of their access, and a removal that left it usable would be a removal in name only. It is also the case removal is asked for most: an invitation is undone before it is accepted, not after. Nothing to rotate then — they never held the key, and member_revoke says so by returning no reminder rather than by leaving the caller to work it out.

Node authority is established the same way, once per node: meshbay-node operator pair prints a code, the operator types it into their own browser, and the node pins that identity. It is never learned from the hub — a hub able to name the operator's key could install itself as node administrator, which is NS4/M3.

Code lifetimes differ because the acts differ:

Code Default Setting
Member invitation 7 days [node] invite_ttl_hours
Operator pairing 24 h [node] pair_ttl_hours
Device add request 1 h [node] device_request_ttl_minutes

An invitation waits for someone to read their messages; a pairing code is typed during the SSH session that printed it.

Why a code and not something lighter — the question is what stops the hub from being bob on his first connection:

Option What an active hub can do
Wrap for whatever key the peer presents Forge a token for bob, present its own key, receive the key worse than nothing
Bind to the key the inviter fetched from the hub Substitute at invite time H3, relocated
TOFU: first connection wins Race the real bob with a forged token small window, total consequence
Safety-number comparison Nothing — but it needs two humans reading digits at the worst moment correct, unusable as a default
One-time pairing code Nothing: the code never reaches the hub adopted

Invitation links. A link lets somebody who has no account yet register and land in the group without typing a code. The code above cannot do that: it names an account when it is issued, and someone who has not registered has none. So a link carries two secrets with two jobs:

Half Secret Checked by Grants
Hub ticket 128 random bits, stored as sha256 the hub, against the redeeming account's verified address — the blind index of the address the inviter typed membership, i.e. the right to reach the node (§7.1). Nothing about keys
Node code the ordinary 40-bit code, with kind = "link" the issuing node only pinning and the group key, as for any code
https://<hub>/#/invite?v=1&g=<group_id>&t=<ticket>&n=<node_pk, base64url>&c=<CODE>

Everything is after #, so no part of it is sent to the hub in a request; the page reads it and rewrites the address to #/invite at once. g lets the client check that the group the hub answers is the one the inviter meant; n names the node that issued the code, which is the only node holding it.

At the node the code is therefore a bearer code, and everything else about it is fixed: a member of one named group, once, never an operator, not spendable by someone already an active member of that group, at most twenty outstanding per group, cancellable by a handle unrelated to the code (MESHBAY_NODE_PROTOCOL.md §8.6). kind is a column rather than an empty account id, so a row bound to nobody can never be read as bound to anybody (AV1).

The rules the client follows, each of which a test holds:

  • it sends the code only to the node the link names, and only once that node has signed its challenge with that key (§5.2). Without the signature the code went to whichever peer answered signaling, and in a group hosted twice, to whichever host answered first;
  • it never sends the code to the hub (unless the inviter asked the hub to mail it — below), and it keeps the pending invitation in the tab's sessionStorage only, cleared on success, refusal, expiry and sign-out;
  • joining is one explicit click after the invitation is shown — group, inviter — never automatic, or a link would add anyone to any group without asking;
  • a link whose origin is not this hub is refused (the desktop client accepts one pasted in), so a ticket is never sent to another hub.

What it holds against, per the convention at the top. A third party holding the link — a messaging service that previews it, a forwarded mail — gets nothing without the invitee's mailbox. An active hub is not stopped: it verifies the addresses it compares, and could already be anybody. A hijacked signaling path no longer receives the code on a native client; a browser is still T3, exactly as for a typed code. What it costs: the hub learns that an owner invited an address (as a blind index and a masked form, until the link is used or expires). Addresses are compared case-insensitively and nothing more, so +tags and dots must match.

Mailing the code is the inviter's choice, and it gives up the row above. The Members tab can ask the hub to mail the code to the invitee's address on file (invite-notify); that request carries the code, so an active hub holds it and can join in the invitee's place. It is offered because a code that arrives on its own is worth more to most groups than the property, and it is stated rather than hidden: the box reads "Send the invitation by e-mail (may land in spam)", is checked by default, and is remembered per account (the invite_email preference), so an operator who unticks it once is not asked to again. Unticked, the hub is never called and the table above holds exactly. The CLI mails nothing.

The same box sits under the link form, sharing the same preference. Ticked, the hub mails the link to the address the ticket is bound to — no second address, and the hub writes the link itself from its own origin, the ticket it has just minted, and the node key and code it is handed, each checked against its exact shape, so a hub-signed mail cannot be made to carry any other URL. Because that mail reaches an address the hub may have no relationship with, at the request of anyone who owns a group, it is also counted per sending account (AV29). A refused mail does not cancel the link, which is shown either way.

Delegation is designed and deferred. invite_create is authorised as a role check against the roster rather than an equality test against the operator, and the delegate role value is reserved, so a group admin who does not run the node becomes a roster row and a CLI command — no protocol change, no migration.

3.5 Open-join groups

A group whose join_policy is open pins on first contact (TOFU) and wraps the key immediately. A code there protects nothing — the hub can create an account, join through the front door, and be a legitimate member — so it would be pure friction.

Stated plainly, per the convention: in an open-join group the hub can obtain the group key. That is a property of open joining, not a defect of this design. Content in such a group is protected from the network and from non-members, and from nobody else.

Note the axis. visibility (public/private) controls discoverability and swarm hash registration (H7). join_policy (open/request/invite) controls admission. Only the second decides whether a code is required: a public group with join_policy = "invite" keeps the code, because being findable is not being open.

join_policy is read from node.toml, never from the hub. A hub able to declare a group open would be handed its key. An unknown group reads as invite.

3.6 Passphrase change and recovery

Changing a known passphrase re-wraps every reachable node's identity bundle from the old bundle_key to the new one before touching the hub — if the fan-out fails, the account is unchanged. Only then is POST /v1/users/password called with the old and new auth_key. Nodes that were unreachable are named to the user, with the operator fallback (member unpin plus a fresh code) as the way to fix each one. Every refresh-token family is revoked.

Recovering a lost passphrase splits into what each key can reach:

Recovered by
Hub login (auth_key) an email code alone
Per-node identity keys — group key unwrap, provable upload ownership, chat identity, device countersigning the recovery key, per reachable node
An identity on a node with no recovery-wrapped copy, or offline at recovery time operator member unpin plus a fresh code

The recovery key is a full-entropy 32-byte secret R generated by the client, rendered as a grouped mnemonic. recovery_key = HKDF-SHA256(R, info = "meshbay:recovery:v1:" + username) — HKDF and not Argon2, because R has 256 bits and there is nothing to brute-force. Every time an identity bundle is written to a node, a second copy is written beside it wrapped under recovery_key (bundle_enc_recovery, additive on the wire). R is a pass-through: offered in the registration email by default, never written to any database, never logged.

The reset endpoints are built to leak nothing. POST /v1/users/password/reset-request requires the username and the email on file as a pair, checked against a blind index and never decrypted; a mismatch, an unknown username and a non-active account all take the identical no-op path and return the same 200 {"status": "sent_if_exists"}. So it cannot be used to spray reset mail at an inbox from a username alone.

A reset deletes every UserDevice row on the hub. Three different things are called "device" here and only one is touched:

What it is A reset
user_devices (hub) an Ed25519 key that lets a client skip the passphrase prompt on launch. A hub-login convenience — no group key is wrapped for it deleted
per-node identity (identities on each node) the keys that unwrap the group key, prove upload ownership and sign chat — this is group access recovered from the recovery copy, or via the operator fallback
the roster pin which identities a node has admitted untouched

Deleting user_devices costs one passphrase prompt per client, which is the point: after a "control may be lost" event, a laptop still carrying a stored hub-auth key must stop signing in on its own.

3.7 The keypair bundle, and what it is worth (C4)

A bundle carries one node's identity keys, encrypted under the owner's passphrase, stored on that node. It is what lets a second browser open the same account there — the ordinary expectation, and the only mechanism available to a browser, which keeps nothing durable of its own.

Why Argon2id. PBKDF2 is compute-only, which is exactly what a GPU is good at. Measured: PBKDF2-SHA512 600k costs 241 ms per guess on one core, Argon2id 128 MB / t=3 costs 88 ms — the defender pays less — but only one of them forces an attacker to find 128 MB per guess.

The honest size of the gain. On a single high-end card the ceiling moves from roughly 8k guesses/s to roughly 2k: a factor of four, not a thousand. What it really buys is the cost of scale — 128 MB per lane caps a 24 GB card near 187 concurrent guesses and makes custom hardware unattractive, where SHA-512 silicon is cheap.

The passphrase decides this, not the KDF. At ~2k guesses/s a dictionary-and-rules run of 10⁹ candidates takes about six days on one card. Four random words (~52 bits) outlasts the sun. No parameter choice saves a weak passphrase; it only moves it from hours to days.

Operational facts that constrain changes:

  • Argon2id runs in WebAssembly, vendored under static/vendor/ with its provenance. The CSP forbids external hosts and must keep wasm-unsafe-eval in script-src.
  • Never change these parameters in one place. keyderive.js, keyderive.py, the desktop client and the test harness are held byte-identical by test_bundle_kdf_parity.py. A mismatch does not look like an error — it looks like an account nobody can open.
  • Bundles carry an MBK2 marker; an older PBKDF2 form is still readable and is re-encrypted on the next backup.
  • Cost is paid once per sign-in (≈650 ms bundle + ≈239 ms auth_key). Reloading a page derives nothing: the key lives in IndexedDB.
  • The pre-proof window that serves bundles is bounded (4 fetches) and audited.

C4 is reduced, not closed. Bundles still sit on disks their owner does not control. It closes for a native device unconditionally, because that device's key is in no bundle anywhere. It closes for an account only when no browser needs a bundle on that node — which needs device_policy {allow_bundle: false}, signed by a pinned key so the decision is the user's and never the hub's (open item O3).


4. Cryptography

4.1 Key hierarchy

User identity key      Ed25519      signing, authentication      — per node (§3.2)
User exchange key      X25519       key agreement                — per node
Group encryption key   AEAD 256-bit content and index encryption — the group secret
Chat epoch key         32 bytes     per group, per epoch         — node-generated (§4.5)
Session keys           X25519/HKDF  per-connection, from DTLS/TLS

Every private key lives in an encrypted keystore on the machine that owns it. The hub never sees one.

Domain separation is consistent and mandatory. Every derivation uses a distinct info string, and the AES variant adds an :aes suffix so two ciphers can never derive the same key from one group key. This is a small detail that prevents cross-protocol key reuse, and it is checked rather than assumed.

4.2 Group key wrapping (ECIES)

wrap:    sk_eph, pk_eph = X25519.generate()             # fresh per bundle
         shared   = X25519(sk_eph, pk_recipient)
         wrap_key = HKDF(shared, salt=pk_eph, info="meshbay:gek_wrap:v1", len=32)
         wrapped  = AEAD(wrap_key).encrypt(nonce, gek, aad=pk_recipient)
         bundle   = pk_eph ‖ nonce ‖ wrapped

unwrap:  shared   = X25519(sk_recipient, pk_eph)         # same derivation

Three properties are why this shape:

  • The ephemeral keypair is fresh per bundle, so the same key to the same recipient produces different ciphertext every time.
  • The AAD binds the bundle to its recipient, so a bundle reused for a different member is rejected by the tag rather than by a check somebody has to remember to write.
  • A wrong private key fails at the AEAD tag — an immediate, unambiguous refusal.

The node produces every copy of the key itself, from its own CSPRNG. Nothing arriving over MNP can activate a group key (C5b). Read that precisely: it targets key material arriving from outside, not the instruction. An operator-signed gek_rotate where the node generates the key is a different shape and is allowed. The initial gek-init stays local, because with no key there is no completed session to carry a signed op.

4.3 On-the-fly encryption

Files are stored in plaintext on the operator's disk and encrypted at read time. This avoids double storage and makes key rotation feasible without re-encrypting terabytes.

disk (plaintext) → compress → per-chunk AEAD under a group-derived key → transport → client
  • Chunk size 1 MB: amortises AEAD overhead and enables seeking, because each chunk is independently decryptable.
  • chunk_key = HKDF(GEK, salt=None, info="file:" ‖ blake3(file) ‖ ":chunk:" ‖ index). The salt is omitted deliberately: the group key is CSPRNG output and already uniform, so the file and chunk context belongs in info, which is the correct HKDF usage (M5, first review).
  • Chunk authentication is the AEAD tag, not a per-chunk signature. The tag authenticates the ciphertext under a key only members hold, which is what the signature was for.
  • Compression precedes encryption, because compression is ineffective on ciphertext.
  • Upload chunk size is 48 KB, which is what fits the SCTP limit after msgpack overhead.

Crypto is not the bottleneck: encrypt-and-send costs single-digit milliseconds per megabyte against tens to hundreds for the network.

4.4 The group envelope

meshbay_common/groupbox.py, mirrored by sealGroup/openGroup in static/crypto.js, is one envelope with purpose-separated subkeys derived from the group key. AAD is "<msg_type>|<group_id>".

Purpose Info string Carries
index meshbay:index:v1 index_sync, index_delta
ack meshbay:ack:v1 the handshake ack's configuration payload
upload meshbay:upload:v1 file_upload / file_upload_ack
chat_keys meshbay:chat_keys:v1 chat epoch key delivery
roster meshbay:roster:v1 group_roster_req / resp

Never reuse the chunk key with a pseudo-file for this. The purposes are what keep two message classes from sharing a key.

Three things about the sealed messages must stay straight:

  • Sealing the ack line is integrity, not confidentiality. The handshake transcript names no ack field, so is_node_admin, enabled_apps, the roots table and the rest were authenticated by the channel alone. The AEAD tag comes from a key the hub does not hold.
  • Sealing the index is defence in depth against our own next bug, of a class that has shipped twice (C1, C6). It buys nothing against an observer, the hub or a member. That is the whole claim.
  • A payload that does not open ends the session, never a default. An unopenable enabled_apps reads as "the operator disabled every app" and an unopenable index as "the group is empty" — both indistinguishable from legitimate states.

What stays in clear, and none of it is content: the handshake itself; type, v, group_id and the ack's node_pk/proof/sig, because a receiver must route and authenticate before it would trust a decryption; upload_id and chunk_index, because the node routes and orders on them; index_progress, which is counters only, every 2 s; transfer-lease fields, which are an opaque id and two numbers (§5.5); and the media-metadata replies.

4.5 Chat encryption

Chat is encrypted with one key per group, per epoch, per device, derived by name from an epoch key the node generates and delivers wrapped under the group key:

epoch_key      32 random bytes, generated BY THE NODE           (C5b)
device_key(d)  = HKDF(epoch_key, info="meshbay:chat:dev:v1|"+group_id+"|"+d)
                 where d = base64(device pk_ed25519), the roster's own identifier

Every member derives every device's key from the epoch key, so nothing is distributed per device and nothing is stored per device.

There is no mutable sending state at all. That is the point, and it is a stronger guarantee than per-device chains rather than a weaker one: a shared chain advanced by two clients produces key and nonce reuse (first review C1, one level down), and a design with no sending state cannot have that hazard.

Two clients of one account normally share a device key. A second browser recovers the existing identity from the keypair bundle rather than minting a new one; device linking is the exception, not the rule. What makes that safe is the nonce, not the derivation: 96 random bits, never a counter. Two independent senders under one key collide only on the birthday bound, which at chat volume is unreachable; two senders advancing one counter collide immediately.

Each message is sealed and then signed over the ciphertext with the device key the node pinned, so a receiver verifies before decrypting and anyone holding the roster can verify. The AAD binds the group and the epoch, so a ciphertext cannot be replayed into another group or attributed to another epoch. sender_name lives inside the sealed payload — as a wire field it was free to spoof.

sender_id stays a clear field set by the node from the authenticated session (NS6). It is what the store keys on and what the UI groups by; it is not what authenticates the message. The device claim is checked against the connection's own device_hello, or a member could sign as anyone. Replay is refused by a unique (device, nonce) — a replay is a validly signed copy, so nothing about the signature refuses it.

Epochs. A new epoch is opened when, and only when, the set of devices that may read future messages shrinks: member revoke, member unpin, revoke_device, gek_rotate, or an explicit chat rotate. Epoch 1 is opened at group load — a group with no epoch is a group nobody can speak in.

Old epochs are kept and still delivered. That is what keeps history readable to everyone who could already read it, and it is why rotating the group key is a re-wrap rather than the destruction of the whole archive: the archive is not encrypted under the group key, only the epoch keys are wrapped with it in transit. Nothing anywhere deletes an epoch. Epoch keys are stored ECIES-wrapped to the node's own X25519 key in bundles.db, never raw — a plaintext key store beside chat.db would collapse the threat model silently, and it is the obvious thing to write.

Why not a ratchet. Under group-key distribution and server-served history, the node must retain each chain's earliest key, and a chain key at iteration i yields every message key from i onward by pure HKDF. Forward secrecy is therefore zero either way. What a ratchet was left buying is a large amount of stateful client code with silent failure modes, three of which are concrete: any member could sign as any other, a second device dropped the first's chain, and the skipped-key cache grew without bound (§13.6, F1–F3). Forward secrecy is given up deliberately and on the record. If it ever becomes a real requirement it belongs in 1:1 DM, where there is no server-side history to contradict it.

Neither a sender-key nor a ratchet implementation exists in the tree. Both were written, neither was ever called, and both are deleted — the reasoning that ruled them out lives at the top of chatbox.py, the module that replaced them, where it stands on its own instead of pointing at a file to compare against.

Kept code that nothing calls is worse than absent code. It reads as an alternative somebody may reach for, its green tests read as evidence of a protection that is not in the product, and it has to be maintained past every refactor to stay compiling — maintenance spent on a decision already made. If forward secrecy ever becomes a real requirement, it belongs in 1:1 DM, where there is no server-side history to contradict it, and it starts from the requirement rather than from a module somebody left behind.

What chat encryption protects against, in the words the user-facing docs should use: someone who obtains the node's storage without the keystore passphrase — a hosting provider imaging the machine, a leaked backup, a seizure where the passphrase is not surrendered. It does not protect chat from the operator or any current member (they hold the group key, and the chat key is delivered under it); from anyone holding any one device of any member; from a former member, for messages sent before the epoch changed; from the hub as regards metadata; or from the node as regards links posted, which it fetches to unfurl.

Deliberately not encrypted, stated so nobody reads more into the feature than it does: sender_id, timestamps, message sizes and the fact of a message are in the clear to the node, which is the relay and cannot route otherwise. The hub learns per message the group, the time and the sender's account id, so it can skip the author when creating notifications — it can build a social graph with timings without reading a word, and that is a known metadata leak rather than a solved problem. Attachments are ordinary files on a root and stay plaintext on disk; the reference to one is inside the sealed payload, but the file and its name are in the index.

4.6 Parameters

Parameter Value
Node keystore KDF Argon2id 256 MB, t=3, lanes=4 — recorded per envelope, so raising it does not orphan existing keystores
Hub password verifier Argon2id 64 MiB, t=3, lanes=4 (pw_version 4), over the client-derived auth_key — RFC 9106's second recommended setting. An older hash is verified at its own version's parameters and rewritten at the current ones on the next sign-in
Browser bundle key Argon2id 128 MB, t=3, p=1
Browser auth_key PBKDF2-SHA512, 600 000 iterations

Why the hub verifier is 64 MiB and not more. What it protects against is an offline attacker holding the database; online guessing is bounded by the sign-in lockout (§7.7), where Argon2's cost plays no part. That attacker pays, per guess, the client's 600 000 PBKDF2-SHA512 iterations and the hub's Argon2id, because auth_key is 256 bits and cannot be searched directly. Above 64 MiB the memory multiplies the attacker's cost by a constant factor — at most 16 at 256 MB, less once PBKDF2 is counted — which moves a weak passphrase from cracked-soon to cracked-later and a strong one from out of reach to out of reach. On the hub the same memory is paid at every sign-in, one derivation at a time (§13.5b, AV9): measured on meshbay.org, 450 ms at 256 MB against 105 ms at 64 MiB. The passphrase floor is what separates the two cases, not the verifier.

The stored pw_version always names the parameters the hash was made with. Verification reads them from it, so hash_password takes the version it is hashing for; the legacy raw-password scheme stays at version 2 and the auth_key scheme is every version from 3 up. | Chunk cipher | AEAD, 1 MB chunks, per-chunk key by HKDF | | Chat nonce | 96 random bits per message, never a counter | | Invite / pair codes | 40 bits, Crockford base32, single use, stored as sha256 |


5. The node protocol (MNP)

5.1 Transports

Listener Role Status
WebRTC DataChannel primary, browser and desktop client the path everything is tested on
QUIC LAN, port-forwarded, hub-less group:// off by default ([node] quic_enabled) — nothing ships a QUIC client yet, and it does not implement chat (below)
~~TCP + TLS~~ — does not exist (C6)
~~HTTP file API~~ — does not exist (C1)

QUIC carries no chat, deliberately. It implements neither the per-device sealing nor the device identification the chat rules require, and a message reaching a group's archive without them would be a plaintext row in an encrypted history — indistinguishable from one somebody actually wrote. The handler was removed rather than gated: refusing there would mean maintaining a second, weaker set of rules for a transport with no client, which is how a bypass branch survives. An unimplemented message type is logged and dropped, like every other one this transport does not have.

That is the general form of the parity rule. A transport either implements a content rule fully or does not serve that content at all. "Implements the authentication but not the authorization" is the shape of finding C6 and of the third review's M2, twice.

ICE/STUN is the NAT traversal mechanism, for native clients too — via aiortc in Python. punch_nat() is a direct-connection helper, not a traversal stack: one UDP probe to one address, no STUN client, no candidate gathering, no dual-stack fallback, and it requires the client to already know its own external address. It was validated on one ISP and one NAT type. ICE has been validated across two ISPs, two browsers, IPv4 STUN and IPv6 direct, and 4G CGNAT (§11.1) — no TURN relay is needed.

Several STUN servers, two providers deep, because a single server is a silent single point of failure that adds the full gathering timeout to every connection when it is slow:

stun:stun.l.google.com:19302
stun:stun1.l.google.com:19302
stun:stun.cloudflare.com:3478

Both sides carry the same defaults and gather independently; neither learns which server the other used. A STUN server learns the querier's public IP and NAT mapping — that is its purpose. No content, no credentials and no group metadata passes through it; adding one is trusting its operator to learn your NAT topology, nothing more.

The node's list is editable three ways (Node page, meshbay-node stun, node.toml) and hot-swapped on save. The browser's list is hardcoded and not configurable: the hub relays SDP, not ICE policy, and no mechanism exists — or is wanted — for a node to push ICE configuration to a browser.

ICE interface filtering (ice_interfaces) is node-side only, and controls which local addresses the node offers as candidates. Two modes: auto (excludes virtual and VPN adapters by heuristic) and manual (an explicit whitelist). It exists because VPN clients add virtual interfaces whose .local mDNS candidates a remote node spends seconds failing to resolve. The browser's own gathering is governed by the engine and is outside this application's control.

5.2 The handshake

One implementation, meshbay_common/handshake.py, called by both transports. Two implementations of one security check is C6 waiting to happen.

client → node   handshake            {token, group_id, nonce_c, v, v_min}
node            authorize_token()    JWT · scope · denylist · group_id · membership · hosting
node → client   handshake_challenge  {nonce_s, node_pk, sig}   sig: Ed25519 over the challenge (3.4)
                                     ── pre-proof window: bundle fetch, join ──
client → node   handshake_response   {proof}
node            verify               HMAC(GEK, client transcript)
node → client   handshake_ack        {proof, sig, node_pk, sealed configuration}
client          verify               HMAC(GEK, node transcript) + Ed25519(node_pk),
                                     and that ack.node_pk is the key announced above

The transcript is length-prefixed, domain-separated and role-bound:

"meshbay:mnp:handshake:v1" ‖ len‖role ‖ len‖group_id ‖ len‖nonce_c ‖ len‖nonce_s ‖ len‖binding

Every field is length-prefixed so the concatenation is unambiguous, and the role is bound in so a client proof can never be replayed as a node proof (L4).

Since 3.4 the node signs its challenge, with the key it announces, over "meshbay:mnp:challenge:v1" ‖ len‖group_id ‖ len‖nonce_c ‖ len‖nonce_s ‖ len‖binding. A join is sent in the pre-proof window, before the ack that proves the node, so without it an invitation code went to whichever peer answered signaling. What it buys, per the convention at the top: a client that knows which node it means to reach can refuse to send a code anywhere else — against a hijacked signaling path and against a second host of the same group. It proves a key, not the right one: it helps only a client that already knows which key to expect. A wrong signature is refused; an absent one is an older node, discovered from its answer.

Channel binding is mandatory and an absent one is refused — never degraded to nonce-only, which would silently drop MitM detection:

Transport Anchor
WebRTC both DTLS certificate fingerprints
QUIC SHA-256 of the server certificate. On a resumed session, which carries no certificate, the anchor travels with the session ticket — sound, because the ticket derives from the handshake where the certificate was presented

Authentication is mutual (C3). The node proves possession of the group key over a client-chosen nonce and signs the transcript with its long-term key. The client verifies both, refuses a bare ack, and TOFU-pins pk_node per node, refusing a changed key outright with a deliberate reset path in Settings for a legitimate rotation.

node_pk is announced in the challenge because joining needs it before the ack: a first-time member signs a transcript naming this node and has no group key to complete a handshake with. It is unverified at that point and is never a substitute for the ack.

The pre-proof window is three messages, and that is a bound to defend. Only the two bundle fetches and join_request are served before the proof, because each is something a caller needs in order to prove possession of the group key: a returning browser has to recover its identity, and a new member has no key to prove with. Nothing else qualifies. Device messages are authenticated-only — a device request is countersigned later by a device already pinned, so requiring the caller to finish its own handshake first costs nothing and keeps the surface at three. The bundle fetches are additionally bounded (4 per connection) and audited, because that window is the neighbourhood C4 and C5b came from.

Pinning is defence in depth, not the primary control. A substituted node already fails the key proof. Pinning covers the case where an attacker holds the group key — an ex-member, a leaked key — and swaps the node underneath, which the proof alone cannot distinguish from the genuine node.

Authorization rules:

  • group_id is mandatory. Omitting it once skipped the membership check and fell back to the node's first group (M1).
  • scope == "user" is enforced by default; node-scoped daemon tokens are refused on the client path (M9, NS7).
  • The denylist is consulted for user, jti and group.
  • The node refuses connections when it holds no group key — there is no gek_required: false bypass (NS8).

Refusals carry a code, not only a sentence, because a client can act on a code. not_a_member in particular is usually a token issued before the person was added to the group — groups is baked in at sign-in and the hub pushes no updates — so the client refreshes once and retries rather than telling someone who was invited a minute ago that they are not a member.

5.3 Correlation and liveness

Every reply carries the request id it answers (req_id). Matching by arrival order is a guess that fails silently and asymmetrically: the victim is never the request that was answered wrongly, it is the unrelated one now waiting for a reply already delivered elsewhere. The node stamps req_id on the reply from _send, via a ContextVar so a handler's spawned work still answers under the right id, and never on a broadcast, which answers nothing.

PING/PONG is liveness on an already-open channel and never a discovery mechanism — opening a connection costs a full ICE/DTLS handshake, measured at 0.6–7 s.

5.4 Operator-signed operations

Destructive and privileged operations require an Ed25519 signature over a structured transcript, never a JWT. The hub controls JWT issuance and can therefore never establish node-level authority.

"meshbay:admin:v1" ‖ len‖op ‖ len‖node_pk ‖ len‖group_id ‖ len‖subject ‖ len‖nonce ‖ len‖ts

TTL 120 s. The client reconstructs the transcript from announced fields and refuses to sign if the operation or subject is not what the user asked for (H5) — a challenge of opaque random bytes signed blind is an unbound signing oracle. The transcript's subject names the outcome, not the operation: what the operator is shown before signing has to be what happens.

Verification is against roster.operator_pks(), rebuilt from node state, never from anything in the response.

Operation Authority
file_delete the operator, or any non-revoked device of the uploading account
dir_delete the operator alone, and only on an empty directory
invite_create the operator (or a delegate, when delegation ships)
invite_link_create, invite_cancel the operator
gek_rotate operator-signed; the node generates the key itself
initial gek-init local admin API or CLI only
root add/remove/update/eject/plug, apps_enabled, app directories, transfer limits operator-signed
~~gek_bundle_store~~ the message does not exist. No member ever hands the node key material

gek_bundle_store was deleted rather than gated. The operator's X25519 public key is announced in the handshake ack, so any member could wrap a key of their choosing for it; a path that does not exist cannot be mis-authorised (C5b).

Authorisation is against the account, not the key. With several devices per person, verifying against the exact uploading key would refuse a person's desktop the right to delete what their phone uploaded. uploader_pk stops being the authorisation key and becomes the audit record of which device acted. This remains roster-rooted, not token-rooted: a hub minting a token that claims to be someone holds no key the node pinned for them, so the signature fails.

Ownership is recorded by the node, from the authenticated session, and it is durable. The record is written when the last chunk lands and the file reaches its final name; the entry is stamped from it when the indexer creates the entry, which is a later moment — the entry does not exist while the upload is still arriving, so an attribution written against the index at the end of the upload matches nothing, silently, and leaves every uploaded file owned by nobody. It is stored beside the hash cache rather than on the entry alone, because the index is rebuilt from disk at every start and an owner the node forgets on restart is a right quietly taken away. It is validated against a live stat(), so whatever later occupies that path inherits nothing.

It is asserted by the node, not proved by the uploader. A member verifies nothing here: they are told who uploaded a file, by the node that served them the index — the same trust they already extend to every other field in it. The overwrite path where an attacker could become a file's recorded uploader (C5a) is closed by the no-overwrite rule (§6.4), not by a signature. An upload transcript the uploader signs — over node, group, root, path, content hash, account and timestamp, stored with the entry — would make it verifiable by any member instead. That is an open item (§15.3), not something the product has.

One implementation, several front doors. The meshbay_node/ops/ package holds every operation, one module per domain, all re-exported by ops/__init__.py. The loopback API, the CLI and the signed MNP handlers all call these functions; they take the daemon state, raise OpError, and know nothing about HTTP. Two implementations of one operation with two authorization checks is C1/C6 one size down.

5.5 Transfer leases

A download used to be invisible to the node: a client sent eight independent chunk requests and reassembled the answers, and nothing said a transfer had started or ended. There was nothing to count and therefore nothing to cap.

The unit is the lease — the node's record that a peer is transferring something, held for the length of the transfer and released by name (transfer_open / transfer_close / transfer_state). Six properties are decisions:

  • tr is drawn by the client. Re-opening after a reconnect with the same tr is idempotent, so a reconnect cannot charge a member twice for one transfer.
  • A lease is scoped to the connection, never to the account. It dies with the session, which makes the primary reclaim deterministic.
  • A lease covers a job, not a file. A directory zip is dozens of files and one lease.
  • Nothing is persisted. A restart drops every session anyway; a lease that outlived the process would be a slot nothing can release.
  • Leases are counted, not bytes. What a slot protects is concurrency — open file handles, disk seeks, the channel buffer each transfer keeps full.
  • Per-member first, then node-wide. A member at their own cap queues behind their own transfers and never holds a node-wide slot a second member has none of. Reversed, whoever arrives first takes everything.
Bound Default Why it exists
Node-wide concurrent transfers 8 per kind operator's machine. Downloads and uploads are separate pools (max_concurrent_downloads, max_concurrent_uploads, §6.8), because the two cost different things and one queue for both makes each cap mean nothing. Streaming is a third pool again (max_concurrent_streams): a member watching a film is not charged a download slot
Per account, per group 2 absent means this, not "unlimited"
Grant deadline 30 s a grant nobody takes up is a slot nobody can use
Idle timeout 120 s catches a peer that vanished without the connection noticing
Queued per member per kind 32 an unbounded queue is how a node runs out of memory politely
Missed grants before closing 3 without a bound the requeue is a permanent cycle

Browsing a group is never subject to a transfer slot. Not the poster grid, not the album covers, not the video thumbnails, not the file list, not opening a photo or a document to look at it. A member must be able to browse a group that is at capacity exactly as they browse an idle one. That is a requirement, not a tuning parameter, and it is met structurally rather than by choosing a lucky threshold.

Navigation proper never touches this path at all: listings and metadata are their own message types, sealed under the group key, with no relationship to a chunk request. That half needs no rule — only a test that fails if someone later routes a listing through the chunk path.

The chunk path itself carries three genuinely different things, and they are distinguished structurally, by what the id resolves to and by which function asked:

What Resolves to Rule
Thumbnails, posters, cover art, cached audio conversions a media-cache id, not an index entry Never leased, never counted, never queued. One chunk each, out of a bounded cache the node built itself
Looking at one file — a photo full size, a document, an image a real index entry, fetched whole Not leased, subject to the bound below
Downloading and uploading a real index entry Leased

The rule stated as a sentence someone can check by reading:

A transfer is something the transfers panel shows. If it does not appear there, it does not take a slot. The two sets are the same three call sites, which is what makes this verifiable rather than a judgement call at each new one.

Which of the two a request is, is the node's answer and not the client's. tr is drawn by the client, so it is a claim: the node resolves it against its own record and serves the chunk as a leased transfer only when that record is a granted lease of this connection. A lease it holds but has not granted is refused with a code — a member reading while queued is the cap not applying — and an id it has no record of is treated as leaseless and bounded below, because that is also what a reconnect looks like from here, where the session's leases died with the old connection and the client is re-opening them. Read as a bare presence check, the field made every cap on this page decoration: any non-empty string skipped the ceiling, and the queue held only the clients that chose to wait.

Why the exemption is expressed in concurrency and not in size or bytes, which is the durable part of this decision:

  • A size threshold does not separate the two. A raw photo out of a camera is 60–80 MB and is browsing; a 40 MB archive is a download. Any threshold letting the photo through lets the archive through too.
  • A byte-rate budget does not either. It would have to be large enough for that same photo, at which point it is large enough to be a download channel.
  • Concurrency is the thing being rationed, so concurrency is what the exemption is expressed in.

The bound is therefore a ceiling on how many distinct files one session may read leaselessly at once (12, with a 60-second idle expiry). Per session and not per member, because this is a ceiling on what one connection can do while claiming to be browsing, not a resource pool — a member with three tabs open is browsing in three tabs. Two rules keep it from becoming a bug: a file already being read is always admitted, whatever the count, because refusing a chunk halfway through a photo is worse than never having admitted it; and entries expire on idle, because a viewer closed mid-file stops asking and says nothing, and dead entries would eventually refuse every later preview.

A preview never shows "waiting", because a preview never queues. If the bound is somehow reached, the request is refused with a stated reason and the person tries again — it does not silently become a queued transfer in a panel they were not looking at.

What the residual is, stated plainly. A client that lies — labelling a bulk download as a view — gets that bound's worth of files at a time instead of its member cap. It is bounded, it is audited, and it is the same class of statement as the cap itself: this is a fairness control among cooperating clients, in the company of the stream cap. It is not a defence against a member determined to saturate the operator's disk, and must never be described as one — that member is a member, and the answer to them is member revoke.

The numbers are visible or the queue is unprovable. transfer_state carries used, cap and ahead so a client can say "waiting — 2 of your 2 slots are busy" rather than showing a bare spinner; the same counters reach the loopback API for the CLI and the Node page, and a periodic debug line. When someone reports a transfer stuck at "waiting", that line is the only thing that will say whether the node ever had them in a queue at all.

The lease module is free of asyncio and of the transport: it decides, and the caller does the I/O. A queue that reveals itself only through a DataChannel is a queue nobody can prove things about.

5.6 Versioning and flag days

MNP and MHP version independently of the package version. Every wire message carries v; both peers declare v and v_min on the handshake and refuse each other with a code (version_too_old / version_too_new / version_unreadable).

A mismatch is a refusal, not a field that turns up missing. A stated refusal is a bug report; a feature that quietly does not work is a support case.

Additive changes are MINOR and cost nothing. A change to what a peer must be able to do is MAJOR even when the messages are additive — a peer that cannot ask for a transfer lease is either refused, or not refused and transferring outside every cap the operator set.

There is no compatibility switch, by policy. An opt-in flag leaves the old branch reachable on every node, which is C6's lesson one feature later. Where a break is required, MNP_MIN_SUPPORTED moves with MNP_VERSION and the deployment is coordinated: the hub serves the SPA, so a browser picks up the new client on reload; the desktop client ships its own UI, which is why GET /v1/hub/version carries client.minimum and the client checks it before connecting and says "this version can no longer connect" rather than showing a handshake refusal nobody can act on.

Every requirement is true of every peer the client can reach. The floor moves with each MAJOR, so check_version refuses at the handshake any peer that cannot meet one: an upload is sealed or it is not sent; a transfer has a real lease or it does not run; there is one app-directories op and no wrappers behind it. The client records the version its peer declared, for diagnostics, and branches on none of it.

A capability flag on a peer whose floor already guarantees the capability is a branch that can only ever take one path — until somebody lowers the floor, at which point it silently takes the other. A field kept "just in case" is how the branches come back.

The floor is not the current version, and MINOR additions are why. It is MNP_MIN_SUPPORTED in handshake.py, it equals the last MAJOR, and 3.1, 3.2, 3.3 and 3.4 have all been added above it without moving it. So a peer can be reachable and still not do something the current version can, and the client has to cope with that — by reading the peer's own answer, never by comparing version numbers. 3.2's audio tracks are the worked example: the node lists them in stream_init, the client draws its selector from that list, and a node that sends no list gets no selector. 3.3's subtitles repeat it exactly, and add the case where the list is narrower than the file: only the tracks the node can convert to WebVTT are named, so what the menu offers is what will actually appear, and a track's ordinal is therefore not its position in the list. That is not the branch the box above refuses. The branch it refuses is a flag the client sets from a version it parsed; this is the node stating what it has, in the same message the feature already needed, and it takes exactly one path per peer because the peer said which.

Where a break leaves data behind, a migration runs with the node stopped, backs the database up first and is idempotent. But a migration that has to be remembered is a migration that does not happen, so anything that can be a read-time fallback is one instead: the roster reads an older settings key when the new one is unset and leaves it behind on the first write, and node.toml keeps being read in its older spelling. Only a transformation nothing can infer — two settings that disagree, where only the operator knows which they meant — is allowed to need a step somebody has to run.


6. The node

6.1 Node authority

The node is the content authority. Its authority comes from its own roster, established locally by pairing, and from nowhere else — never from the hub, never from a config file, never auto-pinned from the keystore (NS4). A configuration that still names a legacy admin key is warned about at startup and never obeyed.

The division of trust: the hub certifies identity; the node authorises content operations. Hub membership lets someone reach a node; the node's roster decides whether it wraps anything for them.

6.2 Roots

A group's content is a set of named roots, each mapping to a local directory, forming one union virtual root:

/                      (group virtual root)
├── Films/         →   D:\Media\Films
├── Music/         →   E:\Audio                 (external drive, removable)
└── Documents/     →   C:\Users\me\Share

The name is the chosen directory's basename, derived once at add time and stored. Never recomputed from the path: renaming a folder on disk would otherwise silently re-identify a whole library and break every stored reference to it. Four rules make basename naming safe:

  • A duplicate basename is refused, case-insensitively. Collisions are common in practice (D:\Films and E:\Films). Refusing is correct; an explicit alias is the escape hatch (open item O11).
  • No root may contain another, compared case-insensitively and after canonicalisation. Two nested roots would index the same bytes twice under two identities.
  • The basename becomes a path segment every member sees, so it must itself pass the portability rules (§10) — a root Windows cannot write to is a root nobody on Windows can download from.
  • Every index path carries a root segment, uniformly, including in a single-root deployment. One code path, not two.

Five consequences, none optional:

  1. The root name is part of a file's identity, so renaming a root rewrites every path under it. Renaming is explicit and warned, never a cosmetic setting.
  2. Availability is per root. With one directory an unplugged disk was a hazard; with named roots it is a supported state — one root freezes, the others carry on.
  3. Free space, quotas and capacity are per root — different volumes. Anything the UI says about space names which root it means.
  4. Path resolution is per root, in one place: a request names <root>/<path>, RootSet.resolve() resolves it against that root's canonical path and refuses .., absolute segments, symlinks and anything escaping the root.
  5. kind is a view hint (generic/video/audio/photo), nothing more.

Each root is read-only or read-write.

  • writable = false (the default) means read-only for everyone, including the operator. A published library that quietly accepts writes from whoever holds admin authority is not one, so refusing the operator is the point rather than the defect.
  • writable = true means any group member may upload there.

Several roots may be writable and none need be — a fully read-only group is valid. The operator toggles this with a signed op.

There is one answer to "may this member write", and it is the root. A single flag over the group cannot express "this library is published read-only and that folder is a drop box", which is the ordinary arrangement — so the group-wide switch that used to exist is gone entirely: the message, the signed operation, the field on the handshake ack, and the upload alias each root used to carry beside writable. Two sources for one question is one too many: whichever the code consulted first decided it, and a client falling back to the group flag whenever a root omitted writable is exactly that bug with a compatibility justification. Two names for one boolean is the same fault one size down.

A control that writes must name where. With two writable roots the node cannot choose without guessing, and a guess sends a member's file to a disk the operator did not intend. The client names a root, never a path; everything below the root is decided by the node.

A root that goes away must freeze, not empty. The indexer runs a watchdog observer and rebuilds on change; unmounting a volume either emits deletions for the whole tree or presents an empty directory to the next scan. Both propagate as though the owner erased their library. So a root has two independent runtime states:

  • ejected — operator-controlled, persisted in roster.db.
  • available — computed as not ejected and is_live(). This is what clients and the indexer see.

The distinction matters: between clicking eject and physically unplugging, is_live() is still true, and without ejected the availability sweep would immediately flip the root back.

Eject stops that root's observer, marks it unavailable, freezes its entries and propagates the availability change to connected peers — the operator can then safely unplug. Plug checks the path is accessible first, then rescans: the plan called for a reconciliation, but a device people carry around can come back arbitrarily different, and the hash cache means unchanged files are not re-read.

ejected is persisted and restored at startup, because a restart is exactly what an operator does after noticing a drive fell off, and an in-memory flag would let the following scan read the empty mount point as an erased library. It lives in roster.db and not in node.toml: it is runtime state, and an operator's hand-written config must not be rewritten because a USB drive was unplugged.

Auto-eject is the safety net. If a removable root's path disappears, the availability sweep sets ejected as though the operator had clicked it, and reports it so the daemon persists it. Nothing is deleted: index entries, cached metadata, thumbnails, chat history referencing those files and app directory configurations all survive, the last flagged as temporarily invalid rather than wrong.

6.3 Indexing

The index is content-addressed: GroupIndex is keyed by blake3, so the same bytes at two paths inside one group are one entry. This is why a scan can report ten files and index nine, and it decides how reconciliation must work:

Anything comparing disk against index must compare ids, not paths. Comparing paths makes the second path of a duplicated file look like a missed event, every sweep, forever — rewriting the entry, bumping the version and pushing an index update to every connected peer.

Hashing is partial above 40 MB. A hash exists for content identity, and a 4 GB file does not need 4 GB of I/O to be identified with overwhelming probability:

Size Method hash_version
≤ 40 MB full read 1
> 40 MB blake3 over the first 20 MB ‖ last 20 MB ‖ 5 MB at the midpoint 2

Head and tail catch container headers, trailers and files that differ only at one end; the mid-sample catches files sharing a header and trailer. Below the threshold a partial read would sample the whole file anyway, so the full path is simpler and produces the same value — which is what keeps small files cross-comparable between nodes of different versions. A large file indexed by a node of each version produces different ids and does not merge in cross-group search; that resolves itself when both upgrade, and is the accepted cost of not reading 4 TB to build a library.

hash_version is an additive index field with a default, so an entry written before it deserialises correctly and needs no protocol bump. The hash cache carries the column and auto-migrates on open; large files are re-hashed lazily on the first scan after an upgrade.

Periodic reconciliation is mandatory on every platform, not a backstop: ReadDirectoryChangesW drops events under load on Windows, and inotify is unreliable on a FUSE-mounted volume on Linux (§10).

Progress accounting covers the real-time path too. A file's size joins the total the moment its debounce timer is first scheduled — not on every re-trigger, or a cancelled-and-rescheduled timer double-counts — and joins the scanned total when its hash finishes. The scanning flag clears only when no timer is pending and no hash is running, because the in-flight hash of a large file is the entire reason to show progress.

index_progress is deliberately not sealed: counters only, every 2 s.

6.4 Uploads

Five protections, and they are the substance:

  • a filename allowlist;
  • no overwrite — a colliding name gets a free one. The check is Path.exists(), and stat() is itself case-insensitive on NTFS and exFAT, so this already holds there;
  • strict chunk ordering;
  • a size cap — 8 GB per file by default, and the operator's to set (max_upload_gb in node.toml, on the Node page, or meshbay-node transfers max-size), because the disk that fills is theirs and a Pi and a film server are not the same machine. It is read from the transport context on every chunk, so a change applies to an upload already running. A file past the ceiling is refused at the chunk that crosses it and its .part is deleted. There is deliberately no aggregate quota yet, and that gap is named in §15.3 rather than left to be discovered: a member can still fill the operator's disk one capped file at a time;
  • the target root must be writable and available, enforced by the node.

There is no quarantine subdirectory. A folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. What made a quarantine worth having was never the subdirectory — it is the four rules above, and they are unchanged. The client now names the destination folder, which is safe for exactly one reason: it is resolved through RootSet.resolve() (§6.2). A member answers "which of this group's folders", never "which path on the operator's disk".

If the named root is unavailable the upload fails with a stated reason and never falls back to another; if the group has no writable root, uploads are refused rather than guessed.

The node enforces it; the interface merely stops offering it. Each root's writable flag rides the handshake ack and the index payload, so a client knows whether to draw the Upload button and the chat paperclip, and changes are broadcast to everyone connected. None of that is the control: a member on an old tab, or one speaking MNP directly, is refused by the node.

Files can be dropped onto the Files tab, files and folders alike, under the Upload button's rule (a folder on screen, in a writable root, in a group — not in Search). A drop is decided whole before anything is sent: a name already in the folder refuses it, compared without case, because the node's no-overwrite rule would otherwise store a colliding file under a free name nobody asked for and refuse a colliding folder half-way through; and a name outside the filename allowlist refuses it too, from a client copy of the rule that a test holds to the node's answers. A folder is rebuilt one dir_create at a time, parents first, and its files are fed to the transfer store a few at a time so the member's queue (§5.5) never reaches its cap. None of this is a control — the node still enforces every rule above; it is what keeps a drop from ending in a partial copy.

In a group with no writable root the interface says so plainly rather than picking one — a fallback that chooses whatever comes first only moves the failure to send time, where the person has already chosen the file.

Archives are not the node's business. A member downloading a folder as a zip fetches the same encrypted chunks as any other download and assembles the archive in the browser. The node serves no bundles, holds no temporary files, and cannot be asked to compress anything — one fewer place where a request turns into work on someone else's disk.

Directory creation is not privileged. Directory removal is, and is refused unless the directory is empty. The emptiness rule is the safety property: whatever the caller intended and whatever the client sent, the operation cannot destroy content. It is checked twice — before the challenge is issued and again after the signature returns — because a file can land during the round trip to the operator's browser.

6.5 Derived data and enrichment

The governing rule:

Enrichment happens on the client, from data it already has. What the client cannot compute, the node produces — and where the node produces it for everybody, it caches it in its own data_dir, never in a shared root.

The client half is unchanged and is why a chat image thumbnail costs the node nothing: the browser already decrypted the image and scales it itself.

The node half was decided against the alternative and the reasoning is worth keeping, because it looks like a violation of "no derived state" and is not:

  1. A third-party API quota is per credential, not per device. A token shipped inside every install and called from every client scales with the number of devices in existence. A node making the calls on behalf of its own members makes the number of nodes the denominator, and one lookup per unique title serves every member indefinitely.
  2. Thin clients benefit from a node that does more. A phone should receive a small ready-made image and a JSON blob, not decode video or hold its own multi-gigabyte cache.
  3. Keeping the credential server-side is a strict improvement over shipping it to every renderer, and it preserves the desktop client's tested invariant: it issues no request outside /v1/ and the signaling socket.

The cache lives in the node's own data_dir — beside chat.db, audit.db, bundles.db — and never inside a shared root. That is not a compromise, it is strictly better, for reasons independent of sovereignty: a shared root is routinely a read-only backup mount or a share the node cannot write to; a dot-prefixed folder is not hidden on Windows and would appear as an ordinary folder full of previews of a private group's content to anyone who plugs the drive into another machine; and it would need filtering out of every listing path consistently, forever.

Delivery reuses the chunk path. A thumbnail is addressed by its own blake3 exactly as a file is by its id, so the chunk handler resolves a requested id against either a real file or the media cache. Same transport, same group-derived encryption, same backpressure — no parallel mechanism, and no new authorization surface.

Cache lifecycle is tied to the index. A file's thumbnail and its third-party match are pruned by the same event that removes its index entry. Third-party metadata is cached per external id with its own refresh window, since several files of one show share one fetch.

A visible node-wide toggle turns third-party calls off entirely, independently of any application being enabled, for an operator who wants zero third-party network traffic. This is genuinely new node behaviour — egress to a third party, and a disk-resident cache with a real deletion obligation — and it is stated rather than left implicit.

Anything that shells out to a media tool obeys three rules, all of which this codebase has paid for:

  • its own small bounded pool with a short timeout, never the streaming pool — a grid of fifty videos would otherwise exhaust every streaming slot on the node, since a stream slot is held for the length of a film;
  • drain the pipes, then wait with a timeout, and release the slot regardless — a process that outruns a paced reader cannot finish closing while its stdout is full, SIGKILL or not;
  • a background task must be held, or the loop may collect it mid-flight and the slot is lost for good.

Link previews are a further instance. A URL pasted in chat is unfurled by the node: the browser cannot (a strict img-src/connect-src, and CORS), and a direct fetch would leak every reader's IP to the linked host on each render. The card text lives in a bounded in-memory TTL cache; the image rides the same blake3-keyed store as any other thumbnail. The new surface is SSRF, because the URL is a member's choice and it triggers an outbound request from the operator's machine: http(s) only, no credentials, a port allowlist, every resolved address must be globally routable, redirects followed by hand so each hop is re-checked, the connect address re-checked against the checked one, a response-size guard, and a per-member rate limit. The operator can switch previews off per group.

Every new outbound or cross-trust surface needs a bound and a named adversary in the same commit. That is the standing rule this section exists to enforce.

6.6 Chat storage

One SQLite database per group at data_dir/<group_id>/chat.db. Rows carry the sealed ciphertext, the epoch, the sending device, the nonce and the signature, with a unique (device, nonce) refusing replays.

Paging is backwards — get_recent / get_before / has_before — because a chat opens at the newest page. A forwards pager is not what a chat opens with.

meshbay-node chat prune <days> deletes messages only, never an epoch key. An epoch with no messages is harmless; an epoch key deleted while messages still need it is an unreadable archive.

A message is bounded in size and in rate, like every other member-supplied write. Sending one costs the operator a row that nothing expires, every other connected member a relayed copy, and every member of the group a notification — so the two bounds answer the two halves: 64 KB of ciphertext for what one message may cost, and 60 a minute per account per group for how often one member may impose it. Both are checked before anything is stored or relayed, and a refusal names itself (chat_too_large, chat_rate_limited) and is audited.

The numbers are meant to be invisible. The sealed payload is the text, a thread id, a display name and a timestamp — an attachment is a file on a root and travels as a reference (§4.5) — so 64 KB is some sixty thousand characters, and sixty a minute is far above a person typing. The rate is keyed by account, not by connection: a second tab does not make anyone type faster, and keying on the session would hand a script one budget per socket it opens.

There is deliberately no node-wide chat ceiling beside the per-account one, and the contrast with link previews is the reason. A preview spends the node's egress and its third-party quota, which is one shared thing and deserves a shared bound; a chat message spends the sender's own group. A node-wide ceiling would let a busy group silence a quiet one — the same defect this bound closes, one level up.

6.7 Operator surface

Two personas need different tools, and the headless one is the normal deployment:

Operator Reaches the node via
Desktop the desktop client's Node page
Headless / SSH the CLI

Every operation is reachable over SSH with no browser on the host, with one exception named in §15.3 rather than left to be discovered: revoking a single device of a member is an interface action, and the CLI's only lever is member unpin, which removes all of that account's devices at once. status deliberately reads the keystore and config directly so it works while the daemon is stopped — the state an operator is most often in, since the daemon will not stay up before its key is linked or a group exists.

meshbay-node status
meshbay-node group list|add|remove
meshbay-node root list|add|remove|set|eject|plug
meshbay-node gek init|rotate
meshbay-node operator pair
meshbay-node member list|invite [--link]|cancel|revoke|unpin
meshbay-node file list|rm
meshbay-node video rematch
meshbay-node chat status|rotate|encrypt-history|prune
meshbay-node transfers show|set|max-size|per-member
meshbay-node denylist show|clear
meshbay-node stun list|add|remove|reset
meshbay-node reload|restart-daemon

This block is a list of verbs that exist. It carried member device list|revoke for as long as device linking has been built, and no such verb was ever written — device falls through to the username resolver and fails. A command list in a specification is read as a promise by whoever writes the next document from it, which is exactly how it was found (§15.3).

member revoke/unpin resolve a username against the roster and refuse an unknown one rather than acting on nobody — a typo must not look like success. Revocation tells the operator what it does not do: the ex-member stops receiving the key on their next connection but still holds the current one, so the message ends with the command that rotates it.

The node's local control API is JSON only, on loopback, behind a per-run session token (X-MeshBay-Token, printed at startup, file mode 0600). "Localhost only" is not authentication: any local process can reach it, as can a page in the operator's browser via DNS rebinding — and this API re-initialises group keys, issues invitations and reads the audit log. There is no server-rendered dashboard; the desktop client's Node page and the CLI are the two consumers, and each operation endpoint is one _op(...) line onto ops (§5.4).

Over MNP, the node's own controls need a proved operator device. The node-wide surface — node_status, which lists every group on the machine with each root's absolute path, plus node_settings_set, roster_read, denylist_read, denylist_clear and node_reload — is reachable when two things hold: the account is the one the node belongs to, and the device on the connection has proved (device_hello, §3.3) a key the roster holds as an operator. The first alone is a claim in a token the hub issued, and NS4 does not allow it to be authority: a hub that can name the operator is a hub that can be one. The second is what it cannot forge, since it holds no user keys and cannot countersign a device — the same property device linking rests on. A browser that has never been paired therefore reads nothing here, exactly as it can already sign nothing (§5.4).

The accepted cost, recorded as a choice: on a headless server the only admin path is the CLI. The CLI covers every operation, so this is acceptable — but it is a real capability reduction, not an oversight.

MNP is the path that must exist; loopback is the fallback. The operator of a node is not necessarily sitting at it. Any operator-facing control needs its MNP route first, or it renders for nobody on the web.

6.8 Node settings

Eight [node]settings affect what the node does rather than how it starts, and their value is invisible until something goes wrong — so they are surfaced on the Node page:

Setting Default What it controls
invite_ttl_hours 168 how long a member invitation stays valid
pair_ttl_hours 24 how long an operator pairing code stays valid
device_request_ttl_minutes 60 how long a device request waits for approval. Comfort, not security: the code is bound to the keys by its hash
max_concurrent_streams 8 simultaneous video streams. One process per viewer, ~50 MB each; a slot is held for the length of a film, so this counts viewers
max_concurrent_downloads 8 node-wide download leases (§5.5)
max_concurrent_uploads 8 node-wide upload leases. A separate pool from downloads, because the two cost different things and one queue for both makes each cap meaningless
max_upload_gb 8 the largest single upload, §6.4. Fractions are allowed, and it is read per chunk so a change reaches an upload already running
transcode_incompatible_video true whether browser-incompatible video is transcoded during streaming. Unlike remuxing this costs real CPU per viewer

hardware_video_encode (default true) governs whether that transcode may use the GPU. It is node.toml only and not on the page: the capability is established by encoding and reading the result back (hwaccel.py), so the setting means "use it if it works" and there is nothing for an operator to decide from a panel.

Turning transcoding off does not mean the same thing for every source. A codec with an MSE codec string falls back to a copy and the viewer's own decoder decides; a codec with none has nothing to fall back to and the stream is refused, naming this setting.

Settings are persisted in both roster.db and node.toml: the database for immediate effect with no restart, the file so the value survives a wipe or a fresh install. On startup the file is read and a database override wins. The TOML write is a targeted line replacement, never a round-trip through a writer — that file is hand-written, full of comments recording decisions, and a setting changed from a panel must not rewrite the operator's file.

Transfer limits (§5.5) follow the same pattern, with per-group per-member caps as an operator-signed op.


7. The hub

7.1 Role — chosen, not minimal

Hub minimisation was considered and deferred, and may be dropped (decision D4). The hub keeps serving the web UI and remains in the trusted path by choice. That is a legitimate product call; what follows from it is carried deliberately rather than by accident (§2.3).

Stores: accounts (username, encrypted email, status, role), the group registry and membership, IP logs (one year, legal retention), node registrations, refresh tokens, notifications, the moderation blocklist, instance policy, and per-account device keys for hub login.

Does not store: file content, file names, private-group indexes, message content, private keys, group keys, keypair bundles, user identity keys, node IPs beyond ephemeral signaling.

Knows, unavoidably: who is a member of what, when nodes connect, and when a chat message was posted in which group and by which account id. The last is a stable identifier the hub needs in order to skip the author when creating notifications; it carries no content, and it is a known metadata leak rather than a solved problem.

Decides nothing about keys. Hub membership lets someone reach a node; the node's roster decides whether it wraps anything for them.

user_devices is not the key directory that was H3: nothing reads it but the hub, nothing wraps a group key for it, and it is a different key from the per-node identities. What it does cost is metadata — the hub knows how many devices an account has and when each last signed in.

7.2 Node registration and signaling

Registration on the node socket requires a node-scoped token, verifies the node record against the token subject, and derives group claims from the database: a node may narrow the set to what it hosts but cannot widen it, and cannot displace a live registration (C2). Narrowing goes all the way down: an empty claim is a claim on nothing, never on everything. Reading it as "all of this account's groups" made an unconfigured node a registered source for groups it could not serve — including other members' — and since /v1/groups/{id}/nodes answers in registration order, one such node reaching the hub first made a group unopenable for everyone in it (2026-09-11). The ceiling applies to every message that changes the set, not only to the registration: a node that may narrow on connecting and widen on reload has no ceiling.

The socket is accepted before anyone is known, so the auth message must arrive within ten seconds or the socket is closed with 4001: an unbounded first read is a connection any stranger holds open for free. The node sends it on connecting.

A client must therefore treat that list as candidates rather than a ranking, and try the next node on a not_hosted refusal (MESHBAY_NODE_PROTOCOL.md §6.3).

The node authenticates to the hub with an Ed25519 signature over a domain-separated timestamped message — no password and no auth key on a node — and receives a scope: "node" token that is refused for group management. The operator manages groups from a client (NS7).

Signaling is rate-limited, SDP-size bounded, capped per user, and the caller must share an active group with the target node. Otherwise any authenticated user could make a third party's machine allocate peer connections on demand (H6). The address in a NAT-punch request must match the caller's source address.

The offer ceilings bound a member and never an ordinary one (AV30). They are sized against an account with twenty groups on three devices, because a refused offer is indistinguishable, to the reader, from a node that is down:

  • Per account and per node, a budget: a burst of 120 offers, refilled at two a second (OFFER_BURST, OFFER_REFILL_PER_S). This is what bounds a member's cost to one machine, and it spares the others — an account that has spent it on one node still reaches every other. It is counted by account, not by address, because a mobile carrier puts many subscribers behind one IPv4 address.
  • Per account, 32 offers pending at once, across devices and nodes — an offer is pending for the node's round trip, or up to the fifteen-second answer timeout when a connected node is silent.
  • Per address and per node, 600 a minute: a coarse guard in front of authentication, and nothing more.
  • Every refusal carries Retry-After, and a client sends the same offer again on 429, 502 or 503 — the hub busy or restarting — with growing, jittered waits (postOffer in transport.js). Any other failure, 404 for a node that is not connected above all, fails at once, so a dead node costs no time.

The node bounds its own total separately (MAX_PEER_SESSIONS in webrtc_server.py), which is the limit that protects the machine whatever the number of members.

A node registered for no group shares one with nobody, and is refused rather than exempted. Written as "check membership if the node claims any group", the rule skipped itself — membership, group status and the public-group gate together — for precisely the node that AV1 made commonplace: the unconfigured one, hosting nothing, which is also the one least able to absorb the work. No legitimate connection is lost, because such a node refuses the handshake anyway (group_id is mandatory, M1, and a node holding no group key refuses, NS8); the refusal simply stops happening at the operator's expense.

X-Forwarded-For is honoured only from a trusted proxy, rightmost hop (M7), through one helper so the behaviour is defined in one place — including for the rate limiter, whose keying otherwise collapses to a single global bucket behind a loopback proxy (third review L10).

7.3 Groups

A group's identity is its UUID, everywhere: the route, the node's configuration, membership. A group name is unique per owner account, case-insensitively and trimmed, enforced by a functional unique index; two different owners may each have a photos. Names are displayed as name@owner, which is a label plus a create-time check and not an addressing scheme. The handle is hub-local: the same name@owner on two federated hubs are different groups, and a federated row shows its source hub rather than an account.

visibility and join_policy are the two independent axes described in §3.5. join_policy is read from the node's own configuration, never from the hub.

Public group creation is quota'd — ten live public groups per owner account, staff exempt. Public groups are the ones that cost other people something: they appear in the directory and are brokered to strangers. The check is at creation only, which is correct because the update endpoint refuses to change visibility.

An invitation link's hub half is a ticket, and it grants membership to one account. group_invite_links holds sha256(ticket), the invitee's address as the same blind index users has, and a masked form for the owner's list — never the address. Redeeming requires a signed-in account whose verified address has that index; anyone else holding the ticket is told only that it is not for this account, and an unknown, used, expired or cancelled ticket, or a group no longer active, is one uniform refusal. Creating a link says nothing about whether the address has an account (M1). At most twenty outstanding per group, as on the node; a lifetime clamped to thirty days; a node token may create one for its own operator's group (the CLI) and may not ask for mail. The list answered to the owner holds the links nobody has used yet, which are the ones there is still something to do about: a redeemed one has become the member row it produced, and showing both says the same thing twice. The row itself outlives the list by thirty days, so the account that used a link is answered on a reload or in a second tab rather than refused. What the binding holds against, per the convention: third parties — a messaging service that previews the link, a forwarded mail — and not this hub, which verifies the addresses it compares and could already be anybody.

Which nodes host a group is answered to its members. For a public group that is everyone, which is what public means; for a private one it is the membership row and nothing else. Answering any authenticated account — as it did while only the public case was checked — hands whoever knows the group id the identities of the machines hosting it, and an ex-member knows that id for ever. Nothing needs it before joining: an open join writes the membership row first, and an invitation registers the invitee's when the code is created.

7.4 Instance policy

hub_settings is a key/value table an admin edits at runtime. It is instance policy, not group content: it says how this hub behaves and holds nothing about any group's files, index, membership or keys, so §1.3 is untouched.

The first entry is allow_public_groups. Switched off, server-side and read live on every path the hub mediates: creating a public group is refused (staff included — the way back is to re-enable, not to slip past), the public directory returns nothing local and federated, open-joining is refused, a non-member is handed no node to connect to, the "this node hosts an open group, admit anyone" signaling fallback is dropped, and the federation export advertises nothing.

It is a directory-and-brokering control, not a remote kill. Existing members keep their membership and their access. A node whose operator set join_policy = "open" still pins and serves whoever reaches it directly over MNP; what the switch removes is the hub-provided ways to find and reach such a node.

What the hub answers without an account is a reviewed list. test_unauthenticated_surface.py walks every route and fails on one that takes no authentication dependency and is not listed there with its reason; routes that authenticate in their own body (a signature, an e-mailed code, an MHP token) are listed with what they check. The hub publishes no API description — no /docs, /redoc or /openapi.json — in the code, not in a proxy rule, so a packaged install behind any proxy publishes none either.

The mail bounds (mail.*) and the sign-in lockout (login.max_failures, default 4, and login.lockout_minutes, default 60 — §7.7) live in the same table for the same reason: they are what an operator changes while the hub is serving, from the panel, without a restart. Each value is clamped to published bounds, and max_failures = 0 turns the lockout off.

7.5 Moderation

Two verbs on a group, and they are distinct things:

suspend revoke
Hub status = "suspended" status = "revoked"
Node nothing signed revocation broadcast → denylist + live sessions dropped, persisted across a restart
Reversible from the panel yes no

The client shows the real state, not a blanket one. Revocation is honoured by nodes and the denylist survives a restart (H4); signaling refuses a group that is not active.

Moderator is not administrator. The user-patch handler is split by field: a moderator may act on the fields moderation needs and may not write role.

Content reporting requires authentication, distinct reporters and a rate limit, and is refused when public groups are off. An unauthenticated endpoint that blocklists a content hash after two reports is a network-wide censorship and DoS primitive for anyone who learns a public file's id.

The exact-hash CSAM check is structural, not yet functional — production databases are perceptual — and is stated as such so it is not relied on operationally.

7.6 Federation (MHP)

Federation is closed in the code, and every MHP route refuses with a stated 503. federation.FEDERATION_ENABLED is the only thing that decides it — a constant rather than a setting, because a switch in an admin panel invites an operator to turn on something that has never worked between two machines. /v1/hub/info reports it, since the mhp_version beside it would otherwise be a claim this hub does not honour.

The reason is not the design below. It is that nothing has ever run it: two hubs have never completed one authenticated request between them (AV14 — the issuer signed with a key bound before it was loaded and named itself after the reference deployment whatever it was called, and the verifier named no audience for the aud the issuer sets). Both were found by reading, and both stood for a month behind a green suite, because a second implementation of a peer proves the protocol and nothing about two machines — the sentence §12 already writes about a second implementation of the client. It re-opens when a second hub has been stood up and the exchange run both ways.

What the closure does not touch: the public directory still reads whatever federated_groups holds, which is nothing, because nothing can arrive.

Peer hubs exchange directory rows and revocations. The trust rules, which hold when it re-opens:

  • a pushed row's source is bound to the signer, not taken from the payload;
  • the token audience is checked;
  • a push is capped, and replays are rejected;
  • revocation acts on the peer's own directory entries — it does not reach nodes, and nothing local hosts a federated group.

7.7 Account lifecycle

A user can delete their own account from Settings behind a passphrase re-entry — a live token may be a borrowed laptop, and the bar for something with blast radius is proof of the passphrase. An admin can delete one too.

The row is tombstoned rather than dropped: username released, email and password hash cleared, node linking key dropped, memberships, notifications, refresh tokens, node registrations, device keys and public-swarm sources removed, active tokens refused at once by a status check rather than left to expire. Device keys go because the desktop client keeps its half: left on the tombstone, the key would refuse that installation to the next account created from it.

Two things survive on purpose:

  • The IP log, for its legal retention period, and it stays attributable — detaching it would keep the data and lose the only thing it is for. The name is copied onto those rows as the account goes, since the join that used to supply it would answer with the tombstone.
  • Everything on a node. Files, the pinned identity and the keypair bundle live on machines the hub does not command — the same sovereignty that makes admission work. Deleting a hub account is not an erasure request to the operators who host you; the operator surface is where that happens, and the docs must say so.

Groups the account owns decide between the two routes:

  • The owner's own deletion is refused while the account still owns groups, rather than cascading into other people's data — the owner can hand them over first.
  • An administrator's deletion deletes them with the account. It is the route an erasure ordered by an authority takes, and it cannot wait on the person it is about. Everything on the hub that references those groups goes too (db/purge.py, which finds the referencing tables from the schema, so none is left to fail a foreign key on PostgreSQL). Then a signed revocation for the account and for each group goes to every connected node: an access token already issued stays valid on a node until it expires, and the revocation is what makes the nodes refuse the account and close the groups' sessions now. A node that is offline misses it.

Registration is gated by a CAPTCHA whenever one is configured — unconditionally, not only when some other field is absent, or the real client's ordinary request skips it. The desktop client renders the widget too.

Sessions end three ways, all admin settings in hours (session.*, §7.4):

  • A browser signs itself out after browser_idle_hours with no input and no <video>/<audio> playing. The page measures it, because the hub cannot: it hears a renewal from any open tab, attended or not, and nothing while a film plays. The last-active time is shared by the browser's tabs and survives the browser being closed. The desktop application is exempt — its owner's machine, which signs back in with its device key, also after a renewal fails.
  • A refresh token unused for refresh_idle_hours stops renewing, never below the access token's life plus an hour, and no session renews past max_hours after its sign-in.
  • Signing out revokes the refresh token on the hub, not only in the browser, and "sign out everywhere" revokes every one the account holds. Access tokens already issued run out on their own.

Passphrase sign-in locks per username. After login.max_failures wrong passphrases (§7.4) the name is refused with 429 account_locked and a Retry-After for login.lockout_minutes, without the passphrase being checked. The per-IP rate limit bounds one address, and IPv6 hands every subscriber a /64 of them; an online guess targets an account, so the account is what is counted. The rules that make this safe:

  • Counted by the name as typed, existing or not. An unknown name locks exactly like a real one, so login stays uniform (M1). The key is a hash: people type passphrases into the username field.
  • The attempt is taken before the check, in one statement — an INSERT … ON CONFLICT DO UPDATE … WHERE … RETURNING — so a concurrent burst gets no more attempts than the limit. A request that checked no passphrase gives its attempt back.
  • Every path that checks the passphrase counts on the same row: sign-in, passphrase change and account deletion. A right passphrase clears it; failures older than the window age out.
  • A lockout refuses passphrase sign-in and nothing else. Open sessions, token renewal and device sign-in continue, and a reset code sent to the address on file clears it — so a stranger who locks a public username costs its owner at most a new sign-in (AV26). A session learns its own lockout from /v1/users/me, because a passphrase change re-wraps every node's bundle before the hub accepts the new passphrase and must not start when the hub would refuse.

8. Clients

8.1 Two clients, deliberately

Hub-served web SPA Desktop client
Distribution served by the hub installed, signed release
Code integrity T3 accepted — the hub can inject detectable if reproducible builds ship
Key storage IndexedDB / sessionStorage, plus a bundle on each node OS-protected local storage; keys never bundled
Transport WebRTC WebRTC + QUIC (sidecar, for hub-less group://)
Downloads to disk where the engine allows it native, streamed, unlimited
Positioning convenience tier — zero install recommended for sensitive use

The SPA is not deprecated and stays. It is the zero-install path, and the objective is explicit: a native client must not prevent web use. It must be labelled honestly — the application page states that the hub serves this code, and the docs never claim end-to-end integrity for that path.

Several browsers, one identity per node. A browser keeps nothing durable the user controls, so the identity it creates for a node is left with that node, encrypted under the passphrase. Any other browser recovers it there with the passphrase alone: same identity, same pin, no second code. Joining a different node creates a different key and needs that operator's code — the first contact it has always needed. This is what makes the product behave the way people expect, and it is also C4, with a blast radius of one node.

8.2 The desktop client

Electron, with an optional Python sidecar for hub-less group:// over QUIC.

The shell choice follows from what the interface actually depends on: not "the web" in general but engine-class APIs — RTCPeerConnection, WebCrypto X25519/Ed25519, MSE, service workers, File System Access. Keeping the engine keeps the transport, crypto, key-derivation, download and player modules as the client. They are not browser workarounds to be deleted once native; they are the implementation.

The non-negotiable: UI assets ship inside the package and load from disk. A shell pointing at the hub's application URL is a browser with a different icon and fixes nothing.

What running it establishes, and what each fact costs:

  • A CSP in a meta tag silently drops frame-ancestors. It is sent as a header by the protocol handler.
  • Service workers do not work on a custom scheme. The application has none and uses the native save dialog; the worker stays for the browser.
  • A secure context is what makes crypto exist at all — without it the whole subtle-crypto surface is undefined, AEAD included.
  • The renderer cannot call the hub. Its custom-scheme origin is refused by CORS, and the hub deliberately has no CORS middleware — its API is reachable from no web origin. Every hub call leaves from the main process, which refuses any origin that is not the signed-in hub. A script served by the hub is refused by the policy, which is T3's mitigation demonstrated rather than asserted.
  • The device's hub key lives in the main process, never in the renderer. Generated, stored and used there; the interface asks for a signature and is never handed a key. Same rule as the save dialog, and for the same reason: the renderer parses decrypted content from nodes, which is attacker-controlled input.
  • OS-backed secret storage is real on a desktop and honest without one. With a keyring it is keyring-backed; headless, the same code reports unavailable and refuses to store rather than downgrading silently.
  • Installation places files, never secrets. No key generation in a package's post-install step or an installer custom action — a golden image would give every machine the same key.
  • Hardware video decoding is asked for and then verified. Chromium ships VA-API off on Linux, so the client enables it where a render node and a driver are present, and then asks navigator.mediaCapabilities whether the codec the node streams really decodes powerEfficiently. A no moves to the next GL backend on the next launch, and an exhausted list drops the switches — a feature name that a Chromium release renamed must not pass for a feature that is on, and --ignore-gpu-blocklist must not survive on a machine it did not help. The package recommends the drivers; nothing requires them.

Two guards apply to anything the hub can display inside the application, which is a phishing surface: plain text or a very restricted markup subset, never raw HTML; and a visually distinct region labelled as a message from the hub operator, never a modal that can imitate application UI.

The download page is a security page: it publishes the release key fingerprint, and a hostile hub serves that page too. The fingerprint must also be published somewhere the hub does not control, or the relocation of trust is circular.

8.3 One UI source

packages/meshbay-hub/src/meshbay_hub/static/ is the interface, for the web and the application alike. The client's build copies it and CI fails if the copy drifts. Never edit the copy by hand.

Shipping the UI in a package creates version skew for the first time. Today the SPA and the hub deploy together, so a response shape and its caller change in one commit. Once the UI is installed rather than served, the hub API is a compatibility surface — which is why GET /v1/hub/version carries a minimum client version, and why the client checks it before connecting (§5.6). Cheap now, awkward later.

A second copy of the hub address is what breaks the application, not the protocol. A module that decides where the hub is — "empty string, same origin" — is true of a page the hub served and false of one loaded from a package, where a relative API call hits the application's own protocol handler and sign-up and sign-in fail. There is one seam, platform.hubBase(), and a test refuses any file that decides where the hub is or fetches the API relative to the page origin.

8.4 URL space

Space Served by Seen by the desktop client
landing, about, downloads, news the static site overlay ❌ never
the application the hub (SPA) ❌ never
the versioned API and the signaling socket the hub ✅ only this

The desktop client issues no request outside the API and the signaling socket. That is a testable invariant, and it is what any feature adding third-party egress must preserve — which is one of the reasons enrichment is node-side (§6.5).

8.5 Downloads and streaming

Downloads go to disk, never through RAM, on every platform. There are three mechanisms — a granted directory handle, a service worker streaming a response, and a blob as the floor — and which exist depends on the engine.

A fallback chain reaches its floor silently. Where the first two do not exist, every download went through memory and the only visible symptom was a save dialog at the end instead of the start. When a chain degrades, check what the floor costs on every platform that will reach it.

Specific rules the download path is built on:

  • A service worker being active is not the page being controlled. An uncontrolled page's requests never reach the fetch handler, so the stream is handed over and never asked for. Require the controller, and have the worker confirm it served the request.
  • An idle service worker is killed, and a streaming response is not "something to do". The page pings the worker while it writes and the worker answers, because receiving a message is the event that resets the timer.
  • Every await on a download path is bounded and says which chunk it gave up on. An unbounded await is a freeze nobody can report.
  • A filename in a content-disposition header needs both forms. The RFC 5987 encoding uses ' as a delimiter and the standard escaper does not escape it, so a plain ASCII fallback rides alongside — the next surprise loses accents rather than the whole name.
  • Three headers decide whether a page may frame itself and they must agree — the frame source policy, the frame-ancestors policy and the legacy frame option. Same-origin framing is what a streamed download needs; refusing every foreign origin is unaffected.

Pause and resume, and where resumability actually lives. A pause button that quietly restarts a download from zero is worse than no pause button, so the interface offers only what the target can do.

Resumability is a property of the target, not of the platform. The same engine yields a resumable target from a granted folder and an unresumable one from a service worker, on the same page, for two files in the same batch. So each target declares it itself and the record travels with the transfer; the widget renders from that record. This is the fallback-chain rule (above) applied one level up: whatever tier a transfer ended up on, it records which, and nothing infers it from the platform.

Two design decisions rather than implementation details:

  • A paused transfer holds nothing. Resuming rejoins the queue at the tail and the interface says so. Anything else lets one member close the node by pausing.
  • A resumed file is truncated down to the last whole chunk, never appended to. Writes are sequential whole chunks, so a size that is not a chunk multiple means an interrupted write — and a silently corrupted download is worse than a failed one. The resume record also stores the file's content id, so a resume whose file is no longer in the index fails with "this file has changed on the node", which is the truth.

Uploads resume through the seal, not around it. The node already holds the partial state; the question "how much do you have" is asked as an ordinary sealed upload message with no bytes and a probe index, and answered inside the sealed ack. Asking on the lease message instead would have put the operator's filenames on an unsealed message — precisely what sealing the write path bought. The partial state is keyed by member, directory and filename in the group context rather than the session, so a reconnect finds it, and an orphaned partial with no live lease is reaped on a timer and at startup. The four upload protections (§6.4) are untouched by any of this.

Video streaming is fragmented-MP4 remux (or transcode where the codec has no MSE string) fed to a source buffer, with the node holding one slot per viewer.

  • The re-encode runs on the GPU wherever one works — VA-API on Linux, Quick Sync or NVENC on Windows. Which one is established by encoding 1080p and reading the file back, never inferred from the hardware or from ffmpeg's encoder list, and nothing is accepted that does not produce the exact profile and level stream_init announces: an encoder that wrote another level would make the node's own codec string a lie, and the client checks that string before it trusts a byte. Every mode falls back to libx264 — per source codec, because a GPU that decodes HEVC may have no decoder for MPEG-4 Part 2 and only asking it finds out. This is what lets a low-power node keep transcode_incompatible_video on: the setting refuses a capability, and hardware encoding is the mechanism that makes refusing it unnecessary.

  • Flow control is a window, not a debt. Read-ahead is bounded by time past the playhead, with a small window of segments in flight topped up as they land, driven by a clock and by playback and never by arriving data. Granting a credit per append pulls at network speed, fills the buffer ceiling, and then goes silent for the length of the accumulated balance.

  • The read-ahead bound is a byte budget, stated in seconds. What a browser limits is bytes, so a bound written in seconds has to be sized for the highest-bitrate file in the library and every ordinary file then holds a fraction of what the same browser would have taken — measured against a 100 MB ceiling, a 1.0 Mbit/s film reached its ninety seconds on 20.7 MB and a 2.3 Mbit/s one on 45.6 MB. On a mobile network that unspent ceiling is the whole of the margin there is for a dead spot. The budget is therefore in bytes, converted with the file's own average bitrate, and the fixed bound is its floor and never its ceiling: nothing pulls less than it used to.
  • A ceiling that varies by engine and device is walked up to, not declared. The two ways to find a limit nobody publishes are to overshoot it or to approach it, and overshooting costs a refused append every time it is tried: a flat budget against a smaller ceiling measured 1386 refusals on a film that had none. So the budget never exceeds one step past the most read-ahead that SourceBuffer has actually accepted, and a refusal takes the ceiling down below where it broke and leaves it there — backing off and climbing again is a sawtooth with a refused append at every peak.
  • The budget only grows while the film is playing. What it buys is a film that survives a dead spot, and a film nobody has started cannot be interrupted. Left to grow on the loading screen it filled a small engine's whole buffer — 40 MB where the fixed bound took 8 — with film on a connection that is paying for it, and reached a ceiling the fixed bound never came near. Autoplay is blocked on a phone more often than not, so that is the ordinary first state of every playback. Pausing does not shrink it: the lead already paid for is the viewer's.
  • A reconnection carries the read-ahead across; it does not spend it. The old stream dies with the connection, so something must be asked for again, and the question is where. Restarting at the playhead goes through the reinit that empties the source buffer — which a seek needs and a reconnection does not: the viewer navigated nowhere, the film kept playing out of the buffer for the whole outage, and emptying it at the moment the link returns is what finally stops the picture. So where there is read-ahead worth keeping, the stream carries on from the end of the buffer and what is there is left alone. The two modes are not interchangeable — resuming keeps the buffer and must not move the playhead, seeking discards it and must — and the choice is one pure function so it can be tested by being run. Below a few seconds of read-ahead, and for a stream that already ended, the seek is still what happens: there is nothing to save, and it is the proven path. The node lands on a keyframe at or before what is asked for, so the new material overlaps the tail of the old and the coded frame processing replaces it; a start that somehow fell past the buffer would leave a gap, which is a permanent silent stall, so that case falls back to the reinit rather than trusting the reasoning.
  • An append refused for quota is not retried until something frees room. It will be refused again for as long as the answer has not changed, and the pipeline's clock calls the append path several times a second: the retry alone cost minutes of refusals while the film played far enough for eviction to have anything to drop. Eviction is the only thing that gives a SourceBuffer room back, so its success is what lifts the hold — not a timer, and not the playhead, which moves long before the answer changes. This cannot deadlock: a full buffer is one holding film the playhead has not reached.
  • Credit follows the buffer, decided in one place. The append path grants nothing, because the buffer's update event fires for evictions too — crediting from it pays the node for the player's own housekeeping.
  • Flow-control accounting comes before every early return. A segment that arrived is no longer in flight, whatever is then done with it.
  • A new stream starts from a known state, reset at the start of the stream and never in the teardown of the one before, which is skippable.
  • A viewer holding credit deliberately must say so, or the node's stall timeout ends a film that is merely paused.
  • Seeking restarts the source with an index seek before the input, clamped away from the end and echoed back; the client supplies the timestamp offset, because copying timestamps does not preserve position.
  • stream_init.start names where the picture begins, never where the viewer dragged, and on the copy path that position is measured, never predicted. The client builds SourceBuffer.timestampOffset out of this number and a subtitle cue carries the source's own absolute time, so every second of disagreement puts a line on screen a second away from the voice saying it. Reading the key frames and taking the last one at or before the request answers a different question twice over: Matroska's Cues index only some keyframes, so an index seek backs off to an indexed one that can be much earlier, and the landing point moves with which streams are mapped, because the container is positioned where every mapped stream has data — one real seek answered 4909.863 s from the frame list and delivered 4907.236 s. So the node runs the same seek with the same mapping, copies one frame under -copyts, and reads the answer back: 0.06–0.07 s, cheaper than the scan it replaced. Only the copy path needs it — re-encoded video begins exactly where it is asked to.
  • A seek trims what it can, and what it can differs per stream — which is a desync, not an inconvenience. Accurate seeking cannot trim copied video, which must begin on a keyframe, but it does trim re-encoded audio to the exact request. The output then carries video from the keyframe and audio from the request, a whole GOP apart, with a hole between them. So accurate seeking is off wherever video is copied and on wherever it is re-encoded, which is the only place the two can agree on where to begin.

Every timestamp was correct while this was happening. First PTS per stream, durations, spans, and the browser's own audio/video delta through MediaSource all agreed, because both streams genuinely sat where the container said. Only the content at a given instant was displaced. A property that every timestamp-shaped check confirms is not thereby true: this one needed the output decoded and compared against the source, frame against frame and envelope against envelope. - The viewer picks the audio track, and picking one is a seek. One ffmpeg carries one audio track, so there is nothing to switch inside a running stream: the node is asked again at the current position and the source buffer is reset the way any seek resets it. It costs nothing extra — audio is transcoded on every stream anyway — and mapping the first track unconditionally, which is what this replaced, made a dubbed library playable in one language only. The node reports the tracks it found and the one it used; the client draws its selector from that list and from no version number, which is what keeps the addition MINOR. - Losing a peer must stop its work, not merely forget it — anything holding a resource is shut down on the way out, or a closed tab transcodes for the length of the credit timeout. - Resume positions are per file, per device, in local storage. No protocol, and nothing new learns what you watch.


9. Group applications

9.1 The plug-in architecture

A group's UI is a set of pluggable applications, not one page. Two things motivated the split: one file had become the thing every unrelated change touched, and the roadmap wanted several more group-level surfaces — none of which need a protocol change, because the indexer already classifies files as video, audio or image and they read the same index, chunk and stream messages the explorer already uses.

group-page.js  ─┬─ the shell: the connection, the file index, the tab bar,
                │  the video/preview modals — nothing app-specific
                ├─ apps.js ─── the registry: [{ key, icon, labelKey, Component, Settings? }]
                ├─ chat-app.js · files-app.js · video-app.js · music-app.js · photos-app.js
                └─ group-settings.js — not an app, always present, never toggleable

Settings is not an app and cannot be disabled — it is the one way back if everything else were turned off.

Shared infrastructure lives in its own modules (icon.js, file-utils.js, hub-client.js, settings-ui.js, folder-tree.js, source-merge.js) rather than being re-exported from the shell, because the shell importing an app that imports the shell is a cycle, and ES modules answer that with a temporal-dead-zone error at first render: the component simply does not appear, with nothing in the console to say why.

9.2 What every application receives

The shell builds one props object per render and spreads it into whichever application is active. Every registered component gets the same context and destructures what it needs — a new application does not get a bespoke prop list.

Prop Why it is here rather than local state
entries, nodeDirs, nodeRoots the group's file index. Chat needs it too, for image attachments — lifting it avoids two copies going stale against each other
applyIndex(...) anything that mutates files calls this, so every application sees the result
onPreview(entry) opens the shell's modal; an application does not own modal state
transportRef, gekRef refs, never state, so a reconnect does not re-render every application
deviceReady the exception, and why it is a prop. A ref not re-rendering is right for a transport reached into on demand and wrong for a fact about the connection an application renders from
mayUpload computed once; a second derivation would eventually disagree with the first

An application that needs local state owns it. One pattern is worth carrying: any notion of "current location within the group" resets on group change, because a directory from the group just left rarely exists in the one just entered.

9.3 Enablement and settings

enabled_apps is a per-group setting on the same pattern as everything else the operator decides: stored on the node (roster.db — not the hub, not node.toml, for the reason in §1.3 and §6.8), changed by a signed operator instruction, and enforced by the node refusing an unrecognised or empty set.

Validation happens before a signature is ever requested. The whole set is signed in one message rather than one op per application, so ticking several boxes costs one signature. The subject is the sorted, comma-joined list, built identically on both sides so the two arrive at identical bytes.

The node's allow-list is the server-side enforcement — a client that names an application this node does not know is refused, and an application the node refused could not demonstrate anything. That entry and the client's registry line are the whole of what adding an application costs.

A group that has said nothing gets Chat and Files. files is always enabled and not toggleable, and is added to the list at every writer so the two agree. The ability to hide it was misleading: the protocol permits root exploration regardless, so hiding the tab only ever misled.

A group with no stored set gets the default, never the whole registry. Falling back to everything registered would turn on an application nobody chose — including one shipped behind a development flag.

Changes are broadcast to everyone connected, so a disabled tab disappears without waiting for a reconnection. A client that has not yet received the list shows everything registered — a node that predates an application hides nothing.

An application's directories are the same shape one level down: one generic signed op (app_directories) keyed by the application's own registry name, stored under <key>_directories, one MNP message, one loopback route. Adding an application adds no function, no message type and no route — which is what "plug-in architecture" has to mean to be worth the phrase.

The per-application variants are gone, and the reason is the interesting part. Three messages, three signed ops, three handlers and three ops wrappers were the same instruction three times, differing only in the key they wrote and whether they carried a string or a list. That shape is what made adding an application mean adding a message type, an op, a handler and a widget — and it meant three validation paths, of which the older ones validated nothing: a typo was stored, matched no entry, and the application showed an empty tab with no way to tell "misconfigured" from "no files yet". One op has one validation path, and an unknown application name is refused rather than stored.

What stays is a read fallback: the roster still reads the older per-app keys out of its settings table, because that is a key on an operator's disk rather than on the wire, and a node upgraded into this must find its own configuration (§5.6).

Each application's settings pane is its own file, named in its registry entry, and every pane takes the same props and nothing else. The split is the point:

What every application has, the page does generically; what one application alone has, the pane does itself. Pointing an application at folders goes through the shared saver; a third-party credential or a per-group switch is the pane's own business, made with the transport it is handed.

An application that only needs directories therefore touches neither the settings page nor the shell.

The folder picker asks the node for nothing. The tree is derived from paths the client already holds, so it shows what the group's index contains and no more. There is no folder-browsing protocol and this does not add one.

9.4 Adding an application

  1. <name>-app.js exporting a component with the standard props shape.
  2. Register it in the registry. key is the wire identifier: it must match the node's allow-list and it is the row the application's directories are stored under. One identifier per application, everywhere. Register the component — and its settings pane — through lazy(() => import('./<name>-app.js'), 'Name') (lazy.js), not a static import: the module is then fetched the first time the tab is opened instead of with the first page, and test_first_load_is_lean.py (add the file to its ON_DEMAND set) refuses a static import that would bring it back.
  3. <name>-app-settings.js if it has anything to configure. Do not import the settings page — that is the cycle in §9.1.
  4. Add the key to the node's allow-list, and — if the application keeps directories — to NodeDaemon.APP_DIR_KEYS, the one list a group's context is built from. Nothing else on the node may name an application.
  5. i18n: at minimum a tab label key in all ten catalogues. A settings key added to the client must be added ten times; write the table and generate the insert.
  6. Nothing to register for caching. Every file under static/ feeds the /a/<hash>/ fingerprint, subdirectories included, so a new or changed module moves the URL by itself — which matters for a module reached through the registry, since nothing imports it by name.
  7. Add the file to the test file-set lists — hook ordering, sticky headers, transport contracts. A file missing from those lists is never checked, which fails silently rather than loudly.
  8. Toolbars pin. A toolbar is a direct child of the page root and is opaque, or content scrolls visibly through it; if anything pins below it, it must publish its own height, which is never a constant because it wraps on a phone. An application with no toolbar renders none — an empty band still holds a strip of the page open.

No protocol change, no hub change, no daemon change. Steps 4 and 7 are the only node-side and test-side touches, and both are allow-lists.

A list of application names is only ever kept in one place, and everything downstream is derived from it. There were three. The daemon built a group's context from one; the handshake ack was assembled from a copy that had already lost an entry — the reference application's, so the one application that exists to prove a new one needs no special-casing was the single one whose directories never reached a client; and the client shell named three applications by hand while the live-update path beside it was already generic.

The fix that holds is removing the copies, not syncing them: the ack emits whatever <app>_directories the context carries, and the shell reads the ack's own keys. Neither can drift, because neither has anything of its own to drift from.

Where the one list lives matters too. It is on the daemon, which is what wires a group's context; the roster, the operator ops, the config and the root set must name no application at all, and a test holds them to it. That is the property the reference application exists to demonstrate, and it is the reason a first attempt at this fix — moving the list to the roster, where the directory storage lives — was wrong and was caught.

A reference application exists in the tree behind a development flag. Every other test of this architecture reads source for the absence of application names, which proves nobody wrote a special case — not that a new application works. Writing a real one immediately found two places the claim was only nearly true.

9.5 Files

The explorer: roots, folders, sorting, selection, upload where the current root is writable and available, download, folder-as-zip, and the eject/plug control beside each removable root. Entries from unavailable roots are filtered out.

9.6 Chat

Messages, threads, attachments, and link previews (§6.5). The composer gates on the connection having identified a device (§3.3) and on a writable root existing for attachments; the attachment directory is a single writable folder chosen by the operator, and the paperclip is disabled with a stated reason when it is not usable.

Own-ness is decided from the account id, in one place, never by comparing display names — and the optimistic local echo carries an explicit flag rather than inventing an identity for itself.

9.7 Videos

A poster browser over the video files in the group's configured folders.

Two modes, both driven by the index: a poster grid with third-party metadata and artwork, falling back to a thumbnail card with the cleaned filename when the lookup returns nothing or a low-confidence match; and a flat, folder-driven list with no third-party dependency, which keeps working with the service switched off node-wide. The mode toggle is a per-device display preference in local storage — it has no authority implication, so it needs none of the signed-op treatment.

A library is read a page at a time, in Videos and Music alike, on the group page and in Search: the pinned toolbar carries previous/next arrows and the range shown, and a page is a slice of exactly what the mode would otherwise draw, in the same order (pager.js). The size — 50 by default, 10 to 200 in steps of 10 — is a per-account preference on the hub (media_page_size, behind the same allowlist as the others), because it describes the reader and not the group's content (§1.3). Show merging (below) runs before the page is cut, so a merged show counts as one card.

Scoping to folders is not a display preference, because it decides what every member's tab shows. It is a per-group setting changed by a signed op, broadcast to connected members, and validated before a signature is requested: a candidate path is resolved against the group's real root set and must name a real, readable directory, so a stale path never reaches the operator's browser as a signing prompt.

Grouping is by unit, not by file: one card per film, one per show — expanding to seasons and episodes. Seasons carry their own text where the service supplies it, falling back to the show's. An operator can correct a wrong automatic match, and correcting one applies to the unit rather than to a single file (for shows) or to the one file (for films), because those are the units each actually is.

The filename parser is a fallback, and directory context is what bare-filename parsing cannot supply. Matching is a scored ladder rather than the first candidate to clear a threshold, with a year-exact rescue for a weak top hit.

Thumbnails, probes and metadata are node-side (§6.5). Index-time probing runs in its own bounded pool after a file is first seen: the file appears in the index immediately with size and hash, and an index delta fills in the technical and parsed fields once ready. No scan is blocked waiting for enrichment.

9.8 Music

An album browser and a player over the audio files in the group's configured folders.

Metadata mostly already exists in the files themselves, which is the real difference from Videos. The order of trust is embedded tags, then filename and folder parsing for what tags do not supply, then a third-party lookup for canonical spelling, a missing field, or cover art where none is embedded — node-side and cached like any other enrichment, and needing no credential, unlike Videos.

No playback protocol change is needed at all. A track is a few megabytes, so playback reuses the ordinary download-and-decrypt pipeline and hands a blob to an audio element. No streaming request, no transcode pool, no stream slot, nothing added to the node's streaming machinery. The one exception is narrow and extension-gated: two container formats tag perfectly and decode in no mainstream engine, so for those the node performs a one-shot whole-file conversion, cached under its own content hash and served through the ordinary chunk path.

That conversion has a size ceiling, and it is the cache's rather than memory's. The result is one row in a 512 MB least-recently-used store sized for thumbnails, posters and subtitle tracks; a source hours long would be a single row that evicts most of the cache to fit and is evicted again by the next few thumbnails. Whole-file is the right shape for the few-megabyte sources this exists for, and the wrong one past that — serving audio of that length means streaming the conversion, which is a different feature and not this one.

The player is persistent across tabs, at shell level: closing the tab must not stop the music. The next queued track is prefetched while one plays — client-side only, a small in-memory cache evicted as the queue moves.

Music is scoped to folders on the same mechanism as Videos. The reason is not cost — tag reads are cheap — it is mixing: a shared tree with more than one kind of thing under it puts everything into one undifferentiated view with no way to narrow it.

9.9 Photos

An album browser over the image files in the group's configured folders. It is smaller than Videos and Music, deliberately, in two ways:

  • One album-grid view, no mode toggle, because there is nothing to fall back from.
  • No third-party service at all — there is nothing to match a photo to. It already is what it is, per its own folder and filename.

"Several root folders rather than one" used to be the third, and is not a difference any more. The generic app_directories op (§9.3) gave every application a set of folders, so what Photos needed — a library scattered across directories — is what Videos and Music get as well. The pane that chooses them is the same shared widget in multi mode for all three; Chat's is the only single one, because an attachment is written to exactly one place. It is worth recording because the per-application shapes converging is the whole point of §9.3, and this section had been left stating the difference it removed.

An album is a directory, exactly as a season is a folder in Videos. Thumbnails are node-side, orientation-corrected, and delivered through the same chunk path.

EXIF is read locally on the node and narrowed on purpose: a capture time and a camera, and never GPS, anywhere, in any field a client receives. The claim this supports is precise, and the one it must not make matters more:

GPS is in the file, for most phone photos, in the original bytes any member with file access can already download. What this design controls is what the application computes and surfaces — not what the underlying file contains.

9.10 Playlists

Playlists are the first feature to need per-account state that spans several groups on several nodes. Built 2026-09-16; full design, and every place the design was wrong before it was built, in playlists.md.

A playlist belongs to one account and is never shared with other members. That scope is what keeps the merge problem small.

Where the state lives. Not the hub — and the operative rule is narrower than "the hub stores nothing about a user", because it already holds small per-account preferences behind an allowlist. The rule is:

No content metadata on the hub.

A playlist is literally a list of content hashes of private-group files, plus the titles needed to render while nodes are offline. That is the exact object H7 removed from the hub, and it is the same rule that keeps resume positions local: nothing new learns what you watch. An encrypted blob on the hub is technically trivial and is still refused.

It lives on the node instead, as opaque per-account blobs in bundles.db — the same shape as the keypair bundle, which the node already stores and cannot read. No new trust boundary: the node is not asked to hold a kind of thing it does not already hold for that same account.

One blob per playlist, plus a small manifest — not one blob for the collection, and the reason is write amplification rather than size. Under a single blob, starring one track rewrites and re-uploads the whole collection to every node reached; split, it rewrites that one playlist. The manifest — names, revisions, tombstones, counts, a few KB — is also the only thing every menu needs, so "add to playlist" draws instantly with every node offline, and a body is fetched only when its playlist is opened or played. Blobs are compressed before sealing and padded after: the payload is repetitive enough to be worth a factor of three, and the padding is what stops a ciphertext length from counting somebody's tracks. The node caps each blob and the account's total, and refuses rather than truncates — a truncating cap silently loses tracks, which is the failure the whole design exists to prevent.

The key is the one thing that must not be got wrong. Identity keys are per node (§3.2), so a blob encrypted under one is unreadable from every other node — the precise opposite of the requirement. The only secret an account holds everywhere is the bundle key, so playlist_key = HKDF(bundle_key, info = "meshbay:playlists:v1"): one derivation at sign-in, two handles, no second Argon2 run, and a purpose-separated subkey rather than the bundle key reused with a different AAD (§4.4's rule). The nonce is 96 random bits and never a counter, for exactly the reason chat's is (§4.5): two devices of one account derive the same key, which is the point. The AAD names the blob's kind, so one playlist's body cannot be served in place of another's.

Merge is the hard third, and the granularity is what makes it tractable. The unit is one playlist, not the collection — which the per-playlist blob now makes true of the storage as well, so two devices editing two playlists do not even write the same row. Revision counters order writes, never the wall clock; and a deletion is a tombstone, never an absence — an absence is indistinguishable from a device that has not seen the addition yet. A node that is offline for a month therefore cannot corrupt anything: it holds an older revision of some playlists and is overwritten per playlist, not wholesale.

The interface is a menu inside Music, not a page. Music's player is already persistent at shell level and already holds a queue that crosses groups; loading a playlist replaces that queue, and below that call a playlist and an album are indistinguishable — so auto-advance, shuffle and prefetch are unchanged by construction. The one genuine code change is that the queue can today only be replaced: "play next" and "add to queue" require it to become appendable, which makes it a small reducer rather than three pieces of component state. Everything else is a context menu on a cover or a track row, and one button in Music's sticky toolbar. Because Music is mounted by both the group page and the Search page (§9.11), a playlist built inside a group is managed from the consolidated view with no second surface and no application-registry entry.

Unavailability is answered at play time, not at add time. Whether a group is reachable is only knowable by dialing, and refusing to add a track because its node is off tonight loses the user's intent permanently to a condition that lasts an evening. So adding never dials; playback skips, distinguishing a file that will not decode (a property of that file) from a group that does not answer (a property of that group, whose tracks are then skipped together).

It adds no dialing and no new streaming path. Sync rides connections the client already makes, and playback is unchanged (§9.8).

9.11 Cross-group search and source merging

The Search page mounts the same media applications across every group the reader belongs to. The difference lives entirely on the entries, in fields the group page never sets: which group serves an entry, that group's transport and key, a connection generation to use as a refetch key, and the merged source list.

A file shared by two groups is one entry, not two. Identity is the content hash: two entries with the same id are the same file, whatever group announced them and whatever their path. Within one group this cannot arise, because the index is already keyed by hash; the duplication is created by concatenating N independently keyed indexes, and by nothing else.

One source is chosen per logical unit — a film, a show, an album, a photo album — not per file, so a show's episodes never stream from two different nodes. A group hosted by the local node wins; otherwise the pick is deterministic and pseudo-random, stable for one reader and spread across readers. If the chosen source is unreachable the unit fails over. Which source was picked is never shown; the badge names the group when there is one source and counts them when there are more.

Two rules for a new application here:

  • Use the shared source tag rather than an entry's group name — a merged entry has several groups. Pass it the whole unit, not the entry the card was drawn from, which is usually chosen for its thumbnail and would under-report.
  • Never re-derive unit keys. Call the application's own exported grouping function. A copy keeps agreeing until one of them changes, and the symptom is a show whose episodes stream from two different nodes.

A group's index is never kept in browser storage. The browser holds keys and playlists; it does not hold a copy of what a group contains. Such a cache existed, for a cross-group search that read it instead of dialling, and it outlived that search by weeks — writing a cleartext file listing that nothing read and no sign-out removed. Its only remaining use would be to draw a group's files while its node is unreachable, and that is refused on its own merits: a listing that cannot be opened is worse than an honest absence.

A group whose node is down is the normal case, and nothing waits for it. A node is a machine in somebody's house, so with a handful of groups one of them is always off. Each index is drawn the moment it arrives rather than when its neighbours finish; the limit on how many groups are dialled at once is a ceiling on concurrency and never a batch, because a batch waits for its slowest member and a silent node is always the slowest there is; and a group that did not answer last time is dialled last, remembered per browser, because a ceiling alone still lets silent nodes hold every place at once. The ordering is advisory — storage that refuses, a private window or a first visit all fall back to the hub's own order, which costs that one sweep and then corrects itself.

Search reaches each group with one connection, through one pool. The sweep, the warm-up and a tile asking for its group all go through ConnectionPool, which negotiates at most six at once (MAX_IN_FLIGHT) and keeps what the sweep opened for the tiles, up to its size; a group past that connects when a tile of it scrolls into view. So a visit costs one offer per group, a refresh costs none for a connection that answers a short ping, and a connection whose index is being read is never the one evicted. Separate ceilings for each of these add up behind each other's backs, past what the hub admits per account (§7.2).

Search dials many nodes at once, so its deadline measures stalling rather than elapsed time. Opening one group from the sidebar waits out the transport's own budgets; in Search a dead group holds one of the pool's places for the deadline — so the deadline is what a page of unreachable groups costs in spinner, and raising it to accommodate a slow link taxes every dead one. A hub that refuses an offer for load is answering, and each retry counts as progress. A node that is not there reports no progress and fails in one window. A node that answers ICE, then opens a channel, buys another window at each step, to a ceiling, because a deadline that only ever resets has none. A flat budget cannot tell a slow link from a node that is not answering, and set low enough for the fan-out it cuts off connections that are still arriving: the same group then fails in Search and opens from the sidebar, against the same node.

The Files explorer is deliberately not merged, and not "mostly not": there each group is a top-level folder, and merging would remove a file from one of them. A test refuses a build that changes this.

A group can be left out of Search — search_listed, a per-group setting on the node, signed by the operator and carried in the sealed ack. Search reads it after the handshake and stops there: no index is asked for, cached or merged, in any of the four views. The case it exists for is a family album that should not turn up in the middle of a film library.

It is a listing preference and it protects nothing, against anyone. The node cannot tell Search's request from the group page's and serves the same index to both; every member lists the whole group by opening it; a client that ignores the flag lists the group in Search too. It must never be described as "private" or "confidential". What it costs is one handshake per unlisted group, because only the node knows the setting — a hub-side flag would save that and put group state on the hub (§1.3).

One consequence is deliberate and is not a bug: albums are keyed by directory, so two groups whose roots have different basenames put the same photo into two differently-named albums, and the merge — scoped to a unit — leaves it in both. They are two albums.


10. Filesystem portability

exFAT and NTFS on Windows are the common case, not an edge case. Most users are expected to share from an external drive. The consequences below are correctness requirements, not compatibility notes.

Property What has to be true
Case-insensitive, case-preserving The index needs a canonical identity and a case-folding collision check at scan time, reported to the operator rather than resolved silently. One directory indexed as two roots is the same problem one level up
Unicode normalization A name written on one platform in decomposed form and on another in composed form are different byte strings. Normalize to NFC for identity and comparison; preserve the original bytes for display and for opening the file
Reserved names and characters A group indexed on one platform can hold names another cannot create. The client sanitises on save and says so; the upload allowlist is the intersection across platforms, or some files are simply undownloadable
Path length limits Use extended-length paths on Windows, in the node and the client alike
Coarse timestamps, local time mtime alone is not a change detector. Size plus mtime with tolerance, and rehash when in doubt
Watcher reliability Change notification drops events under load on Windows, and inotify is unreliable on a FUSE mount. Periodic reconciliation is mandatory on both platforms
No symlinks, no POSIX permissions Simplifications: nothing to defend against, and the node runs as the user anyway

Case folding is for comparisons the code makes itself — index identity, collision reporting, root names, nesting checks. It is not needed for the no-overwrite rule, where the filesystem's own case-insensitive stat() already gives a colliding upload a free name.

A volume that disappears freezes its root's subtree and never empties it (§6.2). Emptying propagates deletions for a whole library as though the owner had erased it.

Never assume POSIX, systemd, or case sensitivity. One platform-specific trap with no counterpart elsewhere: a service unit with filesystem-protection options gets its own mount namespace, so a volume mounted on the host after the service started is invisible inside it — the directory reads as empty with everything else configured correctly.


11. Platforms

11.1 NAT traversal, measured

QUIC native path — residential ISP A to a hosted VPS: port-restricted cone NAT, direct connection established from the server socket's own probe.

WebRTC browser path — mobile and laptop clients to nodes behind two different residential ISPs:

Path Result
LAN, IPv6 direct OK, ~100 ms
Mobile data, IPv6 inter-network OK, ~600 ms
Mobile data, IPv4 only, STUN hole-punch OK, ~650 ms
Laptop → second ISP's node, IPv6 inter-network OK, ~7 s
Laptop → second ISP's node, IPv4 only, STUN hole-punch OK, ~6.9 s

Two ISPs validated, both without TURN. The hub relays under a kilobyte of signaling; the data path is peer to peer.

11.2 Windows

The bulk of the codebase is portable because the portability rules in §10 were treated as correctness from the start. What the port needed is registered as W1–W9 (§13.7) and is done; packaging is built and awaits a clean-machine run.

Two Windows-specific design points worth stating here:

  • Autostart has two modes, chosen at install and switchable afterwards from the Node page: a per-user startup launcher (the default) and a scheduled-task service mode. A per-user default is right for the desktop persona; a service is what a machine that must serve while nobody is logged in needs.
  • A user service unit cannot carry a system unit's user directive. Two unit templates exist, held apart by a test that parses directives rather than searching the file — searching matched the comment explaining why the directive is absent.

The hub stays Linux. macOS is not planned. Windows on ARM and store packaging are out of scope.

11.3 Android

A client, not a host. The platform is hostile to hosting a node — background execution, storage, battery — and fine as a client, which is one of the reasons enrichment is node-side (§6.5).

11.4 Casting

An HTTP relay in the desktop client serves a standard fragmented-MP4 stream that any LAN renderer can play; the relay is device-agnostic. Chromecast discovery and control ship. DLNA/UPnP is designed and not built: it is a second device backend beside the first, not a second relay.

Subtitles are rebased onto the relay's clock before they are sent. The node extracts a track whole, so its cues carry the film's timeline, and the player can use them unchanged because its SourceBuffer is given timestampOffset = start. The relay has no such offset: it forwards the node's fragments, which begin at zero at the seek point. A receiver is therefore sent the cues shifted by -start, recomputed at every restart of the relay, and cues that end before the stream begins are dropped rather than clamped to zero.

The subtitle is served from the relay's own port, behind the same token as the stream, and with CORS. A receiver fetches a side-loaded track with XHR from its own origin rather than handing it to a media element, so without Access-Control-Allow-Origin it fails as a network error and the film plays on with no subtitles and no message. The stream carries the same headers, because a receiver given a side-loaded track reads the media through the same checked path: on one and not the other, the load fails whole. They widen nothing the token does not already govern. The subtitle URL carries a version because a receiver caches a track by address: changing the cues behind a fixed URL leaves the previous language on screen.


12. Testing posture

Security tests are negative assertions — "this attack does not work" — and are verified to fail against the pre-fix source before being trusted. A suite that tests only that features work will happily pin a vulnerability in place as expected behaviour, and a refactor that accidentally fixed one would be reported as a regression. That has happened here, to four findings at once.

Rules that follow from what has actually escaped this suite:

  • A syntax check validates names not at all, and a module syntax check must force the module parser or it accepts template syntax pasted into an object literal and reports success.
  • A second implementation of the client proves the protocol and nothing about the client. An end-to-end harness written in the right order by construction cannot see ordering or lifecycle faults. Source-reading tests are weak evidence and are sometimes the only evidence available; prefer ones that re-derive a value from the source over ones that restate it.
  • A test that models a fix agrees with it by construction. Lift the real functions out of the source as text and execute them; model the environment, never the code under test.
  • A test that reads source and inspects "the first" occurrence of a call stops guarding anything the moment a new call is inserted before it — and keeps passing.
  • Measure, do not read. A stylesheet does not tell you where anything lands; a browser measuring the real stylesheet does. Assert on geometry, and check the test fails with the fix removed.
  • A fixture narrower than real data tests the fixture. Names are the one thing a file browser cannot be given short.
  • Pace a probe like the real thing. A stress probe fast enough to finish inside a timeout hides every time-based fault.
  • "It works now" is not evidence against a race. Force the worst case.
  • Exercise every browser's branch of a shared path before shipping a fix for one of them, and finish with a live pass: launching the real thing finds what source reading cannot.
  • A passing suite over code nothing calls is evidence about that code, never about the product. Two full protocol implementations sat green and unreferenced for months (§13.3 L7). Before trusting a suite, check that production imports what it tests.
  • A comment that contradicts the constant beside it is worse than no comment — one of them is wrong and the reader cannot tell which. The same holds for a document: a number restated away from its definition is a number that will drift.

13. Register of labelled findings and decisions

Each entry states the rule the label names today. Where the label originally named a defect, the subject is given in one clause so a code comment citing it can be understood, not so the incident can be retold.

13.1 First review (design review)

Label The rule it names
C1 Group chat has no shared mutable sending state. A pairwise ratchet shared across a group produces key and nonce reuse; the design that shipped has no sending state at all, which is stronger than partitioning it per device (§4.5)
C2 Tokens carry a groups claim and the node verifies membership before serving content (§5.2)
S1 Every admin endpoint has an authorization check
S2 Email addresses are encrypted at rest, with a blind index for lookup
S3 Token revocation reaches nodes over the hub socket, and the denylist persists (§7.5)
S4 AEAD nonces are 96-bit, per NIST SP 800-38D
S5 Refresh tokens rotate, one-time-use, with family-based reuse detection
M1 (first review) Account enumeration is a known, bounded property of the account-management endpoints; login, device auth and reset-request are uniform
M2 (first review) Node TLS certificates are transport confidentiality only; identity is the Ed25519 key checked at the MNP layer
M3 (first review) Rate limits on key-adjacent endpoints
M4 (first review) Delegation is designed and deferred; the role check is written so it drops in (§3.4)
M5 Chunk key derivation uses info, with salt=None — correct HKDF usage, because the group key is already uniform CSPRNG output (§4.3)
M6 (first review) Argon2id production parameters are applied, and recorded per envelope so they can be raised (§4.6)
N1–N5 Notes, no action: the forward-secrecy model is appropriate to the deployment; login error messages are correct; the hub's legal exposure model is well-positioned; the NAT probe payload is fine; the web/CLI derivation difference is by design

13.2 Node sovereignty

Label The rule it names
NS1 The client proves possession of the group key in the handshake, and the node verifies it
NS2 Admin operations are Ed25519 challenge-response over a structured transcript, never a token (§5.4)
NS3 The node never serves the group key in plaintext. There is no request message for it; the constants are gone from the protocol
NS4 Operator authority comes from the node's roster and from nowhere else. No auto-pin from the keystore, no resolution through the hub, no config key — a config naming one is warned about and never obeyed (§3.4, §6.1)
NS5 The proof is bound to the transport channel (DTLS fingerprints / certificate hash), so a signaling relay that substitutes its own cannot produce it (§5.2)
NS6 sender_id is enforced from the authenticated session, never the wire. It is what the store keys on; it is not what authenticates a message — the device signature is (§4.5)
NS7 The node authenticates to the hub with Ed25519 and no password, and its token's scope is refused for group management (§7.2)
NS8 The node refuses connections when it holds no group key. There is no bypass switch

13.3 Second review (code review) — the default numbering

Critical

Label The rule it names
C1 All content travels over the authenticated protocol. The node exposes no unauthenticated HTTP surface; the per-group file API that served private indexes and plaintext files on all interfaces was deleted rather than repaired, because it duplicated MNP without any of its controls
C2 A node's signaling identity is resolved against the database and derived from it, never taken from the client's first message (§7.2)
C3 Authentication is mutual: the node proves key possession over the client's nonce and signs the transcript, and the client verifies both and pins the key (§5.2)
C4 Keypair bundles are per node, Argon2id-protected, and closed for native devices — and open for any account that also uses a browser (§3.7)
C5 The two halves below, cited together where a comment means "a member must not be able to write what the node then trusts"
C5a Uploads cannot overwrite, are allowlisted, ordered and capped, and ownership is signed by the uploader (§6.4)
C5b No key material arrives from outside. The member-supplied bundle message does not exist; the node generates every copy of a group key itself (§4.2)
C6 One handshake implementation, shared by every transport. A transport that accepts a bare token is the standing example of what an opt-in compatibility branch costs (§5.2, §5.6)

High

Label The rule it names
H1 Per-group isolation on a multi-group node: the chat store, the peer registry and the broadcast set are per group
H2 Every value that originates outside the node — filenames chosen by members, usernames originating at the hub — is escaped where it is rendered. A CSP contains exfiltration but cannot prevent injected inline script, so escaping is the actual fix
H3 No public key is ever fetched from a directory to wrap a group key for. The node wraps for a key the recipient proved possession of, bound to an account by a code the hub never sees (§3.4)
H4 Revocation reaches nodes, drops live sessions, and persists across a restart (§7.5)
H5 An admin challenge is a structured, domain-separated transcript naming the operation and subject, and the client refuses to sign anything that is not what the user asked for (§5.4)
H6 Unauthenticated work a node will do is bounded: a small pre-handshake buffer, a transcode semaphore, per-user pending-offer caps, and a membership check on signaling (§7.2)
H7 Only public groups register content hashes with the hub. Private groups register nothing, and the swarm route requires authentication

Medium

Label The rule it names
M1 group_id is mandatory on the handshake — there is no fallback to the node's first group (§5.2)
M2 The node keystore uses the full Argon2id parameters, recorded per envelope (§4.6)
M3 Operator authority is the roster pin, established locally by pairing. Asking the hub for the operator's key — the obvious-looking fix — would let the hub install itself as node administrator (§3.4)
M4 Every reply carries the request id it answers. Arrival-order matching is a guess that fails silently and asymmetrically (§5.3)
M5 The index has one protection level, not one per transport (§4.4)
M6 Audit rows are attributed to the row's own subject, never backfilled across rows
M7 Client addresses are taken from a trusted proxy's rightmost hop, in one helper (§7.2)
M8 A node announcement requires proof of possession of the key it announces
M9 Node-scoped tokens are refused on the client path (§5.2)

Low

Label The rule it names
L1 The wire contract carries no constants for messages that do not exist
L2 MNP negotiates versions explicitly and refuses with a code (§5.6)
L3 Errors returned to a peer name no filesystem path and no exception detail
L4 Transcripts are length-prefixed, and an empty channel binding raises rather than degrading the proof to nonce-only (§5.2)
L5 The hub serves a CSP and security headers on the application it serves
L6 Registration validates the email field it declares
L7 A module nothing imports is not a protection, and it is not kept. The sender-key and ratchet implementations this finding named were unreferenced for months and are now deleted; a green test suite over uncalled code is evidence about that code, never about the product (§4.5)
L8 An uploader record identifies a file by its id, not by a name at a root

13.4 Third review

Label The rule it names
H1 (third review) Moderator is not administrator. The user-patch handler is split by field so a moderator cannot write role (§7.5)
H2 (third review) Content reporting requires authentication, distinct reporters and a rate limit, and is refused when public groups are off (§7.5)
M1 (third review) The registration CAPTCHA gate is unconditional when a captcha is configured — never conditioned on a field the real client always sends (§7.7)
M2a sender_id comes from the authenticated session (NS6). Now guaranteed by there being one chat implementation: QUIC does not carry chat at all (§5.1)
M2b Chat broadcast is per group (H1), on the one transport that carries chat
M2c No transport runs a synchronous media process on the event loop, and every one is capped
M3 (third review) Link-preview SSRF is gated: rate limit, port allowlist, globally-routable check, per-hop re-check, connect-address re-check, size guard (§6.5)
M4 (third review) Federation binds a pushed row's source to the signer, checks the token audience, caps the push, rejects replays, and scopes revocation to the peer's own entries (§7.6)
M5 (third review) A CSP and security headers apply to the hub-served application, verified against the running app — a mis-tuned CSP shows as a blank page
M6 (third review) Withdrawn. It misread the node registering a hub membership during the CLI invite flow — which is deliberate — as authorization drift
L1–L11 (third review) Opportunistic hardening: relay-registry proof of possession; delete orphaned modules rather than leaving them to be rewired; decide and document account enumeration; an aggregate upload quota; header-only control-API tokens; a freshness bound on revocation replay; state that the exact-hash content check is structural; validate group-name length and charset; require exp and bind an audience on token decode; key the rate limiter through the same client-address helper as everything else; keep diagnostic logging truncated

Two structural recommendations from that review stand as rules:

  • Make transport parity a test, not a habit. Shared helpers in common code with a test that fails if a transport calls a chat or stream path that bypasses them.
  • Every new outbound or cross-trust surface needs a rate limit and a named adversary in the same commit (§6.5).

13.5 Standing trust limits

Label The rule it names
T1 The password split. The hub never sees a passphrase; it holds a verifier for a client-derived auth_key. The passphrase floor can therefore only be enforced client-side (§3.1)
T2 The hub was the key directory. Closed by admission redesign, not by safety numbers: the invite path reads no directory at all (§3.4). Reclassified as H3
T3 The hub serves the SPA. Accepted permanently for browser users. It is the only remaining way an active hub reads content, it is an artifact-level attack rather than a silent lie, and it does not exist for a native client — whose value is realised by reproducible builds, not by packaging (§2.3, §8.2)

13.5b Availability between members (AV)

Added 2026-09-12, after the finding that produced it, and extended the same day when users.py and admin.py were read under it. The first three reviews asked who can read what, who can impersonate whom, and what a hostile node can forge. None asked what a legitimate but misconfigured or careless member costs everyone else — and that is the question a group platform lives on, because every member was invited by someone who trusted them and none of them is an attacker. C2 had asked "can a node claim a group its owner is not in?" and the answer was correctly no; nobody had asked what happens when a node claims one its owner is in but which it does not host, which is how a group went dark for all of its members on 2026-09-11 with its real host online throughout.

The lens, for anything reviewed from here: a participant supplies input; who else bears the cost? Where the answer is "someone other than the sender", there must be a ceiling, and it must apply on every path that writes the state — not only the one where the ceiling was first thought of.

Two shapes recur, and are worth naming because each accounts for several entries. A limit written on one of several equivalent paths — three endpoints send mail and one of them had neither a rate limit nor a captcha; every list in admin.py is bounded and the two outside it were not; the group claim was bounded at registration and not on reload. And a bound that counts the wrong thing — a per-IP rate limit bounds a caller, never the mailbox that receives what they cause, which is why AV10 is a cooldown per account under a limit per IP rather than a tighter limit.

Open for decision, not a defect: moderation.AUTO_BLOCK_THRESHOLD is 3. Three distinct accounts blocking a hash adds it to the list every node enforces, network-wide, automatically, with manual admin removal the only undo. That is already far better than the anonymous version it replaced, and it is still a censorship primitive an attacker buys for the price of three email addresses. Raising it buys little; requiring the reporting accounts to be more than a day old would cost a patient attacker a day and cost an honest reporter nothing after their first. Left as it is because it is a moderation policy rather than a bug, and the person who sets that policy is the operator.

admin.py was read under this lens and needed nothing. Its moderator/admin line is drawn explicitly — a moderator may not change a role, may not revoke, and may not touch an admin's account at all — self-modification is refused, and every list it serves is bounded. It is the part of the hub where the question had already been asked.

Label The rule it names
AV1 An empty claim is a claim on nothing. A node's group set is authorized ∩ claimed, and an absent or empty group_ids registers it for no group rather than all of its owner's — on registration and on update_groups alike (§7.2)
AV2 A client treats the hub's node list as candidates, not a ranking, and tries the next one on a not_hosted refusal (MESHBAY_NODE_PROTOCOL.md §6.3)
AV3 A node speaks only for the groups it is registered for. chat_notify names a group and is checked against that node's set before a notification is written for anyone, and it is rate-limited per node — the fan-out is one write per member
AV4 Nobody names a third party's address. A swarm source publishes a transport and a port, never a host; where a peer is comes from its node record, stamped with the address its announce arrived from. The number of hashes one account may claim is bounded
AV5 An answer is accepted only from the node the offer was sent to. A peer_id is bound to its node, so no connected node can resolve another's pending offer
AV6 A relay proves possession of its approved key. A public key is not a password, and the register call is unauthenticated by design — it is not a user — so the proof is the only thing standing between a stranger and where nodes send relayed traffic
AV7 A node bounds how many peers it holds and how long an unproven one lasts. The hub's cap is per calling account, which is a limit on each member and not on the machine, so without this an operator's exposure grew with the size of their groups
AV8 One account cannot make the hub mail another at will. The invitation email's subject comes from the group row, never from the request, and the endpoint is metered
AV9 No mail is sent from the event loop. smtplib is synchronous and waits up to ten seconds; called from an async handler that wait is the whole instance's, not one request's. Every send goes through mail.send_off_loop. Argon2 is held to the same rule: every derivation runs on one dedicated worker thread (auth.*_off_loop), never on the loop and never two at a time, because two concurrent lanes=4 derivations deadlock in OpenSSL. So is the node's disk: every filesystem call on a group's content — the stat as much as the read, since a stat is what wakes a sleeping disk — goes through roots.off_disk, onto one worker thread per root set. A spun-down or network-mounted root answers its first syscall in seconds, and on the loop that is every group, every stream and the hub socket waiting for a platter. ffmpeg's own output too, through asyncio.to_thread rather than that per-root thread: a temp file is not a group root and has no platter to serialise against, but a whole transcode read inline is still tens of megabytes of blocking read
AV10 Every path that makes the hub send mail is metered, per account. A rate limit that counts by IP bounds a caller, not an inbox. Changing one's address mails an arbitrary stranger, so it carries a cooldown and a daily ceiling; a reset request and a registration resend carry cooldowns
AV13 The mail server is not a relay, and mail.py is where that is decided. Every message passes one function; purpose is keyword-required and checked against a closed list, so a helper that names anything else does not send and one that names nothing is a TypeError. Under it sit a bound per recipient — the thing a person being mail-bombed actually experiences, unmoved by which account, address or endpoint asks — and an instance-wide hourly ceiling, because registration is open and "per account" is a bound an attacker buys more of. The recipient's daily cap counts every purpose; its cooldown is per family, invitations on one side and the account's own steps on the other, because one cooldown for both refused — silently — the sign-up code of everyone who registered within two minutes of receiving an invitation link
AV11 A namespace a client writes into is closed, and its rows are capped. The preference key space is an allow-list plus default_tab:<group_id> checked as a group id, the value is length-bounded, and the row count per account is bounded
AV12 Every list has an upper bound on limit and a floor under offset. Including the ones that take no authentication at all — the public group directory and the content blocklist
AV16 A bound the operator can see and change, and that a restart does not forget. The mail allowance lives in mail_quota, not in a module dict — a deploy used to hand out a fresh budget, and the hub is deployed often. The values are settings with defaults in hub.toml and a block in the admin panel, because the hour a budget runs out is not when anyone wants to edit a file and restart; /v1/admin/mail says how much of the hour is left, which was previously visible only as an absence of mail
AV17 A global ceiling being reached is an event, not an absence. When the hourly budget runs out the administrators are notified — once per hour, because a flood is what spends it and one alert per refusal buries the message under its own cause — and the panel says which of the two ceilings fell: newcomers turned away, or somebody locked out of their account unable to get back in
AV19 Nothing carries the path to the migrations. meshbay-hub migrate derives it from the installed package, so the RPM, the DEB, a venv and a checkout all agree. A unit naming alembic.ini names a file whose %(here)s stops being true the moment packaging moves it
AV18 The hub runs on exactly one worker, and says so at startup. _connected_nodes, _node_groups, _webrtc_answers and the relay registry are per-process: a second worker makes a node intermittently unreachable for half its members, which is a symptom that describes something else entirely
AV14 MHP binds its audience, and the hub reads its own identity at call time. A token is minted for one peer and accepted by that peer only. federation.py bound _hub_id and _hub_sk_pem at import, which is before load_hub_keypair runs, so it signed with None and called itself meshbay.org whatever the instance was named — and the verifier named no audience for the aud the issuer sets, which PyJWT refuses outright. MHP could not complete one authenticated request between two hubs
AV15 A hash is checked for shape before it is a key lookup, on the unauthenticated blocklist endpoints a node consults
AV20 Chat is bounded in size and in rate, like every other member-supplied write (§6.6). A message is a row on the operator's disk that nothing expires, a relayed copy for every connected member and a notification for every member of the group; the only ceiling was the frame size. Uploads had carried four protections and a cap since C5a because somebody asked what one member costs the others on that path, and nobody had asked it on this one
AV21 A lease is what the node granted, not what the client called it (§5.5). tr was read as a boolean, so any non-empty string skipped the leaseless ceiling and every cap behind it, and a queued transfer was held back only by the honesty of the client waiting in the queue
AV22 The node's own controls take no authority from a hub token (§6.7). node_status, node_settings_set, roster_read, denylist_read, denylist_clear and node_reload were gated on the account id in the JWT, which is the hub's to choose — NS4 and M3 with the check written the other way round. The gate is a proved operator device, which a hub holding no user keys cannot produce
AV23 An upload's owner is recorded when the upload ends and applied when the entry is created, which are different moments (§5.4). Written against the index at the end of the upload it matched nothing, every time, and left every uploaded file owned by nobody — so no member could delete what they had sent
AV24 A node registered for no group is refused signaling, not exempted from it (§7.2). The membership check was written as "if the node claims any group", so it skipped itself — membership, group status and the public-group gate together — for the node AV1 made commonplace: the unconfigured one, which is also the one least able to absorb the work
AV25 Which nodes host a group is answered to its members (§7.3). Only the public case checked, so a private group told any authenticated account that knew its id which machines hosted it — and an ex-member knows that id for ever
AV26 A sign-in lockout refuses passphrase sign-in and nothing else (§7.7). It is keyed by username, usernames are public, and so anyone can spend somebody else's attempts. Open sessions, renewal and device sign-in are untouched and a reset code ends it, which bounds what a stranger buys to one forced sign-in. The lockout is a DoS primitive by construction; this is the ceiling on it
AV27 A free-text third-party search is bounded per member and per node (§6.5). tmdb_search_req spends the operator's credential, which TMDB rates and the whole group's automatic matching depends on, so one member holding a search box degrades the library for everyone. Per member and not per connection — three tabs is one person — and kept in the group context so a reconnect does not reset it. The refusal is an error, because an empty result list is what "no such film" looks like
AV29 An invitation link is bounded on both halves and its mail on the sender (§3.4, §7.3). Twenty outstanding per group on the node (bearer codes) and on the hub (tickets); and because a link mail reaches an address the hub has no relationship with, at the request of anyone who owns a group, it is counted per sending account per day (mail.invite_link_daily_cap, 10), under the recipient and instance bounds and outside the recovery reserve (invite_link is not a recovery purpose)
AV28 How many node keys one account may announce is bounded (§7.2). Each is a row plus an IP-log row under a one-year retention, so an account in a loop writes a year of storage on the operator's disk having paid only for signatures. Proof of possession (M8) settles whose key it is and not how many. Counted only where a row is added: re-announcing a key already held keeps working at the ceiling, or a node that reached it could never refresh its address again
AV30 What one member's offers cost a node is bounded per account and per node, and the bound admits the heaviest ordinary account (§7.2). Each offer makes the node allocate a peer connection. A budget of 120 per node refilled at two a second bounds a member there without touching their other nodes, and it is counted by account because a mobile carrier shares one IPv4 address among many subscribers. Pending offers are capped at 32 per account. Both refusals carry Retry-After and the client retries them, because a refused offer otherwise reads as a node that is down

13.6 Chat design findings

Label The rule it names
F1 A sender's signing key is bound to the roster, never generated fresh inside a distribution any member can produce. Otherwise every member can replace another's chain and forge them silently
F2 A second device cannot destroy the first's ability to send or be read — there is no per-device chain to drop (§4.5)
F3 There is no skipped-key cache to grow without bound
F4 Rotating the group key is a re-wrap, not the destruction of the archive. Epoch keys are wrapped at delivery, never at rest (§4.5)
F5 History is served to devices that were not present, which is what "load older" means and is incompatible with a ratchet
F6 Ciphertext is not corrupted on the history path
F7 One account can hold two connected devices: the peer registry is keyed per connection, and a broadcast excludes the sending session, not the sending account
R1–R22 The regression register for that work — each row a concrete way the feature could break something that already worked, with the guard that stops it. R1 (rotation keeps history readable), R5 (an older client is refused with a stated reason, not left showing gibberish), R11 (own-ness from the account id), R15 (no epoch key in a plaintext store), R16 (retention deletes messages, never epoch keys) are the ones cited elsewhere

13.7 Windows port

Label The rule it names
W1 Platform directories: no hardcoded XDG paths
W2 Signal handling is platform-guarded
W3 Daemon lifecycle: a per-user startup launcher by default, a scheduled-task service mode offered, switchable after install (§11.2)
W4 Packaging: one per-user installer carrying client and node, with media tools bundled
W5 File permission calls are skipped where they have no meaning
W6 Media-tool discovery fails at startup with a stated reason rather than at first use
W7 CLI messages name the right paths and commands for the platform
W8 The test suite is green on the platform, encoding included
W9 ICE interface matching works where an adapter's name is a GUID rather than a kernel name, and fails open

13.8 Desktop client decisions

Label The decision
E1 Electron, plus an optional Python sidecar for hub-less group:// over QUIC
E2 Device linking: an already-pinned key countersigns, bound by a one-time code the new device generates (§3.3)
E3 Hybrid account creation: register with a passphrase-derived auth_key — the recovery path — then authenticate day to day with a device key
E4 Signed admin ops over MNP for everything group-scoped; first run stays local; the loopback API is never exposed to the network
E5 LAN enrolment of a headless node is out of scope for v1, kept implementable
E6 The browser SPA stays. A native client must not prevent web use
E7 Several named roots per group, forming one virtual root (§6.2)
E8 exFAT/NTFS and Windows are the common case. Linux ships first; that is build order, not population (§10)
E9 Group-related server state lives on the node. Always (§1.3)
O1 Initial key setup in the pre-proof window — deferred; that window is where C4 and C5b came from
O2 A LAN enrolment door — one endpoint, bounded window, one-time code, closing permanently on success
O3 device_policy {allow_bundle: false}, signed by a pinned key — the mechanism that actually closes C4 (§3.7)
O4 Isolating the node-admin panel from the process holding user keys
O5 An unlock key in the environment, for the node
O6 The engine version floor, verified rather than assumed
O8 A minimum client version in the hub version endpoint — done (§5.6)
O10 Canonical file identity across filesystems, defined once and shared (§10)
O11 A root alias where the basename cannot be used (§6.2)
O12 Derived data: revised — the node caches durably in its own data_dir (§6.5)
O13 Hub identity pinning. The client points at a hub by URL and nothing pins that hub's identity. Bounded, because a substituted hub can neither read content nor ship the code to a native client — worth doing all the same
V1–V13, P1–P5 Per-application open items: wording of a disabled-service state, whether artwork reuses the chunk path, cache TTL, multi-track surfacing, HEIC/RAW support, a fuller EXIF panel, lightbox preloading, album-boundary behaviour, cover selection

14. Decisions that are not revisited

14.1 Structural

  1. Multi-group on a single port.
  2. Signaling punch and connect through the hub socket.
  3. Chat is a core feature, not a module.
  4. Chat encryption is a sealed archive with per-device keys and signed messages — not sender keys, not a ratchet (§4.5).
  5. Tokens carry group claims and the node checks them.
  6. Admin roles are hub configuration.
  7. Refresh-token rotation is family-based.
  8. Email is encrypted at rest.
  9. Argon2id parameters are versioned and migrate on login.
  10. Browser transport is WebRTC DataChannel with ICE/STUN.
  11. The hub is a registrar and signaling relay, never in the data path.
  12. Chat is stored on nodes, never on the hub.
  13. The web UI is a small-framework SPA: dark/light, responsive, i18n in ten languages.
  14. Site-specific pages are an overlay, separate from the generic hub.
  15. Video streaming is fragmented-MP4 remux on the node, source buffer in the client.
  16. ICE is primary for browser and native alike. QUIC is kept at parity for LAN, port-forwarded and hub-less access. TCP+TLS and the node HTTP API do not exist.
  17. punch_nat() is a direct-connection helper, not a traversal stack (§5.1).
  18. The desktop shell is Electron. What is unchanged and non-negotiable: UI assets ship inside the package and load from disk (§8.2).
  19. A second device is admitted by device linking, not by an operator code (§3.3).
  20. Private keys never leave the device on native clients. Qualified: a browser has no durable storage of its own and still needs a bundle on each node, so C4 closes for an account only when it opts out of browser use.
  21. Hub minimisation is enforced by an acceptance test, not by policy. The hub must be unable to see keys, content or file listings.
  22. No new code exchanges between people. Safety numbers are refused for identity verification, permanently. The device-linking code is between a person's own devices and is unaffected. The total user-visible cost of the whole authorship story is one notice: "this account's key changed".

14.2 Client architecture

# Decision
D1 The hub keeps serving the web UI. It is the zero-install path and it stays. What must then be true: a strict CSP, a signed digest of the served bundle that any third party can verify, an explicit reduced-trust notice, and docs that never claim end-to-end integrity for that path
D2 A native client, offered alongside the SPA — not as a replacement. The browser-extension options (an extension that verifies the served bundle; an extension that ships the UI) were analysed and are not taken up
D3 Transport: ICE primary, QUIC at parity, TCP and HTTP removed
D4 Hub minimisation deferred, and may be dropped. The hub stays in the trusted path by choice

The reason a native client is justified, and the reason it is not justified. It is justified on product grounds: durable keys, no browser tab, background connectivity, better video, hub-less access. It is not the fix for T3 unless reproducible builds ship with it — a binary from the same operator relocates trust rather than removing it. What genuinely changes is detectability: a browser attack is one HTTP response aimed at one user, leaving no artifact; a native attack requires shipping a build, which is hashable, archivable and comparable.

Its real costs were under-weighted once and are recorded: patch velocity is owned rather than inherited from a browser vendor, and the renderer parses attacker-controlled content from nodes. Accepted deliberately.

For node operators specifically, the CLI beats every client. The operator holds the group key and is the content authority; a CLI removes their dependency on hub-served code at a fraction of any client's cost. If only one thing were built against T3, it should be that — and it was.


15. State of the build

15.1 Built and running

The hub, the node daemon, both transports, the unified handshake, admission and pairing, invitation links, device linking with member-visible evidence, per-node identity, named roots with RO/RW and eject/plug, the indexer with partial hashing, uploads, the sealed index and sealed upload path, encrypted chat with epochs, video streaming with seeking, audio-language and subtitle selection, transfer leases with queueing, pause and resume, the group-application framework with Chat, Files, Videos, Music and Photos, cross-group search with source merging, per-account playlists, casting to a Chromecast with subtitles rebased onto the relay's clock, the operator CLI and loopback control API, the desktop client through its identity and download stages, account recovery, and the Windows port through packaging.

The packages install: a machine has been taken from the built artefacts to a running hub and node on Ubuntu 26.04 (.deb), Fedora 44 (.rpm) and Windows 11 (.exe). What is not done is signing them (Stage D11, D12), which is a different question from whether they install.

Managing a node and setting one up are built, and not where the desktop stages put them. §8.3 is why: one UI source means a panel written for the desktop client is a panel the browser does not have, so both live in static/ and the client ships them. node-page.js is the management surface over the operator ops — status, groups, roots with eject, plug and add, roster, peers, pairing, audit export. create-group-page.js is the first-run path: it detects the node, provisions it, takes the group's name and roots, initialises the key and pairs. The one part that is genuinely native is the service unit, and the desktop main process runs it — systemctl --user on Linux, Task Scheduler on Windows.

15.2 Not built

Stage D9 Python sidecar — group:// over QUIC
Stage D11 Windows code signing — the installer runs on Windows 11; the binary is unsigned
Stage D12 Release key, signed repositories, updates through the OS
— DLNA/UPnP casting (§11.4)
— Bitmap subtitles (PGS, VOBSUB — about a fifth of the embedded streams). No WebVTT without OCR; they are not listed rather than listed and blank. Burn-in covers them and costs -c:v copy, which is what the eight-slot sizing assumes never happens
— Delegation (§3.4)
— Tier 3 roster attestation (§3.3)
— Android client
— Federation between two hubs. The protocol is written and switched off in the code (§7.6); what is not built is one run between two machines

15.3 Open, and why each is where it is

Item Status
C4 for browser-using accounts Open until the signed bundle opt-out ships (O3)
T3 for browser users Accepted permanently. Removed for native clients, and that removal's value depends on reproducible builds
Hub identity pinning (O13) Nothing pins the hub's key. Bounded, because a substituted hub can neither read content nor ship code to a native client
Aggregate upload quota Per-file caps exist and the operator sets theirs (§6.4); a per-user or per-group total does not
A signed upload transcript Ownership is recorded by the node and verifiable by nobody else (§5.4). Making it provable is a transcript the uploader signs, stored with the entry — designed in outline, not built
Forward secrecy in group chat Given up deliberately and on the record (§4.5). If it becomes a requirement it belongs in 1:1 DM
Metadata at the hub Membership, and who posted in which group and when. A known leak, not a solved problem (§7.1)
The exact-hash content check Structural, not functional (§7.5)
QUIC Off by default, and not at parity: it serves the index and file chunks with no transfer lease, no leaseless ceiling and no root-availability check, does its file I/O on the event loop, and returns exception text to the peer (L3). No client speaks it. Either it comes to parity or it goes; until then §5.1's "chat is the only gap" is the one sentence here that overstates the code
The relay registry Closed in the code: relay.RELAYS_ENABLED is False and every /v1/relays route answers 503, as federation does. Nothing in the tree calls them, node or client, and §11.1 measured two ISPs with no TURN relay needed. Kept code that nothing calls is what L7 says not to keep; it stays only as the proof-of-possession design (AV6) until a node needs a relay or it is deleted
A very high bitrate wedges the player against a small buffer ceiling Where even the floor read-ahead does not fit — ninety seconds plus the minute kept behind, at the file's bitrate, above what the engine will hold — the film stalls: measured on the harness at 9.3 Mbit/s against a 100 MB ceiling, 100.8 s of film played in 900 s of wall clock. Predates the byte budget and is unchanged by it, to the tenth of a second; what the budget did change there is the refusal count, 1560 → 2. The fix is not a bound at all, it is a second stage of buffer outside the SourceBuffer, which means gating the append path — the riskiest change in this area and not one to make alongside another
The reconnect backoff only wakes on visibilitychange So a tab that stays visible through an outage — which is what a screen wake lock guarantees while a film is playing — waits out the full backoff, up to 30 s, after the network is already back. Nothing listens for online
Per-device revocation has no CLI A device is revoked over MNP (roster.revoke_device), from a device the node has already pinned. On a headless node the operator's only lever is member unpin, which removes every device of that account — so the per-device control the roster is built around is reachable from an interface and from nowhere else. §6.7 listed a meshbay-node member device list\|revoke verb that was never written, and that listing is how this was found: USERGUIDE.md was the first document written by reading the CLI rather than this specification, and the verb it copied out did not run
Migrations run on SQLite only The chain reaches head and agrees with the models there (§12), which is not where it ships. The exposure is one revision deep, not the whole chain: every revision behind the first packaged release was development that no installation ever ran, so nothing replays them on PostgreSQL. What is unguarded is the next migration — a default, an index type or a constraint PostgreSQL refuses reaches a deploy without the suite saying so
The loopback path removes access without writing an audit entry ops.revoke_member and ops.unpin_member log to the daemon's log and nothing to audit.db; the MNP admin handlers doing the same work audit member_revoke and member_unpin. So a removal made from the node page or the CLI — the two doors an operator sitting at their own machine actually uses — leaves the journal showing an admission and then, whenever that person next connects, a join_refused with nothing in between to explain it. §5.4's signed transcript is not what is missing: a loopback caller is authorized by being on localhost with the run token and signs nothing, so the gap is the record, not the authority. Found by reading a node's audit log for a refusal whose cause was six hours earlier and unrecorded
The Create group wizard calls two hooks after an early return CreateGroupWizard (create-group-page.js) returns during node detection, before its useRef/useEffect for provisioning, so the hook count changes between renders. Preact tolerates a list that grows, and nothing is known to break; test_hook_ordering.py checks declaration order, not this. Found while tracing the frozen-fields report, which had another cause (ask.js)
A node key is read from the terminal or the desktop client, never a browser Accepted. meshbay-node status on the node's own machine and Node → Overview in the desktop client are the two places the key can be read; the Node page is Electron-only, because platform.node resolves to "not available" without the bridge, and no hub route exposes the key. The create-group wizard links it automatically over that same bridge, so the manual paste in Profile → Link Node exists for the operator who runs the node from a terminal and the hub from a browser — who has a terminal by definition. Anyone linking a node is already at a shell prompt, so a browser-reachable copy would buy nothing and widen what the hub knows about the node
Listing a group's folders walks every root on the event loop index_sync_message (transport/wire.py) builds its dirs field with list_dirs, an rglob("*") over every root, and nothing sends it off the loop: the WebRTC index_sync handler, the daemon's index push and QUIC all call it inline. So each index request from any member is a directory walk of the whole library that every other peer on the node waits behind. test_disk_io_off_loop.py never saw it, because it reads the transport's own modules and the walk is one call away in wire.py. Found by widening what that test reads, not by a symptom
The transcoded-seek test passes without transcoding test_a_transcoded_video_keeps_accurate_seeking (test_stream_seek_audio_alignment.py) forces the re-encode branch by swapping the module's BROWSER_INCOMPATIBLE_VIDEO_CODECS, then checks only that the result has no audio gap. The copy path also leaves no gap on that clip, so pointing the swap at a module the streaming code does not read still passes: the test cannot tell that the branch it is named after never ran. It should assert the re-encode happened (the re-encoding log line, or the encoder in the ffmpeg argv). Found by breaking the swap on purpose while moving the streaming code

16. Concordance

Older documents, notes outside this repository and every earlier revision in git cite sections of the documents this one replaces. Those documents are no longer in the tree — they were removed on 2026-09-11, once their content was here and this table could resolve every reference they left behind. git log -- docs/ recovers any of them.

The code does not need this table: since 2026-09-19 every comment and test cites MESHBAY_DESIGN.md and a section directly, and a new one must. What this resolves is what cannot be rewritten — a commit message, a git blame, a superseded document kept under docs/, a note somebody holds elsewhere.

Cited as Read
draft-v5 §2, draft-v6 §4 — security claims §2.2
draft-v5 §3 — transport, NAT traversal §5.1
draft-v5 §4, §4.1–4.4 — handshake, transcript, channel binding, mutual auth §5.2
draft-v5 §5.1, draft-v6 §2.3, §2.4b — privileged operations, key activation, authorship §5.4
draft-v5 §5.2, draft-v6 §2.1, §2.1b — uploads §6.4
draft-v5 §5.2 — nothing derived beside the originals §6.5
draft-v5 §5.2b — removing a directory §6.4
draft-v5 §5.3, §5.4 — operator interface, local admin UI §6.7
draft-v5 §5.5, invite-pairing-v1.md §3, §5, §7 — admission and key delivery §3.4
draft-v5 §6.1, draft-v6 §2.5, §2.9 — hub role, group names §7.1, §7.3
draft-v5 §6.2–§6.4 — node registration, signaling, client addresses §7.2
draft-v5 §7 — cryptography §4.1–§4.4, §4.6
draft-v5 §7.1 — the keypair bundle and C4 §3.7
draft-v5 §8.1, §8.2, desktop-client-v1.md §2 — the two clients, the shell §8.1, §8.2
draft-v5 §9, draft-v6 §5 — open items §15.3
draft-v5 §10 — testing posture §12
draft-v6 §2.2, desktop-client-v1.md §4, §4.5, §4.6 — device linking §3.3
desktop-client-v1.md §4.8 — authorship, the tiers §3.3, §5.4
desktop-client-v1.md §5, §5.1, auth-confirm.md — accounts, change and recovery §3.1, §3.6
desktop-client-v1.md §6.7, refactor-groups.md §1.1, §1.5b — roots, RO/RW, eject §6.2
desktop-client-v1.md §6.8, draft-v6 §3, WINDOWS-PORT.md — portability §10
desktop-client-v1.md §6.9 — a root that goes away §6.2
desktop-client-v1.md §6.10, mediacenter.md §2, §5.2, §5.3 — views not a catalogue, derived data §6.5, §9.1
desktop-client-v1.md §8.1 — URL space §8.4
desktop-client-v1.md §10b, refactor-node-ui.md — group settings, the node's admin surface §9.3, §6.7
draft-v6 §2.7, apps.md §1–§3 — the application framework §9.1–§9.3
apps.md §2b, refactoring-search.md — cross-group search, source merging §9.11
playlists.md — per-account cross-group state §9.10
apps.md §4, refactor-groups.md §3, §4.1 — adding an application §9.4
draft-v6 §2.8 — instance policy, suspend vs revoke §7.4, §7.5
draft-v6 §2.10, refactor-groups.md §1.7 — link previews §6.5, §9.6
draft-v6 §2.11 — node settings §6.8
draft-v6 §2.12 — STUN and ICE filtering §5.1
mediacenter.md §3, §4, §5.4–§5.7 — Videos §9.7
musicbay.md §2, §3, §4, §5, §6 — Music §9.8
photos.md §2, §3, §4, §5 — Photos §9.9
refactor-groups.md §1.6 — generic app-directory ops §9.3
chat-sender-keys.md §5, §6, §8, §13 — chat encryption, sender authentication §4.5, §3.3
indexing-v2.md — partial-read hashing §6.3
per-node-identity-v1.md — identity keys per node §3.2
captcha.md — registration gate §7.7
tmp-decisions.md D1–D4 §14.2
first-review.md, second-review.md, third-review.md — any finding §13, which defines every label. §13.1 is the first review, §13.3 the second, §13.4 the third
USERGUIDE.md — anything it was wrong and is gone. §2 for the trust model, §3 for identity, §6.7 for the operator surface
QUICKSTART.md — anything it was stale and is gone. PACKAGING-GUIDE.md installs; §6.7 lists the operator commands
devel-phases-next.md structural decisions §14.1
cast-smart-tv.md §11.4

Older references to draft-v3 and draft-v4 sections point into old-draft.md, which is a historical archive and is not authoritative.