diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/MESHBAY_DESIGN.md | 456 | ||||
| -rw-r--r-- | docs/MESHBAY_HTTP_API.md | 233 | ||||
| -rw-r--r-- | docs/MESHBAY_NODE_PROTOCOL.md | 97 | ||||
| -rw-r--r-- | docs/QUICKSTART.md | 2 | ||||
| -rw-r--r-- | docs/USERGUIDE.md | 63 | ||||
| -rw-r--r-- | docs/generate_http_api.py | 164 | ||||
| -rw-r--r-- | docs/transfers-v1.md | 5 |
7 files changed, 861 insertions, 159 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 2419916..fdf08f9 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -16,8 +16,8 @@ > them — it names the invariant that holds today, not the incident that produced > it. §13 is the register of those labels. > -> Wire versions at the time of writing: **MNP 5.0** (oldest peer accepted 4.0), -> **MHP 0.1**, packages **0.17.0**. The normative source for the wire format is +> Wire versions at the time of writing: **MNP 6.0** (oldest peer accepted 4.0), +> **MHP 0.1**, packages **0.18.0**. The normative source for the wire format is > `MESHBAY_NODE_PROTOCOL.md`; this document states the design the protocol > serves, not its byte layout. @@ -33,6 +33,7 @@ | `QUICKSTART.md` | one machine to a working group, for somebody who has installed nothing | | `USERGUIDE.md` | using a group and running a node, for the person who does either | | `MESHBAY_NODE_PROTOCOL.md` | the MNP wire format, message by message | +| `MESHBAY_HTTP_API.md` | every route of the hub and of the node's control API, generated from the code | | `transfers-v1.md` | the transfer system's failure-mode analysis, kept because a synthesis cannot carry "every way a slot can be lost" | | `playlists.md` | the playlist design and its interface in full, with what building it corrected (§9.10) | | `cast-smart-tv.md` | the DLNA/UPnP device backend — designed, not built (§11.4) | @@ -99,7 +100,7 @@ opens them. │ └─────────┘ MHP 0.1 │ signalling (SDP/ICE, <1 KB), presence, revocation push MHP │ - ┌────┴────┐ MNP 5.0 ┌──────────┐ + ┌────┴────┐ MNP 6.0 ┌──────────┐ │ node │◄──────── WebRTC DataChannel / QUIC ──────────►│ client │ └─────────┘ index, file chunks, streams, chat, admin └──────────┘ holds the files browser SPA or desktop @@ -134,76 +135,67 @@ paths, no filenames**. --- -## 2. Trust model +## 2. Who you trust -### 2.1 Adversaries +MeshBay lets you share things with the people you choose, and keeps everyone else +out. In plain terms, here is who can see your group — and the one caveat worth +knowing. -Every claim below is written against one of these, and they are the only ones the -document uses: +### 2.1 Inside your group -| Adversary | What they can do | -|---|---| -| **Passive hub** | Read everything the hub legitimately stores and relays | -| **Active hub** | Also lie: forge tokens, invent accounts, substitute values it publishes, ship modified client code to a browser | -| **Malicious node operator** | Read and alter everything on their own machine, including the plaintext files they host | -| **Malicious group member** | Everything a member may do, plus anything the protocol fails to refuse | -| **Network attacker** | Observe and tamper with traffic between any two parties | -| **Local attacker** | Reach loopback services and files on a client or node machine | -| **Registered hub user with no membership** | Reach every hub endpoint that does not check membership | -| **Federated peer hub** | Push directory rows and revocations over MHP | +A group lives on a node, run by its operator. Everyone in the group — the operator +and the members — shares its files, its index and its chat. That is what a group +*is*: the people you decided to let in, the same as a shared folder or a team +workspace. They are the people you trust, and they are the only ones who can read +what you put there. -### 2.2 Security claims +Two things follow from that by design — they are the shape of a shared space, not +gaps in it: -| Claim | Passive hub | Active hub | Malicious node operator | Malicious member | Network attacker | -|---|---|---|---|---|---| -| Data never transits the hub | ✅ | ✅ | — | — | ✅ | -| File content is unreadable | ✅ | ❌ **T3** (browser) · ✅ native | ❌ by design — the operator hosts the files | ❌ members share the group key | ✅ | -| The file index is unreadable | ✅ | ❌ T3 · ✅ native | ❌ | ❌ | ✅ | -| Chat content is unreadable | ✅ | ❌ T3 · ✅ native | ❌ — the operator is a member | ❌ | ✅ | -| Chat is unreadable **from a copy of the node's storage that lacks its unlock key** — not from a whole disk by default (§4.5) | ✅ | ✅ | — the operator holds the unlock key | ✅ | ✅ | -| Content cannot be modified | ✅ | ✅ | ❌ by design | ✅ | ✅ | -| The node cannot be impersonated | ✅ | ✅ | — | ✅ | ✅ | -| Client code integrity | ❌ **T3, accepted** (browser) · ✅ ships in the package (native) | ❌ T3 · ⚠️ native: **detectable, not prevented** | ✅ | ✅ | ✅ | -| The hub cannot obtain the group key | ✅ | ✅ **except** in an open-join group, where it can join legitimately (§7.3), and for an invitation the inviter asked the hub to mail, whose code it then holds (§3.4) | — | — | ✅ | -| Node content authority | ✅ | ✅ | ✅ sovereign | ✅ | ✅ | -| Devices cannot be added by the hub | ✅ | ✅ — the hub holds no user key and cannot countersign | ⚠️ a node adds a device only to itself, where it already reads everything | ✅ | ✅ | -| Chat senders are authenticated to each other | ✅ | ✅ | ⚠️ only for accounts the reader has already seen (§3.3) | ✅ | ✅ | -| Keypair bundles (**C4**) | ✅ it holds the pepper and no bundle | ⚠️ it can fetch a bundle with a token it mints and holds the pepper: an offline passphrase search, as T3 already concedes for browsers · none to fetch for an account without browser access | ✅ no offline search: the bundle does not open without the hub's pepper — only sign-in attempts, bounded and audited · none on disk for an account without browser access | — | — | -| Deleting your account erases you | ✅ hub-side | ✅ hub-side | ❌ files, pinned identity and bundle stay on the node (§7.7) | — | — | -| Your identity keys stay yours | ✅ | ⚠️ as the row above | ✅ the bundle they hold does not open without the pepper, and a leaked bundle key opens **that node's** bundle only | ✅ | ✅ | +- Your operator hosts your files, so they can change what they host. A node's + content is the operator's to keep and to serve (§7). +- Leaving a group removes you on the hub, but copies already on a node stay with + its operator (§7.7) — as with anything you have handed to someone in person. -### 2.3 What the project must not claim +### 2.2 Everyone else sees nothing -Three sentences are forbidden, each for a deliberate reason: +Outside your group nobody can read what you share — not the hub that connects you, +not anyone watching the network, not the operator of any *other* group: -- **"Everything is encrypted and unreadable by other parties, even the hub."** - A hub that ships the code can lift keys from the page regardless of protocol - design (**T3**). That is an artifact-level attack, not a silent directory lie, - and it is removed for native clients — not for browsers. -- **"A native client makes the hub untrusted."** It converts an undetectable, - per-request, per-user attack into a persistent artifact that can be hashed and - compared. That value is realised by reproducible builds and published hashes, - not by the packaging format. A build signed with a key the hub operator holds - *relocates* trust; it does not remove it. -- **"C4 is closed."** It is closed for an account whose identities the desktop - application keeps (browser access off, §3.7): nothing of them is on any node. - For an account with browser access the bundle is still on each node, sealed - under the passphrase **and** a pepper the hub holds — no operator can search it - offline, but an active hub can, and that is T3's adversary already. **An account - is only as strong as its weakest client.** +- Your content never passes through the hub in the clear. The hub introduces nodes + to one another and relays sealed traffic; it does not hold what you share (§5). +- The hub holds no key of yours. It cannot read your group, add a device to your + account, or act as you — and it cannot hand your group's key to anyone, save + where you ask it to (an open-join group, or an invitation you asked it to mail; + §7.3, §3.4). +- Each node carries its own identity keys, so a key that somehow leaked would open + that one node and no other (§3), and a copy of a node's storage without its + unlock key reveals none of its chat (§4.5). + +### 2.3 Your operator hosts you, but cannot become you + +Keeping your content is the deal you made with your operator. Turning that into +*being* you is the line they cannot cross. The identity bundle stored on their +node opens only with your passphrase and a secret the hub releases to no one but a +session that has already proved the passphrase or a device key; the only attempts +left to an operator are ordinary sign-ins, which the hub counts and locks out. And +reading what *they* host never reaches what *other* operators host (§3.2). -"End-to-end" here describes **client ↔ node**, never client ↔ client. Members and -the operator read everything in their group; that is what a group is. +### 2.4 The one honest caveat: the browser -### 2.4 One boundary worth naming +Used in a web browser, MeshBay is a page the hub sends you on each visit — so a hub +that had been taken over could send an altered page and lift your keys from it. +This is true of **every** web application; most simply never say so, and no +protocol can prevent it, because the attack is in the code itself rather than in +the messages. So we never claim your content is unreadable *even by the hub* when +you use a browser. -An operator hosts your content by design. They should not be able to become -*you*. The bundle on their disk does not let them try: it opens only with the -passphrase and a pepper the hub hands to nobody but a session that proved the -passphrase or a device key, so the only guesses left to them are sign-ins, which -the hub counts and locks out. And were a bundle key to leak all the same, it -opens **the bundle on that node** and no other. Reading what they host is by -design; reading what *other* operators host is not, and does not follow (§3.2). +The desktop and mobile apps close this. Their code is installed once, from a +package anyone can rebuild from source and check against a published hash, so a +tampered build is caught instead of silently trusted. For the strongest assurance, +use the app. This is the project's single standing limit for browser use — tracked +as **T3** — and the full, adversary-by-adversary analysis behind every statement +in this section is in §13.9 for readers who want it. --- @@ -401,7 +393,11 @@ Four properties, each load-bearing: attempt is an audit event. 3. **The node's roster is the authority**, not hub membership. A hub that invents an account, adds it to a group and mints it a token gets - `not_authorized_for_group`. + `not_authorized_for_group`. The Members list says the same: it shows the + accounts the node has admitted (the sealed group roster, §11.7 of the protocol). + One the hub counts as a member but that has not presented its code yet is + shown to the owner alone, as waiting for its code; when the roster cannot be + read, the hub's list is shown. 4. **Wrapping happens on every connection.** Nothing is stored per member, so key rotation propagates by itself and revocation actually takes effect. (Rotating the key after a revocation is still required — the ex-member holds the current @@ -786,10 +782,10 @@ Three properties are why this shape: The node produces every copy of the key itself, from its own CSPRNG. **Nothing arriving over MNP can activate a group key** (**C5b**). Read that precisely: it -targets *key material arriving from outside*, not the instruction. An -operator-signed `gek_rotate` where the node generates the key is a different shape -and is allowed. The initial `gek-init` stays local, because with no key there is -no completed session to carry a signed op. +targets *key material arriving from outside*, not the instruction: a key the node +generates itself on an operator's instruction is a different shape. Initialising +and rotating the group key are local (loopback API, CLI); the signed `chat_epoch` +is the MNP instance of that shape. ### 4.3 On-the-fly encryption @@ -898,7 +894,7 @@ signature refuses it. **Epochs.** A new epoch is opened when, and only when, the set of devices that may read *future* messages shrinks: `member revoke`, `member unpin`, `revoke_device`, -`gek_rotate`, or an explicit `chat rotate`. Epoch 1 is opened at group load — a +or an explicit `chat rotate`. Epoch 1 is opened at group load — a group with no epoch is a group nobody can speak in. **Old epochs are kept and still delivered.** That is what keeps history readable @@ -1233,9 +1229,9 @@ from anything in the response. | `dir_delete` | the operator alone, and only on an empty directory | | `invite_create` | the operator (or a delegate, when delegation ships) | | `invite_link_create`, `invite_cancel` | the operator | -| `gek_rotate` | operator-signed; the node generates the key itself | -| initial `gek-init` | **local admin API or CLI only** | -| root add/remove/update/eject/plug, `apps_enabled`, app directories, transfer limits | operator-signed | +| group key init and rotation | **local admin API or CLI only**; the node generates the key itself | +| root add, root `writable`/`removable`, hosting a group, transfer limits | **local admin API or CLI only** (MNP 6.0) | +| root remove/eject/plug, `apps_enabled`, app directories | operator-signed | | ~~`gek_bundle_store`~~ | **the message does not exist.** No member ever hands the node key material | `gek_bundle_store` was deleted rather than gated. The operator's X25519 public key @@ -1451,8 +1447,8 @@ checks the version its peer declared and **branches on none of it**. **The floor is not necessarily the current version, and what is added above it is why.** It is `MNP_MIN_SUPPORTED` in `handshake.py`, and it is the last MAJOR that had to refuse at the handshake: 3.1–3.4 were added above the 3.0 floor without moving it, -and 5.0, a MAJOR confined to four signed operations that a peer across the break -refuses to sign, sits above the 4.0 floor. So a +and 5.0 and 6.0 — MAJORs confined to a few signed operations, four whose subjects +changed and then three removed — sit above the 4.0 floor. So a peer can be reachable and still not do something the current version can, and the client has to cope with that — **by reading the peer's own answer, never by comparing version numbers**. @@ -1541,7 +1537,16 @@ Five consequences, none optional: - `writable = true` means any group member may upload there. Several roots may be writable and none need be — a fully read-only group is valid. -The operator toggles this with a signed op. + +**What widens the sharing is decided on the node's own machine.** Adding a root, +hosting a group over a directory, and switching `writable` or `removable` go through +the loopback API (the desktop application) or the CLI — never over MNP, since 6.0. +A signed op proves that the operator's key signed, not that they meant it: in a +browser that key is driven by code the hub serves (T3), and in the desktop +application by a renderer that parses content from nodes. Either could otherwise +have shared any folder on the machine, writable, from anywhere. The operator still +sees every root and its flags from any browser; removing, ejecting and plugging stay +signed ops, because they narrow what is shared or restore what already was. > **There is one answer to "may this member write", and it is the root.** A single > flag over the group cannot express "this library is published read-only and that @@ -1564,7 +1569,8 @@ the whole tree or presents an empty directory to the next scan. Both propagate a though the owner erased their library. So a root has two independent runtime states: -- **`ejected`** — operator-controlled, persisted in `roster.db`. +- **`ejected`** — set by the operator, or by the safety net below; persisted in + `roster.db`, with which of the two set it. - **`available`** — computed as `not ejected and is_live()`. This is what clients and the indexer see. @@ -1585,12 +1591,21 @@ let the following scan read the empty mount point as an erased library. It lives hand-written config must not be rewritten because a USB drive was unplugged. **Auto-eject is the safety net.** If a `removable` root's path disappears, the -availability sweep sets `ejected` as though the operator had clicked it, and -reports it so the daemon persists it. Nothing is deleted: index entries, cached +availability sweep sets `ejected` and reports it so the daemon persists it, marked +as the safety net's. Nothing is deleted: index entries, cached metadata, thumbnails, chat history referencing those files and app directory configurations all survive, the last flagged as temporarily invalid rather than wrong. +**The safety net's eject undoes itself; the operator's never does.** At startup +and at every reconcile, an auto-ejected root whose path is readable again is +checked against what the hash cache knows was under it: a few of those files, +at the same path with the same size and mtime. One found, and the root is +plugged back and rescanned, like a plug. None found, and it stays ejected: an +empty mount point or another drive mounted in its place is exactly what the eject +protects the index from. The case this serves is ordinary: a node started with +the session, before the desktop has mounted its USB drives. + ### 6.3 Indexing The index is **content-addressed**: `GroupIndex` is keyed by blake3, so the same @@ -1888,28 +1903,27 @@ is not authentication: any local process can reach it, as can a page in the operator's browser via DNS rebinding — and this API re-initialises group keys, issues invitations and reads the audit log. There is no server-rendered dashboard; the desktop client's Node page and the CLI are the two consumers, and each -operation endpoint is one `_op(...)` line onto `ops` (§5.4). +operation endpoint is one `_op(...)` line onto `ops` (§5.4). Its routes are listed +in `MESHBAY_HTTP_API.md`. -**Over MNP, the node's own controls need a proved operator device.** The -node-wide surface — `node_status`, which lists every group on the machine with -each root's absolute path, plus `node_settings_set`, `roster_read`, -`denylist_read`, `denylist_clear` and `node_reload` — is reachable when two -things hold: the account is the one the node belongs to, *and* the device on the -connection has proved (`device_hello`, §3.3) a key the roster holds as an -operator. The first alone is a claim in a token the hub issued, and **NS4** does -not allow it to be authority: a hub that can name the operator is a hub that can -be one. The second is what it cannot forge, since it holds no user keys and -cannot countersign a device — the same property device linking rests on. A -browser that has never been paired therefore reads nothing here, exactly as it -can already sign nothing (§5.4). +**The node's own controls are not on MNP.** Its status — which lists every group +on the machine with each root's absolute path — settings, roster, denylist and +reload were MNP messages gated on a proved operator device (`device_hello`, §3.3), +because the account id in a token is the hub's to choose (**NS4**). No client ever +sent them, and MNP 6.0 removed them with the four signed ops in the same position +(`gek_rotate`, `member_unpin`, `transfer_limits`, `group_detach`): the desktop +client's Node page and the CLI do this work over loopback. A door nobody calls is +an untested way in, and one that does not exist needs no gate. **The accepted cost, recorded as a choice:** on a headless server the only admin path is the CLI. The CLI covers every operation, so this is acceptable — but it is a real capability reduction, not an oversight. -> **MNP is the path that must exist; loopback is the fallback.** The operator of a -> node is not necessarily sitting at it. Any operator-facing control needs its MNP -> route first, or it renders for nobody on the web. +> **What the operator must see needs an MNP route; what widens the node does not +> get one.** The operator of a node is not necessarily sitting at it, so a view that +> only loopback can fill renders for nobody on the web — which is why the roots +> table rides in the index. But sharing a folder, opening it to writes and the +> node's own controls are decided at the node (§6.2, MNP 6.0). ### 6.8 Node settings @@ -1952,12 +1966,14 @@ an operator-signed op. ## 7. The hub +Its routes are listed in `MESHBAY_HTTP_API.md`. + ### 7.1 Role — chosen, not minimal Hub minimisation was considered and **deferred, and may be dropped** (decision D4). The hub keeps serving the web UI and remains in the trusted path by choice. That is a legitimate product call; what follows from it is carried deliberately rather than -by accident (§2.3). +by accident (§2.4). **Stores:** accounts (username, encrypted email, status, role), the group registry and membership, IP logs (one year, legal retention), node registrations, refresh @@ -2072,10 +2088,17 @@ A group's **identity is its UUID**, everywhere: the route, the node's configurat membership. A group **name is unique per owner account**, case-insensitively and trimmed, enforced by a functional unique index; two different owners may each have a `photos`. Names are displayed as `name@owner`, which is a label plus a create-time -check and **not an addressing scheme**. The handle is hub-local: the same +check and **not an identity**. The handle is hub-local: the same `name@owner` on two federated hubs are different groups, and a federated row shows its source hub rather than an account. +The client also accepts the handle in the address, as an alias for the UUID +(§8.4): `#/name@owner`, optionally followed by a path inside the group. It is +resolved **in the client, against the account's own `/v1/groups/mine`**, and no +hub route answers "which group is called this" — so a handle tells nobody +anything they could not already see, and cannot be used to probe for a group. +A rename breaks the handle links to a group and none of its `#/group/<id>` ones. + `visibility` and `join_policy` are the two independent axes described in §3.5. `join_policy` is read from the node's own configuration, never from the hub. @@ -2529,6 +2552,26 @@ that decides where the hub is or fetches the API relative to the page origin. That is a testable invariant, and it is what any feature adding third-party egress must preserve — which is one of the reasons enrichment is node-side (§6.5). +**Inside the application, every route is a fragment** (`#/…`). What follows `#` +is never sent to a server, so it is in no hub or proxy log and no `Referer`; +that is what lets an invitation carry its code (§3.4), and it is why the same +router runs unchanged on `app://meshbay` and in the Android WebView, where no +server could answer a path. Two forms name a group: + +| Route | Meaning | +|---|---| +| `#/group/<uuid>` | the group — every link the application draws | +| `#/name@owner` | the same group by its handle (§7.3); the address shows this form while a group is open, written with `replace` so it is not a history entry | +| `#/name@owner/<root>/<dir>/<file>` | a file: Files opens on its folder and the file is downloaded. A folder instead of a file opens Files there. The path is taken out of the address once acted on, so a reload does not download twice | + +The owner is after the **last** `@` (a username cannot contain one); each path +segment is percent-decoded on its own. Opened signed out, the sign-in form +stands in for the page and the address is left alone, so signing in lands on +it. A download started this way has no user gesture behind it, so where a browser +offers a Save As dialog it takes the fallback a dialog refused for want of a +gesture already takes (`file-utils.js` `_openDownloadTarget`): streamed to the +download folder. `static/group-link.js`. + ### 8.5 Downloads and streaming **Downloads go to disk, never through RAM, on every platform.** There are three @@ -2774,6 +2817,7 @@ destructures what it needs — a new application does not get a bespoke prop lis | `transportRef`, `gekRef` | **refs**, never state, so a reconnect does not re-render every application | | `deviceReady` | **the exception, and why it is a prop.** A ref not re-rendering is right for a transport reached into on demand and wrong for a *fact about the connection* an application renders from | | `mayUpload` | computed once; a second derivation would eventually disagree with the first | +| `linkFor(entry \| folderPath)` | the `#/name@owner/path` link "Copy link" puts on the clipboard (§8.4), or null where none can be named. The group page builds it from the hub's row; Search from each result's own group and unprefixed path. An application offers the action only when this returns a link, and copies with `copy-link.js` `copyLink` | An application that needs local state owns it. One pattern is worth carrying: **any notion of "current location within the group" resets on group change**, because a @@ -2882,6 +2926,16 @@ There is no folder-browsing protocol and this does not add one. application with no toolbar renders none — an empty band still holds a strip of the page open. +9. **Licence.** An application may be under any licence, provided it reaches the + interface only through the *application interface*: the props of §9.2, the + registry fields above, the exports of `i18n.js`, `icon.js`, `file-utils.js`, + `settings-ui.js` and `folder-tree.js`, `style.css`'s classes and the + catalogues' keys (`static/licenses/APPLICATION-EXCEPTION.txt`, an AGPL §7 + permission). Importing any other module of the interface makes the + application a work based on it, under the AGPL. Its registry line and its + catalogue entries are changes to the interface and stay AGPL. Starting from + `helloworld-app.js`, which is 0BSD, brings no AGPL code along. + No protocol change, no hub change, no daemon change. Steps 4 and 7 are the only node-side and test-side touches, and both are allow-lists. @@ -3345,6 +3399,45 @@ A client, not a host. The platform is hostile to *hosting* a node — background execution, storage, battery — and fine as a *client*, which is one of the reasons enrichment is node-side (§6.5). +**The application is the desktop client's design in a system WebView** +(`packages/meshbay-android/`). What §8.2 depends on is not Electron but an +engine with `RTCPeerConnection`, WebCrypto X25519/Ed25519 and MSE, the +interface loaded from the package, and a privileged side reached through a +narrow bridge — and Android has all three: + +- **The interface is the hub's `static/`, copied at build time** (§8.3) and + served from the APK by an asset loader under + `https://appassets.androidplatform.net` — a secure context, so `crypto.subtle` + exists. Nothing the hub serves is ever loaded into the WebView; the policy is + the desktop's, sent as a header. +- **The bridge offers the desktop preload's `window.meshbay`** wherever it + offers anything. What a phone does not have — the node, shared folders, the + tray — is **absent, not a function that refuses**: `platform.js` decides what + to show from whether an object exists. The bridge answers the packaged + origin's top-level document only (`addWebMessageListener`, `isMainFrame`); a + same-origin child frame does get the port, and is refused there. +- **Keys are held natively** (§3.7, §14.1 #20): device key, bundle key and every + node identity, under an Android Keystore key, never handed to the page. The + Kotlin keyring is a third implementation of the bundle format and the + transcripts, so **`tests/vectors/keyring.json` — generated from the desktop + keyring and checked against the specification — is what every implementation + must reproduce**; a list the vectors cannot hold (the admin operations that + may be signed) is compared by source. +- **Hub calls leave from native code**, to the signed-in hub only, as on the + desktop. **Downloads go to disk** through the Storage Access Framework or the + Downloads collection, as pending files until complete; uploads come through + the system picker. +- **A hidden page is frozen by Chromium sixty seconds after it is hidden** — + measured, and whatever the process's importance: a foreground service, wake + locks and the renderer's priority policy do not prevent it. A cast's pipeline + lives in the page (WebRTC → decrypt → relay), so while one runs the shell + keeps the WebView reported visible and holds a media-playback foreground + service; nowhere else, because a page never hidden is never throttled. + +Release builds are signed with the development key until the release key exists +(Stage D12); §2.3's sentence about who holds a signing key applies to whichever +store distributes them. + ### 11.4 Casting An HTTP relay in the desktop client serves a standard fragmented-MP4 stream that @@ -3352,6 +3445,91 @@ any LAN renderer can play; the relay is device-agnostic. Chromecast discovery an control ship. DLNA/UPnP is designed and not built: it is a second device backend beside the first, not a second relay. +**Discovery lists receivers as they answer.** The mDNS scan runs six seconds, +because a receiver coming back from a reset can take that long, but most answer +within two. The main process holds what the scan has found and the page polls it +(`cast:scan`, then `cast:devices`) for as long as the picker is open, rather than +waiting on one call for the whole scan; a poll uses the same checked `handle()` +door as every other call, where a pushed event would be a second one. + +**The Android relay is the desktop's, with three things the phone found.** +(1) **The header is everything before the first moof**, however many chunks it +arrives in: the node's first chunk can be the 28-byte ftyp alone, and served as +the header it left the receiver without a moov. (2) **What a receiver has not +read waits in a spool file, not in memory.** A fragment is a segment — 5 to +10 MB at a film's bitrate — and the page runs ahead of the television by its +whole read-ahead; dropped past the desktop's 8 MB bound, each lost fragment +froze the picture for its length. (3) **A seek's first segments are held while +it lands** (`video-player.js`): they are the new stream, its header first, and +they arrived while `reinitAt` waited on the SourceBuffer and were dropped as the +old film's — the local player never noticed, a relay restarted there did. The +local element keeps playing while a cast runs, because its playhead paces the +stream, and is muted. + +**While a receiver plays the film, the player is its remote.** Phone and +television start apart and drift, so a scrubber on the local playhead lied about +where the film was and a seek from it landed off by the gap. The page polls +`cast:status`, whose `chromecast` carries the receiver's `playerState` and +`position` (stream seconds, zero at the relay's start; the page adds the +stream's start), and shows that over the local picture with play/pause +(`cast:chromecast:pause`/`play`), ±30 s and a scrubber. A seek is the ordinary +seek: it restarts the relay and reloads the receiver there, and until that +lands the target is shown, not the old stream's position. Pausing the receiver +pauses the local element too, which otherwise goes on fetching for nobody. The +copy-URL cast has no receiver to read and keeps the ordinary player. + +**A television is chosen for the session, not for one film** +(`cast-session.js`). Any cast button — Videos', Photos' and Music's toolbars, a +photo album's bar, the lightbox, the video player, the music bar — sets it, and +it is held in memory only. A film opened +while one is set starts its cast as a restart does: pending until the first +segment, then the relay and `connect` (or `reload`, when the receiver is already +connected), with the player drawn as the remote from the start. *Stop casting* +clears it, from the player's remote or from any button. + +**A photo or a music track is served whole at `/file`, one at a time** +(`cast:file:begin`, `:write`, `:end`), its cover at `/cover`. It crosses in +pieces — binary frames of a megabyte on Android — because a lossless track is +too large for one bridge message, and on Android it is written to the relay's +spool directory rather than held. The relay starts for it if nothing else has, +serves it behind the stream's token with a versioned address (a receiver handed +the same URL twice shows what it already has), honours byte ranges (a receiver +seeks in a track that way), and accepts only pictures and the audio types the +music player plays. **The shell, not the page, says what the receiver loads:** +the relay's file is loaded as what the relay was told it is — a photo as a +picture (`streamType: NONE`), a track as music (`BUFFERED`, with title, artist, +album and the relay's cover URL), at `startAt` seconds for a track picked up +mid-song — any other relay URL as the stream, and on Android no URL but the +relay's own is accepted at all. A file does not start the foreground service; a +photo's load answers as soon as the receiver accepts it, since a photo never +reaches PLAYING. + +**A photo is scaled before it leaves.** The page fits it to 1920×1080, applies +its EXIF orientation and re-encodes it as JPEG: a camera original is too large +for a receiver to decode in good time, and some formats it does not decode at +all. Only the newest photo asked for is sent; paging quickly shows where the +reader stopped. + +**With a television chosen, the music bar is its remote.** Each track, once +decrypted, goes to the relay with its cover instead of to the `<audio>` +element, which keeps the source for its duration only. Play, pause, seek +(`cast:chromecast:seek`) and previous drive the receiver; the bar's clock is the +receiver's, polled once a second; and the receiver's IDLE/FINISHED moves the +queue on as `ended` does, once per track, because the receiver keeps reporting +it until it is given something else. The session records which view the +television is showing (`castSession.owner`): a film or a photo opened meanwhile +takes it, the bar pauses and stops reading the receiver, and play gives it the +track back from where it was. A television chosen mid-track picks the track up +where it was; *Stop casting* carries it on locally from where the television +was. + +**The interface itself is not cast.** The default media receiver plays what it +is given and renders nothing of its own, and a rendered interface sent to it as +live video does not work: it starts a progressive stream that arrives at real +time only when the stream carries audio and its first seconds arrive faster than +real time, and then holds that lead as latency — five seconds, measured. Showing +the application on a television needs a registered receiver of our own. + **Subtitles are rebased onto the relay's clock before they are sent.** The node extracts a track whole, so its cues carry the film's timeline, and the player can use them unchanged because its SourceBuffer is given `timestampOffset = @@ -3539,7 +3717,7 @@ Two structural recommendations from that review stand as rules: |---|---| | **T1** | **The password split.** The hub never sees a passphrase; it holds a verifier for a client-derived `auth_key`. The passphrase floor can therefore only be enforced client-side (§3.1) | | **T2** | The hub was the key directory. **Closed** by admission redesign, not by safety numbers: the invite path reads no directory at all (§3.4). Reclassified as **H3** | -| **T3** | **The hub serves the SPA. Accepted permanently for browser users.** It is the only remaining way an active hub reads content, it is an artifact-level attack rather than a silent lie, and it does not exist for a native client — whose value is realised by reproducible builds, not by packaging (§2.3, §8.2) | +| **T3** | **The hub serves the SPA. Accepted permanently for browser users.** It is the only remaining way an active hub reads content, it is an artifact-level attack rather than a silent lie, and it does not exist for a native client — whose value is realised by reproducible builds, not by packaging (§13.9, §8.2) | ### 13.5b Availability between members (`AV`) @@ -3605,7 +3783,7 @@ had already been asked. | **AV15** | **A hash is checked for shape before it is a key lookup**, on every blocklist endpoint, the administrator's included | | **AV20** | **Chat is bounded in size and in rate, like every other member-supplied write** (§6.6). A message is a row on the operator's disk that nothing expires, a relayed copy for every connected member and a notification for every member of the group; the only ceiling was the frame size. Uploads had carried four protections and a cap since C5a because somebody asked what one member costs the others on that path, and nobody had asked it on this one | | **AV21** | **A lease is what the node granted, not what the client called it** (§5.5). `tr` was read as a boolean, so any non-empty string skipped the leaseless ceiling and every cap behind it, and a queued transfer was held back only by the honesty of the client waiting in the queue | -| **AV22** | **The node's own controls take no authority from a hub token** (§6.7). `node_status`, `node_settings_set`, `roster_read`, `denylist_read`, `denylist_clear` and `node_reload` were gated on the account id in the JWT, which is the hub's to choose — NS4 and M3 with the check written the other way round. The gate is a proved operator device, which a hub holding no user keys cannot produce | +| **AV22** | **The node's own controls take no authority from a hub token** (§6.7). `node_status`, `node_settings_set`, `roster_read`, `denylist_read`, `denylist_clear` and `node_reload` were gated on the account id in the JWT, which is the hub's to choose — NS4 and M3 with the check written the other way round. The gate became a proved operator device, which a hub holding no user keys cannot produce; since MNP 6.0 the messages are gone and these controls are loopback and CLI only | | **AV23** | **An upload's owner is recorded when the upload ends and applied when the entry is created**, which are different moments (§5.4). Written against the index at the end of the upload it matched nothing, every time, and left every uploaded file owned by nobody — so no member could delete what they had sent | | **AV24** | **A node registered for no group is refused signaling, not exempted from it** (§7.2). The membership check was written as "if the node claims any group", so it skipped itself — membership, group status and the public-group gate together — for the node AV1 made commonplace: the unconfigured one, which is also the one least able to absorb the work | | **AV25** | **Which nodes host a group is answered to its members** (§7.3). Only the public case checked, so a private group told any authenticated account that knew its id which machines hosted it — and an ex-member knows that id for ever | @@ -3672,6 +3850,72 @@ had already been asked. | **O13** | **Hub identity pinning.** The client points at a hub by URL and nothing pins that hub's identity. Bounded, because a substituted hub can neither read content nor ship the code to a native client — worth doing all the same | | **V1–V13**, **P1–P5** | Per-application open items: wording of a disabled-service state, whether artwork reuses the chunk path, cache TTL, multi-track surfacing, HEIC/RAW support, a fuller EXIF panel, lightbox preloading, album-boundary behaviour, cover selection | +### 13.9 Formal adversary model + +§2 states the trust model for a human reader. This restates the same ground +formally, for auditors and implementers: the adversaries every claim is written +against, the claim-by-adversary matrix, and the over-claims the project refuses to +make. Nothing here is new — it is §2 with the proofs shown. + +"End-to-end" throughout describes **client ↔ node**, never client ↔ client. The +operator and the members read everything in their group; that is what a group is. + +#### Adversaries + +Every claim is written against one of these, and they are the only ones the +document uses: + +| Adversary | What they can do | +|---|---| +| **Passive hub** | Read everything the hub legitimately stores and relays | +| **Active hub** | Also lie: forge tokens, invent accounts, substitute values it publishes, ship modified client code to a browser | +| **Malicious node operator** | Read and alter everything on their own machine, including the plaintext files they host | +| **Malicious group member** | Everything a member may do, plus anything the protocol fails to refuse | +| **Network attacker** | Observe and tamper with traffic between any two parties | +| **Local attacker** | Reach loopback services and files on a client or node machine | +| **Registered hub user with no membership** | Reach every hub endpoint that does not check membership | +| **Federated peer hub** | Push directory rows and revocations over MHP | + +#### Claim matrix + +| Claim | Passive hub | Active hub | Malicious node operator | Malicious member | Network attacker | +|---|---|---|---|---|---| +| Data never transits the hub | ✅ | ✅ | — | — | ✅ | +| File content is unreadable | ✅ | ❌ **T3** (browser) · ✅ native | ❌ by design — the operator hosts the files | ❌ members share the group key | ✅ | +| The file index is unreadable | ✅ | ❌ T3 · ✅ native | ❌ | ❌ | ✅ | +| Chat content is unreadable | ✅ | ❌ T3 · ✅ native | ❌ — the operator is a member | ❌ | ✅ | +| Chat is unreadable **from a copy of the node's storage that lacks its unlock key** — not from a whole disk by default (§4.5) | ✅ | ✅ | — the operator holds the unlock key | ✅ | ✅ | +| Content cannot be modified | ✅ | ✅ | ❌ by design | ✅ | ✅ | +| The node cannot be impersonated | ✅ | ✅ | — | ✅ | ✅ | +| Client code integrity | ❌ **T3, accepted** (browser) · ✅ ships in the package (native) | ❌ T3 · ⚠️ native: **publicly verifiable, not prevented** | ✅ | ✅ | ✅ | +| The hub cannot obtain the group key | ✅ | ✅ **except** in an open-join group, where it can join legitimately (§7.3), and for an invitation the inviter asked the hub to mail, whose code it then holds (§3.4) | — | — | ✅ | +| Node content authority | ✅ | ✅ | ✅ sovereign | ✅ | ✅ | +| Devices cannot be added by the hub | ✅ | ✅ — the hub holds no user key and cannot countersign | ⚠️ a node adds a device only to itself, where it already reads everything | ✅ | ✅ | +| Chat senders are authenticated to each other | ✅ | ✅ | ⚠️ only for accounts the reader has already seen (§3.3) | ✅ | ✅ | +| Keypair bundles (**C4**) | ✅ it holds the pepper and no bundle | ⚠️ it can fetch a bundle with a token it mints and holds the pepper: an offline passphrase search, as T3 already concedes for browsers · none to fetch for an account without browser access | ✅ no offline search: the bundle does not open without the hub's pepper — only sign-in attempts, bounded and audited · none on disk for an account without browser access | — | — | +| Deleting your account erases you | ✅ hub-side | ✅ hub-side | ❌ files, pinned identity and bundle stay on the node (§7.7) | — | — | +| Your identity keys stay yours | ✅ | ⚠️ as the row above | ✅ the bundle they hold does not open without the pepper, and a leaked bundle key opens **that node's** bundle only | ✅ | ✅ | + +#### Over-claims the project refuses to make + +Three sentences are forbidden, each for a deliberate reason: + +- **"Everything is encrypted and unreadable by other parties, even the hub."** + A hub that ships the code can lift keys from the page regardless of protocol + design (**T3**). That is an artifact-level attack, not a silent directory lie, + and it is removed for native clients — not for browsers. +- **"A native client makes the hub untrusted."** It converts an undetectable, + per-request, per-user attack into a persistent artifact that can be hashed and + compared. That value is realised by reproducible builds and published hashes, + not by the packaging format. A build signed with a key the hub operator holds + *relocates* trust; it does not remove it. +- **"C4 is closed."** It is closed for an account whose identities the desktop + application keeps (browser access off, §3.7): nothing of them is on any node. + For an account with browser access the bundle is still on each node, sealed + under the passphrase **and** a pepper the hub holds — no operator can search it + offline, but an active hub can, and that is T3's adversary already. **An account + is only as strong as its weakest client.** + --- ## 14. Decisions that are not revisited @@ -3743,7 +3987,9 @@ pause and resume, the group-application framework with Chat, Files, Videos, Music and Photos, cross-group search with source merging, per-account playlists, casting to a Chromecast with subtitles rebased onto the relay's clock, the operator CLI and loopback control API, the desktop client through its identity -and download stages, account recovery, and the Windows port through packaging. +and download stages, account recovery, the Windows port through packaging, and +the Android client — the shell, native keys, downloads and uploads, and casting +(§11.3). The packages install: a machine has been taken from the built artefacts to a running hub and node on **Ubuntu 26.04 (`.deb`), Fedora 44 (`.rpm`) and @@ -3771,13 +4017,19 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | — | **Bitmap subtitles** (PGS, VOBSUB — about a fifth of the embedded streams). No WebVTT without OCR; they are not listed rather than listed and blank. Burn-in covers them and costs `-c:v copy`, which is what the eight-slot sizing assumes never happens | | — | Delegation (§3.4) | | — | Tier 3 roster attestation (§3.3) | -| — | Android client | +| — | **Android: phone behaviour and release** — the back button driving the page, recovery from a network handover, keeping a download alive with the screen off, lock-screen media controls, and a release key (§11.3) | | — | **Federation between two hubs.** The protocol is written and switched off in the code (§7.6); what is not built is one run between two machines | ### 15.3 Open, and why each is where it is | Item | Status | |---|---| +| **The desktop cast relay drops whole segments** | `cast-relay.js` drops a fragment once 8 MB wait for a receiver, and a fragment is a segment, 5 to 10 MB at a film's bitrate: the picture freezes for its length. Its backlog is bounded in fragments only. The Android relay spools a receiver's lead to disk and bounds its backlog in bytes (§11.4); the desktop has neither | +| **The desktop cast relay takes the first chunk for the whole header** | The node's first chunk can be the ftyp alone, the moov in the next; the receiver then gives up. Fixed in the Android relay (§11.4), not in `cast-relay.js` | +| **A cast restart may pull far ahead** | Seen once on an emulator with a synthetic film: after the restart's reinit the node reported `duration=None`, and the page pulled most of the film at network speed. Not reproduced on a real film; the suspicion is a read-ahead budget computed without a duration | +| **"Copy stream URL" after picking a receiver casts to that receiver** | The player keeps the last device chosen, so the copy-only path reconnects it instead of only starting the relay | +| **The home page says "No groups yet" when the hub cannot be reached** | An unreachable hub reads as an account with no groups, rather than as an error | +| **`meshbay-node init` says "Settings → Link Node"** | The control is on the Profile page, as QUICKSTART says | | **C4** for accounts with browser access | Closed against operators by the pepper; **open against an active hub**, which holds the pepper and can fetch a bundle with a token it mints — the adversary T3 already concedes for browsers (§3.7) | | **A desktop that never held an identity cannot open its recovery copy** | The application opens a node's passphrase copy, not the recovery copy: after a reset, an identity it never held is recovered from a browser (§3.6). And an identity the application mints gets a recovery copy only when the recovery key is entered on its Profile page with browser access on | | **T3** for browser users | **Accepted permanently.** Removed for native clients, and that removal's value depends on reproducible builds | @@ -3791,10 +4043,12 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | **The reconnect backoff only wakes on `visibilitychange`** | So a tab that stays visible through an outage — which is what a screen wake lock guarantees while a film is playing — waits out the full backoff, up to 30 s, after the network is already back. Nothing listens for `online` | | **Per-device revocation has no CLI** | A device is revoked over MNP (`roster.revoke_device`), from a device the node has already pinned. On a headless node the operator's only lever is `member unpin`, which removes **every** device of that account — so the per-device control the roster is built around is reachable from an interface and from nowhere else. §6.7 listed a `meshbay-node member device list\|revoke` verb that was never written, and that listing is how this was found: `USERGUIDE.md` was the first document written by reading the CLI rather than this specification, and the verb it copied out did not run | | **Migrations run on SQLite only** | The chain reaches head and agrees with the models there (§12), which is not where it ships. **The exposure is one revision deep, not the whole chain**: every revision behind the first packaged release was development that no installation ever ran, so nothing replays them on PostgreSQL. What is unguarded is the *next* migration — a default, an index type or a constraint PostgreSQL refuses reaches a deploy without the suite saying so | -| **The loopback path removes access without writing an audit entry** | `ops.revoke_member` and `ops.unpin_member` log to the daemon's log and nothing to `audit.db`; the MNP admin handlers doing the same work audit `member_revoke` and `member_unpin`. So a removal made from the node page or the CLI — the two doors an operator sitting at their own machine actually uses — leaves the journal showing an admission and then, whenever that person next connects, an `auth_failed` ("not admitted by the roster") with nothing in between to explain it. §5.4's signed transcript is not what is missing: a loopback caller is authorized by being on localhost with the run token and signs nothing, so the gap is the record, not the authority. Found by reading a node's audit log for a refusal whose cause was six hours earlier and unrecorded | +| **The loopback path removes access without writing an audit entry** | `ops.revoke_member` and `ops.unpin_member` log to the daemon's log and nothing to `audit.db`; the MNP `member_revoke` handler doing the same work audits it (and `member_unpin` did, until it left MNP in 6.0). So a removal made from the node page or the CLI — the two doors an operator sitting at their own machine actually uses — leaves the journal showing an admission and then, whenever that person next connects, an `auth_failed` ("not admitted by the roster") with nothing in between to explain it. §5.4's signed transcript is not what is missing: a loopback caller is authorized by being on localhost with the run token and signs nothing, so the gap is the record, not the authority. Found by reading a node's audit log for a refusal whose cause was six hours earlier and unrecorded | | **The Create group wizard calls two hooks after an early return** | `CreateGroupWizard` (`create-group-page.js`) returns during node detection, before its `useRef`/`useEffect` for provisioning, so the hook count changes between renders. Preact tolerates a list that grows, and nothing is known to break; `test_hook_ordering.py` checks declaration order, not this. Found while tracing the frozen-fields report, which had another cause (`ask.js`) | | **A node key is read from the terminal or the desktop client, never a browser** | **Accepted.** `meshbay-node status` on the node's own machine and Node → Overview in the desktop client are the two places the key can be read; the Node page is Electron-only, because `platform.node` resolves to "not available" without the bridge, and no hub route exposes the key. The create-group wizard links it automatically over that same bridge, so the manual paste in **Profile → Link Node** exists for the operator who runs the node from a terminal and the hub from a browser — who has a terminal by definition. Anyone linking a node is already at a shell prompt, so a browser-reachable copy would buy nothing and widen what the hub knows about the node | | **Listing a group's folders walks every root on the event loop** | `index_sync_message` (`transport/wire.py`) builds its `dirs` field with `list_dirs`, an `rglob("*")` over every root, and nothing sends it off the loop: the WebRTC `index_sync` handler, the daemon's index push and QUIC all call it inline. So each index request from any member is a directory walk of the whole library that every other peer on the node waits behind. `test_disk_io_off_loop.py` never saw it, because it reads the transport's own modules and the walk is one call away in `wire.py`. Found by widening what that test reads, not by a symptom | +| **A loopback eject or plug reaches open pages late** | `ops.eject_root` and `ops.plug_root` flip the live set and tell nobody; over MNP the broadcast `root_eject_ack` / `root_plug_ack` is what moves every open table. So an eject made from the desktop application or the CLI shows on members' pages only with the next index push — for a plug, the end of its rescan; for an eject, whatever changes next. `ops.update_root` had the same silence and now calls `DirectoryIndexer.publish_roots`; the same call belongs in these two. Found while moving the `writable`/`removable` switches to the loopback door (MNP 6.0) | +| **A group key rotated from the Node page leaves the chat key where it was** | `ops.set_gek(rotate=True)` replaces the group key and opens no chat epoch; the MNP `gek_rotate` handler opened one itself (`_new_chat_epoch`), and it was the only door that did — but no client ever sent it, and it is gone since 6.0. The removals that matter (revoke, unpin, device revoke) open an epoch in `ops` for every door, so what is missing is the follow-through for an operator who rotates by hand: §4.5's "rotate after a removal" means it for chat too. The fix is the `_after_removal` shape — `open_chat_epoch` inside `ops.set_gek` when `rotated`. Found while removing the MNP message | | **The transcoded-seek test passes without transcoding** | `test_a_transcoded_video_keeps_accurate_seeking` (`test_stream_seek_audio_alignment.py`) forces the re-encode branch by swapping the module's `BROWSER_INCOMPATIBLE_VIDEO_CODECS`, then checks only that the result has no audio gap. The copy path also leaves no gap on that clip, so pointing the swap at a module the streaming code does not read still passes: the test cannot tell that the branch it is named after never ran. It should assert the re-encode happened (the `re-encoding` log line, or the encoder in the ffmpeg argv). Found by breaking the swap on purpose while moving the streaming code | --- @@ -3814,7 +4068,7 @@ superseded document kept under `docs/`, a note somebody holds elsewhere. | Cited as | Read | |---|---| -| `draft-v5 §2`, `draft-v6 §4` — security claims | §2.2 | +| `draft-v5 §2`, `draft-v6 §4` — security claims | §13.9 | | `draft-v5 §3` — transport, NAT traversal | §5.1 | | `draft-v5 §4`, §4.1–4.4 — handshake, transcript, channel binding, mutual auth | §5.2 | | `draft-v5 §5.1`, `draft-v6 §2.3`, `§2.4b` — privileged operations, key activation, authorship | §5.4 | diff --git a/docs/MESHBAY_HTTP_API.md b/docs/MESHBAY_HTTP_API.md new file mode 100644 index 0000000..74b9d77 --- /dev/null +++ b/docs/MESHBAY_HTTP_API.md @@ -0,0 +1,233 @@ +# MeshBay HTTP API + +> **Generated** by `docs/generate_http_api.py` from the routes themselves. Do not +> edit this file: change the route's docstring and run the script again. A test +> fails when the two disagree. + +MeshBay has two HTTP APIs: the **hub's**, for accounts, groups and signaling, and +the **node's control API**, which only its own machine reaches. Files, the index +and chat do not use either: they travel between a client and a node over MNP +(`MESHBAY_NODE_PROTOCOL.md`). Why each API is shaped as it is, who may call +what and what the hub must never see are in `MESHBAY_DESIGN.md`; this file lists +what exists. + +`examples/` has small Python programs that use both. + +## Hub + +Under the hub's address, `https://meshbay.org` on the reference deployment. The +hub also serves the web application at `/` and `/app/`, which are not listed. + +| Auth | What the request carries | +|---|---| +| none | No session. Any credential is in the request itself, as the route says | +| user | `Authorization: Bearer`, a person's session. A node's token is refused | +| user or node | A person's session, or a node daemon's token from `/v1/nodes/auth` | +| node | A node daemon's token only | +| moderator | A person's session, for an account with the moderator or admin role | +| admin | A person's session, for an account with the admin role | +| peer hub | Another hub's MHP token. Every route answers 503 unless federation is enabled | + +### Instance + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/v1/hub/info` | none | Versions and the instance policy a client needs before signing in. | +| GET | `/v1/hub/pubkey` | none | Hub Ed25519 public key PEM — cached by nodes on first contact. | +| GET | `/v1/hub/version` | none | Version check endpoint for clients to detect updates. | + +### Accounts + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/users/register` | none | Create an account. | +| POST | `/v1/users/verify-email` | none | Verify a registration email with the code received by mail. | +| POST | `/v1/users/me/bundle-pepper` | user | The pepper, for a session that has just been given the passphrase again. | +| POST | `/v1/users/login` | none | Sign in with the auth key derived from the passphrase. | +| POST | `/v1/users/devices` | user | Register a device's hub authentication key. | +| GET | `/v1/users/devices` | user or node | The account's registered devices. | +| DELETE | `/v1/users/devices/{device_id}` | user | Retire a device's hub key. | +| POST | `/v1/users/auth` | none | Sign in with a registered device key. | +| POST | `/v1/users/token/refresh` | none | Exchange a refresh token for a new session. | +| GET | `/v1/users/me` | user or node | The signed-in account: id, name, e-mail, role, status. | +| PATCH | `/v1/users/me` | user | Update the signed-in account. | +| POST | `/v1/users/verify-email-change` | user | Confirm an email change with the code sent to the new address. | +| POST | `/v1/users/logout` | none | End this session on the hub, not only in the browser. | +| POST | `/v1/users/me/sessions/revoke` | user | Sign out everywhere: no refresh token of this account renews any more. | +| POST | `/v1/users/password` | user | Change the passphrase, proving the current one. | +| POST | `/v1/users/password/reset-request` | none | Send a reset code by e-mail, when the username and the address match. | +| POST | `/v1/users/password/reset` | none | Set a new passphrase with the code received by e-mail. | +| GET | `/v1/users/me/preferences` | user or node | The account's stored interface preferences. | +| PUT | `/v1/users/me/preferences/{key:path}` | user | Store one interface preference. | +| DELETE | `/v1/users/me/preferences/{key:path}` | user | Remove one interface preference. | +| PUT | `/v1/users/me/node_key` | user | Link a node daemon's Ed25519 public key to the operator's account. | +| DELETE | `/v1/users/me/node_key` | user or node | Remove the linked node key from the operator's account. | +| DELETE | `/v1/users/me` | user | Erase your own account. | +| GET | `/v1/users/{username}/pubkeys` | user or node | Resolve a username to its account id, and its node's linking key. | + +### Nodes + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/nodes/mnp-token` | user | Mint the short-lived token a member presents to a node in the MNP handshake. | +| POST | `/v1/nodes/auth` | none | Authenticate a node daemon via Ed25519 challenge-response. | +| POST | `/v1/nodes/announce` | user or node | Register a node record. | +| GET | `/v1/nodes/{node_id}` | user or node | A node's public record: owner, key, endpoint hint. | + +### Groups + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/v1/groups/mine` | user or node | List groups the current user belongs to. | +| GET | `/v1/groups/invitations` | user or node | Groups somebody added this account to, waiting for it to say yes. | +| POST | `/v1/groups/{group_id}/invitation/accept` | user | Accept an invitation: the account becomes a member of the group. | +| POST | `/v1/groups/{group_id}/invitation/decline` | user | Decline an invitation to a group. | +| POST | `/v1/groups/{group_id}/activity` | user or node | Bump a group's last_activity_at. | +| GET | `/v1/groups/{group_id}/nodes` | user or node | Return online nodes that serve a group (for WebRTC connection). | +| GET | `/v1/groups` | none | List/search public groups — local and optionally federated. | +| GET | `/v1/groups/{group_id}/members` | user or node | A group's members, for its members only. | +| POST | `/v1/groups/{group_id}/join` | user | Join an open group. | +| POST | `/v1/groups` | user | Create a group, owned by the caller. | +| DELETE | `/v1/groups/{group_id}/members/{username}` | user | Remove someone from a group. | +| POST | `/v1/groups/{group_id}/leave` | user | Leave a group you are a member of. | +| PATCH | `/v1/groups/{group_id}` | user | Change the group's description. | +| POST | `/v1/groups/{group_id}/members/{username}` | user or node | Add an account to a group the caller owns. | +| POST | `/v1/groups/{group_id}/mute` | user | Turn this group's notifications on or off, for this account. | +| DELETE | `/v1/groups/{group_id}` | user | Delete a group. | +| POST | `/v1/groups/{group_id}/invite-notify` | user | Send an invitation email to a member who was just invited. | +| GET | `/v1/groups/{group_id}/hosts` | user | The nodes that host a group or asked to, for its owner. | +| POST | `/v1/groups/{group_id}/hosts/{node_id}` | user | Approve a node that asked to host this group. | +| DELETE | `/v1/groups/{group_id}/hosts/{node_id}` | user | Withdraw an approval, or turn a request down. | + +### Invitation links + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/groups/{group_id}/invite-links` | user or node | Mint the ticket for a link whose node half already exists. | +| GET | `/v1/groups/{group_id}/invite-links` | user or node | The owner's view: the links nobody has used yet, masked. | +| DELETE | `/v1/groups/{group_id}/invite-links/{link_id}` | user or node | Take the ticket back. | +| POST | `/v1/invite-links/preview` | user | What the confirmation screen shows before anyone joins anything. | +| POST | `/v1/invite-links/redeem` | user | Membership for the first account that asks, once — and the same answer again for that account, because a second tab or a reload is the same person. | + +### Revocation and the node socket + +| Method | Path | Auth | What | +|---|---|---|---| +| WS | `/v1/nodes/ws` | none | Persistent WebSocket connection for nodes, authenticated by the node's token in the first message. | +| POST | `/v1/nodes/{node_id}/incoming` | user or node | Signal a node that a client wants to connect (NAT punch coordination). | +| POST | `/v1/admin/revoke` | admin | Revoke a user or group. | + +### Moderation + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/reports` | user | Report a file of a public group, as a member of that group. | +| GET | `/v1/blocklist` | node | The content blocklist, a page at a time, for a node hosting a public group. | +| GET | `/v1/admin/blocklist` | admin | The content blocklist. | +| POST | `/v1/admin/blocklist` | admin | Add a content hash (BLAKE3) to the blocklist. | +| DELETE | `/v1/admin/blocklist/{content_hash}` | admin | Remove a content hash from the blocklist. | +| GET | `/v1/admin/reports` | moderator | Hashes waiting for a decision, oldest first, with what was said about them. | +| POST | `/v1/admin/reports/{content_hash}/block` | admin | Block reported content and close its reports. | +| POST | `/v1/admin/reports/{content_hash}/dismiss` | admin | Dismiss the reports on a piece of content. | + +### Federation (MHP) + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/mhp/info` | peer hub | Return this hub's identity for peer registration. | +| GET | `/mhp/directory` | peer hub | This hub's public groups, for a peer hub presenting an MHP token. | +| POST | `/mhp/directory` | peer hub | A peer hub's public groups, pushed with a single-use MHP token. | +| POST | `/mhp/revoke` | peer hub | Act on a revocation from a peer hub. | +| POST | `/mhp/peers` | admin | Admin: register a trusted peer hub. | +| GET | `/mhp/peers` | admin | Admin: list registered peer hubs. | + +### Health + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/v1/health` | none | Liveness: database reachable, version, connected nodes. | + +### Signaling + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/nodes/{node_id}/webrtc/offer` | user or node | Browser sends WebRTC SDP offer for a node. | + +### Administration + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/v1/admin/settings` | moderator | Instance-wide policy an admin controls from the panel. | +| PATCH | `/v1/admin/settings` | admin | Change instance policy. | +| GET | `/v1/admin/mail` | moderator | Is the hub still sending, and how much of the hour is left. | +| GET | `/v1/admin/stats` | moderator | Account, group and node counts. | +| GET | `/v1/admin/users` | moderator | Search and list accounts. | +| GET | `/v1/admin/users/{user_id}` | moderator | One account, with its group count. | +| PATCH | `/v1/admin/users/{user_id}` | moderator | Change an account's status, or its role (admin only). | +| DELETE | `/v1/admin/users/{user_id}` | admin | Erase an account, and every group it owns. | +| GET | `/v1/admin/groups` | moderator | List groups with their member counts. | +| PATCH | `/v1/admin/groups/{group_id}` | moderator | Change a group's status. | +| GET | `/v1/admin/nodes` | moderator | Registered nodes, with the address the hub saw them announce from. | +| GET | `/v1/admin/logs` | moderator | The connection log, filtered by account and event. | + +### Notifications + +| Method | Path | Auth | What | +|---|---|---|---| +| GET | `/v1/notifications` | user or node | The account's notifications, newest first. | +| POST | `/v1/notifications/{notification_id}/read` | user or node | Dismiss one. | +| DELETE | `/v1/notifications/{notification_id}` | user or node | Dismiss one. | +| DELETE | `/v1/notifications` | user or node | Throw them all away. | +| POST | `/v1/notifications/read-all` | user or node | Dismiss every one — the same thing as `DELETE ""`, under the name an older client knows it by. | + +## Node control API + +`http://127.0.0.1:<ui_port>`, port 18000 unless `ui_port` in `node.toml` says +otherwise, and never on another address. Every request carries the token the +daemon writes to `<data_dir>/ui-token` (`~/.local/share/meshbay/ui-token` on +Linux), as the `X-MeshBay-Token` header or the `t` query parameter. The daemon +draws a new one at each start and deletes the file when it stops. + +| Method | Path | What | +|---|---|---| +| GET | `/api/status` | The daemon's state, and what it still needs: a linked key, a group, an operator, a group key. | +| DELETE | `/api/unlink` | Unlink the node's key from its hub account. | +| GET | `/api/groups` | The groups this node hosts, with live status, and whether an operator is paired. | +| POST | `/api/groups/attach` | Host a group that exists on the hub: add it to node.toml with its first folder, then reload. | +| POST | `/api/groups/detach` | Stop hosting a group: remove it from node.toml, then reload. | +| DELETE | `/api/groups/{group_id}/files/{file_id}` | Delete a file from the group's folder on disk. | +| GET | `/api/denylist` | What the node currently refuses. | +| POST | `/api/denylist/clear` | Drop denylist entries: all of them, or one identifier. | +| GET | `/api/index-cache` | Size of the index cache. | +| POST | `/api/index-cache/prune` | Drop index cache rows that no longer match a file on disk. | +| POST | `/api/groups/{group_id}/video/rematch` | Forget the automatic matches of the group's videos, so they are looked up again. | +| GET | `/api/groups/{group_id}/files` | The group's files, from its index. | +| GET | `/api/peers` | The connected peers. | +| GET | `/api/audit` | The audit log, filtered by time, account and event. | +| POST | `/api/operator/pair` | A one-time code that pairs an application as this node's operator. | +| GET | `/api/roster` | The pinned identities, for one group or all. | +| POST | `/api/groups/{group_id}/invites` | An invitation code for one account, for this group. | +| POST | `/api/groups/{group_id}/invite-links` | A whole invitation link: the node's code, then the hub's ticket. | +| DELETE | `/api/groups/{group_id}/invite-links/{invite_id}` | Take an invitation link back, on the node and on the hub. | +| GET | `/api/resolve` | Map a username to an account id, through the hub. | +| POST | `/api/members/{user_id}/revoke` | Stop serving the group key to a member. | +| POST | `/api/members/{user_id}/unpin` | Forget a pinned identity, so the person can pair again with a new key. | +| GET | `/api/groups/{group_id}/chat` | What the operator needs to decide anything about the group's chat. | +| POST | `/api/groups/{group_id}/chat/epoch` | Open a new chat epoch. | +| POST | `/api/groups/{group_id}/chat/encrypt-history` | Re-encrypt the messages written before the group's chat was encrypted. | +| POST | `/api/groups/{group_id}/chat/prune` | Delete chat messages older than a number of days. | +| POST | `/api/groups/{group_id}/gek` | Generate the group key, or rotate it with ?rotate=true. | +| POST | `/api/groups/{group_id}/roots` | Add a folder to a group. | +| PATCH | `/api/groups/{group_id}/roots/{root_name}` | Make a folder writable or removable, or not. | +| PUT | `/api/groups/{group_id}/roots/{root_name}/eject` | Eject a removable folder so its disk can be unplugged. | +| PUT | `/api/groups/{group_id}/roots/{root_name}/plug` | Bring an ejected folder back. | +| DELETE | `/api/groups/{group_id}/roots/{root_name}` | Remove a folder from a group. | +| GET | `/api/groups/{group_id}/index-status` | One group's indexing progress. | +| GET | `/api/index-status` | Every group's indexing progress. | +| PUT | `/api/groups/{group_id}/apps` | Which applications members see for the group. | +| POST | `/api/reload` | Reload node.toml. | +| POST | `/api/shutdown` | Stop the daemon. | +| GET | `/api/node-settings` | The node's effective settings. | +| PUT | `/api/node-settings` | Change node settings, written to roster.db and node.toml. | +| GET | `/api/transfers` | Live transfer leases and queue depth. | +| PUT | `/api/groups/{group_id}/transfer-limits` | How many transfers one member may run at once in this group. | diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index e7fce90..0b5213c 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1,6 +1,6 @@ # MeshBay Node Protocol (MNP) -**Wire version:** `5.0` — `meshbay_common/__init__.py` (`MNP_VERSION`) +**Wire version:** `6.0` — `meshbay_common/__init__.py` (`MNP_VERSION`) **Oldest peer accepted:** `4.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) **Normative implementation:** `meshbay-common` (`protocol.py`, `handshake.py`, `groupbox.py`, `chatbox.py`, `adminop.py`, `join.py`, `device.py`, `crypto.py`, @@ -162,7 +162,7 @@ Codes in use: | Upload | `upload_not_sealed`, `no_group_key`, `lease_not_granted`, `upload_incomplete`, `bad_chunk_encoding`, `bad_chunk_index`, `invalid_filename`, `no_roots`, `no_such_root`, `no_writable_root`, `root_read_only`, `root_unavailable`, `no_such_directory`, `already_exists`, `not_started`, `too_large` (§11.4) | | Directories | `root_read_only`, `root_unavailable` (§11.5) | | Chat | `chat_too_large`, `chat_rate_limited` (§11.7) | -| Operator controls | `not_operator`, `too_many_pending`, `too_large` (§10.4) | +| Operator controls | `too_many_pending`, `too_large` (§10.4) | | Metadata | `transcode_not_applicable`, `tmdb_search_rate_limited` (§11.9) | | Moderation | `content_blocked` — a file the hub's content blocklist names, in a public group (§11.3) | @@ -1156,8 +1156,6 @@ broadcast, every connected peer in the group learns the change without reconnect | `invite_link_create` | `link:<group_id>`, the session's group | operator only | `invite_link_result{code, invite_id, expires_at, group_id}` | no — the code is shown once | | `invite_cancel` | `invite_id` (32 hex) | operator only | `ack{detail: "invite_cancelled", invite_id}` | no | | `member_revoke` | `user_id` | operator | `member_revoke_ack` | no | -| `member_unpin` | `user_id` | operator | `member_unpin_ack{user_id}` | no | -| `gek_rotate` | `group_id` | operator | `gek_rotate_ack{group_id, authorized_members, note}` | no | | `apps_enabled` | the app set | operator | `apps_enabled_ack{apps}` | yes | | `set_scan_settings` | the interval/debounce pair | operator | `set_scan_settings_ack{...}` | yes | | `tmdb_config` | `{token, language}` — `token` is `null` (unchanged), `""` (clear) or `sha256:<hex>` of the token, never the token | operator | `tmdb_config_ack{token_customized, language}` | yes (never the token) | @@ -1165,24 +1163,33 @@ broadcast, every connected peer in the group learns the change without reconnect | `tmdb_override` | `file_id=..,tmdb_id=..,media_type=..` | operator | `tmdb_override_ack{file_id, tmdb_id, media_type}` | yes | | `tmdb_rematch` | `file_id=..` | operator | `tmdb_rematch_ack{file_id}` | yes | | `musicbrainz_enabled` | `enabled` | operator | `musicbrainz_enabled_ack{enabled}` | yes | -| `root_add` | `{path, name, kind, writable, removable}` | operator | `root_add_ack` | no | | `root_remove` | the root name | operator | `root_remove_ack` | no | -| `root_update` | `<root>:rw=on\|off,rem=on\|off` | operator | `root_update_ack` | yes | | `root_eject`, `root_plug` | the root name | operator | `root_eject_ack` / `root_plug_ack` | yes | | `app_directories` | `<app>:<dir>,<dir>,...` | operator | `app_directories_ack{app, dirs}` | yes | | `chat_directory` | the path | operator | `chat_directory_ack{path}` | yes | | `chat_link_preview` | `on\|off` | operator | `chat_link_preview_ack{enabled}` | yes | | `search_listed` | `on\|off` | operator | `search_listed_ack{listed}` | yes | -| `transfer_limits` | `d=<n>,u=<n>` | operator | `transfer_limits_ack{limits}` | yes | | `chat_epoch` | `group_id` | operator | `chat_epoch_ack{epoch}` | yes | -| `group_attach` | `{name, shared_dir, writable}` | operator | `group_attach_ack` | no | -| `group_detach` | the group name | operator | `group_detach_ack` | no | **Upload policy is not in this table**, and that is the design: whether a member may -write is a property of each root (`root_update`), not a switch over the group. A single +write is a property of each root (its `writable` flag), not a switch over the group. A single group-wide flag cannot express "this library is published read-only and that folder is a drop box", which is the ordinary arrangement. +**Nothing in this table widens what the node shares**, and that is the design too. +Adding a directory to a group, hosting a new group over a directory, and switching +a root's `writable` or `removable` flag are done on the node's own machine — the +desktop application over the loopback API, or the CLI — and never over MNP. Until 6.0 +they were the signed ops `root_add`, `group_attach` and `root_update`. A signature +proves that the operator's key signed, not that the operator meant it: in a browser +that key is driven by code the hub serves (T3), and in the desktop application by a +renderer that parses content from nodes. Either could have shared any folder on the +operator's machine, writable, from anywhere. What is left here narrows (`root_remove`, +`root_eject`) or restores what the operator already shared (`root_plug`). The +operator still sees the table from any browser: it rides in `index_sync` and +`index_delta`, which is also how a change made on the node's machine reaches every +open page. + `app_directories` is the **only** way an application's folders are set: one message for every application, keyed by the app's own registry name, so adding an application adds no message type, no signed op and no handler. @@ -1201,18 +1208,14 @@ Their *storage* keys survive on the node — `Roster.LEGACY_DIR_KEYS` still read operator's disk rather than on the wire, and a node upgraded into this has to find its own configuration. -A second family of operator messages is **not** signed: `node_status`, `roster_read`, -`denylist_read`, `denylist_clear`, `node_settings_set`, `node_reload`. These are gated -by `_operator_device()`, which asks two things: the authenticated session's `user_id` -is the account the node records as its own (`node_user_id`, `is_node_admin()`), **and** -the device on this connection has proved, with `device_hello` (§9.4), a key the roster -holds as an operator. Anything else is refused with code `not_operator`. The first -alone would be a claim in a token the hub issued, and a hub that can name the operator -is a hub that can be one; the second is what it cannot forge, since it holds no user -keys. Three of them only read; the other three run through the same `ops` entry points -as the CLI and the loopback admin API. The distinction from the signed table above: -a signed op proves possession of an operator key for *this* operation, while these -prove it once per connection, through the device the connection identified. +**The node's own controls are not MNP messages.** Its status (every group, with each +root's absolute path), settings, roster, denylist and reload; rotating a group key; +forgetting a pinned identity; the per-member transfer caps; no longer hosting a group — +the Node page in the desktop application and the CLI do these over the loopback API, on +the operator's own machine. Until 6.0 MNP carried them too (`node_status`, +`node_settings_set`, `roster_read`, `denylist_read`, `denylist_clear`, `node_reload`, +`gek_rotate`, `member_unpin`, `transfer_limits`, `group_detach`), and no client ever +sent one: a door nobody calls is an untested way in (§10.5). **The subject covers everything the node acts on.** The signature covers `op`, the node, the group, the subject, the nonce and the time — nothing else of the request — @@ -1231,16 +1234,14 @@ operations (`too_many_pending` beyond), each at most 64 KiB of subject and paylo Rules that hold across the table: * **No MNP message can activate a group key.** The rule (I2) targets key material - arriving from outside, not the instruction: `gek_rotate` and `chat_epoch` are allowed - precisely because the node generates the new key itself with its own CSPRNG. Initial - `gek-init` stays local — with no group key there is no completed session to carry a - signed op anyway. + arriving from outside, not the instruction: `chat_epoch` is allowed precisely + because the node generates the new key itself with its own CSPRNG. Initialising and + rotating the group key stay local (loopback, CLI). * **There is no operation by which key material reaches the node** (§12). The node wraps for a key the recipient has proved possession of, so no such message is needed — and a path that does not exist cannot be mis-authorized, which is I10 applied to a message instead of a version. -* An operator cannot revoke or unpin **themselves** over the connection their pin - authorizes. +* An operator cannot revoke **themselves** over the connection their pin authorizes. * Rotation is what actually removes a revoked member's access. Revocation stops the node serving the *next* key; the ex-member still holds the current one, and content they already downloaded stays readable. The ack says so in words. @@ -1408,7 +1409,7 @@ The control plane is not covered either — see §14.2. **Nonce collision, since `upload` is the first purpose with volume.** One subkey per purpose and a fresh 96-bit random nonce per message: at one message per 48 KiB chunk, 2³² chunks is 200 TB uploaded under a single GEK before the collision probability -reaches 2⁻³², and `gek_rotate` exists. Deriving the nonce from the payload instead +reaches 2⁻³², and the group key can be rotated. Deriving the nonce from the payload instead would be worse, not better — two chunks of identical bytes are ordinary in a file. **Failure is fatal, never degraded** (I8). A client that cannot open an index message @@ -1466,9 +1467,9 @@ The **lease** is that object, and every transfer runs under one. **Caps.** Node-wide, 8 concurrent per kind by default; per member per group, 2 by default. A group with no value of its own gets the default, never "unlimited": reading an absent setting as no limit would leave the node-wide cap as the only control, which -is the situation leases exist to end. The per-group value is a signed operator -operation (`transfer_limits`, §10.4, bounded to 1–32; zero is refused, because a member -who may not transfer at all is a member the operator revokes). The node-wide values are +is the situation leases exist to end. The per-group value is set on the node's machine +(`ops.set_transfer_limits`, loopback and CLI; bounded to 1–32, zero is refused, because +a member who may not transfer at all is a member the operator revokes). The node-wide values are daemon settings. A member's own cap rides on every `transfer_state`, so the interface can say "2 of your 2 slots are busy" rather than draw a spinner that explains nothing. @@ -1935,7 +1936,7 @@ collide immediately. The design degrades correctly into the deployment that exis chain-based one would not have. **Epochs.** A new epoch is opened when the set of devices that may read *future* -messages shrinks — member revoke, member unpin, device revoke, `gek_rotate` — and by +messages shrinks — member revoke, member unpin, device revoke — and by hand with the signed `chat_epoch` op (§10.4). Old epochs are kept and still delivered to current members, which is what keeps history readable to the people who could already read it. The epoch key is wrapped under the group key **at delivery**, never stored @@ -2121,16 +2122,12 @@ it back (§3.5). | `admin_response` | C→N | auth | the operator's signature over that transcript | | `client_diag` | C→N | auth | the video player's own view of a stream, written to the node's log beside its own (a stream event at INFO, the periodic state at DEBUG); the node acts on none of it and sends no reply | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | -| `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | -| `transfer_limits` / `_ack` | C→N / N⇒C | signed | per-member transfer caps for this group | | `chat_epoch` / `_ack` | C→N / N⇒C | signed | open a new chat epoch by hand | | `app_directories` / `_ack` | C→N / N⇒C | signed | one application's folders, keyed by app name | | `chat_directory` / `_ack` | C→N / N⇒C | signed | where chat attachments are written | | `chat_link_preview` / `_ack` | C→N / N⇒C | signed | whether the node unfurls posted links | | `search_listed` / `_ack` | C→N / N⇒C | signed | whether members' cross-group Search lists this group | -| `root_update` / `_ack` | C→N / N⇒C | signed | a root's `writable` / `removable` flags | | `root_eject` / `_ack`, `root_plug` / `_ack` | C→N / N⇒C | signed | take a removable root offline, put it back | -| `gek_rotate` / `_ack` | C→N / N→C | signed | node generates a new group key | | `apps_enabled` / `_ack` | C→N / N⇒C | signed | which group apps are shown | | `set_scan_settings` / `_ack` | C→N / N⇒C | signed | reconcile interval and debounce | | `tmdb_config` / `_ack` | C→N / N⇒C | signed | node-wide TMDB token and language | @@ -2138,14 +2135,7 @@ it back (§3.5). | `tmdb_override` / `_ack` | C→N / N⇒C | signed | correct a wrong automatic match | | `tmdb_rematch` / `_ack` | C→N / N⇒C | signed | drop one file's cached match | | `musicbrainz_enabled` / `_ack` | C→N / N⇒C | signed | per-group MusicBrainz on/off | -| `root_add` / `_ack`, `root_remove` / `_ack` | C→N / N→C | signed | add or remove a shared directory | -| `group_attach` / `_ack`, `group_detach` / `_ack` | C→N / N→C | signed | start or stop hosting a group | -| `node_status` / `_ack` | C→N / N→C | auth (operator) | all groups, roots, daemon state | -| `roster_read` / `_ack` | C→N / N→C | auth (operator) | pinned identities and members | -| `denylist_read` / `_ack` | C→N / N→C | auth (operator) | current refusals | -| `denylist_clear` / `_ack` | C→N / N→C | auth (operator) | remove entries | -| `node_settings_set` / `_ack` | C→N / N→C | auth (operator) | change daemon settings | -| `node_reload` / `_ack` | C→N / N→C | auth (operator) | re-read `node.toml` | +| `root_remove` / `_ack` | C→N / N→C | signed | remove a shared directory | | `error` | N→C | any | refusal, with `detail` and optionally `code`, `req_id`, and the `upload_id` / `tr` / `file_id` it is about | | `ack` | N→C | auth | generic acknowledgement (chat, keypair bundle store) | @@ -2182,7 +2172,7 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`5.0`**; oldest peer +MNP versions independently of the package version. Current: **`6.0`**; oldest peer accepted: **`4.0`**. 4.0 is the floor: a member presents a short-lived node-audience token bound to one node @@ -2197,6 +2187,14 @@ selection, per-account blobs. four fail with a refusal and everything else works; no node accepts the old subjects, so nothing is left unsigned on either side. That is why the floor did not move. +6.0 is a MAJOR that removes three signed operations — `root_add`, `root_update`, +`group_attach` — rather than changing any (§10.4: nothing over MNP widens what a node +shares). A 5.x client that sends one gets no answer, as for any unknown type, and +everything else it does still works; the floor stays at 4.0. The same version removes +ten operator messages no client ever sent (§10.4, "The node's own controls"). The desktop application +also refuses to sign them, so a node older than 6.0 cannot be driven into them by a +script in its page either. + The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; `MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about whether an older peer can still do anything useful: @@ -2336,9 +2334,8 @@ walks through the gate meant to stop it. file chunks, stream segments, uploads, the chat keys and the group roster. It does not cover the admin and configuration acks (`app_directories_ack`, `root_*_ack` and the rest), which carry the same folder names the sealed index carries; the media-metadata replies (`media_meta_resp`, `music_meta_resp`, - `link_preview_resp`), which carry titles, artists and synopses; `node_status_ack`, - which carries absolute paths on the operator's disk to an operator session; the - identity replies (`roster_read_ack`, `device_list_result`); or `invite_result` and + `link_preview_resp`), which carry titles, artists and synopses; the identity reply + `device_list_result`; or `invite_result` and `invite_link_result`, which carry a pairing code. All are inside DTLS/TLS and none reaches the hub, but none is behind the group key. * **Transfer messages are in clear on purpose**, and that is a deliberate line rather @@ -2414,7 +2411,7 @@ LP(x) = uint32be(len(x)) || x every field, no exceptions | Constant | Value | Source | |---|---|---| -| `MNP_VERSION` | `5.0` | `meshbay_common/__init__.py` | +| `MNP_VERSION` | `6.0` | `meshbay_common/__init__.py` | | `MNP_MIN_SUPPORTED` | `4.0` | `handshake.py` | | `MNP_AUD` / `HUB_API_AUD` | `meshbay:mnp` / `meshbay:hub-api` | `tokens.py` | | MNP token lifetime | 900 s | `meshbay-hub/auth.py` (`issue_mnp_token`) | diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index ce1eec1..ea75cab 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -359,7 +359,7 @@ A healthy node reads roughly like this: ``` hub https://meshbay.org (user yourname) node key 7mK2p...= -daemon running — ok +daemon running node_id 82.65.x.x:0 groups 1 files 4213 peers 1 config /home/you/.config/meshbay/node.toml diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index e92f6bc..ea81c76 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -297,6 +297,19 @@ file browser — sort, select, download, preview. - **Right-click a file or folder** for the same actions as the toolbar, listing only the ones that apply to it. On a ticked row the menu acts on everything ticked, like the toolbar does. +- **A link to a group, a folder or a file.** While a group is open the address + bar shows `https://<hub>/#/name@owner` — the name under the group's title. + Add a path after it to point inside the group: + `#/name@owner/root/folder/photo.jpg` downloads that file, and a folder opens + Files there. Only members get anywhere with such a link — anyone else is told + the group is unknown — and someone not signed in is asked to sign in first, + then taken where the link pointed. Renaming the group breaks these links. +- **Copy link** gives you that address for one file or folder: right-click it, + or tick it and use the link button in the toolbar (the way on a phone). Music + has it in a track's menu (**⋯** on a phone), Photos when you right-click a + photo or in the photo viewer's bar, and the video player and file preview + have a link button next to Download. Search offers the same, pointing at the + group each result comes from. ### Chat @@ -352,7 +365,9 @@ An album browser and a player, over the folders the operator chose for it. ### Photos An album browser over the folders the operator chose for it, where **an album -is a folder**. Thumbnails come from the node, already rotated correctly. +is a folder**. Thumbnails come from the node, already rotated correctly. A +photo opens full size, with a slideshow button that moves on every five +seconds and stops at the album's last photo. **Location data is never shown.** Photos shows when a picture was taken and what took it, and no coordinates anywhere. Worth knowing, though: the @@ -392,9 +407,32 @@ resume. ### Casting to a TV -**Desktop application only.** A film playing in the application can be sent to a -cast-capable TV or dongle on the same network: *Cast to device* in the player, -pick one from the list. +**The desktop and Android applications.** Films, photos and music can be sent +to a cast-capable TV or dongle on the same network. The cast button is in the +Videos and Music toolbars, at the top of Photos, in an album's bar, in the photo +viewer and in the music bar — in a group and in Search alike — and *Cast to +device* is in the video player. Devices appear as +they answer, usually within a couple of seconds; the search carries on for a +few more, for a TV that is still waking up. + +**A TV, once chosen, stays chosen** until *Stop casting*, from any cast button +or from the player. Every film opened plays on it, and the player opens as its +remote: the position shown is the TV's, with play/pause, 30-second jumps and a +slider to go anywhere in the film. Every photo opened in the viewer is shown on +it, and a slideshow pages the TV along with the screen. Choosing a TV from an +album's bar opens its first photo. Music plays on it too, with the cover, title +and artist on the screen: the music bar becomes its remote — play, pause, the +slider, previous and next — and the queue moves on when the TV reaches the end +of a track. Choosing the TV in the middle of a song carries it on there; *Stop +casting* carries it on here. On a phone, a film goes on playing silently on the +phone while the TV shows it, and the screen can be turned off, as it can while +music plays on the TV. + +Photos are sent scaled to the TV's screen, upright, as an ordinary picture. +The menus, covers and summaries stay on your phone or computer: the TV shows +what is playing, not the application. It plays it in the TV's standard cast +player, which shows its own name, *Default Media Receiver*, on the screen; a +player of MeshBay's own is not built. The application decrypts the film and relays it to the TV itself, over your LAN. The TV is not a group member and holds no key — which is also why the relay's @@ -435,6 +473,11 @@ list you build yourself with invitation codes. Two things follow from that: | **The CLI**, over SSH | no browser needed, and it works while the daemon is stopped | | **A paired browser** | the group's Settings and Members tabs, and the Node page in the desktop application | +Sharing is decided at the node. Adding a directory, and switching it read-write +or removable, work only on the machine running the node — the desktop +application, or `meshbay-node root`. From any other browser the group's +Settings tab still lists the directories and their switches, read-only. + On a headless server the CLI is the only path, and it covers everything you need to run a group. One gap is known: removing **one** device of one member is only doable from the interface (§7). Start with: @@ -960,12 +1003,18 @@ Better to know now than to go looking for it: is nothing to check a download against and no updates through your distribution. Until that ships, take them from the download page and from nowhere else. -- **There is no Android client.** A phone browser works. +- **The Android application is not released yet.** It works — groups, chat, + films, downloads, casting — but is built and installed by hand and signed + with a development key, so there is no store page and no update channel. The + back button and the lock screen do not drive it yet. Music keeps playing + with the screen off, from one track to the next, with a notification shown + while it plays. A phone browser works too. - **A node cannot be hosted on Android**, and is not planned to be. - **Hubs do not talk to each other yet.** Everyone in a group needs an account on the same hub. -- **Casting reaches Chromecast devices** from the desktop application. Support - for other TV protocols is designed but not built. +- **Casting reaches Chromecast devices** from the desktop and Android + applications, for films, photos and music. Support for other TV protocols is + designed but not built. - **Some subtitle tracks cannot be shown** — the ones stored as images rather than text, roughly one embedded track in five. Displaying them would need text recognition. diff --git a/docs/generate_http_api.py b/docs/generate_http_api.py new file mode 100644 index 0000000..d064519 --- /dev/null +++ b/docs/generate_http_api.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +""" +Write docs/MESHBAY_HTTP_API.md from the routes of the hub and of the node's +control API. + + python docs/generate_http_api.py + +Generated rather than written: a list of a hundred routes kept by hand is wrong +by the next one added. `test_http_api_doc.py` fails when the file and the code +disagree, and when a route has no docstring to describe it. +""" + +import inspect +import re +import sys +from pathlib import Path + +from fastapi.routing import APIRoute, APIWebSocketRoute + +OUT = Path(__file__).resolve().parent / "MESHBAY_HTTP_API.md" + +# The hub's routers, by module, in the order the hub includes them. +SECTIONS = { + "hub": "Instance", + "users": "Accounts", + "nodes": "Nodes", + "groups": "Groups", + "invite_links": "Invitation links", + "revocation": "Revocation and the node socket", + "moderation": "Moderation", + "federation": "Federation (MHP)", + "health": "Health", + "signaling": "Signaling", + "admin": "Administration", + "notifications": "Notifications", +} + +# Strongest first: a route is labelled by the first dependency it carries. +AUTH = [ + ("require_admin", "admin"), + ("require_moderator", "moderator"), + ("require_node_scope", "node"), + ("require_user_scope", "user"), + ("get_current_user", "user or node"), + ("_decode_token", "token"), + ("_federation_open", "peer hub"), +] + +AUTH_LEGEND = """\ +| Auth | What the request carries | +|---|---| +| none | No session. Any credential is in the request itself, as the route says | +| user | `Authorization: Bearer`, a person's session. A node's token is refused | +| user or node | A person's session, or a node daemon's token from `/v1/nodes/auth` | +| node | A node daemon's token only | +| moderator | A person's session, for an account with the moderator or admin role | +| admin | A person's session, for an account with the admin role | +| peer hub | Another hub's MHP token. Every route answers 503 unless federation is enabled | +""" + +HEADER = """\ +# MeshBay HTTP API + +> **Generated** by `docs/generate_http_api.py` from the routes themselves. Do not +> edit this file: change the route's docstring and run the script again. A test +> fails when the two disagree. + +MeshBay has two HTTP APIs: the **hub's**, for accounts, groups and signaling, and +the **node's control API**, which only its own machine reaches. Files, the index +and chat do not use either: they travel between a client and a node over MNP +(`MESHBAY_NODE_PROTOCOL.md`). Why each API is shaped as it is, who may call +what and what the hub must never see are in `MESHBAY_DESIGN.md`; this file lists +what exists. + +`examples/` has small Python programs that use both. +""" + +HUB_INTRO = """\ + +## Hub + +Under the hub's address, `https://meshbay.org` on the reference deployment. The +hub also serves the web application at `/` and `/app/`, which are not listed. + +""" + +NODE_INTRO = """\ +## Node control API + +`http://127.0.0.1:<ui_port>`, port 18000 unless `ui_port` in `node.toml` says +otherwise, and never on another address. Every request carries the token the +daemon writes to `<data_dir>/ui-token` (`~/.local/share/meshbay/ui-token` on +Linux), as the `X-MeshBay-Token` header or the `t` query parameter. The daemon +draws a new one at each start and deletes the file when it stops. + +| Method | Path | What | +|---|---|---| +""" + + +def flatten(routes): + for r in routes: + if hasattr(r, "original_router"): + yield from flatten(r.original_router.routes) + elif isinstance(r, (APIRoute, APIWebSocketRoute)): + yield r + + +def summary(route) -> str: + """The docstring's first sentence.""" + doc = inspect.getdoc(route.endpoint) or "" + first = " ".join(doc.split("\n\n")[0].split()) + return re.split(r"(?<=[.!?])\s+(?=[A-Z`])", first)[0].replace("|", "\\|") + + +def method(route) -> str: + if isinstance(route, APIWebSocketRoute): + return "WS" + return ", ".join(sorted(route.methods - {"HEAD"})) + + +def auth(route) -> str: + names = set() + + def walk(dependant): + for d in dependant.dependencies: + if d.call is not None: + names.add(getattr(d.call, "__name__", "")) + walk(d) + + walk(route.dependant) + return next((label for name, label in AUTH if name in names), "none") + + +def hub_routes() -> list: + from meshbay_hub.app import create_app + return [r for r in flatten(create_app().routes) + if r.endpoint.__module__.rsplit(".", 1)[-1] != "webapp"] + + +def node_routes() -> list: + from meshbay_node.ui.app import create_ui_app + return list(flatten(create_ui_app({}).routes)) + + +def render() -> str: + out = [HEADER, HUB_INTRO, AUTH_LEGEND] + by_module: dict[str, list] = {} + for r in hub_routes(): + by_module.setdefault(r.endpoint.__module__.rsplit(".", 1)[-1], []).append(r) + for module, routes in by_module.items(): + out.append(f"\n### {SECTIONS.get(module, module.replace('_', ' ').capitalize())}\n\n") + out.append("| Method | Path | Auth | What |\n|---|---|---|---|\n") + for r in routes: + out.append(f"| {method(r)} | `{r.path}` | {auth(r)} | {summary(r)} |\n") + out.append("\n" + NODE_INTRO) + for r in node_routes(): + out.append(f"| {method(r)} | `{r.path}` | {summary(r)} |\n") + return "".join(out) + + +if __name__ == "__main__": + OUT.write_text(render(), encoding="utf-8") + print(f"wrote {OUT}", file=sys.stderr) diff --git a/docs/transfers-v1.md b/docs/transfers-v1.md index a07d7d6..b87e07c 100644 --- a/docs/transfers-v1.md +++ b/docs/transfers-v1.md @@ -607,6 +607,11 @@ signs names the outcome. `_do_transfer_limits` + `transfer_limits_ack` to the group's peers, surfaced in the group Settings tab as a section beside the scan settings. +> **Since MNP 6.0 the signed op and its ack are gone.** No client ever sent it — +> the Settings section was not built — so the value is set on the node's machine: +> `PUT /api/groups/{id}/transfer-limits` on loopback, and +> `meshbay-node transfers set`. `ops.set_transfer_limits` is unchanged. + **Absent means the default (2), not unlimited.** Deliberately unlike `member_upload`'s "absent means allowed": a group that predates the setting and came back unlimited would leave the node-wide cap as the only control, which is |