diff options
Diffstat (limited to 'docs/USERGUIDE.md')
| -rw-r--r-- | docs/USERGUIDE.md | 291 |
1 files changed, 181 insertions, 110 deletions
diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index 71b0060..a8991b1 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -141,14 +141,59 @@ The JWT payload contains: |---|---| | `iss` | Hub ID (`meshbay.org`) | | `sub` | Your `user_id` (UUID4) | -| `pk_user` | Your Ed25519 public key (base64) | | `hub_id` | `meshbay.org` | | `jti` | UUID4 — unique per token, enables revocation, prevents replay | +| `groups` | The `group_id`s you are a member of, for node-side authorization | +| `scope` | `user` for a browser, `node` for a daemon | | `iat` | Issued at (Unix timestamp) | | `exp` | Expires at (Unix timestamp, 1 hour from issue) | +The token carries **no public key of yours**. It used to carry `pk_user`, and a node +recorded that key as the uploader of a file — which meant the party issuing tokens +decided who was allowed to delete it. The hub certifies *accounts*; keys are generated +on each node and pinned there (§4). + Nodes verify this JWT locally using the hub's cached Ed25519 public key. No hub roundtrip is needed — verified at 884µs in testing. This means your files remain accessible even if the hub is temporarily unreachable. +### Deleting your account + +**Settings → Delete account.** You re-enter your passphrase: a live session may be a +borrowed laptop or a tab left open, and this cannot be undone. A hub administrator can +also delete an account, from Administration → Users. + +What deletion does: + +- Releases the username — someone else may register it afterwards +- Clears the email and password hash, and drops the node linking key +- Removes group memberships, notifications and refresh tokens +- Refuses any access token still in its hour of validity, immediately + +What deletion does **not** do: + +- **It does not touch anything on a node.** Your files stay where you uploaded them, and + so do the identity pinned in the node's roster and the keypair bundle it holds for you. + Nodes are other people's machines; the hub cannot command them. To be removed there, + ask the operator — `meshbay-node member unpin <user>` and deleting your files are + their commands to run (§4). +- **It does not erase the connection log.** IP records are kept for their legal retention + period and stay attributable; that is what they exist for. + +Deletion is refused while you still own a group. Hand the group over or delete it first — +otherwise its members would be stranded. The error names the groups blocking you. + +``` +DELETE /v1/users/me +Authorization: Bearer <access_token> +{"auth_key": "<derived from your passphrase, as at login>"} +→ 200 {"status": "deleted", "username": "alice_test"} +→ 403 {"detail": "Passphrase does not match"} +→ 409 {"detail": "This account still owns groups: ..."} +``` + +Node registrations are removed as well, so a deleted operator's nodes stop being +announced. The daemons keep running and keep their data — again, the hub does not +command them. + --- ## 3. Groups @@ -226,6 +271,35 @@ Rotating the GEK (`meshbay-node gek-init`) makes the node encrypt new content wi A former member can still decrypt content they already received: there is no retroactive re-encryption, and there is no way to reach into someone's disk. Revocation controls what happens next, not what already happened. +### Notifications + +The bell in the top bar counts what you have not read. Clicking an entry takes you to +what it is about and dismisses it. + +- **Chat is one entry per group, not one per message.** A conversation that has been busy + all afternoon is a single line whose date moves to the last thing said and which turns + unread again each time. Opening the group clears it. +- **You are never notified of your own messages.** The node names the author when it + tells the hub a message was posted, and the hub skips them. +- **An invitation disappears once you have joined**, i.e. after you enter the pairing + code — not when you first look at it. +- **Muting a group works from anywhere.** The setting lives on the hub with your + membership, so a muted group creates no notification at all rather than hiding one + after the fact. It follows you to another browser. (It used to be a checkbox in the + browser's local storage that nothing read, so it did nothing.) +- **Clear all** empties the list in one action. + +``` +GET /v1/notifications → {"notifications": [{id, kind, group_id, title, link, read, created_at}], "unread": 3} +POST /v1/notifications/{id}/read → mark one read +POST /v1/notifications/read-all → mark every one read +DELETE /v1/notifications → delete them all +POST /v1/groups/{group_id}/mute {"muted": true} +``` + +`GET /v1/groups/mine` reports `muted` for each group, so the browser shows the checkbox +in the state the hub actually holds. + --- ## 4. Setting up a Node @@ -386,6 +460,25 @@ The node detects type by file extension and MIME sniffing: | `archive` | zip, tar, gz, bz2, xz, 7z | | `other` | everything else | +### Where uploaded files land + +Everything a member sends arrives in **`shared_dir/uploads/`** — both files uploaded from +the Files panel and attachments sent in the chat. One visible directory, so an operator can +look at what was sent, move it, or empty it without hunting through the tree. + +- Filenames are checked against a conservative allowlist and nothing is ever overwritten: + a colliding name gets a suffix, and the sender is told the name it was stored under. +- Chat thumbnails are scaled by the browser from the file itself. The node writes no + derived images, so nothing accumulates beside your files. +- Uploads are attributed to the identity the node pinned for that member, and that is what + decides who may delete the file later — not anything the hub says. + +### Creating a directory + +Any active member can create a directory from the Files panel (**New folder**). It is +created relative to the folder you are looking at, under `shared_dir`, and the same name +rules apply. Paths that try to leave the shared root are refused. + ### Files are stored in plaintext on disk The node holds your files in plaintext. Encryption happens at read time — the node encrypts each 1 MB chunk using a per-chunk key derived from the GEK before sending it over the wire. This means: @@ -398,134 +491,84 @@ The node holds your files in plaintext. Encryption happens at read time — the ## 6. Accessing Files -### Browse the index +**There is no HTTP file API.** Files are requested over MNP — the node's authenticated +message channel, carried by WebRTC DataChannel or QUIC — and nothing on the node answers +an unauthenticated request. The `GET /index`, `GET /file/{id}` and `GET /stream/...` +endpoints documented before 0.2.0 were removed (findings C1 and C6): they served the index +and file bytes to anyone holding a token, outside the handshake that decides what a peer +is allowed to see. Port 19001 is the MNP listener, not a web server. -``` -GET /index -Authorization: Bearer <access_token> - -→ [ - {"id": "blake3hash", "name": "filename", "size": 1234, "type": "video"}, - ... - ] -``` +The node's only HTTP surface is its admin UI, bound to loopback and requiring a token +(§4). It is for the operator, on the machine, over SSH. -```bash -curl -s -H "Authorization: Bearer $TOKEN" http://meshbay.org:19001/index | python3 -m json.tool -``` +### Browse the index -### Download a full file +After the handshake, ask for the index: ``` -GET /file/{file_id} -Authorization: Bearer <access_token> - -→ raw file bytes (Content-Type set by file type) +→ {"type": "index_sync", "v": "0.1"} +← {"type": "index_sync", "entries": [{"id": "<blake3 hex>", "name": "...", "size": 1234, + "type": "video", "path": "uploads/"}, ...]} ``` -```bash -curl -s -H "Authorization: Bearer $TOKEN" \ - http://meshbay.org:19001/file/<file_id> \ - -o output.txt -``` +For a private group the index itself is encrypted with the GEK, so a peer that never +proved possession of the key is served nothing to read. -For private groups, the returned bytes are the concatenated encrypted chunks. You must decrypt them client-side using the GEK (see the Quickstart download script for the full decryption flow). - -### Download an encrypted chunk +### Download a file, chunk by chunk ``` -GET /file/{file_id}/{chunk_index} -Authorization: Bearer <access_token> - -→ { - "ct_b64": "base64 ChaCha20-Poly1305 ciphertext", - "nonce_b64": "base64 12-byte nonce", - "file_hash_b64": "base64 blake3 of full file (HKDF salt input)", - "chunk_index": 0, - "plaintext_size": 1048576 - } +→ {"type": "file_req", "v": "0.1", "file_id": "<blake3 hex>", "chunk_index": 0} +← {"type": "file_chunk", "file_id": ..., "chunk_index": 0, + "nonce": <bytes>, "ct": <bytes>, "plaintext_size": 1048576} ``` -Chunks are 1 MB. Chunk 0 is the first megabyte. For a 5 MB file, request chunks 0–4. +Chunks are 1 MB. Chunk 0 is the first megabyte; for a 5 MB file, request 0–4. -**Per-chunk key derivation:** +**Per-chunk key derivation** — each chunk has an independent key derived from the GEK and +the chunk's position, so a leaked chunk key opens exactly one chunk of one file, and a +player can seek without decrypting from the start: ```python chunk_key = HKDF( algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + file_hash_bytes + b":chunk:" + chunk_index.to_bytes(4, "big") + info=b"file:" + file_hash_bytes + b":chunk:" + chunk_index.to_bytes(4, "big"), ).derive(gek) plaintext = ChaCha20Poly1305(chunk_key).decrypt(nonce, ciphertext, None) ``` -Each chunk has an independent key derived from the GEK and the chunk's position. This enables seeking in media files without decrypting from the start. +The browser client derives the same key the same way but uses **AES-GCM**: WebCrypto has +no ChaCha20-Poly1305. The node picks the cipher from what the peer negotiated at +handshake; the key schedule above is identical in both. -### Node info endpoint +### Identifying the node -``` -GET / -→ { - "file_count": 2, - "pk_node": "base64 Ed25519 public key", - "group_id": "uuid" - } -``` - -The node's Ed25519 public key (`pk_node`) is also what nodes use for TLS certificate pinning. Clients retrieve it from the hub via `GET /v1/nodes/{node_id}` and validate the node's self-signed TLS cert against it. +The node's `pk_node` (Ed25519) comes from the hub — `GET /v1/nodes/{node_id}` — and the +client checks the handshake signature against it. A node that cannot sign the transcript +with the key the hub published for it is refused, so hub signaling can introduce you to a +node but cannot substitute one. --- ## 7. Video Streaming -The node generates HLS (HTTP Live Streaming) segments on the fly for video files. - -### HLS endpoint +Video is streamed over the same MNP channel and played through Media Source Extensions. +The node transcodes to fragmented MP4 on the fly and encrypts each segment exactly like a +file chunk, so a standard `<video src=...>` cannot play it — the segments are ciphertext +until the client decrypts them. ``` -GET /stream/{file_id}/index.m3u8 -Authorization: Bearer <access_token> - -→ M3U8 playlist with segment URLs +→ {"type": "stream_req", "v": "0.1", "file_id": "<blake3 hex>"} +← {"type": "stream_init", "codec": "video/mp4; codecs=\"avc1.640028,mp4a.40.2\"", + "duration": 5124.3} +← {"type": "stream_data", "segment_index": 0, "nonce": <bytes>, "ct": <bytes>} +← ... one message per segment ... +← {"type": "stream_end"} ``` -Each segment is a GEK-encrypted 1 MB chunk served as `application/octet-stream`. The browser-side player must decrypt segments before handing them to the media element. - -### Play in a browser - -Because segments are encrypted, a standard `<video>` tag cannot play them directly. Use the Media Source Extensions (MSE) API with a JavaScript decryption shim: - -```html -<video id="player" controls></video> - -<script type="module"> -import { MeshBayPlayer } from "https://meshbay.org/static/player.js"; - -const player = new MeshBayPlayer({ - element: document.getElementById("player"), - nodeUrl: "http://meshbay.org:19001", - fileId: "YOUR_FILE_ID", - gek: gek_bytes, // Uint8Array(32), obtained from GEK unwrap - token: access_token, -}); -player.load(); -</script> -``` - -`player.js` handles: -- Fetching the M3U8 playlist -- Requesting encrypted chunks on demand -- Deriving per-chunk keys from the GEK -- Decrypting with WebCrypto (AES-256-GCM — see §8 for cipher choice) -- Feeding plaintext segments to MSE - -### Seeking - -Because each chunk uses an independently derived key, seeking jumps directly to the target chunk without decrypting preceding chunks. The HLS playlist embeds the `chunk_index` for each segment so the player computes the correct key immediately. - -### Supported formats for streaming - -The node can serve HLS for any container it can segment at 1 MB boundaries: MP4, MKV, WebM. The browser must support the codec in the file. For broad compatibility, H.264 + AAC in MP4 is recommended. +Each `stream_data` segment is decrypted with the chunk key for its `segment_index` and +appended to a `SourceBuffer`. The web client does this in `static/app.js` (`VideoPlayer`); +ffmpeg must be installed on the node for transcoding. --- @@ -732,7 +775,8 @@ Usually a malformed request body. Check that: ## 11. API Reference -All hub endpoints are under `https://meshbay.org`. Node endpoints are under `http://<node-ip>:<port>`. +All hub endpoints are under `https://meshbay.org`. **Nodes have no public HTTP API** — +they speak MNP (§6), and their only HTTP surface is the operator's admin UI on loopback. ### Hub API @@ -750,7 +794,9 @@ All hub endpoints are under `https://meshbay.org`. Node endpoints are under `htt | POST | `/v1/users/register` | None | Register account. Body: `username, email, auth_key`. No keys — identity keypairs are per node. Returns `user_id`. | | POST | `/v1/users/login` | None | Authenticate. Body: `username, password`. Returns `access_token, refresh_token`. | | POST | `/v1/users/token/refresh` | Refresh token | Issue new access token. Body: `refresh_token`. | -| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for any user (used by group admins for GEK wrapping). | +| GET | `/v1/users/{username}/pubkeys` | Access token | Returns `user_id`, `username` and `pk_node_ed25519` only. It no longer returns identity keys: wrapping the group key for whatever this endpoint answered was finding H3, and the node wraps it now (§3). | +| DELETE | `/v1/users/me` | Access token | Delete your own account. Body: `auth_key` — the passphrase is re-checked. `409` if you still own groups. | +| DELETE | `/v1/admin/users/{user_id}` | Access token (hub admin) | Delete someone else's account. Same tombstone, same refusal if they own groups. | **Node management** @@ -768,12 +814,20 @@ All hub endpoints are under `https://meshbay.org`. Node endpoints are under `htt | GET | `/v1/groups/{group_id}` | None / Access token | Group metadata. | | DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke and delete group. | -**GEK distribution (private groups)** +**GEK distribution — removed.** The hub used to carry wrapped group keys between members. +It does not any more: the node holds the GEK and wraps it itself, for a key the recipient +proved possession of over an authenticated channel (§3, and `docs/invite-pairing-v1.md`). +There is no hub endpoint that touches group key material. + +**Notifications** | Method | Path | Auth | Description | |---|---|---|---| -| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload GEK bundle for a member. Body: `pk_eph_b64, nonce_b64, wrapped_b64`. | -| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve your own GEK bundle. Returns `pk_eph_b64, nonce_b64, wrapped_b64`. | +| GET | `/v1/notifications` | Access token | Your notifications, newest first, plus an `unread` count. Chat is one entry per group. | +| POST | `/v1/notifications/{id}/read` | Access token | Mark one read. | +| POST | `/v1/notifications/read-all` | Access token | Mark all read. | +| DELETE | `/v1/notifications` | Access token | Delete all of yours. | +| POST | `/v1/groups/{group_id}/mute` | Access token (member) | Body: `muted`. A muted group creates no notifications at all. | **Revocation** @@ -785,18 +839,35 @@ All hub endpoints are under `https://meshbay.org`. Node endpoints are under `htt ### Node API -| Method | Path | Auth | Description | -|---|---|---|---| -| GET | `/` | None | Node info: `file_count, pk_node, group_id`. | -| GET | `/index` | Bearer token | Group file index. Returns array of `{id, name, size, type}`. | -| GET | `/file/{file_id}` | Bearer token | Download full file (encrypted bytes for private groups). | -| GET | `/file/{file_id}/{chunk_index}` | Bearer token | Download single 1 MB encrypted chunk as JSON: `ct_b64, nonce_b64, file_hash_b64, chunk_index, plaintext_size`. | -| GET | `/stream/{file_id}/index.m3u8` | Bearer token | HLS playlist for video streaming. | +The unauthenticated HTTP API (`/`, `/index`, `/file/{id}`, `/stream/...`) was **removed in +0.2.0**, findings C1 and C6. Everything a member does now goes through MNP after a +handshake that establishes what they are allowed to see: + +| Client → node | Node → client | Purpose | +|---|---|---| +| `handshake` | `handshake_ack` | Token, group, and the node's signature over the transcript | +| `handshake_challenge` (node first) | `handshake_response` | Proof the peer holds the GEK | +| `join_request` | `join_result` | Pairing code, or recognition of a pinned identity; carries the wrapped GEK on success | +| `index_sync` | `index_sync` / `index_delta` | The group index | +| `file_req` | `file_chunk` | One encrypted 1 MB chunk | +| `file_upload` | `file_upload_ack` | Upload into `uploads/`; the ack names the file as stored | +| `dir_create` | `dir_create_ack` | Create a directory | +| `file_delete` | `file_delete_ack` | Delete a file you uploaded | +| `stream_req` | `stream_init`, `stream_data`, `stream_end` | MSE video | +| `chat_msg`, `chat_hist` | `chat_hist_resp` | Chat and its history | +| `gek_bundle_fetch` | `gek_bundle_resp` | Your own wrapped group key | +| `keypair_bundle_fetch` / `_store` / `_delete` | `keypair_bundle_resp` | Your encrypted keypair backup on that node | + +**Auth:** the JWT is verified offline against the hub's Ed25519 public key — the hub is not +contacted (~884µs). The token proves which account you are; it does not decide what the +node serves you. That is the node's roster and the GEK proof. -**Auth:** all node endpoints that require a Bearer token verify the JWT offline using the hub's Ed25519 public key. The hub is not contacted during verification (~884µs). +**The node's admin UI** (`/api/*`, loopback, token required) is the operator's, not a +member API. --- -*MeshBay protocol: MNP v0.1 over TCP+TLS 1.3. QUIC transport planned for v2.* +*MeshBay protocol: MNP v0.1 over WebRTC DataChannel (browsers) and QUIC (native). The +TCP+TLS transport was removed in 0.2.0 — finding C6.* *Hub: https://meshbay.org — FastAPI + PostgreSQL + Caddy.* *Packages: python3-meshbay-common, meshbay-hub, meshbay-node.* |