From 50ebb4f2e620dad8e1fbca8307b97c5e10e7e6c0 Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Tue, 18 Aug 2026 02:14:39 +0200 Subject: docs: settle the desktop client, and draft v6 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A design discussion on 2026-08-17 settled Phase 13 and, in doing so, changed four things the spec states. v6 restates only those; v5 stays authoritative for everything it does not touch, per the convention v5 itself used with v4. What changed: * The native shell is **Electron**, not pywebview — structural decision 18 reversed. The SPA depends on Chromium-class APIs (WebRTC, WebCrypto X25519/Ed25519, MSE, Service Workers), so keeping Chromium keeps transport.js, crypto.js, keyderive.js, downloads.js and sw.js *as the client*. A system webview meant reimplementing ~2500-3000 lines. The old "69 % reused" figure was measured against an app.js of ~2600 lines; it is 4586. * A group's content is **several named roots**, not one directory, because the planned video and audio libraries will not live in one folder on one disk. * **Device linking**: one person may hold several devices on a node, admitted by a key the node already pinned and bound by a one-time code the new device generates. Without it a native client is refused where a browser is not, and an account created natively could never be opened in a browser. * **Authorship is authenticated, not asserted** — chat senders sign, uploads have a provable owner, and delete authorization moves from the uploading key to the account. And one rule v5 assumed without writing down: **group-related server state lives on the node.** Verified for multi-root — SwarmSource carries hashes and endpoints, no paths. Also here: the Caddy configuration, which was a snippet in the roadmap that would have broken the SPA (it predates /a// asset versioning and would have 404ed /sw.js, silently killing streamed downloads on Firefox and Safari); and downloads.html, which becomes a security page once a release key exists. Phase 15 was re-read against device linking and is wrong as written: a sender key must be per **device**, never per person, or two devices sharing a chain produce key and nonce reuse — C1 again, one level down. senderkeys.py already fails this silently. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 99 ++- devel-phases-next.md | 414 +++++++++---- docs/desktop-client-v1.md | 1087 +++++++++++++++++++++++++++++++++ docs/meshbay-draft-v5.md | 46 +- docs/meshbay-draft-v6.md | 270 ++++++++ packaging/caddy/meshbay.org.Caddyfile | 64 ++ second-review.md | 2 +- site/about.html | 2 +- site/downloads.html | 83 ++- site/index.html | 2 +- tmp-decisions.md | 24 +- 11 files changed, 1946 insertions(+), 147 deletions(-) create mode 100644 docs/desktop-client-v1.md create mode 100644 docs/meshbay-draft-v6.md create mode 100644 packaging/caddy/meshbay.org.Caddyfile diff --git a/CLAUDE.md b/CLAUDE.md index 76a7aa6..fcdb28a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,7 +3,9 @@ ## What this project is MeshBay is a decentralized peer-to-peer platform for file sharing, video streaming, and group messaging. -See `docs/meshbay-draft-v5.md` for the architecture specification (v3/v4 superseded). +See `docs/meshbay-draft-v6.md` for the architecture specification. v6 restates only +what changed on 2026-08-17; `docs/meshbay-draft-v5.md` remains authoritative for +everything v6 does not touch (v3/v4 superseded). ## Repository structure @@ -127,7 +129,13 @@ Scope: `hub`, `node`, `common`, or omitted for cross-cutting warned about at startup, never obeyed. Never auto-pin again, and never resolve the operator's key through the hub - **NS5** DTLS channel binding in GEK-HMAC — `HMAC(GEK, nonce || offer_fp || answer_fp)` detects WebRTC signaling MitM ✅ DONE -- **NS6** Chat `sender_id` enforced from authenticated session — prevents impersonation ✅ DONE +- **NS6** Chat `sender_id` enforced from authenticated session — prevents impersonation ✅ DONE. + **Superseded as sufficient (2026-08-17):** that is the node's word. Messages must be + **signed with the sender's device key**, and clients pin `account → device keys`. + Likewise `_admin_exec_file_delete` authorizes against `entry.uploader_pk` — the exact + uploading key — which **device linking breaks**: it must become any non-revoked device + of `uploader_id`, resolved through the roster (never through a token claim). + See `docs/desktop-client-v1.md` §4.8 - **NS7** Node Ed25519 auth — node daemon authenticates to hub via `POST /v1/nodes/auth` (Ed25519 signed timestamp), no auth_key/password on node. JWT `scope: "node"` blocks group management (create/add/delete/join). Operator manages groups from browser only. ✅ DONE - **NS8** GEK-required enforcement — node REFUSES connections when GEK is None (no `gek_required: false` bypass). GEK initialization via node local admin UI only. ✅ DONE @@ -238,6 +246,83 @@ keypair bundle, or anything that looks like a user's public key. the group named in the invitation (an operator pairs node-wide while opening a group) +## Desktop client — decided, not built (2026-08-17) + +See `docs/desktop-client-v1.md`. Nothing here is implemented; it is the design and the +decision record for Phase 13. Read it before touching the roster, registration, or +anything that assumes one key per person. + +- **Electron**, not pywebview — structural decision 18 is reversed. The SPA depends on + Chromium-class APIs (WebRTC, WebCrypto X25519/Ed25519, MSE, Service Workers), so + keeping Chromium keeps `transport.js`, `crypto.js`, `keyderive.js`, `downloads.js` and + `sw.js` **as the client**. They are no longer on the "delete once native" list. A + Python sidecar reusing `quic_client.py` covers hub-less `group://` only +- **UI assets ship inside the package**, unchanged and non-negotiable. A shell pointing + at the hub's `/app/` fixes nothing +- **Device linking**: `identities.user_id` is a PRIMARY KEY and `pin_identity` does + `INSERT OR REPLACE` — one key per person per node, silently overwritten. Both must + change. A second device is admitted by the **already-pinned key countersigning**, + bound by a one-time code the new device generates; the hub holds no user keys and so + cannot produce that signature. Never make the approval a human comparing digits — + that is the safety-number ritual 12.1 was abandoned for +- **C4 is not fully closed by going native.** It closes for a native device + unconditionally, and stays open for any account that also uses a browser, which needs + a bundle on each node. An account is only as strong as its weakest client +- **`gek_rotate` may become a signed MNP op** — the C5b rule forbids *key material + arriving from outside*, not an operator-signed instruction where the node generates + the key itself. The initial `gek-init` stays local: with no GEK there is no session +- **Installation places files, never secrets.** No key generation in `%post`/`postinst` + or an MSI custom action — a golden image would give every machine the same key +- **A group has several named roots, not one `shared_dir`.** The name is the chosen + directory's **basename**, derived once at add time and *stored* — recomputing it from + the path re-identifies a whole library the day someone renames a folder. Duplicates + refused case-insensitively, no root nested in another, one operator-designated upload + target, availability per root, and `kind` + `layout` reserved for the planned + video/audio libraries. `config.py:103` is the single string this replaces +- **The planned video/audio libraries are VIEWS over the file index, not a catalogue.** + No metadata store, no server-side database, ever, and nothing reaching the hub — it + keeps no file names for private groups (H7). A file stays tied to its representation on + the filesystem: folders are the categories, and moving a file makes it a different + file. Everything a view needs already exists (whole-group index cached client-side, + 10b.5/10b.6). The only non-free piece is a video thumbnail +- **Enrichment happens on the client; what it cannot compute, the node produces on demand + and the asking device caches.** Neither node nor hub keeps durable derived state. This + is already the rule for chat thumbnails (draft-v5 §5.2) and it is the answer for video + thumbnails too — a frame grab is strictly less than the decoding the node already does + for streaming, over the same authorized path +- **A root that goes away must freeze, not empty.** `indexer.py` runs a watchdog + `Observer` and rebuilds on any change; unmounting a USB drive either emits deletions + for the whole tree or presents an empty directory to the next rescan. Both propagate as + though the owner erased their library. The per-root "unavailable" state ships **before** + root selection is offered +- **exFAT/NTFS and Windows are the common case, not an edge case.** Most users are + expected to share from an external exFAT or NTFS drive, on Windows, whatever the build + order says. Consequences that are correctness, not portability: filenames need NFC normalization for identity while keeping original bytes + for display; Windows reserved names and `MAX_PATH` affect what can be downloaded; + `ReadDirectoryChangesW` drops events under load, so periodic reconciliation is + mandatory. Never assume POSIX, systemd or case sensitivity. The upload no-overwrite check was *not* affected — `Path.exists()` is already case-insensitive there (checked 2026-08-18); case folding is for comparisons the code makes itself +- **Shipping the UI in a package creates version skew for the first time.** Today the SPA + and the hub deploy together, so a `/v1/` response shape and its caller change in one + commit. Once the UI is installed rather than served, `/v1/` is a compatibility surface + and `GET /v1/hub/version` needs a minimum client version — cheap now, awkward later + +- **A content-addressed index cannot represent the same bytes at two paths.** + `GroupIndex` is keyed by blake3, so `clip.mp4` at a root and in `uploads/` with + identical content is **one** entry — which is also why a scan can report ten + files and index nine. Reconciliation compares *paths*, so it decided the + second path was a missed event every 60 s, rewrote the entry, bumped the + version and pushed an index update to every connected peer. Found by watching + a live node, not by a test. Anything comparing disk against index must check + the id, not the path + +- **A CLI branch nobody has run is not covered by anything.** `reload` shipped + with `subprocess` unimported and crashed on first use; the module compiles + fine, which is the same "syntax, not names" trap already recorded for the SPA. + `test_cli_dispatch.py` walks every verb with the daemon stubbed, and refuses + to let a verb be added to the parser without an entry there. It also stubs + `os.kill` — the first version of that test SIGHUPed the developer's own + running node + ## Two lessons that cost four rounds of live testing - **`QE/deploy/e2e.py` cannot test `app.js`.** It is a second implementation of the @@ -416,7 +501,12 @@ keypair bundle, or anything that looks like a user's public key. gathering, one ISP validated. **ICE/STUN (WebRTC) is the traversal path**, for native clients too (via `aiortc` in Python) - Argon2id 256 MB was applied to the **hub only**; `crypto.py` keystore is still 64 MB -- Sender keys must be distributed **pairwise to identity keys**, never GEK-derived +- Sender keys must be distributed **pairwise to identity keys**, never GEK-derived — and + **per device, never per person** (2026-08-17). Two devices sharing one sending chain + both advance it, producing key/nonce reuse: that is C1 again, one level down. + `GroupSenderKeyStore.add_sender` currently does `self._states[dist.sender_id] = ...`, + so a second device under the same `sender_id` silently overwrites the first. Revoking a + device must rotate, like revoking a member. See `devel-phases-next.md` §15.0b - Chat is plaintext on the wire and at rest; the index is plaintext on the WebRTC path ## Known calibration TODOs @@ -521,7 +611,8 @@ SFR residential Fedora 44 → meshbay.org OVH VPS: | pkX from private key | `static/transport.js` | `_pkFromSk()` — JWK export to derive X25519 public key | | Group delete (hub) | `meshbay_hub.api.groups` | `DELETE /v1/groups/{group_id}` — admin only | | JWT scope enforcement | `meshbay_hub.api.deps` | `require_user_scope` — blocks node-scoped tokens from mutations | -| Node local admin UI | `meshbay_node.ui.app` | Dashboard, peers, groups, audit log (localhost:18000) | +| Operator operations | `meshbay_node.ops` | **One implementation, several front doors.** The loopback API, the CLI and the signed MNP handlers all call these; they take the daemon `state`, raise `OpError`, and know nothing about HTTP. Two implementations of one operation with two authorization checks is C1/C6 one size down | +| Node local admin UI | `meshbay_node.ui.app` | Dashboard, peers, groups, audit log (localhost:18000). Each operation endpoint is one `_op(...)` line — logic there is a rule the MNP path does not have | | Demo scripts | — | `QE/demo-v1/*.py`, `QE/demo-v2/*.py`, `QE/demo-v3/*.py` (not versioned) | | Video flow control (browser) | `static/app.js` | `pump()` — the only place credit is granted. Read-ahead bounded by `BUFFER_AHEAD_S` of film, `STREAM_WINDOW` segments in flight, driven by a clock and by playback, never by arriving data | | Player under test | `tests/harness/mse_harness.mjs` | Runs the real `pump`/`flushQueue`/`evictBehind` against a fake SourceBuffer with a ceiling. Do not write a second model of them | diff --git a/devel-phases-next.md b/devel-phases-next.md index db9267d..338beb2 100644 --- a/devel-phases-next.md +++ b/devel-phases-next.md @@ -1,7 +1,9 @@ # MeshBay — Next Implementation Phases > Base: Phases 1–12 complete (except 10.9 → Phase 18). Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org. Node daemon is production-ready (WebRTC, WS, chat, HTTP, index push, swarm all wired). -> Architecture reference: **docs/meshbay-draft-v5.md** (v4 superseded 2026-08-13) +> Architecture reference: **docs/meshbay-draft-v6.md** (2026-08-17; v5 remains +> authoritative for everything v6 does not restate, v4 superseded 2026-08-13). +> Desktop client, roots, device linking: **docs/desktop-client-v1.md** > First security review: first-review.md (2026-08-10) > **Second security review: second-review.md (2026-08-13) — 6 critical, 7 high findings.** > @@ -300,27 +302,28 @@ and requests to `/app/` fall through to the hub. ### Caddy integration -Recommended Caddyfile snippet for meshbay.org: -``` -meshbay.org { - root * /path/to/meshbay/site - try_files {path} {path}.html - file_server - - handle /v1/* { - reverse_proxy localhost:8000 - } - handle /app* { - reverse_proxy localhost:8000 - } - handle /style.css { - reverse_proxy localhost:8000 - } - handle /*.js { - reverse_proxy localhost:8000 - } -} -``` +**The real configuration lives at `packaging/caddy/meshbay.org.Caddyfile`** (added +2026-08-17). Use it, not the snippet this section used to carry. + +The snippet that was here served `site/` from the root with `try_files` and proxied +`/v1/*`, `/app*`, `/style.css` and `/*.js` to the hub. It predates asset versioning and +**would have broken the SPA**: the module graph is served under `/a//`, which +`handle /*.js` does not match, and neither does `/locales/*.js`. Worse, `/sw.js` would +have 404ed — the service worker has to stay at the root or its scope stops covering the +pages it intercepts downloads for, which breaks streamed downloads on Firefox and Safari +without any visible error. + +The rule is inverted: an **allowlist** of site paths served statically, everything else +proxied to the hub. The hub mounts its whole static directory at `/` (`app.py`), so it +owns the root namespace by default and the site takes only what it names. + +**Consequence to be aware of:** with the site overlay in front, `/` is the landing page, +so `webapp.py`'s `GET /` (which returns the SPA shell) is unreachable on meshbay.org. That +route stays — a **generic** hub with no site overlay should serve the application at its +root. The overlay is meshbay.org-specific by design. + +`site/` is **not** pushed by the hub deploy procedure; it syncs separately to +`/srv/meshbay/site`. ### Hub mirror (design only — implementation deferred) @@ -785,17 +788,31 @@ metadata minimization, residual schema cleanup. --- -## Phase 13 — Native desktop client (pywebview + aiortc) +## Phase 13 — Native desktop client (Electron + optional Python sidecar) -> **Status (2026-08-13): 13.1 active, 13.2–13.11 DEFERRED to after Phase 15**, pending -> decision D2 in `tmp-decisions.md` (browser extension vs native client vs both). +> **Reworked 2026-08-17 by operator decision. `docs/desktop-client-v1.md` is +> authoritative for this phase** — shell, device linking, account creation, node +> management, packaging and the open items. The milestone table below is the summary; +> the design and its rationale are there. +> +> **The shell changed: pywebview → Electron.** Structural decision 18 is reversed. The +> reason is measured, not aesthetic: the SPA depends on Chromium-class APIs (WebRTC, +> WebCrypto X25519/Ed25519, MSE, Service Workers, File System Access), so keeping +> Chromium keeps `transport.js`, `crypto.js`, `keyderive.js`, `downloads.js` and `sw.js` +> unchanged, while a system webview meant reimplementing ~2500–3000 lines plus a loopback +> media server and native dialogs. The old "69 % reused" figure was measured against an +> `app.js` of ~2600 lines; it is **4586** as of 2026-08-17, and `app.js` now holds 2 direct +> `crypto.subtle` calls — the three-globals seam had already leaked. > -> **13.1 (platform adapter split) proceeds regardless** — it is pure refactoring whose -> acceptance criterion is "the browser SPA behaves identically", and it is the prerequisite -> for every option under D2. +> **One blocking addition: device linking (13.3b).** Identity keys are per node, so a +> native client holding its own keys is refused by `webrtc_server.py:886-894` where a +> browser is not. Without linking, an account created natively could never be opened in a +> browser without an operator code per node — the "native must not prevent web use" +> objective fails outright. See `docs/desktop-client-v1.md` §4. -**Objective:** ship a desktop application with durable key storage, hub-independent -`group://` access, and a better media path than the browser allows. +**Objective:** ship a desktop application whose UI is not served by the hub, with durable +local key storage, at feature parity with the web for a standard user, that does not +prevent web use, and that can manage locally installed nodes. > ⚠️ **Do not justify this phase as "the fix for T3".** An earlier draft of > `second-review.md` claimed a native client makes code integrity independent of the hub. @@ -803,31 +820,37 @@ metadata minimization, residual schema cleanup. > operator holds relocates the trust rather than removing it. What native actually changes is > **detectability** — an attack must ship as an artifact that can be hashed and compared > instead of a one-off HTTP response — and that value is realised only by **18.7 reproducible -> builds** plus published hashes. Native also *costs* the browser sandbox, hands you patch -> velocity for WebKitGTK and every bundled dependency, and adds the loopback media server, -> the IPC bridge and the updater as new attack surface. +> builds** plus published hashes. +> +> **Cost line revised 2026-08-17.** Two of the three costs recorded here were pywebview's, +> not native's. Electron with `sandbox` and `contextIsolation` **keeps** the Chromium +> renderer sandbox — the strongest available — so "native costs the browser sandbox" is +> false for this shell, and the D2 table in `tmp-decisions.md` is wrong on that row. The +> loopback media server is deleted (13.6). What remains, and is real: **we own Chromium's +> patch cadence**, the renderer parses attacker-controlled content from nodes, npm enters +> the supply chain, and the updater is new surface. > > The security-per-effort ranking is: **11.5 ≫ 12 ≫ 14 (CLI) ≫ 13.** This phase is justified -> on product grounds. It permanently closes **C4** as a side effect, but C4 can also be closed -> in a browser by not storing keypair bundles remotely at all. +> on product grounds. It closes **C4** for native devices, but **not for accounts that also +> use a browser** — see `docs/desktop-client-v1.md` §5.1. Reproducible builds are unusually +> tractable here: the UI has no bundler or minifier, and Electron's own binaries have +> published upstream hashes. -### Why this is cheap +### Why this is cheap (re-measured 2026-08-17) -The SPA never touches a browser crypto or network primitive directly: `app.js` contains -**0** occurrences of `crypto.subtle` and **0** of `RTCPeerConnection`. All crypto and -transport go through three injected globals (`window.MeshBayCrypto`, `MeshBayKeys`, -`MeshBayTransport` — 16 call sites) and all hub I/O through one function (`hubFetch`, 30 -call sites). That is the seam. +| Asset | Lines | Under Electron | Under a system webview | +|---|---|---|---| +| `style.css`, `i18n.js`, `vendor/`, `zipstream.js`, `transfers.js` | ~4200 | reuse | reuse | +| `app.js` | 4586 | reuse, minus the hub base URL | reuse, minus storage glue + MSE player | +| `transport.js`, `crypto.js`, `keyderive.js` | 1907 | **reuse** | delete and reimplement | +| `downloads.js`, `sw.js` | 363 | **reuse** | delete | -| Asset | Lines | Native | -|---|---|---| -| `style.css`, `i18n.js`, `vendor/htm-preact.js` | 1708 | **reuse as-is** | -| `app.js` — components, routing, theme, admin | ~2050 | **reuse as-is** | -| `app.js` — storage glue, `hubFetch`, download/upload callbacks, MSE `VideoPlayer` | ~550 | rewrite | -| `transport.js`, `crypto.js`, `keyderive.js` | 1145 | **delete** | +`HUB = ''` (`app.js:12`) becoming a configurable absolute base URL is the one structural +change. That is what 13.1 exists for, and under Electron it is nearly all of it. -≈ **69 % reused unchanged**, and the 31 % that is not is largely code `second-review.md` -says to delete anyway (WebCrypto AES variant, PBKDF2 password split, keypair bundles). +The old plan also budgeted a loopback media server (WebKitGTK MSE is unreliable), a SQLite +index cache (IndexedDB is restricted under `file://`) and native file dialogs. Chromium +provides all three, so those milestones are deleted rather than rewritten. ### Non-negotiable @@ -835,30 +858,48 @@ says to delete anyway (WebCrypto AES variant, PBKDF2 password split, keypair bun `https://meshbay.org/app/` is a browser with a different icon and fixes nothing. The hub is used for the API only, and the bundle is covered by 13.9 signing. +Shell hardening is part of that and is not optional: `contextIsolation` and `sandbox` on, +`nodeIntegration` off, a custom `app://` protocol rather than `file://`, navigation to +non-local targets refused, and a strict in-package CSP that **keeps `wasm-unsafe-eval`** — +the bundle KDF is Argon2id in WebAssembly, and a policy forbidding it locks every user out +of their keys. + ### Milestones -| # | Component | Description | -|---|---|---| -| 13.1 | Platform adapter split | Extract `platform-web.js` (WebRTC/WebCrypto/fetch — today's behaviour) and `platform-native.js` (pywebview bridge). `app.js` imports neither directly. **Acceptance: the browser SPA is byte-for-byte functional after the split** — this lands first, on its own, with no native code | -| 13.2 | pywebview shell + Python bridge | `meshbay-client` package; `window.pywebview.api.*` implements the same surface as the three globals; single-instance, tray, window state | -| 13.3 | Local keystore + Ed25519 client auth — **the real close for C4**, now partially mitigated by Argon2id (draft-v5 §7.1) | Reuse `keystore.py` (Argon2id 256 MB, OS keychain later). Client authenticates like the daemon does: signed timestamp, `POST /v1/users/auth`. **No password on the wire, no `auth_key`/`bundle_key`, no keypair bundle anywhere** → closes **C4** permanently | -| 13.4 | aiortc client transport | `RTCPeerConnection` + `createDataChannel` + `createOffer` in Python; ICE/STUN via `aioice` — the traversal path validated on 2 ISPs. Calls the unified handshake from 11.5.4. QUIC (`quic_client.py`) retained as opt-in for LAN / port-forwarded / hub-less `group://` | -| 13.5 | Local index cache | SQLite in the client profile dir, replacing IndexedDB (also restricted under `file://` in some WebViews) | -| 13.6 | Loopback media server | Python decrypts and serves with HTTP Range; `