# MeshBay — Next Implementation Phases > Base: Phases 1–12 complete (except 10.9 → Phase 18). Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org. Node daemon is production-ready (WebRTC, WS, chat, HTTP, index push, swarm all wired). > Architecture reference: docs/meshbay-draft-v4.md > First security review: first-review.md (2026-08-10) > **Second security review: second-review.md (2026-08-13) — 6 critical, 7 high findings.** > > ⛔ **Phase 11.5 is BLOCKING.** No feature phase starts until C1–C6 and H1–H7 are closed. > The current build must not host real private data: the node's HTTP API serves private > group content unauthenticated (C1), any user can hijack a node's signaling identity (C2), > and an active hub can obtain any group key through the key directory it controls (H3). > > **Phases renumbered 2026-08-13** (old → new): 12→14, 13→15, 14→16, 15→17, 16→18, 17→19. > New: 11.5 (security remediation), 12 (hub minimization), 13 (native desktop client). --- ## Phase 7 — Node v2 : production, streaming, chat ✅ DONE Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests. | # | Component | Status | |---|---|---| | 7.0 | JWT group claims + node authz check | ✅ | | 7.1 | QUIC 0-RTT session resumption | ✅ | | 7.2 | Signaling `client_incoming`/`punch_ready` + jti denylist push | ✅ | | 7.3 | Multi-group daemon (1-port multiplexing) | ✅ | | 7.4 | HLS streaming via QUIC | ✅ | | 7.5 | Chat: Sender Keys protocol + storage + MNP wire | ✅ | | 7.6 | Chat: local web UI + WS push to members | ✅ | --- ## Phase 8 — Hub v2: admin, federation, security ✅ DONE Commit: 46918ec — 20 files, +508/−90 lines, 117 tests. Deployed to meshbay.org. Existing emails encrypted. DB schema migrated. | # | Component | Status | |---|---|---| | 8.1 | Admin roles — config-based `require_admin` | ✅ S1 resolved | | 8.2 | Email encrypted at rest — AES-256-GCM, HKDF | ✅ S2 resolved | | 8.3 | Refresh token rotation — family-based reuse detection | ✅ S5 resolved | | 8.4 | Federation DB persistence (HubPeer model) | ✅ | | 8.5 | Federation token verification async (DB-backed) | ✅ | | 8.6 | CSAM hash check in swarm registration | ✅ | | 8.7 | Rate limiting on auth endpoints (5/10/20 per min) | ✅ | | 8.8 | Healthcheck endpoint (GET /v1/health) | ✅ | | 8.9 | IP log cleanup background task (365-day retention) | ✅ | | 8.10 | Argon2id bumped to 256 MB (pw_version=2, rehash on login) | ✅ | --- ## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE Commit: ab4d389 — 27 files, +3053/−330 lines, 132 tests. Deployed to meshbay.org + Orange node. Tested browser → node P2P through two ISP NATs. **Objective:** a web browser can connect P2P to a node behind residential NAT, browse files, download, stream video, and chat — with zero data through the hub. **Architecture decisions (settled 2026-08-10):** ### Transport: WebRTC DataChannel for browsers Native clients (desktop, Android) use QUIC with `punch_nat()` — already validated in demo-v2 on SFR residential (Port-Restricted Cone NAT). Browsers cannot use QUIC for NAT traversal because WebTransport does not allow the browser to choose its UDP source port. Port-Restricted Cone NAT requires the client to connect from the exact port the node probed — impossible for browsers. **Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC stack handles NAT traversal automatically. The node uses `aiortc` (same author as `aioquic`, already referenced in draft-v3 as [future]). ICE is strictly superior to our custom `punch_nat()` for this use case: - Both sides send STUN binding requests simultaneously → mutual hole-punching - No need for the client to pre-announce its port - Handles both sides behind NAT - Battle-tested by billions of users (Google Meet, Discord, etc.) The MNP protocol (handshake, file_request, file_chunk, chat_message, etc.) runs identically over WebRTC DataChannel as over QUIC streams. Same E2E encryption. **Node dual transport:** - QUIC (port 19000) — native clients, already in place - WebRTC DataChannel — browsers, using `aiortc` ### Signaling: hub WebSocket relay The hub relays WebRTC signaling (SDP offer/answer, ICE candidates) between browser and node. This is the same role described in draft-v3 section 4.1.3: "NAT traversal coordination [...] stateless [...] <1 KB per message." ``` Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer {sdp, ice_candidates} Hub → Node (WS) : {type: "webrtc_offer", sdp, ice_candidates, peer_id} Node → Hub (WS) : {type: "webrtc_answer", sdp, ice_candidates, peer_id} Hub → Browser (SSE) : {sdp, ice_candidates} ``` After signaling, the DataChannel is P2P. Hub is no longer involved. ### UI: Preact SPA - **Framework:** Preact (~3 KB gzipped) + preact-router - **Build:** esbuild (single binary, no node_modules bloat) for minification - **Theming:** CSS `prefers-color-scheme` + localStorage toggle (dark/light) - **i18n:** JSON translation files loaded client-side, English default - **Responsive:** sidebar collapses to hamburger on mobile viewports - **Crypto:** existing `crypto.js` (SubtleCrypto AES-GCM) for E2E decryption ### Hub role (reminder — fundamental constraint) The hub is a registrar and signaling facilitator. It stores ONLY: - User accounts (login, encrypted email, public keys, keypair bundle) - Group metadata (name, admin, members, GEK bundles — no file indexes) - Node registrations (endpoint hints, public keys) All data (files, streams, chat messages, directory indexes) lives on mesh nodes. Clients (web or native) transfer data E2E with nodes. The hub never touches content. This is non-negotiable. ### Chat/forum storage Chat messages are stored on the node(s) hosting the group, not on the hub. The browser retrieves chat history from the node via DataChannel, same as files. If no node in the group is online, the group (including chat) is unavailable. This is inherent to the P2P model and acceptable. ### File search Content is not indexed on the hub. Search works client-side: - Node provides a Mesh Group Index (file metadata: names, paths, sizes, hashes) - For private groups, the index is GEK-encrypted — hub stores it opaque, client decrypts - Browser caches decrypted indexes in IndexedDB (~50–100 MB quota, extensible) - Search runs locally on cached indexes — instant, no network call, no hub involvement ### Milestones | # | Component | Files | Priority | |---|---|---|---| | 9.1 | **Spike: WebRTC DataChannel on node** | `aiortc` integration, 4 tests (handshake, file transfer, auth, guard) | ✅ | | 9.2 | WebRTC signaling endpoints on hub | `hub/api/signaling.py` — relay SDP/ICE, 2 tests | ✅ | | 9.3 | WebRTC→MNP transport adapter on node | `node/transport/webrtc_server.py` + hub_client WebRTC handler | ✅ | | 9.4 | `transport.js` — browser WebRTC client | `static/transport.js` — connect, handshake, fetch, msgpack | ✅ | | 9.5 | **Spike: E2E browser→NAT→node file transfer** | Mobile 4G → SFR NAT → node, IPv4 STUN + IPv6 validated | ✅ | | 9.6 | Preact SPA shell (login, routing, theme) | `static/app.js`, `static/style.css`, `static/vendor/htm-preact.js` | ✅ | | 9.7 | Group list + file explorer UI | `app.js` GroupPage, `groups.py` nodes endpoint, `revocation.py` group tracking | ✅ | | 9.8 | File download via DataChannel | AES-GCM chunks, GEK delivery, progress bar, browser download | ✅ | | 9.9 | Video streaming via DataChannel | Chunk download → Blob URL, video overlay with native controls | ✅ | | 9.10 | Chat/forum UI via DataChannel | ChatPanel component, chat history MNP, peer broadcast, tabs UI | ✅ | | 9.11 | i18n framework + English strings | `static/i18n.js` — t() lookup, ESM, localStorage lang, all strings extracted | ✅ | | 9.12 | Settings UI (profile, theme, language) | SettingsPage component, system theme support, sidebar link | ✅ | | 9.13 | Tests: unit + integration | WebRTC transport, MNP over DataChannel | ✅ | | 9.14 | Performance: pipelined download | sliding window (8 concurrent chunks) | ✅ | | 9.15 | Performance: binary wire format | raw bytes via msgpack, no base64 (+33%) | ✅ | | 9.16 | Performance: avoid redundant I/O | file_hash from index, not re-read per chunk | ✅ | | 9.17 | Large file download to disk | File System Access API (`showSaveFilePicker`) | ✅ | **Critical path validated (2026-08-10):** 9.1 → 9.5 all pass. WebRTC DataChannel works browser → node through two different ISP residential NATs: **SFR residential NAT** (mobile 4G → node behind SFR Port-Restricted Cone + CGNAT): | Test | ICE path | Result | |---|---|---| | WiFi LAN (same network) | IPv6 direct | OK, ~100ms | | Mobile 4G SFR + IPv6 | IPv6 inter-network | OK, ~600ms | | Mobile 4G SFR + IPv4 only | STUN hole-punch IPv4 | OK, ~650ms | **Orange Livebox NAT** (laptop browser → node behind Orange residential NAT, cross-site): | Test | ICE path | Result | |---|---|---| | Chrome laptop → Orange node | IPv6 inter-network | OK, ~7000ms | | Firefox laptop → Orange node | IPv6 inter-network | OK, ~6700ms | | Firefox laptop → Orange node (IPv6 disabled) | STUN hole-punch IPv4 | OK, ~6900ms | Two ISPs validated, both Chrome and Firefox. No TURN relay needed. ICE/STUN handles all tested NAT types automatically. **Performance optimizations (2026-08-11):** - Initial transfer speed: ~2 MB/s (sequential, base64, redundant I/O) - After file_hash fix (9.16): ~3 MB/s (eliminated 78 GB redundant reads on 279 MB file) - After pipelining (9.14): ~5 MB/s (8-chunk sliding window, concurrent requests) - After binary wire format (9.15): eliminated 33% base64 inflation + removed redundant per-chunk fields (sig, hashes, pk_node) — AES-GCM tag already authenticates ciphertext, DTLS authenticates transport - Large file support (9.17): `showSaveFilePicker` (Chrome/Edge) streams decrypted chunks directly to disk — flat ~8 MB RAM regardless of file size. Firefox/Safari fall back to Blob-in-RAM approach. **Indexer debounce (2026-08-11):** - File copy triggers multiple watchdog events at different file sizes → duplicate index entries with different blake3 hashes. Fixed with 2-second debounce + path-based dedup (remove old entry before adding new). **Known remaining items for future phases:** - ~~True video streaming (MSE or Service Worker)~~ → Phase 10c (2026-08-11) - Multiple shared directories per node (UI + config) - Multi-node per user support **Dependencies added:** - `aiortc>=1.9` in `meshbay-node/pyproject.toml` ✅ - `esbuild` as a dev tool (single binary, not npm) — needed for 9.6+ - `preact` + `preact-router` (ESM imports, no npm needed — CDN or vendored) --- ## Phase 10 — meshbay.org site + admin/moderation UI Commit: 8fa298e (10.1–10.4), 022da76 (10.5–10.10) — 155 tests. **Objective:** meshbay.org becomes both a production hub and the project's public website, with admin/moderation interfaces and user-facing features. ### Site architecture Two layers, cleanly separated: - **Generic hub** (API + web app) — reusable by any hub operator - **Site overlay** — meshbay.org-specific pages (landing, /downloads, /about) The site overlay is served by Caddy (static files) with priority over the hub. The hub serves the SPA for authenticated users at `/app/`. ``` site/ # meshbay.org-specific (not in generic hub package) ├── index.html # Landing page — project promotion ├── downloads.html # Package repos (placeholder, Phase 13) ├── about.html # Project info, GitHub link, contact └── assets/ └── site.css # Landing page styles (dark/light aware) ``` ### User roles | Role | Capabilities | |---|---| | `user` | Standard user — browse, download, chat, manage own profile | | `moderator` | Review reports, suspend content/groups/users | | `admin` | All moderator rights + hub management (same as moderator for now, distinction reserved for future federation/mirror) | Role stored as `role` column on User model (`user` | `moderator` | `admin`). `require_moderator` dependency (checks role ≥ moderator OR config allowlist). `require_admin` checks role = admin OR config allowlist (backward compat). Config-listed admin usernames are synced to `role = "admin"` in DB at startup. ### Milestones | # | Component | Status | |---|---|---| | 10.1 | Landing page + /downloads + /about | ✅ | | 10.2 | Moderator role + `require_moderator` dependency + admin API | ✅ | | 10.3 | Moderation UI (user/group suspend, blocklist management) | ✅ | | 10.4 | Admin UI (stats, user list, group list, audit logs viewer, blocklist) | ✅ | | 10.5 | Notification system (invitations, role changes, account status) | ✅ | | 10.6 | User settings (profile, role display, per-group notification mute) | ✅ | | 10.7 | Public group search (name keyword filtering) | ✅ | | 10.8 | Front page (notification feed with unread badge) | ✅ | | 10.9 | Package repositories (APT/DNF) | Deferred to Phase 13 | | 10.10 | Auto-update check endpoint (`GET /v1/hub/version`) | ✅ | ### API endpoints (10.2, 10.5, 10.7, 10.10) | Method | Path | Auth | Description | |---|---|---|---| | GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) | | GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) | | GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable) | | GET | `/v1/admin/users/{id}` | Moderator+ | User detail (email decrypted, group count) | | PATCH | `/v1/admin/users/{id}` | Moderator+ | Update role or status (triggers notification) | | GET | `/v1/admin/groups` | Moderator+ | List groups (with member count) | | PATCH | `/v1/admin/groups/{id}` | Moderator+ | Update group status | | GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user) | | GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) | | POST | `/v1/notifications/{id}/read` | Access token | Mark single notification read | | POST | `/v1/notifications/read-all` | Access token | Mark all notifications read | | GET | `/v1/groups?q=` | None | Search public groups by name | | GET | `/v1/hub/version` | None | Version check (hub, MNP, MHP versions) | ### Admin UI (10.3–10.4) Admin page at `#/admin` in SPA, accessible to moderators and admins. Five tabs: Stats, Users, Groups, Logs, Blocklist. - **Stats:** card grid (users, groups, nodes, online nodes) - **Users:** searchable table, inline role dropdown, suspend/unsuspend buttons, detail overlay - **Groups:** table with member count, suspend/unsuspend - **Logs:** filterable IP audit log table, paginated (50/page, load more) - **Blocklist:** existing `/v1/admin/blocklist` endpoints, add/remove hashes ### SPA route change SPA now also served at `/app/` and `/app/{path}` (in addition to `/`). With Caddy site overlay, Caddy serves `site/index.html` at `/`, and requests to `/app/` fall through to the hub. ### Caddy integration Recommended Caddyfile snippet for meshbay.org: ``` meshbay.org { root * /path/to/meshbay/site try_files {path} {path}.html file_server handle /v1/* { reverse_proxy localhost:8000 } handle /app* { reverse_proxy localhost:8000 } handle /style.css { reverse_proxy localhost:8000 } handle /*.js { reverse_proxy localhost:8000 } } ``` ### Hub mirror (design only — implementation deferred) A mirror hub is a complete replica of the primary hub (same user DB, same groups, same GEK bundles, same storage). Purpose: load distribution via DNS round-robin. **Design constraints:** - Shared Ed25519 private key (transferred once at setup, securely) - PostgreSQL logical replication for active-active read/write on both mirrors - Both mirrors can issue JWTs (same signing key) - DNS round-robin (2+ A records on meshbay.org) - If one mirror goes down, the other continues serving **Not implemented now.** The design must not prevent future implementation: - Hub config and private key paths must be externalizable - No hub-specific state that can't be replicated - JWT verification must not depend on hub-local state --- ## Phase 10b — Self-service UI + client-side features Pending commit — 166 tests. **Objective:** make the web SPA fully self-service — users can create groups, manage members, join open groups, upload files, and search across all cached group file indexes. No admin intervention needed for basic operations. ### Self-service features | # | Component | Status | |---|---|---| | 10b.1 | Group creation UI (CreateGroupPage) | ✅ | | 10b.2 | Member management + invite (MembersPanel) | ✅ | | 10b.3 | Group join flow (open groups self-join) | ✅ | | 10b.4 | File upload (client → node via MNP FILE_UPLOAD) | ✅ | | 10b.5 | IndexedDB caching (group file indexes cached locally) | ✅ | | 10b.6 | Cross-group file search (SearchPage — client-side, no hub) | ✅ | ### New API endpoints (10b.1–10b.3) | Method | Path | Auth | Description | |---|---|---|---| | POST | `/v1/groups` | Access token | Create a new group (name, visibility, join_policy) | | GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) | | POST | `/v1/groups/{id}/join` | Access token | Self-join open group (checks join_policy) | | POST | `/v1/groups/{id}/members/{username}/gek` | Access token | Store GEK bundle for invitee | | GET | `/v1/groups/{id}/gek` | Access token | Get own GEK bundle (for wrapping) | ### New MNP message types (10b.4) | Type | Direction | Description | |---|---|---| | `file_upload` | client → node | Push encrypted file chunk (filename, chunk_index, total_chunks, data) | | `file_upload_ack` | node → client | Acknowledge chunk receipt | Node stores uploads in `shared_root/.uploads/` as `.part` files during transfer, renames to final location on last chunk. Filename sanitized (no path traversal). ### Browser crypto additions (10b.2) AES-256-GCM ECIES variant for GEK wrapping in browsers. WebCrypto does not support ChaCha20-Poly1305, so a parallel ECIES scheme uses AES-256-GCM with a distinct HKDF info string (`meshbay:gek_wrap:v1:aes` vs `meshbay:gek_wrap:v1`). Both Python and browser implement the AES variant for interop. Functions added to `crypto.js`: `generateGEK()`, `wrapGEK()`, `unwrapGEK()`, `encryptChunk()`, `b64encode()`. Functions added to `crypto.py`: `wrap_gek_aes()`, `unwrap_gek_aes()`. ### IndexedDB caching (10b.5) When a group's file index is fetched from a node, it is cached in IndexedDB (`meshbay` database, `group_indexes` store). On subsequent visits, cached entries are shown immediately while the live connection is established. This gives instant file list display even before WebRTC connects. Cache key: `groupId`. Stored: `{ groupId, groupName, entries[], cachedAt }`. Best-effort — failures are silently ignored. ### Cross-group file search (10b.6) SearchPage component at `#/search`. Searches file names and paths across ALL cached group indexes in IndexedDB. Pure client-side — no hub involvement. Results link back to the group page. Accessible from sidebar. ### Tests added - 8 tests: group self-service (create, join open, join invite rejected, join already member, members list, non-member denied, search, join triggers notification) - 3 tests: AES GEK wrap/unwrap (round-trip, wrong key rejected, differs from ChaCha20 wrap) --- ## Phase 10c — MSE video streaming (real-time playback) Pending commit — 167 tests. **Objective:** replace the download-then-play video player with real-time MSE (MediaSource Extensions) streaming. Playback starts within seconds instead of waiting for the full file download. ### Architecture ``` Browser Node │ │ ├── stream_req {file_id} ──────►│ │ ├── ffprobe → codec info │◄──── stream_init {codec,dur} ──┤ │ ├── ffmpeg -c copy → fMP4 pipe │◄──── stream_data {seg 0, ct} ──┤ (256 KB encrypted segments) │◄──── stream_data {seg 1, ct} ──┤ │ ... │ │◄──── stream_end ───────────────┤ │ │ MediaSource → SourceBuffer │ ├── appendBuffer(decrypted) │ ├── video.play() after ~2-3s │ ``` **Key design decisions:** 1. **Node-side remux via ffmpeg** — `ffmpeg -c copy -movflags frag_keyframe+empty_moov+default_base_moof -f mp4 pipe:1` remuxes any video format (MP4, MKV, AVI, WebM, MOV) into fragmented MP4 (fMP4) that MSE can consume. No transcoding — just remuxing. Near-zero CPU overhead. 2. **Codec detection via ffprobe** — the node probes the video to determine the exact codec string for MSE SourceBuffer creation (e.g., `avc1.640028,mp4a.40.2` for H.264 High@4.0 + AAC-LC). This ensures the browser creates the correct decoder. 3. **Same encryption model** — each 256 KB fMP4 segment is encrypted with AES-256-GCM using the same key derivation as file downloads (GEK + file_hash + segment_index → HKDF → chunk_key). E2E encryption is maintained. 4. **Progressive SourceBuffer append** — the browser creates a MediaSource, opens a SourceBuffer with the probed codec, and appends decrypted segments as they arrive. SourceBuffer handles partial MP4 boxes internally. Playback starts after ~2-3 segments (~512 KB buffered). ### Supported codecs | Codec | MSE string | Browser support | |---|---|---| | H.264 (AVC) | `avc1.PPCCLL` | Chrome, Firefox, Safari, Edge | | H.265 (HEVC) | `hev1.1.6.L93.B0` | Safari, Chrome (partial) | | VP9 | `vp09.00.10.08` | Chrome, Firefox | | AV1 | `av01.0.01M.08` | Chrome, Firefox | | AAC | `mp4a.40.2` | All | | MP3 | `mp4a.6b` | All | | Opus | `opus` | Chrome, Firefox | | AC-3 | `ac-3` | Safari, Chrome | ### New MNP message types | Type | Direction | Description | |---|---|---| | `stream_req` | client → node | Request MSE video stream for file_id | | `stream_init` | node → client | Codec string + duration (probed via ffprobe) | | `stream_data` | node → client | Encrypted fMP4 segment (256 KB, AES-GCM) | | `stream_end` | node → client | End of stream signal | ### Milestones | # | Component | Status | |---|---|---| | 10c.1 | MNP protocol: STREAM_REQUEST/INIT/DATA/END message types | ✅ | | 10c.2 | Node: ffprobe codec detection + MSE codec string derivation | ✅ | | 10c.3 | Node: ffmpeg fMP4 remux + encrypted segment streaming | ✅ | | 10c.4 | Transport: event-based stream message dispatch | ✅ | | 10c.5 | Browser: MSE VideoPlayer (MediaSource + SourceBuffer) | ✅ | | 10c.6 | Tests: stream_request error handling | ✅ | ### File changes **Modified:** - `packages/meshbay-common/src/meshbay_common/protocol.py` — STREAM_REQUEST/INIT/DATA/END - `packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py` — `_probe_video()`, `_stream_video()` handler - `packages/meshbay-hub/src/meshbay_hub/static/transport.js` — `requestStream()`, stream event handlers - `packages/meshbay-hub/src/meshbay_hub/static/app.js` — MSE-based VideoPlayer component - `packages/meshbay-hub/src/meshbay_hub/static/style.css` — streaming progress bar - `packages/meshbay-hub/src/meshbay_hub/static/i18n.js` — buffering/MSE error strings - `packages/meshbay-node/tests/test_webrtc_transport.py` — stream_request error test ### Known limitations (future work) - No seeking beyond buffered range (user must wait for data to arrive) - No adaptive bitrate (single quality stream) - Requires ffmpeg/ffprobe on the node (already a dependency for the old STREAM_SEGMENT handler) --- ## Phase 11 — Node daemon: production-ready ✅ DONE Pending commit — 171 tests. **Objective:** the node daemon (`meshbay-node`) runs as a complete, self-contained service. Previously the daemon only started QUIC/TCP servers and the local web UI; everything browser-facing (WebRTC, hub WS, chat store, HTTP API) was only wired in QE demo scripts. This phase moved all that logic into the daemon. ### What changed **`daemon.py` — complete rewrite.** The daemon now starts all transports and services in a single process: 1. Keystore + hub login (unchanged) 2. Per-group directory indexers (unchanged) 3. **ChatStore** per group (new) — SQLite DB in `~/.local/share/meshbay/{group_id}/chat.db` 4. **WebRTC transport** (new) — browser clients via DataChannel, wired as `on_webrtc_offer` callback on the hub WS 5. QUIC + TCP servers (unchanged) 6. **Hub WebSocket** (new) — `maintain_ws()` as asyncio task, receives signaling, revocation tokens, WebRTC offers. Auto-reconnect on disconnect. 7. **HTTP file API** (new) — one `create_http_app()` per group on configured port 8. Local web UI (unchanged) 9. **Graceful shutdown** (enhanced) — cancels WS task, closes WebRTC peers, closes chat stores, stops HTTP/QUIC/TCP servers, stops indexers **`hub_client.py`** — added `_ws` tracking, `send_ws()` for chat notifications, and `register_swarm()` for file hash registration with the hub. **`config.py`** — added `data_dir` field (default `~/.local/share/meshbay/`) for chat DBs and other persistent state. **`meshbay-node.service`** — updated systemd unit with `StateDirectory=meshbay`, `ProtectSystem=strict`, `ReadWritePaths` for config and data directories. **Index push (11.5):** when watchdog detects file changes, the debounced `on_change` callback fires `_on_index_change` on the daemon, which pushes a full `INDEX_SYNC` to all WebRTC peers in that group. Only peers whose `_group_id` matches receive the push. **Swarm registration (11.9):** on startup and on each index change, the daemon registers all file hashes with the hub's `/v1/swarm/register` endpoint. This allows other nodes/clients to discover which nodes host which content. ### Milestones | # | Component | Status | |---|---|---| | 11.1 | Daemon: hub WS integration | ✅ | | 11.2 | Daemon: WebRTC transport | ✅ | | 11.3 | Daemon: chat store | ✅ | | 11.4 | Daemon: HTTP file API | ✅ | | 11.5 | Daemon: index push on change | ✅ | | 11.6 | Daemon: node_user_id + hub_ws context | ✅ | | 11.7 | Daemon: graceful shutdown | ✅ | | 11.8 | Systemd unit file | ✅ | | 11.9 | Swarm registration | ✅ | | 11.10 | Integration test | ✅ (4 tests: lifecycle, no-groups, index push, group filtering) | ### File changes **Modified:** - `packages/meshbay-node/src/meshbay_node/daemon.py` — complete rewrite - `packages/meshbay-node/src/meshbay_node/hub_client.py` — `_ws` tracking, `send_ws()` - `packages/meshbay-node/src/meshbay_node/config.py` — `data_dir` field - `packaging/systemd/meshbay-node.service` — hardening, StateDirectory **Added:** - `packages/meshbay-node/tests/test_daemon.py` — 2 integration tests --- ## Phase 11.5 — Security remediation ⛔ BLOCKING > Source: `second-review.md` (2026-08-13). Finding IDs in brackets. > **No other phase starts until section J acceptance criteria pass.** **Objective:** close the gap between what the documents describe and what the code enforces. The Phase 12 sovereignty work (GEK-HMAC proof, DTLS channel binding, Ed25519 admin challenge) is sound but was implemented on one of four paths into the node. This phase reduces the node to two paths and brings both to the same standard. ### Transport decision (settled 2026-08-13) | Listener | Fate | Reason | |---|---|---| | WebRTC DataChannel (aiortc) | **Primary** — browser + native | ICE/STUN is the only NAT traversal validated here (2 ISPs, 2 browsers, IPv4 STUN + IPv6, 4G CGNAT) | | QUIC 19000 | **Kept, brought to parity** | LAN, port-forwarded, and hub-less `group://` direct access | | TCP+TLS 18001 | **Removed** | Superseded; no GEK proof; nothing uses it | | HTTP 19001 | **Removed** | Source of C1; duplicates MNP without any of its controls | > `punch_nat()` is a single UDP probe (`quic_server.py:446`) with no STUN client, no > candidate gathering and no dual-stack fallback — `aioice` is pulled in by `aiortc` only. > It is a direct-connection helper, **not** a traversal stack. ICE remains the primary path. ### A — Reduce the node's exposed surface | # | Component | Finding | Done when | |---|---|---|---| | 11.5.1 | Delete `transport/http_server.py` + daemon wiring (`daemon.py:341-366`) | **C1** | No listener on `0.0.0.0` other than QUIC; no endpoint serves file bytes or an index without a completed handshake | | 11.5.2 | Delete `transport/server.py` + `transport/client.py` (TCP+TLS) | C6 scope | `ChunkServer` gone from `daemon.py`; port 18001 unbound | | 11.5.3 | Node admin UI stays loopback + gains a session token in the URL | H2 | UI unreachable without the token printed at daemon startup | ### B — One handshake, two transports | # | Component | Finding | Done when | |---|---|---|---| | 11.5.4 | Extract `meshbay_common/handshake.py`: JWT verify → `scope == "user"` → denylist → **mandatory** `group_id` in claims → group hosted → GEK challenge → proof verify → ack | **C6**, M1, M9 | Single implementation; `webrtc_server.py` and `quic_server.py` contain no JWT logic of their own | | 11.5.5 | Both transports call it; test parametrized over `[webrtc, quic]` | C6 | A test that adds a step to the handshake fails for any transport that skips it | | 11.5.6 | **Spike:** channel binding for QUIC. No DTLS fingerprint exists — bind to the QUIC server certificate hash as the analogue (`sha256(server_cert) ‖ sha256(client_cert)`); prefer an RFC 5705 TLS exporter if `aioquic` can expose one | C6/NS5 | QUIC handshake proof is bound to the connection, not replayable across connections | ### C — Mutual authentication | # | Component | Finding | Done when | |---|---|---|---| | 11.5.7 | Node proves GEK possession over a client nonce **and** signs the transcript with `sk_node`: `Ed25519(sk_node, "meshbay:node_proof:v1" ‖ nonce_c ‖ binding)` | **C3** | Client rejects a peer that cannot produce both | | 11.5.8 | Client pins `pk_node` (TOFU on first connect, persisted); key change raises a blocking warning | C3 | Swapping the node's key surfaces to the user instead of silently succeeding | | 11.5.9 | Node WS registration: require `scope == "node"`, verify `Node.user_id == payload["sub"]`, derive `group_ids` **from the DB**, refuse to overwrite a live registration | **C2** | A user-scoped token, or a mismatched `node_id`, is rejected at `/v1/nodes/ws` | | 11.5.10 | `POST /v1/nodes/announce` requires proof of possession of `sk_node`; one active record per user | M8 | Announcing someone else's `pk_node` fails | ### D — MNP authorization | # | Component | Finding | Done when | |---|---|---|---| | 11.5.11 | `gek_bundle_store` requires an Ed25519 admin challenge; **delete `_try_activate_gek`** — GEK activation is local-UI/CLI only | **C5b** | A member cannot change the group's active GEK | | 11.5.12 | Upload: per-user quarantine `.uploads/{user_id}/`, refuse to overwrite an existing index entry, size cap + per-user quota, filename allowlist (`[A-Za-z0-9._-]`) | **C5a**, H2 | A member cannot replace another member's file, and cannot inject markup via a filename | | 11.5.13 | Admin challenge becomes a structured transcript: `"meshbay:file_delete:v1" ‖ node_pk ‖ group_id ‖ file_id ‖ nonce ‖ ts`; client displays what it signs | **H5** | No path exists where a peer obtains a signature over bytes it fully chose | | 11.5.14 | `gek_bundle_fetch` / `keypair_bundle_fetch` move **after** proof verification; interim rate-limit + audit on the pre-proof window | C4 (partial) | Pre-proof window serves nothing; full fix lands in 13.3 | ### E — Isolation | # | Component | Finding | Done when | |---|---|---|---| | 11.5.15 | `chat_store` and `_peers` resolve from `_group_ctx()`, one peer registry per group (`daemon.py:249`, `webrtc_server.py:602,617,650`) | **H1** | Two-group / two-user test proves neither history nor broadcast crosses groups | ### F — Node admin UI | # | Component | Finding | Done when | |---|---|---|---| | 11.5.16 | `html.escape()` on every interpolated value (`ui/app.py:363`), `textContent` in the audit page (`:632`), CSP header | **H2** | A file named `` renders as text | ### G — Revocation | # | Component | Finding | Done when | |---|---|---|---| | 11.5.17 | Handle `target == "group"` on the node; persist the denylist to `data_dir`; check group status in `webrtc_offer` | **H4** | Revoking a group drops live sessions and blocks new signaling | ### H — Privacy | # | Component | Finding | Done when | |---|---|---|---| | 11.5.18 | Swarm registers hashes for `visibility == "public"` groups only; fix the mis-mounted route (`/v1/groups/v1/swarm/...`); authenticate the lookup | **H7** | No private-group content hash ever reaches the hub | ### I — Resource limits | # | Component | Finding | Done when | |---|---|---|---| | 11.5.19 | Pre-handshake buffer cap (a few KB, not 64 MB); `asyncio.Semaphore` around ffmpeg; delete the synchronous `subprocess.run` in `_do_stream_segment`; per-user signaling rate limit + membership check before relaying an offer; validate `peer_ip` against the request source | **H6** | One client cannot stall the daemon's event loop or exhaust its memory/CPU | ### J — Crypto hygiene, hub fixes, acceptance | # | Component | Finding | Done when | |---|---|---|---| | 11.5.20 | Keystore Argon2id → 256 MB, parameters stored per-node in `node.toml` (not a `meshbay_common` constant); raise the password minimum | M2 | `calibrate-argon2` writes usable config; `crypto.py:173` no longer hardcodes 64 MB | | 11.5.21 | Length-prefix every field in the HMAC transcript; **reject** empty DTLS fingerprints instead of proceeding | L4 | A missing fingerprint fails the handshake rather than degrading it to nonce-only | | 11.5.22 | Hub: fix IPLog backfill (`users.py:118-122`), trusted-proxy XFF, scrub `str(e)` from peer-visible errors, drop `GEK_REQUEST`/`GEK_RESPONSE` constants, validate email | M6, M7, L3, L1, L6 | Compliance log attributes each row to the right account | | 11.5.23 | Regression suite | all | See below | **Required regression tests (all must exist and fail on reintroduction):** ``` test_no_unauthenticated_content — every node listener refuses index/chunks pre-handshake test_handshake_parity[webrtc,quic] — identical checks on both transports test_group_isolation — 2 groups × 2 users: chat + peers never cross test_upload_cannot_overwrite — member B cannot replace member A's file test_gek_store_requires_admin — member cannot store/activate a GEK test_ws_node_identity — user token / foreign node_id rejected test_node_proof_required — client aborts when the node cannot prove GEK + sk_node test_ui_escapes_filenames — markup in a filename renders inert test_swarm_public_only — private hashes never registered ``` **Acceptance criteria for the phase:** with a hub whose signing key is in the attacker's hands, an attacker who is not a group member obtains **no** index entry, **no** file byte, **no** chat message, and cannot write to any node. A member who is not the node operator cannot delete or overwrite another member's file, and cannot change the group key. --- ## Phase 12 — Hub minimization: registrar and nothing more **Objective:** reduce the hub to its legitimate role and make that reduction *structural* rather than a matter of good behaviour. The hub must not be able to see private keys, unencrypted content, or file listings — not "does not currently", but "cannot". ### What the hub is allowed to know | Category | Allowed | Notes | |---|---|---| | Account: username, encrypted email, public keys, status, role | ✅ | Required to be a registrar | | Group registry: id, admin, visibility, join policy, membership | ✅ | Required to issue the `groups` claim | | Public group name + description | ✅ | Required for discovery | | IP logs | ✅ | Legal retention, 1 year | | Signaling relay (SDP/ICE, in-memory, seconds) | ✅ | Never persisted | | **Private keys, keypair bundles, GEK bundles** | ❌ | Removed in Phase 12 (old); 13.3 removes the last copies | | **File content, file names, file hashes, index** | ❌ | H7 was leaking hashes; 11.5.18 closes it | | **Message content or per-message metadata** | ❌ | `chat_notify` currently leaks it — 12.3 | | **Private group name / description** | ❌ (target) | 12.5 | ### Milestones | # | Component | Description | |---|---|---| | 12.1 | Route inventory + blindness test | Enumerate every hub route; assert no response body can contain key material, content, a file name or a content hash. Runs in CI, fails the build on regression | | 12.2 | **Key transparency + safety numbers** [H3] | Append-only, hub-signed key log; clients pin the key they first saw and audit the log; key change raises a blocking warning; safety-number comparison UI between two members. This is the fix for the last structural way a hub can read content | | 12.3 | Chat metadata minimization | `chat_notify` (`webrtc_server.py:634-644` → `revocation.py:101-128`) currently tells the hub *who* posted in *which* group and *when*. Drop `sender_name`, make notification opt-in per group, coalesce and delay to blunt timing correlation | | 12.4 | Swarm hardening | Enforce 11.5.18 at the API layer too: reject registration for a group the hub knows is private; authenticate `GET /v1/swarm/{hash}` | | 12.5 | Opaque private-group metadata | For `visibility == "private"`, store name/description as a member-encrypted blob; the hub holds an opaque value and an id. Public groups unchanged (discovery needs plaintext) | | 12.6 | SPA integrity + honest labelling | Strict CSP, SRI on the bundle, hub publishes a signed digest of the served bundle that native clients and extensions can verify; `/app/` carries an explicit "reduced trust — this hub serves this code" notice | | 12.7 | Remove dead crypto plumbing | Drop residual columns/migrations/constants from the pre-Phase-12 GEK era so the schema cannot be quietly repopulated | | 12.8 | Written threat model | One page: passive hub, active hub, malicious node operator, malicious member, network attacker, local attacker — and for each claim, which adversary it holds against. Referenced from draft-v5 | **Acceptance criteria:** a hub operator holding root on the server, the full PostgreSQL database, the Ed25519 signing key, and the ability to forge any JWT can obtain: no private key, no GEK, no file content, no file name, no content hash, no message content, and no private group name. Every remaining capability is on the list above and is documented in 12.8. Any attempt to substitute a public key is detectable by clients via 12.2. --- ## Phase 13 — Native desktop client (pywebview + aiortc) > **Status (2026-08-13): 13.1 active, 13.2–13.11 DEFERRED to after Phase 15**, pending > decision D2 in `tmp-decisions.md` (browser extension vs native client vs both). > > **13.1 (platform adapter split) proceeds regardless** — it is pure refactoring whose > acceptance criterion is "the browser SPA behaves identically", and it is the prerequisite > for every option under D2. **Objective:** ship a desktop application with durable key storage, hub-independent `group://` access, and a better media path than the browser allows. > ⚠️ **Do not justify this phase as "the fix for T3".** An earlier draft of > `second-review.md` claimed a native client makes code integrity independent of the hub. > That was wrong: a binary downloaded from `meshbay.org` and signed with a key the hub > operator holds relocates the trust rather than removing it. What native actually changes is > **detectability** — an attack must ship as an artifact that can be hashed and compared > instead of a one-off HTTP response — and that value is realised only by **18.7 reproducible > builds** plus published hashes. Native also *costs* the browser sandbox, hands you patch > velocity for WebKitGTK and every bundled dependency, and adds the loopback media server, > the IPC bridge and the updater as new attack surface. > > The security-per-effort ranking is: **11.5 ≫ 12 ≫ 14 (CLI) ≫ 13.** This phase is justified > on product grounds. It permanently closes **C4** as a side effect, but C4 can also be closed > in a browser by not storing keypair bundles remotely at all. ### Why this is cheap The SPA never touches a browser crypto or network primitive directly: `app.js` contains **0** occurrences of `crypto.subtle` and **0** of `RTCPeerConnection`. All crypto and transport go through three injected globals (`window.MeshBayCrypto`, `MeshBayKeys`, `MeshBayTransport` — 16 call sites) and all hub I/O through one function (`hubFetch`, 30 call sites). That is the seam. | Asset | Lines | Native | |---|---|---| | `style.css`, `i18n.js`, `vendor/htm-preact.js` | 1708 | **reuse as-is** | | `app.js` — components, routing, theme, admin | ~2050 | **reuse as-is** | | `app.js` — storage glue, `hubFetch`, download/upload callbacks, MSE `VideoPlayer` | ~550 | rewrite | | `transport.js`, `crypto.js`, `keyderive.js` | 1145 | **delete** | ≈ **69 % reused unchanged**, and the 31 % that is not is largely code `second-review.md` says to delete anyway (WebCrypto AES variant, PBKDF2 password split, keypair bundles). ### Non-negotiable **UI assets ship inside the package and load from disk.** A shell that points its WebView at `https://meshbay.org/app/` is a browser with a different icon and fixes nothing. The hub is used for the API only, and the bundle is covered by 13.9 signing. ### Milestones | # | Component | Description | |---|---|---| | 13.1 | Platform adapter split | Extract `platform-web.js` (WebRTC/WebCrypto/fetch — today's behaviour) and `platform-native.js` (pywebview bridge). `app.js` imports neither directly. **Acceptance: the browser SPA is byte-for-byte functional after the split** — this lands first, on its own, with no native code | | 13.2 | pywebview shell + Python bridge | `meshbay-client` package; `window.pywebview.api.*` implements the same surface as the three globals; single-instance, tray, window state | | 13.3 | Local keystore + Ed25519 client auth | Reuse `keystore.py` (Argon2id 256 MB, OS keychain later). Client authenticates like the daemon does: signed timestamp, `POST /v1/users/auth`. **No password on the wire, no `auth_key`/`bundle_key`, no keypair bundle anywhere** → closes **C4** permanently | | 13.4 | aiortc client transport | `RTCPeerConnection` + `createDataChannel` + `createOffer` in Python; ICE/STUN via `aioice` — the traversal path validated on 2 ISPs. Calls the unified handshake from 11.5.4. QUIC (`quic_client.py`) retained as opt-in for LAN / port-forwarded / hub-less `group://` | | 13.5 | Local index cache | SQLite in the client profile dir, replacing IndexedDB (also restricted under `file://` in some WebViews) | | 13.6 | Loopback media server | Python decrypts and serves with HTTP Range; `