path: root/devel-phases-next.md

# MeshBay — Next Implementation Phases

> Base: Phases 1–11 complete (except 10.9 → Phase 13, 11.5/11.9 deferred). 169 tests. Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org. Node daemon is now production-ready (WebRTC, WS, chat, HTTP all wired).
> Architecture reference: docs/meshbay-draft-v4.md
> First security review: first-review.md (2026-08-10)

---

## Phase 7 — Node v2 : production, streaming, chat ✅ DONE

Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests.

| # | Component | Status |
|---|---|---|
| 7.0 | JWT group claims + node authz check | ✅ |
| 7.1 | QUIC 0-RTT session resumption | ✅ |
| 7.2 | Signaling `client_incoming`/`punch_ready` + jti denylist push | ✅ |
| 7.3 | Multi-group daemon (1-port multiplexing) | ✅ |
| 7.4 | HLS streaming via QUIC | ✅ |
| 7.5 | Chat: Sender Keys protocol + storage + MNP wire | ✅ |
| 7.6 | Chat: local web UI + WS push to members | ✅ |

---

## Phase 8 — Hub v2: admin, federation, security ✅ DONE

Commit: 46918ec — 20 files, +508/−90 lines, 117 tests.
Deployed to meshbay.org. Existing emails encrypted. DB schema migrated.

| # | Component | Status |
|---|---|---|
| 8.1 | Admin roles — config-based `require_admin` | ✅ S1 resolved |
| 8.2 | Email encrypted at rest — AES-256-GCM, HKDF | ✅ S2 resolved |
| 8.3 | Refresh token rotation — family-based reuse detection | ✅ S5 resolved |
| 8.4 | Federation DB persistence (HubPeer model) | ✅ |
| 8.5 | Federation token verification async (DB-backed) | ✅ |
| 8.6 | CSAM hash check in swarm registration | ✅ |
| 8.7 | Rate limiting on auth endpoints (5/10/20 per min) | ✅ |
| 8.8 | Healthcheck endpoint (GET /v1/health) | ✅ |
| 8.9 | IP log cleanup background task (365-day retention) | ✅ |
| 8.10 | Argon2id bumped to 256 MB (pw_version=2, rehash on login) | ✅ |

---

## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE

Commit: ab4d389 — 27 files, +3053/−330 lines, 132 tests.
Deployed to meshbay.org + Orange node. Tested browser → node P2P through two ISP NATs.

**Objective:** a web browser can connect P2P to a node behind residential NAT,
browse files, download, stream video, and chat — with zero data through the hub.

**Architecture decisions (settled 2026-08-10):**

### Transport: WebRTC DataChannel for browsers

Native clients (desktop, Android) use QUIC with `punch_nat()` — already validated
in demo-v2 on SFR residential (Port-Restricted Cone NAT).

Browsers cannot use QUIC for NAT traversal because WebTransport does not allow
the browser to choose its UDP source port. Port-Restricted Cone NAT requires the
client to connect from the exact port the node probed — impossible for browsers.

**Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC
stack handles NAT traversal automatically. The node uses `aiortc` (same author as
`aioquic`, already referenced in draft-v3 as [future]).

ICE is strictly superior to our custom `punch_nat()` for this use case:
- Both sides send STUN binding requests simultaneously → mutual hole-punching
- No need for the client to pre-announce its port
- Handles both sides behind NAT
- Battle-tested by billions of users (Google Meet, Discord, etc.)

The MNP protocol (handshake, file_request, file_chunk, chat_message, etc.) runs
identically over WebRTC DataChannel as over QUIC streams. Same E2E encryption.

**Node dual transport:**
- QUIC (port 19000) — native clients, already in place
- WebRTC DataChannel — browsers, using `aiortc`

### Signaling: hub WebSocket relay

The hub relays WebRTC signaling (SDP offer/answer, ICE candidates) between
browser and node. This is the same role described in draft-v3 section 4.1.3:
"NAT traversal coordination [...] stateless [...] <1 KB per message."

```
Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer  {sdp, ice_candidates}
Hub → Node  (WS)      : {type: "webrtc_offer", sdp, ice_candidates, peer_id}
Node → Hub  (WS)      : {type: "webrtc_answer", sdp, ice_candidates, peer_id}
Hub → Browser (SSE)   : {sdp, ice_candidates}
```

After signaling, the DataChannel is P2P. Hub is no longer involved.

### UI: Preact SPA

- **Framework:** Preact (~3 KB gzipped) + preact-router
- **Build:** esbuild (single binary, no node_modules bloat) for minification
- **Theming:** CSS `prefers-color-scheme` + localStorage toggle (dark/light)
- **i18n:** JSON translation files loaded client-side, English default
- **Responsive:** sidebar collapses to hamburger on mobile viewports
- **Crypto:** existing `crypto.js` (SubtleCrypto AES-GCM) for E2E decryption

### Hub role (reminder — fundamental constraint)

The hub is a registrar and signaling facilitator. It stores ONLY:
- User accounts (login, encrypted email, public keys, keypair bundle)
- Group metadata (name, admin, members, GEK bundles — no file indexes)
- Node registrations (endpoint hints, public keys)

All data (files, streams, chat messages, directory indexes) lives on mesh nodes.
Clients (web or native) transfer data E2E with nodes. The hub never touches
content. This is non-negotiable.

### Chat/forum storage

Chat messages are stored on the node(s) hosting the group, not on the hub.
The browser retrieves chat history from the node via DataChannel, same as files.
If no node in the group is online, the group (including chat) is unavailable.
This is inherent to the P2P model and acceptable.

### File search

Content is not indexed on the hub. Search works client-side:
- Node provides a Mesh Group Index (file metadata: names, paths, sizes, hashes)
- For private groups, the index is GEK-encrypted — hub stores it opaque, client decrypts
- Browser caches decrypted indexes in IndexedDB (~50–100 MB quota, extensible)
- Search runs locally on cached indexes — instant, no network call, no hub involvement

### Milestones

| # | Component | Files | Priority |
|---|---|---|---|
| 9.1 | **Spike: WebRTC DataChannel on node** | `aiortc` integration, 4 tests (handshake, file transfer, auth, guard) | ✅ |
| 9.2 | WebRTC signaling endpoints on hub | `hub/api/signaling.py` — relay SDP/ICE, 2 tests | ✅ |
| 9.3 | WebRTC→MNP transport adapter on node | `node/transport/webrtc_server.py` + hub_client WebRTC handler | ✅ |
| 9.4 | `transport.js` — browser WebRTC client | `static/transport.js` — connect, handshake, fetch, msgpack | ✅ |
| 9.5 | **Spike: E2E browser→NAT→node file transfer** | Mobile 4G → SFR NAT → node, IPv4 STUN + IPv6 validated | ✅ |
| 9.6 | Preact SPA shell (login, routing, theme) | `static/app.js`, `static/style.css`, `static/vendor/htm-preact.js` | ✅ |
| 9.7 | Group list + file explorer UI | `app.js` GroupPage, `groups.py` nodes endpoint, `revocation.py` group tracking | ✅ |
| 9.8 | File download via DataChannel | AES-GCM chunks, GEK delivery, progress bar, browser download | ✅ |
| 9.9 | Video streaming via DataChannel | Chunk download → Blob URL, video overlay with native controls | ✅ |
| 9.10 | Chat/forum UI via DataChannel | ChatPanel component, chat history MNP, peer broadcast, tabs UI | ✅ |
| 9.11 | i18n framework + English strings | `static/i18n.js` — t() lookup, ESM, localStorage lang, all strings extracted | ✅ |
| 9.12 | Settings UI (profile, theme, language) | SettingsPage component, system theme support, sidebar link | ✅ |
| 9.13 | Tests: unit + integration | WebRTC transport, MNP over DataChannel | ✅ |
| 9.14 | Performance: pipelined download | sliding window (8 concurrent chunks) | ✅ |
| 9.15 | Performance: binary wire format | raw bytes via msgpack, no base64 (+33%) | ✅ |
| 9.16 | Performance: avoid redundant I/O | file_hash from index, not re-read per chunk | ✅ |
| 9.17 | Large file download to disk | File System Access API (`showSaveFilePicker`) | ✅ |

**Critical path validated (2026-08-10):** 9.1 → 9.5 all pass. WebRTC DataChannel
works browser → node through two different ISP residential NATs:

**SFR residential NAT** (mobile 4G → node behind SFR Port-Restricted Cone + CGNAT):

| Test | ICE path | Result |
|---|---|---|
| WiFi LAN (same network) | IPv6 direct | OK, ~100ms |
| Mobile 4G SFR + IPv6 | IPv6 inter-network | OK, ~600ms |
| Mobile 4G SFR + IPv4 only | STUN hole-punch IPv4 | OK, ~650ms |

**Orange Livebox NAT** (laptop browser → node behind Orange residential NAT, cross-site):

| Test | ICE path | Result |
|---|---|---|
| Chrome laptop → Orange node | IPv6 inter-network | OK, ~7000ms |
| Firefox laptop → Orange node | IPv6 inter-network | OK, ~6700ms |
| Firefox laptop → Orange node (IPv6 disabled) | STUN hole-punch IPv4 | OK, ~6900ms |

Two ISPs validated, both Chrome and Firefox. No TURN relay needed.
ICE/STUN handles all tested NAT types automatically.

**Performance optimizations (2026-08-11):**
- Initial transfer speed: ~2 MB/s (sequential, base64, redundant I/O)
- After file_hash fix (9.16): ~3 MB/s (eliminated 78 GB redundant reads on 279 MB file)
- After pipelining (9.14): ~5 MB/s (8-chunk sliding window, concurrent requests)
- After binary wire format (9.15): eliminated 33% base64 inflation + removed
  redundant per-chunk fields (sig, hashes, pk_node) — AES-GCM tag already
  authenticates ciphertext, DTLS authenticates transport
- Large file support (9.17): `showSaveFilePicker` (Chrome/Edge) streams decrypted
  chunks directly to disk — flat ~8 MB RAM regardless of file size. Firefox/Safari
  fall back to Blob-in-RAM approach.

**Indexer debounce (2026-08-11):**
- File copy triggers multiple watchdog events at different file sizes → duplicate
  index entries with different blake3 hashes. Fixed with 2-second debounce +
  path-based dedup (remove old entry before adding new).

**Known remaining items for future phases:**
- ~~True video streaming (MSE or Service Worker)~~ → Phase 10c (2026-08-11)
- Multiple shared directories per node (UI + config)
- Multi-node per user support

**Dependencies added:**
- `aiortc>=1.9` in `meshbay-node/pyproject.toml` ✅
- `esbuild` as a dev tool (single binary, not npm) — needed for 9.6+
- `preact` + `preact-router` (ESM imports, no npm needed — CDN or vendored)

---

## Phase 10 — meshbay.org site + admin/moderation UI

Commit: 8fa298e (10.1–10.4), 022da76 (10.5–10.10) — 155 tests.

**Objective:** meshbay.org becomes both a production hub and the project's public
website, with admin/moderation interfaces and user-facing features.

### Site architecture

Two layers, cleanly separated:
- **Generic hub** (API + web app) — reusable by any hub operator
- **Site overlay** — meshbay.org-specific pages (landing, /downloads, /about)

The site overlay is served by Caddy (static files) with priority over the hub.
The hub serves the SPA for authenticated users at `/app/`.

```
site/                       # meshbay.org-specific (not in generic hub package)
├── index.html              # Landing page — project promotion
├── downloads.html          # Package repos (placeholder, Phase 13)
├── about.html              # Project info, GitHub link, contact
└── assets/
    └── site.css            # Landing page styles (dark/light aware)
```

### User roles

| Role | Capabilities |
|---|---|
| `user` | Standard user — browse, download, chat, manage own profile |
| `moderator` | Review reports, suspend content/groups/users |
| `admin` | All moderator rights + hub management (same as moderator for now, distinction reserved for future federation/mirror) |

Role stored as `role` column on User model (`user` | `moderator` | `admin`).
`require_moderator` dependency (checks role ≥ moderator OR config allowlist).
`require_admin` checks role = admin OR config allowlist (backward compat).
Config-listed admin usernames are synced to `role = "admin"` in DB at startup.

### Milestones

| # | Component | Status |
|---|---|---|
| 10.1 | Landing page + /downloads + /about | ✅ |
| 10.2 | Moderator role + `require_moderator` dependency + admin API | ✅ |
| 10.3 | Moderation UI (user/group suspend, blocklist management) | ✅ |
| 10.4 | Admin UI (stats, user list, group list, audit logs viewer, blocklist) | ✅ |
| 10.5 | Notification system (invitations, role changes, account status) | ✅ |
| 10.6 | User settings (profile, role display, per-group notification mute) | ✅ |
| 10.7 | Public group search (name keyword filtering) | ✅ |
| 10.8 | Front page (notification feed with unread badge) | ✅ |
| 10.9 | Package repositories (APT/DNF) | Deferred to Phase 13 |
| 10.10 | Auto-update check endpoint (`GET /v1/hub/version`) | ✅ |

### API endpoints (10.2, 10.5, 10.7, 10.10)

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) |
| GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) |
| GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable) |
| GET | `/v1/admin/users/{id}` | Moderator+ | User detail (email decrypted, group count) |
| PATCH | `/v1/admin/users/{id}` | Moderator+ | Update role or status (triggers notification) |
| GET | `/v1/admin/groups` | Moderator+ | List groups (with member count) |
| PATCH | `/v1/admin/groups/{id}` | Moderator+ | Update group status |
| GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user) |
| GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) |
| POST | `/v1/notifications/{id}/read` | Access token | Mark single notification read |
| POST | `/v1/notifications/read-all` | Access token | Mark all notifications read |
| GET | `/v1/groups?q=` | None | Search public groups by name |
| GET | `/v1/hub/version` | None | Version check (hub, MNP, MHP versions) |

### Admin UI (10.3–10.4)

Admin page at `#/admin` in SPA, accessible to moderators and admins.
Five tabs: Stats, Users, Groups, Logs, Blocklist.

- **Stats:** card grid (users, groups, nodes, online nodes)
- **Users:** searchable table, inline role dropdown, suspend/unsuspend buttons, detail overlay
- **Groups:** table with member count, suspend/unsuspend
- **Logs:** filterable IP audit log table, paginated (50/page, load more)
- **Blocklist:** existing `/v1/admin/blocklist` endpoints, add/remove hashes

### SPA route change

SPA now also served at `/app/` and `/app/{path}` (in addition to `/`).
With Caddy site overlay, Caddy serves `site/index.html` at `/`,
and requests to `/app/` fall through to the hub.

### Caddy integration

Recommended Caddyfile snippet for meshbay.org:
```
meshbay.org {
    root * /path/to/meshbay/site
    try_files {path} {path}.html
    file_server

    handle /v1/* {
        reverse_proxy localhost:8000
    }
    handle /app* {
        reverse_proxy localhost:8000
    }
    handle /style.css {
        reverse_proxy localhost:8000
    }
    handle /*.js {
        reverse_proxy localhost:8000
    }
}
```

### Hub mirror (design only — implementation deferred)

A mirror hub is a complete replica of the primary hub (same user DB, same groups,
same GEK bundles, same storage). Purpose: load distribution via DNS round-robin.

**Design constraints:**
- Shared Ed25519 private key (transferred once at setup, securely)
- PostgreSQL logical replication for active-active read/write on both mirrors
- Both mirrors can issue JWTs (same signing key)
- DNS round-robin (2+ A records on meshbay.org)
- If one mirror goes down, the other continues serving

**Not implemented now.** The design must not prevent future implementation:
- Hub config and private key paths must be externalizable
- No hub-specific state that can't be replicated
- JWT verification must not depend on hub-local state

---

## Phase 10b — Self-service UI + client-side features

Pending commit — 166 tests.

**Objective:** make the web SPA fully self-service — users can create groups,
manage members, join open groups, upload files, and search across all cached
group file indexes. No admin intervention needed for basic operations.

### Self-service features

| # | Component | Status |
|---|---|---|
| 10b.1 | Group creation UI (CreateGroupPage) | ✅ |
| 10b.2 | Member management + invite (MembersPanel) | ✅ |
| 10b.3 | Group join flow (open groups self-join) | ✅ |
| 10b.4 | File upload (client → node via MNP FILE_UPLOAD) | ✅ |
| 10b.5 | IndexedDB caching (group file indexes cached locally) | ✅ |
| 10b.6 | Cross-group file search (SearchPage — client-side, no hub) | ✅ |

### New API endpoints (10b.1–10b.3)

| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/v1/groups` | Access token | Create a new group (name, visibility, join_policy) |
| GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) |
| POST | `/v1/groups/{id}/join` | Access token | Self-join open group (checks join_policy) |
| POST | `/v1/groups/{id}/members/{username}/gek` | Access token | Store GEK bundle for invitee |
| GET | `/v1/groups/{id}/gek` | Access token | Get own GEK bundle (for wrapping) |

### New MNP message types (10b.4)

| Type | Direction | Description |
|---|---|---|
| `file_upload` | client → node | Push encrypted file chunk (filename, chunk_index, total_chunks, data) |
| `file_upload_ack` | node → client | Acknowledge chunk receipt |

Node stores uploads in `shared_root/.uploads/` as `.part` files during transfer,
renames to final location on last chunk. Filename sanitized (no path traversal).

### Browser crypto additions (10b.2)

AES-256-GCM ECIES variant for GEK wrapping in browsers. WebCrypto does not
support ChaCha20-Poly1305, so a parallel ECIES scheme uses AES-256-GCM with
a distinct HKDF info string (`meshbay:gek_wrap:v1:aes` vs `meshbay:gek_wrap:v1`).
Both Python and browser implement the AES variant for interop.

Functions added to `crypto.js`: `generateGEK()`, `wrapGEK()`, `unwrapGEK()`,
`encryptChunk()`, `b64encode()`.

Functions added to `crypto.py`: `wrap_gek_aes()`, `unwrap_gek_aes()`.

### IndexedDB caching (10b.5)

When a group's file index is fetched from a node, it is cached in IndexedDB
(`meshbay` database, `group_indexes` store). On subsequent visits, cached
entries are shown immediately while the live connection is established. This
gives instant file list display even before WebRTC connects.

Cache key: `groupId`. Stored: `{ groupId, groupName, entries[], cachedAt }`.
Best-effort — failures are silently ignored.

### Cross-group file search (10b.6)

SearchPage component at `#/search`. Searches file names and paths across ALL
cached group indexes in IndexedDB. Pure client-side — no hub involvement.
Results link back to the group page. Accessible from sidebar.

### Tests added

- 8 tests: group self-service (create, join open, join invite rejected, join already member, members list, non-member denied, search, join triggers notification)
- 3 tests: AES GEK wrap/unwrap (round-trip, wrong key rejected, differs from ChaCha20 wrap)

---

## Phase 10c — MSE video streaming (real-time playback)

Pending commit — 167 tests.

**Objective:** replace the download-then-play video player with real-time
MSE (MediaSource Extensions) streaming. Playback starts within seconds
instead of waiting for the full file download.

### Architecture

```
Browser                          Node
  │                                │
  ├── stream_req {file_id} ──────►│
  │                                ├── ffprobe → codec info
  │◄──── stream_init {codec,dur} ──┤
  │                                ├── ffmpeg -c copy → fMP4 pipe
  │◄──── stream_data {seg 0, ct} ──┤  (256 KB encrypted segments)
  │◄──── stream_data {seg 1, ct} ──┤
  │  ...                           │
  │◄──── stream_end ───────────────┤
  │                                │
  MediaSource → SourceBuffer       │
  ├── appendBuffer(decrypted)      │
  ├── video.play() after ~2-3s     │
```

**Key design decisions:**

1. **Node-side remux via ffmpeg** — `ffmpeg -c copy -movflags frag_keyframe+empty_moov+default_base_moof -f mp4 pipe:1` remuxes any video format (MP4, MKV, AVI, WebM, MOV) into fragmented MP4 (fMP4) that MSE can consume. No transcoding — just remuxing. Near-zero CPU overhead.

2. **Codec detection via ffprobe** — the node probes the video to determine the exact codec string for MSE SourceBuffer creation (e.g., `avc1.640028,mp4a.40.2` for H.264 High@4.0 + AAC-LC). This ensures the browser creates the correct decoder.

3. **Same encryption model** — each 256 KB fMP4 segment is encrypted with AES-256-GCM using the same key derivation as file downloads (GEK + file_hash + segment_index → HKDF → chunk_key). E2E encryption is maintained.

4. **Progressive SourceBuffer append** — the browser creates a MediaSource, opens a SourceBuffer with the probed codec, and appends decrypted segments as they arrive. SourceBuffer handles partial MP4 boxes internally. Playback starts after ~2-3 segments (~512 KB buffered).

### Supported codecs

| Codec | MSE string | Browser support |
|---|---|---|
| H.264 (AVC) | `avc1.PPCCLL` | Chrome, Firefox, Safari, Edge |
| H.265 (HEVC) | `hev1.1.6.L93.B0` | Safari, Chrome (partial) |
| VP9 | `vp09.00.10.08` | Chrome, Firefox |
| AV1 | `av01.0.01M.08` | Chrome, Firefox |
| AAC | `mp4a.40.2` | All |
| MP3 | `mp4a.6b` | All |
| Opus | `opus` | Chrome, Firefox |
| AC-3 | `ac-3` | Safari, Chrome |

### New MNP message types

| Type | Direction | Description |
|---|---|---|
| `stream_req` | client → node | Request MSE video stream for file_id |
| `stream_init` | node → client | Codec string + duration (probed via ffprobe) |
| `stream_data` | node → client | Encrypted fMP4 segment (256 KB, AES-GCM) |
| `stream_end` | node → client | End of stream signal |

### Milestones

| # | Component | Status |
|---|---|---|
| 10c.1 | MNP protocol: STREAM_REQUEST/INIT/DATA/END message types | ✅ |
| 10c.2 | Node: ffprobe codec detection + MSE codec string derivation | ✅ |
| 10c.3 | Node: ffmpeg fMP4 remux + encrypted segment streaming | ✅ |
| 10c.4 | Transport: event-based stream message dispatch | ✅ |
| 10c.5 | Browser: MSE VideoPlayer (MediaSource + SourceBuffer) | ✅ |
| 10c.6 | Tests: stream_request error handling | ✅ |

### File changes

**Modified:**
- `packages/meshbay-common/src/meshbay_common/protocol.py` — STREAM_REQUEST/INIT/DATA/END
- `packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py` — `_probe_video()`, `_stream_video()` handler
- `packages/meshbay-hub/src/meshbay_hub/static/transport.js` — `requestStream()`, stream event handlers
- `packages/meshbay-hub/src/meshbay_hub/static/app.js` — MSE-based VideoPlayer component
- `packages/meshbay-hub/src/meshbay_hub/static/style.css` — streaming progress bar
- `packages/meshbay-hub/src/meshbay_hub/static/i18n.js` — buffering/MSE error strings
- `packages/meshbay-node/tests/test_webrtc_transport.py` — stream_request error test

### Known limitations (future work)

- No seeking beyond buffered range (user must wait for data to arrive)
- No adaptive bitrate (single quality stream)
- Requires ffmpeg/ffprobe on the node (already a dependency for the old STREAM_SEGMENT handler)

---

## Phase 11 — Node daemon: production-ready ✅ DONE

Pending commit — 169 tests.

**Objective:** the node daemon (`meshbay-node`) runs as a complete, self-contained
service. Previously the daemon only started QUIC/TCP servers and the local web UI;
everything browser-facing (WebRTC, hub WS, chat store, HTTP API) was only wired
in QE demo scripts. This phase moved all that logic into the daemon.

### What changed

**`daemon.py` — complete rewrite.** The daemon now starts all transports and
services in a single process:

1. Keystore + hub login (unchanged)
2. Per-group directory indexers (unchanged)
3. **ChatStore** per group (new) — SQLite DB in `~/.local/share/meshbay/{group_id}/chat.db`
4. **WebRTC transport** (new) — browser clients via DataChannel, wired as
   `on_webrtc_offer` callback on the hub WS
5. QUIC + TCP servers (unchanged)
6. **Hub WebSocket** (new) — `maintain_ws()` as asyncio task, receives signaling,
   revocation tokens, WebRTC offers. Auto-reconnect on disconnect.
7. **HTTP file API** (new) — one `create_http_app()` per group on configured port
8. Local web UI (unchanged)
9. **Graceful shutdown** (enhanced) — cancels WS task, closes WebRTC peers, closes
   chat stores, stops HTTP/QUIC/TCP servers, stops indexers

**`hub_client.py`** — added `_ws` tracking and `send_ws()` method so the WebRTC
context can send `chat_notify` messages to the hub for real-time chat notifications.

**`config.py`** — added `data_dir` field (default `~/.local/share/meshbay/`)
for chat DBs and other persistent state.

**`meshbay-node.service`** — updated systemd unit with `StateDirectory=meshbay`,
`ProtectSystem=strict`, `ReadWritePaths` for config and data directories.

### Milestones

| # | Component | Status |
|---|---|---|
| 11.1 | Daemon: hub WS integration | ✅ |
| 11.2 | Daemon: WebRTC transport | ✅ |
| 11.3 | Daemon: chat store | ✅ |
| 11.4 | Daemon: HTTP file API | ✅ |
| 11.5 | Daemon: index push on change | Deferred — indexer `on_change` callback not yet implemented |
| 11.6 | Daemon: node_user_id + hub_ws context | ✅ |
| 11.7 | Daemon: graceful shutdown | ✅ |
| 11.8 | Systemd unit file | ✅ |
| 11.9 | Swarm registration | Deferred — hub endpoint exists but node-side trigger not yet wired |
| 11.10 | Integration test | ✅ (2 tests: full lifecycle + no-groups-exit) |

### File changes

**Modified:**
- `packages/meshbay-node/src/meshbay_node/daemon.py` — complete rewrite
- `packages/meshbay-node/src/meshbay_node/hub_client.py` — `_ws` tracking, `send_ws()`
- `packages/meshbay-node/src/meshbay_node/config.py` — `data_dir` field
- `packaging/systemd/meshbay-node.service` — hardening, StateDirectory

**Added:**
- `packages/meshbay-node/tests/test_daemon.py` — 2 integration tests

### Deferred items

- **11.5 Index push**: requires `DirectoryIndexer` to have an `on_change` callback
  that fires when watchdog detects file changes, then the daemon pushes `INDEX_DELTA`
  to all connected WebRTC peers. The indexer currently rebuilds the full index but
  doesn't notify consumers of incremental changes.
- **11.9 Swarm registration**: hub `/v1/swarm/register` endpoint exists. Node needs
  to register file hashes after each index rebuild. Depends on 11.5 (index change events).

---

## Phase 12 — Node CLI + management

**Objective:** `meshbay-node` CLI becomes a full management tool, not just a
daemon launcher. Users can manage groups, members, and node state from the
command line.

### Milestones

| # | Component | Description |
|---|---|---|
| 12.1 | `meshbay-node status` | Show daemon state: groups, peers, connected members, uptime |
| 12.2 | `meshbay-node group list` | List configured groups with online status |
| 12.3 | `meshbay-node group create` | Create group on hub, add to config, generate GEK |
| 12.4 | `meshbay-node group join` | Join existing group, fetch GEK, add to config |
| 12.5 | `meshbay-node member invite` | Wrap GEK for new member, push bundle to hub |
| 12.6 | `meshbay-node member remove` | Rotate GEK, re-wrap for remaining members, push to hub |
| 12.7 | `meshbay-node member list` | List group members with online status |
| 12.8 | Config reload (SIGHUP) | Daemon reloads config and adds/removes groups without restart |

### Architecture

CLI commands communicate with the running daemon via a local Unix socket
(`/run/meshbay-node.sock`). The daemon exposes a small internal API for
status queries and management operations. If the daemon is not running,
commands that require it fail with a clear error.

---

## Phase 13 — Chat encryption (Sender Keys) + retention

**Objective:** implement spec section 6.6 — group chat messages are encrypted
with the Sender Keys protocol. Currently, chat messages are stored and
transmitted as plaintext payloads (relying on transport encryption only).

### Background

`meshbay_common.senderkeys` (Phase 7.5) implements the Sender Keys protocol,
but the node chat flow (`_do_chat_message`) stores raw payloads without
encrypting them. The Sender Keys module provides:
- Per-sender chain key derivation (ratcheting)
- Symmetric encryption of group messages
- Key distribution via pairwise GEK-wrapped channels

### Milestones

| # | Component | Description |
|---|---|---|
| 13.1 | Node: sender key init | Generate sender key on group join, distribute to members |
| 13.2 | Node: encrypt chat on send | Encrypt payload with sender's chain key before broadcast |
| 13.3 | Node: decrypt chat on receive | Decrypt incoming chat messages, handle out-of-order |
| 13.4 | Key rotation on member removal | Admin removes member → all remaining members rotate keys |
| 13.5 | Chat retention config | Per-group `max_age_days` setting, periodic cleanup in ChatStore |
| 13.6 | MNP version negotiation | Handshake declares supported version range, not just single `v` field |

### Security note

Without Sender Keys, any node operator (or anyone with filesystem access to
the node) can read all chat messages in plaintext. With Sender Keys, messages
are encrypted with per-sender chain keys that the node operator does NOT
possess — only group members with the distributed sender keys can decrypt.
This is a fundamental security upgrade for group privacy.

---

## Phase 14 — Android client MVP

**Objective:** Android app for account creation, group browsing, file download,
chat. No node functionality on mobile (client-only).

**Stack:** Kotlin native + Jetpack Compose. QUIC via `quiche` (Cloudflare, Rust
JNI binding). Crypto via Bouncy Castle JVM. Same NAT traversal as desktop native
clients (`punch_nat` + QUIC).

| # | Component | Tech | Priority |
|---|---|---|---|
| 14.1 | Hub client (auth, groups, GEK) | Kotlin + Retrofit | High |
| 14.2 | Crypto (Ed25519, X25519, ChaCha20) | Bouncy Castle JVM | High |
| 14.3 | QUIC client | quiche (Rust JNI) | High |
| 14.4 | NAT traversal (STUN + punch) | Kotlin native UDP | High |
| 14.5 | File browser + download | Kotlin + streaming IO | High |
| 14.6 | Chat UI | Jetpack Compose | Medium |
| 14.7 | Contact list integration | Android Contacts API (permission-gated) | Medium |
| 14.8 | Account creation from app | Registration flow + keypair bundle | High |

**Cross-device compatibility:** the user may switch between web and Android.
The `keypair_bundle` (encrypted, stored on hub) enables this — same credentials,
same keys on both platforms. Notification state and read markers should sync
via hub (small encrypted blob per user, minimal storage).

**Upload from mobile:** posting photos/videos to a group. The mobile uploads to
the group's node(s), not to the hub. The node stores it. MNP protocol extended
with an `upload` message type for client→node push.

**Out of scope:** node functionality on mobile, Mac/iPhone support.

---

## Phase 15 — Network resilience (optional, low priority)

**Objective:** handle edge cases — symmetric NAT (CGNAT mobile), TURN relay,
0-RTT reconnection. Not needed for typical residential users.

| # | Component | Priority |
|---|---|---|
| 15.1 | Mesh Relay TURN server | Low |
| 15.2 | Relay registration via MHP | Low |
| 15.3 | Node fallback to relay after ICE failure | Low |
| 15.4 | QUIC 0-RTT (session tickets) | Medium |
| 15.5 | Connection pool (1 QUIC conn = N requests) | Medium |
| 15.6 | Test CGNAT mobile 4G | Low |

**Note:** enterprise users behind restrictive firewalls can configure port
forwarding themselves. This phase targets the ~15% of residential connections
where even ICE/STUN fails (symmetric NAT behind CGNAT). Not a priority —
the user explicitly deprioritized this.

---

## Phase 16 — RPM/DEB packaging + CI

| # | Component |
|---|---|
| 16.1 | RPM build pipeline (Fedora, RHEL) |
| 16.2 | DEB build pipeline (Ubuntu, Debian) |
| 16.3 | GitHub Actions CI (pytest + ruff on PR) |
| 16.4 | Release signing (GPG key) |
| 16.5 | Repo apt/dnf on meshbay.org/packages/ |
| 16.6 | Android APK distribution on meshbay.org/downloads/ |

---

## Phase 17 — Extension module sandbox (future)

**Objective:** implement spec section 12 — Python extension modules that can
react to group events, access the file index, and send messages, running in
a sandboxed subprocess with limited permissions.

| # | Component | Description |
|---|---|---|
| 17.1 | Module manifest loader | Parse `module.toml`, validate permissions |
| 17.2 | Sandboxed subprocess | `read_index()`, `send_message()`, `receive_events()` API |
| 17.3 | Permission enforcement | No filesystem/network beyond group context |
| 17.4 | Module marketplace on hub | List/install/rate extension modules |

**Low priority.** This is an extensibility feature for power users and
community developers. Core functionality must be complete and stable first.

---

## Recommended order

```
Phase 12 (Node CLI)        ← management UX, now the critical path
Phase 13 (Sender Keys)     ← chat security upgrade
Phase 14 (Android)         ← mobile client, long effort
Phase 16 (Packaging)       ← distribution
Phase 15 (Resilience)      ← optional, edge cases only
Phase 17 (Extensions)      ← future, community-driven
```

Phase 11 (daemon) is complete. Phase 12 (CLI) is now the critical path — without
it, managing groups and members requires manual API calls. After that, Phase 13
(Sender Keys) closes the chat encryption gap flagged in the security review.

---

## Structural decisions (all resolved)

1. Multi-group on a single QUIC port ✅ (Phase 7)
2. Signaling punch/connect via hub WS ✅ (Phase 7)
3. Chat is a core feature, not a module ✅ (draft v3)
4. Chat encryption: Sender Keys ✅ (security review)
5. JWT group claims required ✅ (security review)
6. Admin model: config-based ✅ (Phase 8)
7. Refresh token rotation: family-based ✅ (Phase 8)
8. Email encrypted at rest: AES-256-GCM ✅ (Phase 8)
9. Argon2id params: 256 MB, pw_version for migration ✅ (Phase 8)
10. **Browser transport: WebRTC DataChannel + ICE/STUN** ✅ (decided 2026-08-10)
11. **Hub role: registrar + signaling ONLY, never in data path** ✅ (reinforced 2026-08-10)
12. **Chat stored on nodes, not hub** ✅ (decided 2026-08-10)
13. **Web UI: Preact SPA, dark/light, responsive, i18n** ✅ (decided 2026-08-10)
14. **Site overlay: meshbay.org-specific pages separate from generic hub** ✅ (decided 2026-08-10)
15. **MSE streaming: ffmpeg fMP4 remux on node, SourceBuffer on browser** ✅ (Phase 10c)