path: root/devel-phases-next.md

# MeshBay — Next Implementation Phases

> Base: Phases 1–10b complete (except 10.9 → Phase 13). 166 tests. Web SPA + admin panel + self-service UI live on meshbay.org.
> Architecture reference: docs/meshbay-draft-v4.md
> First security review: first-review.md (2026-08-10)

---

## Phase 7 — Node v2 : production, streaming, chat ✅ DONE

Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests.

| # | Component | Status |
|---|---|---|
| 7.0 | JWT group claims + node authz check | ✅ |
| 7.1 | QUIC 0-RTT session resumption | ✅ |
| 7.2 | Signaling `client_incoming`/`punch_ready` + jti denylist push | ✅ |
| 7.3 | Multi-group daemon (1-port multiplexing) | ✅ |
| 7.4 | HLS streaming via QUIC | ✅ |
| 7.5 | Chat: Sender Keys protocol + storage + MNP wire | ✅ |
| 7.6 | Chat: local web UI + WS push to members | ✅ |

---

## Phase 8 — Hub v2: admin, federation, security ✅ DONE

Commit: 46918ec — 20 files, +508/−90 lines, 117 tests.
Deployed to meshbay.org. Existing emails encrypted. DB schema migrated.

| # | Component | Status |
|---|---|---|
| 8.1 | Admin roles — config-based `require_admin` | ✅ S1 resolved |
| 8.2 | Email encrypted at rest — AES-256-GCM, HKDF | ✅ S2 resolved |
| 8.3 | Refresh token rotation — family-based reuse detection | ✅ S5 resolved |
| 8.4 | Federation DB persistence (HubPeer model) | ✅ |
| 8.5 | Federation token verification async (DB-backed) | ✅ |
| 8.6 | CSAM hash check in swarm registration | ✅ |
| 8.7 | Rate limiting on auth endpoints (5/10/20 per min) | ✅ |
| 8.8 | Healthcheck endpoint (GET /v1/health) | ✅ |
| 8.9 | IP log cleanup background task (365-day retention) | ✅ |
| 8.10 | Argon2id bumped to 256 MB (pw_version=2, rehash on login) | ✅ |

---

## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE

Commit: ab4d389 — 27 files, +3053/−330 lines, 132 tests.
Deployed to meshbay.org + Orange node. Tested browser → node P2P through two ISP NATs.

**Objective:** a web browser can connect P2P to a node behind residential NAT,
browse files, download, stream video, and chat — with zero data through the hub.

**Architecture decisions (settled 2026-08-10):**

### Transport: WebRTC DataChannel for browsers

Native clients (desktop, Android) use QUIC with `punch_nat()` — already validated
in demo-v2 on SFR residential (Port-Restricted Cone NAT).

Browsers cannot use QUIC for NAT traversal because WebTransport does not allow
the browser to choose its UDP source port. Port-Restricted Cone NAT requires the
client to connect from the exact port the node probed — impossible for browsers.

**Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC
stack handles NAT traversal automatically. The node uses `aiortc` (same author as
`aioquic`, already referenced in draft-v3 as [future]).

ICE is strictly superior to our custom `punch_nat()` for this use case:
- Both sides send STUN binding requests simultaneously → mutual hole-punching
- No need for the client to pre-announce its port
- Handles both sides behind NAT
- Battle-tested by billions of users (Google Meet, Discord, etc.)

The MNP protocol (handshake, file_request, file_chunk, chat_message, etc.) runs
identically over WebRTC DataChannel as over QUIC streams. Same E2E encryption.

**Node dual transport:**
- QUIC (port 19000) — native clients, already in place
- WebRTC DataChannel — browsers, using `aiortc`

### Signaling: hub WebSocket relay

The hub relays WebRTC signaling (SDP offer/answer, ICE candidates) between
browser and node. This is the same role described in draft-v3 section 4.1.3:
"NAT traversal coordination [...] stateless [...] <1 KB per message."

```
Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer  {sdp, ice_candidates}
Hub → Node  (WS)      : {type: "webrtc_offer", sdp, ice_candidates, peer_id}
Node → Hub  (WS)      : {type: "webrtc_answer", sdp, ice_candidates, peer_id}
Hub → Browser (SSE)   : {sdp, ice_candidates}
```

After signaling, the DataChannel is P2P. Hub is no longer involved.

### UI: Preact SPA

- **Framework:** Preact (~3 KB gzipped) + preact-router
- **Build:** esbuild (single binary, no node_modules bloat) for minification
- **Theming:** CSS `prefers-color-scheme` + localStorage toggle (dark/light)
- **i18n:** JSON translation files loaded client-side, English default
- **Responsive:** sidebar collapses to hamburger on mobile viewports
- **Crypto:** existing `crypto.js` (SubtleCrypto AES-GCM) for E2E decryption

### Hub role (reminder — fundamental constraint)

The hub is a registrar and signaling facilitator. It stores ONLY:
- User accounts (login, encrypted email, public keys, keypair bundle)
- Group metadata (name, admin, members, GEK bundles — no file indexes)
- Node registrations (endpoint hints, public keys)

All data (files, streams, chat messages, directory indexes) lives on mesh nodes.
Clients (web or native) transfer data E2E with nodes. The hub never touches
content. This is non-negotiable.

### Chat/forum storage

Chat messages are stored on the node(s) hosting the group, not on the hub.
The browser retrieves chat history from the node via DataChannel, same as files.
If no node in the group is online, the group (including chat) is unavailable.
This is inherent to the P2P model and acceptable.

### File search

Content is not indexed on the hub. Search works client-side:
- Node provides a Mesh Group Index (file metadata: names, paths, sizes, hashes)
- For private groups, the index is GEK-encrypted — hub stores it opaque, client decrypts
- Browser caches decrypted indexes in IndexedDB (~50–100 MB quota, extensible)
- Search runs locally on cached indexes — instant, no network call, no hub involvement

### Milestones

| # | Component | Files | Priority |
|---|---|---|---|
| 9.1 | **Spike: WebRTC DataChannel on node** | `aiortc` integration, 4 tests (handshake, file transfer, auth, guard) | ✅ |
| 9.2 | WebRTC signaling endpoints on hub | `hub/api/signaling.py` — relay SDP/ICE, 2 tests | ✅ |
| 9.3 | WebRTC→MNP transport adapter on node | `node/transport/webrtc_server.py` + hub_client WebRTC handler | ✅ |
| 9.4 | `transport.js` — browser WebRTC client | `static/transport.js` — connect, handshake, fetch, msgpack | ✅ |
| 9.5 | **Spike: E2E browser→NAT→node file transfer** | Mobile 4G → SFR NAT → node, IPv4 STUN + IPv6 validated | ✅ |
| 9.6 | Preact SPA shell (login, routing, theme) | `static/app.js`, `static/style.css`, `static/vendor/htm-preact.js` | ✅ |
| 9.7 | Group list + file explorer UI | `app.js` GroupPage, `groups.py` nodes endpoint, `revocation.py` group tracking | ✅ |
| 9.8 | File download via DataChannel | AES-GCM chunks, GEK delivery, progress bar, browser download | ✅ |
| 9.9 | Video streaming via DataChannel | Chunk download → Blob URL, video overlay with native controls | ✅ |
| 9.10 | Chat/forum UI via DataChannel | ChatPanel component, chat history MNP, peer broadcast, tabs UI | ✅ |
| 9.11 | i18n framework + English strings | `static/i18n.js` — t() lookup, ESM, localStorage lang, all strings extracted | ✅ |
| 9.12 | Settings UI (profile, theme, language) | SettingsPage component, system theme support, sidebar link | ✅ |
| 9.13 | Tests: unit + integration | WebRTC transport, MNP over DataChannel | ✅ |
| 9.14 | Performance: pipelined download | sliding window (8 concurrent chunks) | ✅ |
| 9.15 | Performance: binary wire format | raw bytes via msgpack, no base64 (+33%) | ✅ |
| 9.16 | Performance: avoid redundant I/O | file_hash from index, not re-read per chunk | ✅ |
| 9.17 | Large file download to disk | File System Access API (`showSaveFilePicker`) | ✅ |

**Critical path validated (2026-08-10):** 9.1 → 9.5 all pass. WebRTC DataChannel
works browser → node through two different ISP residential NATs:

**SFR residential NAT** (mobile 4G → node behind SFR Port-Restricted Cone + CGNAT):

| Test | ICE path | Result |
|---|---|---|
| WiFi LAN (same network) | IPv6 direct | OK, ~100ms |
| Mobile 4G SFR + IPv6 | IPv6 inter-network | OK, ~600ms |
| Mobile 4G SFR + IPv4 only | STUN hole-punch IPv4 | OK, ~650ms |

**Orange Livebox NAT** (laptop browser → node behind Orange residential NAT, cross-site):

| Test | ICE path | Result |
|---|---|---|
| Chrome laptop → Orange node | IPv6 inter-network | OK, ~7000ms |
| Firefox laptop → Orange node | IPv6 inter-network | OK, ~6700ms |
| Firefox laptop → Orange node (IPv6 disabled) | STUN hole-punch IPv4 | OK, ~6900ms |

Two ISPs validated, both Chrome and Firefox. No TURN relay needed.
ICE/STUN handles all tested NAT types automatically.

**Performance optimizations (2026-08-11):**
- Initial transfer speed: ~2 MB/s (sequential, base64, redundant I/O)
- After file_hash fix (9.16): ~3 MB/s (eliminated 78 GB redundant reads on 279 MB file)
- After pipelining (9.14): ~5 MB/s (8-chunk sliding window, concurrent requests)
- After binary wire format (9.15): eliminated 33% base64 inflation + removed
  redundant per-chunk fields (sig, hashes, pk_node) — AES-GCM tag already
  authenticates ciphertext, DTLS authenticates transport
- Large file support (9.17): `showSaveFilePicker` (Chrome/Edge) streams decrypted
  chunks directly to disk — flat ~8 MB RAM regardless of file size. Firefox/Safari
  fall back to Blob-in-RAM approach.

**Indexer debounce (2026-08-11):**
- File copy triggers multiple watchdog events at different file sizes → duplicate
  index entries with different blake3 hashes. Fixed with 2-second debounce +
  path-based dedup (remove old entry before adding new).

**Known remaining items for future phases:**
- True video streaming (MSE or Service Worker) — currently downloads full file first
- Multiple shared directories per node (UI + config)
- Multi-node per user support

**Dependencies added:**
- `aiortc>=1.9` in `meshbay-node/pyproject.toml` ✅
- `esbuild` as a dev tool (single binary, not npm) — needed for 9.6+
- `preact` + `preact-router` (ESM imports, no npm needed — CDN or vendored)

---

## Phase 10 — meshbay.org site + admin/moderation UI

Commit: 8fa298e (10.1–10.4), 022da76 (10.5–10.10) — 155 tests.

**Objective:** meshbay.org becomes both a production hub and the project's public
website, with admin/moderation interfaces and user-facing features.

### Site architecture

Two layers, cleanly separated:
- **Generic hub** (API + web app) — reusable by any hub operator
- **Site overlay** — meshbay.org-specific pages (landing, /downloads, /about)

The site overlay is served by Caddy (static files) with priority over the hub.
The hub serves the SPA for authenticated users at `/app/`.

```
site/                       # meshbay.org-specific (not in generic hub package)
├── index.html              # Landing page — project promotion
├── downloads.html          # Package repos (placeholder, Phase 13)
├── about.html              # Project info, GitHub link, contact
└── assets/
    └── site.css            # Landing page styles (dark/light aware)
```

### User roles

| Role | Capabilities |
|---|---|
| `user` | Standard user — browse, download, chat, manage own profile |
| `moderator` | Review reports, suspend content/groups/users |
| `admin` | All moderator rights + hub management (same as moderator for now, distinction reserved for future federation/mirror) |

Role stored as `role` column on User model (`user` | `moderator` | `admin`).
`require_moderator` dependency (checks role ≥ moderator OR config allowlist).
`require_admin` checks role = admin OR config allowlist (backward compat).
Config-listed admin usernames are synced to `role = "admin"` in DB at startup.

### Milestones

| # | Component | Status |
|---|---|---|
| 10.1 | Landing page + /downloads + /about | ✅ |
| 10.2 | Moderator role + `require_moderator` dependency + admin API | ✅ |
| 10.3 | Moderation UI (user/group suspend, blocklist management) | ✅ |
| 10.4 | Admin UI (stats, user list, group list, audit logs viewer, blocklist) | ✅ |
| 10.5 | Notification system (invitations, role changes, account status) | ✅ |
| 10.6 | User settings (profile, role display, per-group notification mute) | ✅ |
| 10.7 | Public group search (name keyword filtering) | ✅ |
| 10.8 | Front page (notification feed with unread badge) | ✅ |
| 10.9 | Package repositories (APT/DNF) | Deferred to Phase 13 |
| 10.10 | Auto-update check endpoint (`GET /v1/hub/version`) | ✅ |

### API endpoints (10.2, 10.5, 10.7, 10.10)

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) |
| GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) |
| GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable) |
| GET | `/v1/admin/users/{id}` | Moderator+ | User detail (email decrypted, group count) |
| PATCH | `/v1/admin/users/{id}` | Moderator+ | Update role or status (triggers notification) |
| GET | `/v1/admin/groups` | Moderator+ | List groups (with member count) |
| PATCH | `/v1/admin/groups/{id}` | Moderator+ | Update group status |
| GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user) |
| GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) |
| POST | `/v1/notifications/{id}/read` | Access token | Mark single notification read |
| POST | `/v1/notifications/read-all` | Access token | Mark all notifications read |
| GET | `/v1/groups?q=` | None | Search public groups by name |
| GET | `/v1/hub/version` | None | Version check (hub, MNP, MHP versions) |

### Admin UI (10.3–10.4)

Admin page at `#/admin` in SPA, accessible to moderators and admins.
Five tabs: Stats, Users, Groups, Logs, Blocklist.

- **Stats:** card grid (users, groups, nodes, online nodes)
- **Users:** searchable table, inline role dropdown, suspend/unsuspend buttons, detail overlay
- **Groups:** table with member count, suspend/unsuspend
- **Logs:** filterable IP audit log table, paginated (50/page, load more)
- **Blocklist:** existing `/v1/admin/blocklist` endpoints, add/remove hashes

### SPA route change

SPA now also served at `/app/` and `/app/{path}` (in addition to `/`).
With Caddy site overlay, Caddy serves `site/index.html` at `/`,
and requests to `/app/` fall through to the hub.

### Caddy integration

Recommended Caddyfile snippet for meshbay.org:
```
meshbay.org {
    root * /path/to/meshbay/site
    try_files {path} {path}.html
    file_server

    handle /v1/* {
        reverse_proxy localhost:8000
    }
    handle /app* {
        reverse_proxy localhost:8000
    }
    handle /style.css {
        reverse_proxy localhost:8000
    }
    handle /*.js {
        reverse_proxy localhost:8000
    }
}
```

### Hub mirror (design only — implementation deferred)

A mirror hub is a complete replica of the primary hub (same user DB, same groups,
same GEK bundles, same storage). Purpose: load distribution via DNS round-robin.

**Design constraints:**
- Shared Ed25519 private key (transferred once at setup, securely)
- PostgreSQL logical replication for active-active read/write on both mirrors
- Both mirrors can issue JWTs (same signing key)
- DNS round-robin (2+ A records on meshbay.org)
- If one mirror goes down, the other continues serving

**Not implemented now.** The design must not prevent future implementation:
- Hub config and private key paths must be externalizable
- No hub-specific state that can't be replicated
- JWT verification must not depend on hub-local state

---

## Phase 10b — Self-service UI + client-side features

Pending commit — 166 tests.

**Objective:** make the web SPA fully self-service — users can create groups,
manage members, join open groups, upload files, and search across all cached
group file indexes. No admin intervention needed for basic operations.

### Self-service features

| # | Component | Status |
|---|---|---|
| 10b.1 | Group creation UI (CreateGroupPage) | ✅ |
| 10b.2 | Member management + invite (MembersPanel) | ✅ |
| 10b.3 | Group join flow (open groups self-join) | ✅ |
| 10b.4 | File upload (client → node via MNP FILE_UPLOAD) | ✅ |
| 10b.5 | IndexedDB caching (group file indexes cached locally) | ✅ |
| 10b.6 | Cross-group file search (SearchPage — client-side, no hub) | ✅ |

### New API endpoints (10b.1–10b.3)

| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/v1/groups` | Access token | Create a new group (name, visibility, join_policy) |
| GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) |
| POST | `/v1/groups/{id}/join` | Access token | Self-join open group (checks join_policy) |
| POST | `/v1/groups/{id}/members/{username}/gek` | Access token | Store GEK bundle for invitee |
| GET | `/v1/groups/{id}/gek` | Access token | Get own GEK bundle (for wrapping) |

### New MNP message types (10b.4)

| Type | Direction | Description |
|---|---|---|
| `file_upload` | client → node | Push encrypted file chunk (filename, chunk_index, total_chunks, data) |
| `file_upload_ack` | node → client | Acknowledge chunk receipt |

Node stores uploads in `shared_root/.uploads/` as `.part` files during transfer,
renames to final location on last chunk. Filename sanitized (no path traversal).

### Browser crypto additions (10b.2)

AES-256-GCM ECIES variant for GEK wrapping in browsers. WebCrypto does not
support ChaCha20-Poly1305, so a parallel ECIES scheme uses AES-256-GCM with
a distinct HKDF info string (`meshbay:gek_wrap:v1:aes` vs `meshbay:gek_wrap:v1`).
Both Python and browser implement the AES variant for interop.

Functions added to `crypto.js`: `generateGEK()`, `wrapGEK()`, `unwrapGEK()`,
`encryptChunk()`, `b64encode()`.

Functions added to `crypto.py`: `wrap_gek_aes()`, `unwrap_gek_aes()`.

### IndexedDB caching (10b.5)

When a group's file index is fetched from a node, it is cached in IndexedDB
(`meshbay` database, `group_indexes` store). On subsequent visits, cached
entries are shown immediately while the live connection is established. This
gives instant file list display even before WebRTC connects.

Cache key: `groupId`. Stored: `{ groupId, groupName, entries[], cachedAt }`.
Best-effort — failures are silently ignored.

### Cross-group file search (10b.6)

SearchPage component at `#/search`. Searches file names and paths across ALL
cached group indexes in IndexedDB. Pure client-side — no hub involvement.
Results link back to the group page. Accessible from sidebar.

### Tests added

- 8 tests: group self-service (create, join open, join invite rejected, join already member, members list, non-member denied, search, join triggers notification)
- 3 tests: AES GEK wrap/unwrap (round-trip, wrong key rejected, differs from ChaCha20 wrap)

---

## Phase 11 — Android client MVP

**Objective:** Android app for account creation, group browsing, file download,
chat. No node functionality on mobile (client-only).

**Stack:** Kotlin native + Jetpack Compose. QUIC via `quiche` (Cloudflare, Rust
JNI binding). Crypto via Bouncy Castle JVM. Same NAT traversal as desktop native
clients (`punch_nat` + QUIC).

| # | Component | Tech | Priority |
|---|---|---|---|
| 11.1 | Hub client (auth, groups, GEK) | Kotlin + Retrofit | High |
| 11.2 | Crypto (Ed25519, X25519, ChaCha20) | Bouncy Castle JVM | High |
| 11.3 | QUIC client | quiche (Rust JNI) | High |
| 11.4 | NAT traversal (STUN + punch) | Kotlin native UDP | High |
| 11.5 | File browser + download | Kotlin + streaming IO | High |
| 11.6 | Chat UI | Jetpack Compose | Medium |
| 11.7 | Contact list integration | Android Contacts API (permission-gated) | Medium |
| 11.8 | Account creation from app | Registration flow + keypair bundle | High |

**Cross-device compatibility:** the user may switch between web and Android.
The `keypair_bundle` (encrypted, stored on hub) enables this — same credentials,
same keys on both platforms. Notification state and read markers should sync
via hub (small encrypted blob per user, minimal storage).

**Upload from mobile:** posting photos/videos to a group. The mobile uploads to
the group's node(s), not to the hub. The node stores it. MNP protocol extended
with an `upload` message type for client→node push.

**Out of scope:** node functionality on mobile, Mac/iPhone support.

---

## Phase 12 — Network resilience (optional, low priority)

**Objective:** handle edge cases — symmetric NAT (CGNAT mobile), TURN relay,
0-RTT reconnection. Not needed for typical residential users.

| # | Component | Priority |
|---|---|---|
| 12.1 | Mesh Relay TURN server | Low |
| 12.2 | Relay registration via MHP | Low |
| 12.3 | Node fallback to relay after ICE failure | Low |
| 12.4 | QUIC 0-RTT (session tickets) | Medium |
| 12.5 | Connection pool (1 QUIC conn = N requests) | Medium |
| 12.6 | Test CGNAT mobile 4G | Low |

**Note:** enterprise users behind restrictive firewalls can configure port
forwarding themselves. This phase targets the ~15% of residential connections
where even ICE/STUN fails (symmetric NAT behind CGNAT). Not a priority —
the user explicitly deprioritized this.

---

## Phase 13 — RPM/DEB packaging + CI

| # | Component |
|---|---|
| 13.1 | RPM build pipeline (Fedora, RHEL) |
| 13.2 | DEB build pipeline (Ubuntu, Debian) |
| 13.3 | GitHub Actions CI (pytest + ruff on PR) |
| 13.4 | Release signing (GPG key) |
| 13.5 | Repo apt/dnf on meshbay.org/packages/ |
| 13.6 | Android APK distribution on meshbay.org/downloads/ |

---

## Recommended order

```
Phase 9  (Web client)      ← core product: browser P2P to nodes
Phase 10 (Site + admin UI) ← public-facing site, moderation, admin
Phase 11 (Android)         ← mobile client, long effort
Phase 13 (Packaging)       ← distribution
Phase 12 (Resilience)      ← optional, edge cases only
```

Phase 9 is the critical path. Milestone 9.5 (spike: browser → NAT → node file
transfer) is the single most important validation in the project. If it works,
the web client is viable. If not, the architecture needs fundamental rethinking.

---

## Structural decisions (all resolved)

1. Multi-group on a single QUIC port ✅ (Phase 7)
2. Signaling punch/connect via hub WS ✅ (Phase 7)
3. Chat is a core feature, not a module ✅ (draft v3)
4. Chat encryption: Sender Keys ✅ (security review)
5. JWT group claims required ✅ (security review)
6. Admin model: config-based ✅ (Phase 8)
7. Refresh token rotation: family-based ✅ (Phase 8)
8. Email encrypted at rest: AES-256-GCM ✅ (Phase 8)
9. Argon2id params: 256 MB, pw_version for migration ✅ (Phase 8)
10. **Browser transport: WebRTC DataChannel + ICE/STUN** ✅ (decided 2026-08-10)
11. **Hub role: registrar + signaling ONLY, never in data path** ✅ (reinforced 2026-08-10)
12. **Chat stored on nodes, not hub** ✅ (decided 2026-08-10)
13. **Web UI: Preact SPA, dark/light, responsive, i18n** ✅ (decided 2026-08-10)
14. **Site overlay: meshbay.org-specific pages separate from generic hub** ✅ (decided 2026-08-10)