diff options
| -rw-r--r-- | CLAUDE.md | 8 | ||||
| -rw-r--r-- | devel-phases.md | 404 | ||||
| -rw-r--r-- | docs/devel-phases-next.md (renamed from devel-phases-next.md) | 36 | ||||
| -rw-r--r-- | docs/first-review.md (renamed from first-review.md) | 2 | ||||
| -rw-r--r-- | docs/meshbay-draft-v1-fr.md | 442 | ||||
| -rw-r--r-- | docs/meshbay-draft-v1.md | 442 | ||||
| -rw-r--r-- | docs/meshbay-draft-v2-fr.md | 550 | ||||
| -rw-r--r-- | docs/meshbay-draft-v2.md | 550 | ||||
| -rw-r--r-- | docs/meshbay-draft-v3.md | 893 | ||||
| -rw-r--r-- | docs/meshbay-draft-v4.md | 1368 | ||||
| -rw-r--r-- | docs/meshbay-draft-v5.md | 2 | ||||
| -rw-r--r-- | docs/old-draft.md | 4497 | ||||
| -rw-r--r-- | docs/poc-v1-fr.md | 432 | ||||
| -rw-r--r-- | docs/poc-v1.md | 767 | ||||
| -rw-r--r-- | docs/second-review.md (renamed from second-review.md) | 4 | ||||
| -rw-r--r-- | docs/tmp-decisions.md (renamed from tmp-decisions.md) | 6 |
16 files changed, 4526 insertions, 5877 deletions
@@ -104,7 +104,7 @@ Scope: `hub`, `node`, `common`, or omitted for cross-cutting - **Never log GEK, private keys, or plaintext passwords** — even at DEBUG level - **meshbay.org is internet-facing** — open port → test → close port + kill processes in same block -## First security review (2026-08-10) — see `first-review.md` +## First security review (2026-08-10) — see `docs/first-review.md` **Critical (before Phase 7):** - **C1** Chat: Sender Keys protocol, NOT shared Double Ratchet (pairwise protocol @@ -167,9 +167,9 @@ Scope: `hub`, `node`, `common`, or omitted for cross-cutting **Architecture validated:** crypto primitives, GEK wrapping (ECIES), trust model, key hierarchy, on-the-fly encryption, transport abstraction, DTLS channel binding. -## Second security review (2026-08-13) — see `second-review.md` +## Second security review (2026-08-13) — see `docs/second-review.md` -**6 critical, 7 high findings. Phase 11.5 is BLOCKING — see `devel-phases-next.md`.** +**6 critical, 7 high findings. Phase 11.5 is BLOCKING — see `docs/devel-phases-next.md`.** The current build must not host real private data. The claims above about node sovereignty and P2P crypto material were **overstated**. The @@ -644,7 +644,7 @@ anything that assumes one key per person. both advance it, producing key/nonce reuse: that is C1 again, one level down. `GroupSenderKeyStore.add_sender` currently does `self._states[dist.sender_id] = ...`, so a second device under the same `sender_id` silently overwrites the first. Revoking a - device must rotate, like revoking a member. See `devel-phases-next.md` §15.0b + device must rotate, like revoking a member. See `docs/devel-phases-next.md` §15.0b - Chat is plaintext on the wire and at rest; the index is plaintext on the WebRTC path ## Known calibration TODOs diff --git a/devel-phases.md b/devel-phases.md deleted file mode 100644 index b5bef49..0000000 --- a/devel-phases.md +++ /dev/null @@ -1,404 +0,0 @@ -# MeshBay — Development Phases - -> Reference: architecture spec in `docs/meshbay-draft-v3.md` -> POC results: `poc/spike-results.md` - ---- - -## Phase 1 — Foundations ✅ DONE - -**Goal:** validate all blocking concepts before writing production code. - -### Deliverables - -| Item | Status | Notes | -|---|---|---| -| POC Spike 1 — Crypto primitives | ✅ | All 22 tests pass. ChaCha20 1MB in 1.1ms. | -| POC Spike 2 — Hub skeleton | ✅ | 12/12 endpoints. JWT EdDSA offline verify in 884µs. | -| POC Spike 3 — Node registration | ✅ | Full handshake. jti bug found and fixed. | -| POC Spike 4 — NAT traversal | ✅ | Cone NAT on SFR. UDP P2P works. UPnP disabled (SFR). | -| POC Spike 5 — Encrypted transfer | ✅ | 1MB P2P. 3.2ms encrypt, 3.9ms decrypt. 4.3MB/s. | -| POC Spike 6 — GEK distribution | ✅ | X25519+HKDF wrap/unwrap. 0.48ms/0.59ms. Hub opaque. | -| Security cleanup meshbay.org | ✅ | UFW: 22/80/443 only. No services exposed. | -| Git monorepo | ✅ | 3 packages: meshbay-common, meshbay-hub, meshbay-node. | -| Draft v3 | ✅ | POC findings integrated. All corrections applied. | -| CLAUDE.md conventions | ✅ | Python 3.12+, uv, ruff, SemVer, commit format. | - -### Key findings from POC - -- jti mandatory in all JWTs (Ed25519 is deterministic — same payload = same token) -- Argon2id at 64MB/3iter = 78ms — increase to 256MB for production (~500ms target) -- NAT order: STUN/hole-punching is priority 2, not UPnP (UPnP disabled on tested SFR box) -- TCP+TLS for v1 transport; QUIC in v2 -- GEK wrapping: ephemeral X25519 + HKDF(salt=pk_eph) + ChaCha20-Poly1305(aad=pk_recipient) - ---- - -## Phase 2 — Node v1 ✅ DONE - -**Goal:** working Mesh Node: indexes a directory, registers with hub, -serves encrypted chunks over TCP+TLS, local web UI on localhost:18000. - -**Transport:** TCP+TLS 1.3 (QUIC in v2). Self-signed cert per node. -Node identity verified via Ed25519 PK from hub, not TLS cert chain (client uses CERT_NONE). - -### Milestones - -| # | Component | File(s) | Tests | -|---|---|---|---| -| 2.1 | Keystore | `meshbay_node/keystore.py` | 10/10 | -| 2.2 | Hub client | `meshbay_node/hub_client.py` | 6/6 | -| 2.3 | Directory indexer | `meshbay_node/indexer/indexer.py` | 5/5 | -| 2.4 | Mesh Group Index | `meshbay_node/indexer/group_index.py` | 5/5 | -| 2.5 | TCP+TLS chunk server | `meshbay_node/transport/server.py` | 3/3 | -| 2.6 | TCP+TLS chunk client | `meshbay_node/transport/client.py` | included above | -| 2.7 | TLS cert helper | `meshbay_node/transport/tls_cert.py` | — | -| 2.8 | Config (TOML + env) | `meshbay_node/config.py` | — | -| 2.9 | Local web UI | `meshbay_node/ui/app.py` | — | -| 2.10 | Daemon + CLI | `meshbay_node/daemon.py` | — | - -**Total: 29/29 tests passing** - -### Python dependencies (dev venv — `/home/cbesson/meshbay/.venv`) - -Installed packages (freeze) as of Phase 2 completion: - -``` -aioice==0.10.2 # ICE/STUN for NAT traversal -blake3==1.0.9 # fast content hashing -cryptography==50.0.0 # Ed25519, X25519, ChaCha20, Argon2id, AES-GCM -fastapi==0.141.1 # local web UI + hub POC -httpx==0.28.1 # hub client HTTP -meshbay-common==0.1.0 # editable install -meshbay-node==0.1.0 # editable install -msgpack==1.2.1 # wire serialisation -PyJWT==2.13.0 # JWT EdDSA -pytest==9.1.1 # test runner -pytest-asyncio==1.4.0 # async test support -uvicorn==0.52.1 # ASGI server (local UI) -watchdog==6.0.0 # filesystem watcher -zstandard==0.25.0 # zstd compression -``` - -Also installed transitively: pydantic 2.13.4, starlette 1.6.0, anyio 4.14.2, uvloop 0.22.1. - -### meshbay_common/crypto.py — Argon2id NOTE - -Current params: `iterations=3, memory_cost=65536` (64MB) → ~78ms on dev laptop. -**Must increase to `memory_cost=262144` (256MB) before production keystore use.** -Run `meshbay-node calibrate-argon2` on target hardware to tune. - -### Out of scope for Phase 2 - -Multiple groups, chat, QUIC, module sandbox, mobile pairing, HLS streaming. - ---- - -## Phase 3 — Hub v1 production ✅ DONE - -**Goal:** replace POC in-memory hub with a production-ready service on meshbay.org. -PostgreSQL persistence, HTTPS via Caddy, all endpoints hardened, legal IP logging, deploy. - -### Environment - -- **Server:** meshbay.org — Ubuntu 26.04 LTS, Python 3.14.4, OVH VPS -- **Database:** PostgreSQL 16 (to install) -- **Proxy:** Caddy (to install — handles Let's Encrypt automatically) -- **Service:** systemd `meshbay-hub.service` - -### Python dependencies to add (Phase 3) - -``` -sqlalchemy>=2.0 # async ORM (SQLAlchemy 2.x) -alembic>=1.13 # DB migrations -asyncpg>=0.30 # PostgreSQL async driver -aiosqlite>=0.20 # SQLite async driver (tests only) -slowapi>=0.1 # rate limiting (FastAPI middleware) -``` - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 3.1 | DB models | `meshbay_hub/db/models.py` | ✅ | -| 3.2 | DB engine + session | `meshbay_hub/db/engine.py` | ✅ | -| 3.3 | Alembic migrations | `meshbay_hub/db/migrations/` | ✅ initial_schema | -| 3.4 | Hub config | `meshbay_hub/config.py` | ✅ | -| 3.5 | Auth (JWT + Argon2id) | `meshbay_hub/auth.py` | ✅ | -| 3.6 | API deps | `meshbay_hub/api/deps.py` | ✅ | -| 3.7 | Hub info router | `meshbay_hub/api/hub.py` | ✅ | -| 3.8 | Users router | `meshbay_hub/api/users.py` | ✅ | -| 3.9 | Nodes router | `meshbay_hub/api/nodes.py` | ✅ | -| 3.10 | Groups router | `meshbay_hub/api/groups.py` | ✅ | -| 3.11 | Rate limiting | `meshbay_hub/api/middleware.py` | ✅ slowapi | -| 3.12 | App factory + lifespan | `meshbay_hub/app.py` | ✅ | -| 3.13 | Hub daemon CLI | `meshbay_hub/daemon.py` | ✅ | -| 3.14 | PostgreSQL 16 | meshbay.org | ✅ DB: meshbay_hub | -| 3.15 | Caddy + Let's Encrypt | meshbay.org Caddyfile | ✅ HTTPS auto-cert | -| 3.16 | Systemd service | `/etc/systemd/system/meshbay-hub.service` | ✅ | -| 3.17 | Deploy + smoke test | https://meshbay.org | ✅ 11/11 endpoints | - -**Total: 40 local tests (SQLite) + 11/11 smoke tests HTTPS production** - -### Phase 3 deployment details (meshbay.org) - -- PostgreSQL 16, user `meshbay`, DB `meshbay_hub` -- Caddy auto-handles Let's Encrypt for `meshbay.org` and `www.meshbay.org` -- `www.meshbay.org` → 301 redirect → `meshbay.org` -- TLS setup documented in `docs/HTTPS.md` -- Hub listens on `127.0.0.1:8000`, Caddy proxies 80/443 -- Systemd service: `meshbay-hub.service` (restart-on-failure) -- Hub config: `~/.config/meshbay/hub.toml` -- Hub keypair: `~/.config/meshbay/hub_private.pem` (chmod 600) -- Source deployed at: `~/meshbay-hub/` (common_pkg + hub_pkg) - -### Additional Python dependencies added in Phase 3 - -``` -sqlalchemy==2.0.51 # async ORM -alembic==1.19.1 # DB migrations -asyncpg==0.31.0 # PostgreSQL async driver -aiosqlite # SQLite async (tests only) -slowapi==0.1.10 # rate limiting -hatchling==1.31.0 # build backend (needed for pip install) -``` - -### Notes - -- Initial DB schema created via `init_db()` (`create_all`) — Alembic tracks future changes -- IP logging records: account_create, login, login_fail, group_create, node_announce -- Rate limiting active on /v1/users/register and /v1/users/login (slowapi) -- Revocation, CSAM hash matching, moderation deferred to Phase 5 - -### Phase 3 scope - -**In scope:** -- All POC endpoints from Spike 2+6, production-ready -- PostgreSQL via SQLAlchemy async + Alembic migrations -- IP logging (creation, login, group events) — 1-year retention, legal compliance -- Access token (JWT, 1h) + refresh token (30d, stored hashed) -- Rate limiting on auth endpoints -- Hub config file `/etc/meshbay/hub.toml` or env vars -- HTTPS via Caddy + auto Let's Encrypt on meshbay.org -- Systemd service with restart-on-failure - -**Deferred to later:** -- Revocation push (WebSocket signaling to nodes) -- CSAM hash matching (NCMEC/IWF integration) -- Moderation flow (blocklist + takedown) -- MHP federation -- RPM/DEB packaging - -### Testing strategy - -- Unit tests with SQLite in-memory (`aiosqlite`) — no PostgreSQL needed locally -- API tests via `httpx.AsyncClient` + `ASGITransport` — no network -- All tests run in the existing `.venv` after adding Phase 3 deps - ---- - -## Phase 4 — Integration & Web Client ✅ DONE - -**Goal:** end-to-end working product in a browser: login via hub, discover groups, -browse files on a node, download and stream public content. - -### Architecture decision - -Browsers cannot make raw TCP connections — the node must speak HTTP. -- Node adds an **HTTP file API** (port 19001) serving public content via standard `fetch()` -- Private content (GEK decrypt in browser) deferred to Phase 5 (requires WebCrypto + ChaCha20 WASM) -- Hub serves the web application at `meshbay.org/app/` - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 4.1 | Node HTTP file API | `meshbay_node/transport/http_server.py` | ✅ 7/7 tests | -| 4.2 | Hub web app (HTML/JS) | `meshbay_hub/api/webapp.py` + `static/app.js` | ✅ live on meshbay.org | -| 4.3 | Group listing endpoint | `meshbay_hub/api/groups.py` GET /v1/groups | ✅ | -| 4.4 | End-to-end integration test | local node ↔ hub ↔ browser | ✅ smoke test | -| 4.5 | HLS streaming (public) | node `/hls/{id}/playlist.m3u8` + `.ts` via ffmpeg | ✅ included in 4.1 | - -**Total: 47/47 tests. https://meshbay.org live with web client.** - -### Phase 4 deployment - -- `https://meshbay.org/` — web app (HTML/JS SPA) -- `https://meshbay.org/app.js` — JS client -- `https://meshbay.org/v1/groups` — public group listing (no auth) -- Node HTTP API (port 19001): `/index`, `/file/{id}`, `/hls/{id}/*.m3u8`, `/hls/{id}/*.ts` -- HLS streaming uses ffmpeg for on-the-fly segmentation -- Public content: no auth for index, auth required for chunks -- Private content (GEK decrypt in browser): deferred to Phase 5 - -### Dependencies added in Phase 4 - -``` -# Node (runtime) -ffmpeg (system package) — HLS segmentation via subprocess -``` - -### Phase 4 scope - -**In scope (public content only):** -- Node HTTP API: serve public Mesh Group Index (JSON) + file chunks (binary) -- Hub web app: login, group discovery, file browser, download link -- HLS basic streaming: node segments video on-the-fly, browser plays natively -- JWT auth passed as query param or header to node HTTP API - -**Deferred to Phase 5:** -- Private group content in browser (requires ChaCha20-Poly1305 via WASM) -- Chat UI -- Multi-group node -- Android client - ---- - -## Phase 5 — QUIC, Federation, Moderation ✅ DONE (Mobile deferred) - -**Goal:** full decentralization, mobile support, community infrastructure. - -| # | Component | Notes | -|---|---|---| -| # | Component | File(s) | Status | -|---|---|---|---| -| 5.1 | QUIC transport (MNP v2) | `transport/quic_server.py` + `quic_client.py` | ✅ 3/3 tests | -| 5.2 | MHP federation | `meshbay_hub/api/federation.py` | ✅ GET/POST /mhp/* | -| 5.3 | Mesh Relay | `meshbay_hub/api/relay.py` | ✅ register+list+approve | -| 5.4 | Android client | — | ⏳ deferred (different tech) | -| 5.5 | Content replication | — | ⏳ deferred | -| 5.6 | iOS client | — | ⏳ after Android | -| 5.7 | Revocation push | `api/revocation.py` + `node/revocation.py` | ✅ 3/3 tests | -| 5.8 | CSAM hash matching | `meshbay_hub/csam.py` | ✅ CSAMChecker + admin API | -| 5.9 | Moderation | `api/moderation.py` | ✅ 6/6 tests | -| 5.10 | RPM/DEB packaging | `packaging/` | ✅ spec + control + systemd | - -**Total: 59/59 tests.** - -### Phase 5 bugs fixed - -- **QUIC**: `asyncio.Event` race condition in client recv loop (quic_event_received - overwrote `_stream_events[0]` created by `_recv`). Fixed with `asyncio.Queue` - (no shared mutable state between coroutines). -- **QUIC**: `verify_peer` parameter renamed to `verify_mode` in aioquic 1.3.0. -- **QUIC**: `connect()` returns protocol directly (not `(transport, proto)` tuple). - -### Phase 5 dependencies added - -``` -aioquic==1.3.0 # QUIC transport (Cloudflare-maintained) -websockets # hub→node revocation push -``` - -### Deferred to future - -- Android/iOS client (Kotlin/Flutter — different tech stack, dedicated effort) -- Content replication (node-to-node) → Phase 6 -- MHP federation persistence (currently in-memory) → Phase 6 - ---- - -## Phase 6 — Chat, Multi-group, Federation persistence, Replication ✅ DONE - -**Goal:** complete the product with group chat (Double Ratchet), multi-group node support, -persistent MHP federation, content replication, and browser private group decryption. - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 6.1 | Double Ratchet chat | `meshbay_common/ratchet.py` | ✅ 11/11 tests | -| 6.2 | Multi-group node | `meshbay_node/config.py` [[groups]] | ✅ | -| 6.3 | MHP federation persistence | `FederatedGroup` + `SwarmSource` DB tables | ✅ | -| 6.4 | Content replication | `node/replication.py` + hub `/v1/swarm/*` | ✅ | -| 6.5 | Browser private group | `webcrypto.py` + `static/crypto.js` | ✅ 4/4 tests | -| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | -| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | -| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` + `QE/demo-v2/` (non versionné) | ✅ testés | -| 6.9 | QUICKSTART réécrit | `docs/QUICKSTART.md` | ✅ | -| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | -| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | -| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` (non versionné) | ✅ testés | -| 6.9 | QUICKSTART réécrit | `docs/QUICKSTART.md` | ✅ | - -**Total: 81/81 tests.** - ---- - -## Phase 7 — Node v2: production, streaming, chat ✅ DONE - -**Goal:** multi-group node, Sender Keys chat, QUIC 0-RTT, jti denylist push, HLS streaming. - -Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests. - -See `devel-phases-next.md` for details. - ---- - -## Phase 8 — Hub v2: admin, federation, security ✅ DONE - -**Goal:** admin roles, email encryption, refresh token rotation, rate limiting, healthcheck. - -Commit: 46918ec — 20 files, +508/−90 lines, 117 tests. -Deployed to meshbay.org. All security review items S1/S2/S5 resolved. - -See `devel-phases-next.md` for details. - ---- - -## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE - -**Goal:** browser connects P2P to a node behind residential NAT via WebRTC DataChannel. -Full SPA: login, groups, file browser, download, video playback, chat, i18n, dark/light. - -### Milestones - -| # | Component | Status | -|---|---|---| -| 9.1–9.5 | WebRTC DataChannel spike + E2E NAT validation | ✅ | -| 9.6–9.12 | Preact SPA (login, groups, files, video, chat, i18n, settings) | ✅ | -| 9.13 | Tests: 132 passing | ✅ | -| 9.14–9.16 | Performance: pipelining, binary wire format, I/O reduction | ✅ | -| 9.17 | Large file download: File System Access API (stream to disk) | ✅ | - -**Total: 132/132 tests. Deployed to meshbay.org + Orange node (2026-08-11).** - -### Key technical decisions - -- **Transport:** WebRTC DataChannel (aiortc on node) — browsers can't use QUIC for NAT traversal -- **Wire format:** length-prefixed msgpack, binary chunk fields (no base64) -- **UI:** Preact + htm ESM (vendored, no build step, no CDN, no npm) -- **Crypto:** WebCrypto SubtleCrypto AES-256-GCM for E2E chunk decryption in browser -- **Large files:** File System Access API (`showSaveFilePicker`) — stream to disk, ~8 MB RAM -- **Indexer:** 2s debounce + path-based dedup for file copy events - -### NAT traversal validated - -Two ISPs (SFR + Orange residential NAT), Chrome + Firefox, IPv4 STUN + IPv6 direct. -No TURN relay needed. See `devel-phases-next.md` for detailed test matrix. - -### QE deployment state (2026-08-11) - -- **Hub (meshbay.org):** running as `meshbay-hub.service`, DB has 3 users - (admin, cbesson, grenet), 1 group (`d3bbd90b`), `admin_usernames = ["admin"]` -- **Node (Orange host via `ssh cbesson@localhost -p 2222`):** running as - `nohup .venv/bin/python3 QE/demo-v3/run_node_simple.py`, user grenet, - connected via WS to hub, WebRTC + QUIC dual transport -- **Credentials:** `QE/demo-v3/creds.json` (not versioned) -- **Shared dir on node:** `~/meshbay/QE/demo-v3/shared/` -- All 3 users password: see creds.json - -### Dependencies added - -- `aiortc>=1.9` in meshbay-node (WebRTC DataChannel) -- `preact` + `htm` vendored as `static/vendor/htm-preact.js` (ESM, ~3 KB gzipped) - ---- - -## Conventions - -- Commits: `feat(node):`, `fix(hub):`, `chore(common):`, `docs:`, `test(node):` -- Branch per feature/fix, merge to `main` (when remote configured) -- **Always close test UFW ports after any spike on meshbay.org** -- **Never commit key material** (keystore.enc, hub_private.pem, *.key, node_state.json, unlock.key) -- meshbay.org is internet-facing: only run known-safe services, close ports after tests diff --git a/devel-phases-next.md b/docs/devel-phases-next.md index 00cd3e3..6730ca0 100644 --- a/devel-phases-next.md +++ b/docs/devel-phases-next.md @@ -1,9 +1,9 @@ # MeshBay — Next Implementation Phases > Base: Phases 1–12 complete (except 10.9 → Phase 18). Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org. Node daemon is production-ready (WebRTC, WS, chat, HTTP, index push, swarm all wired). -> Architecture reference: **docs/meshbay-draft-v6.md** (2026-08-17; v5 remains +> Architecture reference: **meshbay-draft-v6.md** (2026-08-17; v5 remains > authoritative for everything v6 does not restate, v4 superseded 2026-08-13). -> Desktop client, roots, device linking: **docs/desktop-client-v1.md** +> Desktop client, roots, device linking: **desktop-client-v1.md** > First security review: first-review.md (2026-08-10) > **Second security review: second-review.md (2026-08-13) — 6 critical, 7 high findings.** > @@ -756,7 +756,7 @@ that it stays in the trusted path by choice. **H3 is closed (2026-08-14), by removing the lookup rather than by verifying it.** The plan here was key transparency and safety numbers: keep fetching the invitee's key from the hub, and give humans a way to notice a substitution. What shipped instead is the -invite redesign in `docs/invite-pairing-v1.md` — the node holds the GEK and wraps it +invite redesign in `invite-pairing-v1.md` — the node holds the GEK and wraps it itself, for a key the recipient proves possession of, and identities are bound to accounts by one-time codes the hub never sees. @@ -777,7 +777,7 @@ group key". They are no longer load-bearing. | # | Component | Description | |---|---|---| -| 12.1 | ~~Key transparency + safety numbers~~ [H3] | ✅ **DONE 2026-08-14**, by a different design — see above and `docs/invite-pairing-v1.md` | +| 12.1 | ~~Key transparency + safety numbers~~ [H3] | ✅ **DONE 2026-08-14**, by a different design — see above and `invite-pairing-v1.md` | | 12.2 | Served-SPA integrity | Strict CSP, Subresource Integrity on the bundle, and a signed digest of the served bundle published by the hub so a native client or extension can verify what the browser was given. **Now the highest-value item here**: T3 is the only remaining way an active hub reads content, and it can also lift a pairing code out of the page it served. **The CSP must keep `wasm-unsafe-eval` in `script-src`** — the bundle KDF is Argon2id in WebAssembly since 2026-08-14, and a policy that forbids it locks every user out of their keys | | 12.3 | Honest labelling | `/app/` states plainly that the hub serves this code and what that implies. Docs stop claiming end-to-end integrity for the hub-served path — the claim that holds is "the hub cannot read your content unless it ships you malicious client code" | | 12.4 | Written threat model | One page: passive hub, active hub, malicious node operator, malicious member, network attacker, local attacker — and for each claim, which adversary it holds against. This is what stops the overclaiming pattern the second review kept finding | @@ -790,7 +790,7 @@ metadata minimization, residual schema cleanup. ## Phase 13 — Native desktop client (Electron + optional Python sidecar) -> **Reworked 2026-08-17 by operator decision. `docs/desktop-client-v1.md` is +> **Reworked 2026-08-17 by operator decision. `desktop-client-v1.md` is > authoritative for this phase** — shell, device linking, account creation, node > management, packaging and the open items. The milestone table below is the summary; > the design and its rationale are there. @@ -808,7 +808,7 @@ metadata minimization, residual schema cleanup. > native client holding its own keys is refused by `webrtc_server.py:886-894` where a > browser is not. Without linking, an account created natively could never be opened in a > browser without an operator code per node — the "native must not prevent web use" -> objective fails outright. See `docs/desktop-client-v1.md` §4. +> objective fails outright. See `desktop-client-v1.md` §4. **Objective:** ship a desktop application whose UI is not served by the hub, with durable local key storage, at feature parity with the web for a standard user, that does not @@ -832,7 +832,7 @@ prevent web use, and that can manage locally installed nodes. > > The security-per-effort ranking is: **11.5 ≫ 12 ≫ 14 (CLI) ≫ 13.** This phase is justified > on product grounds. It closes **C4** for native devices, but **not for accounts that also -> use a browser** — see `docs/desktop-client-v1.md` §5.1. Reproducible builds are unusually +> use a browser** — see `desktop-client-v1.md` §5.1. Reproducible builds are unusually > tractable here: the UI has no bundler or minifier, and Electron's own binaries have > published upstream hashes. @@ -871,7 +871,7 @@ of their keys. | 13.1 | Hub base URL + storage/save-file adapter | 1 | `HUB = ''` becomes configurable; a thin adapter for storage and saving. **Acceptance: the browser SPA behaves identically.** Much smaller than the old platform split — Chromium stays Chromium | | 13.2 | Electron shell | 1 | `app://` via `protocol.handle`, CSP, preload with an enumerated API, sandbox, single instance, tray, window state | | 13.3 | Local key storage + device auth | 1 | `safeStorage` (OS keychain) with an explicit fallback where no keyring exists; device Ed25519 auth on the `POST /v1/nodes/auth` pattern. **`POST /v1/users/auth` does not exist yet and must be written** | -| **13.3b** | **Device linking** | 1 | **Blocking.** One-time code generated by the new device, countersigned by an already-pinned key. `identities` gains a device dimension; `pin_identity`'s `INSERT OR REPLACE` must go. `docs/desktop-client-v1.md` §4 | +| **13.3b** | **Device linking** | 1 | **Blocking.** One-time code generated by the new device, countersigned by an already-pinned key. `identities` gains a device dimension; `pin_identity`'s `INSERT OR REPLACE` must go. `desktop-client-v1.md` §4 | | 13.3c | Hybrid registration | 1 | Register with a passphrase-derived `auth_key` (existing endpoint, browser-compatible at once, and the only account recovery path), device key thereafter | | ~~13.4~~ | aiortc client transport | — | **Deleted** — Chromium provides WebRTC. `transport.js` is kept as the client | | ~~13.5~~ | SQLite index cache | — | **Deleted** — IndexedDB works under `app://` | @@ -897,7 +897,7 @@ of their keys. **Revised 2026-08-17:** `transport.js`, `crypto.js`, `keyderive.js`, `downloads.js`, `sw.js` and the MSE path are **no longer on this list** — under Electron they are the client, not browser workarounds. The keypair-bundle deletions still apply, but only for accounts that -opt out of browser use (`docs/desktop-client-v1.md` §5.1); the browser path needs them. +opt out of browser use (`desktop-client-v1.md` §5.1); the browser path needs them. **Kept regardless:** WebRTC transport, hub signaling relay, DTLS channel binding. These carry NAT traversal and are not browser workarounds. @@ -929,7 +929,7 @@ a browser.** | # | Component | Description | |---|---|---| | 14.1 | `meshbay-node status` | ✅ DONE — hub, node public key, daemon state, groups, admin-key pinning. Reads the keystore directly so it works while the daemon is stopped | -| 14.1b | ~~`meshbay-node ui`~~ | **Removed 2026-09-01** (`docs/refactor-node-ui.md` phase 5). The server-rendered admin page it opened is gone; the CLI and the desktop client's Node page use the loopback control API directly | +| 14.1b | ~~`meshbay-node ui`~~ | **Removed 2026-09-01** (`refactor-node-ui.md` phase 5). The server-rendered admin page it opened is gone; the CLI and the desktop client's Node page use the loopback control API directly | | 14.1c | `meshbay-node gek-init` | ✅ DONE — initialises a group key via the daemon's loopback API. Was previously only possible by clicking a button in a browser on the node's own machine | | 14.2 | `meshbay-node group list` | ✅ **DONE 2026-08-18** — groups with roots, key state, file and peer counts | | 14.3 | `meshbay-node group create` | Create group on hub, add to config, generate GEK | @@ -958,7 +958,7 @@ remaining commands are written against a single internal module rather than besi authorization lives in one place, and 13.12 is adapter code. The refactor is cheaper now, with six commands left, than after 13.12 exists. -See `docs/desktop-client-v1.md` §6.6. +See `desktop-client-v1.md` §6.6. **Done 2026-08-18.** `meshbay_node/ops.py` holds each operation once; the loopback API is a one-line adapter per endpoint (`_op()` translates `OpError` into a JSON response) and the @@ -1015,7 +1015,7 @@ recipient's `pk_x25519` (the existing `wrap_gek_aes` primitive), or run the exis ### 15.0b — A sender key is per DEVICE, never per person (added 2026-08-17) -**This phase predates device linking (`docs/desktop-client-v1.md` §4) and is wrong as +**This phase predates device linking (`desktop-client-v1.md` §4) and is wrong as written.** One person now holds several identity keys on one node — a browser and a desktop client, up to the device cap. Two consequences, and the first is the whole decision: @@ -1067,11 +1067,11 @@ Three additions once devices exist, all of which belong in the user-facing docs: it, because the distribution channel has no forward secrecy. Not a regression — chat is plaintext at rest today — but it means Sender Keys is worth measurably less to a browser-using account than to a native one, which is the same asymmetry as everywhere - else in `docs/desktop-client-v1.md` §5.1. + else in `desktop-client-v1.md` §5.1. - **Sender authentication is now a requirement, not an accepted limitation** (operator decision, 2026-08-17). A sender key proves *a device*; it does not prove which account that device belongs to, and NS6's enforcement of `sender_id` from the session is - the node's word. The design is in `docs/desktop-client-v1.md` §4.8: **sign every message + the node's word. The design is in `desktop-client-v1.md` §4.8: **sign every message with the sender's device key** (independent of encryption, so it can land before this phase), **pin `account → device keys` client-side** using the device-add countersignatures as evidence, and optionally have the **operator sign a roster @@ -1090,7 +1090,7 @@ Three additions once devices exist, all of which belong in the user-facing docs: | 15.3 | Node: decrypt chat on receive | Decrypt incoming chat messages, handle out-of-order | | 15.4 | Key rotation on removal | Member removed **or device revoked or unpinned** → all remaining devices rotate | | 15.5 | Chat retention config | Per-group `max_age_days` setting, periodic cleanup in ChatStore | -| 15.6 | MNP version negotiation | Handshake declares a supported version range, not a single `v` field (L2 — today `v` is sent by everyone and checked by no one). **Move this earlier**: it pairs with the minimum-client-version check, and version skew begins the day the desktop client ships (`docs/desktop-client-v1.md` §2.6), not when chat is encrypted | +| 15.6 | MNP version negotiation | Handshake declares a supported version range, not a single `v` field (L2 — today `v` is sent by everyone and checked by no one). **Move this earlier**: it pairs with the minimum-client-version check, and version skew begins the day the desktop client ships (`desktop-client-v1.md` §2.6), not when chat is encrypted | | 15.7 | Chat attachments | Attachments are ordinary files on the node and remain plaintext at rest. Either encrypt them under the sender key, or document the asymmetry explicitly. Note they now land in the **operator-designated upload root** (§6.7 of the desktop-client doc) | --- @@ -1103,7 +1103,7 @@ Three additions once devices exist, all of which belong in the user-facing docs: > exists and, in two places, one that was deliberately dismantled. Corrections are listed > below rather than silently applied, because the same mistakes are easy to make twice. > -> **Shares the desktop design** (`docs/desktop-client-v1.md`): keys generated and kept +> **Shares the desktop design** (`desktop-client-v1.md`): keys generated and kept > locally, device Ed25519 authentication, no keypair bundles, and **an Android client is > simply another device** under device linking. Do not re-derive a second crypto, auth or > admission model here. @@ -1161,7 +1161,7 @@ use case appears. app and expect their backlog — which argues for the sealed state handover rather than the accept-and-explain option. - **Version skew is worse than on desktop.** An installed client meets a newer hub - (`docs/desktop-client-v1.md` §2.6), and store review latency means a fix cannot be + (`desktop-client-v1.md` §2.6), and store review latency means a fix cannot be pushed quickly. The minimum-client-version check is not optional here. - **Multi-root** falls out for free if the app is built after Stage A; it must not assume a group is one directory. @@ -1327,7 +1327,7 @@ silent overwrite that becomes a hole the moment more than one key per person is Chromium-class APIs, so a system webview meant reimplementing ~2500–3000 lines and losing the renderer sandbox. What is unchanged and non-negotiable: **UI assets ship inside the package and load from disk**, never fetched from the hub, or T3 is not - fixed. See `docs/desktop-client-v1.md` §2. + fixed. See `desktop-client-v1.md` §2. 18b. **A second device is admitted by device linking, not by an operator code.** The already-pinned key countersigns; the binding is a one-time code the new device generates and displays, never a human comparing digits. The hub cannot produce that diff --git a/first-review.md b/docs/first-review.md index dd1847c..fffac4c 100644 --- a/first-review.md +++ b/docs/first-review.md @@ -2,7 +2,7 @@ > Date: 2026-08-10 > Scope: design-level review of the cryptographic architecture, trust model, and -> security properties as specified in `docs/meshbay-draft-v3.md` and implemented +> security properties as specified in draft v3 (archived in `old-draft.md`) and implemented > through Phases 1-6 (81 tests, demo-v2 validated). > > This review does NOT assess the security of the demo/test deployment. It evaluates diff --git a/docs/meshbay-draft-v1-fr.md b/docs/meshbay-draft-v1-fr.md deleted file mode 100644 index 5f8ea6a..0000000 --- a/docs/meshbay-draft-v1-fr.md +++ /dev/null @@ -1,442 +0,0 @@ -# MeshBay — Brouillon d'Architecture v1 - -> Statut : brouillon préliminaire — de nombreux points restent ouverts, marqués [TBD] - ---- - -## 1. Présentation du projet - -MeshBay est une plateforme décentralisée, pair-à-pair, pour le partage de fichiers, le streaming vidéo et la messagerie de groupe. Elle combine une fédération d'identité (via les Mesh Hubs) avec un échange de données véritablement pair-à-pair (via les Mesh Nodes), dans l'objectif d'être résiliente, résistante à la censure et accessible aux utilisateurs. - -**Principes fondamentaux :** -- Les données ne transitent jamais par un serveur central — seuls l'identité et le routage le font -- Chiffrement de bout en bout pour tout contenu privé (fichiers, index, messages) -- L'opérateur du node est l'hébergeur légal et porte l'entière responsabilité de son contenu -- Le hub est un registrar léger, pas un hébergeur ni un indexeur de contenu -- Open source, auto-hébergeable à chaque niveau - -**Domaine :** meshbay.org - ---- - -## 2. Terminologie - -| Terme | Rôle | -|---|---| -| **Mesh Hub** | Serveur d'autorité d'identité et registre de groupes | -| **Mesh Node** | Programme local sur la machine de l'utilisateur hébergeur | -| **Mesh Client** | Navigateur web ou application Android (utilisateur final) | -| **Mesh Relay** | Relais TURN de secours opéré par la communauté | -| **MNP** | Mesh Node Protocol — protocole P2P entre nodes et clients | -| **MHP** | Mesh Bay Hub Protocol — protocole de fédération inter-hubs | -| **GEK** | Group Encryption Key — clé symétrique de chiffrement du groupe | -| **Mesh Directory** | Registre public des groupes (niveau hub) | -| **Mesh Group Index** | Listing chiffré des fichiers d'un groupe (niveau node) | - ---- - -## 3. Composants du système - -### 3.1 Mesh Hub - -Serveur léger agissant comme un registrar. Il est intentionnellement maintenu minimal pour réduire l'exposition légale et la charge opérationnelle. - -**Ce que le hub stocke :** -- Comptes utilisateurs : nom d'utilisateur, email haché, `PK_user` (empreinte de clé publique), ID du hub, statut -- Registre de groupes : nom, `PK_group`, adresse du node hébergeur, visibilité, liste des membres avec bundles GEK chiffrés -- Listes de révocation (utilisateurs et groupes) -- Hubs pairs enregistrés (liste d'autorisation explicite — pas de découverte automatique) - -**Ce que le hub ne stocke jamais :** -- Contenu de fichiers ou métadonnées -- Index de groupes privés -- Contenu de messages -- Adresses IP des nodes (gérées par le service de signaling éphémère) - -**Interactions hub — quand est-il sollicité ?** - -| Événement | Charge hub | Fréquence | -|---|---|---| -| Création de compte | Hash du credential, stockage PK | Une fois | -| Login | Vérification credentials, émission JWT signé | Par session (~30j de validité) | -| Création de groupe | Enregistrement nom, PK_group, node | Une fois par groupe | -| Ajout/suppression membre | Stockage/suppression bundle GEK chiffré | Sur action admin | -| Discovery d'un groupe | Retour adresse node + PK_node + bundle GEK | Par accès initial | -| Signaling NAT | Relais de quelques messages WebSocket (<1 Ko) | Par nouvelle connexion P2P | -| Recherche publique | Délégation de requête aux nodes à la demande | Sur demande | -| Sync fédération MHP | Échange mises à jour du Mesh Directory | Background, périodique | -| Révocation | Émission token de révocation signé | Rare | - -**Le hub n'est jamais dans le chemin des données après l'établissement de la connexion initiale.** - -**Le JWT comme passeport hors-ligne :** -Le hub émet un JWT signé avec sa clé privée Ed25519. Les nodes vérifient ce JWT localement en utilisant la clé publique connue du hub — aucun aller-retour hub requis par requête. Validité JWT : ~30 jours. - -**Stack technique :** -- Langage : Python -- Framework : FastAPI + Uvicorn -- Base de données : PostgreSQL + SQLAlchemy + Alembic -- Déploiement : derrière un reverse proxy Apache (ProxyPass) -- Authentification : système propre (JWT signé Ed25519, sans dépendance OAuth) - -**Création de compte :** [TBD] — email seul dans un premier temps, numéro de téléphone associable par la suite. Via l'app Android, les deux collectés par défaut. Comptes fusionnables. - -### 3.2 Mesh Node - -Programme local tournant sur la machine de l'utilisateur hébergeur. Le node est l'hébergeur effectif de tout le contenu. - -**Responsabilités :** -- Surveiller et indexer les répertoires partagés (Mesh Group Index) -- Servir fichiers et flux vidéo aux membres du groupe -- Gérer toutes les clés cryptographiques localement (keystore, protégé par mot de passe) -- Gérer les connexions P2P et la traversée NAT -- Exécuter le protocole MNP -- Héberger le sandbox de modules Python -- Servir l'interface web locale (localhost) -- [Futur] Recevoir et redistribuer une vidéo éphémère depuis mobile - -**Plateforme :** Linux en priorité, cross-platform dès le départ (Windows/macOS). Python assure la portabilité. - -**Stack technique :** -- Langage : Python (principal), extensions Rust uniquement si strictement nécessaire pour les parties critiques en performance -- QUIC : `aioquic` -- ICE/STUN : `aioice` -- WebRTC (futur) : `aiortc` -- Crypto : `cryptography` (PyCA, backed OpenSSL, accélération matérielle) -- Sérialisation : `msgpack` -- Compression : `zstandard` (zstd) -- Surveillance fichiers : `watchdog` -- BDD locale : SQLite -- Interface web locale : servie par le node sur localhost (port [TBD]) - -**Appairage node avec mobile :** QR code depuis l'interface web locale [futur]. - -### 3.3 Mesh Client - -Navigateur web ou application Android. Consomme le contenu depuis le node ; gère le compte via le hub. - -**Opérations côté hub :** -- Création de compte et login -- Recherche et découverte de groupes publics -- Gestion de l'appartenance aux groupes - -**Opérations côté node (P2P direct) :** -- Navigation dans les fichiers (Mesh Group Index) -- Lecture du fil de messages (avec pièces jointes, façon Signal) -- Téléchargement de fichiers -- Streaming vidéo (VOD) -- [Futur] Flux vidéo éphémère - -**Modes client** [à concevoir] : -- Mode explorateur : navigation dans les fichiers d'un groupe -- Mode flux : fil de messages avec pièces jointes -- Articulation UI hub/node à définir - -### 3.4 Mesh Relay - -Relais TURN opéré par la communauté. Utilisé uniquement en dernier recours quand toutes les méthodes de connexion P2P échouent. Le trafic est toujours chiffré E2E — le relais ne voit que des paquets QUIC opaques et ne peut pas lire le contenu. - -Non opéré par meshbay.org. Un protocole d'enregistrement des relais auprès des hubs est [TBD]. - ---- - -## 4. Modèle de groupe - -Les groupes sont l'unité organisationnelle centrale. - -| Paramètre | Options | -|---|---| -| Visibilité | Public / Privé | -| Politique d'adhésion | Libre / Sur demande / Sur invitation uniquement | -| Admin | L'opérateur du node hébergeur (hébergeur légal) | - -Un groupe public fonctionne comme un forum thématique : fichiers partagés, fil de discussion, liste de membres. Il peut être à entrée libre, sur demande ou sur invitation, indépendamment de sa visibilité publique. - -Le contenu d'un groupe privé (fichiers, index, messages) est toujours chiffré E2E avec la GEK. Seuls les membres possédant la GEK peuvent déchiffrer quoi que ce soit. - -**Adressage des groupes** [TBD] : -``` -meshbay.org/u/username/groupname — groupe public via hub -meshbay.org/g/groupname — groupe public direct -group://<PK_group_fingerprint>@<node_addr> — accès direct sans hub -``` - ---- - -## 5. Architecture cryptographique - -### 5.1 Hiérarchie de clés - -``` -Clé d'identité utilisateur Ed25519 Signature, authentification -Clé d'échange utilisateur X25519 Accord de clé -Clé d'identité groupe Ed25519 Signature métadonnées groupe (tenue par le node admin) -Clé de chiffrement groupe ChaCha20 Chiffrement contenu et index (symétrique, 256 bits) -Clés de session X25519/HKDF Perfect forward secrecy par connexion P2P -``` - -Toutes les clés privées sont stockées exclusivement sur le node (ou l'appareil client), dans un keystore local protégé par mot de passe. Le hub ne voit jamais aucune clé privée. - -### 5.2 Gestion de la GEK - -**Création de groupe :** -1. Le node admin génère la GEK (ChaCha20-Poly1305, 256 bits, CSPRNG) -2. La GEK est chiffrée pour chaque membre via accord de clé X25519 + HKDF -3. Les bundles GEK chiffrés sont stockés sur le hub (ou sur le node — [TBD]) - -**Ajout de membre :** -- GEK chiffrée avec la `PK_user` du nouveau membre et distribuée - -**Révocation de membre :** -- Le node admin génère une nouvelle GEK -- Re-chiffrement pour tous les membres restants -- Les nouveaux contenus sont chiffrés avec la nouvelle GEK -- L'ancien membre conserve la capacité de déchiffrer le contenu précédemment reçu (compromis acceptable — re-chiffrement complet non prévu) - -### 5.3 Chiffrement à la volée pour le transfert de fichiers - -Les fichiers sont stockés en clair sur le disque de l'hébergeur. Le node chiffre à la lecture avant transmission. - -``` -Disque (clair) → [Node] → compression zstd → chiffrement GEK (par chunk) → session QUIC → [Client] → déchiffrement QUIC → déchiffrement GEK → clair -``` - -**Stratégie de chunking :** -- Taille de chunk : 1 Mo (amortit l'overhead AEAD, permet le seek) -- Dérivation de clé par chunk : - `chunk_key = HKDF(GEK, "file:" || blake3(fichier) || "chunk:" || index)` -- Chaque chunk déchiffrable indépendamment (permet le seek vidéo) -- Compresser avant chiffrer (la compression zstd est inutile après chiffrement) - -**Authentification des chunks :** -Chaque chunk (ou lot) est signé avec la clé Ed25519 du node. Le client vérifie avant déchiffrement. Prévient l'injection de données par un relais compromis. - -### 5.4 Sécurité du transport - -- Protocole principal : **QUIC** (TLS 1.3 intégré, UDP, multiplexé) -- Clés de session par connexion via X25519 ECDH + HKDF -- La couche QUIC est indépendante de la couche applicative GEK — deux couches de chiffrement indépendantes - -### 5.5 Chiffrement du chat - -La messagerie de groupe utilise l'algorithme **Double Ratchet** (comme Signal) : -- Forward secrecy et break-in recovery par message -- Chaque message chiffré indépendamment -- Implémentation : bibliothèque Python ou Rust existante [TBD] - ---- - -## 6. Réseau et connectivité - -### 6.1 Traversée NAT — ordre des tentatives - -``` -1. IPv6 disponible des deux côtés → connexion directe, aucun problème NAT -2. UPnP / NAT-PMP sur le routeur → le node ouvre un port automatiquement -3. ICE + STUN / UDP hole punching → fonctionne pour ~80-85% des cas -4. Mesh Relay (fallback TURN) → opéré par la communauté, trafic E2E chiffré -``` - -**Signaling** (étapes 3/4) : coordonné via WebSocket du hub, <1 Ko par tentative, sans état après connexion établie. - -**Couverture étape 4 :** ~15-20% des connexions (NAT symétrique des deux côtés, CGNAT). Le relais ne voit que des paquets QUIC chiffrés. - -### 6.2 MNP — Mesh Node Protocol - -Protocole applicatif sur QUIC. Blocs définis : - -- **Handshake** : échange de clés, vérification d'appartenance au groupe (présentation JWT) -- **Sync d'index** : delta de Mesh Group Index chiffré à la connexion -- **Transfert de fichiers** : requête/réponse par chunk avec vérification de hash -- **Streaming VOD** : segments HLS/DASH, chiffrés par segment avec des clés dérivées de la GEK -- **Messagerie** : messages Double Ratchet encapsulés dans des frames MNP -- **[Futur] Flux éphémère** : type `ephemeral_stream` avec métadonnées TTL - -### 6.3 Diffusion de contenu public - -Les fichiers publics sont identifiés par leur hash `blake3`. Plusieurs nodes peuvent servir le même fichier : - -1. Le Node A possède le fichier public X (hash H) -2. Tout node qui obtient X et choisit de le mirrorer s'enregistre auprès du hub : "je sers le hash H" -3. Le hub maintient : `{ blake3_hash → [node_A, node_B, ...] }` -4. Un client demande X → le hub retourne la liste des sources → le client récupère des chunks en parallèle depuis plusieurs nodes - -**Transport contenu public :** TLS uniquement (pas de GEK). Contenu signé avec la clé Ed25519 du node original pour vérification d'authenticité par les clients, même servi depuis un miroir. Possibilité laissée ouverte d'ajouter une GEK pour des groupes "publics réservés aux inscrits" dans une révision future. - ---- - -## 7. Index - -### 7.1 Mesh Directory (niveau hub) - -Registre public des groupes. Échangé entre hubs via MHP. - -Format : msgpack, signé par la clé Ed25519 du hub. - -Champs par entrée : nom de groupe, `PK_group`, hub hébergeur, description, tags de type de contenu, politique d'adhésion. - -### 7.2 Mesh Group Index (niveau node) - -Listing des fichiers d'un groupe. Généré et maintenu par le node hébergeur. - -Format : msgpack → compressé zstd → chiffré GEK (groupes privés) ou signé en clair (groupes publics). - -Structure d'une entrée : -```python -{ - "id": "<blake3_hash>", - "name": "fichier.mkv", - "path": "Films/2024/", # relatif au répertoire partagé - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # secondes, pour les médias - "thumb_hash":"<blake3>", # hash de la miniature (miniature aussi chiffrée GEK) - "added_at": 1720000000 -} -``` - -**Mises à jour delta :** chaque mise à jour porte `{base_version, additions, deletions}` — pas de re-chiffrement complet à chaque changement. - -**Transit :** les nodes poussent les deltas d'index aux membres connectés sur modification. Les membres tirent l'index complet à la première connexion. Le hub ne stocke aucun contenu d'index — seulement l'adresse du node pour le routage. - -### 7.3 Recherche - -**Groupes privés :** la recherche est entièrement locale sur l'appareil du client. Le client maintient un cache local chiffré de tous les index des groupes dont il est membre. Aucun appel réseau, aucune implication du hub, résultats instantanés. - -**Groupes publics :** le client interroge les nodes directement à la demande. Le hub fournit le routage (quel node héberge quel groupe) mais n'effectue aucune recherche de contenu lui-même. - -**Interface web du hub — recherche :** délègue la requête aux nodes concernés à la demande. Le hub ne stocke rien de cette interaction. Micro-cache en mémoire des résultats : **TTL 60 secondes maximum, RAM uniquement, jamais écrit sur disque, contenu public uniquement.** Ceci relève du caching technique (DSA EU Article 13) et ne constitue pas de l'indexation. - ---- - -## 8. Fédération inter-hubs (MHP) - -### 8.1 Hiérarchie des hubs - -``` -Root Hub (meshbay.org) - ├── Full Hub (auto-hébergé, CA déléguée) - │ └── émet des credentials utilisateurs, gère ses propres groupes - │ └── peut se fédérer avec d'autres Full Hubs via MHP - └── Mirror Hub - └── héberge uniquement le Mesh Directory public (pas de comptes utilisateurs) -``` - -Un Full Hub reçoit un certificat signé par le Root Hub (ou un Full Hub parent) prouvant son autorité. Les clients vérifient la chaîne. Un Mirror Hub ne peut que répliquer des données publiques. - -### 8.2 Principes de conception MHP - -- Sélection explicite des pairs : chaque hub maintient une liste d'autorisation de hubs de confiance -- Pas de découverte automatique de hubs -- Données échangées : Mesh Directory (groupes publics), listes de révocation, credentials utilisateurs cross-hub -- Authentification cross-hub : l'utilisateur du Hub A présente un JWT signé par Hub A ; Hub B vérifie en utilisant la clé publique de Hub A (récupérée une fois à la première interaction, mise en cache) - -### 8.3 Accès client cross-hub - -Client de Hub A accédant à un groupe sur Hub B : -1. Le Mesh Directory de Hub A ou un lien direct amène le client vers Hub B -2. Le client présente son JWT Hub A directement à Hub B -3. Hub B vérifie la signature JWT avec la clé publique de Hub A -4. Hub B émet un token local de courte durée pour cette session -5. Le client rejoint le node normalement - ---- - -## 9. Modération - -### 9.1 Contenu public - -``` -Signalement #1 → suspension automatique de l'accès public au contenu - → notification à l'opérateur du node -Une republication autorisée -Signalement #2 → escalade vers les modérateurs du hub -Confirmé → groupe révoqué sur le hub local - → révocation propagée aux hubs fédérés via MHP -``` - -Mécanisme : hash blake3 du contenu ajouté à la liste de blocage du hub. Le node reçoit un avis de révocation signé et coupe l'accès public. - -### 9.2 CSAM - -Hash matching contre la base de données NCMEC/IWF sur tout contenu public lors de l'enregistrement. La participation démontre la bonne foi et réduit significativement l'exposition légale. Pas de scanning de contenu privé/chiffré. - -### 9.3 Copyright - -Cadre de notification légale DMCA/équivalent (takedown sur notification). Pas de blocage technique automatique — trop complexe, trop de faux positifs (fair use, variations régionales). Le hub peut révoquer sur demande légale confirmée. - -### 9.4 Contenu privé - -Non modérable directement (chiffré E2E par conception). Seule action disponible : révoquer l'utilisateur ou le groupe au niveau du hub sur demande légale formelle. Le hub émet un token de révocation signé que les nodes de tous les membres peuvent vérifier. - ---- - -## 10. Système de modules Python - -Le node peut charger des modules d'extension (Python) s'exécutant dans un sous-processus sandbox. - -**Manifeste de module** (capacités déclarées) : -```python -{ - "name": "group-chat", - "version": "1.0.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**APIs disponibles (restreintes) :** -- `read_index()` — lecture de l'index courant du groupe (lecture seule) -- `send_message(content)` — poster un message dans le fil du groupe -- `receive_events(handler)` — s'abonner aux événements du groupe (nouveau fichier, nouveau message) - -**Non disponible :** -- Accès réseau arbitraire -- Accès au système de fichiers hors du contexte du groupe -- Appels système - -**Premier module officiel :** fil de discussion de groupe (façon Signal, avec pièces jointes). Fourni avec le node. - ---- - -## 11. Cadre légal - -**Opérateur du node :** hébergeur légal principal du contenu. Entièrement responsable de ce qu'il partage. Le logiciel node communique clairement cela lors de l'installation. - -**Opérateur du hub :** registrar, pas hébergeur de contenu. Stocke un minimum de données personnelles. Opère le mécanisme de takedown. Participe au hash matching CSAM. Exposition légale analogue à celle d'un bureau d'enregistrement de domaines. - -**Auteur du protocole/logiciel :** protégé par les usages non-contrefaisants substantiels. Pas de facilitation active de l'infraction. - -**Minimisation des données du hub :** -- Email stocké haché après vérification [TBD] -- Pas de journalisation des IP (ou suppression automatique après 24h) -- Aucune métadonnée de contenu stockée -- Adresse courante du node gérée uniquement par le service de signaling éphémère - ---- - -## 12. Fonctionnalités futures (notées, non conçues) - -- **Réplication de contenu entre nodes :** node-à-node, autorisée par l'admin, sans implication du hub -- **Push vidéo depuis mobile :** mobile filme → pousse vers le node hébergeur → distribué comme flux éphémère avec TTL aux membres du groupe. Type MNP `ephemeral_stream` réservé. -- **Protocole d'enregistrement des Mesh Relays :** relais TURN communautaires enregistrés auprès des hubs -- **Appairage node-mobile :** QR code depuis l'interface web locale -- **Téléchargement multi-sources :** récupération de chunks en parallèle depuis plusieurs nodes pour un même fichier public (swarm) -- **Client iOS** -- **Chiffrement at-rest sur le node :** optionnel, pour les nodes déployés sur des serveurs distants - ---- - -## 13. Questions ouvertes [TBD] - -1. **Stockage des bundles GEK :** sur le hub ou sur le node uniquement ? Hub = discovery plus facile ; node uniquement = plus décentralisé -2. **Schéma d'adressage des groupes :** format URL final -3. **Périmètre de l'interface web locale du hub pour la V1 :** configuration uniquement, ou aussi navigation dans les groupes ? -4. **Création de compte :** email seul pour commencer, téléphone associable — à confirmer -5. **Implémentation du chat :** module bundlé ou fonctionnalité core ? -6. **Maturité de la lib QUIC :** évaluation de `aioquic` en production à effectuer -7. **Bibliothèque Double Ratchet :** identifier la meilleure implémentation Python -8. **Protocole d'enregistrement des relais :** à concevoir lors de l'introduction des relais communautaires -9. **Échange de répertoire cross-hub :** fréquence, résolution de conflits -10. **Port de l'interface web locale du node :** à définir -11. **Stratégie d'expiration et renouvellement des JWT** -12. **Format du keystore et mécanisme de déverrouillage au démarrage du node** diff --git a/docs/meshbay-draft-v1.md b/docs/meshbay-draft-v1.md deleted file mode 100644 index 1eb1758..0000000 --- a/docs/meshbay-draft-v1.md +++ /dev/null @@ -1,442 +0,0 @@ -# MeshBay — Architecture Draft v1 - -> Status: preliminary draft — many points still open, marked [TBD] - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), aiming to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | Encrypted file listing for a group (node level) | - ---- - -## 3. System Components - -### 3.1 Mesh Hub - -A lightweight server acting as a registrar. It is intentionally kept minimal to reduce legal exposure and operational burden. - -**What the hub stores:** -- User accounts: username, hashed email, `PK_user` (public key fingerprint), hub ID, status -- Group registry: name, `PK_group`, hosting node address, visibility, member list with encrypted GEK bundles -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node IP addresses (handled by ephemeral signaling service) - -**Hub interactions — when is it called?** - -| Event | Hub load | Frequency | -|---|---|---| -| Account creation | Hash credential, store PK | Once | -| Login | Verify credentials, issue signed JWT | Per session (~30-day validity) | -| Group creation | Register name, PK_group, node | Once per group | -| Member add/remove | Store/remove encrypted GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay a few WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate query to nodes at request time | On demand | -| MHP federation sync | Exchange public directory updates | Background, periodic | -| Revocation | Issue signed revocation token | Rare | - -**The hub is never in the data path after initial connection setup.** - -**JWT as offline passport:** -The hub issues a JWT signed with its Ed25519 private key. Nodes verify this JWT locally using the hub's known public key — no hub roundtrip required per request. JWT validity: ~30 days. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: behind Apache reverse proxy (ProxyPass) -- Authentication: own system (JWT signed with Ed25519, no OAuth dependency) - -**Account creation:** [TBD] — email only at first, phone number associable later. Via Android app, both collected by default. Fusionable accounts. - -### 3.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) -- Serve files and video streams to group members -- Manage all cryptographic keys locally (keystore, password-protected) -- Handle P2P connections and NAT traversal -- Run the MNP protocol -- Host the Python module sandbox -- Serve the local web UI (localhost) -- Receive and redistribute ephemeral video from mobile [future] - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -**Tech stack:** -- Language: Python (primary), Rust extensions only if strictly necessary for hot paths -- QUIC: `aioquic` -- ICE/STUN: `aioice` -- WebRTC (future): `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by the node on localhost (port [TBD]) - -**Node pairing with mobile:** QR code from local web UI [future]. - -### 3.3 Mesh Client - -Web browser or Android app. Consumes content from the node; manages account via the hub. - -**Hub-side operations (via hub):** -- Account creation and login -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing (Mesh Group Index) -- Message feed reading (with attachments, Signal-like) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: browse files in a group -- Feed mode: message thread with attachments -- Hub/Node UI split to be defined - -### 3.4 Mesh Relay - -Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail. Traffic is always E2E encrypted — the relay sees only opaque QUIC packets and cannot read content. - -Not operated by meshbay.org. A protocol for relay registration with hubs is [TBD]. - ---- - -## 4. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: shared files, message thread, member list. It can be open entry, request-based, or invitation-only regardless of its public visibility. - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members with the GEK can decrypt anything. - -**Group addressing** [TBD]: -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — direct public group -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` - ---- - -## 5. Cryptographic Architecture - -### 5.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys are stored exclusively on the node (or client device), in a password-protected local keystore. The hub never sees any private key. - -### 5.2 GEK Management - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK is encrypted for each member using X25519 key agreement + HKDF -3. Encrypted GEK bundles stored on hub (or node — [TBD]) - -**Member addition:** -- GEK encrypted with new member's `PK_user` and distributed - -**Member revocation:** -- Admin node generates a new GEK -- Re-encrypts for all remaining members -- New content encrypted with new GEK -- Former member retains ability to decrypt previously received content (acceptable trade-off — full re-encryption not planned) - -### 5.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time before transmission. - -``` -Disk (plaintext) → [Node] → zstd compress → GEK encrypt (per-chunk) → QUIC session → [Client] → QUIC decrypt → GEK decrypt → plaintext -``` - -**Chunking strategy:** -- Chunk size: 1 MB (amortizes AEAD overhead, allows seeking) -- Per-chunk key derivation: - `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` -- Each chunk independently decryptable (enables video seeking) -- Compress before encrypt (zstd compression is useless after encryption) - -**Chunk authentication:** -Each chunk (or batch) is signed with the node's Ed25519 key. The client verifies before decryption. Prevents data injection by a compromised relay. - -### 5.4 Transport Security - -- Primary protocol: **QUIC** (TLS 1.3 integrated, UDP-based, multiplexed) -- Per-connection session keys via X25519 ECDH + HKDF -- The QUIC layer is independent from the GEK application layer — two independent encryption layers - -### 5.5 Chat Encryption - -Group messaging uses the **Double Ratchet algorithm** (as in Signal): -- Forward secrecy and break-in recovery per message -- Each message independently encrypted -- Implementation: existing Python or Rust library [TBD] - ---- - -## 6. Network and Connectivity - -### 6.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection, no NAT issue -2. UPnP / NAT-PMP on router → node opens port automatically -3. ICE + STUN / UDP hole punching → works for ~80-85% of cases -4. Mesh Relay (TURN fallback) → community-operated, E2E encrypted traffic -``` - -**Signaling** (steps 3/4): coordinated via hub WebSocket, <1 KB per attempt, stateless after connection established. - -**Step 4 coverage:** ~15-20% of connections (symmetric NAT on both sides, CGNAT). The relay sees only encrypted QUIC packets. - -### 6.2 MNP — Mesh Node Protocol - -Application-level protocol over QUIC. Defined blocks: - -- **Handshake**: key exchange, group membership verification (JWT presentation) -- **Index sync**: encrypted delta Mesh Group Index on connection -- **File transfer**: chunk request/response with hash verification -- **VOD streaming**: HLS/DASH segments, encrypted per-segment with GEK-derived keys -- **Messaging**: Double Ratchet messages encapsulated in MNP frames -- **[Future] Ephemeral stream**: `ephemeral_stream` type with TTL metadata - -### 6.3 Public Content Delivery - -Public files are identified by their `blake3` hash. Multiple nodes can serve the same file: - -1. Node A has public file X (hash H) -2. Any node that obtains X and chooses to mirror it registers with the hub: "I serve hash H" -3. Hub maintains: `{ blake3_hash → [node_A, node_B, ...] }` -4. Client requests X → hub returns source list → client fetches in parallel chunks from multiple nodes - -**Public content transport:** TLS only (no GEK). Content signed with the original node's Ed25519 key for authenticity verification by clients, even when served from a mirror. Door left open for GEK on "registered-users-only public" groups in a future revision. - ---- - -## 7. Indexes - -### 7.1 Mesh Directory (hub level) - -Public registry of groups. Exchanged between hubs via MHP. - -Format: msgpack, signed by hub's Ed25519 key. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy. - -### 7.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: msgpack → zstd compressed → GEK encrypted (private groups) or plaintext signed (public groups). - -Entry structure: -```python -{ - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", # relative to shared directory - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash":"<blake3>", # thumbnail hash (thumbnail also GEK-encrypted) - "added_at": 1720000000 -} -``` - -**Delta updates:** each update carries `{base_version, additions, deletions}` — no full re-encryption on every change. - -**Transit:** nodes push index deltas to connected members on change. Members pull full index on first connection. Hub stores no index content — only the node address for routing. - -### 7.3 Search - -**Private groups:** search is entirely local on the client device. The client maintains a local encrypted cache of all indexes for groups it belongs to. No network call, no hub involvement, instant results. - -**Public groups:** client queries node(s) directly at request time. Hub provides routing (which node hosts which group) but performs no content lookup itself. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this interaction. In-memory micro-cache of results: **60-second TTL maximum, RAM only, never written to disk, public content only.** This qualifies as technical caching (EU DSA Article 13) and does not constitute indexing. - ---- - -## 8. Hub Federation (MHP) - -### 8.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, CA-delegated) - │ └── issues user credentials, manages its own groups - │ └── can federate with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub) proving its authority. Clients verify the chain. A Mirror Hub can only replicate public directory data. - -### 8.2 MHP Design Principles - -- Explicit peer selection: each hub maintains an allowlist of trusted peer hubs -- No automatic hub discovery -- Exchanged data: Mesh Directory (public groups), revocation lists, cross-hub user credentials -- Cross-hub authentication: user from Hub A presents JWT signed by Hub A; Hub B verifies using Hub A's public key (fetched once on first interaction, cached) - -### 8.3 Cross-Hub Client Access - -Client from Hub A accessing a group on Hub B: -1. Hub A's public directory or direct link leads client to Hub B -2. Client presents Hub A JWT to Hub B directly -3. Hub B verifies JWT signature using Hub A's public key -4. Hub B issues a short-lived local token for this session -5. Client proceeds to node as normal - ---- - -## 9. Moderation - -### 9.1 Public Content - -``` -Report #1 → automatic suspension of public access to content - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: blake3 hash of content added to hub blocklist. Node receives signed revocation notice and cuts public access. - -### 9.2 CSAM - -Hash matching against NCMEC/IWF database on all public content at registration time. Participation demonstrates good faith and significantly reduces legal exposure. No scanning of private/encrypted content. - -### 9.3 Copyright - -DMCA/legal notice framework (takedown on notification). No automated technical blocking — too complex, too many false positives (fair use, regional variations). Hub can revoke on confirmed legal request. - -### 9.4 Private Content - -Not directly moderatable (E2E encrypted by design). Only action available: revoke user or group at hub level on formal legal request. Hub issues a signed revocation token that all group members' nodes can verify. - ---- - -## 10. Python Module System - -The node can load extension modules (Python) that run in a sandboxed subprocess. - -**Module manifest** (declared capabilities): -```python -{ - "name": "group-chat", - "version": "1.0.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs (restricted):** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post a message to the group thread -- `receive_events(handler)` — subscribe to group events (new file, new message) - -**Not available:** -- Arbitrary network access -- Filesystem access outside the group context -- System calls - -**First official module:** group chat thread (Signal-like, with attachments). Bundled with node. - ---- - -## 11. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node software clearly communicates this at setup. - -**Hub operator:** registrar, not content host. Stores minimal PII. Operates takedown mechanism. Participates in CSAM hash matching. Analogous to a domain registrar in legal exposure terms. - -**Protocol/software author:** protected by substantial non-infringing uses. No active facilitation of infringement. - -**Hub data minimization:** -- Email stored hashed after verification [TBD] -- No IP address logging (or auto-deletion after 24h) -- No content metadata stored -- Node current address managed by ephemeral signaling service only - ---- - -## 12. Future Features (noted, not designed) - -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Mobile video push:** mobile films → pushes to hosting node → distributed as ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Mesh Relay registration protocol:** community TURN relays registered with hubs -- **Node mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from multiple nodes for same public file (swarm) -- **iOS client** -- **At-rest encryption on node:** optional, for nodes deployed on remote servers - ---- - -## 13. Open Questions [TBD] - -1. **GEK bundle storage:** on hub or on node only? Hub = easier discovery; node only = more decentralized -2. **Group address scheme:** final URL format -3. **Hub local web UI scope for V1:** config only, or also group browsing? -4. **Account creation:** email only to start, phone associable — confirm -5. **Chat implementation:** bundled module or core feature? -6. **QUIC library maturity:** `aioquic` production readiness assessment needed -7. **Double Ratchet library:** identify best Python implementation -8. **Relay registration protocol:** design when community relays are introduced -9. **Cross-hub directory exchange:** frequency, conflict resolution -10. **Node port for local web UI:** to assign -11. **JWT expiry and refresh strategy** -12. **Keystore format and unlock mechanism on node startup** diff --git a/docs/meshbay-draft-v2-fr.md b/docs/meshbay-draft-v2-fr.md deleted file mode 100644 index c61203e..0000000 --- a/docs/meshbay-draft-v2-fr.md +++ /dev/null @@ -1,550 +0,0 @@ -# MeshBay — Brouillon d'Architecture v2 - -> Statut : brouillon préliminaire — points ouverts marqués [TBD] -> Changements depuis v1 : logs IP (légal), versionnage des protocoles, dimensionnement matériel, stratégie JWT, propositions keystore, chat en core, relay déplacé en futur, miroir hub en futur, GEK clarifié, port 18000, lazy admin keystore. - ---- - -## 1. Présentation du projet - -MeshBay est une plateforme décentralisée pair-à-pair pour le partage de fichiers, le streaming vidéo et la messagerie de groupe. Elle combine une fédération d'identité (via les Mesh Hubs) avec un échange de données véritablement pair-à-pair (via les Mesh Nodes), conçue pour être résiliente, résistante à la censure et accessible aux utilisateurs. - -**Principes fondamentaux :** -- Les données ne transitent jamais par un serveur central — seuls l'identité et le routage le font -- Chiffrement de bout en bout pour tout contenu privé (fichiers, index, messages) -- L'opérateur du node est l'hébergeur légal et porte l'entière responsabilité de son contenu -- Le hub est un registrar léger, pas un hébergeur ni un indexeur -- Open source, auto-hébergeable à chaque niveau - -**Domaine :** meshbay.org (configurable dans tout le code source) - ---- - -## 2. Terminologie - -| Terme | Rôle | -|---|---| -| **Mesh Hub** | Serveur d'autorité d'identité et registre de groupes | -| **Mesh Node** | Programme local sur la machine de l'utilisateur hébergeur | -| **Mesh Client** | Navigateur web ou application Android (utilisateur final) | -| **Mesh Relay** | Relais TURN communautaire [futur] | -| **MNP** | Mesh Node Protocol — protocole P2P entre nodes et clients | -| **MHP** | Mesh Bay Hub Protocol — protocole de fédération inter-hubs | -| **GEK** | Group Encryption Key — clé symétrique de chiffrement des groupes privés | -| **Mesh Directory** | Registre public des groupes (niveau hub) | -| **Mesh Group Index** | Listing des fichiers d'un groupe (niveau node, chiffré pour les groupes privés) | - ---- - -## 3. Versionnage des protocoles - -Tous les protocoles (MNP, MHP, API REST du hub) portent une information de version explicite. - -**Format :** `MAJOR.MINOR` -- Incrément MAJOR : changement cassant, incompatible -- Incrément MINOR : ajout rétrocompatible - -**Négociation :** lors du handshake, les deux parties déclarent leur plage de versions supportées. Le MINOR le plus élevé mutuellement supporté au sein du même MAJOR est utilisé. En l'absence de version commune, la connexion est refusée avec une erreur explicite. - -**Politique de support :** une version supporte le MAJOR courant et au moins les deux MINOR précédents (N-2). - -**Implémentation :** champ `version` dans chaque en-tête de message msgpack. L'étape de handshake précède tous les autres échanges. - ---- - -## 4. Composants du système - -### 4.1 Mesh Hub - -Serveur léger agissant comme un registrar. Intentionnellement minimal pour limiter l'exposition légale et le coût opérationnel. - -**Ce que le hub stocke :** -- Comptes utilisateurs : nom d'utilisateur, email (conservé pour la récupération de compte — voir §4.1.1), numéro de téléphone optionnel, `PK_user`, ID du hub, statut, timestamp de création -- Registre de groupes : nom, `PK_group`, identifiant du node hébergeur, visibilité, politique d'adhésion, liste des membres avec bundles GEK chiffrés (groupes privés uniquement) -- Logs de connexion obligatoires (voir §4.1.2) -- Listes de révocation (utilisateurs et groupes) -- Hubs pairs enregistrés (liste d'autorisation explicite — pas de découverte automatique) - -**Ce que le hub ne stocke jamais :** -- Contenu de fichiers ou métadonnées -- Index de groupes privés -- Contenu de messages -- IP courante des nodes (gérée par le signaling éphémère — voir §4.1.3) - -#### 4.1.1 Données de compte - -L'email est conservé en clair (non haché) pour permettre : -- La récupération de compte (réinitialisation de mot de passe) -- Les notifications légales -- Le contact en cas d'abus - -Numéro de téléphone : optionnel, associable après la création du compte. Sur Android, les deux sont collectés à l'inscription. Les comptes sont fusionnables (email + téléphone pointant vers le même compte). - -Email et téléphone sont stockés chiffrés au repos dans la base de données. - -#### 4.1.2 Logs IP obligatoires (conformité légale) - -Les cadres légaux (LCEN en France, directive e-Commerce UE, DSA) imposent aux prestataires de conserver des logs de connexion. Le hub enregistre les événements suivants avec horodatage et adresse IP : - -| Événement | Rétention | -|---|---| -| Création de compte | 1 an minimum | -| Login (succès et échec) | 1 an minimum | -| Création de groupe | 1 an minimum | -| Adhésion / départ d'un groupe | 1 an minimum | -| Suppression de groupe | 1 an minimum | -| Actions de révocation | 1 an minimum | - -Les logs sont stockés dans une table séparée à accès contrôlé. Ils ne sont utilisés qu'à des fins de conformité légale et ne sont pas exposés aux utilisateurs ou opérateurs sauf sur demande légale. - -#### 4.1.3 Service de signaling - -La coordination de la traversée NAT est gérée par un endpoint WebSocket léger, logiquement séparé de l'API principale du hub. Il est sans état : l'état de connexion est maintenu uniquement en mémoire et effacé après l'établissement de la connexion P2P (typiquement en quelques secondes). Aucun stockage persistant des IP des nodes. - -**Résumé des interactions hub :** - -| Événement | Charge crypto hub | Fréquence | -|---|---|---| -| Création de compte | Hash Argon2, stockage PK | Une fois | -| Login | Vérification mot de passe, émission JWT (signature Ed25519) | Par session | -| Création de groupe | Enregistrement métadonnées | Une fois par groupe | -| Ajout/suppression membre | Stockage/suppression bundle GEK | Sur action admin | -| Discovery de groupe | Retour adresse node + PK_node + bundle GEK | Par accès initial | -| Signaling NAT | Relais messages WebSocket (<1 Ko) | Par nouvelle connexion P2P | -| Recherche publique | Délégation aux nodes, cache 60s en mémoire | Sur demande | -| Sync fédération MHP | Échange Mesh Directory | Background, périodique | -| Révocation | Signature Ed25519 token de révocation | Rare | - -**Le hub n'est jamais dans le chemin des données après l'établissement de la connexion. La vérification des JWT par les nodes est locale (Ed25519, aucun aller-retour hub).** - -#### 4.1.4 Stratégie JWT - -Deux tokens émis à la connexion : - -**Access token** (JWT, signé Ed25519) : -- Validité : 1 heure -- Payload : `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, claim d'appartenance aux groupes signé par le hub -- Présenté aux nodes pour authentification et vérification d'accès aux groupes -- Vérifié localement par les nodes avec la clé publique connue du hub — aucun aller-retour hub -- Fenêtre de compromission : 1 heure maximum - -**Refresh token** (opaque, 256 bits aléatoires) : -- Validité : 30–90 jours [TBD durée exacte] -- Stocké de manière sécurisée côté client uniquement -- Utilisé exclusivement avec le hub pour obtenir un nouvel access token -- Révocable immédiatement par le hub (invalide tous les renouvellements futurs pour ce token) -- Stocké côté serveur sous forme de valeur hachée - -**Flux de révocation :** le hub invalide le refresh token → le prochain renouvellement d'access token échoue → l'accès aux nodes expire au plus dans 1 heure. - -**Stack technique :** -- Langage : Python -- Framework : FastAPI + Uvicorn -- Base de données : PostgreSQL + SQLAlchemy + Alembic -- Déploiement : reverse proxy Apache (ProxyPass + terminaison SSL) -- Authentification : système propre (JWT Ed25519, Argon2id pour le hachage des mots de passe) -- Hub accessible par domaine et par IP directe (avertissement certificat auto-signé attendu pour l'accès par IP ; documenté) - -### 4.2 Mesh Node - -Programme local sur la machine de l'utilisateur hébergeur. Le node est l'hébergeur effectif de tout le contenu. - -**Responsabilités :** -- Surveiller et indexer les répertoires partagés (Mesh Group Index) -- Servir fichiers et flux vidéo aux membres du groupe -- Gérer toutes les clés cryptographiques localement (keystore chiffré) -- Gérer les connexions P2P et la traversée NAT -- Exécuter le protocole MNP -- Héberger le sandbox de modules Python d'extension -- Servir l'interface web locale (localhost:18000) -- Héberger le chat de groupe (fonctionnalité core) - -**Plateforme :** Linux en priorité, cross-platform dès le départ (Windows/macOS). Python assure la portabilité. - -#### 4.2.1 Keystore et déverrouillage - -Les clés privées (identité utilisateur, identité groupe, copies GEK) sont stockées dans un fichier keystore local chiffré. - -**Format :** conteneur msgpack chiffré avec AES-256-GCM, clé dérivée du mot de passe maître par Argon2id (paramétré pour ~1s de dérivation sur le matériel cible). - -**Trois modes de déverrouillage :** - -| Mode | Fonctionnement | Niveau de sécurité | -|---|---|---| -| **Sécurisé (défaut)** | Mot de passe saisi au démarrage via terminal ou interface web locale | Élevé | -| **Lazy (fichier)** | Mot de passe ou clé dérivée stocké dans `~/.config/meshbay/unlock.key` (chmod 600), lu automatiquement au démarrage | Moyen — acceptable pour une machine home physiquement sécurisée. Risque documenté lors de la configuration. | -| **Service (headless)** | Variable d'environnement `MESHBAY_UNLOCK_KEY`, définie via `EnvironmentFile=` systemd pointant vers un fichier chmod 600 | Moyen-élevé — pratique standard pour les déploiements serveur | - -Futur : intégration keychain OS (libsecret/GNOME Keyring sur Linux, Windows Credential Manager, Keychain macOS). - -#### 4.2.2 Dimensionnement matériel - -La contrainte principale est la **bande passante montante**, pas le CPU ou la RAM. - -| Scénario | Utilisateurs simultanés | Upload requis | CPU | RAM | -|---|---|---|---|---| -| Fichiers + chat, peu de streaming | 10 | 20–50 Mbps | 2 cœurs | 512 Mo | -| Streaming 1080p actif (5–6 flux) | 10 | 50–80 Mbps | 2–4 cœurs | 1 Go | -| Usage mixte | 50 | 200–300 Mbps | 4 cœurs | 2 Go | -| Streaming actif | 50 | 400 Mbps | 4–8 cœurs | 2–4 Go | -| Tous usages | 100 | 800 Mbps–1 Gbps | 8 cœurs | 4–8 Go | - -Au-delà de 20–30 utilisateurs en streaming actif, un serveur dédié est nécessaire. Une connexion fibre domestique (100–500 Mbps symétrique) convient pour un petit groupe. - -**Stack technique :** -- Langage : Python (principal). Extension Rust uniquement si un chemin critique s'avère insuffisant. -- Couche d'abstraction transport : interface `Transport` découplant QUIC du fallback TCP+TLS -- QUIC : `aioquic` (maintenu par des ingénieurs Cloudflare). Fallback : TCP + TLS 1.3 + HTTP/2 si QUIC s'avère insuffisant en production -- ICE/STUN : `aioice` -- WebRTC [futur] : `aiortc` -- Crypto : `cryptography` (PyCA, backed OpenSSL, accélération matérielle AES-NI/ChaCha) -- Sérialisation : `msgpack` -- Compression : `zstandard` (zstd) -- Surveillance fichiers : `watchdog` -- BDD locale : SQLite -- Interface web locale : servie par le node sur `localhost:18000` - -### 4.3 Mesh Client - -Navigateur web ou application Android. Consomme le contenu depuis les nodes ; gère le compte via le hub. - -**Opérations côté hub :** -- Création de compte et login (Android : email + téléphone à l'inscription) -- Recherche et découverte de groupes publics -- Gestion de l'appartenance aux groupes - -**Opérations côté node (P2P direct) :** -- Navigation dans les fichiers via Mesh Group Index -- Chat de groupe (messages + pièces jointes, façon Signal — fonctionnalité core) -- Téléchargement de fichiers -- Streaming vidéo (VOD) -- [Futur] Flux vidéo éphémère - -**Modes client** [à concevoir] : -- Mode explorateur : navigateur de fichiers pour le contenu du groupe -- Mode flux : fil de chat avec pièces jointes -- Articulation UI hub/node à définir ; l'app Android se connectera directement au node rapidement après la création du compte - -### 4.4 Mesh Relay - -**[Fonctionnalité future]** Relais TURN opéré par la communauté. Utilisé uniquement en dernier recours quand toutes les méthodes de connexion P2P échouent (~15–20% des connexions). Le trafic est toujours chiffré E2E — le relais ne voit que des paquets QUIC opaques. - -Non opéré par meshbay.org. Un protocole d'enregistrement des relais (hub-médié) sera conçu lors de l'introduction de cette fonctionnalité. N'impacte pas le design actuel. - ---- - -## 5. Modèle de groupe - -Les groupes sont l'unité organisationnelle centrale. - -| Paramètre | Options | -|---|---| -| Visibilité | Public / Privé | -| Politique d'adhésion | Libre / Sur demande / Sur invitation uniquement | -| Admin | L'opérateur du node hébergeur (hébergeur légal) | - -Un groupe public fonctionne comme un forum thématique : fichiers, fil de chat, liste de membres. La politique d'adhésion est indépendante de la visibilité (un groupe public peut nécessiter une approbation pour rejoindre). - -Le contenu d'un groupe privé (fichiers, index, messages) est toujours chiffré E2E avec la GEK. Seuls les membres possédant la GEK peuvent déchiffrer quoi que ce soit. - -**Adressage des groupes :** -``` -meshbay.org/u/username/groupname — groupe public via hub -meshbay.org/g/groupname — groupe public (raccourci) -group://<PK_group_fingerprint>@<node_addr> — accès direct sans hub -``` -`meshbay.org` est entièrement configurable dans le code source (constante/fichier de config). Le hub est accessible par domaine ou par IP (accès par IP nécessite un certificat auto-signé ; avertissement navigateur attendu et documenté). - ---- - -## 6. Architecture cryptographique - -### 6.1 Hiérarchie de clés - -``` -Clé d'identité utilisateur Ed25519 Signature, authentification -Clé d'échange utilisateur X25519 Accord de clé -Clé d'identité groupe Ed25519 Signature métadonnées groupe (tenue par le node admin) -Clé de chiffrement groupe ChaCha20 Chiffrement contenu et index privés (symétrique, 256 bits) -Clés de session X25519/HKDF Perfect forward secrecy par connexion P2P -``` - -Toutes les clés privées stockées exclusivement sur le node (ou l'appareil client) dans le keystore chiffré. Le hub ne voit jamais aucune clé privée. - -### 6.2 Gestion de la GEK - -**Périmètre :** la GEK s'applique uniquement aux groupes privés. Les groupes publics utilisent TLS uniquement (pas de chiffrement applicatif). - -**Création de groupe :** -1. Le node admin génère la GEK (ChaCha20-Poly1305, 256 bits, CSPRNG) -2. GEK chiffrée pour chaque membre via accord de clé X25519 + HKDF -3. Bundles GEK chiffrés stockés sur le hub (blobs opaques — le hub ne peut pas les déchiffrer ; charge négligeable : ~200–400 octets par membre par groupe) - -**Justification du stockage sur hub :** les membres peuvent récupérer leur bundle GEK même si le node est hors ligne. L'exposition du hub est minimale — il stocke du texte chiffré qu'il ne peut pas lire. - -**Ajout de membre :** -- GEK chiffrée avec la `PK_user` du nouveau membre et uploadée sur le hub - -**Révocation de membre :** -- Le node admin génère une nouvelle GEK -- Re-chiffrement pour tous les membres restants, upload des nouveaux bundles -- Les nouveaux contenus sont chiffrés avec la nouvelle GEK -- L'ancien membre conserve la capacité de déchiffrer le contenu précédemment reçu (compromis acceptable — re-chiffrement rétroactif complet non prévu) - -### 6.3 Chiffrement à la volée pour le transfert de fichiers - -Les fichiers sont stockés en clair sur le disque de l'hébergeur. Le node chiffre à la lecture. - -``` -Disque (clair) → compression zstd → chiffrement GEK (par chunk) → session QUIC → Client → déchiffrement QUIC → déchiffrement GEK → clair -``` - -**Chunking :** -- Taille de chunk : 1 Mo (amortit l'overhead AEAD ; permet le seek) -- Dérivation de clé par chunk : `chunk_key = HKDF(GEK, "file:" || blake3(fichier) || "chunk:" || index)` -- Chaque chunk déchiffrable indépendamment → permet le seek VOD -- Compresser avant chiffrer (la compression est inefficace sur du texte chiffré) - -**Authentification des chunks :** chaque chunk signé avec la clé Ed25519 du node. Le client vérifie avant déchiffrement. Prévient l'injection de données par un relais compromis. - -**Optimisations chiffrement :** -- `cryptography` (PyCA) utilise OpenSSL, contourne le GIL Python pour les ops crypto -- ChaCha20-Poly1305 : ~500 Mo/s sans AES-NI ; AES-256-GCM : >2 Go/s avec AES-NI -- Pour un home node (50 Mbps upload = 6 Mo/s), le chiffrement n'est pas le goulot d'étranglement -- Pipeline asyncio (lecture → compression → chiffrement → envoi) sans charger les fichiers entiers en mémoire -- Clés de chunk dérivées par batch au début du transfert, pas chunk par chunk - -### 6.4 Sécurité du transport - -- Principal : **QUIC** (TLS 1.3 intégré, UDP, streams multiplexés) -- Fallback : **TCP + TLS 1.3 + HTTP/2** (même protocole applicatif, performances moindres) -- Interface transport abstraite dans le code — swappable sans changer le protocole applicatif -- Clés de session par connexion via X25519 ECDH + HKDF (indépendantes de la couche GEK) - -### 6.5 Chiffrement du chat - -Le chat de groupe est une **fonctionnalité core** (pas un module d'extension). Utilise l'algorithme **Double Ratchet** (comme Signal) : -- Forward secrecy et break-in recovery par message -- Chaque message chiffré indépendamment -- Pièces jointes : chiffrées avec la clé de message Double Ratchet courante, hash inclus dans le message -- Implémentation Python : [TBD — évaluer les bibliothèques existantes] - ---- - -## 7. Réseau et connectivité - -### 7.1 Traversée NAT — ordre des tentatives - -``` -1. IPv6 disponible des deux côtés → connexion directe -2. UPnP / NAT-PMP sur le routeur → le node ouvre un port automatiquement -3. ICE + STUN / UDP hole punching → ~80–85% de réussite -4. Mesh Relay (TURN) → [fonctionnalité future] -``` - -Sans l'étape 4, ~15% des connexions entre peers sous NAT symétrique échoueront. Comportement documenté jusqu'à l'implémentation du Mesh Relay. - -Signaling (étape 3) : coordonné via l'endpoint WebSocket du hub, <1 Ko par tentative, sans état persistant. - -### 7.2 MNP — Mesh Node Protocol - -Protocole applicatif sur QUIC (ou fallback TCP+TLS). Tous les messages portent un champ `version`. - -**Types de messages définis :** - -| Type | Description | -|---|---| -| `handshake` | Échange de clés, présentation JWT, négociation de version | -| `index_sync` | Delta de Mesh Group Index chiffré | -| `file_request` | Demande de chunk(s) d'un fichier par hash + index de chunk | -| `file_chunk` | Données de chunk + signature | -| `stream_segment` | Segment HLS/DASH (VOD), chiffré avec clé dérivée de la GEK | -| `chat_message` | Frame de message chiffré Double Ratchet | -| `chat_attachment` | Métadonnées de pièce jointe + clé ; données transférées comme chunks de fichier | -| `ephemeral_stream` | [réservé, futur] Vidéo éphémère avec métadonnées TTL | - -### 7.3 Diffusion de contenu public — Swarm - -Fichiers publics identifiés par leur hash `blake3`. Plusieurs nodes peuvent servir le même fichier : - -1. Tout node possédant un fichier public et choisissant de le mirrorer s'enregistre : `{ hash → adresse_node }` auprès du hub -2. Le hub maintient une table de sources : `{ blake3_hash → [node_A, node_B, ...] }` -3. Un client demande un fichier → le hub retourne la liste des sources → le client récupère des chunks en parallèle depuis plusieurs nodes -4. Intégrité vérifiée par hash blake3 sur chaque chunk - -**Transport :** TLS uniquement pour le contenu public (pas de GEK). Contenu signé avec la clé Ed25519 du node original — les clients vérifient l'authenticité même depuis un miroir. - ---- - -## 8. Index - -### 8.1 Mesh Directory (niveau hub) - -Registre public des groupes, échangé entre hubs via MHP. - -Format : `msgpack`, signé avec la clé Ed25519 du hub, porte un champ `version`. - -Champs par entrée : nom de groupe, `PK_group`, hub hébergeur, description, tags de type de contenu, politique d'adhésion, date de création. - -### 8.2 Mesh Group Index (niveau node) - -Listing des fichiers d'un groupe. Généré et maintenu par le node hébergeur. - -Format : `msgpack` → `zstd` → chiffré GEK (groupes privés) ou signé en clair Ed25519 (groupes publics). - -Structure d'une entrée : -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "fichier.mkv", - "path": "Films/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # secondes, pour les médias - "thumb_hash": "<blake3>", # miniature aussi chiffrée GEK - "added_at": 1720000000 -} -``` - -Mises à jour delta : `{ base_version, additions, deletions }` — pas de re-chiffrement complet à chaque changement. - -Transit : les nodes poussent les deltas d'index aux membres connectés sur modification ; les membres tirent l'index complet à la première connexion. Le hub ne stocke aucun contenu d'index. - -### 8.3 Recherche - -**Groupes privés :** entièrement locale sur l'appareil du client. Le client maintient un cache local chiffré de tous les index des groupes dont il est membre. Aucun appel réseau, aucune implication du hub, résultats instantanés. - -**Groupes publics :** le client interroge les nodes directement à la demande. Le hub fournit le routage uniquement. - -**Interface web du hub — recherche :** délègue la requête aux nodes concernés à la demande. Le hub ne stocke rien de cette interaction. Micro-cache en mémoire des résultats : **TTL 60 secondes maximum, RAM uniquement, jamais écrit sur disque, contenu public uniquement.** Relève du caching technique (DSA UE Article 13) — pas de l'indexation. - ---- - -## 9. Fédération inter-hubs (MHP) - -### 9.1 Hiérarchie des hubs - -``` -Root Hub (meshbay.org) - ├── Full Hub (auto-hébergé, CA déléguée) - │ └── émet des credentials utilisateurs, gère ses propres groupes - │ └── se fédère avec d'autres Full Hubs via MHP - └── Mirror Hub - └── héberge uniquement le Mesh Directory public (pas de comptes, pas d'émission de clés) -``` - -Un Full Hub reçoit un certificat signé par le Root Hub (ou un Full Hub parent). Les Mirror Hubs ne peuvent que répliquer les données publiques. Promotion/rétrogradation possible sans casser le protocole. - -### 9.2 Conception MHP - -- Sélection explicite des pairs : chaque hub maintient une liste d'autorisation de hubs de confiance -- Pas de découverte automatique de hubs -- Données échangées : Mesh Directory (groupes publics), listes de révocation, données d'authentification cross-hub -- Tous les messages MHP portent un champ `version` - -### 9.3 Accès client cross-hub - -1. Le client (utilisateur Hub A) découvre un groupe sur Hub B via le Mesh Directory ou un lien direct -2. Le client présente son JWT Hub A directement à Hub B -3. Hub B vérifie le JWT avec la clé publique de Hub A (récupérée une fois, mise en cache) -4. Hub B émet un token de session local de courte durée -5. Le client se connecte au node normalement - ---- - -## 10. Modération - -### 10.1 Contenu public - -``` -Signalement #1 → suspension automatique de l'accès public au contenu - → notification à l'opérateur du node -Une republication autorisée -Signalement #2 → escalade vers les modérateurs du hub -Confirmé → groupe révoqué sur le hub local - → révocation propagée aux hubs fédérés via MHP -``` - -Mécanisme : hash `blake3` du contenu ajouté à la liste de blocage du hub. Token de révocation signé envoyé au node. - -### 10.2 CSAM - -Hash matching contre la base de données NCMEC/IWF sur le contenu public lors de l'enregistrement. Pas de scanning du contenu privé/chiffré. La participation est obligatoire pour les opérateurs de hub et réduit significativement l'exposition légale. - -### 10.3 Copyright - -Cadre de notification légale DMCA/équivalent. Takedown sur notification. Pas de blocage technique automatique (risque de faux positifs, fair use). Le hub peut révoquer sur demande légale confirmée. - -### 10.4 Contenu privé - -Non modérable directement (chiffré E2E par conception). Action disponible : révoquer l'utilisateur ou le groupe au niveau du hub sur demande légale formelle. Le hub émet un token de révocation signé Ed25519 vérifiable offline par les nodes de tous les membres. - ---- - -## 11. Système de modules Python d'extension - -Le node charge des modules d'extension (Python) dans un sous-processus sandbox. **Le chat est une fonctionnalité core intégrée, pas un module.** - -**Manifeste de module :** -```python -{ - "name": "mon-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**APIs disponibles :** -- `read_index()` — lecture de l'index courant du groupe (lecture seule) -- `send_message(content)` — poster dans le fil du groupe -- `receive_events(handler)` — s'abonner aux événements du groupe - -**Non disponible :** accès réseau arbitraire, accès au système de fichiers hors du contexte du groupe, appels système. - ---- - -## 12. Cadre légal - -**Opérateur du node :** hébergeur légal principal du contenu. Entièrement responsable de ce qu'il partage. Le logiciel node communique cela explicitement lors de l'installation. - -**Opérateur du hub (meshbay.org) :** registrar, pas hébergeur de contenu. Stocke un minimum de données. Opère le mécanisme de takedown. Participe au hash matching CSAM. Exposition légale analogue à celle d'un bureau d'enregistrement de domaines. - -**Auteur du protocole/logiciel :** protégé par les usages non-contrefaisants substantiels. - -**Données du hub :** -- Email et téléphone optionnel : conservés pour la récupération de compte et la conformité légale -- Mot de passe : haché Argon2id, jamais stocké en clair -- Logs de connexion : conservés selon les obligations légales (1 an minimum) -- Métadonnées de contenu : jamais stockées -- IP courante des nodes : non persistée (signaling éphémère) - ---- - -## 13. Fonctionnalités futures - -- **Mesh Relay :** relais TURN communautaires, protocole d'enregistrement via hub, trafic E2E chiffré -- **Réplication de contenu entre nodes :** node-à-node, autorisée par l'admin, sans implication du hub -- **Miroir de hub (répartition de charge) :** réplication complète du hub (BDD users, registre de groupes, bundles GEK) pour distribuer la charge. Nécessite une stratégie de BDD distribuée (streaming replication PostgreSQL ou équivalent). Complexe — à concevoir quand nécessaire. -- **Push vidéo depuis mobile → node :** mobile filme → pousse vers le node hébergeur → flux éphémère avec TTL distribué aux membres du groupe. Type MNP `ephemeral_stream` réservé. -- **Appairage node–mobile :** QR code depuis l'interface web locale -- **Téléchargement multi-sources :** récupération de chunks en parallèle depuis le swarm pour les fichiers publics -- **Client iOS** -- **Chiffrement at-rest sur le node :** optionnel pour les nodes déployés sur des serveurs distants -- **Intégration keychain OS pour le déverrouillage du keystore** - ---- - -## 14. Questions ouvertes [TBD] - -1. **Durée de validité du refresh token :** 30 ou 90 jours ? -2. **Schéma d'adressage des groupes :** confirmation du format URL final -3. **Bibliothèque Double Ratchet :** identifier la meilleure implémentation Python -4. **Emplacement des bundles GEK pour les groupes à accès mixte** (public restreint aux inscrits) : hub ou node ? -5. **Fréquence de sync fédération MHP et résolution de conflits** -6. **Stratégie de réplication pour le miroir de hub** (quand implémenté) -7. **Stockage des pièces jointes du chat :** stockées sur le node comme des fichiers ordinaires, ou store séparé ? -8. **Conception du protocole d'enregistrement des relais** (quand implémenté) -9. **Claims du payload JWT :** champs exacts à inclure pour la vérification d'accès aux groupes par le node -10. **Paramètres Argon2id :** calibrage pour le matériel cible (home server vs VPS) diff --git a/docs/meshbay-draft-v2.md b/docs/meshbay-draft-v2.md deleted file mode 100644 index 493e877..0000000 --- a/docs/meshbay-draft-v2.md +++ /dev/null @@ -1,550 +0,0 @@ -# MeshBay — Architecture Draft v2 - -> Status: preliminary draft — open points marked [TBD] -> Changes from v1: IP logging (legal), protocol versioning, hardware sizing, JWT strategy, keystore proposals, chat as core, relay moved to future, hub mirror future, GEK clarified, port 18000, lazy admin keystore. - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user`, hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database. - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip).** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, hub-signed groups membership claim -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) -- Serve files and video streams to group members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal -- Run the MNP protocol -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) -- Host the group chat (core feature) - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity, group identity, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id (tuned for ~1s derivation on target hardware). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples QUIC from TCP+TLS fallback -- QUIC: `aioquic` (Cloudflare-maintained). Fallback: TCP + TLS 1.3 + HTTP/2 if QUIC proves insufficient in production -- ICE/STUN: `aioice` -- WebRTC [future]: `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. - -**Hub-side operations:** -- Account creation and login (Android: email + phone at registration) -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing via Mesh Group Index -- Group chat (messages + attachments, Signal-like — core feature) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: file browser for group content -- Feed mode: chat thread with attachments -- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (~15–20% of connections). Traffic is always E2E encrypted — the relay sees only opaque QUIC packets. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK encrypted for each member via X25519 key agreement + HKDF -3. Encrypted GEK bundles stored on hub (opaque blobs — hub cannot decrypt them; charge is negligible: ~200–400 bytes per member per group) - -**Storing on hub rationale:** members can retrieve their GEK bundle even when the node is offline. Hub exposure is minimal — it stores ciphertext it cannot read. - -**Member addition:** -- GEK encrypted with new member's `PK_user` and uploaded to hub - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → QUIC session → Client → QUIC decrypt → GEK decrypt → plaintext -``` - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~500 MB/s on hardware without AES-NI; AES-256-GCM: >2 GB/s with AES-NI -- For typical home node (50 Mbps upload = 6 MB/s), encryption is not the bottleneck -- For high-concurrency scenarios: asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -- Primary: **QUIC** (TLS 1.3 integrated, UDP, multiplexed streams) -- Fallback: **TCP + TLS 1.3 + HTTP/2** (same application protocol, lower performance) -- Transport interface abstracted in code — swappable without protocol changes -- Per-connection session keys via X25519 ECDH + HKDF (independent of GEK layer) - -### 6.5 Chat Encryption - -Group chat is a **core feature** (not an extension module). Uses the **Double Ratchet algorithm** (as in Signal): -- Forward secrecy and break-in recovery per message -- Each message independently encrypted -- Attachment files: encrypted with the current Double Ratchet message key, hash included in message -- Python implementation: [TBD — evaluate existing libraries] - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection -2. UPnP / NAT-PMP on router → node opens port automatically -3. ICE + STUN / UDP hole punching → ~80–85% success rate -4. Mesh Relay (TURN) → [future feature] -``` - -Without step 4, ~15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -Signaling (step 3): coordinated via hub WebSocket endpoint, <1 KB per attempt, no persistent state. - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over QUIC (or TCP+TLS fallback). All messages carry a `version` field. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + signature | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Double Ratchet encrypted message frame | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000 -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Hub Federation (MHP) - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 10. Moderation - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 11. Python Extension Module System - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 12. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) - ---- - -## 13. Future Features - -- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **iOS client** -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** - ---- - -## 14. Open Questions [TBD] - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **Double Ratchet library:** identify best Python implementation -4. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? -5. **MHP federation sync frequency and conflict resolution** -6. **Hub mirror replication strategy** (when implemented) -7. **Chat attachment storage:** stored on node like regular files, or separate store? -8. **Relay registration protocol design** (when implemented) -9. **JWT payload claims:** exact fields to include for node group-access verification -10. **Argon2id parameters:** tuning for target hardware (home server vs. VPS) diff --git a/docs/meshbay-draft-v3.md b/docs/meshbay-draft-v3.md deleted file mode 100644 index 97c391d..0000000 --- a/docs/meshbay-draft-v3.md +++ /dev/null @@ -1,893 +0,0 @@ -# MeshBay — Architecture Draft v3 - -> Status: preliminary draft — open points marked [TBD] -> Changes from v2: jti mandatory in JWT (Spike 3), Argon2id params corrected (Spike 1), NAT traversal order corrected (Spike 4), transport flipped to TCP+TLS 1.3 v1 / QUIC v2, GEK wrapping protocol confirmed with exact parameters (Spike 6), hub API table expanded with 4 new endpoints (Spike 6), package structure decided (3 packages, uv monorepo), key persistence requirement added (Spike 6), new sections: Hub API Reference, TCP+TLS Transport v1, Package Structure. - ---- - -## Changes from v2 - -The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | -| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | -| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | -| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | -| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | -| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | -| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | -| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) -- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. -> -> This bug was found and fixed during Spike 3. - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -#### 4.1.5 Hub API Reference - -Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. - -**Hub metadata:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | -| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | - -**User management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | -| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | -| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | -| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | - -**Node management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | -| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | - -**Group management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | -| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | -| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | -| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | - -**GEK distribution (private groups):** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | -| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | - -**Revocation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/revoke/user/{user_id}` | Access token (admin) | Revoke a user account | [TBD] | -| POST | `/v1/revoke/group/{group_id}` | Access token (admin) | Revoke a group | [TBD] | -| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) — one directory per group -- Serve files, video streams, and group chat to members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) -- Run the MNP protocol (QUIC v2, TCP+TLS v1) -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) - -**Multi-group architecture (decided Phase 7):** -A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated -by port — the MNP handshake identifies the target group via the `group_id` claim -in the client JWT. The server routes each connection to the appropriate -DirectoryIndexer and GEK after JWT verification. -Rationale: one NAT hole to maintain, one port to forward manually if needed. - -**Authorization invariant:** the node MUST verify that the JWT's `groups` claim -contains the requested group_id before serving any content. Without this check, -any authenticated user could access any group on the node. This is enforced at -the MNP handshake layer, not the transport layer. - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. - -**Argon2id parameters (production):** -- `iterations = 4` -- `memory_cost = 262144` (256 MB) -- `parallelism = 1` (or match CPU count — tune to target hardware) -- Target derivation time: ~500 ms on a home server - -> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. - -**CLI calibration:** -``` -meshbay-node --calibrate-argon2 -``` -This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. - -**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. -- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) -- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) -- ICE/STUN: `aioice` -- WebRTC [future]: `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. - -**Hub-side operations:** -- Account creation and login (Android: email + phone at registration) -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing via Mesh Group Index -- Group chat (messages + attachments, Signal-like — core feature) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: file browser for group content -- Feed mode: chat thread with attachments -- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - -### 4.5 Package Structure and Repository - -**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). - -``` -meshbay/ -├── packages/ -│ ├── meshbay-common/ # Shared crypto, serialization, protocol types -│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) -│ └── meshbay-node/ # Node daemon + local web UI -├── poc/ # POC and spikes — reference implementation -│ ├── spike1_crypto/ -│ ├── spike2_hub/ -│ ├── spike3_node_reg/ -│ ├── spike4_nat/ -│ ├── spike5_transfer/ -│ ├── spike6_gek/ -│ └── spike-results.md -├── docs/ -│ └── meshbay-draft-v3.md -└── pyproject.toml # Workspace root -``` - -**Three packages:** - -| Package | RPM name | Contents | -|---|---|---| -| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | -| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | -| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | - -**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. - -**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: -- Exact crypto parameter choices (Spike 1) -- GEK wrapping/unwrapping implementation (Spike 6) -- Hub API skeleton (Spike 2) -- NAT detection and STUN interaction (Spike 4) -- TCP file transfer pipeline (Spike 5) - -Developers implementing production features should read the corresponding spike before writing production code. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. - -### 6.1.1 Key Generation Strategies - -Three strategies, depending on client type: - -**A — CLI / native node (Argon2id derivation)** -Keys are derived deterministically from `username + password`: -``` -salt = SHA-256("meshbay:v1:" + username) -seed = Argon2id(password, salt, length=64) -sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) -sk_x25519 = X25519.from_private_bytes(seed[32:]) -``` -Same credentials → same keys on any machine. Password recovery = key recovery. -Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. - -**B — Web browser (random keypairs + encrypted bundle)** -Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them -with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub -alongside the public keys. On subsequent logins, the hub returns the bundle -and the browser decrypts it locally with the password. - -The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. -Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. - -**C — Native node with keystore file** -Random keypairs generated once, stored in the Argon2id-encrypted keystore file -(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. - -**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). -A user who registered via CLI (A) and later tries to recover via web (B) with the same -password will get different keypairs. This is by design: users pick one registration path. -Cross-path recovery requires the admin to issue new GEK bundles. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** - -``` -Admin side (wrap_gek): - sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle - shared = X25519(sk_eph, pk_recipient) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - nonce = random_bytes(12) - wrapped = ChaCha20-Poly1305(wrap_key).encrypt( - nonce, gek, aad=pk_recipient) # aad binds bundle to recipient - bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire - # hub stores as opaque 48-byte blob - # (without pk_eph in compact form — see note) - -Member side (unwrap_gek): - shared = X25519(sk_recipient, pk_eph) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - gek = ChaCha20-Poly1305(wrap_key).decrypt( - nonce, wrapped, aad=pk_recipient) -``` - -> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. - -**Security properties confirmed in Spike 6:** -- Hub never sees the GEK in cleartext -- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls -- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected -- Wrong private key → AEAD authentication tag failure → immediate rejection - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK wrapped for each initial member via the protocol above -3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` -4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` - -**Member addition:** -- Admin fetches new member's `pk_x25519` from hub -- Wraps GEK for them and uploads bundle - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext -``` - -(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** - -| Operation | Time | -|---|---| -| Encrypt + sign (node side) | 3.2 ms | -| Verify + decrypt (client side) | 3.9 ms | -| Total crypto overhead (1 MB) | < 10 ms | -| Network transfer | 99–234 ms (network-limited) | - -Encryption is not the bottleneck. Network latency and bandwidth dominate. - -**Pipeline optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI -- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -**Implementation phases:** - -| Phase | Transport | Status | Notes | -|---|---|---|---| -| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | -| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | - -The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. - -**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. - -**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. - -### 6.5 TCP+TLS 1.3 Transport Implementation (v1) - -**Connection model:** -- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) -- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) -- TLS 1.3 mandatory; TLS 1.2 rejected -- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) -- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain - -**Handshake sequence:** -``` -Client → Node: TCP SYN -Node → Client: TLS ServerHello (self-signed cert, PK_node) -Client: verify cert against hub-fetched PK_node -Client → Node: TLS ClientFinished -Node → Client: MNP handshake request (version negotiation) -Client → Node: MNP handshake response (JWT access token, version) -Node: verify JWT offline (Ed25519, hub public key) -Node → Client: session established -``` - -**Message framing over TCP:** -- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` -- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) -- Each frame carries the MNP `version` field in its header - -**QUIC migration path (v2):** -- Replace TCP length-framing with QUIC streams (one stream per logical exchange) -- MNP handshake maps 1:1 to a QUIC handshake stream -- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) -- Chat messages map to a persistent QUIC stream -- No changes to JWT verification, GEK decryption, or Index sync logic - -**Port allocation:** -- `18000/tcp` — local web UI (loopback only, not exposed externally) -- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) -- Configurable via `~/.config/meshbay/node.toml` - -### 6.6 Chat Encryption and Model - -Group chat is a **core feature** (not an extension module). - -**Model (decided):** between a forum and Signal. -- **Persistent:** messages stored on the node (not ephemeral like Signal by default) -- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages -- **Scope:** per group (not per user pair) -- **Attachments:** files and images, shared like regular group files -- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect -- **Retention:** managed by the group admin (no automatic expiry) - -**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** - -The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): - -- Each group member generates a **sender key** (random symmetric chain key + signing keypair) -- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) -- Each existing member sends their current sender key to the new member -- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) -- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys -- O(N) state per member (one chain per group member), not O(N^2) -- The existing Double Ratchet implementation is kept for future 1:1 direct messaging - -Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. - -> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection (preferred) -2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) -3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) -4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile -``` - -> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. - -**Spike 4 findings:** -- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) -- UDP hole punching functional: bidirectional echo received from OVH VPS -- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results -- No CGNAT: stable public IPv4 (81.220.170.32) - -Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** -Currently the node punches blindly at startup; the client may connect 10-20s later -on an aging NAT entry, causing retransmissions. The coordinated flow uses the -existing hub→node WebSocket (revocation channel): -``` -Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} -Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} -Node : punch_nat(peer_ip, peer_port) immediately -Node → Hub (WS) : {type: "punch_ready"} -Hub → Client: 200 OK "connect now" -Client → QUIC: first packet < 2s after probe → fresh NAT entry -``` -demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. -The probe must come from the QUIC server's own socket (`punch_nat()` via -`_transport.sendto()`). The QUIC client must connect from the same port -as the probe's destination (`local_port=QUIC_PORT`). Handshake time -with proper signaling: < 200ms (vs 12.7s without). - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + Ed25519 signature | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Double Ratchet encrypted message frame | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000 -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Hub Federation (MHP) - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 10. Moderation - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 11. Python Extension Module System - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 12. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) -- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them - ---- - -## 13. Future Features - -- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic. Necessary for symmetric NAT (CGNAT mobile, some professional ISPs). -- **QUIC transport (v2):** replace TCP+TLS 1.3 with QUIC once application protocol is stable. Transport abstraction layer makes this a drop-in replacement. -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **iOS client** -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** -- **WebRTC:** `aiortc` for browser-native P2P (no node required for clients) - ---- - -## 14. Open Questions [TBD] - -**Resolved by POC (no longer open):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R1 | Argon2id parameters: what values target ~500ms? | `iterations=4, memory_cost=262144` (256 MB). Use `meshbay-node --calibrate-argon2` for hardware-specific tuning. | Spike 1 | -| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, group membership claim. `jti` is mandatory (prevents replay, enables revocation). | Spike 3 | -| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). Hub stores opaque 48-byte blobs. | Spike 6 | -| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). UPnP unreliable — demoted to step 3. | Spike 4 | -| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1 (lower complexity, works everywhere). QUIC for v2 via `Transport` abstraction. | Spike 5 | -| R6 | Hub API: which endpoints for GEK distribution? | `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek` | Spike 6 | -| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. uv workspace monorepo. | POC | - -**Resolved by first security review (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R8 | Group chat encryption model? | Sender Keys protocol (Signal Groups approach). Double Ratchet kept for future 1:1 DM only. MLS considered for v2 if groups > 50 members. | Security review C1 | -| R9 | Token denylist distribution? | Push via existing hub→node WebSocket. Node maintains an in-memory jti set. MNP handshake checks the set before accepting a JWT. No periodic polling needed. | Security review S3 | -| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. GEK is CSPRNG output (already uniform), so HKDF extract step doesn't need a random salt. Spec wording corrected to match code (RFC 5869 compliant). | Security review M5 | -| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D recommendation. Code fixed from 128-bit to 96-bit. | Security review S4 | - -**Still open:** - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? -4. **MHP federation sync frequency and conflict resolution** -5. **Hub mirror replication strategy** (when implemented) -6. **Chat attachment storage:** stored on node like regular files, or separate store? -7. **Relay registration protocol design** (when implemented) -8. **QUIC migration timeline:** when is the application protocol considered stable enough to begin v2 transport implementation? -9. **Refresh token rotation:** implement one-time-use refresh tokens (rotate on each use, detect reuse as theft indicator). RFC 6819 §5.2.2.3. -10. **Email encryption at rest:** spec requires encrypted email/phone in DB, implementation stores plaintext. Needs server-side encryption with key from hub config. diff --git a/docs/meshbay-draft-v4.md b/docs/meshbay-draft-v4.md deleted file mode 100644 index 26c7bf6..0000000 --- a/docs/meshbay-draft-v4.md +++ /dev/null @@ -1,1368 +0,0 @@ -# MeshBay — Architecture Draft v4 - -> Status: active development — Phases 1–12 complete (except 10.9 → Phase 13), 191 tests. -> Changes from v3: web client transport (WebRTC DataChannel), web UI architecture, hub roles (admin/moderator), hub mirror design, browser-specific NAT traversal, chat storage clarified, Phase 8 security items resolved, Phase 10 site overlay + admin/moderation UI + notifications + group search + version endpoint, Phase 10b self-service UI (group create/join/invite, file upload, IndexedDB caching, cross-group search), **node sovereignty model** (§4.2.x — node operator is sole content authority, deny-by-default, uploader_id tracking), **cryptographic sovereignty enforcement** (GEK-HMAC handshake challenge, Ed25519 admin challenge-response, gek_req removed), **Phase 12 — P2P crypto material** (GEK+keypair bundles moved off hub to node BundleStore, password split, key persistence in IndexedDB/sessionStorage, DTLS channel binding fix). - ---- - -## Changes from v3 - -The following items are **architectural decisions** driven by Phase 8 implementation and web client design (2026-08-10). They supersede the corresponding text in v3. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | Browser transport | Web browsers use **WebRTC DataChannel** (with ICE/STUN) for P2P to nodes behind NAT. WebTransport cannot work because browsers cannot choose their UDP source port — Port-Restricted Cone NAT requires exact port matching. Native clients (desktop, Android) continue using QUIC with `punch_nat()`. | Web client design session | -| 2 | Hub signaling | Hub WebSocket extended to relay WebRTC signaling (SDP/ICE) between browser and node. <1 KB per message, stateless, no content. Same channel as jti denylist push and `client_incoming`. | Web client design session | -| 3 | Hub role | Reinforced: hub is registrar + signaling facilitator ONLY. Never proxies, stores, or touches content (files, streams, chat, indexes). All data lives on nodes. Clients connect E2E to nodes. | Design constraint | -| 4 | Chat storage | Chat messages stored on node(s) hosting the group, not on the hub. Browser retrieves chat from node via DataChannel. If no node is online, group is unavailable. | Web client design session | -| 5 | Web UI | Preact SPA (~3 KB gzipped), dark/light theme, responsive, i18n (JSON translations). ESM modules, esbuild for minification. No heavy frameworks. | Web client design session | -| 6 | Hub roles | Three roles: `user`, `moderator`, `admin`. Moderator can review reports and suspend content/groups/users. Admin has full hub management. | Web client design session | -| 7 | Site overlay | meshbay.org serves both generic hub functionality and site-specific pages (landing, /downloads, /about). Separated via Caddy static file priority. | Web client design session | -| 8 | Hub mirror | Design defined (future implementation): active-active with shared signing key, PostgreSQL logical replication, DNS round-robin. Not implemented yet. | Web client design session | -| 9 | Security items | S1 (admin authz), S2 (email encryption), S5 (refresh token rotation) resolved in Phase 8. Argon2id bumped to 256 MB with transparent rehash. | Phase 8 implementation | -| 10 | File search | Client-side search on cached indexes (IndexedDB). No hub involvement. Private group indexes are GEK-encrypted — hub stores opaque, client decrypts locally. | Web client design session | -| 11 | P2P crypto material | **ALL crypto material moved off hub to P2P channel.** GEK bundles and keypair bundles stored on node (`BundleStore` SQLite), exchanged via MNP DataChannel. Hub `GEKBundle` model and `/gek` endpoint removed. Hub never touches, stores, or proxies any crypto material. | Phase 12 — T3 attack surface reduction | -| 12 | Password split | Hub receives `auth_key` (PBKDF2-SHA512, auth salt), never raw password. Separate `bundle_key` (PBKDF2-SHA512, bundle salt) encrypts keypair bundles on the node. Hub cannot derive `bundle_key` from `auth_key`. | Phase 12 — T1 | -| 13 | Node auth | Node daemon authenticates to hub via Ed25519 signed timestamp (`POST /v1/nodes/auth`), not password. JWT `scope: "node"` blocks group mutation endpoints. | Phase 12 — NS7 | -| 14 | Key persistence | Browser stores `_bundleKey` (CryptoKey) in IndexedDB and `_sessionKeys` in sessionStorage. Survives page refresh without re-login. Public key derived from recovered private key via JWK export (`_pkFromSk`), no hub dependency. | Phase 12 — browser hardening | -| 15 | DTLS channel binding | Browser saves raw answer SDP before `setRemoteDescription` (Chrome may drop sha-256 fingerprint). GEK-HMAC uses `_rawAnswerSdp` for fingerprint extraction. | Phase 12 — handshake fix | - ---- - -## Changes from v2 - -The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | -| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | -| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | -| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | -| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | -| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | -| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | -| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) -- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. -> -> This bug was found and fixed during Spike 3. - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -#### 4.1.5 Hub API Reference - -Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. - -**Hub metadata:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | -| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | - -**User management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | -| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | -| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | -| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | - -**Node management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | -| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | - -**Group management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | -| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | -| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | -| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | - -**GEK distribution (private groups):** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | -| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | - -**Revocation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/admin/revoke` | Access token (admin) | Revoke a user or group | ✓ Phase 8 | -| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | - -**Admin / moderation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) | ✓ Phase 10 | -| GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) | ✓ Phase 10 | -| GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable by username) | ✓ Phase 10 | -| GET | `/v1/admin/users/{user_id}` | Moderator+ | User detail (email, role, status, group count) | ✓ Phase 10 | -| PATCH | `/v1/admin/users/{user_id}` | Moderator+ | Update user role or status | ✓ Phase 10 | -| GET | `/v1/admin/groups` | Moderator+ | List all groups with member count | ✓ Phase 10 | -| PATCH | `/v1/admin/groups/{group_id}` | Moderator+ | Update group status | ✓ Phase 10 | -| GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user_id) | ✓ Phase 10 | -| GET | `/v1/admin/blocklist` | Admin | List blocked content hashes | ✓ Phase 8 | -| POST | `/v1/admin/blocklist` | Admin | Manually block a content hash | ✓ Phase 8 | -| DELETE | `/v1/admin/blocklist/{hash}` | Admin | Unblock a content hash | ✓ Phase 8 | -| GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) | ✓ Phase 10 | -| POST | `/v1/notifications/{id}/read` | Access token | Mark notification as read | ✓ Phase 10 | -| POST | `/v1/notifications/read-all` | Access token | Mark all notifications as read | ✓ Phase 10 | -| GET | `/v1/groups?q=` | None | Search public groups by name (ilike) | ✓ Phase 10 | -| GET | `/v1/hub/version` | None | Client version check (hub, MNP, MHP) | ✓ Phase 10 | -| GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) | ✓ Phase 10b | -| POST | `/v1/groups/{id}/join` | Access token | Self-join open group | ✓ Phase 10b | - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) — one directory per group -- Serve files, video streams, and group chat to members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) -- Run the MNP protocol (QUIC v2, TCP+TLS v1) -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) - -**Multi-group architecture (decided Phase 7):** -A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated -by port — the MNP handshake identifies the target group via the `group_id` claim -in the client JWT. The server routes each connection to the appropriate -DirectoryIndexer and GEK after JWT verification. -Rationale: one NAT hole to maintain, one port to forward manually if needed. - -**Authorization invariant:** the node MUST verify that the JWT's `groups` claim -contains the requested group_id before serving any content. Without this check, -any authenticated user could access any group on the node. This is enforced at -the MNP handshake layer, not the transport layer. - -#### 4.2.x Node Sovereignty — Content Authorization Model - -The node operator is the **sole authority** over content stored on their machine. -No external actor — including the hub admin — can modify, delete, or control -files on a node they do not operate. This is a non-negotiable design invariant, -enforced by **cryptography**, not just policy. - -**Two trust domains, strictly separated:** - -| Domain | Authority | Scope | -|---|---|---| -| **Hub** | Hub admin / moderator | User accounts, group registry, group membership, GEK distribution, moderation (suspend user/group at hub level) | -| **Node** | Node operator | Files on disk, file deletion, upload acceptance, chat storage, who can do what with node content | - -The hub certifies **identity** (JWT) and **group membership** (`groups` claim). -The node decides **authorization for content operations** based on that identity. -These two concerns must never be conflated. - -##### Cryptographic enforcement — two defense layers - -A malicious hub admin controls the JWT signing key and could forge JWTs to -impersonate any user, including the node operator. Policy-only checks (comparing -`user_id` to `node_user_id`) are insufficient because the hub controls the -identity layer. Two cryptographic mechanisms make this impossible: - -**Layer 1 — GEK proof in handshake (membership verification):** - -After JWT verification, the node challenges the connecting user to prove they -possess the Group Encryption Key (GEK). The hub never has the GEK — it only -stores opaque ECIES-wrapped bundles. Without the GEK, a hub admin who forges -a JWT still cannot access any group content. - -``` -Client → Node: handshake { token, group_id } -Node: verify JWT, verify group_id in claims - nonce = random(32) -Node → Client: handshake_challenge { nonce: base64(nonce) } -Client: proof = HMAC-SHA256(GEK, nonce) -Client → Node: handshake_response { proof: base64(proof) } -Node: verify HMAC — if wrong, reject connection -Node → Client: handshake_ack { is_node_admin, node_pk, v } -``` - -This blocks: content reading, index reading, chat reading, file upload, chat -injection — ALL operations require passing the GEK proof first. - -**Layer 2 — Ed25519 challenge-response for admin operations:** - -The node operator's Ed25519 public key is pinned locally in `node.toml` -(auto-pinned from keystore on first startup). Destructive operations (file -deletion) require the user to sign a random challenge with their Ed25519 -private key. The hub cannot forge this signature. - -``` -Client → Node: file_delete { file_id } -Node: (if uploader → allow immediately) - (else) challenge = random(32) -Node → Client: admin_challenge { challenge: base64(challenge), file_id } -Client: signature = Ed25519.sign(sk_ed, challenge) -Client → Node: admin_response { signature: base64(signature), file_id } -Node: verify(admin_pk_ed25519, signature, challenge) - if valid → delete file -``` - -**Node configuration — admin key pinning:** - -```toml -# node.toml -admin_pk_ed25519 = "base64-encoded-32-bytes-raw-Ed25519-public-key" -``` - -Auto-pinned from the node operator's keystore on first startup. The daemon -logs: "Admin Ed25519 key pinned for node sovereignty". - -**GEK distribution — browser flow (node no longer serves GEK):** - -The node NEVER serves the GEK in plaintext. Browser clients obtain the GEK -from their hub-stored encrypted bundle: - -1. `GET /v1/groups/{id}/gek` → encrypted ECIES bundle (AES-256-GCM variant) -2. Browser unwraps with its X25519 private key (from keypair bundle) -3. Browser uses raw GEK bytes for the handshake HMAC proof -4. Browser imports GEK as HKDF key for chunk decryption - -This eliminates the `gek_req`/`gek_resp` MNP messages from the protocol. - -**Authorization rules for destructive file operations (enforced by the node):** - -| Action | Who can do it | Enforcement point | -|---|---|---| -| Delete a file | Node operator (Ed25519 challenge-response) OR the user who uploaded it | Node (`_do_file_delete`) | -| Delete any file | Node operator only (Ed25519 challenge-response) | Node (`_do_file_delete`) | - -Default posture: **deny.** If the admin key is not pinned, all admin operations -are refused. If the GEK proof fails, the connection is refused entirely. - -**Protocol enforcement — MNP handshake_ack:** - -The handshake_ack message carries `is_node_admin: bool` — the node tells the -client whether the authenticated user is the node operator. Clients MUST use -this node-reported flag (not the hub's `group.admin_id`) to decide whether -to show destructive operations like file deletion. - -``` -handshake_ack: - v: "0.1" - node_pk: "<base64>" - is_node_admin: true | false # node-side authorization, NOT hub-side -``` - -**Index entry — uploader tracking:** - -Each `IndexEntry` carries an `uploader_id` field (user_id of who uploaded the -file, or null for files that pre-existed on disk). This enables the "uploader -can delete their own files" rule without granting node-admin privileges. - -**What the hub admin CANNOT do on a node they don't operate:** -- Delete files (requires Ed25519 key pinned on node — hub can't forge) -- Read files (requires GEK — hub never has it) -- Read index / chat (requires GEK proof in handshake) -- Upload files (requires GEK proof in handshake) -- Impersonate the node operator (JWT forgery blocked by Ed25519 challenge) - -**What the hub admin CAN do (hub-level only):** -- Suspend a user account (blocks JWT issuance → user loses access everywhere) -- Suspend a group (blocks signaling → no new P2P connections to nodes for that group) -- These are hub-level actions that don't touch node content - -**Remaining trust assumptions:** -- The hub serves the SPA code to browsers (a malicious hub could inject JS — fundamentally unsolvable in browser; native client or browser extension required for full integrity) -- The hub relays WebRTC signaling — ✅ MITIGATED: DTLS channel binding in GEK-HMAC proof (`HMAC(GEK, nonce || offer_fp || answer_fp)`) detects fingerprint substitution (MitM) -- The hub receives raw password at login — ✅ MITIGATED: password split (auth_key ≠ bundle_key, independent PBKDF2 derivations). Hub receives auth_key only, cannot derive bundle_key to decrypt keypair bundle. Legacy accounts migrated on first login. -- The hub controls public key distribution — can substitute keys during invite to intercept GEK. Fix: out-of-band key verification (safety numbers) — Phase 12 - -> **Design lesson (2026-08-12):** The initial implementation conflated hub -> `group.admin_id` (who created the group on the hub) with node operator -> authority (who runs the machine). The SPA used the hub's `is_admin` flag -> to show file deletion controls, and the node's delete handler used a -> fail-open check (`if node_user_id and ...` — allowed everyone when -> `node_user_id` was not set). Both violated node sovereignty. Fixed by: -> (1) deny-by-default on the node, (2) `is_node_admin` in handshake_ack, -> (3) `uploader_id` tracking in the index, (4) SPA uses node-reported -> permissions only. Then hardened with cryptographic enforcement: -> (5) GEK-HMAC proof in handshake (blocks forged-JWT access), -> (6) Ed25519 challenge-response for admin ops (blocks identity impersonation), -> (7) removal of `gek_req` endpoint (node never serves GEK in plaintext), -> (8) DTLS channel binding in GEK-HMAC proof to detect WebRTC signaling MitM, -> (9) chat `sender_id` fixed to authenticated identity (prevents impersonation), -> (10) Ed25519 challenge for ALL file deletions — uploaders verified by stored pk, not JWT sub, -> (11) password split — hub receives PBKDF2 auth_key, never raw password (cannot derive bundle_key). - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. - -**Argon2id parameters (production):** -- `iterations = 4` -- `memory_cost = 262144` (256 MB) -- `parallelism = 1` (or match CPU count — tune to target hardware) -- Target derivation time: ~500 ms on a home server - -> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. - -**CLI calibration:** -``` -meshbay-node --calibrate-argon2 -``` -This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. - -**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. -- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) -- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) -- ICE/STUN: `aioice` (already a dependency) -- WebRTC: `aiortc` (browser P2P transport — Phase 9) -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. -The hub is never in the data path — clients connect E2E to nodes for all content. - -**Hub-side operations (HTTPS, lightweight):** -- Account creation, login, token refresh -- Public group search and discovery -- Group membership management, GEK bundle retrieval -- WebRTC signaling relay (SDP/ICE — <1 KB per connection, stateless) -- Notification metadata (invitations, new content indicators) - -**Node-side operations (direct P2P via QUIC or WebRTC DataChannel):** -- File browsing via Mesh Group Index -- File download (chunked, E2E encrypted) -- Video streaming (HLS segments via DataChannel or QUIC stream) -- Group chat (Sender Keys encrypted, stored on node) -- File/photo/video upload (client → node push) - -#### 4.3.1 Web Browser Client - -**Transport:** WebRTC DataChannel with ICE/STUN for NAT traversal. -WebTransport (HTTP/3) is not suitable because browsers cannot choose their UDP -source port — Port-Restricted Cone NAT (confirmed on SFR residential) requires -the client to connect from the exact port the node probed. WebRTC's ICE handles -this automatically via simultaneous STUN binding requests. - -**UI:** Preact SPA (~3 KB gzipped) served by the hub. -- Dark/light theme (CSS `prefers-color-scheme` + user toggle in localStorage) -- Responsive design (sidebar → hamburger menu on mobile) -- i18n: JSON translation files, English default -- Build: esbuild for minification (single binary, no npm dependency) -- Crypto: SubtleCrypto (AES-GCM) for E2E decryption in browser - -**Layout:** -- Left sidebar: group list (ordered by usage — private groups first), navigation -- Top bar: logo ("MeshBay") left, user menu right (settings, profile, language, logout) -- Main content area: file explorer, chat view, or settings depending on context - -**Client modes:** -- Explorer: file/folder browser for group content (read-only browse, download, stream) -- Chat/forum: per-group discussion thread with photo/video posting -- Settings: general, per-group, notifications, privacy, theme, language - -**Local storage:** -- IndexedDB: cached group indexes for instant local search (~50–100 MB quota) -- localStorage: theme preference, language, session state -- `keypair_bundle`: encrypted keypair retrieved from hub, decrypted locally with password - -**File search:** entirely client-side on cached indexes. No hub involvement. -Private group indexes are GEK-encrypted — stored opaque on the hub, decrypted -by the client locally. Search runs against the decrypted index in IndexedDB. - -#### 4.3.2 Android Client - -**Transport:** QUIC with `punch_nat()` — same as desktop native clients. -Android has full UDP access; no WebRTC needed. Uses `quiche` (Cloudflare, Rust -via JNI) for QUIC transport. - -**Stack:** Kotlin + Jetpack Compose. Bouncy Castle JVM for crypto. - -**Capabilities:** same as web browser (browse, download, stream, chat, upload). -Additional: contact list integration (Android Contacts API, permission-gated). -Account creation from app. No node functionality on mobile (client-only). - -**Cross-device compatibility:** the `keypair_bundle` (encrypted, stored on hub) -enables seamless switching between web and Android with the same credentials. -Notification state and read markers sync via hub (small encrypted blob per user). - -**Out of scope:** Mac/iPhone support. Node on mobile. - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - -### 4.5 Package Structure and Repository - -**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). - -``` -meshbay/ -├── packages/ -│ ├── meshbay-common/ # Shared crypto, serialization, protocol types -│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) -│ └── meshbay-node/ # Node daemon + local web UI -├── poc/ # POC and spikes — reference implementation -│ ├── spike1_crypto/ -│ ├── spike2_hub/ -│ ├── spike3_node_reg/ -│ ├── spike4_nat/ -│ ├── spike5_transfer/ -│ ├── spike6_gek/ -│ └── spike-results.md -├── docs/ -│ └── meshbay-draft-v3.md -└── pyproject.toml # Workspace root -``` - -**Three packages:** - -| Package | RPM name | Contents | -|---|---|---| -| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | -| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | -| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | - -**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. - -**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: -- Exact crypto parameter choices (Spike 1) -- GEK wrapping/unwrapping implementation (Spike 6) -- Hub API skeleton (Spike 2) -- NAT detection and STUN interaction (Spike 4) -- TCP file transfer pipeline (Spike 5) - -Developers implementing production features should read the corresponding spike before writing production code. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Node admin | The hosting node operator — sovereign over content, sole delete authority (see §4.2.x) | -| Hub group creator | The user who registered the group on the hub — manages membership and GEK distribution | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. - -### 6.1.1 Key Generation Strategies - -Three strategies, depending on client type: - -**A — CLI / native node (Argon2id derivation)** -Keys are derived deterministically from `username + password`: -``` -salt = SHA-256("meshbay:v1:" + username) -seed = Argon2id(password, salt, length=64) -sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) -sk_x25519 = X25519.from_private_bytes(seed[32:]) -``` -Same credentials → same keys on any machine. Password recovery = key recovery. -Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. - -**B — Web browser (random keypairs + encrypted bundle)** -Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them -with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub -alongside the public keys. On subsequent logins, the hub returns the bundle -and the browser decrypts it locally with the password. - -The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. -Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. - -**C — Native node with keystore file** -Random keypairs generated once, stored in the Argon2id-encrypted keystore file -(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. - -**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). -A user who registered via CLI (A) and later tries to recover via web (B) with the same -password will get different keypairs. This is by design: users pick one registration path. -Cross-path recovery requires the admin to issue new GEK bundles. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** - -``` -Admin side (wrap_gek): - sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle - shared = X25519(sk_eph, pk_recipient) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - nonce = random_bytes(12) - wrapped = ChaCha20-Poly1305(wrap_key).encrypt( - nonce, gek, aad=pk_recipient) # aad binds bundle to recipient - bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire - # hub stores as opaque 48-byte blob - # (without pk_eph in compact form — see note) - -Member side (unwrap_gek): - shared = X25519(sk_recipient, pk_eph) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - gek = ChaCha20-Poly1305(wrap_key).decrypt( - nonce, wrapped, aad=pk_recipient) -``` - -> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. - -**Security properties confirmed in Spike 6:** -- Hub never sees the GEK in cleartext -- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls -- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected -- Wrong private key → AEAD authentication tag failure → immediate rejection - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK wrapped for each initial member via the protocol above -3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` -4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` - -**Member addition:** -- Admin fetches new member's `pk_x25519` from hub -- Wraps GEK for them and uploads bundle - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext -``` - -(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** - -| Operation | Time | -|---|---| -| Encrypt + sign (node side) | 3.2 ms | -| Verify + decrypt (client side) | 3.9 ms | -| Total crypto overhead (1 MB) | < 10 ms | -| Network transfer | 99–234 ms (network-limited) | - -Encryption is not the bottleneck. Network latency and bandwidth dominate. - -**Pipeline optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI -- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -**Implementation phases:** - -| Phase | Transport | Status | Notes | -|---|---|---|---| -| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | -| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | - -The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. - -**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. - -**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. - -### 6.5 TCP+TLS 1.3 Transport Implementation (v1) - -**Connection model:** -- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) -- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) -- TLS 1.3 mandatory; TLS 1.2 rejected -- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) -- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain - -**Handshake sequence:** -``` -Client → Node: TCP SYN -Node → Client: TLS ServerHello (self-signed cert, PK_node) -Client: verify cert against hub-fetched PK_node -Client → Node: TLS ClientFinished -Node → Client: MNP handshake request (version negotiation) -Client → Node: MNP handshake response (JWT access token, version) -Node: verify JWT offline (Ed25519, hub public key) -Node → Client: session established -``` - -**Message framing over TCP:** -- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` -- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) -- Each frame carries the MNP `version` field in its header - -**QUIC migration path (v2):** -- Replace TCP length-framing with QUIC streams (one stream per logical exchange) -- MNP handshake maps 1:1 to a QUIC handshake stream -- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) -- Chat messages map to a persistent QUIC stream -- No changes to JWT verification, GEK decryption, or Index sync logic - -**Port allocation:** -- `18000/tcp` — local web UI (loopback only, not exposed externally) -- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) -- Configurable via `~/.config/meshbay/node.toml` - -### 6.6 Chat Encryption and Model - -Group chat is a **core feature** (not an extension module). - -**Model (decided):** between a forum and Signal. -- **Persistent:** messages stored on the node (not ephemeral like Signal by default) -- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages -- **Scope:** per group (not per user pair) -- **Attachments:** files and images, shared like regular group files -- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect -- **Retention:** managed by the group admin (no automatic expiry) - -**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** - -The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): - -- Each group member generates a **sender key** (random symmetric chain key + signing keypair) -- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) -- Each existing member sends their current sender key to the new member -- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) -- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys -- O(N) state per member (one chain per group member), not O(N^2) -- The existing Double Ratchet implementation is kept for future 1:1 direct messaging - -Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. - -> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection (preferred) -2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) -3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) -4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile -``` - -> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. - -**Spike 4 findings:** -- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) -- UDP hole punching functional: bidirectional echo received from OVH VPS -- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results -- No CGNAT: stable public IPv4 (81.220.170.32) - -Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** -Currently the node punches blindly at startup; the client may connect 10-20s later -on an aging NAT entry, causing retransmissions. The coordinated flow uses the -existing hub→node WebSocket (revocation channel): -``` -Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} -Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} -Node : punch_nat(peer_ip, peer_port) immediately -Node → Hub (WS) : {type: "punch_ready"} -Hub → Client: 200 OK "connect now" -Client → QUIC: first packet < 2s after probe → fresh NAT entry -``` -demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. -The probe must come from the QUIC server's own socket (`punch_nat()` via -`_transport.sendto()`). The QUIC client must connect from the same port -as the probe's destination (`local_port=QUIC_PORT`). Handshake time -with proper signaling: < 200ms (vs 12.7s without). - -#### 7.1.1 Browser-Specific NAT Traversal (WebRTC DataChannel) - -Browsers cannot use the QUIC `punch_nat()` mechanism because WebTransport does -not allow the browser to choose its UDP source port. Port-Restricted Cone NAT -requires exact port matching on both IP and port — impossible for browsers. - -**Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC -stack handles NAT traversal automatically: - -1. Browser gathers ICE candidates via STUN (discovers its external IP:port) -2. Node gathers ICE candidates via `aioice` (discovers its external IP:port) -3. Candidates exchanged via hub signaling (WebSocket relay, <1 KB) -4. ICE connectivity checks: both sides send STUN binding requests simultaneously -5. STUN binding requests serve as NAT hole-punching (both directions) -6. ICE finds a valid candidate pair — DataChannel established -7. MNP protocol runs over DataChannel (same messages, same E2E encryption) - -**Signaling flow:** -``` -Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer {sdp, ice_candidates} -Hub → Node (WS) : {type: "webrtc_offer", sdp, ice_candidates, peer_id} -Node (aiortc) : creates PeerConnection, gathers answer candidates -Node → Hub (WS) : {type: "webrtc_answer", sdp, ice_candidates, peer_id} -Hub → Browser (SSE) : answer SDP + ICE candidates -DataChannel : P2P established — hub no longer involved -``` - -ICE is strictly superior to custom `punch_nat()` for browser use: -- No need for the client to pre-announce its port -- Handles both sides behind NAT simultaneously -- Automatic candidate prioritization and fallback -- Battle-tested by billions of daily users (Google Meet, Discord, Zoom) - -**Node dual transport:** the node listens on both: -- QUIC (UDP port 19000) — native clients (desktop, Android) -- WebRTC — browsers (via `aiortc`, separate UDP socket managed by ICE) - -The MNP application protocol is identical on both transports. Same handshake, -same file_request/file_chunk, same chat_message, same E2E encryption. - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `handshake_challenge` | Node sends GEK proof nonce (base64, 32 bytes random) — see §4.2.x | -| `handshake_response` | Client proves GEK possession: HMAC-SHA256(GEK, nonce) | -| `handshake_ack` | Node response: version, node public key, `is_node_admin` (node-level authorization) | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + Ed25519 signature | -| `file_delete` | Client requests file deletion by file_id | -| `file_delete_ack` | Node confirms deletion | -| `file_upload` | Client pushes file chunk to node | -| `file_upload_ack` | Node acknowledges chunk receipt | -| `admin_challenge` | Node sends Ed25519 sign challenge for admin ops (base64, 32 bytes) | -| `admin_response` | Client returns Ed25519 signature over the challenge | -| `stream_request` | Client requests MSE video stream | -| `stream_init` | Node sends codec info + signals stream start | -| `stream_data` | Node sends encrypted fMP4 segment | -| `stream_end` | Node signals end of stream | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Sender Keys encrypted message frame (group chat) | -| `chat_history` | Client requests chat history | -| `chat_history_response` | Node responds with stored messages | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000, - "uploader_id": "<user_id>" # who uploaded this file (null = pre-existing on disk) -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Web Client UI - -### 9.1 Architecture - -The web client is a Preact SPA served by the hub at `/app/`. It communicates -with the hub via HTTPS (auth, group management, signaling) and with nodes via -WebRTC DataChannel (file transfer, streaming, chat). The hub is never in the -data path. - -**Technology choices:** -- **Preact** (~3 KB gzipped): lightweight React-compatible framework -- **preact-router**: client-side routing (no server round-trips) -- **esbuild**: minification/bundling (single binary, no npm/node_modules) -- **SubtleCrypto**: browser-native AES-GCM for E2E decryption -- **IndexedDB**: local cache for group indexes (client-side search) - -No heavy frameworks (React, Vue, Angular). No build toolchain dependencies beyond -esbuild. ESM modules loaded natively by modern browsers. - -### 9.2 UI Structure - -``` -┌─────────────────────────────────────────────────────────┐ -│ [MeshBay] [User ▾] [⚙] │ -├──────────┬──────────────────────────────────────────────┤ -│ │ │ -│ Groups │ Main content area │ -│ │ │ -│ ● Private│ - File explorer (folders, files, download) │ -│ Group1 │ - Chat/forum view │ -│ Group2 │ - Video player (HLS via MediaSource API) │ -│ │ - Settings │ -│ ○ Public │ - Notifications feed │ -│ Group3 │ │ -│ │ │ -└──────────┴──────────────────────────────────────────────┘ -``` - -- **Left sidebar:** group list, ordered by usage frequency. Private groups first. - Collapses to hamburger menu on mobile viewports. -- **Top bar:** logo (left), user menu dropdown (right) — settings, profile, - language, online/offline status, logout. -- **Main area:** context-dependent content based on selected group and view. - -### 9.3 Views - -**Front page (no group selected):** -- Notification feed, prioritized: known contacts → private group activity → public -- System notifications (maintenance, updates) -- Quick access to recent groups - -**Group view — File Explorer:** -- Directory tree (folders, subfolders) — read-only browsing -- File metadata: name, size, type, date added -- Actions: download, stream (for media files) -- Files fetched directly from node via DataChannel -- Upload: photos/videos posted to the group's shared upload folder -- Delete: node operator can delete any file; uploader can delete their own uploads. - Hub admin has NO delete authority on nodes they don't operate (see §4.2.x). - -**Group view — Chat/Forum:** -- Sender Keys encrypted messages, fetched from node -- Post text, photos, videos (uploads go to node, not hub) -- Optional thread/topic structure for organized discussions -- Real-time push for connected members, pull history on reconnect - -**Group view — Video Player:** -- HLS segments fetched via DataChannel from node -- Decrypted client-side (GEK-derived key per segment) -- Played via MediaSource API (browser-native, no plugins) - -**Settings:** -- General: theme (dark/light/auto), language, notification preferences -- Per-group: notification mute, display options, filtering/blocking -- Privacy: online/offline status, profile visibility -- Profile: display name, avatar, account details - -### 9.4 Theming and i18n - -**Theme:** CSS custom properties for colors, toggled via: -1. `prefers-color-scheme` media query (OS default) -2. User override stored in localStorage -3. Toggle button in top bar or settings - -**i18n:** JSON translation files loaded client-side. -``` -static/i18n/ -├── en.json # English (default, always loaded) -├── fr.json # French (loaded on demand) -└── ... # Other languages added later -``` - -Keys are identifiers, not English text. Translation function: `t('group.join')`. - -### 9.5 meshbay.org Site Overlay - -meshbay.org serves both the generic hub application and site-specific pages: - -``` -site/ # meshbay.org-specific (not packaged with hub) -├── index.html # Landing page — project promotion, features -├── downloads.html # Package repos: Ubuntu, Fedora, Android APK -├── about.html # Project info, team, GitHub, contact -└── assets/ # Landing-specific CSS, images, icons -``` - -Caddy serves `site/` with priority. Requests not matching a static file fall -through to the hub FastAPI application. The hub serves `/app/` (SPA) and `/v1/` -(API). This separation ensures the hub package remains generic and deployable -by any operator, while meshbay.org has its own public-facing identity. - -### 9.6 Hub Mirror (future — design only) - -A mirror hub is a complete active-active replica of the primary hub. - -**Purpose:** load distribution for growing traffic. DNS round-robin (2+ A records). - -**Design:** -- Shared Ed25519 signing key (transferred once, securely) -- PostgreSQL logical replication for bidirectional read/write -- Both mirrors issue JWTs with the same key -- Both mirrors accept registrations, logins, and group operations -- If one mirror goes down, the other serves all traffic - -**Implementation constraints (must not violate in current development):** -- Hub config and key paths must be externalizable (already the case) -- No hub-instance-specific state that cannot be replicated -- JWT verification must not depend on hub-local state (already the case) -- Session state (refresh tokens, IP logs) must be in PostgreSQL (already the case) - -**Not implemented now.** Design documented to avoid blocking decisions. - ---- - -## 10. Hub Federation (MHP) <!-- was §9 in v3 --> - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 11. Moderation <!-- was §10 in v3 --> - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 12. Python Extension Module System <!-- was §11 in v3 --> - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 13. Legal Framework <!-- was §12 in v3 --> - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) -- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them - ---- - -## 14. Future Features <!-- was §13 in v3 --> - -- **Mesh Relay:** community TURN relays, E2E encrypted traffic. Low priority — typical residential NAT works with ICE/STUN. Needed only for symmetric NAT (CGNAT mobile, ~15% of connections). -- ~~**QUIC transport (v2)**~~ ✅ DONE (Phase 5) — QUIC replaces TCP+TLS. -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** design documented in §9.6. Active-active with shared key, PostgreSQL replication, DNS round-robin. Implementation deferred. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** -- ~~**WebRTC**~~ ✅ Validated (Phase 9.1–9.5) — `aiortc` for browser-to-node P2P via DataChannel. Tested on SFR residential NAT (Port-Restricted Cone) + 4G CGNAT. No TURN needed. -- **Extension-triggered views:** local apps providing custom views for group content (gallery, kanban). MNP extension hook reserved. - ---- - -## 15. Open Questions [TBD] - -**Resolved by POC (no longer open):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R1 | Argon2id parameters: what values target ~500ms? | `iterations=3, memory_cost=262144` (256 MB). pw_version=2, transparent rehash on login. | Spike 1 + Phase 8.10 | -| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, `groups` claim. | Spike 3 + Phase 7 | -| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). | Spike 6 | -| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). | Spike 4 | -| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1, QUIC for v2. QUIC is now the active transport (Phase 5). | Spike 5 | -| R6 | Hub API: which endpoints for GEK distribution? | 4 endpoints confirmed. | Spike 6 | -| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. | POC | - -**Resolved by first security review (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R8 | Group chat encryption model? | Sender Keys protocol. Double Ratchet kept for future 1:1 DM. | Security review C1 | -| R9 | Token denylist distribution? | Push via hub→node WebSocket. In-memory jti set on node. | Security review S3 | -| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. RFC 5869 compliant. | Security review M5 | -| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D. | Security review S4 | - -**Resolved by Phase 8 implementation (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R12 | Refresh token rotation? | One-time-use with family-based reuse detection. Old token reuse revokes entire family. | Phase 8.3 | -| R13 | Email encryption at rest? | AES-256-GCM, key derived from hub Ed25519 private key via HKDF(info="meshbay:email:v1"). | Phase 8.2 | -| R14 | Admin authorization model? | Config-based: `admin_usernames` in hub.toml + `MESHBAY_ADMIN_USERS` env var. | Phase 8.1 | -| R15 | QUIC migration timeline? | Done — QUIC is the active transport since Phase 5. | Phase 5 | - -**Resolved by web client design session (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R16 | Browser transport for NAT traversal? | WebRTC DataChannel with ICE/STUN. WebTransport cannot work (port-restricted cone NAT). | Design session | -| R17 | Chat storage location? | On nodes, not hub. Hub never stores content. | Design session | -| R18 | Web UI framework? | Preact SPA (~3 KB), esbuild, dark/light theme, i18n, responsive. | Design session | -| R19 | Hub mirror design? | Active-active, shared signing key, PostgreSQL replication, DNS round-robin. | Design session | - -**Resolved by Phase 9 spike (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R20 | WebRTC DataChannel validation? | Confirmed: browser→NAT→node file transfer works. Tested 3 scenarios on SFR residential (Port-Restricted Cone NAT) + 4G CGNAT: WiFi LAN (IPv6 direct, ~100ms), 4G IPv6 inter-network (~600ms), 4G IPv4 STUN hole-punch (~650ms). No TURN relay needed. | Phase 9.5 spike | - -**Resolved by node sovereignty fix (2026-08-12):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R21 | Who authorizes file deletion on a node? | Node operator (sovereign) OR original uploader. Hub admin has no authority over node content. Enforced: deny-by-default in MNP `file_delete`, `is_node_admin` in handshake_ack, `uploader_id` in IndexEntry. | Security fix — §4.2.x | -| R22 | Can a malicious hub admin access node content? | No. Two cryptographic layers: (1) GEK-HMAC proof in handshake — hub never has the GEK, can't pass the challenge. (2) Ed25519 challenge-response for admin ops — hub can't forge the node operator's signature. `gek_req` endpoint removed. | Crypto enforcement — §4.2.x | - -**Resolved by Phase 12 — P2P crypto material (2026-08-13):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R23 | Where are GEK bundles stored? | On node only (`BundleStore` SQLite, `data_dir/bundles.db`). Hub `GEKBundle` model removed. Exchanged via MNP `gek_bundle_store`/`gek_bundle_fetch`/`gek_bundle_resp` over WebRTC DataChannel. | Phase 12 — T3 | -| R24 | Where are keypair bundles stored? | On node only (`BundleStore`). Encrypted with password-derived AES key (`bundle_key`). Browser pushes after registration, recovers during handshake. Hub `keypair_bundle` column removed. | Phase 12 — T3 | -| R25 | How does the browser recover keys after localStorage cleared? | Transport fetches `keypair_bundle` from node during handshake, decrypts with `_bundleKey` (PBKDF2 from password). Public key derived from private key via JWK export — no hub fetch needed. `_bundleKey` persisted in IndexedDB, `_sessionKeys` in sessionStorage. | Phase 12 | -| R26 | How does the browser handle Chrome SDP re-serialization? | `this._rawAnswerSdp = answer.sdp` saved before `setRemoteDescription`. DTLS fingerprint extracted from raw SDP, not `pc.remoteDescription.sdp` (Chrome may drop sha-256 line when re-serializing multi-hash SDP from aiortc). | Phase 12 | -| R27 | Should the browser auto-regenerate keys on login? | No. Auto-regeneration silently rotates hub keys, breaking GEK unwrap (GEK wrapped for old keys). Keys recovered from node via `_bundleKey`. Regeneration only on explicit user request. | Phase 12 | - -**Still open:** - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? → Resolved: always on node. -4. **MHP federation sync frequency and conflict resolution** -5. **Chat attachment storage:** stored on node like regular files, or separate store? -6. **Relay registration protocol design** (when implemented) diff --git a/docs/meshbay-draft-v5.md b/docs/meshbay-draft-v5.md index e4218ee..de91c80 100644 --- a/docs/meshbay-draft-v5.md +++ b/docs/meshbay-draft-v5.md @@ -10,7 +10,7 @@ > and pairing design, `docs/invite-pairing-v1.md`. For the desktop client — shell, > device linking, account creation, node management — `docs/desktop-client-v1.md` > (2026-08-17) is authoritative and supersedes §8.2 here. -> Supersedes `meshbay-draft-v4.md`. Sections not restated here are unchanged from v4. +> Supersedes draft v4 (archived in `old-draft.md`). Sections not restated here are unchanged from v4. > > v5 exists because the second security review (`second-review.md`, 2026-08-13) found > that v4 described a system the code did not implement, and because several v4 claims diff --git a/docs/old-draft.md b/docs/old-draft.md new file mode 100644 index 0000000..f02c54f --- /dev/null +++ b/docs/old-draft.md @@ -0,0 +1,4497 @@ +# MeshBay — Archived Drafts + +> **Status: historical archive. Nothing here is authoritative.** +> +> This file consolidates the superseded architecture drafts (v1–v4), the original +> POC plan, and the Phase 1–12 development log. They are kept for provenance and +> for the section references (`draft-v3 §4.1.3`, `draft-v4 §6.6`, …) still made +> from live documents and code comments. +> +> For what is true now, read instead: +> +> | Topic | Document | +> |---|---| +> | Current specification | `meshbay-draft-v6.md` (+ `meshbay-draft-v5.md` for what v6 does not restate) | +> | Roadmap | `devel-phases-next.md` | +> | Security findings | `first-review.md`, `second-review.md` | +> | Client architecture decisions | `tmp-decisions.md`, `desktop-client-v1.md` | +> +> Contents of this archive, in order: +> 1. Architecture Draft v1 +> 2. Architecture Draft v2 +> 3. Architecture Draft v3 +> 4. Architecture Draft v4 +> 5. POC v1 +> 6. Development Phases (1–12) + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: Architecture Draft v1 (was docs/meshbay-draft-v1.md) --> +<!-- ============================================================ --> + +# MeshBay — Architecture Draft v1 + +> Status: preliminary draft — many points still open, marked [TBD] + +--- + +## 1. Project Overview + +MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), aiming to be resilient, censorship-resistant, and user-friendly. + +**Core principles:** +- Data never transits through a central server — only identity and routing do +- End-to-end encryption for all private content (files, indexes, messages) +- The node operator is the legal host and is fully responsible for their content +- The hub is a lightweight registrar, not a content host or indexer +- Open source, self-hostable at every level + +**Domain:** meshbay.org + +--- + +## 2. Terminology + +| Term | Role | +|---|---| +| **Mesh Hub** | Identity authority and group registry server | +| **Mesh Node** | Local program on the host user's machine | +| **Mesh Client** | Web browser or Android app (end user) | +| **Mesh Relay** | Community-operated TURN fallback relay | +| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | +| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | +| **GEK** | Group Encryption Key — symmetric key for group content | +| **Mesh Directory** | Public registry of groups (hub level) | +| **Mesh Group Index** | Encrypted file listing for a group (node level) | + +--- + +## 3. System Components + +### 3.1 Mesh Hub + +A lightweight server acting as a registrar. It is intentionally kept minimal to reduce legal exposure and operational burden. + +**What the hub stores:** +- User accounts: username, hashed email, `PK_user` (public key fingerprint), hub ID, status +- Group registry: name, `PK_group`, hosting node address, visibility, member list with encrypted GEK bundles +- Revocation lists (users and groups) +- Registered peer hubs (explicit allowlist — no auto-discovery) + +**What the hub never stores:** +- File content or metadata +- Private group indexes +- Message content +- Node IP addresses (handled by ephemeral signaling service) + +**Hub interactions — when is it called?** + +| Event | Hub load | Frequency | +|---|---|---| +| Account creation | Hash credential, store PK | Once | +| Login | Verify credentials, issue signed JWT | Per session (~30-day validity) | +| Group creation | Register name, PK_group, node | Once per group | +| Member add/remove | Store/remove encrypted GEK bundle | On admin action | +| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | +| NAT signaling | Relay a few WebSocket messages (<1 KB) | Per new P2P connection | +| Public search | Delegate query to nodes at request time | On demand | +| MHP federation sync | Exchange public directory updates | Background, periodic | +| Revocation | Issue signed revocation token | Rare | + +**The hub is never in the data path after initial connection setup.** + +**JWT as offline passport:** +The hub issues a JWT signed with its Ed25519 private key. Nodes verify this JWT locally using the hub's known public key — no hub roundtrip required per request. JWT validity: ~30 days. + +**Tech stack:** +- Language: Python +- Framework: FastAPI + Uvicorn +- Database: PostgreSQL + SQLAlchemy + Alembic +- Deployment: behind Apache reverse proxy (ProxyPass) +- Authentication: own system (JWT signed with Ed25519, no OAuth dependency) + +**Account creation:** [TBD] — email only at first, phone number associable later. Via Android app, both collected by default. Fusionable accounts. + +### 3.2 Mesh Node + +A local program running on the host user's machine. The node is the actual host of all content. + +**Responsibilities:** +- Watch and index shared directories (Mesh Group Index) +- Serve files and video streams to group members +- Manage all cryptographic keys locally (keystore, password-protected) +- Handle P2P connections and NAT traversal +- Run the MNP protocol +- Host the Python module sandbox +- Serve the local web UI (localhost) +- Receive and redistribute ephemeral video from mobile [future] + +**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. + +**Tech stack:** +- Language: Python (primary), Rust extensions only if strictly necessary for hot paths +- QUIC: `aioquic` +- ICE/STUN: `aioice` +- WebRTC (future): `aiortc` +- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated) +- Serialization: `msgpack` +- Compression: `zstandard` (zstd) +- File watching: `watchdog` +- Local DB: SQLite +- Local web UI: served by the node on localhost (port [TBD]) + +**Node pairing with mobile:** QR code from local web UI [future]. + +### 3.3 Mesh Client + +Web browser or Android app. Consumes content from the node; manages account via the hub. + +**Hub-side operations (via hub):** +- Account creation and login +- Public group search and discovery +- Group membership management + +**Node-side operations (direct P2P):** +- File browsing (Mesh Group Index) +- Message feed reading (with attachments, Signal-like) +- File download +- Video streaming (VOD) +- [Future] Ephemeral video feed + +**Client modes** [to be designed]: +- Explorer mode: browse files in a group +- Feed mode: message thread with attachments +- Hub/Node UI split to be defined + +### 3.4 Mesh Relay + +Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail. Traffic is always E2E encrypted — the relay sees only opaque QUIC packets and cannot read content. + +Not operated by meshbay.org. A protocol for relay registration with hubs is [TBD]. + +--- + +## 4. Group Model + +Groups are the core organizational unit. + +| Parameter | Options | +|---|---| +| Visibility | Public / Private | +| Join policy | Open / On request / By invitation only | +| Admin | The hosting node operator (legal host) | + +A public group functions like a themed forum: shared files, message thread, member list. It can be open entry, request-based, or invitation-only regardless of its public visibility. + +A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members with the GEK can decrypt anything. + +**Group addressing** [TBD]: +``` +meshbay.org/u/username/groupname — public group via hub +meshbay.org/g/groupname — direct public group +group://<PK_group_fingerprint>@<node_addr> — hub-less direct access +``` + +--- + +## 5. Cryptographic Architecture + +### 5.1 Key Hierarchy + +``` +User Identity Key Ed25519 Signing, authentication +User Exchange Key X25519 Key agreement +Group Identity Key Ed25519 Group metadata signing (held by admin node) +Group Encryption Key ChaCha20 Content and index encryption (symmetric, 256-bit) +Session Keys X25519/HKDF Perfect forward secrecy per P2P connection +``` + +All private keys are stored exclusively on the node (or client device), in a password-protected local keystore. The hub never sees any private key. + +### 5.2 GEK Management + +**Group creation:** +1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) +2. GEK is encrypted for each member using X25519 key agreement + HKDF +3. Encrypted GEK bundles stored on hub (or node — [TBD]) + +**Member addition:** +- GEK encrypted with new member's `PK_user` and distributed + +**Member revocation:** +- Admin node generates a new GEK +- Re-encrypts for all remaining members +- New content encrypted with new GEK +- Former member retains ability to decrypt previously received content (acceptable trade-off — full re-encryption not planned) + +### 5.3 On-the-Fly Encryption for File Transfer + +Files are stored in plaintext on the host's disk. The node encrypts at read time before transmission. + +``` +Disk (plaintext) → [Node] → zstd compress → GEK encrypt (per-chunk) → QUIC session → [Client] → QUIC decrypt → GEK decrypt → plaintext +``` + +**Chunking strategy:** +- Chunk size: 1 MB (amortizes AEAD overhead, allows seeking) +- Per-chunk key derivation: + `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` +- Each chunk independently decryptable (enables video seeking) +- Compress before encrypt (zstd compression is useless after encryption) + +**Chunk authentication:** +Each chunk (or batch) is signed with the node's Ed25519 key. The client verifies before decryption. Prevents data injection by a compromised relay. + +### 5.4 Transport Security + +- Primary protocol: **QUIC** (TLS 1.3 integrated, UDP-based, multiplexed) +- Per-connection session keys via X25519 ECDH + HKDF +- The QUIC layer is independent from the GEK application layer — two independent encryption layers + +### 5.5 Chat Encryption + +Group messaging uses the **Double Ratchet algorithm** (as in Signal): +- Forward secrecy and break-in recovery per message +- Each message independently encrypted +- Implementation: existing Python or Rust library [TBD] + +--- + +## 6. Network and Connectivity + +### 6.1 NAT Traversal — Attempt Order + +``` +1. IPv6 available on both sides → direct connection, no NAT issue +2. UPnP / NAT-PMP on router → node opens port automatically +3. ICE + STUN / UDP hole punching → works for ~80-85% of cases +4. Mesh Relay (TURN fallback) → community-operated, E2E encrypted traffic +``` + +**Signaling** (steps 3/4): coordinated via hub WebSocket, <1 KB per attempt, stateless after connection established. + +**Step 4 coverage:** ~15-20% of connections (symmetric NAT on both sides, CGNAT). The relay sees only encrypted QUIC packets. + +### 6.2 MNP — Mesh Node Protocol + +Application-level protocol over QUIC. Defined blocks: + +- **Handshake**: key exchange, group membership verification (JWT presentation) +- **Index sync**: encrypted delta Mesh Group Index on connection +- **File transfer**: chunk request/response with hash verification +- **VOD streaming**: HLS/DASH segments, encrypted per-segment with GEK-derived keys +- **Messaging**: Double Ratchet messages encapsulated in MNP frames +- **[Future] Ephemeral stream**: `ephemeral_stream` type with TTL metadata + +### 6.3 Public Content Delivery + +Public files are identified by their `blake3` hash. Multiple nodes can serve the same file: + +1. Node A has public file X (hash H) +2. Any node that obtains X and chooses to mirror it registers with the hub: "I serve hash H" +3. Hub maintains: `{ blake3_hash → [node_A, node_B, ...] }` +4. Client requests X → hub returns source list → client fetches in parallel chunks from multiple nodes + +**Public content transport:** TLS only (no GEK). Content signed with the original node's Ed25519 key for authenticity verification by clients, even when served from a mirror. Door left open for GEK on "registered-users-only public" groups in a future revision. + +--- + +## 7. Indexes + +### 7.1 Mesh Directory (hub level) + +Public registry of groups. Exchanged between hubs via MHP. + +Format: msgpack, signed by hub's Ed25519 key. + +Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy. + +### 7.2 Mesh Group Index (node level) + +File listing for a group. Generated and maintained by the hosting node. + +Format: msgpack → zstd compressed → GEK encrypted (private groups) or plaintext signed (public groups). + +Entry structure: +```python +{ + "id": "<blake3_hash>", + "name": "filename.mkv", + "path": "Movies/2024/", # relative to shared directory + "size": 4294967296, + "type": "video", # video | audio | image | document | archive | other + "duration": 7245, # seconds, for media + "thumb_hash":"<blake3>", # thumbnail hash (thumbnail also GEK-encrypted) + "added_at": 1720000000 +} +``` + +**Delta updates:** each update carries `{base_version, additions, deletions}` — no full re-encryption on every change. + +**Transit:** nodes push index deltas to connected members on change. Members pull full index on first connection. Hub stores no index content — only the node address for routing. + +### 7.3 Search + +**Private groups:** search is entirely local on the client device. The client maintains a local encrypted cache of all indexes for groups it belongs to. No network call, no hub involvement, instant results. + +**Public groups:** client queries node(s) directly at request time. Hub provides routing (which node hosts which group) but performs no content lookup itself. + +**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this interaction. In-memory micro-cache of results: **60-second TTL maximum, RAM only, never written to disk, public content only.** This qualifies as technical caching (EU DSA Article 13) and does not constitute indexing. + +--- + +## 8. Hub Federation (MHP) + +### 8.1 Hub Hierarchy + +``` +Root Hub (meshbay.org) + ├── Full Hub (self-hosted, CA-delegated) + │ └── issues user credentials, manages its own groups + │ └── can federate with other Full Hubs via MHP + └── Mirror Hub + └── hosts public Mesh Directory only (no user accounts) +``` + +A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub) proving its authority. Clients verify the chain. A Mirror Hub can only replicate public directory data. + +### 8.2 MHP Design Principles + +- Explicit peer selection: each hub maintains an allowlist of trusted peer hubs +- No automatic hub discovery +- Exchanged data: Mesh Directory (public groups), revocation lists, cross-hub user credentials +- Cross-hub authentication: user from Hub A presents JWT signed by Hub A; Hub B verifies using Hub A's public key (fetched once on first interaction, cached) + +### 8.3 Cross-Hub Client Access + +Client from Hub A accessing a group on Hub B: +1. Hub A's public directory or direct link leads client to Hub B +2. Client presents Hub A JWT to Hub B directly +3. Hub B verifies JWT signature using Hub A's public key +4. Hub B issues a short-lived local token for this session +5. Client proceeds to node as normal + +--- + +## 9. Moderation + +### 9.1 Public Content + +``` +Report #1 → automatic suspension of public access to content + → node operator notified +One republication allowed +Report #2 → escalated to hub moderators +Confirmed → group revoked on local hub + → revocation propagated to federated hubs via MHP +``` + +Mechanism: blake3 hash of content added to hub blocklist. Node receives signed revocation notice and cuts public access. + +### 9.2 CSAM + +Hash matching against NCMEC/IWF database on all public content at registration time. Participation demonstrates good faith and significantly reduces legal exposure. No scanning of private/encrypted content. + +### 9.3 Copyright + +DMCA/legal notice framework (takedown on notification). No automated technical blocking — too complex, too many false positives (fair use, regional variations). Hub can revoke on confirmed legal request. + +### 9.4 Private Content + +Not directly moderatable (E2E encrypted by design). Only action available: revoke user or group at hub level on formal legal request. Hub issues a signed revocation token that all group members' nodes can verify. + +--- + +## 10. Python Module System + +The node can load extension modules (Python) that run in a sandboxed subprocess. + +**Module manifest** (declared capabilities): +```python +{ + "name": "group-chat", + "version": "1.0.0", + "permissions": ["read_index", "send_message", "receive_events"] +} +``` + +**Available APIs (restricted):** +- `read_index()` — read current group index (read-only) +- `send_message(content)` — post a message to the group thread +- `receive_events(handler)` — subscribe to group events (new file, new message) + +**Not available:** +- Arbitrary network access +- Filesystem access outside the group context +- System calls + +**First official module:** group chat thread (Signal-like, with attachments). Bundled with node. + +--- + +## 11. Legal Framework + +**Node operator:** primary legal host of content. Fully responsible for what they share. Node software clearly communicates this at setup. + +**Hub operator:** registrar, not content host. Stores minimal PII. Operates takedown mechanism. Participates in CSAM hash matching. Analogous to a domain registrar in legal exposure terms. + +**Protocol/software author:** protected by substantial non-infringing uses. No active facilitation of infringement. + +**Hub data minimization:** +- Email stored hashed after verification [TBD] +- No IP address logging (or auto-deletion after 24h) +- No content metadata stored +- Node current address managed by ephemeral signaling service only + +--- + +## 12. Future Features (noted, not designed) + +- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement +- **Mobile video push:** mobile films → pushes to hosting node → distributed as ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. +- **Mesh Relay registration protocol:** community TURN relays registered with hubs +- **Node mobile pairing:** QR code from local web UI +- **Multi-source download:** parallel chunk fetching from multiple nodes for same public file (swarm) +- **iOS client** +- **At-rest encryption on node:** optional, for nodes deployed on remote servers + +--- + +## 13. Open Questions [TBD] + +1. **GEK bundle storage:** on hub or on node only? Hub = easier discovery; node only = more decentralized +2. **Group address scheme:** final URL format +3. **Hub local web UI scope for V1:** config only, or also group browsing? +4. **Account creation:** email only to start, phone associable — confirm +5. **Chat implementation:** bundled module or core feature? +6. **QUIC library maturity:** `aioquic` production readiness assessment needed +7. **Double Ratchet library:** identify best Python implementation +8. **Relay registration protocol:** design when community relays are introduced +9. **Cross-hub directory exchange:** frequency, conflict resolution +10. **Node port for local web UI:** to assign +11. **JWT expiry and refresh strategy** +12. **Keystore format and unlock mechanism on node startup** + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: Architecture Draft v2 (was docs/meshbay-draft-v2.md) --> +<!-- ============================================================ --> + +# MeshBay — Architecture Draft v2 + +> Status: preliminary draft — open points marked [TBD] +> Changes from v1: IP logging (legal), protocol versioning, hardware sizing, JWT strategy, keystore proposals, chat as core, relay moved to future, hub mirror future, GEK clarified, port 18000, lazy admin keystore. + +--- + +## 1. Project Overview + +MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. + +**Core principles:** +- Data never transits through a central server — only identity and routing do +- End-to-end encryption for all private content (files, indexes, messages) +- The node operator is the legal host and is fully responsible for their content +- The hub is a lightweight registrar, not a content host or indexer +- Open source, self-hostable at every level + +**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) + +--- + +## 2. Terminology + +| Term | Role | +|---|---| +| **Mesh Hub** | Identity authority and group registry server | +| **Mesh Node** | Local program on the host user's machine | +| **Mesh Client** | Web browser or Android app (end user) | +| **Mesh Relay** | Community-operated TURN fallback relay [future] | +| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | +| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | +| **GEK** | Group Encryption Key — symmetric key for private group content | +| **Mesh Directory** | Public registry of groups (hub level) | +| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | + +--- + +## 3. Protocol Versioning + +All protocols (MNP, MHP, hub REST API) carry explicit version information. + +**Format:** `MAJOR.MINOR` +- MAJOR bump: breaking change, backward incompatible +- MINOR bump: backward-compatible addition + +**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. + +**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). + +**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. + +--- + +## 4. System Components + +### 4.1 Mesh Hub + +A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. + +**What the hub stores:** +- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user`, hub ID, status, creation timestamp +- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) +- Mandatory connection logs (see §4.1.2) +- Revocation lists (users and groups) +- Registered peer hubs (explicit allowlist — no auto-discovery) + +**What the hub never stores:** +- File content or metadata +- Private group indexes +- Message content +- Node current IP (handled by ephemeral signaling — see §4.1.3) + +#### 4.1.1 Account Data + +Email is kept in full (not hashed) to support: +- Account recovery (password reset) +- Legal notifications +- Abuse contact + +Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). + +Email and phone are stored encrypted at rest in the database. + +#### 4.1.2 Mandatory IP Logging (Legal Compliance) + +Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: + +| Event | Retention | +|---|---| +| Account creation | 1 year minimum | +| Login (success and failure) | 1 year minimum | +| Group creation | 1 year minimum | +| Group join / leave | 1 year minimum | +| Group deletion | 1 year minimum | +| Revocation actions | 1 year minimum | + +Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. + +#### 4.1.3 Signaling Service + +NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. + +**Hub interaction summary:** + +| Event | Hub crypto load | Frequency | +|---|---|---| +| Account creation | Argon2 hash, store PK | Once | +| Login | Verify password, issue JWT (Ed25519 sign) | Per session | +| Group creation | Register metadata | Once per group | +| Member add/remove | Store/remove GEK bundle | On admin action | +| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | +| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | +| Public search | Delegate to nodes, 60s in-memory cache | On demand | +| MHP federation sync | Exchange Mesh Directory | Background, periodic | +| Revocation | Ed25519-sign revocation token | Rare | + +**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip).** + +#### 4.1.4 JWT Strategy + +Two tokens issued at login: + +**Access token** (JWT, signed Ed25519): +- Validity: 1 hour +- Payload: `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, hub-signed groups membership claim +- Presented to nodes for authentication and group access verification +- Verified locally by nodes using the hub's known public key — no hub roundtrip +- Compromise window: 1 hour maximum + +**Refresh token** (opaque, random 256-bit): +- Validity: 30–90 days [TBD exact duration] +- Stored securely on client only +- Used exclusively with the hub to obtain a new access token +- Revocable immediately by the hub (invalidates all future refreshes for this token) +- Stored server-side as a hashed value + +**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. + +**Tech stack:** +- Language: Python +- Framework: FastAPI + Uvicorn +- Database: PostgreSQL + SQLAlchemy + Alembic +- Deployment: Apache reverse proxy (ProxyPass + SSL termination) +- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) +- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) + +### 4.2 Mesh Node + +A local program running on the host user's machine. The node is the actual host of all content. + +**Responsibilities:** +- Watch and index shared directories (Mesh Group Index) +- Serve files and video streams to group members +- Manage all cryptographic keys locally (encrypted keystore) +- Handle P2P connections and NAT traversal +- Run the MNP protocol +- Host the Python extension module sandbox +- Serve the local web UI (localhost:18000) +- Host the group chat (core feature) + +**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. + +#### 4.2.1 Keystore and Unlock + +Private keys (user identity, group identity, GEK copies) are stored in a local encrypted keystore file. + +**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id (tuned for ~1s derivation on target hardware). + +**Three unlock modes:** + +| Mode | How it works | Security level | +|---|---|---| +| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | +| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | +| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | + +Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). + +#### 4.2.2 Hardware Sizing + +The main constraint is **upload bandwidth**, not CPU or RAM. + +| Scenario | Simultaneous users | Upload needed | CPU | RAM | +|---|---|---|---|---| +| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | +| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | +| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | +| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | +| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | + +Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. + +**Tech stack:** +- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. +- Transport abstraction layer: `Transport` interface decouples QUIC from TCP+TLS fallback +- QUIC: `aioquic` (Cloudflare-maintained). Fallback: TCP + TLS 1.3 + HTTP/2 if QUIC proves insufficient in production +- ICE/STUN: `aioice` +- WebRTC [future]: `aiortc` +- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) +- Serialization: `msgpack` +- Compression: `zstandard` (zstd) +- File watching: `watchdog` +- Local DB: SQLite +- Local web UI: served by node on `localhost:18000` + +### 4.3 Mesh Client + +Web browser or Android app. Consumes content from nodes; manages account via hub. + +**Hub-side operations:** +- Account creation and login (Android: email + phone at registration) +- Public group search and discovery +- Group membership management + +**Node-side operations (direct P2P):** +- File browsing via Mesh Group Index +- Group chat (messages + attachments, Signal-like — core feature) +- File download +- Video streaming (VOD) +- [Future] Ephemeral video feed + +**Client modes** [to be designed]: +- Explorer mode: file browser for group content +- Feed mode: chat thread with attachments +- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation + +### 4.4 Mesh Relay + +**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (~15–20% of connections). Traffic is always E2E encrypted — the relay sees only opaque QUIC packets. + +Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. + +--- + +## 5. Group Model + +Groups are the core organizational unit. + +| Parameter | Options | +|---|---| +| Visibility | Public / Private | +| Join policy | Open / On request / By invitation only | +| Admin | The hosting node operator (legal host) | + +A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). + +A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. + +**Group addressing:** +``` +meshbay.org/u/username/groupname — public group via hub +meshbay.org/g/groupname — public group (shorthand) +group://<PK_group_fingerprint>@<node_addr> — hub-less direct access +``` +`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). + +--- + +## 6. Cryptographic Architecture + +### 6.1 Key Hierarchy + +``` +User Identity Key Ed25519 Signing, authentication +User Exchange Key X25519 Key agreement +Group Identity Key Ed25519 Group metadata signing (held by admin node) +Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) +Session Keys X25519/HKDF Perfect forward secrecy per P2P connection +``` + +All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. + +### 6.2 GEK Management + +**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). + +**Group creation:** +1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) +2. GEK encrypted for each member via X25519 key agreement + HKDF +3. Encrypted GEK bundles stored on hub (opaque blobs — hub cannot decrypt them; charge is negligible: ~200–400 bytes per member per group) + +**Storing on hub rationale:** members can retrieve their GEK bundle even when the node is offline. Hub exposure is minimal — it stores ciphertext it cannot read. + +**Member addition:** +- GEK encrypted with new member's `PK_user` and uploaded to hub + +**Member revocation:** +- Admin node generates new GEK +- Re-encrypts for all remaining members, uploads new bundles +- New content encrypted with new GEK from this point +- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) + +### 6.3 On-the-Fly Encryption for File Transfer + +Files are stored in plaintext on the host's disk. The node encrypts at read time. + +``` +Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → QUIC session → Client → QUIC decrypt → GEK decrypt → plaintext +``` + +**Chunking:** +- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) +- Per-chunk key derivation: `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` +- Each chunk independently decryptable → enables VOD seeking +- Compress before encrypt (compression is ineffective on ciphertext) + +**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. + +**Encryption optimization:** +- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops +- ChaCha20-Poly1305: ~500 MB/s on hardware without AES-NI; AES-256-GCM: >2 GB/s with AES-NI +- For typical home node (50 Mbps upload = 6 MB/s), encryption is not the bottleneck +- For high-concurrency scenarios: asyncio pipeline (read → compress → encrypt → send) without loading full files into memory +- GEK-derived chunk keys computed in batch at transfer start, not per-chunk + +### 6.4 Transport Security + +- Primary: **QUIC** (TLS 1.3 integrated, UDP, multiplexed streams) +- Fallback: **TCP + TLS 1.3 + HTTP/2** (same application protocol, lower performance) +- Transport interface abstracted in code — swappable without protocol changes +- Per-connection session keys via X25519 ECDH + HKDF (independent of GEK layer) + +### 6.5 Chat Encryption + +Group chat is a **core feature** (not an extension module). Uses the **Double Ratchet algorithm** (as in Signal): +- Forward secrecy and break-in recovery per message +- Each message independently encrypted +- Attachment files: encrypted with the current Double Ratchet message key, hash included in message +- Python implementation: [TBD — evaluate existing libraries] + +--- + +## 7. Network and Connectivity + +### 7.1 NAT Traversal — Attempt Order + +``` +1. IPv6 available on both sides → direct connection +2. UPnP / NAT-PMP on router → node opens port automatically +3. ICE + STUN / UDP hole punching → ~80–85% success rate +4. Mesh Relay (TURN) → [future feature] +``` + +Without step 4, ~15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. + +Signaling (step 3): coordinated via hub WebSocket endpoint, <1 KB per attempt, no persistent state. + +### 7.2 MNP — Mesh Node Protocol + +Application-level protocol over QUIC (or TCP+TLS fallback). All messages carry a `version` field. + +**Defined message types:** + +| Type | Description | +|---|---| +| `handshake` | Key exchange, JWT presentation, version negotiation | +| `index_sync` | Encrypted Mesh Group Index delta | +| `file_request` | Request chunk(s) of a file by hash + chunk index | +| `file_chunk` | Chunk data + signature | +| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | +| `chat_message` | Double Ratchet encrypted message frame | +| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | +| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | + +### 7.3 Public Content Delivery — Swarm + +Public files identified by `blake3` hash. Multiple nodes can serve the same file: + +1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub +2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` +3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes +4. Integrity verified by blake3 hash on each chunk + +**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. + +--- + +## 8. Indexes + +### 8.1 Mesh Directory (hub level) + +Public registry of groups, exchanged between hubs via MHP. + +Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. + +Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. + +### 8.2 Mesh Group Index (node level) + +File listing for a group. Generated and maintained by the hosting node. + +Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). + +Entry structure: +```python +{ + "version": 1, + "id": "<blake3_hash>", + "name": "filename.mkv", + "path": "Movies/2024/", + "size": 4294967296, + "type": "video", # video | audio | image | document | archive | other + "duration": 7245, # seconds, for media + "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted + "added_at": 1720000000 +} +``` + +Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. + +Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. + +### 8.3 Search + +**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. + +**Public groups:** client queries nodes directly at request time. Hub provides routing only. + +**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. + +--- + +## 9. Hub Federation (MHP) + +### 9.1 Hub Hierarchy + +``` +Root Hub (meshbay.org) + ├── Full Hub (self-hosted, delegated CA) + │ └── issues user credentials, manages own groups + │ └── federates with other Full Hubs via MHP + └── Mirror Hub + └── hosts public Mesh Directory only (no user accounts, no key issuance) +``` + +A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. + +### 9.2 MHP Design + +- Explicit peer selection: each hub maintains an allowlist of trusted peers +- No automatic hub discovery +- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data +- All MHP messages carry `version` field + +### 9.3 Cross-Hub Client Access + +1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link +2. Client presents Hub A JWT directly to Hub B +3. Hub B verifies JWT using Hub A's public key (fetched once, cached) +4. Hub B issues short-lived local session token +5. Client connects to node as normal + +--- + +## 10. Moderation + +### 10.1 Public Content + +``` +Report #1 → automatic suspension of public access + → node operator notified +One republication allowed +Report #2 → escalated to hub moderators +Confirmed → group revoked on local hub + → revocation propagated to federated hubs via MHP +``` + +Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. + +### 10.2 CSAM + +Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. + +### 10.3 Copyright + +DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. + +### 10.4 Private Content + +Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. + +--- + +## 11. Python Extension Module System + +The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** + +**Module manifest:** +```python +{ + "name": "my-extension", + "version": "1.0.0", + "mnp_version": ">=1.0", + "permissions": ["read_index", "send_message", "receive_events"] +} +``` + +**Available APIs:** +- `read_index()` — read current group index (read-only) +- `send_message(content)` — post to group thread +- `receive_events(handler)` — subscribe to group events + +**Unavailable:** arbitrary network, filesystem access outside group context, system calls. + +--- + +## 12. Legal Framework + +**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. + +**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. + +**Protocol/software author:** protected by substantial non-infringing uses. + +**Hub data:** +- Email and optional phone: kept for account recovery and legal compliance +- Password: Argon2id hash, never stored in cleartext +- Connection logs: retained per legal requirements (minimum 1 year) +- Content metadata: never stored +- Node current IP: not persisted (signaling is ephemeral) + +--- + +## 13. Future Features + +- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic +- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement +- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. +- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. +- **Node–mobile pairing:** QR code from local web UI +- **Multi-source download:** parallel chunk fetching from swarm for public files +- **iOS client** +- **At-rest encryption on node:** optional for server-deployed nodes +- **OS keychain integration for keystore unlock** + +--- + +## 14. Open Questions [TBD] + +1. **Refresh token validity:** 30 or 90 days? +2. **Group address scheme:** final URL format confirmation +3. **Double Ratchet library:** identify best Python implementation +4. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? +5. **MHP federation sync frequency and conflict resolution** +6. **Hub mirror replication strategy** (when implemented) +7. **Chat attachment storage:** stored on node like regular files, or separate store? +8. **Relay registration protocol design** (when implemented) +9. **JWT payload claims:** exact fields to include for node group-access verification +10. **Argon2id parameters:** tuning for target hardware (home server vs. VPS) + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: Architecture Draft v3 (was docs/meshbay-draft-v3.md) --> +<!-- ============================================================ --> + +# MeshBay — Architecture Draft v3 + +> Status: preliminary draft — open points marked [TBD] +> Changes from v2: jti mandatory in JWT (Spike 3), Argon2id params corrected (Spike 1), NAT traversal order corrected (Spike 4), transport flipped to TCP+TLS 1.3 v1 / QUIC v2, GEK wrapping protocol confirmed with exact parameters (Spike 6), hub API table expanded with 4 new endpoints (Spike 6), package structure decided (3 packages, uv monorepo), key persistence requirement added (Spike 6), new sections: Hub API Reference, TCP+TLS Transport v1, Package Structure. + +--- + +## Changes from v2 + +The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. + +| # | Category | What changed | Source | +|---|---|---|---| +| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | +| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | +| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | +| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | +| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | +| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | +| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | +| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | + +--- + +## 1. Project Overview + +MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. + +**Core principles:** +- Data never transits through a central server — only identity and routing do +- End-to-end encryption for all private content (files, indexes, messages) +- The node operator is the legal host and is fully responsible for their content +- The hub is a lightweight registrar, not a content host or indexer +- Open source, self-hostable at every level + +**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) + +--- + +## 2. Terminology + +| Term | Role | +|---|---| +| **Mesh Hub** | Identity authority and group registry server | +| **Mesh Node** | Local program on the host user's machine | +| **Mesh Client** | Web browser or Android app (end user) | +| **Mesh Relay** | Community-operated TURN fallback relay [future] | +| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | +| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | +| **GEK** | Group Encryption Key — symmetric key for private group content | +| **Mesh Directory** | Public registry of groups (hub level) | +| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | + +--- + +## 3. Protocol Versioning + +All protocols (MNP, MHP, hub REST API) carry explicit version information. + +**Format:** `MAJOR.MINOR` +- MAJOR bump: breaking change, backward incompatible +- MINOR bump: backward-compatible addition + +**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. + +**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). + +**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. + +--- + +## 4. System Components + +### 4.1 Mesh Hub + +A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. + +**What the hub stores:** +- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp +- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) +- Mandatory connection logs (see §4.1.2) +- Revocation lists (users and groups) +- Registered peer hubs (explicit allowlist — no auto-discovery) + +**What the hub never stores:** +- File content or metadata +- Private group indexes +- Message content +- Node current IP (handled by ephemeral signaling — see §4.1.3) + +#### 4.1.1 Account Data + +Email is kept in full (not hashed) to support: +- Account recovery (password reset) +- Legal notifications +- Abuse contact + +Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). + +Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** + +#### 4.1.2 Mandatory IP Logging (Legal Compliance) + +Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: + +| Event | Retention | +|---|---| +| Account creation | 1 year minimum | +| Login (success and failure) | 1 year minimum | +| Group creation | 1 year minimum | +| Group join / leave | 1 year minimum | +| Group deletion | 1 year minimum | +| Revocation actions | 1 year minimum | + +Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. + +#### 4.1.3 Signaling Service + +NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. + +**Hub interaction summary:** + +| Event | Hub crypto load | Frequency | +|---|---|---| +| Account creation | Argon2 hash, store PK | Once | +| Login | Verify password, issue JWT (Ed25519 sign) | Per session | +| Group creation | Register metadata | Once per group | +| Member add/remove | Store/remove GEK bundle | On admin action | +| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | +| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | +| Public search | Delegate to nodes, 60s in-memory cache | On demand | +| MHP federation sync | Exchange Mesh Directory | Background, periodic | +| Revocation | Ed25519-sign revocation token | Rare | + +**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** + +#### 4.1.4 JWT Strategy + +Two tokens issued at login: + +**Access token** (JWT, signed Ed25519): +- Validity: 1 hour +- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) +- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. +- Presented to nodes for authentication and group access verification +- Verified locally by nodes using the hub's known public key — no hub roundtrip +- Compromise window: 1 hour maximum + +> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. +> +> This bug was found and fixed during Spike 3. + +**Refresh token** (opaque, random 256-bit): +- Validity: 30–90 days [TBD exact duration] +- Stored securely on client only +- Used exclusively with the hub to obtain a new access token +- Revocable immediately by the hub (invalidates all future refreshes for this token) +- Stored server-side as a hashed value + +**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. + +**Tech stack:** +- Language: Python +- Framework: FastAPI + Uvicorn +- Database: PostgreSQL + SQLAlchemy + Alembic +- Deployment: Apache reverse proxy (ProxyPass + SSL termination) +- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) +- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) + +#### 4.1.5 Hub API Reference + +Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. + +**Hub metadata:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | +| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | + +**User management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | +| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | +| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | +| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | + +**Node management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | +| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | + +**Group management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | +| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | +| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | +| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | + +**GEK distribution (private groups):** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | +| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | + +**Revocation:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/revoke/user/{user_id}` | Access token (admin) | Revoke a user account | [TBD] | +| POST | `/v1/revoke/group/{group_id}` | Access token (admin) | Revoke a group | [TBD] | +| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | + +### 4.2 Mesh Node + +A local program running on the host user's machine. The node is the actual host of all content. + +**Responsibilities:** +- Watch and index shared directories (Mesh Group Index) — one directory per group +- Serve files, video streams, and group chat to members +- Manage all cryptographic keys locally (encrypted keystore) +- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) +- Run the MNP protocol (QUIC v2, TCP+TLS v1) +- Host the Python extension module sandbox +- Serve the local web UI (localhost:18000) + +**Multi-group architecture (decided Phase 7):** +A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated +by port — the MNP handshake identifies the target group via the `group_id` claim +in the client JWT. The server routes each connection to the appropriate +DirectoryIndexer and GEK after JWT verification. +Rationale: one NAT hole to maintain, one port to forward manually if needed. + +**Authorization invariant:** the node MUST verify that the JWT's `groups` claim +contains the requested group_id before serving any content. Without this check, +any authenticated user could access any group on the node. This is enforced at +the MNP handshake layer, not the transport layer. + +**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. + +#### 4.2.1 Keystore and Unlock + +Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. + +**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. + +**Argon2id parameters (production):** +- `iterations = 4` +- `memory_cost = 262144` (256 MB) +- `parallelism = 1` (or match CPU count — tune to target hardware) +- Target derivation time: ~500 ms on a home server + +> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. + +**CLI calibration:** +``` +meshbay-node --calibrate-argon2 +``` +This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. + +**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). + +**Three unlock modes:** + +| Mode | How it works | Security level | +|---|---|---| +| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | +| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | +| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | + +Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). + +#### 4.2.2 Hardware Sizing + +The main constraint is **upload bandwidth**, not CPU or RAM. + +| Scenario | Simultaneous users | Upload needed | CPU | RAM | +|---|---|---|---|---| +| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | +| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | +| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | +| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | +| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | + +Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. + +Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. + +**Tech stack:** +- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. +- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. +- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) +- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) +- ICE/STUN: `aioice` +- WebRTC [future]: `aiortc` +- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) +- Serialization: `msgpack` +- Compression: `zstandard` (zstd) +- File watching: `watchdog` +- Local DB: SQLite +- Local web UI: served by node on `localhost:18000` + +### 4.3 Mesh Client + +Web browser or Android app. Consumes content from nodes; manages account via hub. + +**Hub-side operations:** +- Account creation and login (Android: email + phone at registration) +- Public group search and discovery +- Group membership management + +**Node-side operations (direct P2P):** +- File browsing via Mesh Group Index +- Group chat (messages + attachments, Signal-like — core feature) +- File download +- Video streaming (VOD) +- [Future] Ephemeral video feed + +**Client modes** [to be designed]: +- Explorer mode: file browser for group content +- Feed mode: chat thread with attachments +- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation + +### 4.4 Mesh Relay + +**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. + +Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. + +### 4.5 Package Structure and Repository + +**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). + +``` +meshbay/ +├── packages/ +│ ├── meshbay-common/ # Shared crypto, serialization, protocol types +│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) +│ └── meshbay-node/ # Node daemon + local web UI +├── poc/ # POC and spikes — reference implementation +│ ├── spike1_crypto/ +│ ├── spike2_hub/ +│ ├── spike3_node_reg/ +│ ├── spike4_nat/ +│ ├── spike5_transfer/ +│ ├── spike6_gek/ +│ └── spike-results.md +├── docs/ +│ └── meshbay-draft-v3.md +└── pyproject.toml # Workspace root +``` + +**Three packages:** + +| Package | RPM name | Contents | +|---|---|---| +| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | +| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | +| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | + +**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. + +**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: +- Exact crypto parameter choices (Spike 1) +- GEK wrapping/unwrapping implementation (Spike 6) +- Hub API skeleton (Spike 2) +- NAT detection and STUN interaction (Spike 4) +- TCP file transfer pipeline (Spike 5) + +Developers implementing production features should read the corresponding spike before writing production code. + +--- + +## 5. Group Model + +Groups are the core organizational unit. + +| Parameter | Options | +|---|---| +| Visibility | Public / Private | +| Join policy | Open / On request / By invitation only | +| Admin | The hosting node operator (legal host) | + +A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). + +A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. + +**Group addressing:** +``` +meshbay.org/u/username/groupname — public group via hub +meshbay.org/g/groupname — public group (shorthand) +group://<PK_group_fingerprint>@<node_addr> — hub-less direct access +``` +`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). + +--- + +## 6. Cryptographic Architecture + +### 6.1 Key Hierarchy + +``` +User Identity Key Ed25519 Signing, authentication +User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) +Group Identity Key Ed25519 Group metadata signing (held by admin node) +Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) +Session Keys X25519/HKDF Perfect forward secrecy per P2P connection +``` + +All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. + +Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. + +### 6.1.1 Key Generation Strategies + +Three strategies, depending on client type: + +**A — CLI / native node (Argon2id derivation)** +Keys are derived deterministically from `username + password`: +``` +salt = SHA-256("meshbay:v1:" + username) +seed = Argon2id(password, salt, length=64) +sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) +sk_x25519 = X25519.from_private_bytes(seed[32:]) +``` +Same credentials → same keys on any machine. Password recovery = key recovery. +Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. + +**B — Web browser (random keypairs + encrypted bundle)** +Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them +with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub +alongside the public keys. On subsequent logins, the hub returns the bundle +and the browser decrypts it locally with the password. + +The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. +Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. + +**C — Native node with keystore file** +Random keypairs generated once, stored in the Argon2id-encrypted keystore file +(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. + +**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). +A user who registered via CLI (A) and later tries to recover via web (B) with the same +password will get different keypairs. This is by design: users pick one registration path. +Cross-path recovery requires the admin to issue new GEK bundles. + +### 6.2 GEK Management + +**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). + +**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** + +``` +Admin side (wrap_gek): + sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle + shared = X25519(sk_eph, pk_recipient) + wrap_key = HKDF(shared, salt=pk_eph, + info="meshbay:gek_wrap:v1", + length=32) + nonce = random_bytes(12) + wrapped = ChaCha20-Poly1305(wrap_key).encrypt( + nonce, gek, aad=pk_recipient) # aad binds bundle to recipient + bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire + # hub stores as opaque 48-byte blob + # (without pk_eph in compact form — see note) + +Member side (unwrap_gek): + shared = X25519(sk_recipient, pk_eph) + wrap_key = HKDF(shared, salt=pk_eph, + info="meshbay:gek_wrap:v1", + length=32) + gek = ChaCha20-Poly1305(wrap_key).decrypt( + nonce, wrapped, aad=pk_recipient) +``` + +> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. + +**Security properties confirmed in Spike 6:** +- Hub never sees the GEK in cleartext +- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls +- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected +- Wrong private key → AEAD authentication tag failure → immediate rejection + +**Group creation:** +1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) +2. GEK wrapped for each initial member via the protocol above +3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` +4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` + +**Member addition:** +- Admin fetches new member's `pk_x25519` from hub +- Wraps GEK for them and uploads bundle + +**Member revocation:** +- Admin node generates new GEK +- Re-encrypts for all remaining members, uploads new bundles +- New content encrypted with new GEK from this point +- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) + +**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. + +### 6.3 On-the-Fly Encryption for File Transfer + +Files are stored in plaintext on the host's disk. The node encrypts at read time. + +``` +Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext +``` + +(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) + +**Chunking:** +- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) +- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 +- Each chunk independently decryptable → enables VOD seeking +- Compress before encrypt (compression is ineffective on ciphertext) + +**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. + +**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** + +| Operation | Time | +|---|---| +| Encrypt + sign (node side) | 3.2 ms | +| Verify + decrypt (client side) | 3.9 ms | +| Total crypto overhead (1 MB) | < 10 ms | +| Network transfer | 99–234 ms (network-limited) | + +Encryption is not the bottleneck. Network latency and bandwidth dominate. + +**Pipeline optimization:** +- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops +- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI +- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory +- GEK-derived chunk keys computed in batch at transfer start, not per-chunk + +### 6.4 Transport Security + +**Implementation phases:** + +| Phase | Transport | Status | Notes | +|---|---|---|---| +| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | +| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | + +The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. + +**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. + +**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. + +### 6.5 TCP+TLS 1.3 Transport Implementation (v1) + +**Connection model:** +- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) +- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) +- TLS 1.3 mandatory; TLS 1.2 rejected +- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) +- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain + +**Handshake sequence:** +``` +Client → Node: TCP SYN +Node → Client: TLS ServerHello (self-signed cert, PK_node) +Client: verify cert against hub-fetched PK_node +Client → Node: TLS ClientFinished +Node → Client: MNP handshake request (version negotiation) +Client → Node: MNP handshake response (JWT access token, version) +Node: verify JWT offline (Ed25519, hub public key) +Node → Client: session established +``` + +**Message framing over TCP:** +- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` +- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) +- Each frame carries the MNP `version` field in its header + +**QUIC migration path (v2):** +- Replace TCP length-framing with QUIC streams (one stream per logical exchange) +- MNP handshake maps 1:1 to a QUIC handshake stream +- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) +- Chat messages map to a persistent QUIC stream +- No changes to JWT verification, GEK decryption, or Index sync logic + +**Port allocation:** +- `18000/tcp` — local web UI (loopback only, not exposed externally) +- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) +- Configurable via `~/.config/meshbay/node.toml` + +### 6.6 Chat Encryption and Model + +Group chat is a **core feature** (not an extension module). + +**Model (decided):** between a forum and Signal. +- **Persistent:** messages stored on the node (not ephemeral like Signal by default) +- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages +- **Scope:** per group (not per user pair) +- **Attachments:** files and images, shared like regular group files +- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect +- **Retention:** managed by the group admin (no automatic expiry) + +**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** + +The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): + +- Each group member generates a **sender key** (random symmetric chain key + signing keypair) +- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) +- Each existing member sends their current sender key to the new member +- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) +- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys +- O(N) state per member (one chain per group member), not O(N^2) +- The existing Double Ratchet implementation is kept for future 1:1 direct messaging + +Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. + +> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. + +--- + +## 7. Network and Connectivity + +### 7.1 NAT Traversal — Attempt Order + +``` +1. IPv6 available on both sides → direct connection (preferred) +2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) +3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) +4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile +``` + +> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. + +**Spike 4 findings:** +- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) +- UDP hole punching functional: bidirectional echo received from OVH VPS +- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results +- No CGNAT: stable public IPv4 (81.220.170.32) + +Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. + +**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** +Currently the node punches blindly at startup; the client may connect 10-20s later +on an aging NAT entry, causing retransmissions. The coordinated flow uses the +existing hub→node WebSocket (revocation channel): +``` +Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} +Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} +Node : punch_nat(peer_ip, peer_port) immediately +Node → Hub (WS) : {type: "punch_ready"} +Hub → Client: 200 OK "connect now" +Client → QUIC: first packet < 2s after probe → fresh NAT entry +``` +demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. +The probe must come from the QUIC server's own socket (`punch_nat()` via +`_transport.sendto()`). The QUIC client must connect from the same port +as the probe's destination (`local_port=QUIC_PORT`). Handshake time +with proper signaling: < 200ms (vs 12.7s without). + +### 7.2 MNP — Mesh Node Protocol + +Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. + +**Defined message types:** + +| Type | Description | +|---|---| +| `handshake` | Key exchange, JWT presentation, version negotiation | +| `index_sync` | Encrypted Mesh Group Index delta | +| `file_request` | Request chunk(s) of a file by hash + chunk index | +| `file_chunk` | Chunk data + Ed25519 signature | +| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | +| `chat_message` | Double Ratchet encrypted message frame | +| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | +| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | + +### 7.3 Public Content Delivery — Swarm + +Public files identified by `blake3` hash. Multiple nodes can serve the same file: + +1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub +2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` +3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes +4. Integrity verified by blake3 hash on each chunk + +**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. + +--- + +## 8. Indexes + +### 8.1 Mesh Directory (hub level) + +Public registry of groups, exchanged between hubs via MHP. + +Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. + +Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. + +### 8.2 Mesh Group Index (node level) + +File listing for a group. Generated and maintained by the hosting node. + +Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). + +Entry structure: +```python +{ + "version": 1, + "id": "<blake3_hash>", + "name": "filename.mkv", + "path": "Movies/2024/", + "size": 4294967296, + "type": "video", # video | audio | image | document | archive | other + "duration": 7245, # seconds, for media + "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted + "added_at": 1720000000 +} +``` + +Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. + +Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. + +### 8.3 Search + +**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. + +**Public groups:** client queries nodes directly at request time. Hub provides routing only. + +**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. + +--- + +## 9. Hub Federation (MHP) + +### 9.1 Hub Hierarchy + +``` +Root Hub (meshbay.org) + ├── Full Hub (self-hosted, delegated CA) + │ └── issues user credentials, manages own groups + │ └── federates with other Full Hubs via MHP + └── Mirror Hub + └── hosts public Mesh Directory only (no user accounts, no key issuance) +``` + +A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. + +### 9.2 MHP Design + +- Explicit peer selection: each hub maintains an allowlist of trusted peers +- No automatic hub discovery +- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data +- All MHP messages carry `version` field + +### 9.3 Cross-Hub Client Access + +1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link +2. Client presents Hub A JWT directly to Hub B +3. Hub B verifies JWT using Hub A's public key (fetched once, cached) +4. Hub B issues short-lived local session token +5. Client connects to node as normal + +--- + +## 10. Moderation + +### 10.1 Public Content + +``` +Report #1 → automatic suspension of public access + → node operator notified +One republication allowed +Report #2 → escalated to hub moderators +Confirmed → group revoked on local hub + → revocation propagated to federated hubs via MHP +``` + +Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. + +### 10.2 CSAM + +Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. + +### 10.3 Copyright + +DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. + +### 10.4 Private Content + +Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. + +--- + +## 11. Python Extension Module System + +The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** + +**Module manifest:** +```python +{ + "name": "my-extension", + "version": "1.0.0", + "mnp_version": ">=1.0", + "permissions": ["read_index", "send_message", "receive_events"] +} +``` + +**Available APIs:** +- `read_index()` — read current group index (read-only) +- `send_message(content)` — post to group thread +- `receive_events(handler)` — subscribe to group events + +**Unavailable:** arbitrary network, filesystem access outside group context, system calls. + +--- + +## 12. Legal Framework + +**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. + +**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. + +**Protocol/software author:** protected by substantial non-infringing uses. + +**Hub data:** +- Email and optional phone: kept for account recovery and legal compliance +- Password: Argon2id hash, never stored in cleartext +- Connection logs: retained per legal requirements (minimum 1 year) +- Content metadata: never stored +- Node current IP: not persisted (signaling is ephemeral) +- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them + +--- + +## 13. Future Features + +- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic. Necessary for symmetric NAT (CGNAT mobile, some professional ISPs). +- **QUIC transport (v2):** replace TCP+TLS 1.3 with QUIC once application protocol is stable. Transport abstraction layer makes this a drop-in replacement. +- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement +- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. +- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. +- **Node–mobile pairing:** QR code from local web UI +- **Multi-source download:** parallel chunk fetching from swarm for public files +- **iOS client** +- **At-rest encryption on node:** optional for server-deployed nodes +- **OS keychain integration for keystore unlock** +- **WebRTC:** `aiortc` for browser-native P2P (no node required for clients) + +--- + +## 14. Open Questions [TBD] + +**Resolved by POC (no longer open):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R1 | Argon2id parameters: what values target ~500ms? | `iterations=4, memory_cost=262144` (256 MB). Use `meshbay-node --calibrate-argon2` for hardware-specific tuning. | Spike 1 | +| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, group membership claim. `jti` is mandatory (prevents replay, enables revocation). | Spike 3 | +| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). Hub stores opaque 48-byte blobs. | Spike 6 | +| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). UPnP unreliable — demoted to step 3. | Spike 4 | +| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1 (lower complexity, works everywhere). QUIC for v2 via `Transport` abstraction. | Spike 5 | +| R6 | Hub API: which endpoints for GEK distribution? | `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek` | Spike 6 | +| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. uv workspace monorepo. | POC | + +**Resolved by first security review (2026-08-10):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R8 | Group chat encryption model? | Sender Keys protocol (Signal Groups approach). Double Ratchet kept for future 1:1 DM only. MLS considered for v2 if groups > 50 members. | Security review C1 | +| R9 | Token denylist distribution? | Push via existing hub→node WebSocket. Node maintains an in-memory jti set. MNP handshake checks the set before accepting a JWT. No periodic polling needed. | Security review S3 | +| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. GEK is CSPRNG output (already uniform), so HKDF extract step doesn't need a random salt. Spec wording corrected to match code (RFC 5869 compliant). | Security review M5 | +| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D recommendation. Code fixed from 128-bit to 96-bit. | Security review S4 | + +**Still open:** + +1. **Refresh token validity:** 30 or 90 days? +2. **Group address scheme:** final URL format confirmation +3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? +4. **MHP federation sync frequency and conflict resolution** +5. **Hub mirror replication strategy** (when implemented) +6. **Chat attachment storage:** stored on node like regular files, or separate store? +7. **Relay registration protocol design** (when implemented) +8. **QUIC migration timeline:** when is the application protocol considered stable enough to begin v2 transport implementation? +9. **Refresh token rotation:** implement one-time-use refresh tokens (rotate on each use, detect reuse as theft indicator). RFC 6819 §5.2.2.3. +10. **Email encryption at rest:** spec requires encrypted email/phone in DB, implementation stores plaintext. Needs server-side encryption with key from hub config. + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: Architecture Draft v4 (was docs/meshbay-draft-v4.md) --> +<!-- ============================================================ --> + +# MeshBay — Architecture Draft v4 + +> Status: active development — Phases 1–12 complete (except 10.9 → Phase 13), 191 tests. +> Changes from v3: web client transport (WebRTC DataChannel), web UI architecture, hub roles (admin/moderator), hub mirror design, browser-specific NAT traversal, chat storage clarified, Phase 8 security items resolved, Phase 10 site overlay + admin/moderation UI + notifications + group search + version endpoint, Phase 10b self-service UI (group create/join/invite, file upload, IndexedDB caching, cross-group search), **node sovereignty model** (§4.2.x — node operator is sole content authority, deny-by-default, uploader_id tracking), **cryptographic sovereignty enforcement** (GEK-HMAC handshake challenge, Ed25519 admin challenge-response, gek_req removed), **Phase 12 — P2P crypto material** (GEK+keypair bundles moved off hub to node BundleStore, password split, key persistence in IndexedDB/sessionStorage, DTLS channel binding fix). + +--- + +## Changes from v3 + +The following items are **architectural decisions** driven by Phase 8 implementation and web client design (2026-08-10). They supersede the corresponding text in v3. + +| # | Category | What changed | Source | +|---|---|---|---| +| 1 | Browser transport | Web browsers use **WebRTC DataChannel** (with ICE/STUN) for P2P to nodes behind NAT. WebTransport cannot work because browsers cannot choose their UDP source port — Port-Restricted Cone NAT requires exact port matching. Native clients (desktop, Android) continue using QUIC with `punch_nat()`. | Web client design session | +| 2 | Hub signaling | Hub WebSocket extended to relay WebRTC signaling (SDP/ICE) between browser and node. <1 KB per message, stateless, no content. Same channel as jti denylist push and `client_incoming`. | Web client design session | +| 3 | Hub role | Reinforced: hub is registrar + signaling facilitator ONLY. Never proxies, stores, or touches content (files, streams, chat, indexes). All data lives on nodes. Clients connect E2E to nodes. | Design constraint | +| 4 | Chat storage | Chat messages stored on node(s) hosting the group, not on the hub. Browser retrieves chat from node via DataChannel. If no node is online, group is unavailable. | Web client design session | +| 5 | Web UI | Preact SPA (~3 KB gzipped), dark/light theme, responsive, i18n (JSON translations). ESM modules, esbuild for minification. No heavy frameworks. | Web client design session | +| 6 | Hub roles | Three roles: `user`, `moderator`, `admin`. Moderator can review reports and suspend content/groups/users. Admin has full hub management. | Web client design session | +| 7 | Site overlay | meshbay.org serves both generic hub functionality and site-specific pages (landing, /downloads, /about). Separated via Caddy static file priority. | Web client design session | +| 8 | Hub mirror | Design defined (future implementation): active-active with shared signing key, PostgreSQL logical replication, DNS round-robin. Not implemented yet. | Web client design session | +| 9 | Security items | S1 (admin authz), S2 (email encryption), S5 (refresh token rotation) resolved in Phase 8. Argon2id bumped to 256 MB with transparent rehash. | Phase 8 implementation | +| 10 | File search | Client-side search on cached indexes (IndexedDB). No hub involvement. Private group indexes are GEK-encrypted — hub stores opaque, client decrypts locally. | Web client design session | +| 11 | P2P crypto material | **ALL crypto material moved off hub to P2P channel.** GEK bundles and keypair bundles stored on node (`BundleStore` SQLite), exchanged via MNP DataChannel. Hub `GEKBundle` model and `/gek` endpoint removed. Hub never touches, stores, or proxies any crypto material. | Phase 12 — T3 attack surface reduction | +| 12 | Password split | Hub receives `auth_key` (PBKDF2-SHA512, auth salt), never raw password. Separate `bundle_key` (PBKDF2-SHA512, bundle salt) encrypts keypair bundles on the node. Hub cannot derive `bundle_key` from `auth_key`. | Phase 12 — T1 | +| 13 | Node auth | Node daemon authenticates to hub via Ed25519 signed timestamp (`POST /v1/nodes/auth`), not password. JWT `scope: "node"` blocks group mutation endpoints. | Phase 12 — NS7 | +| 14 | Key persistence | Browser stores `_bundleKey` (CryptoKey) in IndexedDB and `_sessionKeys` in sessionStorage. Survives page refresh without re-login. Public key derived from recovered private key via JWK export (`_pkFromSk`), no hub dependency. | Phase 12 — browser hardening | +| 15 | DTLS channel binding | Browser saves raw answer SDP before `setRemoteDescription` (Chrome may drop sha-256 fingerprint). GEK-HMAC uses `_rawAnswerSdp` for fingerprint extraction. | Phase 12 — handshake fix | + +--- + +## Changes from v2 + +The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. + +| # | Category | What changed | Source | +|---|---|---|---| +| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | +| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | +| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | +| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | +| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | +| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | +| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | +| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | + +--- + +## 1. Project Overview + +MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. + +**Core principles:** +- Data never transits through a central server — only identity and routing do +- End-to-end encryption for all private content (files, indexes, messages) +- The node operator is the legal host and is fully responsible for their content +- The hub is a lightweight registrar, not a content host or indexer +- Open source, self-hostable at every level + +**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) + +--- + +## 2. Terminology + +| Term | Role | +|---|---| +| **Mesh Hub** | Identity authority and group registry server | +| **Mesh Node** | Local program on the host user's machine | +| **Mesh Client** | Web browser or Android app (end user) | +| **Mesh Relay** | Community-operated TURN fallback relay [future] | +| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | +| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | +| **GEK** | Group Encryption Key — symmetric key for private group content | +| **Mesh Directory** | Public registry of groups (hub level) | +| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | + +--- + +## 3. Protocol Versioning + +All protocols (MNP, MHP, hub REST API) carry explicit version information. + +**Format:** `MAJOR.MINOR` +- MAJOR bump: breaking change, backward incompatible +- MINOR bump: backward-compatible addition + +**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. + +**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). + +**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. + +--- + +## 4. System Components + +### 4.1 Mesh Hub + +A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. + +**What the hub stores:** +- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp +- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) +- Mandatory connection logs (see §4.1.2) +- Revocation lists (users and groups) +- Registered peer hubs (explicit allowlist — no auto-discovery) + +**What the hub never stores:** +- File content or metadata +- Private group indexes +- Message content +- Node current IP (handled by ephemeral signaling — see §4.1.3) + +#### 4.1.1 Account Data + +Email is kept in full (not hashed) to support: +- Account recovery (password reset) +- Legal notifications +- Abuse contact + +Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). + +Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** + +#### 4.1.2 Mandatory IP Logging (Legal Compliance) + +Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: + +| Event | Retention | +|---|---| +| Account creation | 1 year minimum | +| Login (success and failure) | 1 year minimum | +| Group creation | 1 year minimum | +| Group join / leave | 1 year minimum | +| Group deletion | 1 year minimum | +| Revocation actions | 1 year minimum | + +Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. + +#### 4.1.3 Signaling Service + +NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. + +**Hub interaction summary:** + +| Event | Hub crypto load | Frequency | +|---|---|---| +| Account creation | Argon2 hash, store PK | Once | +| Login | Verify password, issue JWT (Ed25519 sign) | Per session | +| Group creation | Register metadata | Once per group | +| Member add/remove | Store/remove GEK bundle | On admin action | +| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | +| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | +| Public search | Delegate to nodes, 60s in-memory cache | On demand | +| MHP federation sync | Exchange Mesh Directory | Background, periodic | +| Revocation | Ed25519-sign revocation token | Rare | + +**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** + +#### 4.1.4 JWT Strategy + +Two tokens issued at login: + +**Access token** (JWT, signed Ed25519): +- Validity: 1 hour +- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) +- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. +- Presented to nodes for authentication and group access verification +- Verified locally by nodes using the hub's known public key — no hub roundtrip +- Compromise window: 1 hour maximum + +> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. +> +> This bug was found and fixed during Spike 3. + +**Refresh token** (opaque, random 256-bit): +- Validity: 30–90 days [TBD exact duration] +- Stored securely on client only +- Used exclusively with the hub to obtain a new access token +- Revocable immediately by the hub (invalidates all future refreshes for this token) +- Stored server-side as a hashed value + +**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. + +**Tech stack:** +- Language: Python +- Framework: FastAPI + Uvicorn +- Database: PostgreSQL + SQLAlchemy + Alembic +- Deployment: Apache reverse proxy (ProxyPass + SSL termination) +- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) +- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) + +#### 4.1.5 Hub API Reference + +Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. + +**Hub metadata:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | +| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | + +**User management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | +| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | +| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | +| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | + +**Node management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | +| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | + +**Group management:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | +| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | +| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | +| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | + +**GEK distribution (private groups):** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | +| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | + +**Revocation:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| POST | `/v1/admin/revoke` | Access token (admin) | Revoke a user or group | ✓ Phase 8 | +| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | + +**Admin / moderation:** + +| Method | Path | Auth | Description | Status | +|---|---|---|---|---| +| GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) | ✓ Phase 10 | +| GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) | ✓ Phase 10 | +| GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable by username) | ✓ Phase 10 | +| GET | `/v1/admin/users/{user_id}` | Moderator+ | User detail (email, role, status, group count) | ✓ Phase 10 | +| PATCH | `/v1/admin/users/{user_id}` | Moderator+ | Update user role or status | ✓ Phase 10 | +| GET | `/v1/admin/groups` | Moderator+ | List all groups with member count | ✓ Phase 10 | +| PATCH | `/v1/admin/groups/{group_id}` | Moderator+ | Update group status | ✓ Phase 10 | +| GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user_id) | ✓ Phase 10 | +| GET | `/v1/admin/blocklist` | Admin | List blocked content hashes | ✓ Phase 8 | +| POST | `/v1/admin/blocklist` | Admin | Manually block a content hash | ✓ Phase 8 | +| DELETE | `/v1/admin/blocklist/{hash}` | Admin | Unblock a content hash | ✓ Phase 8 | +| GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) | ✓ Phase 10 | +| POST | `/v1/notifications/{id}/read` | Access token | Mark notification as read | ✓ Phase 10 | +| POST | `/v1/notifications/read-all` | Access token | Mark all notifications as read | ✓ Phase 10 | +| GET | `/v1/groups?q=` | None | Search public groups by name (ilike) | ✓ Phase 10 | +| GET | `/v1/hub/version` | None | Client version check (hub, MNP, MHP) | ✓ Phase 10 | +| GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) | ✓ Phase 10b | +| POST | `/v1/groups/{id}/join` | Access token | Self-join open group | ✓ Phase 10b | + +### 4.2 Mesh Node + +A local program running on the host user's machine. The node is the actual host of all content. + +**Responsibilities:** +- Watch and index shared directories (Mesh Group Index) — one directory per group +- Serve files, video streams, and group chat to members +- Manage all cryptographic keys locally (encrypted keystore) +- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) +- Run the MNP protocol (QUIC v2, TCP+TLS v1) +- Host the Python extension module sandbox +- Serve the local web UI (localhost:18000) + +**Multi-group architecture (decided Phase 7):** +A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated +by port — the MNP handshake identifies the target group via the `group_id` claim +in the client JWT. The server routes each connection to the appropriate +DirectoryIndexer and GEK after JWT verification. +Rationale: one NAT hole to maintain, one port to forward manually if needed. + +**Authorization invariant:** the node MUST verify that the JWT's `groups` claim +contains the requested group_id before serving any content. Without this check, +any authenticated user could access any group on the node. This is enforced at +the MNP handshake layer, not the transport layer. + +#### 4.2.x Node Sovereignty — Content Authorization Model + +The node operator is the **sole authority** over content stored on their machine. +No external actor — including the hub admin — can modify, delete, or control +files on a node they do not operate. This is a non-negotiable design invariant, +enforced by **cryptography**, not just policy. + +**Two trust domains, strictly separated:** + +| Domain | Authority | Scope | +|---|---|---| +| **Hub** | Hub admin / moderator | User accounts, group registry, group membership, GEK distribution, moderation (suspend user/group at hub level) | +| **Node** | Node operator | Files on disk, file deletion, upload acceptance, chat storage, who can do what with node content | + +The hub certifies **identity** (JWT) and **group membership** (`groups` claim). +The node decides **authorization for content operations** based on that identity. +These two concerns must never be conflated. + +##### Cryptographic enforcement — two defense layers + +A malicious hub admin controls the JWT signing key and could forge JWTs to +impersonate any user, including the node operator. Policy-only checks (comparing +`user_id` to `node_user_id`) are insufficient because the hub controls the +identity layer. Two cryptographic mechanisms make this impossible: + +**Layer 1 — GEK proof in handshake (membership verification):** + +After JWT verification, the node challenges the connecting user to prove they +possess the Group Encryption Key (GEK). The hub never has the GEK — it only +stores opaque ECIES-wrapped bundles. Without the GEK, a hub admin who forges +a JWT still cannot access any group content. + +``` +Client → Node: handshake { token, group_id } +Node: verify JWT, verify group_id in claims + nonce = random(32) +Node → Client: handshake_challenge { nonce: base64(nonce) } +Client: proof = HMAC-SHA256(GEK, nonce) +Client → Node: handshake_response { proof: base64(proof) } +Node: verify HMAC — if wrong, reject connection +Node → Client: handshake_ack { is_node_admin, node_pk, v } +``` + +This blocks: content reading, index reading, chat reading, file upload, chat +injection — ALL operations require passing the GEK proof first. + +**Layer 2 — Ed25519 challenge-response for admin operations:** + +The node operator's Ed25519 public key is pinned locally in `node.toml` +(auto-pinned from keystore on first startup). Destructive operations (file +deletion) require the user to sign a random challenge with their Ed25519 +private key. The hub cannot forge this signature. + +``` +Client → Node: file_delete { file_id } +Node: (if uploader → allow immediately) + (else) challenge = random(32) +Node → Client: admin_challenge { challenge: base64(challenge), file_id } +Client: signature = Ed25519.sign(sk_ed, challenge) +Client → Node: admin_response { signature: base64(signature), file_id } +Node: verify(admin_pk_ed25519, signature, challenge) + if valid → delete file +``` + +**Node configuration — admin key pinning:** + +```toml +# node.toml +admin_pk_ed25519 = "base64-encoded-32-bytes-raw-Ed25519-public-key" +``` + +Auto-pinned from the node operator's keystore on first startup. The daemon +logs: "Admin Ed25519 key pinned for node sovereignty". + +**GEK distribution — browser flow (node no longer serves GEK):** + +The node NEVER serves the GEK in plaintext. Browser clients obtain the GEK +from their hub-stored encrypted bundle: + +1. `GET /v1/groups/{id}/gek` → encrypted ECIES bundle (AES-256-GCM variant) +2. Browser unwraps with its X25519 private key (from keypair bundle) +3. Browser uses raw GEK bytes for the handshake HMAC proof +4. Browser imports GEK as HKDF key for chunk decryption + +This eliminates the `gek_req`/`gek_resp` MNP messages from the protocol. + +**Authorization rules for destructive file operations (enforced by the node):** + +| Action | Who can do it | Enforcement point | +|---|---|---| +| Delete a file | Node operator (Ed25519 challenge-response) OR the user who uploaded it | Node (`_do_file_delete`) | +| Delete any file | Node operator only (Ed25519 challenge-response) | Node (`_do_file_delete`) | + +Default posture: **deny.** If the admin key is not pinned, all admin operations +are refused. If the GEK proof fails, the connection is refused entirely. + +**Protocol enforcement — MNP handshake_ack:** + +The handshake_ack message carries `is_node_admin: bool` — the node tells the +client whether the authenticated user is the node operator. Clients MUST use +this node-reported flag (not the hub's `group.admin_id`) to decide whether +to show destructive operations like file deletion. + +``` +handshake_ack: + v: "0.1" + node_pk: "<base64>" + is_node_admin: true | false # node-side authorization, NOT hub-side +``` + +**Index entry — uploader tracking:** + +Each `IndexEntry` carries an `uploader_id` field (user_id of who uploaded the +file, or null for files that pre-existed on disk). This enables the "uploader +can delete their own files" rule without granting node-admin privileges. + +**What the hub admin CANNOT do on a node they don't operate:** +- Delete files (requires Ed25519 key pinned on node — hub can't forge) +- Read files (requires GEK — hub never has it) +- Read index / chat (requires GEK proof in handshake) +- Upload files (requires GEK proof in handshake) +- Impersonate the node operator (JWT forgery blocked by Ed25519 challenge) + +**What the hub admin CAN do (hub-level only):** +- Suspend a user account (blocks JWT issuance → user loses access everywhere) +- Suspend a group (blocks signaling → no new P2P connections to nodes for that group) +- These are hub-level actions that don't touch node content + +**Remaining trust assumptions:** +- The hub serves the SPA code to browsers (a malicious hub could inject JS — fundamentally unsolvable in browser; native client or browser extension required for full integrity) +- The hub relays WebRTC signaling — ✅ MITIGATED: DTLS channel binding in GEK-HMAC proof (`HMAC(GEK, nonce || offer_fp || answer_fp)`) detects fingerprint substitution (MitM) +- The hub receives raw password at login — ✅ MITIGATED: password split (auth_key ≠ bundle_key, independent PBKDF2 derivations). Hub receives auth_key only, cannot derive bundle_key to decrypt keypair bundle. Legacy accounts migrated on first login. +- The hub controls public key distribution — can substitute keys during invite to intercept GEK. Fix: out-of-band key verification (safety numbers) — Phase 12 + +> **Design lesson (2026-08-12):** The initial implementation conflated hub +> `group.admin_id` (who created the group on the hub) with node operator +> authority (who runs the machine). The SPA used the hub's `is_admin` flag +> to show file deletion controls, and the node's delete handler used a +> fail-open check (`if node_user_id and ...` — allowed everyone when +> `node_user_id` was not set). Both violated node sovereignty. Fixed by: +> (1) deny-by-default on the node, (2) `is_node_admin` in handshake_ack, +> (3) `uploader_id` tracking in the index, (4) SPA uses node-reported +> permissions only. Then hardened with cryptographic enforcement: +> (5) GEK-HMAC proof in handshake (blocks forged-JWT access), +> (6) Ed25519 challenge-response for admin ops (blocks identity impersonation), +> (7) removal of `gek_req` endpoint (node never serves GEK in plaintext), +> (8) DTLS channel binding in GEK-HMAC proof to detect WebRTC signaling MitM, +> (9) chat `sender_id` fixed to authenticated identity (prevents impersonation), +> (10) Ed25519 challenge for ALL file deletions — uploaders verified by stored pk, not JWT sub, +> (11) password split — hub receives PBKDF2 auth_key, never raw password (cannot derive bundle_key). + +**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. + +#### 4.2.1 Keystore and Unlock + +Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. + +**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. + +**Argon2id parameters (production):** +- `iterations = 4` +- `memory_cost = 262144` (256 MB) +- `parallelism = 1` (or match CPU count — tune to target hardware) +- Target derivation time: ~500 ms on a home server + +> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. + +**CLI calibration:** +``` +meshbay-node --calibrate-argon2 +``` +This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. + +**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). + +**Three unlock modes:** + +| Mode | How it works | Security level | +|---|---|---| +| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | +| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | +| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | + +Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). + +#### 4.2.2 Hardware Sizing + +The main constraint is **upload bandwidth**, not CPU or RAM. + +| Scenario | Simultaneous users | Upload needed | CPU | RAM | +|---|---|---|---|---| +| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | +| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | +| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | +| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | +| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | + +Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. + +Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. + +**Tech stack:** +- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. +- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. +- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) +- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) +- ICE/STUN: `aioice` (already a dependency) +- WebRTC: `aiortc` (browser P2P transport — Phase 9) +- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) +- Serialization: `msgpack` +- Compression: `zstandard` (zstd) +- File watching: `watchdog` +- Local DB: SQLite +- Local web UI: served by node on `localhost:18000` + +### 4.3 Mesh Client + +Web browser or Android app. Consumes content from nodes; manages account via hub. +The hub is never in the data path — clients connect E2E to nodes for all content. + +**Hub-side operations (HTTPS, lightweight):** +- Account creation, login, token refresh +- Public group search and discovery +- Group membership management, GEK bundle retrieval +- WebRTC signaling relay (SDP/ICE — <1 KB per connection, stateless) +- Notification metadata (invitations, new content indicators) + +**Node-side operations (direct P2P via QUIC or WebRTC DataChannel):** +- File browsing via Mesh Group Index +- File download (chunked, E2E encrypted) +- Video streaming (HLS segments via DataChannel or QUIC stream) +- Group chat (Sender Keys encrypted, stored on node) +- File/photo/video upload (client → node push) + +#### 4.3.1 Web Browser Client + +**Transport:** WebRTC DataChannel with ICE/STUN for NAT traversal. +WebTransport (HTTP/3) is not suitable because browsers cannot choose their UDP +source port — Port-Restricted Cone NAT (confirmed on SFR residential) requires +the client to connect from the exact port the node probed. WebRTC's ICE handles +this automatically via simultaneous STUN binding requests. + +**UI:** Preact SPA (~3 KB gzipped) served by the hub. +- Dark/light theme (CSS `prefers-color-scheme` + user toggle in localStorage) +- Responsive design (sidebar → hamburger menu on mobile) +- i18n: JSON translation files, English default +- Build: esbuild for minification (single binary, no npm dependency) +- Crypto: SubtleCrypto (AES-GCM) for E2E decryption in browser + +**Layout:** +- Left sidebar: group list (ordered by usage — private groups first), navigation +- Top bar: logo ("MeshBay") left, user menu right (settings, profile, language, logout) +- Main content area: file explorer, chat view, or settings depending on context + +**Client modes:** +- Explorer: file/folder browser for group content (read-only browse, download, stream) +- Chat/forum: per-group discussion thread with photo/video posting +- Settings: general, per-group, notifications, privacy, theme, language + +**Local storage:** +- IndexedDB: cached group indexes for instant local search (~50–100 MB quota) +- localStorage: theme preference, language, session state +- `keypair_bundle`: encrypted keypair retrieved from hub, decrypted locally with password + +**File search:** entirely client-side on cached indexes. No hub involvement. +Private group indexes are GEK-encrypted — stored opaque on the hub, decrypted +by the client locally. Search runs against the decrypted index in IndexedDB. + +#### 4.3.2 Android Client + +**Transport:** QUIC with `punch_nat()` — same as desktop native clients. +Android has full UDP access; no WebRTC needed. Uses `quiche` (Cloudflare, Rust +via JNI) for QUIC transport. + +**Stack:** Kotlin + Jetpack Compose. Bouncy Castle JVM for crypto. + +**Capabilities:** same as web browser (browse, download, stream, chat, upload). +Additional: contact list integration (Android Contacts API, permission-gated). +Account creation from app. No node functionality on mobile (client-only). + +**Cross-device compatibility:** the `keypair_bundle` (encrypted, stored on hub) +enables seamless switching between web and Android with the same credentials. +Notification state and read markers sync via hub (small encrypted blob per user). + +**Out of scope:** Mac/iPhone support. Node on mobile. + +### 4.4 Mesh Relay + +**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. + +Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. + +### 4.5 Package Structure and Repository + +**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). + +``` +meshbay/ +├── packages/ +│ ├── meshbay-common/ # Shared crypto, serialization, protocol types +│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) +│ └── meshbay-node/ # Node daemon + local web UI +├── poc/ # POC and spikes — reference implementation +│ ├── spike1_crypto/ +│ ├── spike2_hub/ +│ ├── spike3_node_reg/ +│ ├── spike4_nat/ +│ ├── spike5_transfer/ +│ ├── spike6_gek/ +│ └── spike-results.md +├── docs/ +│ └── meshbay-draft-v3.md +└── pyproject.toml # Workspace root +``` + +**Three packages:** + +| Package | RPM name | Contents | +|---|---|---| +| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | +| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | +| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | + +**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. + +**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: +- Exact crypto parameter choices (Spike 1) +- GEK wrapping/unwrapping implementation (Spike 6) +- Hub API skeleton (Spike 2) +- NAT detection and STUN interaction (Spike 4) +- TCP file transfer pipeline (Spike 5) + +Developers implementing production features should read the corresponding spike before writing production code. + +--- + +## 5. Group Model + +Groups are the core organizational unit. + +| Parameter | Options | +|---|---| +| Visibility | Public / Private | +| Join policy | Open / On request / By invitation only | +| Node admin | The hosting node operator — sovereign over content, sole delete authority (see §4.2.x) | +| Hub group creator | The user who registered the group on the hub — manages membership and GEK distribution | + +A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). + +A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. + +**Group addressing:** +``` +meshbay.org/u/username/groupname — public group via hub +meshbay.org/g/groupname — public group (shorthand) +group://<PK_group_fingerprint>@<node_addr> — hub-less direct access +``` +`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). + +--- + +## 6. Cryptographic Architecture + +### 6.1 Key Hierarchy + +``` +User Identity Key Ed25519 Signing, authentication +User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) +Group Identity Key Ed25519 Group metadata signing (held by admin node) +Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) +Session Keys X25519/HKDF Perfect forward secrecy per P2P connection +``` + +All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. + +Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. + +### 6.1.1 Key Generation Strategies + +Three strategies, depending on client type: + +**A — CLI / native node (Argon2id derivation)** +Keys are derived deterministically from `username + password`: +``` +salt = SHA-256("meshbay:v1:" + username) +seed = Argon2id(password, salt, length=64) +sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) +sk_x25519 = X25519.from_private_bytes(seed[32:]) +``` +Same credentials → same keys on any machine. Password recovery = key recovery. +Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. + +**B — Web browser (random keypairs + encrypted bundle)** +Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them +with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub +alongside the public keys. On subsequent logins, the hub returns the bundle +and the browser decrypts it locally with the password. + +The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. +Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. + +**C — Native node with keystore file** +Random keypairs generated once, stored in the Argon2id-encrypted keystore file +(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. + +**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). +A user who registered via CLI (A) and later tries to recover via web (B) with the same +password will get different keypairs. This is by design: users pick one registration path. +Cross-path recovery requires the admin to issue new GEK bundles. + +### 6.2 GEK Management + +**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). + +**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** + +``` +Admin side (wrap_gek): + sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle + shared = X25519(sk_eph, pk_recipient) + wrap_key = HKDF(shared, salt=pk_eph, + info="meshbay:gek_wrap:v1", + length=32) + nonce = random_bytes(12) + wrapped = ChaCha20-Poly1305(wrap_key).encrypt( + nonce, gek, aad=pk_recipient) # aad binds bundle to recipient + bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire + # hub stores as opaque 48-byte blob + # (without pk_eph in compact form — see note) + +Member side (unwrap_gek): + shared = X25519(sk_recipient, pk_eph) + wrap_key = HKDF(shared, salt=pk_eph, + info="meshbay:gek_wrap:v1", + length=32) + gek = ChaCha20-Poly1305(wrap_key).decrypt( + nonce, wrapped, aad=pk_recipient) +``` + +> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. + +**Security properties confirmed in Spike 6:** +- Hub never sees the GEK in cleartext +- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls +- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected +- Wrong private key → AEAD authentication tag failure → immediate rejection + +**Group creation:** +1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) +2. GEK wrapped for each initial member via the protocol above +3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` +4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` + +**Member addition:** +- Admin fetches new member's `pk_x25519` from hub +- Wraps GEK for them and uploads bundle + +**Member revocation:** +- Admin node generates new GEK +- Re-encrypts for all remaining members, uploads new bundles +- New content encrypted with new GEK from this point +- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) + +**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. + +### 6.3 On-the-Fly Encryption for File Transfer + +Files are stored in plaintext on the host's disk. The node encrypts at read time. + +``` +Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext +``` + +(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) + +**Chunking:** +- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) +- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 +- Each chunk independently decryptable → enables VOD seeking +- Compress before encrypt (compression is ineffective on ciphertext) + +**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. + +**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** + +| Operation | Time | +|---|---| +| Encrypt + sign (node side) | 3.2 ms | +| Verify + decrypt (client side) | 3.9 ms | +| Total crypto overhead (1 MB) | < 10 ms | +| Network transfer | 99–234 ms (network-limited) | + +Encryption is not the bottleneck. Network latency and bandwidth dominate. + +**Pipeline optimization:** +- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops +- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI +- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory +- GEK-derived chunk keys computed in batch at transfer start, not per-chunk + +### 6.4 Transport Security + +**Implementation phases:** + +| Phase | Transport | Status | Notes | +|---|---|---|---| +| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | +| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | + +The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. + +**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. + +**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. + +### 6.5 TCP+TLS 1.3 Transport Implementation (v1) + +**Connection model:** +- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) +- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) +- TLS 1.3 mandatory; TLS 1.2 rejected +- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) +- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain + +**Handshake sequence:** +``` +Client → Node: TCP SYN +Node → Client: TLS ServerHello (self-signed cert, PK_node) +Client: verify cert against hub-fetched PK_node +Client → Node: TLS ClientFinished +Node → Client: MNP handshake request (version negotiation) +Client → Node: MNP handshake response (JWT access token, version) +Node: verify JWT offline (Ed25519, hub public key) +Node → Client: session established +``` + +**Message framing over TCP:** +- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` +- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) +- Each frame carries the MNP `version` field in its header + +**QUIC migration path (v2):** +- Replace TCP length-framing with QUIC streams (one stream per logical exchange) +- MNP handshake maps 1:1 to a QUIC handshake stream +- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) +- Chat messages map to a persistent QUIC stream +- No changes to JWT verification, GEK decryption, or Index sync logic + +**Port allocation:** +- `18000/tcp` — local web UI (loopback only, not exposed externally) +- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) +- Configurable via `~/.config/meshbay/node.toml` + +### 6.6 Chat Encryption and Model + +Group chat is a **core feature** (not an extension module). + +**Model (decided):** between a forum and Signal. +- **Persistent:** messages stored on the node (not ephemeral like Signal by default) +- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages +- **Scope:** per group (not per user pair) +- **Attachments:** files and images, shared like regular group files +- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect +- **Retention:** managed by the group admin (no automatic expiry) + +**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** + +The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): + +- Each group member generates a **sender key** (random symmetric chain key + signing keypair) +- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) +- Each existing member sends their current sender key to the new member +- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) +- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys +- O(N) state per member (one chain per group member), not O(N^2) +- The existing Double Ratchet implementation is kept for future 1:1 direct messaging + +Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. + +> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. + +--- + +## 7. Network and Connectivity + +### 7.1 NAT Traversal — Attempt Order + +``` +1. IPv6 available on both sides → direct connection (preferred) +2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) +3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) +4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile +``` + +> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. + +**Spike 4 findings:** +- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) +- UDP hole punching functional: bidirectional echo received from OVH VPS +- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results +- No CGNAT: stable public IPv4 (81.220.170.32) + +Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. + +**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** +Currently the node punches blindly at startup; the client may connect 10-20s later +on an aging NAT entry, causing retransmissions. The coordinated flow uses the +existing hub→node WebSocket (revocation channel): +``` +Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} +Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} +Node : punch_nat(peer_ip, peer_port) immediately +Node → Hub (WS) : {type: "punch_ready"} +Hub → Client: 200 OK "connect now" +Client → QUIC: first packet < 2s after probe → fresh NAT entry +``` +demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. +The probe must come from the QUIC server's own socket (`punch_nat()` via +`_transport.sendto()`). The QUIC client must connect from the same port +as the probe's destination (`local_port=QUIC_PORT`). Handshake time +with proper signaling: < 200ms (vs 12.7s without). + +#### 7.1.1 Browser-Specific NAT Traversal (WebRTC DataChannel) + +Browsers cannot use the QUIC `punch_nat()` mechanism because WebTransport does +not allow the browser to choose its UDP source port. Port-Restricted Cone NAT +requires exact port matching on both IP and port — impossible for browsers. + +**Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC +stack handles NAT traversal automatically: + +1. Browser gathers ICE candidates via STUN (discovers its external IP:port) +2. Node gathers ICE candidates via `aioice` (discovers its external IP:port) +3. Candidates exchanged via hub signaling (WebSocket relay, <1 KB) +4. ICE connectivity checks: both sides send STUN binding requests simultaneously +5. STUN binding requests serve as NAT hole-punching (both directions) +6. ICE finds a valid candidate pair — DataChannel established +7. MNP protocol runs over DataChannel (same messages, same E2E encryption) + +**Signaling flow:** +``` +Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer {sdp, ice_candidates} +Hub → Node (WS) : {type: "webrtc_offer", sdp, ice_candidates, peer_id} +Node (aiortc) : creates PeerConnection, gathers answer candidates +Node → Hub (WS) : {type: "webrtc_answer", sdp, ice_candidates, peer_id} +Hub → Browser (SSE) : answer SDP + ICE candidates +DataChannel : P2P established — hub no longer involved +``` + +ICE is strictly superior to custom `punch_nat()` for browser use: +- No need for the client to pre-announce its port +- Handles both sides behind NAT simultaneously +- Automatic candidate prioritization and fallback +- Battle-tested by billions of daily users (Google Meet, Discord, Zoom) + +**Node dual transport:** the node listens on both: +- QUIC (UDP port 19000) — native clients (desktop, Android) +- WebRTC — browsers (via `aiortc`, separate UDP socket managed by ICE) + +The MNP application protocol is identical on both transports. Same handshake, +same file_request/file_chunk, same chat_message, same E2E encryption. + +### 7.2 MNP — Mesh Node Protocol + +Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. + +**Defined message types:** + +| Type | Description | +|---|---| +| `handshake` | Key exchange, JWT presentation, version negotiation | +| `handshake_challenge` | Node sends GEK proof nonce (base64, 32 bytes random) — see §4.2.x | +| `handshake_response` | Client proves GEK possession: HMAC-SHA256(GEK, nonce) | +| `handshake_ack` | Node response: version, node public key, `is_node_admin` (node-level authorization) | +| `index_sync` | Encrypted Mesh Group Index delta | +| `file_request` | Request chunk(s) of a file by hash + chunk index | +| `file_chunk` | Chunk data + Ed25519 signature | +| `file_delete` | Client requests file deletion by file_id | +| `file_delete_ack` | Node confirms deletion | +| `file_upload` | Client pushes file chunk to node | +| `file_upload_ack` | Node acknowledges chunk receipt | +| `admin_challenge` | Node sends Ed25519 sign challenge for admin ops (base64, 32 bytes) | +| `admin_response` | Client returns Ed25519 signature over the challenge | +| `stream_request` | Client requests MSE video stream | +| `stream_init` | Node sends codec info + signals stream start | +| `stream_data` | Node sends encrypted fMP4 segment | +| `stream_end` | Node signals end of stream | +| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | +| `chat_message` | Sender Keys encrypted message frame (group chat) | +| `chat_history` | Client requests chat history | +| `chat_history_response` | Node responds with stored messages | +| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | +| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | + +### 7.3 Public Content Delivery — Swarm + +Public files identified by `blake3` hash. Multiple nodes can serve the same file: + +1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub +2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` +3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes +4. Integrity verified by blake3 hash on each chunk + +**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. + +--- + +## 8. Indexes + +### 8.1 Mesh Directory (hub level) + +Public registry of groups, exchanged between hubs via MHP. + +Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. + +Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. + +### 8.2 Mesh Group Index (node level) + +File listing for a group. Generated and maintained by the hosting node. + +Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). + +Entry structure: +```python +{ + "version": 1, + "id": "<blake3_hash>", + "name": "filename.mkv", + "path": "Movies/2024/", + "size": 4294967296, + "type": "video", # video | audio | image | document | archive | other + "duration": 7245, # seconds, for media + "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted + "added_at": 1720000000, + "uploader_id": "<user_id>" # who uploaded this file (null = pre-existing on disk) +} +``` + +Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. + +Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. + +### 8.3 Search + +**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. + +**Public groups:** client queries nodes directly at request time. Hub provides routing only. + +**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. + +--- + +## 9. Web Client UI + +### 9.1 Architecture + +The web client is a Preact SPA served by the hub at `/app/`. It communicates +with the hub via HTTPS (auth, group management, signaling) and with nodes via +WebRTC DataChannel (file transfer, streaming, chat). The hub is never in the +data path. + +**Technology choices:** +- **Preact** (~3 KB gzipped): lightweight React-compatible framework +- **preact-router**: client-side routing (no server round-trips) +- **esbuild**: minification/bundling (single binary, no npm/node_modules) +- **SubtleCrypto**: browser-native AES-GCM for E2E decryption +- **IndexedDB**: local cache for group indexes (client-side search) + +No heavy frameworks (React, Vue, Angular). No build toolchain dependencies beyond +esbuild. ESM modules loaded natively by modern browsers. + +### 9.2 UI Structure + +``` +┌─────────────────────────────────────────────────────────┐ +│ [MeshBay] [User ▾] [⚙] │ +├──────────┬──────────────────────────────────────────────┤ +│ │ │ +│ Groups │ Main content area │ +│ │ │ +│ ● Private│ - File explorer (folders, files, download) │ +│ Group1 │ - Chat/forum view │ +│ Group2 │ - Video player (HLS via MediaSource API) │ +│ │ - Settings │ +│ ○ Public │ - Notifications feed │ +│ Group3 │ │ +│ │ │ +└──────────┴──────────────────────────────────────────────┘ +``` + +- **Left sidebar:** group list, ordered by usage frequency. Private groups first. + Collapses to hamburger menu on mobile viewports. +- **Top bar:** logo (left), user menu dropdown (right) — settings, profile, + language, online/offline status, logout. +- **Main area:** context-dependent content based on selected group and view. + +### 9.3 Views + +**Front page (no group selected):** +- Notification feed, prioritized: known contacts → private group activity → public +- System notifications (maintenance, updates) +- Quick access to recent groups + +**Group view — File Explorer:** +- Directory tree (folders, subfolders) — read-only browsing +- File metadata: name, size, type, date added +- Actions: download, stream (for media files) +- Files fetched directly from node via DataChannel +- Upload: photos/videos posted to the group's shared upload folder +- Delete: node operator can delete any file; uploader can delete their own uploads. + Hub admin has NO delete authority on nodes they don't operate (see §4.2.x). + +**Group view — Chat/Forum:** +- Sender Keys encrypted messages, fetched from node +- Post text, photos, videos (uploads go to node, not hub) +- Optional thread/topic structure for organized discussions +- Real-time push for connected members, pull history on reconnect + +**Group view — Video Player:** +- HLS segments fetched via DataChannel from node +- Decrypted client-side (GEK-derived key per segment) +- Played via MediaSource API (browser-native, no plugins) + +**Settings:** +- General: theme (dark/light/auto), language, notification preferences +- Per-group: notification mute, display options, filtering/blocking +- Privacy: online/offline status, profile visibility +- Profile: display name, avatar, account details + +### 9.4 Theming and i18n + +**Theme:** CSS custom properties for colors, toggled via: +1. `prefers-color-scheme` media query (OS default) +2. User override stored in localStorage +3. Toggle button in top bar or settings + +**i18n:** JSON translation files loaded client-side. +``` +static/i18n/ +├── en.json # English (default, always loaded) +├── fr.json # French (loaded on demand) +└── ... # Other languages added later +``` + +Keys are identifiers, not English text. Translation function: `t('group.join')`. + +### 9.5 meshbay.org Site Overlay + +meshbay.org serves both the generic hub application and site-specific pages: + +``` +site/ # meshbay.org-specific (not packaged with hub) +├── index.html # Landing page — project promotion, features +├── downloads.html # Package repos: Ubuntu, Fedora, Android APK +├── about.html # Project info, team, GitHub, contact +└── assets/ # Landing-specific CSS, images, icons +``` + +Caddy serves `site/` with priority. Requests not matching a static file fall +through to the hub FastAPI application. The hub serves `/app/` (SPA) and `/v1/` +(API). This separation ensures the hub package remains generic and deployable +by any operator, while meshbay.org has its own public-facing identity. + +### 9.6 Hub Mirror (future — design only) + +A mirror hub is a complete active-active replica of the primary hub. + +**Purpose:** load distribution for growing traffic. DNS round-robin (2+ A records). + +**Design:** +- Shared Ed25519 signing key (transferred once, securely) +- PostgreSQL logical replication for bidirectional read/write +- Both mirrors issue JWTs with the same key +- Both mirrors accept registrations, logins, and group operations +- If one mirror goes down, the other serves all traffic + +**Implementation constraints (must not violate in current development):** +- Hub config and key paths must be externalizable (already the case) +- No hub-instance-specific state that cannot be replicated +- JWT verification must not depend on hub-local state (already the case) +- Session state (refresh tokens, IP logs) must be in PostgreSQL (already the case) + +**Not implemented now.** Design documented to avoid blocking decisions. + +--- + +## 10. Hub Federation (MHP) <!-- was §9 in v3 --> + +### 9.1 Hub Hierarchy + +``` +Root Hub (meshbay.org) + ├── Full Hub (self-hosted, delegated CA) + │ └── issues user credentials, manages own groups + │ └── federates with other Full Hubs via MHP + └── Mirror Hub + └── hosts public Mesh Directory only (no user accounts, no key issuance) +``` + +A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. + +### 9.2 MHP Design + +- Explicit peer selection: each hub maintains an allowlist of trusted peers +- No automatic hub discovery +- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data +- All MHP messages carry `version` field + +### 9.3 Cross-Hub Client Access + +1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link +2. Client presents Hub A JWT directly to Hub B +3. Hub B verifies JWT using Hub A's public key (fetched once, cached) +4. Hub B issues short-lived local session token +5. Client connects to node as normal + +--- + +## 11. Moderation <!-- was §10 in v3 --> + +### 10.1 Public Content + +``` +Report #1 → automatic suspension of public access + → node operator notified +One republication allowed +Report #2 → escalated to hub moderators +Confirmed → group revoked on local hub + → revocation propagated to federated hubs via MHP +``` + +Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. + +### 10.2 CSAM + +Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. + +### 10.3 Copyright + +DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. + +### 10.4 Private Content + +Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. + +--- + +## 12. Python Extension Module System <!-- was §11 in v3 --> + +The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** + +**Module manifest:** +```python +{ + "name": "my-extension", + "version": "1.0.0", + "mnp_version": ">=1.0", + "permissions": ["read_index", "send_message", "receive_events"] +} +``` + +**Available APIs:** +- `read_index()` — read current group index (read-only) +- `send_message(content)` — post to group thread +- `receive_events(handler)` — subscribe to group events + +**Unavailable:** arbitrary network, filesystem access outside group context, system calls. + +--- + +## 13. Legal Framework <!-- was §12 in v3 --> + +**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. + +**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. + +**Protocol/software author:** protected by substantial non-infringing uses. + +**Hub data:** +- Email and optional phone: kept for account recovery and legal compliance +- Password: Argon2id hash, never stored in cleartext +- Connection logs: retained per legal requirements (minimum 1 year) +- Content metadata: never stored +- Node current IP: not persisted (signaling is ephemeral) +- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them + +--- + +## 14. Future Features <!-- was §13 in v3 --> + +- **Mesh Relay:** community TURN relays, E2E encrypted traffic. Low priority — typical residential NAT works with ICE/STUN. Needed only for symmetric NAT (CGNAT mobile, ~15% of connections). +- ~~**QUIC transport (v2)**~~ ✅ DONE (Phase 5) — QUIC replaces TCP+TLS. +- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement +- **Hub mirror (load balancing):** design documented in §9.6. Active-active with shared key, PostgreSQL replication, DNS round-robin. Implementation deferred. +- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL. MNP `ephemeral_stream` type reserved. +- **Node–mobile pairing:** QR code from local web UI +- **Multi-source download:** parallel chunk fetching from swarm for public files +- **At-rest encryption on node:** optional for server-deployed nodes +- **OS keychain integration for keystore unlock** +- ~~**WebRTC**~~ ✅ Validated (Phase 9.1–9.5) — `aiortc` for browser-to-node P2P via DataChannel. Tested on SFR residential NAT (Port-Restricted Cone) + 4G CGNAT. No TURN needed. +- **Extension-triggered views:** local apps providing custom views for group content (gallery, kanban). MNP extension hook reserved. + +--- + +## 15. Open Questions [TBD] + +**Resolved by POC (no longer open):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R1 | Argon2id parameters: what values target ~500ms? | `iterations=3, memory_cost=262144` (256 MB). pw_version=2, transparent rehash on login. | Spike 1 + Phase 8.10 | +| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, `groups` claim. | Spike 3 + Phase 7 | +| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). | Spike 6 | +| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). | Spike 4 | +| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1, QUIC for v2. QUIC is now the active transport (Phase 5). | Spike 5 | +| R6 | Hub API: which endpoints for GEK distribution? | 4 endpoints confirmed. | Spike 6 | +| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. | POC | + +**Resolved by first security review (2026-08-10):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R8 | Group chat encryption model? | Sender Keys protocol. Double Ratchet kept for future 1:1 DM. | Security review C1 | +| R9 | Token denylist distribution? | Push via hub→node WebSocket. In-memory jti set on node. | Security review S3 | +| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. RFC 5869 compliant. | Security review M5 | +| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D. | Security review S4 | + +**Resolved by Phase 8 implementation (2026-08-10):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R12 | Refresh token rotation? | One-time-use with family-based reuse detection. Old token reuse revokes entire family. | Phase 8.3 | +| R13 | Email encryption at rest? | AES-256-GCM, key derived from hub Ed25519 private key via HKDF(info="meshbay:email:v1"). | Phase 8.2 | +| R14 | Admin authorization model? | Config-based: `admin_usernames` in hub.toml + `MESHBAY_ADMIN_USERS` env var. | Phase 8.1 | +| R15 | QUIC migration timeline? | Done — QUIC is the active transport since Phase 5. | Phase 5 | + +**Resolved by web client design session (2026-08-10):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R16 | Browser transport for NAT traversal? | WebRTC DataChannel with ICE/STUN. WebTransport cannot work (port-restricted cone NAT). | Design session | +| R17 | Chat storage location? | On nodes, not hub. Hub never stores content. | Design session | +| R18 | Web UI framework? | Preact SPA (~3 KB), esbuild, dark/light theme, i18n, responsive. | Design session | +| R19 | Hub mirror design? | Active-active, shared signing key, PostgreSQL replication, DNS round-robin. | Design session | + +**Resolved by Phase 9 spike (2026-08-10):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R20 | WebRTC DataChannel validation? | Confirmed: browser→NAT→node file transfer works. Tested 3 scenarios on SFR residential (Port-Restricted Cone NAT) + 4G CGNAT: WiFi LAN (IPv6 direct, ~100ms), 4G IPv6 inter-network (~600ms), 4G IPv4 STUN hole-punch (~650ms). No TURN relay needed. | Phase 9.5 spike | + +**Resolved by node sovereignty fix (2026-08-12):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R21 | Who authorizes file deletion on a node? | Node operator (sovereign) OR original uploader. Hub admin has no authority over node content. Enforced: deny-by-default in MNP `file_delete`, `is_node_admin` in handshake_ack, `uploader_id` in IndexEntry. | Security fix — §4.2.x | +| R22 | Can a malicious hub admin access node content? | No. Two cryptographic layers: (1) GEK-HMAC proof in handshake — hub never has the GEK, can't pass the challenge. (2) Ed25519 challenge-response for admin ops — hub can't forge the node operator's signature. `gek_req` endpoint removed. | Crypto enforcement — §4.2.x | + +**Resolved by Phase 12 — P2P crypto material (2026-08-13):** + +| # | Question | Resolution | Source | +|---|---|---|---| +| R23 | Where are GEK bundles stored? | On node only (`BundleStore` SQLite, `data_dir/bundles.db`). Hub `GEKBundle` model removed. Exchanged via MNP `gek_bundle_store`/`gek_bundle_fetch`/`gek_bundle_resp` over WebRTC DataChannel. | Phase 12 — T3 | +| R24 | Where are keypair bundles stored? | On node only (`BundleStore`). Encrypted with password-derived AES key (`bundle_key`). Browser pushes after registration, recovers during handshake. Hub `keypair_bundle` column removed. | Phase 12 — T3 | +| R25 | How does the browser recover keys after localStorage cleared? | Transport fetches `keypair_bundle` from node during handshake, decrypts with `_bundleKey` (PBKDF2 from password). Public key derived from private key via JWK export — no hub fetch needed. `_bundleKey` persisted in IndexedDB, `_sessionKeys` in sessionStorage. | Phase 12 | +| R26 | How does the browser handle Chrome SDP re-serialization? | `this._rawAnswerSdp = answer.sdp` saved before `setRemoteDescription`. DTLS fingerprint extracted from raw SDP, not `pc.remoteDescription.sdp` (Chrome may drop sha-256 line when re-serializing multi-hash SDP from aiortc). | Phase 12 | +| R27 | Should the browser auto-regenerate keys on login? | No. Auto-regeneration silently rotates hub keys, breaking GEK unwrap (GEK wrapped for old keys). Keys recovered from node via `_bundleKey`. Regeneration only on explicit user request. | Phase 12 | + +**Still open:** + +1. **Refresh token validity:** 30 or 90 days? +2. **Group address scheme:** final URL format confirmation +3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? → Resolved: always on node. +4. **MHP federation sync frequency and conflict resolution** +5. **Chat attachment storage:** stored on node like regular files, or separate store? +6. **Relay registration protocol design** (when implemented) + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: POC v1 (was docs/poc-v1.md) --> +<!-- ============================================================ --> + +# MeshBay — POC v1 + +> Goal: validate key concepts before committing to a full implementation. +> Scope: Hub/Node exchange in Python, crypto stack, NAT traversal, encrypted file chunk transfer. +> Everything in-memory (no database), minimal code, TCP only (no QUIC yet). + +--- + +## Environment + +### Remote — meshbay.org (Hub) +- OVH VPS, Ubuntu 26.04 LTS, Python 3.14.4 +- Public fixed IP, ports 80 and 443 open +- Clean slate: no web server installed +- SSH access: `ssh cbesson@meshbay.org` + +### Local — Fedora 44 (Node) +- Laptop behind SFR residential NAT (likely Restricted Cone NAT — UPnP supported) +- Python 3.13+ via system packages +- User: `cbesson` (sudoer, no password) + +--- + +## Python Dependencies + +```bash +# Shared (hub and node) +cryptography>=43.0 # Ed25519, X25519, ChaCha20-Poly1305, Argon2id +PyJWT>=2.9 # JWT with EdDSA (Ed25519) support +blake3>=1.0 # Fast content hashing + +# Hub only (meshbay.org) +fastapi>=0.115 +uvicorn[standard]>=0.30 + +# Node only (Fedora laptop) +httpx>=0.28 # Async HTTP client for hub→node calls +aioice>=0.9 # STUN queries for NAT discovery +miniupnpc>=2.2 # UPnP port mapping on SFR box +``` + +Install on each machine: +```bash +python3 -m venv .venv +source .venv/bin/activate +pip install <packages above> +``` + +--- + +## Hub Setup on meshbay.org + +For the POC, uvicorn runs directly on port 80 via iptables redirect (no Caddy/nginx needed yet — HTTPS added before production). + +```bash +# On meshbay.org +# Redirect port 80 → 8000 (persistent via iptables-save if needed) +sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000 + +# Run hub (from poc directory, venv activated) +uvicorn hub:app --host 127.0.0.1 --port 8000 --reload +``` + +> Note: HTTPS (via Caddy + Let's Encrypt) is mandatory before any data beyond this POC. Not in scope here. + +--- + +## Spike Overview + +| # | Name | Where | Validates | Duration | +|---|---|---|---|---| +| 1 | Crypto primitives | Local | Python crypto stack covers all needs | ~1h | +| 2 | Hub skeleton | meshbay.org | Hub API, JWT issuance | ~2h | +| 3 | Node registration | Fedora | Hub-Node handshake, JWT offline verify | ~1h | +| 4 | NAT traversal | Both | SFR box UPnP + STUN, P2P reachability | ~2h | +| 5 | Encrypted transfer | Both | On-the-fly GEK encryption, P2P chunk | ~2h | + +--- + +## Spike 1 — Crypto Primitives (local only) + +**Goal:** confirm `cryptography` (PyCA) covers all MeshBay cryptographic needs without gaps or performance surprises. + +**File:** `spike1_crypto.py` + +**What to test:** + +```python +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey +from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 +from cryptography.hazmat.primitives.kdf.hkdf import HKDF +from cryptography.hazmat.primitives.kdf.argon2 import Argon2id # PyCA 43+ +from cryptography.hazmat.primitives import hashes, serialization +import blake3, os, time +``` + +**Test 1: Ed25519 — hub keypair, sign JWT payload, verify** +```python +sk_hub = Ed25519PrivateKey.generate() +pk_hub = sk_hub.public_key() +msg = b"test payload" +sig = sk_hub.sign(msg) +pk_hub.verify(sig, msg) # raises if invalid +print("Ed25519 OK") +``` + +**Test 2: X25519 — two-party key agreement for GEK wrapping** +```python +sk_a = X25519PrivateKey.generate() +sk_b = X25519PrivateKey.generate() +shared_a = sk_a.exchange(sk_b.public_key()) +shared_b = sk_b.exchange(sk_a.public_key()) +assert shared_a == shared_b +print("X25519 OK") +``` + +**Test 3: GEK derivation and ChaCha20-Poly1305 on a 1 MB chunk** +```python +gek = ChaCha20Poly1305.generate_key() +cipher = ChaCha20Poly1305(gek) +chunk = os.urandom(1024 * 1024) # 1 MB + +t0 = time.perf_counter() +nonce = os.urandom(12) +ct = cipher.encrypt(nonce, chunk, None) +pt = cipher.decrypt(nonce, ct, None) +elapsed = time.perf_counter() - t0 + +assert pt == chunk +print(f"ChaCha20-Poly1305 1MB: {elapsed*1000:.1f} ms") +``` + +**Test 4: HKDF chunk key derivation** +```python +from cryptography.hazmat.primitives.kdf.hkdf import HKDF +from cryptography.hazmat.primitives import hashes +chunk_key = HKDF( + algorithm=hashes.SHA256(), length=32, salt=None, + info=b"file:" + blake3.blake3(chunk).digest() + b":chunk:0" +).derive(gek) +print(f"HKDF derived key: {chunk_key.hex()[:16]}...") +``` + +**Test 5: Argon2id keystore key derivation** +```python +from cryptography.hazmat.primitives.kdf.argon2 import Argon2id +salt = os.urandom(16) +t0 = time.perf_counter() +kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) +key = kdf.derive(b"mypassword") +print(f"Argon2id: {(time.perf_counter()-t0)*1000:.0f} ms, key: {key.hex()[:16]}...") +``` + +**Test 6: PyJWT with Ed25519 (EdDSA)** +```python +import jwt +sk_hub_pem = sk_hub.private_bytes( + serialization.Encoding.PEM, + serialization.PrivateFormat.PKCS8, + serialization.NoEncryption() +) +pk_hub_pem = pk_hub.public_bytes( + serialization.Encoding.PEM, + serialization.PublicFormat.SubjectPublicKeyInfo +) +payload = {"sub": "user_abc", "pk_user": "base64...", "exp": 9999999999} +token = jwt.encode(payload, sk_hub_pem, algorithm="EdDSA") +decoded = jwt.decode(token, pk_hub_pem, algorithms=["EdDSA"]) +assert decoded["sub"] == "user_abc" +print("JWT EdDSA OK") +``` + +**Success criteria:** all tests pass, ChaCha20 1MB < 20ms, Argon2id ~1s. + +--- + +## Spike 2 — Hub Skeleton (meshbay.org) + +**Goal:** minimal FastAPI hub, in-memory storage, 5 endpoints. + +**File:** `hub.py` (on meshbay.org) + +### Hub keypair generation (run once, save to disk) + +```python +# gen_hub_keys.py — run once on meshbay.org +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives import serialization +import base64, json + +sk = Ed25519PrivateKey.generate() +pk = sk.public_key() + +with open("hub_private.pem", "wb") as f: + f.write(sk.private_bytes( + serialization.Encoding.PEM, + serialization.PrivateFormat.PKCS8, + serialization.NoEncryption() + )) +with open("hub_public.pem", "wb") as f: + f.write(pk.public_bytes( + serialization.Encoding.PEM, + serialization.PublicFormat.SubjectPublicKeyInfo + )) +print("Hub keypair generated.") +``` + +### Hub API (`hub.py`) + +```python +from fastapi import FastAPI, HTTPException, Depends, Header +from pydantic import BaseModel +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives import serialization, hashes +from cryptography.hazmat.primitives.kdf.argon2 import Argon2id +import jwt, uuid, os, time, base64 + +app = FastAPI(title="MeshBay Hub POC") + +# Load hub keypair +with open("hub_private.pem", "rb") as f: + HUB_SK_PEM = f.read() +with open("hub_public.pem", "rb") as f: + HUB_PK_PEM = f.read() + +HUB_ID = "meshbay.org" +ACCESS_TOKEN_TTL = 3600 # 1 hour +REFRESH_TOKEN_TTL = 86400 * 30 # 30 days + +# In-memory stores (POC only — not persistent) +users = {} # username → {user_id, pw_hash, pw_salt, pk_ed25519, pk_x25519} +nodes = {} # node_id → {user_id, pk_node, endpoint_hint, registered_at} +refresh_tokens = {} # token → user_id + +# --- Models --- + +class UserRegister(BaseModel): + username: str + password: str + pk_user_ed25519: str # base64 + pk_user_x25519: str # base64 + +class UserLogin(BaseModel): + username: str + password: str + +class NodeAnnounce(BaseModel): + pk_node: str # base64 Ed25519 public key + endpoint_hint: str | None = None # "ip:port" or null + +# --- Helpers --- + +def hash_password(password: str) -> tuple[bytes, bytes]: + salt = os.urandom(16) + kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) + return kdf.derive(password.encode()), salt + +def verify_password(password: str, pw_hash: bytes, salt: bytes) -> bool: + kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) + try: + kdf.verify(password.encode(), pw_hash) + return True + except Exception: + return False + +def issue_access_token(user: dict) -> str: + payload = { + "iss": HUB_ID, + "sub": user["user_id"], + "pk_user": user["pk_ed25519"], + "hub_id": HUB_ID, + "iat": int(time.time()), + "exp": int(time.time()) + ACCESS_TOKEN_TTL, + } + return jwt.encode(payload, HUB_SK_PEM, algorithm="EdDSA") + +def get_current_user(authorization: str = Header(...)) -> dict: + try: + scheme, token = authorization.split() + if scheme.lower() != "bearer": + raise ValueError + payload = jwt.decode(token, HUB_PK_PEM, algorithms=["EdDSA"]) + user_id = payload["sub"] + user = next((u for u in users.values() if u["user_id"] == user_id), None) + if not user: + raise HTTPException(status_code=401, detail="User not found") + return user + except Exception: + raise HTTPException(status_code=401, detail="Invalid token") + +# --- Endpoints --- + +@app.get("/v1/hub/info") +def hub_info(): + return { + "hub_id": HUB_ID, + "pk_hub_ed25519": base64.b64encode( + Ed25519PrivateKey.from_private_bytes( + # shortcut for POC — load pk directly + open("hub_public.pem","rb").read() + ).public_bytes(...) # see note below + ).decode(), + "mnp_version": "0.1", + "mhp_version": "0.1", + } + # Note: return pk_hub_pem directly for POC, nodes store it on first contact + +@app.get("/v1/hub/pubkey") +def hub_pubkey(): + """Return hub Ed25519 public key PEM — cached by nodes on first contact.""" + return {"pk_hub_pem": HUB_PK_PEM.decode()} + +@app.post("/v1/users/register", status_code=201) +def register(body: UserRegister): + if body.username in users: + raise HTTPException(status_code=409, detail="Username taken") + pw_hash, pw_salt = hash_password(body.password) + user_id = str(uuid.uuid4()) + users[body.username] = { + "user_id": user_id, + "username": body.username, + "pw_hash": pw_hash, + "pw_salt": pw_salt, + "pk_ed25519": body.pk_user_ed25519, + "pk_x25519": body.pk_user_x25519, + } + return {"user_id": user_id} + +@app.post("/v1/users/login") +def login(body: UserLogin): + user = users.get(body.username) + if not user or not verify_password(body.password, user["pw_hash"], user["pw_salt"]): + raise HTTPException(status_code=401, detail="Invalid credentials") + access_token = issue_access_token(user) + refresh_token = base64.urlsafe_b64encode(os.urandom(32)).decode() + refresh_tokens[refresh_token] = user["user_id"] + return { + "access_token": access_token, + "refresh_token": refresh_token, + "token_type": "bearer", + "expires_in": ACCESS_TOKEN_TTL, + } + +@app.post("/v1/users/token/refresh") +def refresh(body: dict): + rt = body.get("refresh_token", "") + user_id = refresh_tokens.get(rt) + if not user_id: + raise HTTPException(status_code=401, detail="Invalid refresh token") + user = next((u for u in users.values() if u["user_id"] == user_id), None) + if not user: + raise HTTPException(status_code=401, detail="User not found") + return {"access_token": issue_access_token(user), "token_type": "bearer"} + +@app.post("/v1/nodes/announce", status_code=201) +def announce_node(body: NodeAnnounce, user: dict = Depends(get_current_user)): + node_id = str(uuid.uuid4()) + nodes[node_id] = { + "node_id": node_id, + "user_id": user["user_id"], + "pk_node": body.pk_node, + "endpoint_hint": body.endpoint_hint, + "announced_at": int(time.time()), + } + return {"node_id": node_id} + +@app.get("/v1/nodes/{node_id}") +def get_node(node_id: str, user: dict = Depends(get_current_user)): + node = nodes.get(node_id) + if not node: + raise HTTPException(status_code=404, detail="Node not found") + return { + "node_id": node["node_id"], + "pk_node": node["pk_node"], + "endpoint_hint": node["endpoint_hint"], + } +``` + +**Success criteria:** +- Hub starts, all 6 endpoints respond correctly +- `GET /v1/hub/pubkey` returns the PEM +- `POST /v1/users/register` + `POST /v1/users/login` returns a valid JWT +- JWT verified by `jwt.decode()` with hub public key — passes + +--- + +## Spike 3 — Node Registration (Fedora laptop) + +**Goal:** node generates its keypair, registers a user on the hub, gets a JWT, and verifies it locally without contacting the hub again. + +**File:** `node.py` + +```python +import httpx, asyncio, jwt, base64, os +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey +from cryptography.hazmat.primitives import serialization + +HUB_URL = "http://meshbay.org" # HTTP for POC, HTTPS later + +async def main(): + async with httpx.AsyncClient() as client: + + # 1. Fetch hub public key (first contact — cache this) + r = await client.get(f"{HUB_URL}/v1/hub/pubkey") + hub_pk_pem = r.json()["pk_hub_pem"].encode() + print(f"[node] Hub PK fetched ({len(hub_pk_pem)} bytes)") + + # 2. Generate node identity keypairs + sk_ed = Ed25519PrivateKey.generate() + pk_ed = sk_ed.public_key() + sk_x = X25519PrivateKey.generate() + pk_x = sk_x.public_key() + + pk_ed_b64 = base64.b64encode( + pk_ed.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) + ).decode() + pk_x_b64 = base64.b64encode( + pk_x.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) + ).decode() + + # 3. Register user (skip if already registered) + r = await client.post(f"{HUB_URL}/v1/users/register", json={ + "username": "testnode", + "password": "testpass123", + "pk_user_ed25519": pk_ed_b64, + "pk_user_x25519": pk_x_b64, + }) + print(f"[node] Register: {r.status_code} {r.text}") + + # 4. Login, get access token + r = await client.post(f"{HUB_URL}/v1/users/login", json={ + "username": "testnode", + "password": "testpass123", + }) + data = r.json() + access_token = data["access_token"] + print(f"[node] Login OK, token: {access_token[:40]}...") + + # 5. Verify JWT locally — NO hub roundtrip + decoded = jwt.decode(access_token, hub_pk_pem, algorithms=["EdDSA"]) + print(f"[node] JWT verified locally: sub={decoded['sub']}, exp={decoded['exp']}") + + # 6. Announce node to hub + r = await client.post( + f"{HUB_URL}/v1/nodes/announce", + json={"pk_node": pk_ed_b64, "endpoint_hint": None}, + headers={"Authorization": f"Bearer {access_token}"} + ) + node_id = r.json()["node_id"] + print(f"[node] Node announced: {node_id}") + +asyncio.run(main()) +``` + +**Success criteria:** +- Node registers, logs in, receives JWT +- JWT decoded offline using only the hub's public key — no hub call +- Node announced; `GET /v1/nodes/{node_id}` from hub returns correct PK + +--- + +## Spike 4 — NAT Traversal (both machines) + +**Goal:** discover the local node's external IP:port via STUN and UPnP; test reachability from meshbay.org. + +**File:** `spike4_nat.py` (Fedora laptop) + +### Part A — UPnP (try first, most reliable on SFR box) + +```python +import miniupnpc +import socket + +def try_upnp(internal_port=19000): + u = miniupnpc.UPnP() + u.discoverdelay = 200 + ndevices = u.discover() + if ndevices == 0: + print("UPnP: no IGD found") + return None + + u.selectigd() + external_ip = u.externalipaddress() + local_ip = socket.gethostbyname(socket.gethostname()) + + result = u.addportmapping( + internal_port, 'TCP', local_ip, internal_port, + 'MeshBay POC', '' + ) + if result: + print(f"UPnP: mapped {external_ip}:{internal_port} → {local_ip}:{internal_port}") + return f"{external_ip}:{internal_port}" + else: + print("UPnP: mapping failed") + return None +``` + +### Part B — STUN discovery + +```python +import asyncio +import aioice + +async def stun_discover(local_port=19001): + # Use Cloudflare STUN server + stun_servers = [("stun.cloudflare.com", 3478), ("stun.l.google.com", 19302)] + + connection = aioice.Connection(ice_controlling=True, stun_server=stun_servers[0]) + await connection.gather_candidates() + + for candidate in connection.local_candidates: + if candidate.type == "srflx": # server-reflexive = external address + print(f"STUN srflx: {candidate.host}:{candidate.port}") + return f"{candidate.host}:{candidate.port}" + + print("STUN: no srflx candidate found (may be symmetric NAT)") + return None +``` + +### Part C — Reachability test from meshbay.org + +Once the node has an external address (from UPnP or STUN), it announces it to the hub (`endpoint_hint`). Then from meshbay.org: + +```bash +# On meshbay.org — manually test TCP reachability +nc -zv <external_ip> <external_port> +# or +python3 -c "import socket; s=socket.create_connection(('<external_ip>', <port>), timeout=5); print('REACHABLE'); s.close()" +``` + +And on the Fedora node, a simple listener: +```python +# On Fedora, open a listener on the discovered port +import socket +s = socket.socket() +s.bind(('', 19000)) +s.listen(1) +print("Listening on 19000...") +conn, addr = s.accept() +print(f"Connection from {addr}") +conn.sendall(b"HELLO FROM NODE\n") +conn.close() +``` + +**Expected outcomes on SFR residential:** + +| Method | Expected result | Confidence | +|---|---|---| +| UPnP | Works — SFR La Box supports UPnP IGD | High | +| STUN srflx | Discovered — SFR is cone NAT for residential | High | +| Direct TCP from meshbay.org | Works if UPnP succeeded | High | +| Hole punching only | Depends on NAT type discovered | Medium | + +**Success criteria:** at least one method allows meshbay.org to reach the Fedora node's port directly. + +--- + +## Spike 5 — Encrypted File Transfer (both machines) + +**Goal:** node serves an encrypted file chunk via direct P2P TCP connection; client decrypts and verifies. + +**Prerequisite:** Spike 4 succeeded — external IP:port is known and reachable. + +**File:** `spike5_server.py` (Fedora), `spike5_client.py` (meshbay.org) + +### Node side — serve one encrypted chunk + +```python +# spike5_server.py — Fedora laptop +import asyncio, os, base64 +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey +from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 +from cryptography.hazmat.primitives.kdf.hkdf import HKDF +from cryptography.hazmat.primitives import hashes, serialization +import blake3, struct, json + +# Keypair (reuse from Spike 3 or generate here) +sk_node = Ed25519PrivateKey.generate() +pk_node_bytes = sk_node.public_key().public_bytes( + serialization.Encoding.Raw, serialization.PublicFormat.Raw +) + +# Generate GEK (in a real system, loaded from keystore) +gek_raw = ChaCha20Poly1305.generate_key() +cipher = ChaCha20Poly1305(gek_raw) + +CHUNK_SIZE = 1024 * 1024 # 1 MB + +def make_chunk(file_path: str, chunk_index: int) -> bytes: + """Read, compress (skipped for POC), encrypt, sign a chunk.""" + with open(file_path, "rb") as f: + f.seek(chunk_index * CHUNK_SIZE) + data = f.read(CHUNK_SIZE) + + file_hash = blake3.blake3(open(file_path, "rb").read()).digest() + + # Per-chunk key derivation + chunk_key = HKDF( + algorithm=hashes.SHA256(), length=32, salt=None, + info=b"file:" + file_hash + b":chunk:" + chunk_index.to_bytes(4, "big") + ).derive(gek_raw) + chunk_cipher = ChaCha20Poly1305(chunk_key) + + nonce = os.urandom(12) + ct = chunk_cipher.encrypt(nonce, data, None) + chunk_hash = blake3.blake3(ct).digest() + + # Sign: chunk_index + nonce + ciphertext_hash + sig_payload = chunk_index.to_bytes(4, "big") + nonce + chunk_hash + sig = sk_node.sign(sig_payload) + + return json.dumps({ + "chunk_index": chunk_index, + "nonce": base64.b64encode(nonce).decode(), + "ciphertext": base64.b64encode(ct).decode(), + "chunk_hash": base64.b64encode(chunk_hash).decode(), + "signature": base64.b64encode(sig).decode(), + "pk_node": base64.b64encode(pk_node_bytes).decode(), + "gek_hint": base64.b64encode(gek_raw).decode(), # POC: send GEK in band — never in production! + }).encode() + +async def handle_client(reader, writer): + request = await reader.read(1024) + req = json.loads(request) + chunk_index = req.get("chunk_index", 0) + file_path = req.get("file", "testfile.bin") + + print(f"[node] Client requests chunk {chunk_index} of {file_path}") + chunk_data = make_chunk(file_path, chunk_index) + + writer.write(len(chunk_data).to_bytes(4, "big") + chunk_data) + await writer.drain() + writer.close() + print(f"[node] Chunk {chunk_index} sent ({len(chunk_data)} bytes)") + +async def main(): + # Create a 5MB test file + if not os.path.exists("testfile.bin"): + with open("testfile.bin", "wb") as f: + f.write(os.urandom(5 * 1024 * 1024)) + print("[node] Test file created (5 MB)") + + server = await asyncio.start_server(handle_client, "0.0.0.0", 19000) + print("[node] Serving on port 19000 — waiting for client...") + async with server: + await server.serve_forever() + +asyncio.run(main()) +``` + +### Client side — request, verify, decrypt + +```python +# spike5_client.py — meshbay.org +import asyncio, base64, json +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey +from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 +from cryptography.hazmat.primitives.kdf.hkdf import HKDF +from cryptography.hazmat.primitives import hashes, serialization +import blake3 + +NODE_HOST = "<external_ip>" # from Spike 4 +NODE_PORT = 19000 + +async def main(): + reader, writer = await asyncio.open_connection(NODE_HOST, NODE_PORT) + + # Request chunk 0 + request = json.dumps({"file": "testfile.bin", "chunk_index": 0}).encode() + writer.write(request) + await writer.drain() + + # Receive + length_bytes = await reader.readexactly(4) + length = int.from_bytes(length_bytes, "big") + data = await reader.readexactly(length) + writer.close() + + chunk = json.loads(data) + print(f"[client] Received chunk {chunk['chunk_index']}") + + # 1. Verify signature + pk_node_bytes = base64.b64decode(chunk["pk_node"]) + pk_node = Ed25519PublicKey.from_public_bytes(pk_node_bytes) + ct = base64.b64decode(chunk["ciphertext"]) + nonce = base64.b64decode(chunk["nonce"]) + chunk_hash = base64.b64decode(chunk["chunk_hash"]) + sig = base64.b64decode(chunk["signature"]) + + sig_payload = (0).to_bytes(4, "big") + nonce + chunk_hash + pk_node.verify(sig, sig_payload) # raises on failure + print("[client] Signature OK") + + # 2. Verify ciphertext hash + assert blake3.blake3(ct).digest() == chunk_hash + print("[client] Ciphertext hash OK") + + # 3. Derive chunk key and decrypt (GEK from POC hint — never in production) + gek_raw = base64.b64decode(chunk["gek_hint"]) + # (in production, client has GEK from hub's GEK bundle) + chunk_key = HKDF( + algorithm=hashes.SHA256(), length=32, salt=None, + info=b"file:" + bytes(32) + b":chunk:" + (0).to_bytes(4, "big") + # Note: in production, file_hash is sent separately or in index + ).derive(gek_raw) + plaintext = ChaCha20Poly1305(chunk_key).decrypt(nonce, ct, None) + print(f"[client] Decrypted {len(plaintext)} bytes") + print("[client] Encrypted P2P transfer: SUCCESS") + +asyncio.run(main()) +``` + +**Note on GEK in POC:** the GEK is included in the response as `gek_hint` for POC convenience only. In production, the client gets the GEK from the hub's encrypted GEK bundle (delivered at login, decrypted client-side with the user's X25519 private key). + +**Success criteria:** +- Client receives chunk from node via direct TCP connection +- Signature verification passes +- Ciphertext hash matches +- Decryption produces the original bytes +- End-to-end: `original_bytes == decrypted_bytes` ✓ + +--- + +## What POC Validates (and Doesn't) + +### Validated by these spikes + +| Concept | Spike | Validation | +|---|---|---| +| Python crypto stack is sufficient | 1 | All primitives work, performance acceptable | +| Hub/Node JWT handshake | 2, 3 | JWT issued by hub, verified offline by node | +| Hub-Node REST protocol (minimal MNP/HTTP) | 2, 3 | API contract works end-to-end | +| SFR NAT traversal via UPnP | 4 | P2P reachability confirmed | +| STUN external address discovery | 4 | Confirmed/fallback documented | +| On-the-fly per-chunk encryption | 5 | GEK + HKDF chunk derivation + ChaCha20 | +| Chunk signature and verification | 5 | Ed25519 sign/verify before decryption | +| Real P2P file transfer | 5 | No hub in data path | + +### NOT in scope + +- Database (all in-memory) +- HTTPS / TLS (HTTP for POC) +- QUIC transport (plain TCP) +- GEK bundle distribution via hub (GEK sent in-band for POC) +- Group management +- Chat / Double Ratchet +- Mesh Group Index +- MHP federation +- Android client +- Module system +- Persistence between restarts + +--- + +## Spike Order Dependency Graph + +``` +Spike 1 (crypto) + └──→ Spike 2 (hub skeleton) + └──→ Spike 3 (node registration) + └──→ Spike 4 (NAT traversal) + └──→ Spike 5 (encrypted transfer) +``` + +Spike 1 is a prerequisite for all others. Spikes 2 and 3 can overlap if two people work in parallel. Spike 4 can begin independently once Spike 3 is running. + + +--- + +<!-- ============================================================ --> +<!-- ARCHIVED: Development Phases (1-12) (was docs/devel-phases.md) --> +<!-- ============================================================ --> + +# MeshBay — Development Phases + +> Reference: architecture spec in draft v3 (section 3 of this archive) +> POC results: `poc/spike-results.md` + +--- + +## Phase 1 — Foundations ✅ DONE + +**Goal:** validate all blocking concepts before writing production code. + +### Deliverables + +| Item | Status | Notes | +|---|---|---| +| POC Spike 1 — Crypto primitives | ✅ | All 22 tests pass. ChaCha20 1MB in 1.1ms. | +| POC Spike 2 — Hub skeleton | ✅ | 12/12 endpoints. JWT EdDSA offline verify in 884µs. | +| POC Spike 3 — Node registration | ✅ | Full handshake. jti bug found and fixed. | +| POC Spike 4 — NAT traversal | ✅ | Cone NAT on SFR. UDP P2P works. UPnP disabled (SFR). | +| POC Spike 5 — Encrypted transfer | ✅ | 1MB P2P. 3.2ms encrypt, 3.9ms decrypt. 4.3MB/s. | +| POC Spike 6 — GEK distribution | ✅ | X25519+HKDF wrap/unwrap. 0.48ms/0.59ms. Hub opaque. | +| Security cleanup meshbay.org | ✅ | UFW: 22/80/443 only. No services exposed. | +| Git monorepo | ✅ | 3 packages: meshbay-common, meshbay-hub, meshbay-node. | +| Draft v3 | ✅ | POC findings integrated. All corrections applied. | +| CLAUDE.md conventions | ✅ | Python 3.12+, uv, ruff, SemVer, commit format. | + +### Key findings from POC + +- jti mandatory in all JWTs (Ed25519 is deterministic — same payload = same token) +- Argon2id at 64MB/3iter = 78ms — increase to 256MB for production (~500ms target) +- NAT order: STUN/hole-punching is priority 2, not UPnP (UPnP disabled on tested SFR box) +- TCP+TLS for v1 transport; QUIC in v2 +- GEK wrapping: ephemeral X25519 + HKDF(salt=pk_eph) + ChaCha20-Poly1305(aad=pk_recipient) + +--- + +## Phase 2 — Node v1 ✅ DONE + +**Goal:** working Mesh Node: indexes a directory, registers with hub, +serves encrypted chunks over TCP+TLS, local web UI on localhost:18000. + +**Transport:** TCP+TLS 1.3 (QUIC in v2). Self-signed cert per node. +Node identity verified via Ed25519 PK from hub, not TLS cert chain (client uses CERT_NONE). + +### Milestones + +| # | Component | File(s) | Tests | +|---|---|---|---| +| 2.1 | Keystore | `meshbay_node/keystore.py` | 10/10 | +| 2.2 | Hub client | `meshbay_node/hub_client.py` | 6/6 | +| 2.3 | Directory indexer | `meshbay_node/indexer/indexer.py` | 5/5 | +| 2.4 | Mesh Group Index | `meshbay_node/indexer/group_index.py` | 5/5 | +| 2.5 | TCP+TLS chunk server | `meshbay_node/transport/server.py` | 3/3 | +| 2.6 | TCP+TLS chunk client | `meshbay_node/transport/client.py` | included above | +| 2.7 | TLS cert helper | `meshbay_node/transport/tls_cert.py` | — | +| 2.8 | Config (TOML + env) | `meshbay_node/config.py` | — | +| 2.9 | Local web UI | `meshbay_node/ui/app.py` | — | +| 2.10 | Daemon + CLI | `meshbay_node/daemon.py` | — | + +**Total: 29/29 tests passing** + +### Python dependencies (dev venv — `/home/cbesson/meshbay/.venv`) + +Installed packages (freeze) as of Phase 2 completion: + +``` +aioice==0.10.2 # ICE/STUN for NAT traversal +blake3==1.0.9 # fast content hashing +cryptography==50.0.0 # Ed25519, X25519, ChaCha20, Argon2id, AES-GCM +fastapi==0.141.1 # local web UI + hub POC +httpx==0.28.1 # hub client HTTP +meshbay-common==0.1.0 # editable install +meshbay-node==0.1.0 # editable install +msgpack==1.2.1 # wire serialisation +PyJWT==2.13.0 # JWT EdDSA +pytest==9.1.1 # test runner +pytest-asyncio==1.4.0 # async test support +uvicorn==0.52.1 # ASGI server (local UI) +watchdog==6.0.0 # filesystem watcher +zstandard==0.25.0 # zstd compression +``` + +Also installed transitively: pydantic 2.13.4, starlette 1.6.0, anyio 4.14.2, uvloop 0.22.1. + +### meshbay_common/crypto.py — Argon2id NOTE + +Current params: `iterations=3, memory_cost=65536` (64MB) → ~78ms on dev laptop. +**Must increase to `memory_cost=262144` (256MB) before production keystore use.** +Run `meshbay-node calibrate-argon2` on target hardware to tune. + +### Out of scope for Phase 2 + +Multiple groups, chat, QUIC, module sandbox, mobile pairing, HLS streaming. + +--- + +## Phase 3 — Hub v1 production ✅ DONE + +**Goal:** replace POC in-memory hub with a production-ready service on meshbay.org. +PostgreSQL persistence, HTTPS via Caddy, all endpoints hardened, legal IP logging, deploy. + +### Environment + +- **Server:** meshbay.org — Ubuntu 26.04 LTS, Python 3.14.4, OVH VPS +- **Database:** PostgreSQL 16 (to install) +- **Proxy:** Caddy (to install — handles Let's Encrypt automatically) +- **Service:** systemd `meshbay-hub.service` + +### Python dependencies to add (Phase 3) + +``` +sqlalchemy>=2.0 # async ORM (SQLAlchemy 2.x) +alembic>=1.13 # DB migrations +asyncpg>=0.30 # PostgreSQL async driver +aiosqlite>=0.20 # SQLite async driver (tests only) +slowapi>=0.1 # rate limiting (FastAPI middleware) +``` + +### Milestones + +| # | Component | File(s) | Status | +|---|---|---|---| +| 3.1 | DB models | `meshbay_hub/db/models.py` | ✅ | +| 3.2 | DB engine + session | `meshbay_hub/db/engine.py` | ✅ | +| 3.3 | Alembic migrations | `meshbay_hub/db/migrations/` | ✅ initial_schema | +| 3.4 | Hub config | `meshbay_hub/config.py` | ✅ | +| 3.5 | Auth (JWT + Argon2id) | `meshbay_hub/auth.py` | ✅ | +| 3.6 | API deps | `meshbay_hub/api/deps.py` | ✅ | +| 3.7 | Hub info router | `meshbay_hub/api/hub.py` | ✅ | +| 3.8 | Users router | `meshbay_hub/api/users.py` | ✅ | +| 3.9 | Nodes router | `meshbay_hub/api/nodes.py` | ✅ | +| 3.10 | Groups router | `meshbay_hub/api/groups.py` | ✅ | +| 3.11 | Rate limiting | `meshbay_hub/api/middleware.py` | ✅ slowapi | +| 3.12 | App factory + lifespan | `meshbay_hub/app.py` | ✅ | +| 3.13 | Hub daemon CLI | `meshbay_hub/daemon.py` | ✅ | +| 3.14 | PostgreSQL 16 | meshbay.org | ✅ DB: meshbay_hub | +| 3.15 | Caddy + Let's Encrypt | meshbay.org Caddyfile | ✅ HTTPS auto-cert | +| 3.16 | Systemd service | `/etc/systemd/system/meshbay-hub.service` | ✅ | +| 3.17 | Deploy + smoke test | https://meshbay.org | ✅ 11/11 endpoints | + +**Total: 40 local tests (SQLite) + 11/11 smoke tests HTTPS production** + +### Phase 3 deployment details (meshbay.org) + +- PostgreSQL 16, user `meshbay`, DB `meshbay_hub` +- Caddy auto-handles Let's Encrypt for `meshbay.org` and `www.meshbay.org` +- `www.meshbay.org` → 301 redirect → `meshbay.org` +- TLS setup documented in `HTTPS.md` +- Hub listens on `127.0.0.1:8000`, Caddy proxies 80/443 +- Systemd service: `meshbay-hub.service` (restart-on-failure) +- Hub config: `~/.config/meshbay/hub.toml` +- Hub keypair: `~/.config/meshbay/hub_private.pem` (chmod 600) +- Source deployed at: `~/meshbay-hub/` (common_pkg + hub_pkg) + +### Additional Python dependencies added in Phase 3 + +``` +sqlalchemy==2.0.51 # async ORM +alembic==1.19.1 # DB migrations +asyncpg==0.31.0 # PostgreSQL async driver +aiosqlite # SQLite async (tests only) +slowapi==0.1.10 # rate limiting +hatchling==1.31.0 # build backend (needed for pip install) +``` + +### Notes + +- Initial DB schema created via `init_db()` (`create_all`) — Alembic tracks future changes +- IP logging records: account_create, login, login_fail, group_create, node_announce +- Rate limiting active on /v1/users/register and /v1/users/login (slowapi) +- Revocation, CSAM hash matching, moderation deferred to Phase 5 + +### Phase 3 scope + +**In scope:** +- All POC endpoints from Spike 2+6, production-ready +- PostgreSQL via SQLAlchemy async + Alembic migrations +- IP logging (creation, login, group events) — 1-year retention, legal compliance +- Access token (JWT, 1h) + refresh token (30d, stored hashed) +- Rate limiting on auth endpoints +- Hub config file `/etc/meshbay/hub.toml` or env vars +- HTTPS via Caddy + auto Let's Encrypt on meshbay.org +- Systemd service with restart-on-failure + +**Deferred to later:** +- Revocation push (WebSocket signaling to nodes) +- CSAM hash matching (NCMEC/IWF integration) +- Moderation flow (blocklist + takedown) +- MHP federation +- RPM/DEB packaging + +### Testing strategy + +- Unit tests with SQLite in-memory (`aiosqlite`) — no PostgreSQL needed locally +- API tests via `httpx.AsyncClient` + `ASGITransport` — no network +- All tests run in the existing `.venv` after adding Phase 3 deps + +--- + +## Phase 4 — Integration & Web Client ✅ DONE + +**Goal:** end-to-end working product in a browser: login via hub, discover groups, +browse files on a node, download and stream public content. + +### Architecture decision + +Browsers cannot make raw TCP connections — the node must speak HTTP. +- Node adds an **HTTP file API** (port 19001) serving public content via standard `fetch()` +- Private content (GEK decrypt in browser) deferred to Phase 5 (requires WebCrypto + ChaCha20 WASM) +- Hub serves the web application at `meshbay.org/app/` + +### Milestones + +| # | Component | File(s) | Status | +|---|---|---|---| +| 4.1 | Node HTTP file API | `meshbay_node/transport/http_server.py` | ✅ 7/7 tests | +| 4.2 | Hub web app (HTML/JS) | `meshbay_hub/api/webapp.py` + `static/app.js` | ✅ live on meshbay.org | +| 4.3 | Group listing endpoint | `meshbay_hub/api/groups.py` GET /v1/groups | ✅ | +| 4.4 | End-to-end integration test | local node ↔ hub ↔ browser | ✅ smoke test | +| 4.5 | HLS streaming (public) | node `/hls/{id}/playlist.m3u8` + `.ts` via ffmpeg | ✅ included in 4.1 | + +**Total: 47/47 tests. https://meshbay.org live with web client.** + +### Phase 4 deployment + +- `https://meshbay.org/` — web app (HTML/JS SPA) +- `https://meshbay.org/app.js` — JS client +- `https://meshbay.org/v1/groups` — public group listing (no auth) +- Node HTTP API (port 19001): `/index`, `/file/{id}`, `/hls/{id}/*.m3u8`, `/hls/{id}/*.ts` +- HLS streaming uses ffmpeg for on-the-fly segmentation +- Public content: no auth for index, auth required for chunks +- Private content (GEK decrypt in browser): deferred to Phase 5 + +### Dependencies added in Phase 4 + +``` +# Node (runtime) +ffmpeg (system package) — HLS segmentation via subprocess +``` + +### Phase 4 scope + +**In scope (public content only):** +- Node HTTP API: serve public Mesh Group Index (JSON) + file chunks (binary) +- Hub web app: login, group discovery, file browser, download link +- HLS basic streaming: node segments video on-the-fly, browser plays natively +- JWT auth passed as query param or header to node HTTP API + +**Deferred to Phase 5:** +- Private group content in browser (requires ChaCha20-Poly1305 via WASM) +- Chat UI +- Multi-group node +- Android client + +--- + +## Phase 5 — QUIC, Federation, Moderation ✅ DONE (Mobile deferred) + +**Goal:** full decentralization, mobile support, community infrastructure. + +| # | Component | Notes | +|---|---|---| +| # | Component | File(s) | Status | +|---|---|---|---| +| 5.1 | QUIC transport (MNP v2) | `transport/quic_server.py` + `quic_client.py` | ✅ 3/3 tests | +| 5.2 | MHP federation | `meshbay_hub/api/federation.py` | ✅ GET/POST /mhp/* | +| 5.3 | Mesh Relay | `meshbay_hub/api/relay.py` | ✅ register+list+approve | +| 5.4 | Android client | — | ⏳ deferred (different tech) | +| 5.5 | Content replication | — | ⏳ deferred | +| 5.6 | iOS client | — | ⏳ after Android | +| 5.7 | Revocation push | `api/revocation.py` + `node/revocation.py` | ✅ 3/3 tests | +| 5.8 | CSAM hash matching | `meshbay_hub/csam.py` | ✅ CSAMChecker + admin API | +| 5.9 | Moderation | `api/moderation.py` | ✅ 6/6 tests | +| 5.10 | RPM/DEB packaging | `packaging/` | ✅ spec + control + systemd | + +**Total: 59/59 tests.** + +### Phase 5 bugs fixed + +- **QUIC**: `asyncio.Event` race condition in client recv loop (quic_event_received + overwrote `_stream_events[0]` created by `_recv`). Fixed with `asyncio.Queue` + (no shared mutable state between coroutines). +- **QUIC**: `verify_peer` parameter renamed to `verify_mode` in aioquic 1.3.0. +- **QUIC**: `connect()` returns protocol directly (not `(transport, proto)` tuple). + +### Phase 5 dependencies added + +``` +aioquic==1.3.0 # QUIC transport (Cloudflare-maintained) +websockets # hub→node revocation push +``` + +### Deferred to future + +- Android/iOS client (Kotlin/Flutter — different tech stack, dedicated effort) +- Content replication (node-to-node) → Phase 6 +- MHP federation persistence (currently in-memory) → Phase 6 + +--- + +## Phase 6 — Chat, Multi-group, Federation persistence, Replication ✅ DONE + +**Goal:** complete the product with group chat (Double Ratchet), multi-group node support, +persistent MHP federation, content replication, and browser private group decryption. + +### Milestones + +| # | Component | File(s) | Status | +|---|---|---|---| +| 6.1 | Double Ratchet chat | `meshbay_common/ratchet.py` | ✅ 11/11 tests | +| 6.2 | Multi-group node | `meshbay_node/config.py` [[groups]] | ✅ | +| 6.3 | MHP federation persistence | `FederatedGroup` + `SwarmSource` DB tables | ✅ | +| 6.4 | Content replication | `node/replication.py` + hub `/v1/swarm/*` | ✅ | +| 6.5 | Browser private group | `webcrypto.py` + `static/crypto.js` | ✅ 4/4 tests | +| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | +| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | +| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` + `QE/demo-v2/` (non versionné) | ✅ testés | +| 6.9 | QUICKSTART réécrit | `QUICKSTART.md` | ✅ | +| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | +| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | +| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` (non versionné) | ✅ testés | +| 6.9 | QUICKSTART réécrit | `QUICKSTART.md` | ✅ | + +**Total: 81/81 tests.** + +--- + +## Phase 7 — Node v2: production, streaming, chat ✅ DONE + +**Goal:** multi-group node, Sender Keys chat, QUIC 0-RTT, jti denylist push, HLS streaming. + +Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests. + +See `devel-phases-next.md` for details. + +--- + +## Phase 8 — Hub v2: admin, federation, security ✅ DONE + +**Goal:** admin roles, email encryption, refresh token rotation, rate limiting, healthcheck. + +Commit: 46918ec — 20 files, +508/−90 lines, 117 tests. +Deployed to meshbay.org. All security review items S1/S2/S5 resolved. + +See `devel-phases-next.md` for details. + +--- + +## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE + +**Goal:** browser connects P2P to a node behind residential NAT via WebRTC DataChannel. +Full SPA: login, groups, file browser, download, video playback, chat, i18n, dark/light. + +### Milestones + +| # | Component | Status | +|---|---|---| +| 9.1–9.5 | WebRTC DataChannel spike + E2E NAT validation | ✅ | +| 9.6–9.12 | Preact SPA (login, groups, files, video, chat, i18n, settings) | ✅ | +| 9.13 | Tests: 132 passing | ✅ | +| 9.14–9.16 | Performance: pipelining, binary wire format, I/O reduction | ✅ | +| 9.17 | Large file download: File System Access API (stream to disk) | ✅ | + +**Total: 132/132 tests. Deployed to meshbay.org + Orange node (2026-08-11).** + +### Key technical decisions + +- **Transport:** WebRTC DataChannel (aiortc on node) — browsers can't use QUIC for NAT traversal +- **Wire format:** length-prefixed msgpack, binary chunk fields (no base64) +- **UI:** Preact + htm ESM (vendored, no build step, no CDN, no npm) +- **Crypto:** WebCrypto SubtleCrypto AES-256-GCM for E2E chunk decryption in browser +- **Large files:** File System Access API (`showSaveFilePicker`) — stream to disk, ~8 MB RAM +- **Indexer:** 2s debounce + path-based dedup for file copy events + +### NAT traversal validated + +Two ISPs (SFR + Orange residential NAT), Chrome + Firefox, IPv4 STUN + IPv6 direct. +No TURN relay needed. See `devel-phases-next.md` for detailed test matrix. + +### QE deployment state (2026-08-11) + +- **Hub (meshbay.org):** running as `meshbay-hub.service`, DB has 3 users + (admin, cbesson, grenet), 1 group (`d3bbd90b`), `admin_usernames = ["admin"]` +- **Node (Orange host via `ssh cbesson@localhost -p 2222`):** running as + `nohup .venv/bin/python3 QE/demo-v3/run_node_simple.py`, user grenet, + connected via WS to hub, WebRTC + QUIC dual transport +- **Credentials:** `QE/demo-v3/creds.json` (not versioned) +- **Shared dir on node:** `~/meshbay/QE/demo-v3/shared/` +- All 3 users password: see creds.json + +### Dependencies added + +- `aiortc>=1.9` in meshbay-node (WebRTC DataChannel) +- `preact` + `htm` vendored as `static/vendor/htm-preact.js` (ESM, ~3 KB gzipped) + +--- + +## Conventions + +- Commits: `feat(node):`, `fix(hub):`, `chore(common):`, `docs:`, `test(node):` +- Branch per feature/fix, merge to `main` (when remote configured) +- **Always close test UFW ports after any spike on meshbay.org** +- **Never commit key material** (keystore.enc, hub_private.pem, *.key, node_state.json, unlock.key) +- meshbay.org is internet-facing: only run known-safe services, close ports after tests diff --git a/docs/poc-v1-fr.md b/docs/poc-v1-fr.md deleted file mode 100644 index 262a245..0000000 --- a/docs/poc-v1-fr.md +++ /dev/null @@ -1,432 +0,0 @@ -# MeshBay — POC v1 (FR) - -> Objectif : valider les concepts clés avant de s'engager dans une implémentation complète. -> Périmètre : échange Hub/Node en Python, stack crypto, traversée NAT, transfert chiffré de chunk de fichier. -> Tout en mémoire (pas de base de données), code minimal, TCP uniquement (pas de QUIC pour l'instant). - ---- - -## Environnement - -### Distant — meshbay.org (Hub) -- OVH VPS, Ubuntu 26.04 LTS, Python 3.14.4 -- IP fixe publique, ports 80 et 443 ouverts -- Serveur vierge : aucun serveur web installé -- Accès SSH : `ssh cbesson@meshbay.org` - -### Local — Fedora 44 (Node) -- Laptop derrière NAT résidentiel SFR (vraisemblablement Restricted Cone NAT — UPnP supporté) -- Python 3.13+ via paquets système -- Utilisateur : `cbesson` (sudoer sans mot de passe) - ---- - -## Dépendances Python - -```bash -# Partagé (hub et node) -cryptography>=43.0 # Ed25519, X25519, ChaCha20-Poly1305, Argon2id -PyJWT>=2.9 # JWT avec support EdDSA (Ed25519) -blake3>=1.0 # Hachage rapide du contenu - -# Hub uniquement (meshbay.org) -fastapi>=0.115 -uvicorn[standard]>=0.30 - -# Node uniquement (laptop Fedora) -httpx>=0.28 # Client HTTP async pour les appels node→hub -aioice>=0.9 # Requêtes STUN pour la découverte NAT -miniupnpc>=2.2 # Ouverture de port UPnP sur la box SFR -``` - -Installation sur chaque machine : -```bash -python3 -m venv .venv -source .venv/bin/activate -pip install <paquets ci-dessus> -``` - ---- - -## Configuration du hub sur meshbay.org - -Pour le POC, uvicorn tourne directement sur le port 80 via une redirection iptables (pas de Caddy/nginx pour l'instant — HTTPS ajouté avant la production). - -```bash -# Sur meshbay.org -# Redirection port 80 → 8000 -sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000 - -# Lancer le hub (depuis le répertoire poc, venv activé) -uvicorn hub:app --host 127.0.0.1 --port 8000 --reload -``` - -> Note : HTTPS (via Caddy + Let's Encrypt) est obligatoire avant tout usage réel au-delà de ce POC. - ---- - -## Vue d'ensemble des spikes - -| # | Nom | Où | Ce que ça valide | Durée | -|---|---|---|---|---| -| 1 | Primitives crypto | Local | La stack Python crypto couvre tous les besoins | ~1h | -| 2 | Squelette hub | meshbay.org | API hub, émission JWT | ~2h | -| 3 | Enregistrement node | Fedora | Handshake Hub-Node, vérification JWT offline | ~1h | -| 4 | Traversée NAT | Les deux | UPnP box SFR + STUN, accessibilité P2P | ~2h | -| 5 | Transfert chiffré | Les deux | Chiffrement GEK à la volée, chunk P2P | ~2h | - ---- - -## Spike 1 — Primitives cryptographiques (local uniquement) - -**Objectif :** confirmer que `cryptography` (PyCA) couvre tous les besoins cryptographiques de MeshBay sans lacune ni surprise de performance. - -**Fichier :** `spike1_crypto.py` - -**Test 1 : Ed25519 — keypair hub, signature, vérification** -```python -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey - -sk_hub = Ed25519PrivateKey.generate() -pk_hub = sk_hub.public_key() -msg = b"test payload" -sig = sk_hub.sign(msg) -pk_hub.verify(sig, msg) # lève une exception si invalide -print("Ed25519 OK") -``` - -**Test 2 : X25519 — accord de clé pour l'enveloppement de la GEK** -```python -from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey - -sk_a = X25519PrivateKey.generate() -sk_b = X25519PrivateKey.generate() -shared_a = sk_a.exchange(sk_b.public_key()) -shared_b = sk_b.exchange(sk_a.public_key()) -assert shared_a == shared_b -print("X25519 OK") -``` - -**Test 3 : ChaCha20-Poly1305 sur un chunk de 1 Mo** -```python -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -import os, time - -gek = ChaCha20Poly1305.generate_key() -cipher = ChaCha20Poly1305(gek) -chunk = os.urandom(1024 * 1024) # 1 Mo - -t0 = time.perf_counter() -nonce = os.urandom(12) -ct = cipher.encrypt(nonce, chunk, None) -pt = cipher.decrypt(nonce, ct, None) -elapsed = time.perf_counter() - t0 - -assert pt == chunk -print(f"ChaCha20-Poly1305 1 Mo : {elapsed*1000:.1f} ms") -``` - -**Test 4 : Dérivation de clé de chunk par HKDF** -```python -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes -import blake3 - -chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + blake3.blake3(chunk).digest() + b":chunk:0" -).derive(gek) -print(f"Clé HKDF : {chunk_key.hex()[:16]}...") -``` - -**Test 5 : Argon2id — dérivation de clé keystore** -```python -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id - -salt = os.urandom(16) -t0 = time.perf_counter() -kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) -key = kdf.derive(b"motdepasse") -print(f"Argon2id : {(time.perf_counter()-t0)*1000:.0f} ms") -``` - -**Test 6 : PyJWT avec Ed25519 (EdDSA)** -```python -import jwt -from cryptography.hazmat.primitives import serialization - -sk_pem = sk_hub.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() -) -pk_pem = pk_hub.public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo -) -payload = {"sub": "user_abc", "pk_user": "base64...", "exp": 9999999999} -token = jwt.encode(payload, sk_pem, algorithm="EdDSA") -decoded = jwt.decode(token, pk_pem, algorithms=["EdDSA"]) -assert decoded["sub"] == "user_abc" -print("JWT EdDSA OK") -``` - -**Critères de succès :** tous les tests passent, ChaCha20 1 Mo < 20 ms, Argon2id ~1s. - ---- - -## Spike 2 — Squelette du hub (meshbay.org) - -**Objectif :** hub FastAPI minimal avec stockage en mémoire, 6 endpoints. - -**Fichier :** `hub.py` (sur meshbay.org) - -### Génération du keypair hub (une seule fois) - -```python -# gen_hub_keys.py — exécuter une seule fois sur meshbay.org -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives import serialization - -sk = Ed25519PrivateKey.generate() -with open("hub_private.pem", "wb") as f: - f.write(sk.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() - )) -with open("hub_public.pem", "wb") as f: - f.write(sk.public_key().public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo - )) -print("Keypair hub généré.") -``` - -### Endpoints du hub - -``` -GET /v1/hub/pubkey → PEM de la clé publique Ed25519 du hub -POST /v1/users/register → {username, password, pk_user_ed25519, pk_user_x25519} → {user_id} -POST /v1/users/login → {username, password} → {access_token, refresh_token} -POST /v1/users/token/refresh → {refresh_token} → {access_token} -POST /v1/nodes/announce → (auth) {pk_node, endpoint_hint} → {node_id} -GET /v1/nodes/{node_id} → (auth) {pk_node, endpoint_hint} -``` - -### Structure JWT (access token) - -```json -{ - "iss": "meshbay.org", - "sub": "<user_id>", - "pk_user": "<base64 Ed25519 publique>", - "hub_id": "meshbay.org", - "iat": 1720000000, - "exp": 1720003600 -} -``` - -Signé avec la clé Ed25519 privée du hub. Vérifiable par n'importe qui possédant la clé publique du hub — aucun appel hub requis. - -**Critères de succès :** -- Hub démarre, tous les endpoints répondent correctement -- `POST /v1/users/register` + `POST /v1/users/login` retourne un JWT valide -- `jwt.decode()` avec la clé publique du hub passe sans erreur - ---- - -## Spike 3 — Enregistrement du node (laptop Fedora) - -**Objectif :** le node génère son keypair, s'enregistre sur le hub, obtient un JWT, et le vérifie localement sans contacter le hub. - -**Fichier :** `node.py` - -**Séquence :** -1. Récupérer la clé publique du hub (`GET /v1/hub/pubkey`) — mettre en cache -2. Générer le keypair Ed25519 + X25519 du node -3. Enregistrer l'utilisateur sur le hub -4. Se connecter, recevoir l'access token (JWT) -5. **Vérifier le JWT localement** avec la clé publique du hub — aucun appel réseau -6. Annoncer le node au hub - -**Vérification JWT offline (point clé) :** -```python -# Aucun appel hub — juste la signature Ed25519 -decoded = jwt.decode(access_token, hub_pk_pem, algorithms=["EdDSA"]) -print(f"[node] JWT vérifié localement : sub={decoded['sub']}") -``` - -C'est la validation du concept fondamental : le hub est une autorité d'identité qui émet des credentials vérifiables hors ligne. Après le login, le hub n'est plus dans la boucle. - -**Critères de succès :** -- Node s'enregistre, se connecte, reçoit un JWT -- JWT décodé offline avec la seule clé publique du hub -- Node annoncé ; `GET /v1/nodes/{node_id}` depuis le hub retourne le bon PK - ---- - -## Spike 4 — Traversée NAT (les deux machines) - -**Objectif :** découvrir l'IP:port externe du node local via UPnP et STUN ; tester l'accessibilité depuis meshbay.org. - -**Fichier :** `spike4_nat.py` (laptop Fedora) - -### Partie A — UPnP (à tenter en premier, plus fiable sur box SFR) - -```python -import miniupnpc, socket - -def try_upnp(internal_port=19000): - u = miniupnpc.UPnP() - u.discoverdelay = 200 - if u.discover() == 0: - print("UPnP : aucune IGD trouvée") - return None - - u.selectigd() - external_ip = u.externalipaddress() - local_ip = socket.gethostbyname(socket.gethostname()) - - if u.addportmapping(internal_port, 'TCP', local_ip, internal_port, 'MeshBay POC', ''): - print(f"UPnP : {external_ip}:{internal_port} → {local_ip}:{internal_port}") - return f"{external_ip}:{internal_port}" - print("UPnP : échec du mapping") - return None -``` - -### Partie B — Découverte STUN - -```python -import asyncio, aioice - -async def stun_discover(): - connection = aioice.Connection( - ice_controlling=True, - stun_server=("stun.cloudflare.com", 3478) - ) - await connection.gather_candidates() - - for candidate in connection.local_candidates: - if candidate.type == "srflx": # server-reflexive = adresse externe - print(f"STUN srflx : {candidate.host}:{candidate.port}") - return f"{candidate.host}:{candidate.port}" - - print("STUN : aucun candidat srflx (NAT symétrique possible)") - return None -``` - -### Partie C — Test d'accessibilité depuis meshbay.org - -Le node annonce son `endpoint_hint` au hub. Depuis meshbay.org : - -```bash -# Test TCP depuis meshbay.org -python3 -c " -import socket -s = socket.create_connection(('<ip_externe>', <port>), timeout=5) -print('ACCESSIBLE') -s.close() -" -``` - -Sur le laptop Fedora, un listener simple sur le port découvert : -```python -import socket -s = socket.socket() -s.bind(('', 19000)) -s.listen(1) -print("En écoute sur 19000...") -conn, addr = s.accept() -print(f"Connexion depuis {addr}") -conn.sendall(b"BONJOUR DU NODE\n") -conn.close() -``` - -**Résultats attendus sur SFR résidentiel :** - -| Méthode | Résultat attendu | Niveau de confiance | -|---|---|---| -| UPnP | Fonctionne — La Box SFR supporte UPnP IGD | Élevé | -| STUN srflx | Découvert — SFR est un cone NAT pour le résidentiel | Élevé | -| TCP direct depuis meshbay.org | Fonctionne si UPnP a réussi | Élevé | -| Hole punching seul | Dépend du type NAT découvert | Moyen | - -**Critères de succès :** au moins une méthode permet à meshbay.org d'atteindre directement le port du laptop Fedora. - ---- - -## Spike 5 — Transfert chiffré de fichier (les deux machines) - -**Objectif :** le node sert un chunk de fichier chiffré via connexion TCP P2P directe ; le client déchiffre et vérifie. - -**Prérequis :** Spike 4 réussi — IP:port externe connu et accessible. - -### Côté node (laptop Fedora) - -Pipeline : lire le chunk → dériver la clé via HKDF(GEK, file_hash, chunk_index) → chiffrer ChaCha20-Poly1305 → signer Ed25519 → envoyer. - -**Points clés :** -- Clé par chunk dérivée de la GEK (pas la GEK directement) -- Chaque chunk signaturé avant envoi -- La GEK n'est jamais envoyée en clair en production (envoyée en clair uniquement pour ce POC — voir note ci-dessous) - -### Côté client (meshbay.org) - -Pipeline : recevoir → vérifier signature Ed25519 → vérifier hash blake3 du ciphertext → déchiffrer ChaCha20-Poly1305 → obtenir les octets en clair. - -### Note sur la GEK dans le POC - -Pour ce POC, la GEK est transmise dans la réponse comme `gek_hint` pour des raisons de commodité. **En production, le client obtient la GEK depuis le bundle GEK chiffré du hub** (déchiffré côté client avec sa clé X25519 privée). Le mécanisme de distribution de la GEK est délibérément hors périmètre de ce POC. - -**Critères de succès :** -- Le client reçoit le chunk depuis le node via TCP direct (sans hub dans le chemin) -- La vérification de signature passe ✓ -- Le hash du ciphertext correspond ✓ -- Le déchiffrement produit les octets originaux ✓ -- `octets_originaux == octets_déchiffrés` ✓ - ---- - -## Ce que le POC valide (et ne valide pas) - -### Validé par ces spikes - -| Concept | Spike | Validation | -|---|---|---| -| Stack Python crypto suffisante | 1 | Toutes les primitives fonctionnent, performance acceptable | -| Handshake Hub/Node via JWT | 2, 3 | JWT émis par le hub, vérifié offline par le node | -| Protocole REST Hub-Node minimal | 2, 3 | Contrat API validé bout en bout | -| Traversée NAT SFR via UPnP | 4 | Accessibilité P2P confirmée | -| Découverte adresse externe STUN | 4 | Confirmée / fallback documenté | -| Chiffrement par chunk à la volée | 5 | GEK + HKDF par chunk + ChaCha20 | -| Signature et vérification de chunk | 5 | Ed25519 sign/verify avant déchiffrement | -| Transfert de fichier P2P réel | 5 | Aucun hub dans le chemin des données | - -### Hors périmètre - -- Base de données (tout en mémoire) -- HTTPS / TLS (HTTP pour le POC) -- Transport QUIC (TCP simple) -- Distribution du bundle GEK via hub (GEK transmise en clair pour le POC) -- Gestion de groupes -- Chat / Double Ratchet -- Mesh Group Index -- Fédération MHP -- Client Android -- Système de modules -- Persistance entre les redémarrages - ---- - -## Graphe de dépendance des spikes - -``` -Spike 1 (crypto) - └──→ Spike 2 (squelette hub) - └──→ Spike 3 (enregistrement node) - └──→ Spike 4 (traversée NAT) - └──→ Spike 5 (transfert chiffré) -``` - -Le Spike 1 est un prérequis pour tous les autres. Les Spikes 2 et 3 peuvent être menés en parallèle si deux personnes travaillent. Le Spike 4 peut commencer indépendamment dès que le Spike 3 est fonctionnel. diff --git a/docs/poc-v1.md b/docs/poc-v1.md deleted file mode 100644 index 8f66159..0000000 --- a/docs/poc-v1.md +++ /dev/null @@ -1,767 +0,0 @@ -# MeshBay — POC v1 - -> Goal: validate key concepts before committing to a full implementation. -> Scope: Hub/Node exchange in Python, crypto stack, NAT traversal, encrypted file chunk transfer. -> Everything in-memory (no database), minimal code, TCP only (no QUIC yet). - ---- - -## Environment - -### Remote — meshbay.org (Hub) -- OVH VPS, Ubuntu 26.04 LTS, Python 3.14.4 -- Public fixed IP, ports 80 and 443 open -- Clean slate: no web server installed -- SSH access: `ssh cbesson@meshbay.org` - -### Local — Fedora 44 (Node) -- Laptop behind SFR residential NAT (likely Restricted Cone NAT — UPnP supported) -- Python 3.13+ via system packages -- User: `cbesson` (sudoer, no password) - ---- - -## Python Dependencies - -```bash -# Shared (hub and node) -cryptography>=43.0 # Ed25519, X25519, ChaCha20-Poly1305, Argon2id -PyJWT>=2.9 # JWT with EdDSA (Ed25519) support -blake3>=1.0 # Fast content hashing - -# Hub only (meshbay.org) -fastapi>=0.115 -uvicorn[standard]>=0.30 - -# Node only (Fedora laptop) -httpx>=0.28 # Async HTTP client for hub→node calls -aioice>=0.9 # STUN queries for NAT discovery -miniupnpc>=2.2 # UPnP port mapping on SFR box -``` - -Install on each machine: -```bash -python3 -m venv .venv -source .venv/bin/activate -pip install <packages above> -``` - ---- - -## Hub Setup on meshbay.org - -For the POC, uvicorn runs directly on port 80 via iptables redirect (no Caddy/nginx needed yet — HTTPS added before production). - -```bash -# On meshbay.org -# Redirect port 80 → 8000 (persistent via iptables-save if needed) -sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000 - -# Run hub (from poc directory, venv activated) -uvicorn hub:app --host 127.0.0.1 --port 8000 --reload -``` - -> Note: HTTPS (via Caddy + Let's Encrypt) is mandatory before any data beyond this POC. Not in scope here. - ---- - -## Spike Overview - -| # | Name | Where | Validates | Duration | -|---|---|---|---|---| -| 1 | Crypto primitives | Local | Python crypto stack covers all needs | ~1h | -| 2 | Hub skeleton | meshbay.org | Hub API, JWT issuance | ~2h | -| 3 | Node registration | Fedora | Hub-Node handshake, JWT offline verify | ~1h | -| 4 | NAT traversal | Both | SFR box UPnP + STUN, P2P reachability | ~2h | -| 5 | Encrypted transfer | Both | On-the-fly GEK encryption, P2P chunk | ~2h | - ---- - -## Spike 1 — Crypto Primitives (local only) - -**Goal:** confirm `cryptography` (PyCA) covers all MeshBay cryptographic needs without gaps or performance surprises. - -**File:** `spike1_crypto.py` - -**What to test:** - -```python -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id # PyCA 43+ -from cryptography.hazmat.primitives import hashes, serialization -import blake3, os, time -``` - -**Test 1: Ed25519 — hub keypair, sign JWT payload, verify** -```python -sk_hub = Ed25519PrivateKey.generate() -pk_hub = sk_hub.public_key() -msg = b"test payload" -sig = sk_hub.sign(msg) -pk_hub.verify(sig, msg) # raises if invalid -print("Ed25519 OK") -``` - -**Test 2: X25519 — two-party key agreement for GEK wrapping** -```python -sk_a = X25519PrivateKey.generate() -sk_b = X25519PrivateKey.generate() -shared_a = sk_a.exchange(sk_b.public_key()) -shared_b = sk_b.exchange(sk_a.public_key()) -assert shared_a == shared_b -print("X25519 OK") -``` - -**Test 3: GEK derivation and ChaCha20-Poly1305 on a 1 MB chunk** -```python -gek = ChaCha20Poly1305.generate_key() -cipher = ChaCha20Poly1305(gek) -chunk = os.urandom(1024 * 1024) # 1 MB - -t0 = time.perf_counter() -nonce = os.urandom(12) -ct = cipher.encrypt(nonce, chunk, None) -pt = cipher.decrypt(nonce, ct, None) -elapsed = time.perf_counter() - t0 - -assert pt == chunk -print(f"ChaCha20-Poly1305 1MB: {elapsed*1000:.1f} ms") -``` - -**Test 4: HKDF chunk key derivation** -```python -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes -chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + blake3.blake3(chunk).digest() + b":chunk:0" -).derive(gek) -print(f"HKDF derived key: {chunk_key.hex()[:16]}...") -``` - -**Test 5: Argon2id keystore key derivation** -```python -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id -salt = os.urandom(16) -t0 = time.perf_counter() -kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) -key = kdf.derive(b"mypassword") -print(f"Argon2id: {(time.perf_counter()-t0)*1000:.0f} ms, key: {key.hex()[:16]}...") -``` - -**Test 6: PyJWT with Ed25519 (EdDSA)** -```python -import jwt -sk_hub_pem = sk_hub.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() -) -pk_hub_pem = pk_hub.public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo -) -payload = {"sub": "user_abc", "pk_user": "base64...", "exp": 9999999999} -token = jwt.encode(payload, sk_hub_pem, algorithm="EdDSA") -decoded = jwt.decode(token, pk_hub_pem, algorithms=["EdDSA"]) -assert decoded["sub"] == "user_abc" -print("JWT EdDSA OK") -``` - -**Success criteria:** all tests pass, ChaCha20 1MB < 20ms, Argon2id ~1s. - ---- - -## Spike 2 — Hub Skeleton (meshbay.org) - -**Goal:** minimal FastAPI hub, in-memory storage, 5 endpoints. - -**File:** `hub.py` (on meshbay.org) - -### Hub keypair generation (run once, save to disk) - -```python -# gen_hub_keys.py — run once on meshbay.org -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives import serialization -import base64, json - -sk = Ed25519PrivateKey.generate() -pk = sk.public_key() - -with open("hub_private.pem", "wb") as f: - f.write(sk.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() - )) -with open("hub_public.pem", "wb") as f: - f.write(pk.public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo - )) -print("Hub keypair generated.") -``` - -### Hub API (`hub.py`) - -```python -from fastapi import FastAPI, HTTPException, Depends, Header -from pydantic import BaseModel -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives import serialization, hashes -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id -import jwt, uuid, os, time, base64 - -app = FastAPI(title="MeshBay Hub POC") - -# Load hub keypair -with open("hub_private.pem", "rb") as f: - HUB_SK_PEM = f.read() -with open("hub_public.pem", "rb") as f: - HUB_PK_PEM = f.read() - -HUB_ID = "meshbay.org" -ACCESS_TOKEN_TTL = 3600 # 1 hour -REFRESH_TOKEN_TTL = 86400 * 30 # 30 days - -# In-memory stores (POC only — not persistent) -users = {} # username → {user_id, pw_hash, pw_salt, pk_ed25519, pk_x25519} -nodes = {} # node_id → {user_id, pk_node, endpoint_hint, registered_at} -refresh_tokens = {} # token → user_id - -# --- Models --- - -class UserRegister(BaseModel): - username: str - password: str - pk_user_ed25519: str # base64 - pk_user_x25519: str # base64 - -class UserLogin(BaseModel): - username: str - password: str - -class NodeAnnounce(BaseModel): - pk_node: str # base64 Ed25519 public key - endpoint_hint: str | None = None # "ip:port" or null - -# --- Helpers --- - -def hash_password(password: str) -> tuple[bytes, bytes]: - salt = os.urandom(16) - kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) - return kdf.derive(password.encode()), salt - -def verify_password(password: str, pw_hash: bytes, salt: bytes) -> bool: - kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) - try: - kdf.verify(password.encode(), pw_hash) - return True - except Exception: - return False - -def issue_access_token(user: dict) -> str: - payload = { - "iss": HUB_ID, - "sub": user["user_id"], - "pk_user": user["pk_ed25519"], - "hub_id": HUB_ID, - "iat": int(time.time()), - "exp": int(time.time()) + ACCESS_TOKEN_TTL, - } - return jwt.encode(payload, HUB_SK_PEM, algorithm="EdDSA") - -def get_current_user(authorization: str = Header(...)) -> dict: - try: - scheme, token = authorization.split() - if scheme.lower() != "bearer": - raise ValueError - payload = jwt.decode(token, HUB_PK_PEM, algorithms=["EdDSA"]) - user_id = payload["sub"] - user = next((u for u in users.values() if u["user_id"] == user_id), None) - if not user: - raise HTTPException(status_code=401, detail="User not found") - return user - except Exception: - raise HTTPException(status_code=401, detail="Invalid token") - -# --- Endpoints --- - -@app.get("/v1/hub/info") -def hub_info(): - return { - "hub_id": HUB_ID, - "pk_hub_ed25519": base64.b64encode( - Ed25519PrivateKey.from_private_bytes( - # shortcut for POC — load pk directly - open("hub_public.pem","rb").read() - ).public_bytes(...) # see note below - ).decode(), - "mnp_version": "0.1", - "mhp_version": "0.1", - } - # Note: return pk_hub_pem directly for POC, nodes store it on first contact - -@app.get("/v1/hub/pubkey") -def hub_pubkey(): - """Return hub Ed25519 public key PEM — cached by nodes on first contact.""" - return {"pk_hub_pem": HUB_PK_PEM.decode()} - -@app.post("/v1/users/register", status_code=201) -def register(body: UserRegister): - if body.username in users: - raise HTTPException(status_code=409, detail="Username taken") - pw_hash, pw_salt = hash_password(body.password) - user_id = str(uuid.uuid4()) - users[body.username] = { - "user_id": user_id, - "username": body.username, - "pw_hash": pw_hash, - "pw_salt": pw_salt, - "pk_ed25519": body.pk_user_ed25519, - "pk_x25519": body.pk_user_x25519, - } - return {"user_id": user_id} - -@app.post("/v1/users/login") -def login(body: UserLogin): - user = users.get(body.username) - if not user or not verify_password(body.password, user["pw_hash"], user["pw_salt"]): - raise HTTPException(status_code=401, detail="Invalid credentials") - access_token = issue_access_token(user) - refresh_token = base64.urlsafe_b64encode(os.urandom(32)).decode() - refresh_tokens[refresh_token] = user["user_id"] - return { - "access_token": access_token, - "refresh_token": refresh_token, - "token_type": "bearer", - "expires_in": ACCESS_TOKEN_TTL, - } - -@app.post("/v1/users/token/refresh") -def refresh(body: dict): - rt = body.get("refresh_token", "") - user_id = refresh_tokens.get(rt) - if not user_id: - raise HTTPException(status_code=401, detail="Invalid refresh token") - user = next((u for u in users.values() if u["user_id"] == user_id), None) - if not user: - raise HTTPException(status_code=401, detail="User not found") - return {"access_token": issue_access_token(user), "token_type": "bearer"} - -@app.post("/v1/nodes/announce", status_code=201) -def announce_node(body: NodeAnnounce, user: dict = Depends(get_current_user)): - node_id = str(uuid.uuid4()) - nodes[node_id] = { - "node_id": node_id, - "user_id": user["user_id"], - "pk_node": body.pk_node, - "endpoint_hint": body.endpoint_hint, - "announced_at": int(time.time()), - } - return {"node_id": node_id} - -@app.get("/v1/nodes/{node_id}") -def get_node(node_id: str, user: dict = Depends(get_current_user)): - node = nodes.get(node_id) - if not node: - raise HTTPException(status_code=404, detail="Node not found") - return { - "node_id": node["node_id"], - "pk_node": node["pk_node"], - "endpoint_hint": node["endpoint_hint"], - } -``` - -**Success criteria:** -- Hub starts, all 6 endpoints respond correctly -- `GET /v1/hub/pubkey` returns the PEM -- `POST /v1/users/register` + `POST /v1/users/login` returns a valid JWT -- JWT verified by `jwt.decode()` with hub public key — passes - ---- - -## Spike 3 — Node Registration (Fedora laptop) - -**Goal:** node generates its keypair, registers a user on the hub, gets a JWT, and verifies it locally without contacting the hub again. - -**File:** `node.py` - -```python -import httpx, asyncio, jwt, base64, os -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey -from cryptography.hazmat.primitives import serialization - -HUB_URL = "http://meshbay.org" # HTTP for POC, HTTPS later - -async def main(): - async with httpx.AsyncClient() as client: - - # 1. Fetch hub public key (first contact — cache this) - r = await client.get(f"{HUB_URL}/v1/hub/pubkey") - hub_pk_pem = r.json()["pk_hub_pem"].encode() - print(f"[node] Hub PK fetched ({len(hub_pk_pem)} bytes)") - - # 2. Generate node identity keypairs - sk_ed = Ed25519PrivateKey.generate() - pk_ed = sk_ed.public_key() - sk_x = X25519PrivateKey.generate() - pk_x = sk_x.public_key() - - pk_ed_b64 = base64.b64encode( - pk_ed.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) - ).decode() - pk_x_b64 = base64.b64encode( - pk_x.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) - ).decode() - - # 3. Register user (skip if already registered) - r = await client.post(f"{HUB_URL}/v1/users/register", json={ - "username": "testnode", - "password": "testpass123", - "pk_user_ed25519": pk_ed_b64, - "pk_user_x25519": pk_x_b64, - }) - print(f"[node] Register: {r.status_code} {r.text}") - - # 4. Login, get access token - r = await client.post(f"{HUB_URL}/v1/users/login", json={ - "username": "testnode", - "password": "testpass123", - }) - data = r.json() - access_token = data["access_token"] - print(f"[node] Login OK, token: {access_token[:40]}...") - - # 5. Verify JWT locally — NO hub roundtrip - decoded = jwt.decode(access_token, hub_pk_pem, algorithms=["EdDSA"]) - print(f"[node] JWT verified locally: sub={decoded['sub']}, exp={decoded['exp']}") - - # 6. Announce node to hub - r = await client.post( - f"{HUB_URL}/v1/nodes/announce", - json={"pk_node": pk_ed_b64, "endpoint_hint": None}, - headers={"Authorization": f"Bearer {access_token}"} - ) - node_id = r.json()["node_id"] - print(f"[node] Node announced: {node_id}") - -asyncio.run(main()) -``` - -**Success criteria:** -- Node registers, logs in, receives JWT -- JWT decoded offline using only the hub's public key — no hub call -- Node announced; `GET /v1/nodes/{node_id}` from hub returns correct PK - ---- - -## Spike 4 — NAT Traversal (both machines) - -**Goal:** discover the local node's external IP:port via STUN and UPnP; test reachability from meshbay.org. - -**File:** `spike4_nat.py` (Fedora laptop) - -### Part A — UPnP (try first, most reliable on SFR box) - -```python -import miniupnpc -import socket - -def try_upnp(internal_port=19000): - u = miniupnpc.UPnP() - u.discoverdelay = 200 - ndevices = u.discover() - if ndevices == 0: - print("UPnP: no IGD found") - return None - - u.selectigd() - external_ip = u.externalipaddress() - local_ip = socket.gethostbyname(socket.gethostname()) - - result = u.addportmapping( - internal_port, 'TCP', local_ip, internal_port, - 'MeshBay POC', '' - ) - if result: - print(f"UPnP: mapped {external_ip}:{internal_port} → {local_ip}:{internal_port}") - return f"{external_ip}:{internal_port}" - else: - print("UPnP: mapping failed") - return None -``` - -### Part B — STUN discovery - -```python -import asyncio -import aioice - -async def stun_discover(local_port=19001): - # Use Cloudflare STUN server - stun_servers = [("stun.cloudflare.com", 3478), ("stun.l.google.com", 19302)] - - connection = aioice.Connection(ice_controlling=True, stun_server=stun_servers[0]) - await connection.gather_candidates() - - for candidate in connection.local_candidates: - if candidate.type == "srflx": # server-reflexive = external address - print(f"STUN srflx: {candidate.host}:{candidate.port}") - return f"{candidate.host}:{candidate.port}" - - print("STUN: no srflx candidate found (may be symmetric NAT)") - return None -``` - -### Part C — Reachability test from meshbay.org - -Once the node has an external address (from UPnP or STUN), it announces it to the hub (`endpoint_hint`). Then from meshbay.org: - -```bash -# On meshbay.org — manually test TCP reachability -nc -zv <external_ip> <external_port> -# or -python3 -c "import socket; s=socket.create_connection(('<external_ip>', <port>), timeout=5); print('REACHABLE'); s.close()" -``` - -And on the Fedora node, a simple listener: -```python -# On Fedora, open a listener on the discovered port -import socket -s = socket.socket() -s.bind(('', 19000)) -s.listen(1) -print("Listening on 19000...") -conn, addr = s.accept() -print(f"Connection from {addr}") -conn.sendall(b"HELLO FROM NODE\n") -conn.close() -``` - -**Expected outcomes on SFR residential:** - -| Method | Expected result | Confidence | -|---|---|---| -| UPnP | Works — SFR La Box supports UPnP IGD | High | -| STUN srflx | Discovered — SFR is cone NAT for residential | High | -| Direct TCP from meshbay.org | Works if UPnP succeeded | High | -| Hole punching only | Depends on NAT type discovered | Medium | - -**Success criteria:** at least one method allows meshbay.org to reach the Fedora node's port directly. - ---- - -## Spike 5 — Encrypted File Transfer (both machines) - -**Goal:** node serves an encrypted file chunk via direct P2P TCP connection; client decrypts and verifies. - -**Prerequisite:** Spike 4 succeeded — external IP:port is known and reachable. - -**File:** `spike5_server.py` (Fedora), `spike5_client.py` (meshbay.org) - -### Node side — serve one encrypted chunk - -```python -# spike5_server.py — Fedora laptop -import asyncio, os, base64 -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes, serialization -import blake3, struct, json - -# Keypair (reuse from Spike 3 or generate here) -sk_node = Ed25519PrivateKey.generate() -pk_node_bytes = sk_node.public_key().public_bytes( - serialization.Encoding.Raw, serialization.PublicFormat.Raw -) - -# Generate GEK (in a real system, loaded from keystore) -gek_raw = ChaCha20Poly1305.generate_key() -cipher = ChaCha20Poly1305(gek_raw) - -CHUNK_SIZE = 1024 * 1024 # 1 MB - -def make_chunk(file_path: str, chunk_index: int) -> bytes: - """Read, compress (skipped for POC), encrypt, sign a chunk.""" - with open(file_path, "rb") as f: - f.seek(chunk_index * CHUNK_SIZE) - data = f.read(CHUNK_SIZE) - - file_hash = blake3.blake3(open(file_path, "rb").read()).digest() - - # Per-chunk key derivation - chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + file_hash + b":chunk:" + chunk_index.to_bytes(4, "big") - ).derive(gek_raw) - chunk_cipher = ChaCha20Poly1305(chunk_key) - - nonce = os.urandom(12) - ct = chunk_cipher.encrypt(nonce, data, None) - chunk_hash = blake3.blake3(ct).digest() - - # Sign: chunk_index + nonce + ciphertext_hash - sig_payload = chunk_index.to_bytes(4, "big") + nonce + chunk_hash - sig = sk_node.sign(sig_payload) - - return json.dumps({ - "chunk_index": chunk_index, - "nonce": base64.b64encode(nonce).decode(), - "ciphertext": base64.b64encode(ct).decode(), - "chunk_hash": base64.b64encode(chunk_hash).decode(), - "signature": base64.b64encode(sig).decode(), - "pk_node": base64.b64encode(pk_node_bytes).decode(), - "gek_hint": base64.b64encode(gek_raw).decode(), # POC: send GEK in band — never in production! - }).encode() - -async def handle_client(reader, writer): - request = await reader.read(1024) - req = json.loads(request) - chunk_index = req.get("chunk_index", 0) - file_path = req.get("file", "testfile.bin") - - print(f"[node] Client requests chunk {chunk_index} of {file_path}") - chunk_data = make_chunk(file_path, chunk_index) - - writer.write(len(chunk_data).to_bytes(4, "big") + chunk_data) - await writer.drain() - writer.close() - print(f"[node] Chunk {chunk_index} sent ({len(chunk_data)} bytes)") - -async def main(): - # Create a 5MB test file - if not os.path.exists("testfile.bin"): - with open("testfile.bin", "wb") as f: - f.write(os.urandom(5 * 1024 * 1024)) - print("[node] Test file created (5 MB)") - - server = await asyncio.start_server(handle_client, "0.0.0.0", 19000) - print("[node] Serving on port 19000 — waiting for client...") - async with server: - await server.serve_forever() - -asyncio.run(main()) -``` - -### Client side — request, verify, decrypt - -```python -# spike5_client.py — meshbay.org -import asyncio, base64, json -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes, serialization -import blake3 - -NODE_HOST = "<external_ip>" # from Spike 4 -NODE_PORT = 19000 - -async def main(): - reader, writer = await asyncio.open_connection(NODE_HOST, NODE_PORT) - - # Request chunk 0 - request = json.dumps({"file": "testfile.bin", "chunk_index": 0}).encode() - writer.write(request) - await writer.drain() - - # Receive - length_bytes = await reader.readexactly(4) - length = int.from_bytes(length_bytes, "big") - data = await reader.readexactly(length) - writer.close() - - chunk = json.loads(data) - print(f"[client] Received chunk {chunk['chunk_index']}") - - # 1. Verify signature - pk_node_bytes = base64.b64decode(chunk["pk_node"]) - pk_node = Ed25519PublicKey.from_public_bytes(pk_node_bytes) - ct = base64.b64decode(chunk["ciphertext"]) - nonce = base64.b64decode(chunk["nonce"]) - chunk_hash = base64.b64decode(chunk["chunk_hash"]) - sig = base64.b64decode(chunk["signature"]) - - sig_payload = (0).to_bytes(4, "big") + nonce + chunk_hash - pk_node.verify(sig, sig_payload) # raises on failure - print("[client] Signature OK") - - # 2. Verify ciphertext hash - assert blake3.blake3(ct).digest() == chunk_hash - print("[client] Ciphertext hash OK") - - # 3. Derive chunk key and decrypt (GEK from POC hint — never in production) - gek_raw = base64.b64decode(chunk["gek_hint"]) - # (in production, client has GEK from hub's GEK bundle) - chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + bytes(32) + b":chunk:" + (0).to_bytes(4, "big") - # Note: in production, file_hash is sent separately or in index - ).derive(gek_raw) - plaintext = ChaCha20Poly1305(chunk_key).decrypt(nonce, ct, None) - print(f"[client] Decrypted {len(plaintext)} bytes") - print("[client] Encrypted P2P transfer: SUCCESS") - -asyncio.run(main()) -``` - -**Note on GEK in POC:** the GEK is included in the response as `gek_hint` for POC convenience only. In production, the client gets the GEK from the hub's encrypted GEK bundle (delivered at login, decrypted client-side with the user's X25519 private key). - -**Success criteria:** -- Client receives chunk from node via direct TCP connection -- Signature verification passes -- Ciphertext hash matches -- Decryption produces the original bytes -- End-to-end: `original_bytes == decrypted_bytes` ✓ - ---- - -## What POC Validates (and Doesn't) - -### Validated by these spikes - -| Concept | Spike | Validation | -|---|---|---| -| Python crypto stack is sufficient | 1 | All primitives work, performance acceptable | -| Hub/Node JWT handshake | 2, 3 | JWT issued by hub, verified offline by node | -| Hub-Node REST protocol (minimal MNP/HTTP) | 2, 3 | API contract works end-to-end | -| SFR NAT traversal via UPnP | 4 | P2P reachability confirmed | -| STUN external address discovery | 4 | Confirmed/fallback documented | -| On-the-fly per-chunk encryption | 5 | GEK + HKDF chunk derivation + ChaCha20 | -| Chunk signature and verification | 5 | Ed25519 sign/verify before decryption | -| Real P2P file transfer | 5 | No hub in data path | - -### NOT in scope - -- Database (all in-memory) -- HTTPS / TLS (HTTP for POC) -- QUIC transport (plain TCP) -- GEK bundle distribution via hub (GEK sent in-band for POC) -- Group management -- Chat / Double Ratchet -- Mesh Group Index -- MHP federation -- Android client -- Module system -- Persistence between restarts - ---- - -## Spike Order Dependency Graph - -``` -Spike 1 (crypto) - └──→ Spike 2 (hub skeleton) - └──→ Spike 3 (node registration) - └──→ Spike 4 (NAT traversal) - └──→ Spike 5 (encrypted transfer) -``` - -Spike 1 is a prerequisite for all others. Spikes 2 and 3 can overlap if two people work in parallel. Spike 4 can begin independently once Spike 3 is running. diff --git a/second-review.md b/docs/second-review.md index 1b684af..36caabb 100644 --- a/second-review.md +++ b/docs/second-review.md @@ -2,7 +2,7 @@ > Date: 2026-08-13 > Scope: architecture and security design review of the hub ↔ node ↔ client protocol, -> as specified in `docs/meshbay-draft-v4.md`, `devel-phases.md`, `devel-phases-next.md`, +> as specified in draft v4 and the Phase 1–12 log (both archived in `old-draft.md`), `devel-phases-next.md`, > and as **implemented** in `packages/` (Phases 1–12 + 10b/10c). > > Unlike `first-review.md` (2026-08-10), which was a design-level review, this one reads @@ -399,7 +399,7 @@ Consider a localhost token in the URL to blunt DNS-rebinding against the unauthe > proves possession of over the authenticated channel, and identities are bound to > accounts by one-time codes the hub never sees. Safety numbers would have made the > substitution *detectable by a human who checks*; removing the lookup makes it -> impossible. See `docs/invite-pairing-v1.md` and draft-v5 §5.5. +> impossible. See `invite-pairing-v1.md` and draft-v5 §5.5. > > `gek-init` had the same flaw with the node as the victim — it fetched every member's > public key from the hub and wrapped for the answer. That is gone too. diff --git a/tmp-decisions.md b/docs/tmp-decisions.md index 76d33d2..70f5171 100644 --- a/tmp-decisions.md +++ b/docs/tmp-decisions.md @@ -1,7 +1,7 @@ # Client architecture — decisions > Created 2026-08-13 after the second security review. D1/D2/D3 decided the same day; -> D4 (hub minimization) deferred. Fold into `docs/meshbay-draft-v5.md`. +> D4 (hub minimization) deferred. Fold into `meshbay-draft-v5.md`. > The analysis below is kept as the rationale behind the decisions, not as open questions. --- @@ -11,7 +11,7 @@ | # | Decision | State | |---|---|---| | D1 | Does the hub keep serving the web UI? | ✅ **DECIDED 2026-08-13 — yes** | -| D2 | Browser extension, native desktop client, or both? | ✅ **DECIDED 2026-08-13 — native client, offered alongside the hub-served SPA.** Shell revised 2026-08-17: **Electron**, see `docs/desktop-client-v1.md` | +| D2 | Browser extension, native desktop client, or both? | ✅ **DECIDED 2026-08-13 — native client, offered alongside the hub-served SPA.** Shell revised 2026-08-17: **Electron**, see `desktop-client-v1.md` | | D3 | Transport: aiortc primary, QUIC at parity, TCP+HTTP removed | ✅ Decided 2026-08-13. Unchanged for the **node**; the desktop client uses Chromium's WebRTC rather than aiortc, and QUIC via a Python sidecar | | D4 | Hub minimization (old Phase 12) | ⏸️ **Deferred, may be dropped** | @@ -112,7 +112,7 @@ Phase 13. Full control, durable keys in an OS keystore, QUIC, hub-less access, b Highest effort, and the security argument depends on 18.7. > **Shell revised 2026-08-17: Electron, not pywebview** (+ an optional Python sidecar for -> `group://` over QUIC). See `docs/desktop-client-v1.md` §2. The comparison table below +> `group://` over QUIC). See `desktop-client-v1.md` §2. The comparison table below > was written against pywebview and **two of its rows are wrong for the chosen shell**: > > - *Browser sandbox* — Electron with `sandbox` and `contextIsolation` **keeps** the |