diff options
Diffstat (limited to 'docs/old-draft.md')
| -rw-r--r-- | docs/old-draft.md | 4497 |
1 files changed, 0 insertions, 4497 deletions
diff --git a/docs/old-draft.md b/docs/old-draft.md deleted file mode 100644 index f02c54f..0000000 --- a/docs/old-draft.md +++ /dev/null @@ -1,4497 +0,0 @@ -# MeshBay — Archived Drafts - -> **Status: historical archive. Nothing here is authoritative.** -> -> This file consolidates the superseded architecture drafts (v1–v4), the original -> POC plan, and the Phase 1–12 development log. They are kept for provenance and -> for the section references (`draft-v3 §4.1.3`, `draft-v4 §6.6`, …) still made -> from live documents and code comments. -> -> For what is true now, read instead: -> -> | Topic | Document | -> |---|---| -> | Current specification | `meshbay-draft-v6.md` (+ `meshbay-draft-v5.md` for what v6 does not restate) | -> | Roadmap | `devel-phases-next.md` | -> | Security findings | `first-review.md`, `second-review.md` | -> | Client architecture decisions | `tmp-decisions.md`, `desktop-client-v1.md` | -> -> Contents of this archive, in order: -> 1. Architecture Draft v1 -> 2. Architecture Draft v2 -> 3. Architecture Draft v3 -> 4. Architecture Draft v4 -> 5. POC v1 -> 6. Development Phases (1–12) - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: Architecture Draft v1 (was docs/meshbay-draft-v1.md) --> -<!-- ============================================================ --> - -# MeshBay — Architecture Draft v1 - -> Status: preliminary draft — many points still open, marked [TBD] - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), aiming to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | Encrypted file listing for a group (node level) | - ---- - -## 3. System Components - -### 3.1 Mesh Hub - -A lightweight server acting as a registrar. It is intentionally kept minimal to reduce legal exposure and operational burden. - -**What the hub stores:** -- User accounts: username, hashed email, `PK_user` (public key fingerprint), hub ID, status -- Group registry: name, `PK_group`, hosting node address, visibility, member list with encrypted GEK bundles -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node IP addresses (handled by ephemeral signaling service) - -**Hub interactions — when is it called?** - -| Event | Hub load | Frequency | -|---|---|---| -| Account creation | Hash credential, store PK | Once | -| Login | Verify credentials, issue signed JWT | Per session (~30-day validity) | -| Group creation | Register name, PK_group, node | Once per group | -| Member add/remove | Store/remove encrypted GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay a few WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate query to nodes at request time | On demand | -| MHP federation sync | Exchange public directory updates | Background, periodic | -| Revocation | Issue signed revocation token | Rare | - -**The hub is never in the data path after initial connection setup.** - -**JWT as offline passport:** -The hub issues a JWT signed with its Ed25519 private key. Nodes verify this JWT locally using the hub's known public key — no hub roundtrip required per request. JWT validity: ~30 days. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: behind Apache reverse proxy (ProxyPass) -- Authentication: own system (JWT signed with Ed25519, no OAuth dependency) - -**Account creation:** [TBD] — email only at first, phone number associable later. Via Android app, both collected by default. Fusionable accounts. - -### 3.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) -- Serve files and video streams to group members -- Manage all cryptographic keys locally (keystore, password-protected) -- Handle P2P connections and NAT traversal -- Run the MNP protocol -- Host the Python module sandbox -- Serve the local web UI (localhost) -- Receive and redistribute ephemeral video from mobile [future] - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -**Tech stack:** -- Language: Python (primary), Rust extensions only if strictly necessary for hot paths -- QUIC: `aioquic` -- ICE/STUN: `aioice` -- WebRTC (future): `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by the node on localhost (port [TBD]) - -**Node pairing with mobile:** QR code from local web UI [future]. - -### 3.3 Mesh Client - -Web browser or Android app. Consumes content from the node; manages account via the hub. - -**Hub-side operations (via hub):** -- Account creation and login -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing (Mesh Group Index) -- Message feed reading (with attachments, Signal-like) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: browse files in a group -- Feed mode: message thread with attachments -- Hub/Node UI split to be defined - -### 3.4 Mesh Relay - -Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail. Traffic is always E2E encrypted — the relay sees only opaque QUIC packets and cannot read content. - -Not operated by meshbay.org. A protocol for relay registration with hubs is [TBD]. - ---- - -## 4. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: shared files, message thread, member list. It can be open entry, request-based, or invitation-only regardless of its public visibility. - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members with the GEK can decrypt anything. - -**Group addressing** [TBD]: -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — direct public group -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` - ---- - -## 5. Cryptographic Architecture - -### 5.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys are stored exclusively on the node (or client device), in a password-protected local keystore. The hub never sees any private key. - -### 5.2 GEK Management - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK is encrypted for each member using X25519 key agreement + HKDF -3. Encrypted GEK bundles stored on hub (or node — [TBD]) - -**Member addition:** -- GEK encrypted with new member's `PK_user` and distributed - -**Member revocation:** -- Admin node generates a new GEK -- Re-encrypts for all remaining members -- New content encrypted with new GEK -- Former member retains ability to decrypt previously received content (acceptable trade-off — full re-encryption not planned) - -### 5.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time before transmission. - -``` -Disk (plaintext) → [Node] → zstd compress → GEK encrypt (per-chunk) → QUIC session → [Client] → QUIC decrypt → GEK decrypt → plaintext -``` - -**Chunking strategy:** -- Chunk size: 1 MB (amortizes AEAD overhead, allows seeking) -- Per-chunk key derivation: - `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` -- Each chunk independently decryptable (enables video seeking) -- Compress before encrypt (zstd compression is useless after encryption) - -**Chunk authentication:** -Each chunk (or batch) is signed with the node's Ed25519 key. The client verifies before decryption. Prevents data injection by a compromised relay. - -### 5.4 Transport Security - -- Primary protocol: **QUIC** (TLS 1.3 integrated, UDP-based, multiplexed) -- Per-connection session keys via X25519 ECDH + HKDF -- The QUIC layer is independent from the GEK application layer — two independent encryption layers - -### 5.5 Chat Encryption - -Group messaging uses the **Double Ratchet algorithm** (as in Signal): -- Forward secrecy and break-in recovery per message -- Each message independently encrypted -- Implementation: existing Python or Rust library [TBD] - ---- - -## 6. Network and Connectivity - -### 6.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection, no NAT issue -2. UPnP / NAT-PMP on router → node opens port automatically -3. ICE + STUN / UDP hole punching → works for ~80-85% of cases -4. Mesh Relay (TURN fallback) → community-operated, E2E encrypted traffic -``` - -**Signaling** (steps 3/4): coordinated via hub WebSocket, <1 KB per attempt, stateless after connection established. - -**Step 4 coverage:** ~15-20% of connections (symmetric NAT on both sides, CGNAT). The relay sees only encrypted QUIC packets. - -### 6.2 MNP — Mesh Node Protocol - -Application-level protocol over QUIC. Defined blocks: - -- **Handshake**: key exchange, group membership verification (JWT presentation) -- **Index sync**: encrypted delta Mesh Group Index on connection -- **File transfer**: chunk request/response with hash verification -- **VOD streaming**: HLS/DASH segments, encrypted per-segment with GEK-derived keys -- **Messaging**: Double Ratchet messages encapsulated in MNP frames -- **[Future] Ephemeral stream**: `ephemeral_stream` type with TTL metadata - -### 6.3 Public Content Delivery - -Public files are identified by their `blake3` hash. Multiple nodes can serve the same file: - -1. Node A has public file X (hash H) -2. Any node that obtains X and chooses to mirror it registers with the hub: "I serve hash H" -3. Hub maintains: `{ blake3_hash → [node_A, node_B, ...] }` -4. Client requests X → hub returns source list → client fetches in parallel chunks from multiple nodes - -**Public content transport:** TLS only (no GEK). Content signed with the original node's Ed25519 key for authenticity verification by clients, even when served from a mirror. Door left open for GEK on "registered-users-only public" groups in a future revision. - ---- - -## 7. Indexes - -### 7.1 Mesh Directory (hub level) - -Public registry of groups. Exchanged between hubs via MHP. - -Format: msgpack, signed by hub's Ed25519 key. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy. - -### 7.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: msgpack → zstd compressed → GEK encrypted (private groups) or plaintext signed (public groups). - -Entry structure: -```python -{ - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", # relative to shared directory - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash":"<blake3>", # thumbnail hash (thumbnail also GEK-encrypted) - "added_at": 1720000000 -} -``` - -**Delta updates:** each update carries `{base_version, additions, deletions}` — no full re-encryption on every change. - -**Transit:** nodes push index deltas to connected members on change. Members pull full index on first connection. Hub stores no index content — only the node address for routing. - -### 7.3 Search - -**Private groups:** search is entirely local on the client device. The client maintains a local encrypted cache of all indexes for groups it belongs to. No network call, no hub involvement, instant results. - -**Public groups:** client queries node(s) directly at request time. Hub provides routing (which node hosts which group) but performs no content lookup itself. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this interaction. In-memory micro-cache of results: **60-second TTL maximum, RAM only, never written to disk, public content only.** This qualifies as technical caching (EU DSA Article 13) and does not constitute indexing. - ---- - -## 8. Hub Federation (MHP) - -### 8.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, CA-delegated) - │ └── issues user credentials, manages its own groups - │ └── can federate with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub) proving its authority. Clients verify the chain. A Mirror Hub can only replicate public directory data. - -### 8.2 MHP Design Principles - -- Explicit peer selection: each hub maintains an allowlist of trusted peer hubs -- No automatic hub discovery -- Exchanged data: Mesh Directory (public groups), revocation lists, cross-hub user credentials -- Cross-hub authentication: user from Hub A presents JWT signed by Hub A; Hub B verifies using Hub A's public key (fetched once on first interaction, cached) - -### 8.3 Cross-Hub Client Access - -Client from Hub A accessing a group on Hub B: -1. Hub A's public directory or direct link leads client to Hub B -2. Client presents Hub A JWT to Hub B directly -3. Hub B verifies JWT signature using Hub A's public key -4. Hub B issues a short-lived local token for this session -5. Client proceeds to node as normal - ---- - -## 9. Moderation - -### 9.1 Public Content - -``` -Report #1 → automatic suspension of public access to content - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: blake3 hash of content added to hub blocklist. Node receives signed revocation notice and cuts public access. - -### 9.2 CSAM - -Hash matching against NCMEC/IWF database on all public content at registration time. Participation demonstrates good faith and significantly reduces legal exposure. No scanning of private/encrypted content. - -### 9.3 Copyright - -DMCA/legal notice framework (takedown on notification). No automated technical blocking — too complex, too many false positives (fair use, regional variations). Hub can revoke on confirmed legal request. - -### 9.4 Private Content - -Not directly moderatable (E2E encrypted by design). Only action available: revoke user or group at hub level on formal legal request. Hub issues a signed revocation token that all group members' nodes can verify. - ---- - -## 10. Python Module System - -The node can load extension modules (Python) that run in a sandboxed subprocess. - -**Module manifest** (declared capabilities): -```python -{ - "name": "group-chat", - "version": "1.0.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs (restricted):** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post a message to the group thread -- `receive_events(handler)` — subscribe to group events (new file, new message) - -**Not available:** -- Arbitrary network access -- Filesystem access outside the group context -- System calls - -**First official module:** group chat thread (Signal-like, with attachments). Bundled with node. - ---- - -## 11. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node software clearly communicates this at setup. - -**Hub operator:** registrar, not content host. Stores minimal PII. Operates takedown mechanism. Participates in CSAM hash matching. Analogous to a domain registrar in legal exposure terms. - -**Protocol/software author:** protected by substantial non-infringing uses. No active facilitation of infringement. - -**Hub data minimization:** -- Email stored hashed after verification [TBD] -- No IP address logging (or auto-deletion after 24h) -- No content metadata stored -- Node current address managed by ephemeral signaling service only - ---- - -## 12. Future Features (noted, not designed) - -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Mobile video push:** mobile films → pushes to hosting node → distributed as ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Mesh Relay registration protocol:** community TURN relays registered with hubs -- **Node mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from multiple nodes for same public file (swarm) -- **iOS client** -- **At-rest encryption on node:** optional, for nodes deployed on remote servers - ---- - -## 13. Open Questions [TBD] - -1. **GEK bundle storage:** on hub or on node only? Hub = easier discovery; node only = more decentralized -2. **Group address scheme:** final URL format -3. **Hub local web UI scope for V1:** config only, or also group browsing? -4. **Account creation:** email only to start, phone associable — confirm -5. **Chat implementation:** bundled module or core feature? -6. **QUIC library maturity:** `aioquic` production readiness assessment needed -7. **Double Ratchet library:** identify best Python implementation -8. **Relay registration protocol:** design when community relays are introduced -9. **Cross-hub directory exchange:** frequency, conflict resolution -10. **Node port for local web UI:** to assign -11. **JWT expiry and refresh strategy** -12. **Keystore format and unlock mechanism on node startup** - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: Architecture Draft v2 (was docs/meshbay-draft-v2.md) --> -<!-- ============================================================ --> - -# MeshBay — Architecture Draft v2 - -> Status: preliminary draft — open points marked [TBD] -> Changes from v1: IP logging (legal), protocol versioning, hardware sizing, JWT strategy, keystore proposals, chat as core, relay moved to future, hub mirror future, GEK clarified, port 18000, lazy admin keystore. - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user`, hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database. - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip).** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, hub-signed groups membership claim -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) -- Serve files and video streams to group members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal -- Run the MNP protocol -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) -- Host the group chat (core feature) - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity, group identity, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id (tuned for ~1s derivation on target hardware). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples QUIC from TCP+TLS fallback -- QUIC: `aioquic` (Cloudflare-maintained). Fallback: TCP + TLS 1.3 + HTTP/2 if QUIC proves insufficient in production -- ICE/STUN: `aioice` -- WebRTC [future]: `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. - -**Hub-side operations:** -- Account creation and login (Android: email + phone at registration) -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing via Mesh Group Index -- Group chat (messages + attachments, Signal-like — core feature) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: file browser for group content -- Feed mode: chat thread with attachments -- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (~15–20% of connections). Traffic is always E2E encrypted — the relay sees only opaque QUIC packets. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK encrypted for each member via X25519 key agreement + HKDF -3. Encrypted GEK bundles stored on hub (opaque blobs — hub cannot decrypt them; charge is negligible: ~200–400 bytes per member per group) - -**Storing on hub rationale:** members can retrieve their GEK bundle even when the node is offline. Hub exposure is minimal — it stores ciphertext it cannot read. - -**Member addition:** -- GEK encrypted with new member's `PK_user` and uploaded to hub - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → QUIC session → Client → QUIC decrypt → GEK decrypt → plaintext -``` - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~500 MB/s on hardware without AES-NI; AES-256-GCM: >2 GB/s with AES-NI -- For typical home node (50 Mbps upload = 6 MB/s), encryption is not the bottleneck -- For high-concurrency scenarios: asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -- Primary: **QUIC** (TLS 1.3 integrated, UDP, multiplexed streams) -- Fallback: **TCP + TLS 1.3 + HTTP/2** (same application protocol, lower performance) -- Transport interface abstracted in code — swappable without protocol changes -- Per-connection session keys via X25519 ECDH + HKDF (independent of GEK layer) - -### 6.5 Chat Encryption - -Group chat is a **core feature** (not an extension module). Uses the **Double Ratchet algorithm** (as in Signal): -- Forward secrecy and break-in recovery per message -- Each message independently encrypted -- Attachment files: encrypted with the current Double Ratchet message key, hash included in message -- Python implementation: [TBD — evaluate existing libraries] - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection -2. UPnP / NAT-PMP on router → node opens port automatically -3. ICE + STUN / UDP hole punching → ~80–85% success rate -4. Mesh Relay (TURN) → [future feature] -``` - -Without step 4, ~15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -Signaling (step 3): coordinated via hub WebSocket endpoint, <1 KB per attempt, no persistent state. - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over QUIC (or TCP+TLS fallback). All messages carry a `version` field. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + signature | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Double Ratchet encrypted message frame | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000 -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Hub Federation (MHP) - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 10. Moderation - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 11. Python Extension Module System - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 12. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) - ---- - -## 13. Future Features - -- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **iOS client** -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** - ---- - -## 14. Open Questions [TBD] - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **Double Ratchet library:** identify best Python implementation -4. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? -5. **MHP federation sync frequency and conflict resolution** -6. **Hub mirror replication strategy** (when implemented) -7. **Chat attachment storage:** stored on node like regular files, or separate store? -8. **Relay registration protocol design** (when implemented) -9. **JWT payload claims:** exact fields to include for node group-access verification -10. **Argon2id parameters:** tuning for target hardware (home server vs. VPS) - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: Architecture Draft v3 (was docs/meshbay-draft-v3.md) --> -<!-- ============================================================ --> - -# MeshBay — Architecture Draft v3 - -> Status: preliminary draft — open points marked [TBD] -> Changes from v2: jti mandatory in JWT (Spike 3), Argon2id params corrected (Spike 1), NAT traversal order corrected (Spike 4), transport flipped to TCP+TLS 1.3 v1 / QUIC v2, GEK wrapping protocol confirmed with exact parameters (Spike 6), hub API table expanded with 4 new endpoints (Spike 6), package structure decided (3 packages, uv monorepo), key persistence requirement added (Spike 6), new sections: Hub API Reference, TCP+TLS Transport v1, Package Structure. - ---- - -## Changes from v2 - -The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | -| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | -| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | -| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | -| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | -| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | -| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | -| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) -- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. -> -> This bug was found and fixed during Spike 3. - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -#### 4.1.5 Hub API Reference - -Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. - -**Hub metadata:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | -| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | - -**User management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | -| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | -| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | -| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | - -**Node management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | -| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | - -**Group management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | -| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | -| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | -| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | - -**GEK distribution (private groups):** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | -| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | - -**Revocation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/revoke/user/{user_id}` | Access token (admin) | Revoke a user account | [TBD] | -| POST | `/v1/revoke/group/{group_id}` | Access token (admin) | Revoke a group | [TBD] | -| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) — one directory per group -- Serve files, video streams, and group chat to members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) -- Run the MNP protocol (QUIC v2, TCP+TLS v1) -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) - -**Multi-group architecture (decided Phase 7):** -A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated -by port — the MNP handshake identifies the target group via the `group_id` claim -in the client JWT. The server routes each connection to the appropriate -DirectoryIndexer and GEK after JWT verification. -Rationale: one NAT hole to maintain, one port to forward manually if needed. - -**Authorization invariant:** the node MUST verify that the JWT's `groups` claim -contains the requested group_id before serving any content. Without this check, -any authenticated user could access any group on the node. This is enforced at -the MNP handshake layer, not the transport layer. - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. - -**Argon2id parameters (production):** -- `iterations = 4` -- `memory_cost = 262144` (256 MB) -- `parallelism = 1` (or match CPU count — tune to target hardware) -- Target derivation time: ~500 ms on a home server - -> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. - -**CLI calibration:** -``` -meshbay-node --calibrate-argon2 -``` -This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. - -**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. -- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) -- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) -- ICE/STUN: `aioice` -- WebRTC [future]: `aiortc` -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. - -**Hub-side operations:** -- Account creation and login (Android: email + phone at registration) -- Public group search and discovery -- Group membership management - -**Node-side operations (direct P2P):** -- File browsing via Mesh Group Index -- Group chat (messages + attachments, Signal-like — core feature) -- File download -- Video streaming (VOD) -- [Future] Ephemeral video feed - -**Client modes** [to be designed]: -- Explorer mode: file browser for group content -- Feed mode: chat thread with attachments -- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - -### 4.5 Package Structure and Repository - -**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). - -``` -meshbay/ -├── packages/ -│ ├── meshbay-common/ # Shared crypto, serialization, protocol types -│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) -│ └── meshbay-node/ # Node daemon + local web UI -├── poc/ # POC and spikes — reference implementation -│ ├── spike1_crypto/ -│ ├── spike2_hub/ -│ ├── spike3_node_reg/ -│ ├── spike4_nat/ -│ ├── spike5_transfer/ -│ ├── spike6_gek/ -│ └── spike-results.md -├── docs/ -│ └── meshbay-draft-v3.md -└── pyproject.toml # Workspace root -``` - -**Three packages:** - -| Package | RPM name | Contents | -|---|---|---| -| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | -| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | -| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | - -**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. - -**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: -- Exact crypto parameter choices (Spike 1) -- GEK wrapping/unwrapping implementation (Spike 6) -- Hub API skeleton (Spike 2) -- NAT detection and STUN interaction (Spike 4) -- TCP file transfer pipeline (Spike 5) - -Developers implementing production features should read the corresponding spike before writing production code. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Admin | The hosting node operator (legal host) | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. - -### 6.1.1 Key Generation Strategies - -Three strategies, depending on client type: - -**A — CLI / native node (Argon2id derivation)** -Keys are derived deterministically from `username + password`: -``` -salt = SHA-256("meshbay:v1:" + username) -seed = Argon2id(password, salt, length=64) -sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) -sk_x25519 = X25519.from_private_bytes(seed[32:]) -``` -Same credentials → same keys on any machine. Password recovery = key recovery. -Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. - -**B — Web browser (random keypairs + encrypted bundle)** -Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them -with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub -alongside the public keys. On subsequent logins, the hub returns the bundle -and the browser decrypts it locally with the password. - -The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. -Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. - -**C — Native node with keystore file** -Random keypairs generated once, stored in the Argon2id-encrypted keystore file -(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. - -**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). -A user who registered via CLI (A) and later tries to recover via web (B) with the same -password will get different keypairs. This is by design: users pick one registration path. -Cross-path recovery requires the admin to issue new GEK bundles. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** - -``` -Admin side (wrap_gek): - sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle - shared = X25519(sk_eph, pk_recipient) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - nonce = random_bytes(12) - wrapped = ChaCha20-Poly1305(wrap_key).encrypt( - nonce, gek, aad=pk_recipient) # aad binds bundle to recipient - bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire - # hub stores as opaque 48-byte blob - # (without pk_eph in compact form — see note) - -Member side (unwrap_gek): - shared = X25519(sk_recipient, pk_eph) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - gek = ChaCha20-Poly1305(wrap_key).decrypt( - nonce, wrapped, aad=pk_recipient) -``` - -> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. - -**Security properties confirmed in Spike 6:** -- Hub never sees the GEK in cleartext -- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls -- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected -- Wrong private key → AEAD authentication tag failure → immediate rejection - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK wrapped for each initial member via the protocol above -3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` -4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` - -**Member addition:** -- Admin fetches new member's `pk_x25519` from hub -- Wraps GEK for them and uploads bundle - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext -``` - -(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** - -| Operation | Time | -|---|---| -| Encrypt + sign (node side) | 3.2 ms | -| Verify + decrypt (client side) | 3.9 ms | -| Total crypto overhead (1 MB) | < 10 ms | -| Network transfer | 99–234 ms (network-limited) | - -Encryption is not the bottleneck. Network latency and bandwidth dominate. - -**Pipeline optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI -- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -**Implementation phases:** - -| Phase | Transport | Status | Notes | -|---|---|---|---| -| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | -| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | - -The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. - -**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. - -**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. - -### 6.5 TCP+TLS 1.3 Transport Implementation (v1) - -**Connection model:** -- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) -- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) -- TLS 1.3 mandatory; TLS 1.2 rejected -- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) -- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain - -**Handshake sequence:** -``` -Client → Node: TCP SYN -Node → Client: TLS ServerHello (self-signed cert, PK_node) -Client: verify cert against hub-fetched PK_node -Client → Node: TLS ClientFinished -Node → Client: MNP handshake request (version negotiation) -Client → Node: MNP handshake response (JWT access token, version) -Node: verify JWT offline (Ed25519, hub public key) -Node → Client: session established -``` - -**Message framing over TCP:** -- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` -- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) -- Each frame carries the MNP `version` field in its header - -**QUIC migration path (v2):** -- Replace TCP length-framing with QUIC streams (one stream per logical exchange) -- MNP handshake maps 1:1 to a QUIC handshake stream -- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) -- Chat messages map to a persistent QUIC stream -- No changes to JWT verification, GEK decryption, or Index sync logic - -**Port allocation:** -- `18000/tcp` — local web UI (loopback only, not exposed externally) -- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) -- Configurable via `~/.config/meshbay/node.toml` - -### 6.6 Chat Encryption and Model - -Group chat is a **core feature** (not an extension module). - -**Model (decided):** between a forum and Signal. -- **Persistent:** messages stored on the node (not ephemeral like Signal by default) -- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages -- **Scope:** per group (not per user pair) -- **Attachments:** files and images, shared like regular group files -- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect -- **Retention:** managed by the group admin (no automatic expiry) - -**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** - -The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): - -- Each group member generates a **sender key** (random symmetric chain key + signing keypair) -- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) -- Each existing member sends their current sender key to the new member -- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) -- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys -- O(N) state per member (one chain per group member), not O(N^2) -- The existing Double Ratchet implementation is kept for future 1:1 direct messaging - -Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. - -> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection (preferred) -2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) -3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) -4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile -``` - -> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. - -**Spike 4 findings:** -- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) -- UDP hole punching functional: bidirectional echo received from OVH VPS -- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results -- No CGNAT: stable public IPv4 (81.220.170.32) - -Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** -Currently the node punches blindly at startup; the client may connect 10-20s later -on an aging NAT entry, causing retransmissions. The coordinated flow uses the -existing hub→node WebSocket (revocation channel): -``` -Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} -Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} -Node : punch_nat(peer_ip, peer_port) immediately -Node → Hub (WS) : {type: "punch_ready"} -Hub → Client: 200 OK "connect now" -Client → QUIC: first packet < 2s after probe → fresh NAT entry -``` -demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. -The probe must come from the QUIC server's own socket (`punch_nat()` via -`_transport.sendto()`). The QUIC client must connect from the same port -as the probe's destination (`local_port=QUIC_PORT`). Handshake time -with proper signaling: < 200ms (vs 12.7s without). - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + Ed25519 signature | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Double Ratchet encrypted message frame | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000 -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Hub Federation (MHP) - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 10. Moderation - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 11. Python Extension Module System - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 12. Legal Framework - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) -- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them - ---- - -## 13. Future Features - -- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic. Necessary for symmetric NAT (CGNAT mobile, some professional ISPs). -- **QUIC transport (v2):** replace TCP+TLS 1.3 with QUIC once application protocol is stable. Transport abstraction layer makes this a drop-in replacement. -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **iOS client** -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** -- **WebRTC:** `aiortc` for browser-native P2P (no node required for clients) - ---- - -## 14. Open Questions [TBD] - -**Resolved by POC (no longer open):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R1 | Argon2id parameters: what values target ~500ms? | `iterations=4, memory_cost=262144` (256 MB). Use `meshbay-node --calibrate-argon2` for hardware-specific tuning. | Spike 1 | -| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, group membership claim. `jti` is mandatory (prevents replay, enables revocation). | Spike 3 | -| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). Hub stores opaque 48-byte blobs. | Spike 6 | -| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). UPnP unreliable — demoted to step 3. | Spike 4 | -| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1 (lower complexity, works everywhere). QUIC for v2 via `Transport` abstraction. | Spike 5 | -| R6 | Hub API: which endpoints for GEK distribution? | `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek` | Spike 6 | -| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. uv workspace monorepo. | POC | - -**Resolved by first security review (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R8 | Group chat encryption model? | Sender Keys protocol (Signal Groups approach). Double Ratchet kept for future 1:1 DM only. MLS considered for v2 if groups > 50 members. | Security review C1 | -| R9 | Token denylist distribution? | Push via existing hub→node WebSocket. Node maintains an in-memory jti set. MNP handshake checks the set before accepting a JWT. No periodic polling needed. | Security review S3 | -| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. GEK is CSPRNG output (already uniform), so HKDF extract step doesn't need a random salt. Spec wording corrected to match code (RFC 5869 compliant). | Security review M5 | -| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D recommendation. Code fixed from 128-bit to 96-bit. | Security review S4 | - -**Still open:** - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? -4. **MHP federation sync frequency and conflict resolution** -5. **Hub mirror replication strategy** (when implemented) -6. **Chat attachment storage:** stored on node like regular files, or separate store? -7. **Relay registration protocol design** (when implemented) -8. **QUIC migration timeline:** when is the application protocol considered stable enough to begin v2 transport implementation? -9. **Refresh token rotation:** implement one-time-use refresh tokens (rotate on each use, detect reuse as theft indicator). RFC 6819 §5.2.2.3. -10. **Email encryption at rest:** spec requires encrypted email/phone in DB, implementation stores plaintext. Needs server-side encryption with key from hub config. - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: Architecture Draft v4 (was docs/meshbay-draft-v4.md) --> -<!-- ============================================================ --> - -# MeshBay — Architecture Draft v4 - -> Status: active development — Phases 1–12 complete (except 10.9 → Phase 13), 191 tests. -> Changes from v3: web client transport (WebRTC DataChannel), web UI architecture, hub roles (admin/moderator), hub mirror design, browser-specific NAT traversal, chat storage clarified, Phase 8 security items resolved, Phase 10 site overlay + admin/moderation UI + notifications + group search + version endpoint, Phase 10b self-service UI (group create/join/invite, file upload, IndexedDB caching, cross-group search), **node sovereignty model** (§4.2.x — node operator is sole content authority, deny-by-default, uploader_id tracking), **cryptographic sovereignty enforcement** (GEK-HMAC handshake challenge, Ed25519 admin challenge-response, gek_req removed), **Phase 12 — P2P crypto material** (GEK+keypair bundles moved off hub to node BundleStore, password split, key persistence in IndexedDB/sessionStorage, DTLS channel binding fix). - ---- - -## Changes from v3 - -The following items are **architectural decisions** driven by Phase 8 implementation and web client design (2026-08-10). They supersede the corresponding text in v3. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | Browser transport | Web browsers use **WebRTC DataChannel** (with ICE/STUN) for P2P to nodes behind NAT. WebTransport cannot work because browsers cannot choose their UDP source port — Port-Restricted Cone NAT requires exact port matching. Native clients (desktop, Android) continue using QUIC with `punch_nat()`. | Web client design session | -| 2 | Hub signaling | Hub WebSocket extended to relay WebRTC signaling (SDP/ICE) between browser and node. <1 KB per message, stateless, no content. Same channel as jti denylist push and `client_incoming`. | Web client design session | -| 3 | Hub role | Reinforced: hub is registrar + signaling facilitator ONLY. Never proxies, stores, or touches content (files, streams, chat, indexes). All data lives on nodes. Clients connect E2E to nodes. | Design constraint | -| 4 | Chat storage | Chat messages stored on node(s) hosting the group, not on the hub. Browser retrieves chat from node via DataChannel. If no node is online, group is unavailable. | Web client design session | -| 5 | Web UI | Preact SPA (~3 KB gzipped), dark/light theme, responsive, i18n (JSON translations). ESM modules, esbuild for minification. No heavy frameworks. | Web client design session | -| 6 | Hub roles | Three roles: `user`, `moderator`, `admin`. Moderator can review reports and suspend content/groups/users. Admin has full hub management. | Web client design session | -| 7 | Site overlay | meshbay.org serves both generic hub functionality and site-specific pages (landing, /downloads, /about). Separated via Caddy static file priority. | Web client design session | -| 8 | Hub mirror | Design defined (future implementation): active-active with shared signing key, PostgreSQL logical replication, DNS round-robin. Not implemented yet. | Web client design session | -| 9 | Security items | S1 (admin authz), S2 (email encryption), S5 (refresh token rotation) resolved in Phase 8. Argon2id bumped to 256 MB with transparent rehash. | Phase 8 implementation | -| 10 | File search | Client-side search on cached indexes (IndexedDB). No hub involvement. Private group indexes are GEK-encrypted — hub stores opaque, client decrypts locally. | Web client design session | -| 11 | P2P crypto material | **ALL crypto material moved off hub to P2P channel.** GEK bundles and keypair bundles stored on node (`BundleStore` SQLite), exchanged via MNP DataChannel. Hub `GEKBundle` model and `/gek` endpoint removed. Hub never touches, stores, or proxies any crypto material. | Phase 12 — T3 attack surface reduction | -| 12 | Password split | Hub receives `auth_key` (PBKDF2-SHA512, auth salt), never raw password. Separate `bundle_key` (PBKDF2-SHA512, bundle salt) encrypts keypair bundles on the node. Hub cannot derive `bundle_key` from `auth_key`. | Phase 12 — T1 | -| 13 | Node auth | Node daemon authenticates to hub via Ed25519 signed timestamp (`POST /v1/nodes/auth`), not password. JWT `scope: "node"` blocks group mutation endpoints. | Phase 12 — NS7 | -| 14 | Key persistence | Browser stores `_bundleKey` (CryptoKey) in IndexedDB and `_sessionKeys` in sessionStorage. Survives page refresh without re-login. Public key derived from recovered private key via JWK export (`_pkFromSk`), no hub dependency. | Phase 12 — browser hardening | -| 15 | DTLS channel binding | Browser saves raw answer SDP before `setRemoteDescription` (Chrome may drop sha-256 fingerprint). GEK-HMAC uses `_rawAnswerSdp` for fingerprint extraction. | Phase 12 — handshake fix | - ---- - -## Changes from v2 - -The following items are **mandatory corrections** driven by POC findings (spikes 1–6). They supersede the corresponding text in v2. - -| # | Category | What changed | Source | -|---|---|---|---| -| 1 | JWT | `jti` (UUID4) is now **required** in every access token — prevents replay and enables individual revocation. Without it, two tokens issued in the same second are bit-for-bit identical (Ed25519 is deterministic). | Spike 3 | -| 2 | Argon2id | Parameters updated: `iterations=4`, `memory_cost=262144` (256 MB). Previous params (iterations=3, 64 MB) gave 78 ms — too fast. Target is 500 ms on a home server. CLI calibration command added. | Spike 1 | -| 3 | NAT traversal | Order corrected: IPv6 → **STUN/hole-punching** → UPnP → TURN relay. UPnP moved to step 3 (disabled on tested SFR box). STUN is now priority 2, not UPnP. | Spike 4 | -| 4 | Transport | TCP + TLS 1.3 is now the **v1 implementation**. QUIC is the v2 target. The v2 architecture doc had this reversed (QUIC primary, TCP fallback). A `Transport` abstraction layer ensures the switch requires no protocol-layer changes. | Spike 5 | -| 5 | GEK wrapping | Exact protocol confirmed: ephemeral X25519 + `HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1")` + `ChaCha20-Poly1305(aad=pk_recipient)`. Hub stores opaque 48-byte blobs. | Spike 6 | -| 6 | Hub API | Four new endpoints validated in Spike 6: `GET /v1/users/{username}/pubkeys`, `POST /v1/groups`, `POST /v1/groups/{group_id}/members/{username}/gek`, `GET /v1/groups/{group_id}/gek`. Full table added as §4.1.5. | Spike 6 | -| 7 | Packages | Repository structure decided: 3 packages (`meshbay-common`, `meshbay-hub`, `meshbay-node`) in a uv workspace monorepo. RPM package names defined. | POC structure | -| 8 | Key persistence | X25519 keypairs **must be persisted** client-side before the first hub contact. Lesson from Spike 6 (`bob_state.json` fix). | Spike 6 | - ---- - -## 1. Project Overview - -MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. - -**Core principles:** -- Data never transits through a central server — only identity and routing do -- End-to-end encryption for all private content (files, indexes, messages) -- The node operator is the legal host and is fully responsible for their content -- The hub is a lightweight registrar, not a content host or indexer -- Open source, self-hostable at every level - -**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) - ---- - -## 2. Terminology - -| Term | Role | -|---|---| -| **Mesh Hub** | Identity authority and group registry server | -| **Mesh Node** | Local program on the host user's machine | -| **Mesh Client** | Web browser or Android app (end user) | -| **Mesh Relay** | Community-operated TURN fallback relay [future] | -| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | -| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | -| **GEK** | Group Encryption Key — symmetric key for private group content | -| **Mesh Directory** | Public registry of groups (hub level) | -| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | - ---- - -## 3. Protocol Versioning - -All protocols (MNP, MHP, hub REST API) carry explicit version information. - -**Format:** `MAJOR.MINOR` -- MAJOR bump: breaking change, backward incompatible -- MINOR bump: backward-compatible addition - -**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. - -**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). - -**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. - ---- - -## 4. System Components - -### 4.1 Mesh Hub - -A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. - -**What the hub stores:** -- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user` (Ed25519 + X25519), hub ID, status, creation timestamp -- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) -- Mandatory connection logs (see §4.1.2) -- Revocation lists (users and groups) -- Registered peer hubs (explicit allowlist — no auto-discovery) - -**What the hub never stores:** -- File content or metadata -- Private group indexes -- Message content -- Node current IP (handled by ephemeral signaling — see §4.1.3) - -#### 4.1.1 Account Data - -Email is kept in full (not hashed) to support: -- Account recovery (password reset) -- Legal notifications -- Abuse contact - -Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). - -Email and phone are stored encrypted at rest in the database, using a server-side key derived from the hub's configuration secret (not the database). **[NOT YET IMPLEMENTED — currently stored in plaintext. Tracked as open question #10.]** - -#### 4.1.2 Mandatory IP Logging (Legal Compliance) - -Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: - -| Event | Retention | -|---|---| -| Account creation | 1 year minimum | -| Login (success and failure) | 1 year minimum | -| Group creation | 1 year minimum | -| Group join / leave | 1 year minimum | -| Group deletion | 1 year minimum | -| Revocation actions | 1 year minimum | - -Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. - -#### 4.1.3 Signaling Service - -NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. - -**Hub interaction summary:** - -| Event | Hub crypto load | Frequency | -|---|---|---| -| Account creation | Argon2 hash, store PK | Once | -| Login | Verify password, issue JWT (Ed25519 sign) | Per session | -| Group creation | Register metadata | Once per group | -| Member add/remove | Store/remove GEK bundle | On admin action | -| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | -| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | -| Public search | Delegate to nodes, 60s in-memory cache | On demand | -| MHP federation sync | Exchange Mesh Directory | Background, periodic | -| Revocation | Ed25519-sign revocation token | Rare | - -**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip). Confirmed at 884 µs in Spike 3.** - -#### 4.1.4 JWT Strategy - -Two tokens issued at login: - -**Access token** (JWT, signed Ed25519): -- Validity: 1 hour -- Payload: `jti` (UUID4, **mandatory** — unique per token, enables individual revocation and prevents replay), `user_id`, `PK_user`, `PK_user_x25519`, `hub_id`, `issued_at`, `expires_at`, `groups` (list of group_ids the user is a member of — hub-signed membership claim) -- The `groups` claim is **mandatory** for node-side authorization: the node checks that the requested group_id appears in the JWT before serving any content. Without this claim, any authenticated user could access any group on the node. -- Presented to nodes for authentication and group access verification -- Verified locally by nodes using the hub's known public key — no hub roundtrip -- Compromise window: 1 hour maximum - -> **Why `jti` is mandatory:** Ed25519 signing is deterministic. Two tokens with identical payloads issued within the same second produce the same byte sequence. Without a `jti`, they are indistinguishable — a captured token is replayable forever within its validity window, and individual revocation is impossible. The `jti` also provides the revocation handle: hub stores `jti` of invalidated tokens in a server-side denylist. -> -> This bug was found and fixed during Spike 3. - -**Refresh token** (opaque, random 256-bit): -- Validity: 30–90 days [TBD exact duration] -- Stored securely on client only -- Used exclusively with the hub to obtain a new access token -- Revocable immediately by the hub (invalidates all future refreshes for this token) -- Stored server-side as a hashed value - -**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. For immediate revocation of an active access token: hub adds its `jti` to the token denylist; nodes that cache hub public key will periodically fetch the denylist. - -**Tech stack:** -- Language: Python -- Framework: FastAPI + Uvicorn -- Database: PostgreSQL + SQLAlchemy + Alembic -- Deployment: Apache reverse proxy (ProxyPass + SSL termination) -- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) -- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) - -#### 4.1.5 Hub API Reference - -Complete table of validated and planned hub REST API endpoints. Endpoints marked ✓ were validated in the POC; endpoints marked [TBD] are designed but not yet implemented. - -**Hub metadata:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/hub/info` | None | Hub metadata: hub_id, versions, counters | ✓ Spike 2 | -| GET | `/v1/hub/pubkey` | None | Hub Ed25519 public key (PEM) | ✓ Spike 2 | - -**User management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/users/register` | None | Create user account (username, email, password, pk_ed25519, pk_x25519) | ✓ Spike 2 | -| POST | `/v1/users/login` | None | Authenticate; returns access token + refresh token | ✓ Spike 2 | -| POST | `/v1/users/token/refresh` | Refresh token | Issue new access token | ✓ Spike 2 | -| GET | `/v1/users/{username}/pubkeys` | Access token | Fetch `pk_ed25519` and `pk_x25519` for a user (used for GEK wrapping) | ✓ Spike 6 | - -**Node management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/nodes/announce` | Access token | Register node with endpoint_hint; returns node_id | ✓ Spike 2 | -| GET | `/v1/nodes/{node_id}` | Access token | Retrieve node record (pk_node, endpoint_hint) | ✓ Spike 2 | - -**Group management:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups` | Access token | Create group (name, visibility, join_policy, pk_group) | ✓ Spike 6 | -| GET | `/v1/groups` | None / Access token | List/search public groups; private groups require membership | [TBD] | -| GET | `/v1/groups/{group_id}` | None / Access token | Group metadata | [TBD] | -| DELETE | `/v1/groups/{group_id}` | Access token (admin) | Revoke group | [TBD] | - -**GEK distribution (private groups):** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/groups/{group_id}/members/{username}/gek` | Access token (admin) | Upload opaque 48-byte GEK bundle for a member | ✓ Spike 6 | -| GET | `/v1/groups/{group_id}/gek` | Access token (member) | Retrieve caller's GEK bundle | ✓ Spike 6 | - -**Revocation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| POST | `/v1/admin/revoke` | Access token (admin) | Revoke a user or group | ✓ Phase 8 | -| GET | `/v1/revoke/denylist` | None | Current `jti` denylist for active access tokens | [TBD] | - -**Admin / moderation:** - -| Method | Path | Auth | Description | Status | -|---|---|---|---|---| -| GET | `/v1/users/me` | Access token | Current user info (id, username, role, status) | ✓ Phase 10 | -| GET | `/v1/admin/stats` | Moderator+ | Hub stats (user/group/node counts, online nodes) | ✓ Phase 10 | -| GET | `/v1/admin/users` | Moderator+ | List users (paginated, searchable by username) | ✓ Phase 10 | -| GET | `/v1/admin/users/{user_id}` | Moderator+ | User detail (email, role, status, group count) | ✓ Phase 10 | -| PATCH | `/v1/admin/users/{user_id}` | Moderator+ | Update user role or status | ✓ Phase 10 | -| GET | `/v1/admin/groups` | Moderator+ | List all groups with member count | ✓ Phase 10 | -| PATCH | `/v1/admin/groups/{group_id}` | Moderator+ | Update group status | ✓ Phase 10 | -| GET | `/v1/admin/logs` | Moderator+ | IP audit logs (filterable by event, user_id) | ✓ Phase 10 | -| GET | `/v1/admin/blocklist` | Admin | List blocked content hashes | ✓ Phase 8 | -| POST | `/v1/admin/blocklist` | Admin | Manually block a content hash | ✓ Phase 8 | -| DELETE | `/v1/admin/blocklist/{hash}` | Admin | Unblock a content hash | ✓ Phase 8 | -| GET | `/v1/notifications` | Access token | List notifications (unread_only, paginated) | ✓ Phase 10 | -| POST | `/v1/notifications/{id}/read` | Access token | Mark notification as read | ✓ Phase 10 | -| POST | `/v1/notifications/read-all` | Access token | Mark all notifications as read | ✓ Phase 10 | -| GET | `/v1/groups?q=` | None | Search public groups by name (ilike) | ✓ Phase 10 | -| GET | `/v1/hub/version` | None | Client version check (hub, MNP, MHP) | ✓ Phase 10 | -| GET | `/v1/groups/{id}/members` | Access token | List group members (requires membership) | ✓ Phase 10b | -| POST | `/v1/groups/{id}/join` | Access token | Self-join open group | ✓ Phase 10b | - -### 4.2 Mesh Node - -A local program running on the host user's machine. The node is the actual host of all content. - -**Responsibilities:** -- Watch and index shared directories (Mesh Group Index) — one directory per group -- Serve files, video streams, and group chat to members -- Manage all cryptographic keys locally (encrypted keystore) -- Handle P2P connections and NAT traversal (STUN + QUIC hole punching) -- Run the MNP protocol (QUIC v2, TCP+TLS v1) -- Host the Python extension module sandbox -- Serve the local web UI (localhost:18000) - -**Multi-group architecture (decided Phase 7):** -A node exposes **one QUIC port** for all groups it hosts. Groups are not isolated -by port — the MNP handshake identifies the target group via the `group_id` claim -in the client JWT. The server routes each connection to the appropriate -DirectoryIndexer and GEK after JWT verification. -Rationale: one NAT hole to maintain, one port to forward manually if needed. - -**Authorization invariant:** the node MUST verify that the JWT's `groups` claim -contains the requested group_id before serving any content. Without this check, -any authenticated user could access any group on the node. This is enforced at -the MNP handshake layer, not the transport layer. - -#### 4.2.x Node Sovereignty — Content Authorization Model - -The node operator is the **sole authority** over content stored on their machine. -No external actor — including the hub admin — can modify, delete, or control -files on a node they do not operate. This is a non-negotiable design invariant, -enforced by **cryptography**, not just policy. - -**Two trust domains, strictly separated:** - -| Domain | Authority | Scope | -|---|---|---| -| **Hub** | Hub admin / moderator | User accounts, group registry, group membership, GEK distribution, moderation (suspend user/group at hub level) | -| **Node** | Node operator | Files on disk, file deletion, upload acceptance, chat storage, who can do what with node content | - -The hub certifies **identity** (JWT) and **group membership** (`groups` claim). -The node decides **authorization for content operations** based on that identity. -These two concerns must never be conflated. - -##### Cryptographic enforcement — two defense layers - -A malicious hub admin controls the JWT signing key and could forge JWTs to -impersonate any user, including the node operator. Policy-only checks (comparing -`user_id` to `node_user_id`) are insufficient because the hub controls the -identity layer. Two cryptographic mechanisms make this impossible: - -**Layer 1 — GEK proof in handshake (membership verification):** - -After JWT verification, the node challenges the connecting user to prove they -possess the Group Encryption Key (GEK). The hub never has the GEK — it only -stores opaque ECIES-wrapped bundles. Without the GEK, a hub admin who forges -a JWT still cannot access any group content. - -``` -Client → Node: handshake { token, group_id } -Node: verify JWT, verify group_id in claims - nonce = random(32) -Node → Client: handshake_challenge { nonce: base64(nonce) } -Client: proof = HMAC-SHA256(GEK, nonce) -Client → Node: handshake_response { proof: base64(proof) } -Node: verify HMAC — if wrong, reject connection -Node → Client: handshake_ack { is_node_admin, node_pk, v } -``` - -This blocks: content reading, index reading, chat reading, file upload, chat -injection — ALL operations require passing the GEK proof first. - -**Layer 2 — Ed25519 challenge-response for admin operations:** - -The node operator's Ed25519 public key is pinned locally in `node.toml` -(auto-pinned from keystore on first startup). Destructive operations (file -deletion) require the user to sign a random challenge with their Ed25519 -private key. The hub cannot forge this signature. - -``` -Client → Node: file_delete { file_id } -Node: (if uploader → allow immediately) - (else) challenge = random(32) -Node → Client: admin_challenge { challenge: base64(challenge), file_id } -Client: signature = Ed25519.sign(sk_ed, challenge) -Client → Node: admin_response { signature: base64(signature), file_id } -Node: verify(admin_pk_ed25519, signature, challenge) - if valid → delete file -``` - -**Node configuration — admin key pinning:** - -```toml -# node.toml -admin_pk_ed25519 = "base64-encoded-32-bytes-raw-Ed25519-public-key" -``` - -Auto-pinned from the node operator's keystore on first startup. The daemon -logs: "Admin Ed25519 key pinned for node sovereignty". - -**GEK distribution — browser flow (node no longer serves GEK):** - -The node NEVER serves the GEK in plaintext. Browser clients obtain the GEK -from their hub-stored encrypted bundle: - -1. `GET /v1/groups/{id}/gek` → encrypted ECIES bundle (AES-256-GCM variant) -2. Browser unwraps with its X25519 private key (from keypair bundle) -3. Browser uses raw GEK bytes for the handshake HMAC proof -4. Browser imports GEK as HKDF key for chunk decryption - -This eliminates the `gek_req`/`gek_resp` MNP messages from the protocol. - -**Authorization rules for destructive file operations (enforced by the node):** - -| Action | Who can do it | Enforcement point | -|---|---|---| -| Delete a file | Node operator (Ed25519 challenge-response) OR the user who uploaded it | Node (`_do_file_delete`) | -| Delete any file | Node operator only (Ed25519 challenge-response) | Node (`_do_file_delete`) | - -Default posture: **deny.** If the admin key is not pinned, all admin operations -are refused. If the GEK proof fails, the connection is refused entirely. - -**Protocol enforcement — MNP handshake_ack:** - -The handshake_ack message carries `is_node_admin: bool` — the node tells the -client whether the authenticated user is the node operator. Clients MUST use -this node-reported flag (not the hub's `group.admin_id`) to decide whether -to show destructive operations like file deletion. - -``` -handshake_ack: - v: "0.1" - node_pk: "<base64>" - is_node_admin: true | false # node-side authorization, NOT hub-side -``` - -**Index entry — uploader tracking:** - -Each `IndexEntry` carries an `uploader_id` field (user_id of who uploaded the -file, or null for files that pre-existed on disk). This enables the "uploader -can delete their own files" rule without granting node-admin privileges. - -**What the hub admin CANNOT do on a node they don't operate:** -- Delete files (requires Ed25519 key pinned on node — hub can't forge) -- Read files (requires GEK — hub never has it) -- Read index / chat (requires GEK proof in handshake) -- Upload files (requires GEK proof in handshake) -- Impersonate the node operator (JWT forgery blocked by Ed25519 challenge) - -**What the hub admin CAN do (hub-level only):** -- Suspend a user account (blocks JWT issuance → user loses access everywhere) -- Suspend a group (blocks signaling → no new P2P connections to nodes for that group) -- These are hub-level actions that don't touch node content - -**Remaining trust assumptions:** -- The hub serves the SPA code to browsers (a malicious hub could inject JS — fundamentally unsolvable in browser; native client or browser extension required for full integrity) -- The hub relays WebRTC signaling — ✅ MITIGATED: DTLS channel binding in GEK-HMAC proof (`HMAC(GEK, nonce || offer_fp || answer_fp)`) detects fingerprint substitution (MitM) -- The hub receives raw password at login — ✅ MITIGATED: password split (auth_key ≠ bundle_key, independent PBKDF2 derivations). Hub receives auth_key only, cannot derive bundle_key to decrypt keypair bundle. Legacy accounts migrated on first login. -- The hub controls public key distribution — can substitute keys during invite to intercept GEK. Fix: out-of-band key verification (safety numbers) — Phase 12 - -> **Design lesson (2026-08-12):** The initial implementation conflated hub -> `group.admin_id` (who created the group on the hub) with node operator -> authority (who runs the machine). The SPA used the hub's `is_admin` flag -> to show file deletion controls, and the node's delete handler used a -> fail-open check (`if node_user_id and ...` — allowed everyone when -> `node_user_id` was not set). Both violated node sovereignty. Fixed by: -> (1) deny-by-default on the node, (2) `is_node_admin` in handshake_ack, -> (3) `uploader_id` tracking in the index, (4) SPA uses node-reported -> permissions only. Then hardened with cryptographic enforcement: -> (5) GEK-HMAC proof in handshake (blocks forged-JWT access), -> (6) Ed25519 challenge-response for admin ops (blocks identity impersonation), -> (7) removal of `gek_req` endpoint (node never serves GEK in plaintext), -> (8) DTLS channel binding in GEK-HMAC proof to detect WebRTC signaling MitM, -> (9) chat `sender_id` fixed to authenticated identity (prevents impersonation), -> (10) Ed25519 challenge for ALL file deletions — uploaders verified by stored pk, not JWT sub, -> (11) password split — hub receives PBKDF2 auth_key, never raw password (cannot derive bundle_key). - -**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. - -#### 4.2.1 Keystore and Unlock - -Private keys (user identity Ed25519, user exchange X25519, group identity Ed25519, GEK copies) are stored in a local encrypted keystore file. - -**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id. - -**Argon2id parameters (production):** -- `iterations = 4` -- `memory_cost = 262144` (256 MB) -- `parallelism = 1` (or match CPU count — tune to target hardware) -- Target derivation time: ~500 ms on a home server - -> **Why these parameters:** Spike 1 measured iterations=3, memory=64 MB at 78 ms — far too fast. At 78 ms an attacker can attempt millions of guesses per second-equivalent with a GPU cluster. The target of 500 ms on a home server limits offline dictionary attacks to a tractable rate while remaining acceptable for a node that unlocks once at startup. - -**CLI calibration:** -``` -meshbay-node --calibrate-argon2 -``` -This command iterates through parameter combinations and reports the derivation time on the current hardware. The operator selects parameters meeting the 500 ms target and stores them in `~/.config/meshbay/node.toml`. Recommended starting point: `iterations=4, memory_cost=262144`. - -**Key persistence requirement:** All keypairs (Ed25519 + X25519) **must be written to the keystore before the first hub contact.** If keypairs are generated at registration time but not persisted before the hub call, subsequent runs will regenerate different keypairs, making all stored GEK bundles on the hub undecryptable. This was identified as a real failure mode in Spike 6 (`bob_state.json` fix). - -**Three unlock modes:** - -| Mode | How it works | Security level | -|---|---|---| -| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | -| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | -| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | - -Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). - -#### 4.2.2 Hardware Sizing - -The main constraint is **upload bandwidth**, not CPU or RAM. - -| Scenario | Simultaneous users | Upload needed | CPU | RAM | -|---|---|---|---|---| -| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | -| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | -| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | -| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | -| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | - -Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. - -Crypto overhead is confirmed negligible: Spike 5 measured full encrypt+sign and verify+decrypt at under 10 ms for a 1 MB chunk. Network latency dominates. - -**Tech stack:** -- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. -- Transport abstraction layer: `Transport` interface decouples TCP+TLS 1.3 (v1) from QUIC (v2). Application protocol is identical across both transports. -- v1 transport: **TCP + TLS 1.3** (`asyncio` + `ssl` module, standard library) -- v2 transport (future): **QUIC** (`aioquic`, Cloudflare-maintained) -- ICE/STUN: `aioice` (already a dependency) -- WebRTC: `aiortc` (browser P2P transport — Phase 9) -- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) -- Serialization: `msgpack` -- Compression: `zstandard` (zstd) -- File watching: `watchdog` -- Local DB: SQLite -- Local web UI: served by node on `localhost:18000` - -### 4.3 Mesh Client - -Web browser or Android app. Consumes content from nodes; manages account via hub. -The hub is never in the data path — clients connect E2E to nodes for all content. - -**Hub-side operations (HTTPS, lightweight):** -- Account creation, login, token refresh -- Public group search and discovery -- Group membership management, GEK bundle retrieval -- WebRTC signaling relay (SDP/ICE — <1 KB per connection, stateless) -- Notification metadata (invitations, new content indicators) - -**Node-side operations (direct P2P via QUIC or WebRTC DataChannel):** -- File browsing via Mesh Group Index -- File download (chunked, E2E encrypted) -- Video streaming (HLS segments via DataChannel or QUIC stream) -- Group chat (Sender Keys encrypted, stored on node) -- File/photo/video upload (client → node push) - -#### 4.3.1 Web Browser Client - -**Transport:** WebRTC DataChannel with ICE/STUN for NAT traversal. -WebTransport (HTTP/3) is not suitable because browsers cannot choose their UDP -source port — Port-Restricted Cone NAT (confirmed on SFR residential) requires -the client to connect from the exact port the node probed. WebRTC's ICE handles -this automatically via simultaneous STUN binding requests. - -**UI:** Preact SPA (~3 KB gzipped) served by the hub. -- Dark/light theme (CSS `prefers-color-scheme` + user toggle in localStorage) -- Responsive design (sidebar → hamburger menu on mobile) -- i18n: JSON translation files, English default -- Build: esbuild for minification (single binary, no npm dependency) -- Crypto: SubtleCrypto (AES-GCM) for E2E decryption in browser - -**Layout:** -- Left sidebar: group list (ordered by usage — private groups first), navigation -- Top bar: logo ("MeshBay") left, user menu right (settings, profile, language, logout) -- Main content area: file explorer, chat view, or settings depending on context - -**Client modes:** -- Explorer: file/folder browser for group content (read-only browse, download, stream) -- Chat/forum: per-group discussion thread with photo/video posting -- Settings: general, per-group, notifications, privacy, theme, language - -**Local storage:** -- IndexedDB: cached group indexes for instant local search (~50–100 MB quota) -- localStorage: theme preference, language, session state -- `keypair_bundle`: encrypted keypair retrieved from hub, decrypted locally with password - -**File search:** entirely client-side on cached indexes. No hub involvement. -Private group indexes are GEK-encrypted — stored opaque on the hub, decrypted -by the client locally. Search runs against the decrypted index in IndexedDB. - -#### 4.3.2 Android Client - -**Transport:** QUIC with `punch_nat()` — same as desktop native clients. -Android has full UDP access; no WebRTC needed. Uses `quiche` (Cloudflare, Rust -via JNI) for QUIC transport. - -**Stack:** Kotlin + Jetpack Compose. Bouncy Castle JVM for crypto. - -**Capabilities:** same as web browser (browse, download, stream, chat, upload). -Additional: contact list integration (Android Contacts API, permission-gated). -Account creation from app. No node functionality on mobile (client-only). - -**Cross-device compatibility:** the `keypair_bundle` (encrypted, stored on hub) -enables seamless switching between web and Android with the same credentials. -Notification state and read markers sync via hub (small encrypted blob per user). - -**Out of scope:** Mac/iPhone support. Node on mobile. - -### 4.4 Mesh Relay - -**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (symmetric NAT behind CGNAT, approximately 15–20% of connections in the worst case). Traffic is always E2E encrypted — the relay sees only opaque ciphertext. - -Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. - -### 4.5 Package Structure and Repository - -**Repository layout:** monorepo managed with [uv workspaces](https://docs.astral.sh/uv/concepts/workspaces/). - -``` -meshbay/ -├── packages/ -│ ├── meshbay-common/ # Shared crypto, serialization, protocol types -│ ├── meshbay-hub/ # Hub server (FastAPI + Uvicorn) -│ └── meshbay-node/ # Node daemon + local web UI -├── poc/ # POC and spikes — reference implementation -│ ├── spike1_crypto/ -│ ├── spike2_hub/ -│ ├── spike3_node_reg/ -│ ├── spike4_nat/ -│ ├── spike5_transfer/ -│ ├── spike6_gek/ -│ └── spike-results.md -├── docs/ -│ └── meshbay-draft-v3.md -└── pyproject.toml # Workspace root -``` - -**Three packages:** - -| Package | RPM name | Contents | -|---|---|---| -| `meshbay-common` | `python3-meshbay-common` | Crypto primitives (Ed25519, X25519, ChaCha20, Argon2, HKDF), msgpack schemas, protocol constants, MNP message types | -| `meshbay-hub` | `python3-meshbay-hub` | FastAPI hub application, database models (SQLAlchemy), Alembic migrations, JWT issuance, GEK bundle storage | -| `meshbay-node` | `python3-meshbay-node` | Node daemon, keystore, file watcher, TCP+TLS transport, local web UI, extension module sandbox | - -**`meshbay-hub` and `meshbay-node` both depend on `meshbay-common`.** There is no runtime dependency between hub and node packages. - -**POC directory as reference implementation:** The `poc/` directory contains the working code from spikes 1–6. It is not production code and not packaged, but serves as the canonical reference for: -- Exact crypto parameter choices (Spike 1) -- GEK wrapping/unwrapping implementation (Spike 6) -- Hub API skeleton (Spike 2) -- NAT detection and STUN interaction (Spike 4) -- TCP file transfer pipeline (Spike 5) - -Developers implementing production features should read the corresponding spike before writing production code. - ---- - -## 5. Group Model - -Groups are the core organizational unit. - -| Parameter | Options | -|---|---| -| Visibility | Public / Private | -| Join policy | Open / On request / By invitation only | -| Node admin | The hosting node operator — sovereign over content, sole delete authority (see §4.2.x) | -| Hub group creator | The user who registered the group on the hub — manages membership and GEK distribution | - -A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). - -A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. - -**Group addressing:** -``` -meshbay.org/u/username/groupname — public group via hub -meshbay.org/g/groupname — public group (shorthand) -group://<PK_group_fingerprint>@<node_addr> — hub-less direct access -``` -`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). - ---- - -## 6. Cryptographic Architecture - -### 6.1 Key Hierarchy - -``` -User Identity Key Ed25519 Signing, authentication -User Exchange Key X25519 Key agreement (GEK wrapping, session ECDH) -Group Identity Key Ed25519 Group metadata signing (held by admin node) -Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) -Session Keys X25519/HKDF Perfect forward secrecy per P2P connection -``` - -All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. - -Both `PK_ed25519` and `PK_x25519` are registered with the hub at account creation. The hub exposes them via `GET /v1/users/{username}/pubkeys` so that group admins can wrap GEK bundles for members without any direct contact between nodes. - -### 6.1.1 Key Generation Strategies - -Three strategies, depending on client type: - -**A — CLI / native node (Argon2id derivation)** -Keys are derived deterministically from `username + password`: -``` -salt = SHA-256("meshbay:v1:" + username) -seed = Argon2id(password, salt, length=64) -sk_ed25519 = Ed25519.from_private_bytes(seed[:32]) -sk_x25519 = X25519.from_private_bytes(seed[32:]) -``` -Same credentials → same keys on any machine. Password recovery = key recovery. -Implemented in `meshbay_common/keyderive.py::derive_keys_from_password()`. - -**B — Web browser (random keypairs + encrypted bundle)** -Browser generates random keypairs via WebCrypto `generateKey()`, encrypts them -with a PBKDF2-SHA512 derived key, and uploads the encrypted bundle to the hub -alongside the public keys. On subsequent logins, the hub returns the bundle -and the browser decrypts it locally with the password. - -The hub stores `keypair_bundle` (AES-256-GCM ciphertext) — opaque, cannot decrypt it. -Implemented in `static/keyderive.js`. Python side in `keyderive.py::encrypt_keypair_bundle()`. - -**C — Native node with keystore file** -Random keypairs generated once, stored in the Argon2id-encrypted keystore file -(`~/.config/meshbay/keystore.enc`). Standard operating mode for `meshbay-node`. - -**Algorithm mismatch note:** strategies A and B use different KDFs (Argon2id vs PBKDF2). -A user who registered via CLI (A) and later tries to recover via web (B) with the same -password will get different keypairs. This is by design: users pick one registration path. -Cross-path recovery requires the admin to issue new GEK bundles. - -### 6.2 GEK Management - -**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). - -**GEK wrapping protocol (ECIES-like, confirmed in Spike 6):** - -``` -Admin side (wrap_gek): - sk_eph, pk_eph = X25519.generate() # fresh ephemeral keypair per bundle - shared = X25519(sk_eph, pk_recipient) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - nonce = random_bytes(12) - wrapped = ChaCha20-Poly1305(wrap_key).encrypt( - nonce, gek, aad=pk_recipient) # aad binds bundle to recipient - bundle = pk_eph || nonce || wrapped # 32 + 12 + 32+16 = 92 bytes on wire - # hub stores as opaque 48-byte blob - # (without pk_eph in compact form — see note) - -Member side (unwrap_gek): - shared = X25519(sk_recipient, pk_eph) - wrap_key = HKDF(shared, salt=pk_eph, - info="meshbay:gek_wrap:v1", - length=32) - gek = ChaCha20-Poly1305(wrap_key).decrypt( - nonce, wrapped, aad=pk_recipient) -``` - -> **Hub-stored blob size:** the hub stores the opaque bundle. Spike 6 confirmed the hub stores 48-byte blobs (nonce=12 + ciphertext=20 + tag=16 in the compact wire format used in the spike — `pk_eph` is stored separately in the bundle record). Production schema: hub bundle record = `{ pk_eph (32B), nonce (12B), ciphertext (32B), tag (16B) }` = 92 bytes total per member per group, stored as a single column. - -**Security properties confirmed in Spike 6:** -- Hub never sees the GEK in cleartext -- Ephemeral keypair is unique per bundle — same GEK and same recipient produce different ciphertext across calls -- AAD (`pk_recipient`) binds the bundle to its intended recipient — reuse for a different member is detected and rejected -- Wrong private key → AEAD authentication tag failure → immediate rejection - -**Group creation:** -1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) -2. GEK wrapped for each initial member via the protocol above -3. Wrapped bundles uploaded to hub via `POST /v1/groups/{group_id}/members/{username}/gek` -4. Members retrieve their bundle via `GET /v1/groups/{group_id}/gek` - -**Member addition:** -- Admin fetches new member's `pk_x25519` from hub -- Wraps GEK for them and uploads bundle - -**Member revocation:** -- Admin node generates new GEK -- Re-encrypts for all remaining members, uploads new bundles -- New content encrypted with new GEK from this point -- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) - -**Key persistence requirement:** before uploading a GEK bundle, the recipient's keypairs must already be registered on the hub and persisted locally. If a user registers, generates keypairs, but does not persist them before the first hub contact, subsequent sessions will regenerate different keypairs and all bundles will be undecryptable. The node initializes and persists all keypairs to the keystore before any hub API call. - -### 6.3 On-the-Fly Encryption for File Transfer - -Files are stored in plaintext on the host's disk. The node encrypts at read time. - -``` -Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → TCP+TLS 1.3 session → Client → TLS decrypt → GEK decrypt → plaintext -``` - -(In v2 transport: replace TCP+TLS 1.3 with QUIC — application pipeline is identical.) - -**Chunking:** -- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) -- Per-chunk key derivation: `chunk_key = HKDF(GEK, salt=None, info="file:" || blake3(file) || ":chunk:" || index)` — salt is omitted because the GEK is a CSPRNG output (already uniform); the file/chunk context goes in `info` for domain separation, which is the correct HKDF usage per RFC 5869 -- Each chunk independently decryptable → enables VOD seeking -- Compress before encrypt (compression is ineffective on ciphertext) - -**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. - -**Encryption performance (Spike 5, 1 MB chunk, TCP, Fedora → OVH VPS):** - -| Operation | Time | -|---|---| -| Encrypt + sign (node side) | 3.2 ms | -| Verify + decrypt (client side) | 3.9 ms | -| Total crypto overhead (1 MB) | < 10 ms | -| Network transfer | 99–234 ms (network-limited) | - -Encryption is not the bottleneck. Network latency and bandwidth dominate. - -**Pipeline optimization:** -- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops -- ChaCha20-Poly1305: ~1750 MB/s (Spike 1); AES-256-GCM: >2 GB/s with AES-NI -- asyncio pipeline (read → compress → encrypt → send) without loading full files into memory -- GEK-derived chunk keys computed in batch at transfer start, not per-chunk - -### 6.4 Transport Security - -**Implementation phases:** - -| Phase | Transport | Status | Notes | -|---|---|---|---| -| v1 | TCP + TLS 1.3 | Current implementation target | Standard library (`asyncio` + `ssl`), well-understood, works everywhere | -| v2 | QUIC (TLS 1.3 integrated, UDP, multiplexed streams) | Future upgrade | `aioquic`, no protocol changes needed — only transport layer | - -The `Transport` abstraction interface in `meshbay-node` decouples the application protocol from the underlying transport. Switching from TCP+TLS to QUIC requires implementing a new `Transport` backend with no changes to MNP message handling, GEK pipeline, or NAT traversal logic. - -**Per-connection session keys:** X25519 ECDH + HKDF, independent of the GEK layer. Provides forward secrecy per connection regardless of transport. - -**Rationale for TCP+TLS 1.3 first:** UDP hole-punching (required for QUIC in NAT scenarios) adds complexity in the early implementation. TCP outbound from behind NAT (as used in Spike 5) works without any NAT coordination. TLS 1.3 provides equivalent confidentiality guarantees to QUIC's integrated TLS. QUIC's benefits (0-RTT, multiplexing, no head-of-line blocking) are meaningful for performance but not for correctness — they belong in v2 once the application protocol is stable. - -### 6.5 TCP+TLS 1.3 Transport Implementation (v1) - -**Connection model:** -- Node listens on a configurable TCP port (default: 18000, same as local web UI port — separate socket) -- Clients connect outbound; nodes behind NAT connect outbound to other nodes via hole-punching signaling (see §7.1) -- TLS 1.3 mandatory; TLS 1.2 rejected -- Node presents a self-signed Ed25519 certificate pinned to its `PK_node` (registered on hub) -- Client validates certificate against `PK_node` retrieved from hub — not against a CA chain - -**Handshake sequence:** -``` -Client → Node: TCP SYN -Node → Client: TLS ServerHello (self-signed cert, PK_node) -Client: verify cert against hub-fetched PK_node -Client → Node: TLS ClientFinished -Node → Client: MNP handshake request (version negotiation) -Client → Node: MNP handshake response (JWT access token, version) -Node: verify JWT offline (Ed25519, hub public key) -Node → Client: session established -``` - -**Message framing over TCP:** -- Length-prefixed frames: `[4-byte big-endian length][msgpack payload]` -- Maximum frame size: 2 MB (prevents memory exhaustion; larger transfers use chunked `file_chunk` messages) -- Each frame carries the MNP `version` field in its header - -**QUIC migration path (v2):** -- Replace TCP length-framing with QUIC streams (one stream per logical exchange) -- MNP handshake maps 1:1 to a QUIC handshake stream -- File transfer maps to a dedicated QUIC stream per file (multiplexed, no head-of-line blocking) -- Chat messages map to a persistent QUIC stream -- No changes to JWT verification, GEK decryption, or Index sync logic - -**Port allocation:** -- `18000/tcp` — local web UI (loopback only, not exposed externally) -- `18001/tcp` — MNP P2P listener (exposed externally, TLS required) -- Configurable via `~/.config/meshbay/node.toml` - -### 6.6 Chat Encryption and Model - -Group chat is a **core feature** (not an extension module). - -**Model (decided):** between a forum and Signal. -- **Persistent:** messages stored on the node (not ephemeral like Signal by default) -- **Structured:** optional threads/topics for longer discussions, flat stream for quick messages -- **Scope:** per group (not per user pair) -- **Attachments:** files and images, shared like regular group files -- **Push/pull:** connected members get real-time push (WebSocket); offline members pull history on reconnect -- **Retention:** managed by the group admin (no automatic expiry) - -**Encryption — Sender Keys protocol (decided in first security review, 2026-08-10):** - -The Double Ratchet (implemented in `meshbay_common.ratchet`) is a **pairwise** (1:1) protocol. Using a shared ratchet state for N group members would cause chain key desynchronization and nonce/key reuse — a catastrophic AEAD failure. The architecture uses **Sender Keys** instead (same approach as Signal Groups): - -- Each group member generates a **sender key** (random symmetric chain key + signing keypair) -- On joining a group, the new member's sender key is distributed to all existing members via pairwise channels (GEK-wrapped or direct) -- Each existing member sends their current sender key to the new member -- Messages are encrypted with the sender's chain key (symmetric ratchet, one direction) -- Forward secrecy at **member rotation** granularity: when a member is removed, all remaining members rotate their sender keys -- O(N) state per member (one chain per group member), not O(N^2) -- The existing Double Ratchet implementation is kept for future 1:1 direct messaging - -Attachment files: encrypted with GEK-derived key (same as file chunks), hash referenced in the message. - -> **Why not MLS (RFC 9420)?** MLS provides O(log N) message overhead and per-message forward secrecy via tree-based ratcheting. It is the superior long-term choice, but its complexity is not justified for v1 group sizes (< 50 members). Sender Keys is proven at scale (Signal, WhatsApp) and simpler to implement. Migration to MLS is a v2 option if group sizes grow. - ---- - -## 7. Network and Connectivity - -### 7.1 NAT Traversal — Attempt Order - -``` -1. IPv6 available on both sides → direct connection (preferred) -2. STUN / ICE + UDP hole punching → ~80–85% success rate (Cone NAT confirmed in Spike 4) -3. UPnP / NAT-PMP on router → port mapping if available (NOT reliable — disabled on tested SFR box) -4. Mesh Relay (TURN) → [future feature] — symmetric NAT, CGNAT mobile -``` - -> **Correction from v2:** UPnP was listed as step 2 in v2. Spike 4 showed UPnP disabled on the tested SFR residential gateway. STUN + hole-punching (step 2) is more reliable and does not require router cooperation. UPnP is demoted to step 3 as a best-effort supplement, not a dependency. - -**Spike 4 findings:** -- Cone NAT confirmed on SFR residential (same external port 51250 for two different STUN servers) -- UDP hole punching functional: bidirectional echo received from OVH VPS -- STUN servers tested: `stun.cloudflare.com`, `stun.l.google.com` — both returned consistent results -- No CGNAT: stable public IPv4 (81.220.170.32) - -Without step 4 (Mesh Relay), approximately 15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. - -**Signaling punch/connect (Phase 7.2 — reduces handshake from 12.7s to < 200ms):** -Currently the node punches blindly at startup; the client may connect 10-20s later -on an aging NAT entry, causing retransmissions. The coordinated flow uses the -existing hub→node WebSocket (revocation channel): -``` -Client → Hub : POST /v1/nodes/{id}/incoming {peer_ip, peer_port} -Hub → Node (WS) : {type: "client_incoming", peer_ip, peer_port} -Node : punch_nat(peer_ip, peer_port) immediately -Node → Hub (WS) : {type: "punch_ready"} -Hub → Client: 200 OK "connect now" -Client → QUIC: first packet < 2s after probe → fresh NAT entry -``` -demo-v2 finding: SFR residential is **Port-Restricted Cone NAT**. -The probe must come from the QUIC server's own socket (`punch_nat()` via -`_transport.sendto()`). The QUIC client must connect from the same port -as the probe's destination (`local_port=QUIC_PORT`). Handshake time -with proper signaling: < 200ms (vs 12.7s without). - -#### 7.1.1 Browser-Specific NAT Traversal (WebRTC DataChannel) - -Browsers cannot use the QUIC `punch_nat()` mechanism because WebTransport does -not allow the browser to choose its UDP source port. Port-Restricted Cone NAT -requires exact port matching on both IP and port — impossible for browsers. - -**Solution:** WebRTC DataChannel with ICE/STUN. The browser's built-in WebRTC -stack handles NAT traversal automatically: - -1. Browser gathers ICE candidates via STUN (discovers its external IP:port) -2. Node gathers ICE candidates via `aioice` (discovers its external IP:port) -3. Candidates exchanged via hub signaling (WebSocket relay, <1 KB) -4. ICE connectivity checks: both sides send STUN binding requests simultaneously -5. STUN binding requests serve as NAT hole-punching (both directions) -6. ICE finds a valid candidate pair — DataChannel established -7. MNP protocol runs over DataChannel (same messages, same E2E encryption) - -**Signaling flow:** -``` -Browser → Hub (HTTPS) : POST /v1/nodes/{id}/webrtc/offer {sdp, ice_candidates} -Hub → Node (WS) : {type: "webrtc_offer", sdp, ice_candidates, peer_id} -Node (aiortc) : creates PeerConnection, gathers answer candidates -Node → Hub (WS) : {type: "webrtc_answer", sdp, ice_candidates, peer_id} -Hub → Browser (SSE) : answer SDP + ICE candidates -DataChannel : P2P established — hub no longer involved -``` - -ICE is strictly superior to custom `punch_nat()` for browser use: -- No need for the client to pre-announce its port -- Handles both sides behind NAT simultaneously -- Automatic candidate prioritization and fallback -- Battle-tested by billions of daily users (Google Meet, Discord, Zoom) - -**Node dual transport:** the node listens on both: -- QUIC (UDP port 19000) — native clients (desktop, Android) -- WebRTC — browsers (via `aiortc`, separate UDP socket managed by ICE) - -The MNP application protocol is identical on both transports. Same handshake, -same file_request/file_chunk, same chat_message, same E2E encryption. - -### 7.2 MNP — Mesh Node Protocol - -Application-level protocol over TCP+TLS 1.3 (v1) or QUIC (v2). All messages carry a `version` field. The protocol is transport-agnostic — the `Transport` abstraction layer handles framing differences. - -**Defined message types:** - -| Type | Description | -|---|---| -| `handshake` | Key exchange, JWT presentation, version negotiation | -| `handshake_challenge` | Node sends GEK proof nonce (base64, 32 bytes random) — see §4.2.x | -| `handshake_response` | Client proves GEK possession: HMAC-SHA256(GEK, nonce) | -| `handshake_ack` | Node response: version, node public key, `is_node_admin` (node-level authorization) | -| `index_sync` | Encrypted Mesh Group Index delta | -| `file_request` | Request chunk(s) of a file by hash + chunk index | -| `file_chunk` | Chunk data + Ed25519 signature | -| `file_delete` | Client requests file deletion by file_id | -| `file_delete_ack` | Node confirms deletion | -| `file_upload` | Client pushes file chunk to node | -| `file_upload_ack` | Node acknowledges chunk receipt | -| `admin_challenge` | Node sends Ed25519 sign challenge for admin ops (base64, 32 bytes) | -| `admin_response` | Client returns Ed25519 signature over the challenge | -| `stream_request` | Client requests MSE video stream | -| `stream_init` | Node sends codec info + signals stream start | -| `stream_data` | Node sends encrypted fMP4 segment | -| `stream_end` | Node signals end of stream | -| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | -| `chat_message` | Sender Keys encrypted message frame (group chat) | -| `chat_history` | Client requests chat history | -| `chat_history_response` | Node responds with stored messages | -| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | -| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | - -### 7.3 Public Content Delivery — Swarm - -Public files identified by `blake3` hash. Multiple nodes can serve the same file: - -1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub -2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` -3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes -4. Integrity verified by blake3 hash on each chunk - -**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. - ---- - -## 8. Indexes - -### 8.1 Mesh Directory (hub level) - -Public registry of groups, exchanged between hubs via MHP. - -Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. - -Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. - -### 8.2 Mesh Group Index (node level) - -File listing for a group. Generated and maintained by the hosting node. - -Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). - -Entry structure: -```python -{ - "version": 1, - "id": "<blake3_hash>", - "name": "filename.mkv", - "path": "Movies/2024/", - "size": 4294967296, - "type": "video", # video | audio | image | document | archive | other - "duration": 7245, # seconds, for media - "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted - "added_at": 1720000000, - "uploader_id": "<user_id>" # who uploaded this file (null = pre-existing on disk) -} -``` - -Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. - -Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. - -### 8.3 Search - -**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. - -**Public groups:** client queries nodes directly at request time. Hub provides routing only. - -**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. - ---- - -## 9. Web Client UI - -### 9.1 Architecture - -The web client is a Preact SPA served by the hub at `/app/`. It communicates -with the hub via HTTPS (auth, group management, signaling) and with nodes via -WebRTC DataChannel (file transfer, streaming, chat). The hub is never in the -data path. - -**Technology choices:** -- **Preact** (~3 KB gzipped): lightweight React-compatible framework -- **preact-router**: client-side routing (no server round-trips) -- **esbuild**: minification/bundling (single binary, no npm/node_modules) -- **SubtleCrypto**: browser-native AES-GCM for E2E decryption -- **IndexedDB**: local cache for group indexes (client-side search) - -No heavy frameworks (React, Vue, Angular). No build toolchain dependencies beyond -esbuild. ESM modules loaded natively by modern browsers. - -### 9.2 UI Structure - -``` -┌─────────────────────────────────────────────────────────┐ -│ [MeshBay] [User ▾] [⚙] │ -├──────────┬──────────────────────────────────────────────┤ -│ │ │ -│ Groups │ Main content area │ -│ │ │ -│ ● Private│ - File explorer (folders, files, download) │ -│ Group1 │ - Chat/forum view │ -│ Group2 │ - Video player (HLS via MediaSource API) │ -│ │ - Settings │ -│ ○ Public │ - Notifications feed │ -│ Group3 │ │ -│ │ │ -└──────────┴──────────────────────────────────────────────┘ -``` - -- **Left sidebar:** group list, ordered by usage frequency. Private groups first. - Collapses to hamburger menu on mobile viewports. -- **Top bar:** logo (left), user menu dropdown (right) — settings, profile, - language, online/offline status, logout. -- **Main area:** context-dependent content based on selected group and view. - -### 9.3 Views - -**Front page (no group selected):** -- Notification feed, prioritized: known contacts → private group activity → public -- System notifications (maintenance, updates) -- Quick access to recent groups - -**Group view — File Explorer:** -- Directory tree (folders, subfolders) — read-only browsing -- File metadata: name, size, type, date added -- Actions: download, stream (for media files) -- Files fetched directly from node via DataChannel -- Upload: photos/videos posted to the group's shared upload folder -- Delete: node operator can delete any file; uploader can delete their own uploads. - Hub admin has NO delete authority on nodes they don't operate (see §4.2.x). - -**Group view — Chat/Forum:** -- Sender Keys encrypted messages, fetched from node -- Post text, photos, videos (uploads go to node, not hub) -- Optional thread/topic structure for organized discussions -- Real-time push for connected members, pull history on reconnect - -**Group view — Video Player:** -- HLS segments fetched via DataChannel from node -- Decrypted client-side (GEK-derived key per segment) -- Played via MediaSource API (browser-native, no plugins) - -**Settings:** -- General: theme (dark/light/auto), language, notification preferences -- Per-group: notification mute, display options, filtering/blocking -- Privacy: online/offline status, profile visibility -- Profile: display name, avatar, account details - -### 9.4 Theming and i18n - -**Theme:** CSS custom properties for colors, toggled via: -1. `prefers-color-scheme` media query (OS default) -2. User override stored in localStorage -3. Toggle button in top bar or settings - -**i18n:** JSON translation files loaded client-side. -``` -static/i18n/ -├── en.json # English (default, always loaded) -├── fr.json # French (loaded on demand) -└── ... # Other languages added later -``` - -Keys are identifiers, not English text. Translation function: `t('group.join')`. - -### 9.5 meshbay.org Site Overlay - -meshbay.org serves both the generic hub application and site-specific pages: - -``` -site/ # meshbay.org-specific (not packaged with hub) -├── index.html # Landing page — project promotion, features -├── downloads.html # Package repos: Ubuntu, Fedora, Android APK -├── about.html # Project info, team, GitHub, contact -└── assets/ # Landing-specific CSS, images, icons -``` - -Caddy serves `site/` with priority. Requests not matching a static file fall -through to the hub FastAPI application. The hub serves `/app/` (SPA) and `/v1/` -(API). This separation ensures the hub package remains generic and deployable -by any operator, while meshbay.org has its own public-facing identity. - -### 9.6 Hub Mirror (future — design only) - -A mirror hub is a complete active-active replica of the primary hub. - -**Purpose:** load distribution for growing traffic. DNS round-robin (2+ A records). - -**Design:** -- Shared Ed25519 signing key (transferred once, securely) -- PostgreSQL logical replication for bidirectional read/write -- Both mirrors issue JWTs with the same key -- Both mirrors accept registrations, logins, and group operations -- If one mirror goes down, the other serves all traffic - -**Implementation constraints (must not violate in current development):** -- Hub config and key paths must be externalizable (already the case) -- No hub-instance-specific state that cannot be replicated -- JWT verification must not depend on hub-local state (already the case) -- Session state (refresh tokens, IP logs) must be in PostgreSQL (already the case) - -**Not implemented now.** Design documented to avoid blocking decisions. - ---- - -## 10. Hub Federation (MHP) <!-- was §9 in v3 --> - -### 9.1 Hub Hierarchy - -``` -Root Hub (meshbay.org) - ├── Full Hub (self-hosted, delegated CA) - │ └── issues user credentials, manages own groups - │ └── federates with other Full Hubs via MHP - └── Mirror Hub - └── hosts public Mesh Directory only (no user accounts, no key issuance) -``` - -A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. - -### 9.2 MHP Design - -- Explicit peer selection: each hub maintains an allowlist of trusted peers -- No automatic hub discovery -- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data -- All MHP messages carry `version` field - -### 9.3 Cross-Hub Client Access - -1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link -2. Client presents Hub A JWT directly to Hub B -3. Hub B verifies JWT using Hub A's public key (fetched once, cached) -4. Hub B issues short-lived local session token -5. Client connects to node as normal - ---- - -## 11. Moderation <!-- was §10 in v3 --> - -### 10.1 Public Content - -``` -Report #1 → automatic suspension of public access - → node operator notified -One republication allowed -Report #2 → escalated to hub moderators -Confirmed → group revoked on local hub - → revocation propagated to federated hubs via MHP -``` - -Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. - -### 10.2 CSAM - -Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. - -### 10.3 Copyright - -DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. - -### 10.4 Private Content - -Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. - ---- - -## 12. Python Extension Module System <!-- was §11 in v3 --> - -The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** - -**Module manifest:** -```python -{ - "name": "my-extension", - "version": "1.0.0", - "mnp_version": ">=1.0", - "permissions": ["read_index", "send_message", "receive_events"] -} -``` - -**Available APIs:** -- `read_index()` — read current group index (read-only) -- `send_message(content)` — post to group thread -- `receive_events(handler)` — subscribe to group events - -**Unavailable:** arbitrary network, filesystem access outside group context, system calls. - ---- - -## 13. Legal Framework <!-- was §12 in v3 --> - -**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. - -**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. - -**Protocol/software author:** protected by substantial non-infringing uses. - -**Hub data:** -- Email and optional phone: kept for account recovery and legal compliance -- Password: Argon2id hash, never stored in cleartext -- Connection logs: retained per legal requirements (minimum 1 year) -- Content metadata: never stored -- Node current IP: not persisted (signaling is ephemeral) -- GEK bundles: opaque 48-byte ciphertext blobs; hub cannot decrypt them - ---- - -## 14. Future Features <!-- was §13 in v3 --> - -- **Mesh Relay:** community TURN relays, E2E encrypted traffic. Low priority — typical residential NAT works with ICE/STUN. Needed only for symmetric NAT (CGNAT mobile, ~15% of connections). -- ~~**QUIC transport (v2)**~~ ✅ DONE (Phase 5) — QUIC replaces TCP+TLS. -- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement -- **Hub mirror (load balancing):** design documented in §9.6. Active-active with shared key, PostgreSQL replication, DNS round-robin. Implementation deferred. -- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL. MNP `ephemeral_stream` type reserved. -- **Node–mobile pairing:** QR code from local web UI -- **Multi-source download:** parallel chunk fetching from swarm for public files -- **At-rest encryption on node:** optional for server-deployed nodes -- **OS keychain integration for keystore unlock** -- ~~**WebRTC**~~ ✅ Validated (Phase 9.1–9.5) — `aiortc` for browser-to-node P2P via DataChannel. Tested on SFR residential NAT (Port-Restricted Cone) + 4G CGNAT. No TURN needed. -- **Extension-triggered views:** local apps providing custom views for group content (gallery, kanban). MNP extension hook reserved. - ---- - -## 15. Open Questions [TBD] - -**Resolved by POC (no longer open):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R1 | Argon2id parameters: what values target ~500ms? | `iterations=3, memory_cost=262144` (256 MB). pw_version=2, transparent rehash on login. | Spike 1 + Phase 8.10 | -| R2 | JWT payload claims: what fields for offline node verification? | `jti` (UUID4), `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, `groups` claim. | Spike 3 + Phase 7 | -| R3 | GEK wrapping protocol: exact algorithm? | ECIES-like: ephemeral X25519 + HKDF(salt=pk_eph, info="meshbay:gek_wrap:v1") + ChaCha20-Poly1305(aad=pk_recipient). | Spike 6 | -| R4 | NAT traversal: is STUN/hole-punching sufficient for residential users? | Yes for Cone NAT (SFR, Orange, Free). Relay needed only for symmetric NAT (CGNAT mobile). | Spike 4 | -| R5 | Transport: QUIC or TCP+TLS 1.3 for v1? | TCP+TLS 1.3 for v1, QUIC for v2. QUIC is now the active transport (Phase 5). | Spike 5 | -| R6 | Hub API: which endpoints for GEK distribution? | 4 endpoints confirmed. | Spike 6 | -| R7 | Package structure? | 3 packages: `meshbay-common`, `meshbay-hub`, `meshbay-node`. | POC | - -**Resolved by first security review (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R8 | Group chat encryption model? | Sender Keys protocol. Double Ratchet kept for future 1:1 DM. | Security review C1 | -| R9 | Token denylist distribution? | Push via hub→node WebSocket. In-memory jti set on node. | Security review S3 | -| R10 | Chunk key HKDF: salt or info? | `info` (domain separation), `salt=None`. RFC 5869 compliant. | Security review M5 | -| R11 | AES-GCM keystore IV size? | 96-bit (12 bytes), per NIST SP 800-38D. | Security review S4 | - -**Resolved by Phase 8 implementation (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R12 | Refresh token rotation? | One-time-use with family-based reuse detection. Old token reuse revokes entire family. | Phase 8.3 | -| R13 | Email encryption at rest? | AES-256-GCM, key derived from hub Ed25519 private key via HKDF(info="meshbay:email:v1"). | Phase 8.2 | -| R14 | Admin authorization model? | Config-based: `admin_usernames` in hub.toml + `MESHBAY_ADMIN_USERS` env var. | Phase 8.1 | -| R15 | QUIC migration timeline? | Done — QUIC is the active transport since Phase 5. | Phase 5 | - -**Resolved by web client design session (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R16 | Browser transport for NAT traversal? | WebRTC DataChannel with ICE/STUN. WebTransport cannot work (port-restricted cone NAT). | Design session | -| R17 | Chat storage location? | On nodes, not hub. Hub never stores content. | Design session | -| R18 | Web UI framework? | Preact SPA (~3 KB), esbuild, dark/light theme, i18n, responsive. | Design session | -| R19 | Hub mirror design? | Active-active, shared signing key, PostgreSQL replication, DNS round-robin. | Design session | - -**Resolved by Phase 9 spike (2026-08-10):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R20 | WebRTC DataChannel validation? | Confirmed: browser→NAT→node file transfer works. Tested 3 scenarios on SFR residential (Port-Restricted Cone NAT) + 4G CGNAT: WiFi LAN (IPv6 direct, ~100ms), 4G IPv6 inter-network (~600ms), 4G IPv4 STUN hole-punch (~650ms). No TURN relay needed. | Phase 9.5 spike | - -**Resolved by node sovereignty fix (2026-08-12):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R21 | Who authorizes file deletion on a node? | Node operator (sovereign) OR original uploader. Hub admin has no authority over node content. Enforced: deny-by-default in MNP `file_delete`, `is_node_admin` in handshake_ack, `uploader_id` in IndexEntry. | Security fix — §4.2.x | -| R22 | Can a malicious hub admin access node content? | No. Two cryptographic layers: (1) GEK-HMAC proof in handshake — hub never has the GEK, can't pass the challenge. (2) Ed25519 challenge-response for admin ops — hub can't forge the node operator's signature. `gek_req` endpoint removed. | Crypto enforcement — §4.2.x | - -**Resolved by Phase 12 — P2P crypto material (2026-08-13):** - -| # | Question | Resolution | Source | -|---|---|---|---| -| R23 | Where are GEK bundles stored? | On node only (`BundleStore` SQLite, `data_dir/bundles.db`). Hub `GEKBundle` model removed. Exchanged via MNP `gek_bundle_store`/`gek_bundle_fetch`/`gek_bundle_resp` over WebRTC DataChannel. | Phase 12 — T3 | -| R24 | Where are keypair bundles stored? | On node only (`BundleStore`). Encrypted with password-derived AES key (`bundle_key`). Browser pushes after registration, recovers during handshake. Hub `keypair_bundle` column removed. | Phase 12 — T3 | -| R25 | How does the browser recover keys after localStorage cleared? | Transport fetches `keypair_bundle` from node during handshake, decrypts with `_bundleKey` (PBKDF2 from password). Public key derived from private key via JWK export — no hub fetch needed. `_bundleKey` persisted in IndexedDB, `_sessionKeys` in sessionStorage. | Phase 12 | -| R26 | How does the browser handle Chrome SDP re-serialization? | `this._rawAnswerSdp = answer.sdp` saved before `setRemoteDescription`. DTLS fingerprint extracted from raw SDP, not `pc.remoteDescription.sdp` (Chrome may drop sha-256 line when re-serializing multi-hash SDP from aiortc). | Phase 12 | -| R27 | Should the browser auto-regenerate keys on login? | No. Auto-regeneration silently rotates hub keys, breaking GEK unwrap (GEK wrapped for old keys). Keys recovered from node via `_bundleKey`. Regeneration only on explicit user request. | Phase 12 | - -**Still open:** - -1. **Refresh token validity:** 30 or 90 days? -2. **Group address scheme:** final URL format confirmation -3. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? → Resolved: always on node. -4. **MHP federation sync frequency and conflict resolution** -5. **Chat attachment storage:** stored on node like regular files, or separate store? -6. **Relay registration protocol design** (when implemented) - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: POC v1 (was docs/poc-v1.md) --> -<!-- ============================================================ --> - -# MeshBay — POC v1 - -> Goal: validate key concepts before committing to a full implementation. -> Scope: Hub/Node exchange in Python, crypto stack, NAT traversal, encrypted file chunk transfer. -> Everything in-memory (no database), minimal code, TCP only (no QUIC yet). - ---- - -## Environment - -### Remote — meshbay.org (Hub) -- OVH VPS, Ubuntu 26.04 LTS, Python 3.14.4 -- Public fixed IP, ports 80 and 443 open -- Clean slate: no web server installed -- SSH access: `ssh cbesson@meshbay.org` - -### Local — Fedora 44 (Node) -- Laptop behind SFR residential NAT (likely Restricted Cone NAT — UPnP supported) -- Python 3.13+ via system packages -- User: `cbesson` (sudoer, no password) - ---- - -## Python Dependencies - -```bash -# Shared (hub and node) -cryptography>=43.0 # Ed25519, X25519, ChaCha20-Poly1305, Argon2id -PyJWT>=2.9 # JWT with EdDSA (Ed25519) support -blake3>=1.0 # Fast content hashing - -# Hub only (meshbay.org) -fastapi>=0.115 -uvicorn[standard]>=0.30 - -# Node only (Fedora laptop) -httpx>=0.28 # Async HTTP client for hub→node calls -aioice>=0.9 # STUN queries for NAT discovery -miniupnpc>=2.2 # UPnP port mapping on SFR box -``` - -Install on each machine: -```bash -python3 -m venv .venv -source .venv/bin/activate -pip install <packages above> -``` - ---- - -## Hub Setup on meshbay.org - -For the POC, uvicorn runs directly on port 80 via iptables redirect (no Caddy/nginx needed yet — HTTPS added before production). - -```bash -# On meshbay.org -# Redirect port 80 → 8000 (persistent via iptables-save if needed) -sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000 - -# Run hub (from poc directory, venv activated) -uvicorn hub:app --host 127.0.0.1 --port 8000 --reload -``` - -> Note: HTTPS (via Caddy + Let's Encrypt) is mandatory before any data beyond this POC. Not in scope here. - ---- - -## Spike Overview - -| # | Name | Where | Validates | Duration | -|---|---|---|---|---| -| 1 | Crypto primitives | Local | Python crypto stack covers all needs | ~1h | -| 2 | Hub skeleton | meshbay.org | Hub API, JWT issuance | ~2h | -| 3 | Node registration | Fedora | Hub-Node handshake, JWT offline verify | ~1h | -| 4 | NAT traversal | Both | SFR box UPnP + STUN, P2P reachability | ~2h | -| 5 | Encrypted transfer | Both | On-the-fly GEK encryption, P2P chunk | ~2h | - ---- - -## Spike 1 — Crypto Primitives (local only) - -**Goal:** confirm `cryptography` (PyCA) covers all MeshBay cryptographic needs without gaps or performance surprises. - -**File:** `spike1_crypto.py` - -**What to test:** - -```python -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id # PyCA 43+ -from cryptography.hazmat.primitives import hashes, serialization -import blake3, os, time -``` - -**Test 1: Ed25519 — hub keypair, sign JWT payload, verify** -```python -sk_hub = Ed25519PrivateKey.generate() -pk_hub = sk_hub.public_key() -msg = b"test payload" -sig = sk_hub.sign(msg) -pk_hub.verify(sig, msg) # raises if invalid -print("Ed25519 OK") -``` - -**Test 2: X25519 — two-party key agreement for GEK wrapping** -```python -sk_a = X25519PrivateKey.generate() -sk_b = X25519PrivateKey.generate() -shared_a = sk_a.exchange(sk_b.public_key()) -shared_b = sk_b.exchange(sk_a.public_key()) -assert shared_a == shared_b -print("X25519 OK") -``` - -**Test 3: GEK derivation and ChaCha20-Poly1305 on a 1 MB chunk** -```python -gek = ChaCha20Poly1305.generate_key() -cipher = ChaCha20Poly1305(gek) -chunk = os.urandom(1024 * 1024) # 1 MB - -t0 = time.perf_counter() -nonce = os.urandom(12) -ct = cipher.encrypt(nonce, chunk, None) -pt = cipher.decrypt(nonce, ct, None) -elapsed = time.perf_counter() - t0 - -assert pt == chunk -print(f"ChaCha20-Poly1305 1MB: {elapsed*1000:.1f} ms") -``` - -**Test 4: HKDF chunk key derivation** -```python -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes -chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + blake3.blake3(chunk).digest() + b":chunk:0" -).derive(gek) -print(f"HKDF derived key: {chunk_key.hex()[:16]}...") -``` - -**Test 5: Argon2id keystore key derivation** -```python -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id -salt = os.urandom(16) -t0 = time.perf_counter() -kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) -key = kdf.derive(b"mypassword") -print(f"Argon2id: {(time.perf_counter()-t0)*1000:.0f} ms, key: {key.hex()[:16]}...") -``` - -**Test 6: PyJWT with Ed25519 (EdDSA)** -```python -import jwt -sk_hub_pem = sk_hub.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() -) -pk_hub_pem = pk_hub.public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo -) -payload = {"sub": "user_abc", "pk_user": "base64...", "exp": 9999999999} -token = jwt.encode(payload, sk_hub_pem, algorithm="EdDSA") -decoded = jwt.decode(token, pk_hub_pem, algorithms=["EdDSA"]) -assert decoded["sub"] == "user_abc" -print("JWT EdDSA OK") -``` - -**Success criteria:** all tests pass, ChaCha20 1MB < 20ms, Argon2id ~1s. - ---- - -## Spike 2 — Hub Skeleton (meshbay.org) - -**Goal:** minimal FastAPI hub, in-memory storage, 5 endpoints. - -**File:** `hub.py` (on meshbay.org) - -### Hub keypair generation (run once, save to disk) - -```python -# gen_hub_keys.py — run once on meshbay.org -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives import serialization -import base64, json - -sk = Ed25519PrivateKey.generate() -pk = sk.public_key() - -with open("hub_private.pem", "wb") as f: - f.write(sk.private_bytes( - serialization.Encoding.PEM, - serialization.PrivateFormat.PKCS8, - serialization.NoEncryption() - )) -with open("hub_public.pem", "wb") as f: - f.write(pk.public_bytes( - serialization.Encoding.PEM, - serialization.PublicFormat.SubjectPublicKeyInfo - )) -print("Hub keypair generated.") -``` - -### Hub API (`hub.py`) - -```python -from fastapi import FastAPI, HTTPException, Depends, Header -from pydantic import BaseModel -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives import serialization, hashes -from cryptography.hazmat.primitives.kdf.argon2 import Argon2id -import jwt, uuid, os, time, base64 - -app = FastAPI(title="MeshBay Hub POC") - -# Load hub keypair -with open("hub_private.pem", "rb") as f: - HUB_SK_PEM = f.read() -with open("hub_public.pem", "rb") as f: - HUB_PK_PEM = f.read() - -HUB_ID = "meshbay.org" -ACCESS_TOKEN_TTL = 3600 # 1 hour -REFRESH_TOKEN_TTL = 86400 * 30 # 30 days - -# In-memory stores (POC only — not persistent) -users = {} # username → {user_id, pw_hash, pw_salt, pk_ed25519, pk_x25519} -nodes = {} # node_id → {user_id, pk_node, endpoint_hint, registered_at} -refresh_tokens = {} # token → user_id - -# --- Models --- - -class UserRegister(BaseModel): - username: str - password: str - pk_user_ed25519: str # base64 - pk_user_x25519: str # base64 - -class UserLogin(BaseModel): - username: str - password: str - -class NodeAnnounce(BaseModel): - pk_node: str # base64 Ed25519 public key - endpoint_hint: str | None = None # "ip:port" or null - -# --- Helpers --- - -def hash_password(password: str) -> tuple[bytes, bytes]: - salt = os.urandom(16) - kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) - return kdf.derive(password.encode()), salt - -def verify_password(password: str, pw_hash: bytes, salt: bytes) -> bool: - kdf = Argon2id(salt=salt, length=32, iterations=3, lanes=4, memory_cost=65536) - try: - kdf.verify(password.encode(), pw_hash) - return True - except Exception: - return False - -def issue_access_token(user: dict) -> str: - payload = { - "iss": HUB_ID, - "sub": user["user_id"], - "pk_user": user["pk_ed25519"], - "hub_id": HUB_ID, - "iat": int(time.time()), - "exp": int(time.time()) + ACCESS_TOKEN_TTL, - } - return jwt.encode(payload, HUB_SK_PEM, algorithm="EdDSA") - -def get_current_user(authorization: str = Header(...)) -> dict: - try: - scheme, token = authorization.split() - if scheme.lower() != "bearer": - raise ValueError - payload = jwt.decode(token, HUB_PK_PEM, algorithms=["EdDSA"]) - user_id = payload["sub"] - user = next((u for u in users.values() if u["user_id"] == user_id), None) - if not user: - raise HTTPException(status_code=401, detail="User not found") - return user - except Exception: - raise HTTPException(status_code=401, detail="Invalid token") - -# --- Endpoints --- - -@app.get("/v1/hub/info") -def hub_info(): - return { - "hub_id": HUB_ID, - "pk_hub_ed25519": base64.b64encode( - Ed25519PrivateKey.from_private_bytes( - # shortcut for POC — load pk directly - open("hub_public.pem","rb").read() - ).public_bytes(...) # see note below - ).decode(), - "mnp_version": "0.1", - "mhp_version": "0.1", - } - # Note: return pk_hub_pem directly for POC, nodes store it on first contact - -@app.get("/v1/hub/pubkey") -def hub_pubkey(): - """Return hub Ed25519 public key PEM — cached by nodes on first contact.""" - return {"pk_hub_pem": HUB_PK_PEM.decode()} - -@app.post("/v1/users/register", status_code=201) -def register(body: UserRegister): - if body.username in users: - raise HTTPException(status_code=409, detail="Username taken") - pw_hash, pw_salt = hash_password(body.password) - user_id = str(uuid.uuid4()) - users[body.username] = { - "user_id": user_id, - "username": body.username, - "pw_hash": pw_hash, - "pw_salt": pw_salt, - "pk_ed25519": body.pk_user_ed25519, - "pk_x25519": body.pk_user_x25519, - } - return {"user_id": user_id} - -@app.post("/v1/users/login") -def login(body: UserLogin): - user = users.get(body.username) - if not user or not verify_password(body.password, user["pw_hash"], user["pw_salt"]): - raise HTTPException(status_code=401, detail="Invalid credentials") - access_token = issue_access_token(user) - refresh_token = base64.urlsafe_b64encode(os.urandom(32)).decode() - refresh_tokens[refresh_token] = user["user_id"] - return { - "access_token": access_token, - "refresh_token": refresh_token, - "token_type": "bearer", - "expires_in": ACCESS_TOKEN_TTL, - } - -@app.post("/v1/users/token/refresh") -def refresh(body: dict): - rt = body.get("refresh_token", "") - user_id = refresh_tokens.get(rt) - if not user_id: - raise HTTPException(status_code=401, detail="Invalid refresh token") - user = next((u for u in users.values() if u["user_id"] == user_id), None) - if not user: - raise HTTPException(status_code=401, detail="User not found") - return {"access_token": issue_access_token(user), "token_type": "bearer"} - -@app.post("/v1/nodes/announce", status_code=201) -def announce_node(body: NodeAnnounce, user: dict = Depends(get_current_user)): - node_id = str(uuid.uuid4()) - nodes[node_id] = { - "node_id": node_id, - "user_id": user["user_id"], - "pk_node": body.pk_node, - "endpoint_hint": body.endpoint_hint, - "announced_at": int(time.time()), - } - return {"node_id": node_id} - -@app.get("/v1/nodes/{node_id}") -def get_node(node_id: str, user: dict = Depends(get_current_user)): - node = nodes.get(node_id) - if not node: - raise HTTPException(status_code=404, detail="Node not found") - return { - "node_id": node["node_id"], - "pk_node": node["pk_node"], - "endpoint_hint": node["endpoint_hint"], - } -``` - -**Success criteria:** -- Hub starts, all 6 endpoints respond correctly -- `GET /v1/hub/pubkey` returns the PEM -- `POST /v1/users/register` + `POST /v1/users/login` returns a valid JWT -- JWT verified by `jwt.decode()` with hub public key — passes - ---- - -## Spike 3 — Node Registration (Fedora laptop) - -**Goal:** node generates its keypair, registers a user on the hub, gets a JWT, and verifies it locally without contacting the hub again. - -**File:** `node.py` - -```python -import httpx, asyncio, jwt, base64, os -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey -from cryptography.hazmat.primitives import serialization - -HUB_URL = "http://meshbay.org" # HTTP for POC, HTTPS later - -async def main(): - async with httpx.AsyncClient() as client: - - # 1. Fetch hub public key (first contact — cache this) - r = await client.get(f"{HUB_URL}/v1/hub/pubkey") - hub_pk_pem = r.json()["pk_hub_pem"].encode() - print(f"[node] Hub PK fetched ({len(hub_pk_pem)} bytes)") - - # 2. Generate node identity keypairs - sk_ed = Ed25519PrivateKey.generate() - pk_ed = sk_ed.public_key() - sk_x = X25519PrivateKey.generate() - pk_x = sk_x.public_key() - - pk_ed_b64 = base64.b64encode( - pk_ed.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) - ).decode() - pk_x_b64 = base64.b64encode( - pk_x.public_bytes(serialization.Encoding.Raw, serialization.PublicFormat.Raw) - ).decode() - - # 3. Register user (skip if already registered) - r = await client.post(f"{HUB_URL}/v1/users/register", json={ - "username": "testnode", - "password": "testpass123", - "pk_user_ed25519": pk_ed_b64, - "pk_user_x25519": pk_x_b64, - }) - print(f"[node] Register: {r.status_code} {r.text}") - - # 4. Login, get access token - r = await client.post(f"{HUB_URL}/v1/users/login", json={ - "username": "testnode", - "password": "testpass123", - }) - data = r.json() - access_token = data["access_token"] - print(f"[node] Login OK, token: {access_token[:40]}...") - - # 5. Verify JWT locally — NO hub roundtrip - decoded = jwt.decode(access_token, hub_pk_pem, algorithms=["EdDSA"]) - print(f"[node] JWT verified locally: sub={decoded['sub']}, exp={decoded['exp']}") - - # 6. Announce node to hub - r = await client.post( - f"{HUB_URL}/v1/nodes/announce", - json={"pk_node": pk_ed_b64, "endpoint_hint": None}, - headers={"Authorization": f"Bearer {access_token}"} - ) - node_id = r.json()["node_id"] - print(f"[node] Node announced: {node_id}") - -asyncio.run(main()) -``` - -**Success criteria:** -- Node registers, logs in, receives JWT -- JWT decoded offline using only the hub's public key — no hub call -- Node announced; `GET /v1/nodes/{node_id}` from hub returns correct PK - ---- - -## Spike 4 — NAT Traversal (both machines) - -**Goal:** discover the local node's external IP:port via STUN and UPnP; test reachability from meshbay.org. - -**File:** `spike4_nat.py` (Fedora laptop) - -### Part A — UPnP (try first, most reliable on SFR box) - -```python -import miniupnpc -import socket - -def try_upnp(internal_port=19000): - u = miniupnpc.UPnP() - u.discoverdelay = 200 - ndevices = u.discover() - if ndevices == 0: - print("UPnP: no IGD found") - return None - - u.selectigd() - external_ip = u.externalipaddress() - local_ip = socket.gethostbyname(socket.gethostname()) - - result = u.addportmapping( - internal_port, 'TCP', local_ip, internal_port, - 'MeshBay POC', '' - ) - if result: - print(f"UPnP: mapped {external_ip}:{internal_port} → {local_ip}:{internal_port}") - return f"{external_ip}:{internal_port}" - else: - print("UPnP: mapping failed") - return None -``` - -### Part B — STUN discovery - -```python -import asyncio -import aioice - -async def stun_discover(local_port=19001): - # Use Cloudflare STUN server - stun_servers = [("stun.cloudflare.com", 3478), ("stun.l.google.com", 19302)] - - connection = aioice.Connection(ice_controlling=True, stun_server=stun_servers[0]) - await connection.gather_candidates() - - for candidate in connection.local_candidates: - if candidate.type == "srflx": # server-reflexive = external address - print(f"STUN srflx: {candidate.host}:{candidate.port}") - return f"{candidate.host}:{candidate.port}" - - print("STUN: no srflx candidate found (may be symmetric NAT)") - return None -``` - -### Part C — Reachability test from meshbay.org - -Once the node has an external address (from UPnP or STUN), it announces it to the hub (`endpoint_hint`). Then from meshbay.org: - -```bash -# On meshbay.org — manually test TCP reachability -nc -zv <external_ip> <external_port> -# or -python3 -c "import socket; s=socket.create_connection(('<external_ip>', <port>), timeout=5); print('REACHABLE'); s.close()" -``` - -And on the Fedora node, a simple listener: -```python -# On Fedora, open a listener on the discovered port -import socket -s = socket.socket() -s.bind(('', 19000)) -s.listen(1) -print("Listening on 19000...") -conn, addr = s.accept() -print(f"Connection from {addr}") -conn.sendall(b"HELLO FROM NODE\n") -conn.close() -``` - -**Expected outcomes on SFR residential:** - -| Method | Expected result | Confidence | -|---|---|---| -| UPnP | Works — SFR La Box supports UPnP IGD | High | -| STUN srflx | Discovered — SFR is cone NAT for residential | High | -| Direct TCP from meshbay.org | Works if UPnP succeeded | High | -| Hole punching only | Depends on NAT type discovered | Medium | - -**Success criteria:** at least one method allows meshbay.org to reach the Fedora node's port directly. - ---- - -## Spike 5 — Encrypted File Transfer (both machines) - -**Goal:** node serves an encrypted file chunk via direct P2P TCP connection; client decrypts and verifies. - -**Prerequisite:** Spike 4 succeeded — external IP:port is known and reachable. - -**File:** `spike5_server.py` (Fedora), `spike5_client.py` (meshbay.org) - -### Node side — serve one encrypted chunk - -```python -# spike5_server.py — Fedora laptop -import asyncio, os, base64 -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes, serialization -import blake3, struct, json - -# Keypair (reuse from Spike 3 or generate here) -sk_node = Ed25519PrivateKey.generate() -pk_node_bytes = sk_node.public_key().public_bytes( - serialization.Encoding.Raw, serialization.PublicFormat.Raw -) - -# Generate GEK (in a real system, loaded from keystore) -gek_raw = ChaCha20Poly1305.generate_key() -cipher = ChaCha20Poly1305(gek_raw) - -CHUNK_SIZE = 1024 * 1024 # 1 MB - -def make_chunk(file_path: str, chunk_index: int) -> bytes: - """Read, compress (skipped for POC), encrypt, sign a chunk.""" - with open(file_path, "rb") as f: - f.seek(chunk_index * CHUNK_SIZE) - data = f.read(CHUNK_SIZE) - - file_hash = blake3.blake3(open(file_path, "rb").read()).digest() - - # Per-chunk key derivation - chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + file_hash + b":chunk:" + chunk_index.to_bytes(4, "big") - ).derive(gek_raw) - chunk_cipher = ChaCha20Poly1305(chunk_key) - - nonce = os.urandom(12) - ct = chunk_cipher.encrypt(nonce, data, None) - chunk_hash = blake3.blake3(ct).digest() - - # Sign: chunk_index + nonce + ciphertext_hash - sig_payload = chunk_index.to_bytes(4, "big") + nonce + chunk_hash - sig = sk_node.sign(sig_payload) - - return json.dumps({ - "chunk_index": chunk_index, - "nonce": base64.b64encode(nonce).decode(), - "ciphertext": base64.b64encode(ct).decode(), - "chunk_hash": base64.b64encode(chunk_hash).decode(), - "signature": base64.b64encode(sig).decode(), - "pk_node": base64.b64encode(pk_node_bytes).decode(), - "gek_hint": base64.b64encode(gek_raw).decode(), # POC: send GEK in band — never in production! - }).encode() - -async def handle_client(reader, writer): - request = await reader.read(1024) - req = json.loads(request) - chunk_index = req.get("chunk_index", 0) - file_path = req.get("file", "testfile.bin") - - print(f"[node] Client requests chunk {chunk_index} of {file_path}") - chunk_data = make_chunk(file_path, chunk_index) - - writer.write(len(chunk_data).to_bytes(4, "big") + chunk_data) - await writer.drain() - writer.close() - print(f"[node] Chunk {chunk_index} sent ({len(chunk_data)} bytes)") - -async def main(): - # Create a 5MB test file - if not os.path.exists("testfile.bin"): - with open("testfile.bin", "wb") as f: - f.write(os.urandom(5 * 1024 * 1024)) - print("[node] Test file created (5 MB)") - - server = await asyncio.start_server(handle_client, "0.0.0.0", 19000) - print("[node] Serving on port 19000 — waiting for client...") - async with server: - await server.serve_forever() - -asyncio.run(main()) -``` - -### Client side — request, verify, decrypt - -```python -# spike5_client.py — meshbay.org -import asyncio, base64, json -from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey -from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305 -from cryptography.hazmat.primitives.kdf.hkdf import HKDF -from cryptography.hazmat.primitives import hashes, serialization -import blake3 - -NODE_HOST = "<external_ip>" # from Spike 4 -NODE_PORT = 19000 - -async def main(): - reader, writer = await asyncio.open_connection(NODE_HOST, NODE_PORT) - - # Request chunk 0 - request = json.dumps({"file": "testfile.bin", "chunk_index": 0}).encode() - writer.write(request) - await writer.drain() - - # Receive - length_bytes = await reader.readexactly(4) - length = int.from_bytes(length_bytes, "big") - data = await reader.readexactly(length) - writer.close() - - chunk = json.loads(data) - print(f"[client] Received chunk {chunk['chunk_index']}") - - # 1. Verify signature - pk_node_bytes = base64.b64decode(chunk["pk_node"]) - pk_node = Ed25519PublicKey.from_public_bytes(pk_node_bytes) - ct = base64.b64decode(chunk["ciphertext"]) - nonce = base64.b64decode(chunk["nonce"]) - chunk_hash = base64.b64decode(chunk["chunk_hash"]) - sig = base64.b64decode(chunk["signature"]) - - sig_payload = (0).to_bytes(4, "big") + nonce + chunk_hash - pk_node.verify(sig, sig_payload) # raises on failure - print("[client] Signature OK") - - # 2. Verify ciphertext hash - assert blake3.blake3(ct).digest() == chunk_hash - print("[client] Ciphertext hash OK") - - # 3. Derive chunk key and decrypt (GEK from POC hint — never in production) - gek_raw = base64.b64decode(chunk["gek_hint"]) - # (in production, client has GEK from hub's GEK bundle) - chunk_key = HKDF( - algorithm=hashes.SHA256(), length=32, salt=None, - info=b"file:" + bytes(32) + b":chunk:" + (0).to_bytes(4, "big") - # Note: in production, file_hash is sent separately or in index - ).derive(gek_raw) - plaintext = ChaCha20Poly1305(chunk_key).decrypt(nonce, ct, None) - print(f"[client] Decrypted {len(plaintext)} bytes") - print("[client] Encrypted P2P transfer: SUCCESS") - -asyncio.run(main()) -``` - -**Note on GEK in POC:** the GEK is included in the response as `gek_hint` for POC convenience only. In production, the client gets the GEK from the hub's encrypted GEK bundle (delivered at login, decrypted client-side with the user's X25519 private key). - -**Success criteria:** -- Client receives chunk from node via direct TCP connection -- Signature verification passes -- Ciphertext hash matches -- Decryption produces the original bytes -- End-to-end: `original_bytes == decrypted_bytes` ✓ - ---- - -## What POC Validates (and Doesn't) - -### Validated by these spikes - -| Concept | Spike | Validation | -|---|---|---| -| Python crypto stack is sufficient | 1 | All primitives work, performance acceptable | -| Hub/Node JWT handshake | 2, 3 | JWT issued by hub, verified offline by node | -| Hub-Node REST protocol (minimal MNP/HTTP) | 2, 3 | API contract works end-to-end | -| SFR NAT traversal via UPnP | 4 | P2P reachability confirmed | -| STUN external address discovery | 4 | Confirmed/fallback documented | -| On-the-fly per-chunk encryption | 5 | GEK + HKDF chunk derivation + ChaCha20 | -| Chunk signature and verification | 5 | Ed25519 sign/verify before decryption | -| Real P2P file transfer | 5 | No hub in data path | - -### NOT in scope - -- Database (all in-memory) -- HTTPS / TLS (HTTP for POC) -- QUIC transport (plain TCP) -- GEK bundle distribution via hub (GEK sent in-band for POC) -- Group management -- Chat / Double Ratchet -- Mesh Group Index -- MHP federation -- Android client -- Module system -- Persistence between restarts - ---- - -## Spike Order Dependency Graph - -``` -Spike 1 (crypto) - └──→ Spike 2 (hub skeleton) - └──→ Spike 3 (node registration) - └──→ Spike 4 (NAT traversal) - └──→ Spike 5 (encrypted transfer) -``` - -Spike 1 is a prerequisite for all others. Spikes 2 and 3 can overlap if two people work in parallel. Spike 4 can begin independently once Spike 3 is running. - - ---- - -<!-- ============================================================ --> -<!-- ARCHIVED: Development Phases (1-12) (was docs/devel-phases.md) --> -<!-- ============================================================ --> - -# MeshBay — Development Phases - -> Reference: architecture spec in draft v3 (section 3 of this archive) -> POC results: `poc/spike-results.md` - ---- - -## Phase 1 — Foundations ✅ DONE - -**Goal:** validate all blocking concepts before writing production code. - -### Deliverables - -| Item | Status | Notes | -|---|---|---| -| POC Spike 1 — Crypto primitives | ✅ | All 22 tests pass. ChaCha20 1MB in 1.1ms. | -| POC Spike 2 — Hub skeleton | ✅ | 12/12 endpoints. JWT EdDSA offline verify in 884µs. | -| POC Spike 3 — Node registration | ✅ | Full handshake. jti bug found and fixed. | -| POC Spike 4 — NAT traversal | ✅ | Cone NAT on SFR. UDP P2P works. UPnP disabled (SFR). | -| POC Spike 5 — Encrypted transfer | ✅ | 1MB P2P. 3.2ms encrypt, 3.9ms decrypt. 4.3MB/s. | -| POC Spike 6 — GEK distribution | ✅ | X25519+HKDF wrap/unwrap. 0.48ms/0.59ms. Hub opaque. | -| Security cleanup meshbay.org | ✅ | UFW: 22/80/443 only. No services exposed. | -| Git monorepo | ✅ | 3 packages: meshbay-common, meshbay-hub, meshbay-node. | -| Draft v3 | ✅ | POC findings integrated. All corrections applied. | -| CLAUDE.md conventions | ✅ | Python 3.12+, uv, ruff, SemVer, commit format. | - -### Key findings from POC - -- jti mandatory in all JWTs (Ed25519 is deterministic — same payload = same token) -- Argon2id at 64MB/3iter = 78ms — increase to 256MB for production (~500ms target) -- NAT order: STUN/hole-punching is priority 2, not UPnP (UPnP disabled on tested SFR box) -- TCP+TLS for v1 transport; QUIC in v2 -- GEK wrapping: ephemeral X25519 + HKDF(salt=pk_eph) + ChaCha20-Poly1305(aad=pk_recipient) - ---- - -## Phase 2 — Node v1 ✅ DONE - -**Goal:** working Mesh Node: indexes a directory, registers with hub, -serves encrypted chunks over TCP+TLS, local web UI on localhost:18000. - -**Transport:** TCP+TLS 1.3 (QUIC in v2). Self-signed cert per node. -Node identity verified via Ed25519 PK from hub, not TLS cert chain (client uses CERT_NONE). - -### Milestones - -| # | Component | File(s) | Tests | -|---|---|---|---| -| 2.1 | Keystore | `meshbay_node/keystore.py` | 10/10 | -| 2.2 | Hub client | `meshbay_node/hub_client.py` | 6/6 | -| 2.3 | Directory indexer | `meshbay_node/indexer/indexer.py` | 5/5 | -| 2.4 | Mesh Group Index | `meshbay_node/indexer/group_index.py` | 5/5 | -| 2.5 | TCP+TLS chunk server | `meshbay_node/transport/server.py` | 3/3 | -| 2.6 | TCP+TLS chunk client | `meshbay_node/transport/client.py` | included above | -| 2.7 | TLS cert helper | `meshbay_node/transport/tls_cert.py` | — | -| 2.8 | Config (TOML + env) | `meshbay_node/config.py` | — | -| 2.9 | Local web UI | `meshbay_node/ui/app.py` | — | -| 2.10 | Daemon + CLI | `meshbay_node/daemon.py` | — | - -**Total: 29/29 tests passing** - -### Python dependencies (dev venv — `/home/cbesson/meshbay/.venv`) - -Installed packages (freeze) as of Phase 2 completion: - -``` -aioice==0.10.2 # ICE/STUN for NAT traversal -blake3==1.0.9 # fast content hashing -cryptography==50.0.0 # Ed25519, X25519, ChaCha20, Argon2id, AES-GCM -fastapi==0.141.1 # local web UI + hub POC -httpx==0.28.1 # hub client HTTP -meshbay-common==0.1.0 # editable install -meshbay-node==0.1.0 # editable install -msgpack==1.2.1 # wire serialisation -PyJWT==2.13.0 # JWT EdDSA -pytest==9.1.1 # test runner -pytest-asyncio==1.4.0 # async test support -uvicorn==0.52.1 # ASGI server (local UI) -watchdog==6.0.0 # filesystem watcher -zstandard==0.25.0 # zstd compression -``` - -Also installed transitively: pydantic 2.13.4, starlette 1.6.0, anyio 4.14.2, uvloop 0.22.1. - -### meshbay_common/crypto.py — Argon2id NOTE - -Current params: `iterations=3, memory_cost=65536` (64MB) → ~78ms on dev laptop. -**Must increase to `memory_cost=262144` (256MB) before production keystore use.** -Run `meshbay-node calibrate-argon2` on target hardware to tune. - -### Out of scope for Phase 2 - -Multiple groups, chat, QUIC, module sandbox, mobile pairing, HLS streaming. - ---- - -## Phase 3 — Hub v1 production ✅ DONE - -**Goal:** replace POC in-memory hub with a production-ready service on meshbay.org. -PostgreSQL persistence, HTTPS via Caddy, all endpoints hardened, legal IP logging, deploy. - -### Environment - -- **Server:** meshbay.org — Ubuntu 26.04 LTS, Python 3.14.4, OVH VPS -- **Database:** PostgreSQL 16 (to install) -- **Proxy:** Caddy (to install — handles Let's Encrypt automatically) -- **Service:** systemd `meshbay-hub.service` - -### Python dependencies to add (Phase 3) - -``` -sqlalchemy>=2.0 # async ORM (SQLAlchemy 2.x) -alembic>=1.13 # DB migrations -asyncpg>=0.30 # PostgreSQL async driver -aiosqlite>=0.20 # SQLite async driver (tests only) -slowapi>=0.1 # rate limiting (FastAPI middleware) -``` - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 3.1 | DB models | `meshbay_hub/db/models.py` | ✅ | -| 3.2 | DB engine + session | `meshbay_hub/db/engine.py` | ✅ | -| 3.3 | Alembic migrations | `meshbay_hub/db/migrations/` | ✅ initial_schema | -| 3.4 | Hub config | `meshbay_hub/config.py` | ✅ | -| 3.5 | Auth (JWT + Argon2id) | `meshbay_hub/auth.py` | ✅ | -| 3.6 | API deps | `meshbay_hub/api/deps.py` | ✅ | -| 3.7 | Hub info router | `meshbay_hub/api/hub.py` | ✅ | -| 3.8 | Users router | `meshbay_hub/api/users.py` | ✅ | -| 3.9 | Nodes router | `meshbay_hub/api/nodes.py` | ✅ | -| 3.10 | Groups router | `meshbay_hub/api/groups.py` | ✅ | -| 3.11 | Rate limiting | `meshbay_hub/api/middleware.py` | ✅ slowapi | -| 3.12 | App factory + lifespan | `meshbay_hub/app.py` | ✅ | -| 3.13 | Hub daemon CLI | `meshbay_hub/daemon.py` | ✅ | -| 3.14 | PostgreSQL 16 | meshbay.org | ✅ DB: meshbay_hub | -| 3.15 | Caddy + Let's Encrypt | meshbay.org Caddyfile | ✅ HTTPS auto-cert | -| 3.16 | Systemd service | `/etc/systemd/system/meshbay-hub.service` | ✅ | -| 3.17 | Deploy + smoke test | https://meshbay.org | ✅ 11/11 endpoints | - -**Total: 40 local tests (SQLite) + 11/11 smoke tests HTTPS production** - -### Phase 3 deployment details (meshbay.org) - -- PostgreSQL 16, user `meshbay`, DB `meshbay_hub` -- Caddy auto-handles Let's Encrypt for `meshbay.org` and `www.meshbay.org` -- `www.meshbay.org` → 301 redirect → `meshbay.org` -- TLS setup documented in `HTTPS.md` -- Hub listens on `127.0.0.1:8000`, Caddy proxies 80/443 -- Systemd service: `meshbay-hub.service` (restart-on-failure) -- Hub config: `~/.config/meshbay/hub.toml` -- Hub keypair: `~/.config/meshbay/hub_private.pem` (chmod 600) -- Source deployed at: `~/meshbay-hub/` (common_pkg + hub_pkg) - -### Additional Python dependencies added in Phase 3 - -``` -sqlalchemy==2.0.51 # async ORM -alembic==1.19.1 # DB migrations -asyncpg==0.31.0 # PostgreSQL async driver -aiosqlite # SQLite async (tests only) -slowapi==0.1.10 # rate limiting -hatchling==1.31.0 # build backend (needed for pip install) -``` - -### Notes - -- Initial DB schema created via `init_db()` (`create_all`) — Alembic tracks future changes -- IP logging records: account_create, login, login_fail, group_create, node_announce -- Rate limiting active on /v1/users/register and /v1/users/login (slowapi) -- Revocation, CSAM hash matching, moderation deferred to Phase 5 - -### Phase 3 scope - -**In scope:** -- All POC endpoints from Spike 2+6, production-ready -- PostgreSQL via SQLAlchemy async + Alembic migrations -- IP logging (creation, login, group events) — 1-year retention, legal compliance -- Access token (JWT, 1h) + refresh token (30d, stored hashed) -- Rate limiting on auth endpoints -- Hub config file `/etc/meshbay/hub.toml` or env vars -- HTTPS via Caddy + auto Let's Encrypt on meshbay.org -- Systemd service with restart-on-failure - -**Deferred to later:** -- Revocation push (WebSocket signaling to nodes) -- CSAM hash matching (NCMEC/IWF integration) -- Moderation flow (blocklist + takedown) -- MHP federation -- RPM/DEB packaging - -### Testing strategy - -- Unit tests with SQLite in-memory (`aiosqlite`) — no PostgreSQL needed locally -- API tests via `httpx.AsyncClient` + `ASGITransport` — no network -- All tests run in the existing `.venv` after adding Phase 3 deps - ---- - -## Phase 4 — Integration & Web Client ✅ DONE - -**Goal:** end-to-end working product in a browser: login via hub, discover groups, -browse files on a node, download and stream public content. - -### Architecture decision - -Browsers cannot make raw TCP connections — the node must speak HTTP. -- Node adds an **HTTP file API** (port 19001) serving public content via standard `fetch()` -- Private content (GEK decrypt in browser) deferred to Phase 5 (requires WebCrypto + ChaCha20 WASM) -- Hub serves the web application at `meshbay.org/app/` - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 4.1 | Node HTTP file API | `meshbay_node/transport/http_server.py` | ✅ 7/7 tests | -| 4.2 | Hub web app (HTML/JS) | `meshbay_hub/api/webapp.py` + `static/app.js` | ✅ live on meshbay.org | -| 4.3 | Group listing endpoint | `meshbay_hub/api/groups.py` GET /v1/groups | ✅ | -| 4.4 | End-to-end integration test | local node ↔ hub ↔ browser | ✅ smoke test | -| 4.5 | HLS streaming (public) | node `/hls/{id}/playlist.m3u8` + `.ts` via ffmpeg | ✅ included in 4.1 | - -**Total: 47/47 tests. https://meshbay.org live with web client.** - -### Phase 4 deployment - -- `https://meshbay.org/` — web app (HTML/JS SPA) -- `https://meshbay.org/app.js` — JS client -- `https://meshbay.org/v1/groups` — public group listing (no auth) -- Node HTTP API (port 19001): `/index`, `/file/{id}`, `/hls/{id}/*.m3u8`, `/hls/{id}/*.ts` -- HLS streaming uses ffmpeg for on-the-fly segmentation -- Public content: no auth for index, auth required for chunks -- Private content (GEK decrypt in browser): deferred to Phase 5 - -### Dependencies added in Phase 4 - -``` -# Node (runtime) -ffmpeg (system package) — HLS segmentation via subprocess -``` - -### Phase 4 scope - -**In scope (public content only):** -- Node HTTP API: serve public Mesh Group Index (JSON) + file chunks (binary) -- Hub web app: login, group discovery, file browser, download link -- HLS basic streaming: node segments video on-the-fly, browser plays natively -- JWT auth passed as query param or header to node HTTP API - -**Deferred to Phase 5:** -- Private group content in browser (requires ChaCha20-Poly1305 via WASM) -- Chat UI -- Multi-group node -- Android client - ---- - -## Phase 5 — QUIC, Federation, Moderation ✅ DONE (Mobile deferred) - -**Goal:** full decentralization, mobile support, community infrastructure. - -| # | Component | Notes | -|---|---|---| -| # | Component | File(s) | Status | -|---|---|---|---| -| 5.1 | QUIC transport (MNP v2) | `transport/quic_server.py` + `quic_client.py` | ✅ 3/3 tests | -| 5.2 | MHP federation | `meshbay_hub/api/federation.py` | ✅ GET/POST /mhp/* | -| 5.3 | Mesh Relay | `meshbay_hub/api/relay.py` | ✅ register+list+approve | -| 5.4 | Android client | — | ⏳ deferred (different tech) | -| 5.5 | Content replication | — | ⏳ deferred | -| 5.6 | iOS client | — | ⏳ after Android | -| 5.7 | Revocation push | `api/revocation.py` + `node/revocation.py` | ✅ 3/3 tests | -| 5.8 | CSAM hash matching | `meshbay_hub/csam.py` | ✅ CSAMChecker + admin API | -| 5.9 | Moderation | `api/moderation.py` | ✅ 6/6 tests | -| 5.10 | RPM/DEB packaging | `packaging/` | ✅ spec + control + systemd | - -**Total: 59/59 tests.** - -### Phase 5 bugs fixed - -- **QUIC**: `asyncio.Event` race condition in client recv loop (quic_event_received - overwrote `_stream_events[0]` created by `_recv`). Fixed with `asyncio.Queue` - (no shared mutable state between coroutines). -- **QUIC**: `verify_peer` parameter renamed to `verify_mode` in aioquic 1.3.0. -- **QUIC**: `connect()` returns protocol directly (not `(transport, proto)` tuple). - -### Phase 5 dependencies added - -``` -aioquic==1.3.0 # QUIC transport (Cloudflare-maintained) -websockets # hub→node revocation push -``` - -### Deferred to future - -- Android/iOS client (Kotlin/Flutter — different tech stack, dedicated effort) -- Content replication (node-to-node) → Phase 6 -- MHP federation persistence (currently in-memory) → Phase 6 - ---- - -## Phase 6 — Chat, Multi-group, Federation persistence, Replication ✅ DONE - -**Goal:** complete the product with group chat (Double Ratchet), multi-group node support, -persistent MHP federation, content replication, and browser private group decryption. - -### Milestones - -| # | Component | File(s) | Status | -|---|---|---|---| -| 6.1 | Double Ratchet chat | `meshbay_common/ratchet.py` | ✅ 11/11 tests | -| 6.2 | Multi-group node | `meshbay_node/config.py` [[groups]] | ✅ | -| 6.3 | MHP federation persistence | `FederatedGroup` + `SwarmSource` DB tables | ✅ | -| 6.4 | Content replication | `node/replication.py` + hub `/v1/swarm/*` | ✅ | -| 6.5 | Browser private group | `webcrypto.py` + `static/crypto.js` | ✅ 4/4 tests | -| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | -| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | -| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` + `QE/demo-v2/` (non versionné) | ✅ testés | -| 6.9 | QUICKSTART réécrit | `QUICKSTART.md` | ✅ | -| 6.6 | Dérivation clés depuis password | `meshbay_common/keyderive.py` + `static/keyderive.js` | ✅ 7/7 tests | -| 6.7 | Bundle clés chiffré (web) | Hub: `keypair_bundle` field + migration Alembic | ✅ | -| 6.8 | Scripts démo opérationnels | `QE/demo-v1/` (non versionné) | ✅ testés | -| 6.9 | QUICKSTART réécrit | `QUICKSTART.md` | ✅ | - -**Total: 81/81 tests.** - ---- - -## Phase 7 — Node v2: production, streaming, chat ✅ DONE - -**Goal:** multi-group node, Sender Keys chat, QUIC 0-RTT, jti denylist push, HLS streaming. - -Commit: fc56585 — 26 files, +2155/−159 lines, 109 tests. - -See `devel-phases-next.md` for details. - ---- - -## Phase 8 — Hub v2: admin, federation, security ✅ DONE - -**Goal:** admin roles, email encryption, refresh token rotation, rate limiting, healthcheck. - -Commit: 46918ec — 20 files, +508/−90 lines, 117 tests. -Deployed to meshbay.org. All security review items S1/S2/S5 resolved. - -See `devel-phases-next.md` for details. - ---- - -## Phase 9 — Web client: WebRTC transport + core SPA ✅ DONE - -**Goal:** browser connects P2P to a node behind residential NAT via WebRTC DataChannel. -Full SPA: login, groups, file browser, download, video playback, chat, i18n, dark/light. - -### Milestones - -| # | Component | Status | -|---|---|---| -| 9.1–9.5 | WebRTC DataChannel spike + E2E NAT validation | ✅ | -| 9.6–9.12 | Preact SPA (login, groups, files, video, chat, i18n, settings) | ✅ | -| 9.13 | Tests: 132 passing | ✅ | -| 9.14–9.16 | Performance: pipelining, binary wire format, I/O reduction | ✅ | -| 9.17 | Large file download: File System Access API (stream to disk) | ✅ | - -**Total: 132/132 tests. Deployed to meshbay.org + Orange node (2026-08-11).** - -### Key technical decisions - -- **Transport:** WebRTC DataChannel (aiortc on node) — browsers can't use QUIC for NAT traversal -- **Wire format:** length-prefixed msgpack, binary chunk fields (no base64) -- **UI:** Preact + htm ESM (vendored, no build step, no CDN, no npm) -- **Crypto:** WebCrypto SubtleCrypto AES-256-GCM for E2E chunk decryption in browser -- **Large files:** File System Access API (`showSaveFilePicker`) — stream to disk, ~8 MB RAM -- **Indexer:** 2s debounce + path-based dedup for file copy events - -### NAT traversal validated - -Two ISPs (SFR + Orange residential NAT), Chrome + Firefox, IPv4 STUN + IPv6 direct. -No TURN relay needed. See `devel-phases-next.md` for detailed test matrix. - -### QE deployment state (2026-08-11) - -- **Hub (meshbay.org):** running as `meshbay-hub.service`, DB has 3 users - (admin, cbesson, grenet), 1 group (`d3bbd90b`), `admin_usernames = ["admin"]` -- **Node (Orange host via `ssh cbesson@localhost -p 2222`):** running as - `nohup .venv/bin/python3 QE/demo-v3/run_node_simple.py`, user grenet, - connected via WS to hub, WebRTC + QUIC dual transport -- **Credentials:** `QE/demo-v3/creds.json` (not versioned) -- **Shared dir on node:** `~/meshbay/QE/demo-v3/shared/` -- All 3 users password: see creds.json - -### Dependencies added - -- `aiortc>=1.9` in meshbay-node (WebRTC DataChannel) -- `preact` + `htm` vendored as `static/vendor/htm-preact.js` (ESM, ~3 KB gzipped) - ---- - -## Conventions - -- Commits: `feat(node):`, `fix(hub):`, `chore(common):`, `docs:`, `test(node):` -- Branch per feature/fix, merge to `main` (when remote configured) -- **Always close test UFW ports after any spike on meshbay.org** -- **Never commit key material** (keystore.enc, hub_private.pem, *.key, node_state.json, unlock.key) -- meshbay.org is internet-facing: only run known-safe services, close ports after tests |