diff options
Diffstat (limited to 'docs/meshbay-draft-v2.md')
| -rw-r--r-- | docs/meshbay-draft-v2.md | 550 |
1 files changed, 550 insertions, 0 deletions
diff --git a/docs/meshbay-draft-v2.md b/docs/meshbay-draft-v2.md new file mode 100644 index 0000000..493e877 --- /dev/null +++ b/docs/meshbay-draft-v2.md @@ -0,0 +1,550 @@ +# MeshBay — Architecture Draft v2 + +> Status: preliminary draft — open points marked [TBD] +> Changes from v1: IP logging (legal), protocol versioning, hardware sizing, JWT strategy, keystore proposals, chat as core, relay moved to future, hub mirror future, GEK clarified, port 18000, lazy admin keystore. + +--- + +## 1. Project Overview + +MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), designed to be resilient, censorship-resistant, and user-friendly. + +**Core principles:** +- Data never transits through a central server — only identity and routing do +- End-to-end encryption for all private content (files, indexes, messages) +- The node operator is the legal host and is fully responsible for their content +- The hub is a lightweight registrar, not a content host or indexer +- Open source, self-hostable at every level + +**Domain:** meshbay.org (configurable at compile/deploy time throughout the codebase) + +--- + +## 2. Terminology + +| Term | Role | +|---|---| +| **Mesh Hub** | Identity authority and group registry server | +| **Mesh Node** | Local program on the host user's machine | +| **Mesh Client** | Web browser or Android app (end user) | +| **Mesh Relay** | Community-operated TURN fallback relay [future] | +| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients | +| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol | +| **GEK** | Group Encryption Key — symmetric key for private group content | +| **Mesh Directory** | Public registry of groups (hub level) | +| **Mesh Group Index** | File listing for a group (node level, encrypted for private groups) | + +--- + +## 3. Protocol Versioning + +All protocols (MNP, MHP, hub REST API) carry explicit version information. + +**Format:** `MAJOR.MINOR` +- MAJOR bump: breaking change, backward incompatible +- MINOR bump: backward-compatible addition + +**Negotiation:** during handshake, both parties declare their supported version range. The highest mutually supported MINOR within the same MAJOR is used. If no common version exists, connection is refused with an explicit error. + +**Support policy:** a release supports the current MAJOR and at least the two previous MINOR versions (N-2). + +**Implementation:** a `version` field in every msgpack message header. Handshake step precedes all other exchanges. + +--- + +## 4. System Components + +### 4.1 Mesh Hub + +A lightweight server acting as a registrar. Intentionally minimal to limit legal exposure and operational cost. + +**What the hub stores:** +- User accounts: username, email (stored for account recovery — see §4.1.1), optional phone number, `PK_user`, hub ID, status, creation timestamp +- Group registry: name, `PK_group`, hosting node identifier, visibility, join policy, member list with encrypted GEK bundles (private groups only) +- Mandatory connection logs (see §4.1.2) +- Revocation lists (users and groups) +- Registered peer hubs (explicit allowlist — no auto-discovery) + +**What the hub never stores:** +- File content or metadata +- Private group indexes +- Message content +- Node current IP (handled by ephemeral signaling — see §4.1.3) + +#### 4.1.1 Account Data + +Email is kept in full (not hashed) to support: +- Account recovery (password reset) +- Legal notifications +- Abuse contact + +Phone number: optional, associable after account creation. On Android, both collected at registration. Accounts are fusionable (email + phone pointing to same account). + +Email and phone are stored encrypted at rest in the database. + +#### 4.1.2 Mandatory IP Logging (Legal Compliance) + +Legal frameworks (LCEN in France, EU e-Commerce Directive, DSA) require service providers to retain connection logs. The hub logs the following with timestamp and IP address: + +| Event | Retention | +|---|---| +| Account creation | 1 year minimum | +| Login (success and failure) | 1 year minimum | +| Group creation | 1 year minimum | +| Group join / leave | 1 year minimum | +| Group deletion | 1 year minimum | +| Revocation actions | 1 year minimum | + +Logs are stored in a separate, access-controlled log table. They are not used for any purpose other than legal compliance and are not exposed to users or operators beyond legal requests. + +#### 4.1.3 Signaling Service + +NAT traversal coordination is handled by a lightweight WebSocket endpoint, logically separate from the main hub API. It is stateless: connection state is held in memory only and discarded after P2P connection establishment (typically within seconds). No persistent storage of node IP addresses. + +**Hub interaction summary:** + +| Event | Hub crypto load | Frequency | +|---|---|---| +| Account creation | Argon2 hash, store PK | Once | +| Login | Verify password, issue JWT (Ed25519 sign) | Per session | +| Group creation | Register metadata | Once per group | +| Member add/remove | Store/remove GEK bundle | On admin action | +| Group discovery | Return node address + PK_node + GEK bundle | Per initial access | +| NAT signaling | Relay WebSocket messages (<1 KB) | Per new P2P connection | +| Public search | Delegate to nodes, 60s in-memory cache | On demand | +| MHP federation sync | Exchange Mesh Directory | Background, periodic | +| Revocation | Ed25519-sign revocation token | Rare | + +**The hub is never in the data path after connection setup. JWT verification by nodes is local (Ed25519, no hub roundtrip).** + +#### 4.1.4 JWT Strategy + +Two tokens issued at login: + +**Access token** (JWT, signed Ed25519): +- Validity: 1 hour +- Payload: `user_id`, `PK_user`, `hub_id`, `issued_at`, `expires_at`, hub-signed groups membership claim +- Presented to nodes for authentication and group access verification +- Verified locally by nodes using the hub's known public key — no hub roundtrip +- Compromise window: 1 hour maximum + +**Refresh token** (opaque, random 256-bit): +- Validity: 30–90 days [TBD exact duration] +- Stored securely on client only +- Used exclusively with the hub to obtain a new access token +- Revocable immediately by the hub (invalidates all future refreshes for this token) +- Stored server-side as a hashed value + +**Revocation flow:** hub invalidates the refresh token → next access token renewal fails → node access expires within 1 hour at most. + +**Tech stack:** +- Language: Python +- Framework: FastAPI + Uvicorn +- Database: PostgreSQL + SQLAlchemy + Alembic +- Deployment: Apache reverse proxy (ProxyPass + SSL termination) +- Authentication: own system (Ed25519 JWT, Argon2id for password hashing) +- Hub accessible via domain and directly by IP (self-signed cert warning expected for IP access; documented) + +### 4.2 Mesh Node + +A local program running on the host user's machine. The node is the actual host of all content. + +**Responsibilities:** +- Watch and index shared directories (Mesh Group Index) +- Serve files and video streams to group members +- Manage all cryptographic keys locally (encrypted keystore) +- Handle P2P connections and NAT traversal +- Run the MNP protocol +- Host the Python extension module sandbox +- Serve the local web UI (localhost:18000) +- Host the group chat (core feature) + +**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability. + +#### 4.2.1 Keystore and Unlock + +Private keys (user identity, group identity, GEK copies) are stored in a local encrypted keystore file. + +**Format:** msgpack container encrypted with AES-256-GCM, key derived from master password using Argon2id (tuned for ~1s derivation on target hardware). + +**Three unlock modes:** + +| Mode | How it works | Security level | +|---|---|---| +| **Secure (default)** | Password prompted at startup via terminal or local web UI | High | +| **Lazy file** | Password or derived key stored in `~/.config/meshbay/unlock.key` (chmod 600), read automatically at startup | Medium — acceptable for physically secure home machines. Risk documented at setup. | +| **Service (headless)** | `MESHBAY_UNLOCK_KEY` environment variable, set via systemd `EnvironmentFile=` pointing to a chmod 600 file | Medium-high — standard practice for server deployments | + +Future: OS keychain integration (libsecret/GNOME Keyring on Linux, Windows Credential Manager, macOS Keychain). + +#### 4.2.2 Hardware Sizing + +The main constraint is **upload bandwidth**, not CPU or RAM. + +| Scenario | Simultaneous users | Upload needed | CPU | RAM | +|---|---|---|---|---| +| Files + chat, minimal streaming | 10 | 20–50 Mbps | 2 cores | 512 MB | +| Active 1080p streaming (5–6 streams) | 10 | 50–80 Mbps | 2–4 cores | 1 GB | +| Mixed use | 50 | 200–300 Mbps | 4 cores | 2 GB | +| Active streaming | 50 | 400 Mbps | 4–8 cores | 2–4 GB | +| All use cases | 100 | 800 Mbps–1 Gbps | 8 cores | 4–8 GB | + +Beyond 20–30 active streaming users, a dedicated server is required. A home fiber connection (100–500 Mbps symmetric) is suitable for small groups. + +**Tech stack:** +- Language: Python (primary). Rust extension only if a specific hot path proves insufficient. +- Transport abstraction layer: `Transport` interface decouples QUIC from TCP+TLS fallback +- QUIC: `aioquic` (Cloudflare-maintained). Fallback: TCP + TLS 1.3 + HTTP/2 if QUIC proves insufficient in production +- ICE/STUN: `aioice` +- WebRTC [future]: `aiortc` +- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated AES-NI/ChaCha) +- Serialization: `msgpack` +- Compression: `zstandard` (zstd) +- File watching: `watchdog` +- Local DB: SQLite +- Local web UI: served by node on `localhost:18000` + +### 4.3 Mesh Client + +Web browser or Android app. Consumes content from nodes; manages account via hub. + +**Hub-side operations:** +- Account creation and login (Android: email + phone at registration) +- Public group search and discovery +- Group membership management + +**Node-side operations (direct P2P):** +- File browsing via Mesh Group Index +- Group chat (messages + attachments, Signal-like — core feature) +- File download +- Video streaming (VOD) +- [Future] Ephemeral video feed + +**Client modes** [to be designed]: +- Explorer mode: file browser for group content +- Feed mode: chat thread with attachments +- Hub/node UI articulation to be defined; Android app will connect to node directly as a near-term priority after account creation + +### 4.4 Mesh Relay + +**[Future feature]** Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail (~15–20% of connections). Traffic is always E2E encrypted — the relay sees only opaque QUIC packets. + +Not operated by meshbay.org. A relay registration protocol (hub-mediated) will be designed when this feature is introduced. It does not affect the current design. + +--- + +## 5. Group Model + +Groups are the core organizational unit. + +| Parameter | Options | +|---|---| +| Visibility | Public / Private | +| Join policy | Open / On request / By invitation only | +| Admin | The hosting node operator (legal host) | + +A public group functions like a themed forum: files, chat thread, member list. Join policy is independent of visibility (a public group can require approval to join). + +A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members holding the GEK can decrypt anything. + +**Group addressing:** +``` +meshbay.org/u/username/groupname — public group via hub +meshbay.org/g/groupname — public group (shorthand) +group://<PK_group_fingerprint>@<node_addr> — hub-less direct access +``` +`meshbay.org` is fully configurable throughout the codebase (constant/config file). The hub is reachable via domain or IP (IP access requires self-signed cert; browsers will warn — expected and documented behavior). + +--- + +## 6. Cryptographic Architecture + +### 6.1 Key Hierarchy + +``` +User Identity Key Ed25519 Signing, authentication +User Exchange Key X25519 Key agreement +Group Identity Key Ed25519 Group metadata signing (held by admin node) +Group Encryption Key ChaCha20 Private content and index encryption (symmetric, 256-bit) +Session Keys X25519/HKDF Perfect forward secrecy per P2P connection +``` + +All private keys stored exclusively on the node (or client device) in the encrypted keystore. The hub never sees any private key. + +### 6.2 GEK Management + +**Scope:** GEK applies to private groups only. Public groups use TLS transport only (no application-layer encryption). + +**Group creation:** +1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG) +2. GEK encrypted for each member via X25519 key agreement + HKDF +3. Encrypted GEK bundles stored on hub (opaque blobs — hub cannot decrypt them; charge is negligible: ~200–400 bytes per member per group) + +**Storing on hub rationale:** members can retrieve their GEK bundle even when the node is offline. Hub exposure is minimal — it stores ciphertext it cannot read. + +**Member addition:** +- GEK encrypted with new member's `PK_user` and uploaded to hub + +**Member revocation:** +- Admin node generates new GEK +- Re-encrypts for all remaining members, uploads new bundles +- New content encrypted with new GEK from this point +- Former member can still decrypt previously received content (acceptable trade-off — full retroactive re-encryption not planned) + +### 6.3 On-the-Fly Encryption for File Transfer + +Files are stored in plaintext on the host's disk. The node encrypts at read time. + +``` +Disk (plaintext) → zstd compress → GEK encrypt (per-chunk) → QUIC session → Client → QUIC decrypt → GEK decrypt → plaintext +``` + +**Chunking:** +- Chunk size: 1 MB (amortizes AEAD overhead; enables seeking) +- Per-chunk key derivation: `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)` +- Each chunk independently decryptable → enables VOD seeking +- Compress before encrypt (compression is ineffective on ciphertext) + +**Chunk authentication:** each chunk signed with the node's Ed25519 key. Client verifies before decryption. Prevents data injection by compromised relay. + +**Encryption optimization:** +- `cryptography` (PyCA) uses OpenSSL under the hood, bypasses Python GIL for crypto ops +- ChaCha20-Poly1305: ~500 MB/s on hardware without AES-NI; AES-256-GCM: >2 GB/s with AES-NI +- For typical home node (50 Mbps upload = 6 MB/s), encryption is not the bottleneck +- For high-concurrency scenarios: asyncio pipeline (read → compress → encrypt → send) without loading full files into memory +- GEK-derived chunk keys computed in batch at transfer start, not per-chunk + +### 6.4 Transport Security + +- Primary: **QUIC** (TLS 1.3 integrated, UDP, multiplexed streams) +- Fallback: **TCP + TLS 1.3 + HTTP/2** (same application protocol, lower performance) +- Transport interface abstracted in code — swappable without protocol changes +- Per-connection session keys via X25519 ECDH + HKDF (independent of GEK layer) + +### 6.5 Chat Encryption + +Group chat is a **core feature** (not an extension module). Uses the **Double Ratchet algorithm** (as in Signal): +- Forward secrecy and break-in recovery per message +- Each message independently encrypted +- Attachment files: encrypted with the current Double Ratchet message key, hash included in message +- Python implementation: [TBD — evaluate existing libraries] + +--- + +## 7. Network and Connectivity + +### 7.1 NAT Traversal — Attempt Order + +``` +1. IPv6 available on both sides → direct connection +2. UPnP / NAT-PMP on router → node opens port automatically +3. ICE + STUN / UDP hole punching → ~80–85% success rate +4. Mesh Relay (TURN) → [future feature] +``` + +Without step 4, ~15% of connections between symmetric-NAT peers will fail. This is documented behavior until Mesh Relay is implemented. + +Signaling (step 3): coordinated via hub WebSocket endpoint, <1 KB per attempt, no persistent state. + +### 7.2 MNP — Mesh Node Protocol + +Application-level protocol over QUIC (or TCP+TLS fallback). All messages carry a `version` field. + +**Defined message types:** + +| Type | Description | +|---|---| +| `handshake` | Key exchange, JWT presentation, version negotiation | +| `index_sync` | Encrypted Mesh Group Index delta | +| `file_request` | Request chunk(s) of a file by hash + chunk index | +| `file_chunk` | Chunk data + signature | +| `stream_segment` | HLS/DASH segment (VOD), encrypted with GEK-derived key | +| `chat_message` | Double Ratchet encrypted message frame | +| `chat_attachment` | Attachment metadata + key; data transferred as file chunks | +| `ephemeral_stream` | [reserved, future] Ephemeral video with TTL metadata | + +### 7.3 Public Content Delivery — Swarm + +Public files identified by `blake3` hash. Multiple nodes can serve the same file: + +1. Any node that has a public file and chooses to mirror it registers: `{ hash → node_address }` with the hub +2. Hub maintains a source table: `{ blake3_hash → [node_A, node_B, ...] }` +3. Client requests file → hub returns source list → client fetches chunks in parallel from multiple nodes +4. Integrity verified by blake3 hash on each chunk + +**Transport:** TLS only for public content (no GEK). Content signed with the original node's Ed25519 key — clients verify authenticity even when served from a mirror. + +--- + +## 8. Indexes + +### 8.1 Mesh Directory (hub level) + +Public registry of groups, exchanged between hubs via MHP. + +Format: `msgpack`, signed with hub's Ed25519 key, carries `version` field. + +Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy, creation date. + +### 8.2 Mesh Group Index (node level) + +File listing for a group. Generated and maintained by the hosting node. + +Format: `msgpack` → `zstd` → GEK-encrypted (private groups) or plaintext + Ed25519 signature (public groups). + +Entry structure: +```python +{ + "version": 1, + "id": "<blake3_hash>", + "name": "filename.mkv", + "path": "Movies/2024/", + "size": 4294967296, + "type": "video", # video | audio | image | document | archive | other + "duration": 7245, # seconds, for media + "thumb_hash": "<blake3>", # thumbnail also GEK-encrypted + "added_at": 1720000000 +} +``` + +Delta updates: `{ base_version, additions, deletions }` — no full re-encryption on each change. + +Transit: nodes push index deltas to connected members on change; members pull full index on first connection. Hub stores no index content. + +### 8.3 Search + +**Private groups:** entirely local on the client device. Client maintains a local encrypted cache of all group indexes it has received. No network call, no hub involvement, instant. + +**Public groups:** client queries nodes directly at request time. Hub provides routing only. + +**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this. In-memory micro-cache: **60-second TTL, RAM only, never persisted to disk, public content only.** Qualifies as technical caching under EU DSA Article 13 — not indexing. + +--- + +## 9. Hub Federation (MHP) + +### 9.1 Hub Hierarchy + +``` +Root Hub (meshbay.org) + ├── Full Hub (self-hosted, delegated CA) + │ └── issues user credentials, manages own groups + │ └── federates with other Full Hubs via MHP + └── Mirror Hub + └── hosts public Mesh Directory only (no user accounts, no key issuance) +``` + +A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub). Mirror Hubs can only replicate public directory data. Promotion/demotion is possible without breaking the protocol. + +### 9.2 MHP Design + +- Explicit peer selection: each hub maintains an allowlist of trusted peers +- No automatic hub discovery +- Exchanged: Mesh Directory (public groups), revocation lists, cross-hub user authentication data +- All MHP messages carry `version` field + +### 9.3 Cross-Hub Client Access + +1. Client (Hub A user) discovers a group on Hub B via Mesh Directory or direct link +2. Client presents Hub A JWT directly to Hub B +3. Hub B verifies JWT using Hub A's public key (fetched once, cached) +4. Hub B issues short-lived local session token +5. Client connects to node as normal + +--- + +## 10. Moderation + +### 10.1 Public Content + +``` +Report #1 → automatic suspension of public access + → node operator notified +One republication allowed +Report #2 → escalated to hub moderators +Confirmed → group revoked on local hub + → revocation propagated to federated hubs via MHP +``` + +Mechanism: `blake3` hash added to hub blocklist. Signed revocation token sent to node. + +### 10.2 CSAM + +Hash matching against NCMEC/IWF database on public content at registration time. No scanning of private/encrypted content. Participation is mandatory for hub operators and reduces legal exposure. + +### 10.3 Copyright + +DMCA/legal notice framework. Takedown on notification. No automated technical blocking (false positive risk, fair use). Hub can revoke on confirmed legal request. + +### 10.4 Private Content + +Not directly moderatable (E2E encrypted). Action available: revoke user or group at hub level on formal legal request. Hub issues Ed25519-signed revocation token verifiable by all member nodes offline. + +--- + +## 11. Python Extension Module System + +The node loads extension modules (Python) in a sandboxed subprocess. **Chat is a core built-in feature, not a module.** + +**Module manifest:** +```python +{ + "name": "my-extension", + "version": "1.0.0", + "mnp_version": ">=1.0", + "permissions": ["read_index", "send_message", "receive_events"] +} +``` + +**Available APIs:** +- `read_index()` — read current group index (read-only) +- `send_message(content)` — post to group thread +- `receive_events(handler)` — subscribe to group events + +**Unavailable:** arbitrary network, filesystem access outside group context, system calls. + +--- + +## 12. Legal Framework + +**Node operator:** primary legal host of content. Fully responsible for what they share. Node setup communicates this explicitly. + +**Hub operator (meshbay.org):** registrar, not content host. Stores minimal data. Operates takedown mechanism. Participates in CSAM hash matching. Legal exposure analogous to a domain registrar. + +**Protocol/software author:** protected by substantial non-infringing uses. + +**Hub data:** +- Email and optional phone: kept for account recovery and legal compliance +- Password: Argon2id hash, never stored in cleartext +- Connection logs: retained per legal requirements (minimum 1 year) +- Content metadata: never stored +- Node current IP: not persisted (signaling is ephemeral) + +--- + +## 13. Future Features + +- **Mesh Relay:** community TURN relays, relay registration protocol via hub, E2E encrypted traffic +- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement +- **Hub mirror (load balancing):** full hub replication (user DB, group registry, GEK bundles) for load distribution. Requires distributed DB strategy (PostgreSQL streaming replication or equivalent). Complex — design when needed. +- **Mobile video push → node:** mobile films → pushes to hosting node → ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved. +- **Node–mobile pairing:** QR code from local web UI +- **Multi-source download:** parallel chunk fetching from swarm for public files +- **iOS client** +- **At-rest encryption on node:** optional for server-deployed nodes +- **OS keychain integration for keystore unlock** + +--- + +## 14. Open Questions [TBD] + +1. **Refresh token validity:** 30 or 90 days? +2. **Group address scheme:** final URL format confirmation +3. **Double Ratchet library:** identify best Python implementation +4. **GEK bundle location for groups with mixed access** (public-restricted): hub or node? +5. **MHP federation sync frequency and conflict resolution** +6. **Hub mirror replication strategy** (when implemented) +7. **Chat attachment storage:** stored on node like regular files, or separate store? +8. **Relay registration protocol design** (when implemented) +9. **JWT payload claims:** exact fields to include for node group-access verification +10. **Argon2id parameters:** tuning for target hardware (home server vs. VPS) |