path: root/docs/meshbay-draft-v1.md

# MeshBay — Architecture Draft v1

> Status: preliminary draft — many points still open, marked [TBD]

---

## 1. Project Overview

MeshBay is a decentralized, peer-to-peer platform for file sharing, video streaming, and group messaging. It combines identity federation (via Mesh Hubs) with truly peer-to-peer data exchange (via Mesh Nodes), aiming to be resilient, censorship-resistant, and user-friendly.

**Core principles:**
- Data never transits through a central server — only identity and routing do
- End-to-end encryption for all private content (files, indexes, messages)
- The node operator is the legal host and is fully responsible for their content
- The hub is a lightweight registrar, not a content host or indexer
- Open source, self-hostable at every level

**Domain:** meshbay.org

---

## 2. Terminology

| Term | Role |
|---|---|
| **Mesh Hub** | Identity authority and group registry server |
| **Mesh Node** | Local program on the host user's machine |
| **Mesh Client** | Web browser or Android app (end user) |
| **Mesh Relay** | Community-operated TURN fallback relay |
| **MNP** | Mesh Node Protocol — P2P protocol between nodes and clients |
| **MHP** | Mesh Bay Hub Protocol — inter-hub federation protocol |
| **GEK** | Group Encryption Key — symmetric key for group content |
| **Mesh Directory** | Public registry of groups (hub level) |
| **Mesh Group Index** | Encrypted file listing for a group (node level) |

---

## 3. System Components

### 3.1 Mesh Hub

A lightweight server acting as a registrar. It is intentionally kept minimal to reduce legal exposure and operational burden.

**What the hub stores:**
- User accounts: username, hashed email, `PK_user` (public key fingerprint), hub ID, status
- Group registry: name, `PK_group`, hosting node address, visibility, member list with encrypted GEK bundles
- Revocation lists (users and groups)
- Registered peer hubs (explicit allowlist — no auto-discovery)

**What the hub never stores:**
- File content or metadata
- Private group indexes
- Message content
- Node IP addresses (handled by ephemeral signaling service)

**Hub interactions — when is it called?**

| Event | Hub load | Frequency |
|---|---|---|
| Account creation | Hash credential, store PK | Once |
| Login | Verify credentials, issue signed JWT | Per session (~30-day validity) |
| Group creation | Register name, PK_group, node | Once per group |
| Member add/remove | Store/remove encrypted GEK bundle | On admin action |
| Group discovery | Return node address + PK_node + GEK bundle | Per initial access |
| NAT signaling | Relay a few WebSocket messages (<1 KB) | Per new P2P connection |
| Public search | Delegate query to nodes at request time | On demand |
| MHP federation sync | Exchange public directory updates | Background, periodic |
| Revocation | Issue signed revocation token | Rare |

**The hub is never in the data path after initial connection setup.**

**JWT as offline passport:**
The hub issues a JWT signed with its Ed25519 private key. Nodes verify this JWT locally using the hub's known public key — no hub roundtrip required per request. JWT validity: ~30 days.

**Tech stack:**
- Language: Python
- Framework: FastAPI + Uvicorn
- Database: PostgreSQL + SQLAlchemy + Alembic
- Deployment: behind Apache reverse proxy (ProxyPass)
- Authentication: own system (JWT signed with Ed25519, no OAuth dependency)

**Account creation:** [TBD] — email only at first, phone number associable later. Via Android app, both collected by default. Fusionable accounts.

### 3.2 Mesh Node

A local program running on the host user's machine. The node is the actual host of all content.

**Responsibilities:**
- Watch and index shared directories (Mesh Group Index)
- Serve files and video streams to group members
- Manage all cryptographic keys locally (keystore, password-protected)
- Handle P2P connections and NAT traversal
- Run the MNP protocol
- Host the Python module sandbox
- Serve the local web UI (localhost)
- Receive and redistribute ephemeral video from mobile [future]

**Platform:** Linux primary, cross-platform from the start (Windows/macOS). Python ensures portability.

**Tech stack:**
- Language: Python (primary), Rust extensions only if strictly necessary for hot paths
- QUIC: `aioquic`
- ICE/STUN: `aioice`
- WebRTC (future): `aiortc`
- Crypto: `cryptography` (PyCA, OpenSSL-backed, hardware-accelerated)
- Serialization: `msgpack`
- Compression: `zstandard` (zstd)
- File watching: `watchdog`
- Local DB: SQLite
- Local web UI: served by the node on localhost (port [TBD])

**Node pairing with mobile:** QR code from local web UI [future].

### 3.3 Mesh Client

Web browser or Android app. Consumes content from the node; manages account via the hub.

**Hub-side operations (via hub):**
- Account creation and login
- Public group search and discovery
- Group membership management

**Node-side operations (direct P2P):**
- File browsing (Mesh Group Index)
- Message feed reading (with attachments, Signal-like)
- File download
- Video streaming (VOD)
- [Future] Ephemeral video feed

**Client modes** [to be designed]:
- Explorer mode: browse files in a group
- Feed mode: message thread with attachments
- Hub/Node UI split to be defined

### 3.4 Mesh Relay

Community-operated TURN relay. Used only as last-resort fallback when all P2P connection methods fail. Traffic is always E2E encrypted — the relay sees only opaque QUIC packets and cannot read content.

Not operated by meshbay.org. A protocol for relay registration with hubs is [TBD].

---

## 4. Group Model

Groups are the core organizational unit.

| Parameter | Options |
|---|---|
| Visibility | Public / Private |
| Join policy | Open / On request / By invitation only |
| Admin | The hosting node operator (legal host) |

A public group functions like a themed forum: shared files, message thread, member list. It can be open entry, request-based, or invitation-only regardless of its public visibility.

A private group's content (files, index, messages) is always E2E encrypted with the GEK. Only members with the GEK can decrypt anything.

**Group addressing** [TBD]:
```
meshbay.org/u/username/groupname    — public group via hub
meshbay.org/g/groupname             — direct public group
group://<PK_group_fingerprint>@<node_addr>   — hub-less direct access
```

---

## 5. Cryptographic Architecture

### 5.1 Key Hierarchy

```
User Identity Key      Ed25519     Signing, authentication
User Exchange Key      X25519      Key agreement
Group Identity Key     Ed25519     Group metadata signing (held by admin node)
Group Encryption Key   ChaCha20    Content and index encryption (symmetric, 256-bit)
Session Keys           X25519/HKDF Perfect forward secrecy per P2P connection
```

All private keys are stored exclusively on the node (or client device), in a password-protected local keystore. The hub never sees any private key.

### 5.2 GEK Management

**Group creation:**
1. Admin node generates GEK (ChaCha20-Poly1305, 256-bit, CSPRNG)
2. GEK is encrypted for each member using X25519 key agreement + HKDF
3. Encrypted GEK bundles stored on hub (or node — [TBD])

**Member addition:**
- GEK encrypted with new member's `PK_user` and distributed

**Member revocation:**
- Admin node generates a new GEK
- Re-encrypts for all remaining members
- New content encrypted with new GEK
- Former member retains ability to decrypt previously received content (acceptable trade-off — full re-encryption not planned)

### 5.3 On-the-Fly Encryption for File Transfer

Files are stored in plaintext on the host's disk. The node encrypts at read time before transmission.

```
Disk (plaintext) → [Node] → zstd compress → GEK encrypt (per-chunk) → QUIC session → [Client] → QUIC decrypt → GEK decrypt → plaintext
```

**Chunking strategy:**
- Chunk size: 1 MB (amortizes AEAD overhead, allows seeking)
- Per-chunk key derivation:
  `chunk_key = HKDF(GEK, "file:" || blake3(file) || "chunk:" || index)`
- Each chunk independently decryptable (enables video seeking)
- Compress before encrypt (zstd compression is useless after encryption)

**Chunk authentication:**
Each chunk (or batch) is signed with the node's Ed25519 key. The client verifies before decryption. Prevents data injection by a compromised relay.

### 5.4 Transport Security

- Primary protocol: **QUIC** (TLS 1.3 integrated, UDP-based, multiplexed)
- Per-connection session keys via X25519 ECDH + HKDF
- The QUIC layer is independent from the GEK application layer — two independent encryption layers

### 5.5 Chat Encryption

Group messaging uses the **Double Ratchet algorithm** (as in Signal):
- Forward secrecy and break-in recovery per message
- Each message independently encrypted
- Implementation: existing Python or Rust library [TBD]

---

## 6. Network and Connectivity

### 6.1 NAT Traversal — Attempt Order

```
1. IPv6 available on both sides          → direct connection, no NAT issue
2. UPnP / NAT-PMP on router             → node opens port automatically
3. ICE + STUN / UDP hole punching        → works for ~80-85% of cases
4. Mesh Relay (TURN fallback)            → community-operated, E2E encrypted traffic
```

**Signaling** (steps 3/4): coordinated via hub WebSocket, <1 KB per attempt, stateless after connection established.

**Step 4 coverage:** ~15-20% of connections (symmetric NAT on both sides, CGNAT). The relay sees only encrypted QUIC packets.

### 6.2 MNP — Mesh Node Protocol

Application-level protocol over QUIC. Defined blocks:

- **Handshake**: key exchange, group membership verification (JWT presentation)
- **Index sync**: encrypted delta Mesh Group Index on connection
- **File transfer**: chunk request/response with hash verification
- **VOD streaming**: HLS/DASH segments, encrypted per-segment with GEK-derived keys
- **Messaging**: Double Ratchet messages encapsulated in MNP frames
- **[Future] Ephemeral stream**: `ephemeral_stream` type with TTL metadata

### 6.3 Public Content Delivery

Public files are identified by their `blake3` hash. Multiple nodes can serve the same file:

1. Node A has public file X (hash H)
2. Any node that obtains X and chooses to mirror it registers with the hub: "I serve hash H"
3. Hub maintains: `{ blake3_hash → [node_A, node_B, ...] }`
4. Client requests X → hub returns source list → client fetches in parallel chunks from multiple nodes

**Public content transport:** TLS only (no GEK). Content signed with the original node's Ed25519 key for authenticity verification by clients, even when served from a mirror. Door left open for GEK on "registered-users-only public" groups in a future revision.

---

## 7. Indexes

### 7.1 Mesh Directory (hub level)

Public registry of groups. Exchanged between hubs via MHP.

Format: msgpack, signed by hub's Ed25519 key.

Fields per entry: group name, `PK_group`, hosting hub, description, content type tags, join policy.

### 7.2 Mesh Group Index (node level)

File listing for a group. Generated and maintained by the hosting node.

Format: msgpack → zstd compressed → GEK encrypted (private groups) or plaintext signed (public groups).

Entry structure:
```python
{
  "id":        "<blake3_hash>",
  "name":      "filename.mkv",
  "path":      "Movies/2024/",        # relative to shared directory
  "size":      4294967296,
  "type":      "video",               # video | audio | image | document | archive | other
  "duration":  7245,                  # seconds, for media
  "thumb_hash":"<blake3>",            # thumbnail hash (thumbnail also GEK-encrypted)
  "added_at":  1720000000
}
```

**Delta updates:** each update carries `{base_version, additions, deletions}` — no full re-encryption on every change.

**Transit:** nodes push index deltas to connected members on change. Members pull full index on first connection. Hub stores no index content — only the node address for routing.

### 7.3 Search

**Private groups:** search is entirely local on the client device. The client maintains a local encrypted cache of all indexes for groups it belongs to. No network call, no hub involvement, instant results.

**Public groups:** client queries node(s) directly at request time. Hub provides routing (which node hosts which group) but performs no content lookup itself.

**Hub web UI search:** delegates query to relevant nodes at request time. Hub stores nothing from this interaction. In-memory micro-cache of results: **60-second TTL maximum, RAM only, never written to disk, public content only.** This qualifies as technical caching (EU DSA Article 13) and does not constitute indexing.

---

## 8. Hub Federation (MHP)

### 8.1 Hub Hierarchy

```
Root Hub (meshbay.org)
    ├── Full Hub (self-hosted, CA-delegated)
    │     └── issues user credentials, manages its own groups
    │     └── can federate with other Full Hubs via MHP
    └── Mirror Hub
          └── hosts public Mesh Directory only (no user accounts)
```

A Full Hub receives a certificate signed by the Root Hub (or a parent Full Hub) proving its authority. Clients verify the chain. A Mirror Hub can only replicate public directory data.

### 8.2 MHP Design Principles

- Explicit peer selection: each hub maintains an allowlist of trusted peer hubs
- No automatic hub discovery
- Exchanged data: Mesh Directory (public groups), revocation lists, cross-hub user credentials
- Cross-hub authentication: user from Hub A presents JWT signed by Hub A; Hub B verifies using Hub A's public key (fetched once on first interaction, cached)

### 8.3 Cross-Hub Client Access

Client from Hub A accessing a group on Hub B:
1. Hub A's public directory or direct link leads client to Hub B
2. Client presents Hub A JWT to Hub B directly
3. Hub B verifies JWT signature using Hub A's public key
4. Hub B issues a short-lived local token for this session
5. Client proceeds to node as normal

---

## 9. Moderation

### 9.1 Public Content

```
Report #1  → automatic suspension of public access to content
           → node operator notified
One republication allowed
Report #2  → escalated to hub moderators
Confirmed  → group revoked on local hub
           → revocation propagated to federated hubs via MHP
```

Mechanism: blake3 hash of content added to hub blocklist. Node receives signed revocation notice and cuts public access.

### 9.2 CSAM

Hash matching against NCMEC/IWF database on all public content at registration time. Participation demonstrates good faith and significantly reduces legal exposure. No scanning of private/encrypted content.

### 9.3 Copyright

DMCA/legal notice framework (takedown on notification). No automated technical blocking — too complex, too many false positives (fair use, regional variations). Hub can revoke on confirmed legal request.

### 9.4 Private Content

Not directly moderatable (E2E encrypted by design). Only action available: revoke user or group at hub level on formal legal request. Hub issues a signed revocation token that all group members' nodes can verify.

---

## 10. Python Module System

The node can load extension modules (Python) that run in a sandboxed subprocess.

**Module manifest** (declared capabilities):
```python
{
  "name": "group-chat",
  "version": "1.0.0",
  "permissions": ["read_index", "send_message", "receive_events"]
}
```

**Available APIs (restricted):**
- `read_index()` — read current group index (read-only)
- `send_message(content)` — post a message to the group thread
- `receive_events(handler)` — subscribe to group events (new file, new message)

**Not available:**
- Arbitrary network access
- Filesystem access outside the group context
- System calls

**First official module:** group chat thread (Signal-like, with attachments). Bundled with node.

---

## 11. Legal Framework

**Node operator:** primary legal host of content. Fully responsible for what they share. Node software clearly communicates this at setup.

**Hub operator:** registrar, not content host. Stores minimal PII. Operates takedown mechanism. Participates in CSAM hash matching. Analogous to a domain registrar in legal exposure terms.

**Protocol/software author:** protected by substantial non-infringing uses. No active facilitation of infringement.

**Hub data minimization:**
- Email stored hashed after verification [TBD]
- No IP address logging (or auto-deletion after 24h)
- No content metadata stored
- Node current address managed by ephemeral signaling service only

---

## 12. Future Features (noted, not designed)

- **Content replication between nodes:** node-to-node, admin-authorized, no hub involvement
- **Mobile video push:** mobile films → pushes to hosting node → distributed as ephemeral stream with TTL to group members. MNP `ephemeral_stream` type reserved.
- **Mesh Relay registration protocol:** community TURN relays registered with hubs
- **Node mobile pairing:** QR code from local web UI
- **Multi-source download:** parallel chunk fetching from multiple nodes for same public file (swarm)
- **iOS client**
- **At-rest encryption on node:** optional, for nodes deployed on remote servers

---

## 13. Open Questions [TBD]

1. **GEK bundle storage:** on hub or on node only? Hub = easier discovery; node only = more decentralized
2. **Group address scheme:** final URL format
3. **Hub local web UI scope for V1:** config only, or also group browsing?
4. **Account creation:** email only to start, phone associable — confirm
5. **Chat implementation:** bundled module or core feature?
6. **QUIC library maturity:** `aioquic` production readiness assessment needed
7. **Double Ratchet library:** identify best Python implementation
8. **Relay registration protocol:** design when community relays are introduced
9. **Cross-hub directory exchange:** frequency, conflict resolution
10. **Node port for local web UI:** to assign
11. **JWT expiry and refresh strategy**
12. **Keystore format and unlock mechanism on node startup**