summaryrefslogtreecommitdiffstats
path: root/packages/meshbay-common/src
Commit message (Collapse)AuthorAgeFilesLines
* chore: release 0.2.00.2Christophe Besson2026-08-141-1/+1
| | | | | | | | | | | | All three packages together, as the conventions require, plus the RPM and DEB metadata and their changelogs. The tag said 0.2 while every package announced 0.1.0, which would have shipped an RPM claiming to be the reviewed build while containing a different protocol: the hub schema lost the user identity keys, tokens lost pk_user, and gek_bundle_store left the wire. Pre-1.0, a breaking change bumps MINOR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat!: identity keys per node — C4's blast radius drops to one operatorChristophe Besson2026-08-141-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One keypair was copied to every node its owner joined, so cracking the bundle on any single node yielded the identity used on all of them: their content on other operators' machines, and the ability to sign as them anywhere. That lateral reach was the part of C4 worth attacking. Each node now gets its own keypair, generated the first time its owner joins it and left with that node alone. An operator who cracks what sits on their own disk holds a key that is a stranger to every other node — and on their own node, one that unlocks nothing they did not already hold: they serve the content, the index and every byte of it by design. Nothing changes for the user. A first contact with a node already needed that operator's code, and the key is created in the same step; a second browser still recovers it from the node with the passphrase alone. Two operators can also no longer tell they host the same person by comparing keys. BREAKING, and deliberately without a compatibility path — the deployment is wiped for the next demo: - users.pk_ed25519 / pk_x25519 dropped (migration a7c31f9e40b2) - registration no longer sends or stores a key - PUT /v1/users/me/keys and regenerateKeys() gone; rotation is now `member unpin` plus a fresh code, decided on the machine that pinned it - /pubkeys returns an account id and a node's linking key. It was the directory H3 read, and nothing wraps for it any more - the pk_user JWT claim is gone That last one closed a live defect the inventory turned up: the node recorded pk_user as the uploader's identity and authorized deletion against it, so a hub issuing a token naming its own key could delete anyone's uploads on any node. Attribution now uses the key the node itself pinned. A simplification falls out. Registration generates nothing, so a scripted signup is a real account: `demo.py bootstrap` takes a wiped hub and node to a working demo with no browser, which was impossible while keys were born in one. Also fixes, found by running it on a wiped deployment: the key handed back on a join now belongs to the group the connection is for, not the group named in the invitation — an operator pairs node-wide but redeems the code while opening a group, and expects to read it. Tests: 343, including the two that state the property — a key pinned by one node is refused at another, and someone else's code does not admit it. Verified end to end against a wiped hub and node: bootstrap, pair, invite, join, download, stream, second browser, revoke. Design: docs/per-node-identity-v1.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(client): make the key backup a choice, and raise the passphrase floorChristophe Besson2026-08-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two things the multi-browser story made obvious. **The backup is now opt-out.** Keys are kept, encrypted with the passphrase, on every node whose group you join — that is what lets a second browser recover them, and it is finding C4: a PBKDF2-protected blob on other people's disks, attackable offline at the speed of PBKDF2, which is memory-light and therefore cheap on a GPU. Until now everybody paid that cost, including people who will only ever use one browser and get nothing back for it. Settings → "Use this account on other devices". Turning it off does not merely stop future uploads: the next connection to each node withdraws what that node already holds (new keypair_bundle_delete, which only ever deletes the caller's own, taken from the authenticated session and never from the message). The warning says plainly what it costs — clearing the browser then loses everything encrypted for that account, with no recovery, which is the point of choosing it. Default is on. Silent, unrecoverable key loss is worse for an ordinary user than an exposure the roadmap already tracks, but that is a judgement call and it is now visible and reversible instead of implicit. **Passphrase floor 8 → 12 characters, plus a strength estimate** shown while typing, with a refusal below ~60 bits. This number matters more here than in most applications: it is what stands between a node operator and your identity keys. It has to live in the client — with the password split (T1) the hub never sees a password and cannot enforce anything about one — so the UI says why it is asking, rather than nagging. The estimator is deliberately conservative and dependency-free: character classes and length, penalised for repetition and for the handful of patterns everyone tries. Verified against the live deployment: withdrawing the backup leaves a second browser unable to recover anything, which is exactly what it promises, and re-enabling restores it. Tests: 338. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(client): refresh the token when the node says "not a member"Christophe Besson2026-08-141-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | A member added to a group after they signed in was refused by the node, told "Not a member of this group", and had no way forward but to log out and back in. The hub bakes `groups` into the access token at login and never pushes updates, so the token said they were in nothing while the database said otherwise. This lands on every newly invited member, at their first action, and the message tells them the opposite of the truth — toto2 was a member of newdemo on the hub and read that they were not. The refusal now carries a code the client can act on (`not_a_member`) rather than prose it would have to string-match, and the SPA refreshes the access token once and retries. Refreshing re-reads membership from the database, so the retry succeeds. Once per mount: if a fresh token still says not a member, that is the truth and it gets shown. The SPA had stored a refresh token since Phase 8 and never used it. It does now. Found in a browser, doing the ordinary thing — the automated run never sees it, because e2e.py logs in after being added to the group. Tests: 233 node+common, including a handshake test that the refusal carries the code, and the full e2e run against the live deployment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(node)!: the node wraps the group key — closes H3 and M3Christophe Besson2026-08-143-3/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The invite flow fetched the invitee's pk_x25519 from the hub and wrapped the GEK for whatever came back (app.js:1466, and gek-init did the same server-side). The hub is the key directory, so a hub answering with its own key was handed the group key by an honest member following the protocol exactly. No forgery, no injection, nothing for the client to notice. That was H3. The fix is not safety numbers. Nobody reads the directory any more: - the node holds the GEK and wraps it itself, on every connection, for the X25519 key the joiner signed with their Ed25519 identity in one transcript (meshbay:join:v1), so the identity key vouches for the encryption key; - identities are bound to accounts by a one-time code the hub never sees — 40 bits, single use, one account, bounded per connection AND node-wide; - the node's own roster decides who may receive the key. Hub membership lets someone reach a node; it no longer gets them anything. A hub that invents an account and mints it a token is answered not_authorized_for_group. Safety numbers would have made substitution detectable by a human who checks, at the moment there is nothing to check against — first contact. Removing the lookup makes it impossible, and costs the user one code to pass along. M3 falls out of the same work. The daemon auto-pinned its own keystore key as admin_pk_ed25519 while the browser signs with the user identity key, so every privileged operation failed closed with a signature error that looked like a bug somewhere else; the demo only worked because a deploy script overwrote the value. Authority now comes from the roster, established locally by `operator pair`. Asking the hub for the operator's key — the obvious-looking fix — would have let the hub install itself as node administrator. BREAKING: gek_bundle_store is deleted, not gated. No member hands the node key material at all, so C5b becomes structural rather than an authorization to check. Existing stored bundles are still served, so current deployments keep working. Also: - join_policy (invite|open) is read from node.toml, never from the hub — a hub able to declare a group open would be handed its key. Unknown group ⇒ invite. - admin signatures are verified against the roster on every check, so unpinning takes effect without a restart. admin_pk_ed25519 stays readable as legacy. - two C5b tests were rewritten, deliberately: they asserted that gek_bundle_store demanded an operator signature, and the message is gone. They now assert the stronger property. The file says not to fix these tests, so this is the record of why they changed. - a slice-1 bug found while writing slice 2: connect() never passed skEdB64, so pairing would have failed at runtime with no test able to catch it. Tests: 152 node+common here, including an end-to-end DataChannel run where a member who has never held the group key redeems a code in the pre-proof window and receives the key wrapped for a key only they can open. Design: docs/invite-pairing-v1.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(mnp): unified handshake with mutual authenticationChristophe Besson2026-08-131-0/+203
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Phase 11.5.4/5/7/8 — findings C6 (WebRTC half), C3, L4, M1, M9. New meshbay_common/handshake.py is the single implementation of authorization and proof: JWT verify, scope, denylist, mandatory group_id, membership, hosting. The handshake previously existed three times over and only the newest copy enforced the GEK proof. C3 — mutual authentication. Authentication ran one way: the client proved itself, the node proved nothing. handshake_ack.node_pk was never verified against anything and per-chunk signatures had been dropped in Phase 9.15, so a peer that had hijacked signaling (C2) or been substituted by the hub could accept the client's proof, ignore it, and serve a forged index, forged chat history and a forged is_node_admin flag. The client now sends a nonce; the node answers with its own GEK proof over that nonce AND an Ed25519 signature over the transcript; the browser verifies both and refuses otherwise. It also refuses an unchallenged handshake_ack, which previously let a peer skip proving anything at all. L4 — the proof was nonce ‖ offer_fp ‖ answer_fp: bare concatenation, and a missing fingerprint silently degraded it to nonce-only, dropping MitM detection (NS5). Every field is now length-prefixed and domain-separated, the role is bound so a client proof cannot be replayed as a node proof, and an absent channel binding is refused rather than tolerated. M1 — group_id was optional; omitting it skipped the membership check entirely and fell back to the node's first group. Now mandatory. M9 — node-scoped daemon tokens are refused on the client path. NOT DONE: quic_server.py still runs its own JWT-only handshake, so C6 remains open — a forged or stolen token reaches a node over QUIC and can inject chat without holding the GEK. quic_binding() is written and unit-tested but unwired. 11.5.6 (whether the certificate-hash anchor works with aioquic, or an RFC 5705 exporter is reachable) is unproven. 11.5.8 TOFU pinning of pk_node is not done: the client verifies the node's signature but does not yet remember which key it saw last. Adds packages/meshbay-common/tests/test_handshake.py (18 tests) covering the properties every transport must inherit. WebRTC test helpers rewritten around the shared module; _make_jwt now defaults to the test group, since group_id is mandatory. Tests: 24 webrtc, 176+ node+common. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: swarm privacy, revocation persistence, keystore KDF, audit integrityChristophe Besson2026-08-132-10/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Phase 11.5 hardening batch — H7, H4, M2, M6, M7, L1, L3, L6. H7 — private content hashes leaked to the hub. The daemon registered blake3 hashes for every group it hosted, private ones included, giving the hub a content fingerprint of every private file and letting anyone confirm whether a known file exists in the network. The leak was dormant only because the routes were declared on the groups router with a full path and mounted at /v1/groups/v1/swarm/* — the node's calls 404'd into a swallowed exception. Fixing the path alone would have activated the leak, so both land together: registration is gated on group visibility, the routes moved to a real /v1/swarm router, and the lookup now requires authentication. H4 — revocation was advisory. Group revocations were signed and broadcast by the hub and then dropped by the node, whose handler understood only "user" and "jti", so "suspend a group" enforced nothing. The denylist was also in-memory only, so a restart silently un-revoked everyone. Now persisted to data_dir/denylist.json, group targets honoured on both transports, and live sessions for a revoked group are closed. M2 — the node keystore, which protects the node's Ed25519 and X25519 private keys, was still deriving at 64 MB long after the hub's password verifier moved to 256 MB; the docs recorded the bump as done, true for the hub only. Raising the constant alone would have made every existing keystore permanently undecryptable, so envelopes now record the parameters they were written with and pre-M2 files continue to open under the legacy profile. M6 — registration inserted its audit row with a NULL user_id and then ran UPDATE ip_logs SET user_id=<new> WHERE user_id IS NULL, claiming every unattributed row in the table: failed logins for other usernames, concurrent registrations. In logs retained a year for legal requests, that attributed other people's connections to the wrong account. M7 — X-Forwarded-For was trusted unconditionally at four call sites, so anyone could forge the IP written to the compliance log and evade per-IP rate limits. New netutil.client_ip honours the header only from a trusted proxy and takes the rightmost hop (the one our proxy appended); no direct header reads remain. L1 dead GEK_REQUEST/GEK_RESPONSE constants removed; L3 peer errors no longer echo exception text (paths, internal state); L6 email sanity-checked instead of accepting any string — deliberately not RFC 5322, to avoid a new dependency. test_daemon_index_change_pushes_to_peers asserted that a PRIVATE group's hashes are registered with the hub. Split: private asserts not-called (index push to members still asserted), and a new test proves public groups still register. That is the fourth pre-existing test found asserting a vulnerability as intended behaviour, after gek auto-activation, the transport-wide chat_store and the blind admin challenge. Tests: 116 node, 132 hub+common. Regression suite now 43. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): group isolation, upload confinement, GEK seizure, admin challengeChristophe Besson2026-08-131-0/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Phase 11.5 — findings H1, C5a, H2, C5b, H5 (see second-review.md). Batched together because the node-side changes share webrtc_server.py and cannot be separated into working commits. H1 — cross-group chat leak. chat_store, the peer registry and the display-name cache were read from the shared transport context, and daemon.py hoisted the FIRST group's chat store onto it. On a node hosting several groups every group's messages went to one database, chat_history served them back to members of every other group, and chat broadcast reached all peers regardless of group. All three now resolve through _group_ctx(). C5a — upload confinement. Uploads landed in the shared root under a client-chosen name and overwrote whatever was there. Any member could destroy the operator's files, and by becoming the recorded uploader of the replaced file could then delete it through the uploader path, bypassing the Ed25519 admin challenge. Uploads now go to a per-user quarantine (.uploads/{user_id}/), refuse to overwrite, and enforce chunk ordering, a filename allowlist and a size cap. H2 — stored XSS in the node admin UI. Filenames chosen by any group member were interpolated raw into the localhost UI, which has no authentication, so script execution there equals control of the node admin API. Now html.escape() throughout, textContent in the audit table, plus CSP/nosniff/no-referrer. The CSP contains exfiltration but cannot stop injected inline script — escaping is the fix. C5b — group key seizure. gek_bundle_store wrote whatever any member sent and auto-activated bundles addressed to the node operator. The operator's X25519 public key is public (the node publishes it in handshake_ack), so any member could wrap a key of their choosing for it and take over the group, locking every legitimate member out. Storing now requires an operator signature and _try_activate_gek is removed: nothing arriving over MNP can set a live GEK. H5 — unbound signing oracle. The node challenged with 32 raw random bytes and the client signed them blind, so a signature named no operation, subject, node or time. New meshbay_common/adminop.py defines a length-prefixed, domain-separated transcript; both sides build it independently and the client refuses to sign when the announced op/subject do not match its request. BREAKING: a group admin who does not operate the node can no longer store GEK bundles on it. Invites must be performed by the node operator. Adds tests/test_security_regressions.py. Verified against pre-fix source via git stash. Three pre-existing tests asserted the vulnerable behaviour as a feature and were inverted: gek auto-activation, and the transport-wide chat_store in test_daemon. Tests: 109 node, 132 hub+common. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: Phase 12 — P2P crypto material, password split, node Ed25519 authChristophe Besson2026-08-131-0/+12
| | | | | | | | | | | | | | | | Baseline commit capturing in-progress Phase 12 work that was already present in the working tree (uncommitted) before the Phase 11.5 security remediation begins. Committed as-is, without review or modification, so that remediation changes arrive as a separable diff. Contents: BundleStore (P2P GEK + keypair bundles), password split (auth_key / bundle_key), node Ed25519 auth (POST /v1/nodes/auth, node-scoped JWT), GEK-HMAC handshake proof with DTLS channel binding, Ed25519 admin challenge-response, node local admin UI rewrite, browser key persistence. Not authored in this session — captured to establish a baseline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: Phase 10c — MSE video streaming (real-time playback)Christophe Besson2026-08-111-0/+4
| | | | | | | | | Replace download-then-play VideoPlayer with MSE (MediaSource Extensions) streaming. Node remuxes to fMP4 via ffmpeg, probes codecs with ffprobe, and sends encrypted segments over DataChannel. Browser decrypts and appends to SourceBuffer — playback starts within seconds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(ui): upload chunk size, cached file display, chat names, file delete, ↵Christophe Besson2026-08-111-0/+2
| | | | | | | | | | | | | | | | inline thumbnails - Upload chunks capped at 64KB to avoid WebRTC DataChannel max-message-size - Show cached files immediately while WebRTC connects (tabs visible during connection) - Persist sender_name in chat store (SQLite) — no more UUID display in history - File delete action in menu (node admin only, enforced server-side) - FILE_DELETE / FILE_DELETE_ACK MNP message types - Inline image thumbnails in chat attachments (download+decrypt, Signal-style) - Member panel: "Owner" label instead of "Group admin" to avoid hub/group admin confusion - Create group page: hint about needing a node - Refresh index after chat file attachment upload Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Phase 10b — Self-service UI (group create/join, upload, IndexedDB, ↵Christophe Besson2026-08-112-0/+44
| | | | | | | | | | | | | | | | search) Six self-service features for the web SPA: - Group creation UI with GEK auto-generation (AES-256-GCM ECIES) - Member management + invite by username (GEK wrapping for invitee) - Open group self-join flow (POST /v1/groups/{id}/join) - File upload client→node (FILE_UPLOAD MNP type, .uploads/ staging) - IndexedDB caching of group file indexes (instant display on revisit) - Cross-group file search (SearchPage, pure client-side on cached indexes) 11 new tests (166 total): 8 group self-service + 3 AES GEK wrap/unwrap. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Phase 9 — Web client SPA with WebRTC P2P transportChristophe Besson2026-08-111-0/+4
| | | | | | | | | | | | | | | | Complete browser-based client: Preact SPA with login, group file browser, encrypted download, video playback, group chat, i18n, and dark/light theme. Browser connects P2P to nodes behind residential NAT via WebRTC DataChannel (aiortc). Hub handles signaling only — all data flows E2E. Performance: pipelined downloads (8-chunk sliding window), binary msgpack wire format (no base64), redundant I/O elimination. Large file downloads stream to disk via File System Access API (showSaveFilePicker). Validated on SFR + Orange residential NATs, Chrome + Firefox, IPv4/IPv6. 132 tests passing. Deployed to meshbay.org + Orange node. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Phase 7 — Node v2 (multi-group, Sender Keys, 0-RTT, chat, denylist)Christophe Besson2026-08-102-1/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | | Implements all 8 milestones (7.0-7.7): - 7.0: JWT carries `groups` claim; node verifies group membership at MNP handshake (QUIC + TCP+TLS). Resolves security review C2. - 7.1: QUIC 0-RTT session resumption via stored session tickets (17-21ms reconnect vs 47ms cold). - 7.2: Hub→node WebSocket signaling for NAT punch coordination (`client_incoming`/`punch_ready`) + jti denylist push. Denylist class blocks revoked users/jtis at handshake. - 7.3: Multi-group daemon — one QUIC port serves N groups with per-group GEK, shared_root, and index routing. - 7.4: HLS streaming via QUIC (STREAM_SEGMENT message type, ffmpeg segment extraction). - 7.5: Sender Keys protocol for group chat (Signal Groups approach). Each member has own sending chain key, HKDF chain ratchet, AES-256-GCM encryption, Ed25519 signing. Resolves security review C1. - 7.6: Chat store (SQLite via aiosqlite), CHAT_MESSAGE MNP wire type with peer broadcast, web UI with WebSocket push. - 7.7: Argon2id calibration CLI. First security review included (first-review.md). 109 tests, demo-v3 validated against meshbay.org production hub. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: password-based key derivation + operational QUICKSTARTChristophe Besson2026-08-091-0/+128
| | | | | | | | | | | | | | | | | | | | | keyderive.py: derive Ed25519+X25519 from username+password via Argon2id. Same credentials → same keys on any device. Encrypt/decrypt keypair bundle (AES-256-GCM) for hub storage (web clients). 7/7 tests. Full suite: 81/81. keyderive.js: browser counterpart using PBKDF2-SHA512 + random keypairs encrypted for hub storage. Avoids algorithm mismatch with Python. hub/models.py + users.py: keypair_bundle field added to User, stored on registration, returned in login response for web client key recovery. QUICKSTART.md: fully rewritten. 3 operational scripts in QE/demo-v1/: setup_demo.py — create accounts, group, distribute GEK run_node.py — start HTTP node (watches shared/ directory) download.py — bob login → GEK fetch → decrypt → save All tested locally end-to-end. No invented URLs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 6 complete — chat, multi-group, federation, replication, webcryptoChristophe Besson2026-08-091-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | 6.1 Double Ratchet (meshbay_common/ratchet.py): Forward secrecy, break-in recovery, out-of-order delivery. Signal-spec KDF_RK/KDF_CK via HKDF-SHA256. 11/11 tests. 6.2 Multi-group node (config.py): [[groups]] TOML array, per-group ports, back-compat [group]. 6.3 MHP federation persistence (db/models.py FederatedGroup + SwarmSource): receive_directory() now persists to federated_groups table. list_public_groups() includes federated results with source attribution. 6.4 Content replication (node/replication.py + hub SwarmSource): ContentReplicator: fetch-index, download, hash-verify, register-swarm. Hub: POST /v1/swarm/register, GET /v1/swarm/{hash} for multi-source. 6.5 Browser private group (webcrypto.py + static/crypto.js): AES-256-GCM variant of GEK for WebCrypto-compatible groups. crypto.js: SubtleCrypto importGEK + deriveChunkKey + decryptChunk. Keys distinct from ChaCha20 via :aes HKDF info suffix. 4/4 tests. 74/74 tests total. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat(common): add Double Ratchet algorithm — 6.1Christophe Besson2026-08-091-0/+311
| | | | | | | | | | | | | | | | | RatchetState: full Signal-spec Double Ratchet (DH ratchet + symmetric ratchet). KDF_RK/KDF_CK via HKDF-SHA256. AES-256-GCM message encryption. MKSKIP for out-of-order delivery (max 1000 skipped keys). ChatMessage dataclass with to_dict/from_dict for wire serialisation. Properties validated by tests: ✓ Forward secrecy (consumed keys unreplayable) ✓ Out-of-order delivery ✓ Associated data binding ✓ Break-in recovery (post-ratchet keys independent) ✓ 100-message stress test 11/11 tests in 0.06s. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* chore: initialize monorepo structure for MeshBayChristophe Besson2026-08-093-0/+247
3-package layout: meshbay-common (shared crypto/protocol), meshbay-hub (FastAPI server), meshbay-node (local daemon). Includes validated POC spikes 1-6 in poc/, architecture drafts v1/v2 in docs/, and CLAUDE.md project conventions. All cryptographic primitives extracted from POC into meshbay_common/crypto.py (GEK wrap/unwrap, chunk key derivation, keystore encryption, chunk signing). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>