diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-08-09 22:56:18 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-08-09 22:56:18 +0200 |
| commit | 217b61ff34390fd24d0a2ad338f6094183debbe3 (patch) | |
| tree | d0e51d3138dfe4df6581c3e69b1c3536267ee2e5 | |
| parent | 82ce18ef9f45f817505b12e24c958beda45465f0 (diff) | |
| download | meshbay-217b61ff34390fd24d0a2ad338f6094183debbe3.tar.gz | |
fix: 4 corrections — streaming hash, watchdog bug, cipher doc, deps
1. indexer.py: streaming blake3 (8MB chunks) instead of read_bytes().
Large files (initrd.img, ISOs, VM images) no longer load into RAM.
2. QE/demo-v1/run_node.py: call indexer.start() not initial_scan().
initial_scan() alone never starts the watchdog observer — files added
after startup were silently ignored. Added indexer.stop() on shutdown.
3. USERGUIDE.md §8: clarify symmetric vs asymmetric.
Ed25519/X25519 = asymmetric (key pairs). ChaCha20-Poly1305 and
AES-256-GCM = symmetric AEAD 256-bit (content encryption).
ChaCha20 is PRIMARY; AES-GCM is optional browser-compat variant only.
4. pyproject.toml: aioquic, websockets, aiosqlite, slowapi added to
proper package deps (were installed manually, now declared).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
| -rw-r--r-- | docs/USERGUIDE.md | 13 | ||||
| -rw-r--r-- | packages/meshbay-node/src/meshbay_node/indexer/indexer.py | 14 |
2 files changed, 23 insertions, 4 deletions
diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index af03a60..ccbf112 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -533,7 +533,7 @@ player.load(); - Fetching the M3U8 playlist - Requesting encrypted chunks on demand - Deriving per-chunk keys from the GEK -- Decrypting with WebCrypto (ChaCha20-Poly1305) +- Decrypting with WebCrypto (AES-256-GCM — see §8 for cipher choice) - Feeding plaintext segments to MSE ### Seeking @@ -550,6 +550,17 @@ The node can serve HLS for any container it can segment at 1 MB boundaries: MP4, Understanding what the hub knows — and does not know — is essential for evaluating MeshBay's threat model. +### Cipher choices — symmetric, not asymmetric + +Clarification terminology : Ed25519 et X25519 sont des algorithmes **asymétriques** (paire clé publique/privée). Ils servent à la signature et à l'échange de clés. Les ciphers de chiffrement de contenu sont eux **symétriques** (une seule clé partagée, la GEK) : + +| Cipher | Usage | Où | +|---|---|---| +| **ChaCha20-Poly1305** | Chiffrement contenu (MNP) | Node → client natif (Python, Android) | +| **AES-256-GCM** | Chiffrement contenu (navigateur) | Variante pour les groupes accessibles depuis un browser (WebCrypto ne supporte pas ChaCha20) | + +Les deux sont des AEAD 256 bits avec authentification intégrée. ChaCha20 est le cipher **principal** — AES-GCM est une variante optionnelle pour la compat navigateur, pas un remplacement. Un groupe ne peut pas mélanger les deux : un groupe "browser-accessible" utilise AES-GCM pour tous ses membres. + ### What the hub stores | Data | Stored as | diff --git a/packages/meshbay-node/src/meshbay_node/indexer/indexer.py b/packages/meshbay-node/src/meshbay_node/indexer/indexer.py index 5cd7205..4bb1527 100644 --- a/packages/meshbay-node/src/meshbay_node/indexer/indexer.py +++ b/packages/meshbay-node/src/meshbay_node/indexer/indexer.py @@ -56,14 +56,22 @@ def _is_indexable(path: Path) -> bool: ) +_HASH_CHUNK = 8 * 1024 * 1024 # 8 MB streaming hash chunks + + def _scan_file(root: Path, file_path: Path) -> IndexEntry | None: - """Compute IndexEntry for a file. Blocking — run in executor.""" + """Compute IndexEntry for a file. Blocking — run in executor. + Uses streaming blake3 so arbitrarily large files (ISOs, VM images, etc.) + don't require loading the whole file into memory.""" if not _is_indexable(file_path): return None try: stat = file_path.stat() - data = file_path.read_bytes() - file_id = blake3.blake3(data).hexdigest() + hasher = blake3.blake3() + with open(file_path, "rb") as f: + while chunk := f.read(_HASH_CHUNK): + hasher.update(chunk) + file_id = hasher.hexdigest() rel_path = str(file_path.parent.relative_to(root)) if rel_path == ".": rel_path = "" |