summaryrefslogtreecommitdiffstats
path: root/packages/meshbay-node/src/meshbay_node/indexer/indexer.py
Commit message (Collapse)AuthorAgeFilesLines
* fix: first-run wizard reliability and node startup performanceChristophe Besson2026-08-221-3/+9
| | | | | | | | | | | | | Node daemon no longer blocks startup on slow directory scans — initial indexing runs in the background so the node reaches "running" immediately after transports are up. Fixes the wizard failing to detect the node when large USB/NAS roots take minutes to scan. Also: wizard key-linking deadlock resolved (main.js links during poll), invite form stays in DOM during reconnects (disabled instead of destroyed), pairing code bridges to renderer, and firewall docs for LAN casting added. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(node): several named roots per group, and one implementation per operationChristophe Besson2026-08-181-50/+350
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stage A — a group's content is a set of named roots --------------------------------------------------- `shared_dir` becomes a list of {name, path, kind}. The name is the directory's basename, derived once at add time and *stored*: recomputing it would re-identify a whole library the day someone renames a folder on disk. Duplicate names are refused case-insensitively and no root may contain another — both compared with NFC folding, because most of these directories live on exFAT or NTFS where `Films` and `films` are one directory. Every index path carries its root name, in a one-root group as much as in a five-root one. One path shape has to be got right once; two have to be kept right for ever. **A root that goes away freezes; it never empties.** Unmounting a volume makes watchdog report every file under it as deleted, or presents an empty directory to the next scan. Acting on either propagates deletions for a whole library to every member, as though the owner had erased it. So a deletion is acted on only once its root is confirmed readable, and availability is tracked per root — one unplugged drive leaves the others serving. 12 tests, verified to fail against an indexer without the check. Events are not trusted to be complete either: ReadDirectoryChangesW drops them under load and inotify on a FUSE mount misses changes made outside it. A periodic reconciliation sweep is the only thing that recovers a missed event. MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content hash, a node id and an endpoint — no paths, no filenames — and private groups register nothing (H7). Stage B — one implementation behind every front door ---------------------------------------------------- C1 and C6 were both "a second path into the node with its own weaker handshake". Two implementations of `revoke` with two authorization checks is that shape one size down. `meshbay_node/ops.py` holds each operation once, takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback API is one `_op(...)` line per endpoint; the MNP handlers call the same functions. test_ops.py asserts the shape rather than trusting it. Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload` (SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP operations: rotation is the half of revocation that revocation cannot do, since the ex-member holds the current key, and the node generates the replacement with its own CSPRNG — no key material crosses the wire, which is what the C5b rule is actually about. Two bugs found by running it rather than by testing it ------------------------------------------------------ GroupIndex is keyed by **content hash**, so the same bytes at two paths are one entry — which is also why a scan reports ten files and indexes nine. Reconciliation compared paths, so it decided the second path was a missed event every 60 s, rewrote the entry and pushed an index update to every connected peer. Seen in a live node's log. `meshbay-node reload` crashed on first use with `subprocess` unimported: the module compiles fine, which is the "syntax, not names" trap already recorded for the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be added to the parser without an entry there. Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the wire carried "0.2" — harmless only because nothing imported it. And _do_dir_create/_do_dir_delete referenced an undefined `filename` on their error path. 740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: Phase 9 — Web client SPA with WebRTC P2P transportChristophe Besson2026-08-111-16/+35
| | | | | | | | | | | | | | | | Complete browser-based client: Preact SPA with login, group file browser, encrypted download, video playback, group chat, i18n, and dark/light theme. Browser connects P2P to nodes behind residential NAT via WebRTC DataChannel (aiortc). Hub handles signaling only — all data flows E2E. Performance: pipelined downloads (8-chunk sliding window), binary msgpack wire format (no base64), redundant I/O elimination. Large file downloads stream to disk via File System Access API (showSaveFilePicker). Validated on SFR + Orange residential NATs, Chrome + Firefox, IPv4/IPv6. 132 tests passing. Deployed to meshbay.org + Orange node. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: 4 corrections — streaming hash, watchdog bug, cipher doc, depsChristophe Besson2026-08-091-3/+11
| | | | | | | | | | | | | | | | | | | 1. indexer.py: streaming blake3 (8MB chunks) instead of read_bytes(). Large files (initrd.img, ISOs, VM images) no longer load into RAM. 2. QE/demo-v1/run_node.py: call indexer.start() not initial_scan(). initial_scan() alone never starts the watchdog observer — files added after startup were silently ignored. Added indexer.stop() on shutdown. 3. USERGUIDE.md §8: clarify symmetric vs asymmetric. Ed25519/X25519 = asymmetric (key pairs). ChaCha20-Poly1305 and AES-256-GCM = symmetric AEAD 256-bit (content encryption). ChaCha20 is PRIMARY; AES-GCM is optional browser-compat variant only. 4. pyproject.toml: aioquic, websockets, aiosqlite, slowapi added to proper package deps (were installed manually, now declared). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat(node): add directory indexer and Mesh Group IndexChristophe Besson2026-08-091-0/+213
GroupIndex: msgpack→zstd→GEK-encrypt→sign for private groups, plaintext+sign for public groups. DirectoryIndexer: watchdog-based watcher, async initial scan via thread pool, on_change callback. Delta support (diff between versions). 10/10 tests passing. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>