diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-09-06 01:33:16 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-09-06 01:33:37 +0200 |
| commit | f2da33a648f86e34ddbbb5f6bec124828ef2a847 (patch) | |
| tree | 670cbf62ff92598847e067b5835713dfd854dfb5 /packages/meshbay-common | |
| parent | fff1974edf19cf1186e0f49da5f8a4d237bcb13e (diff) | |
| download | meshbay-f2da33a648f86e34ddbbb5f6bec124828ef2a847.tar.gz | |
feat(node): indexing v2 — partial-read hashing for files above 40 MB
Files above 40 MB are no longer read in full. Instead, blake3 hashes
45 MB of samples (first 20 MB + last 20 MB + 5 MB at 50% offset).
Files at or below 40 MB are unchanged (full read, hash_version 1).
A new `hash_version` field on IndexEntry (default 1) travels on the
wire and through the cache so both versions coexist without breaking
existing nodes or clients.
The IndexCache auto-migrates its schema on open (ALTER TABLE), so no
manual step is required on upgrade. A standalone migration script is
available in QE/migration/ for operators who want to preview or force
a full re-hash.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat (limited to 'packages/meshbay-common')
| -rw-r--r-- | packages/meshbay-common/src/meshbay_common/protocol.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/packages/meshbay-common/src/meshbay_common/protocol.py b/packages/meshbay-common/src/meshbay_common/protocol.py index 4d2dd9d..e502379 100644 --- a/packages/meshbay-common/src/meshbay_common/protocol.py +++ b/packages/meshbay-common/src/meshbay_common/protocol.py @@ -204,6 +204,7 @@ class IndexEntry: track_no: int | None = None # tag or parsed, Music app taken_at: int | None = None # unix timestamp, EXIF DateTimeOriginal — Photos app camera: str | None = None # "Make Model", when both present — Photos app + hash_version: int = 1 # 1 = full-file blake3, 2 = partial-read (45 MB sample) def index_entry_wire(e: IndexEntry) -> dict: @@ -224,6 +225,7 @@ def index_entry_wire(e: IndexEntry) -> dict: "season": e.season, "episode": e.episode, "artist": e.artist, "album": e.album, "track_no": e.track_no, "taken_at": e.taken_at, "camera": e.camera, + "hash_version": e.hash_version, } |