| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
The _enrich_*/_reenrich_* methods and _on_enriched become
EnrichmentMixin in meshbay_node/enrichment.py, with the two directory
helpers only they use. APP_DIR_KEYS stays on NodeDaemon.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The pass before this applied ruff's own fixes. These are the ones needing a
decision, and the point of doing them is that `ruff check .` now passes: a
linter reporting 98 known-acceptable findings reports nothing, because the next
real one arrives invisible.
**Lines over 100 (70).** Mostly wrapped where they stood. Two exceptions: the
aligned trailing comments in `protocol.py`'s message table were shortened rather
than wrapped, because wrapping one row of a table breaks the table; and in
`models.py` the column comments moved above their columns for the same reason.
**Imports below the first statement (14).** `csam.py` kept its FastAPI imports
under a section header halfway down the file; two node tests had a constant and
a `pytestmark` wedged between two import blocks. Moved, not suppressed.
**Bindings nothing reads (4).** Three in tests, where the call stays and only the
name goes — `_user(client, "listener")` is there to create the user, not to
return one. The fourth was in `revocation.py` and was not a lint finding at all:
`_connect_and_listen` opened an httpx stream to the WebSocket URL, did `pass`,
and then opened the real connection through the `websockets` library. One
pointless request per connect, left over from before that library was used
directly. Removed, and `httpx` with it.
**`l` as a name (4)**, **semicolons (6)** in the POC spikes, and the rest.
2893 passed, the same count as the two commits before it.
`meshbay_node/revocation.py` is worth a decision separately: 154 lines that
nothing imports, superseded by `hub_client.maintain_ws`'s `on_revocation`. This
commit only stopped it failing the linter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`ruff check .` had gone unrun long enough to report 568 errors, which is the
same as having no linter: the next real finding would have been invisible in the
noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the
98 it cannot fix are the next commit.
What actually changed: import sorting (225), imports nobody used (87, none of
them a re-export — no `__init__.py` is touched, which was the one way this could
have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69)
and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12
this project requires, `Optional[X]` to `X | None` (24), and f-strings with
nothing to interpolate (19).
Checked rather than assumed: every module in the three packages still imports,
and the suite is 2893 passed — the same count, test for test, as the merge
before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
media_cache held video_meta, photo_meta and thumbs; the audio tags lived
only in the in-memory IndexEntry. So every start re-read every audio file
the node serves, and until that pass landed it served an index with no
artist on any track — one the Music app cannot group. Over a real
6176-file library the pass costs 27.8s cold and 6.4s from audio_meta.
Only what the bytes decided is stored. The filename and folder fallbacks
still run live, or a renamed file would get the old name's answer; the
sibling-cover scan reads the folder, so it stays live too; and a read
that failed is not cached, or one bad read becomes permanent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and
deleted from the tree on 2026-09-11, but ~230 comments across the three
packages still named them — usually written `docs/musicbay.md §3.2`, as
though the file were still in docs/. A reader had to know §16 existed to
resolve any of them. They now name the section directly.
Every mapping comes from §16, the concordance, which already records where
each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the
Videos app and §6.5 where the subject is derived data, photos -> §9.9,
auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 ->
§3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys ->
§4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section.
Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`)
are retargeted too: those collide with real section numbers in the design
document, so leaving them would have been worse than the named form.
Four cases the concordance does not cover, each decided rather than guessed:
Sub-item references into documents that no longer exist — mediacenter's
`§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and
§9.2 do not reproduce. The module-level citation stays; the sub-item
pointer is dropped.
The V-findings keep their labels but lose the dead `§10.1/` prefix.
§13.8 lists V1–V13 as per-application open items, which is not what the
labels mean in these comments, so pointing them at §13.8 would have been
a false citation.
`apps.md §5`'s virtualization requirement has no counterpart anywhere in
the design document. The requirement is stated in the comment itself, so
the citation is dropped rather than aimed at a section that does not say
it.
Comments that attributed a *sentence* to an old note — musicbay's "several
thousand files" example, its "what got measured" note, its measured
~11%/~26% cover-art figures, the "original no root, whole shared tree"
call — state the fact without attribution now. §9.8 does not contain those
sentences and citing it for them would have been wrong.
CLAUDE.md's "a reference to a document that no longer exists" row now says
the concordance is for git history and out-of-tree material; the code cites
sections directly.
Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox
leg of test_sticky_header.py's browser harness, which is broken at the
browser level on this machine — headless Firefox (snap) dies with
`[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders
nothing, and the probe exits `{"error": "no measurement"}` after its full
90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing
here can affect it: every changed line in style.css is inside a comment.
Also checked: ast.parse on every changed .py, `node --check` on every
changed .js, the /* */ balance in style.css, and that no changed line
exceeds the width its file already used.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The rest of AV9's disk half. Serving a file left the loop in the commit before
this one; two paths were still on it.
**The availability poll.** `RootSet.refresh_availability` stats every root, and
eleven call sites reached it from `async def` — the reconcile loop among them, on
a timer. On a sleeping disk that is a stall once per tick, and the stat is also
what keeps the disk awake, so a node paid spin-up for a library nobody was
reading. All eleven now go through `off_disk`, `Root.is_live` included.
**The upload write.** `open`/`write`, and the resolve, the stat, the free-name
search, the rename and the unlink around it. This one could not simply be
awaited: the handler was synchronous, so nothing could come between the
`chunk_index != state.next_index` check and the `advance` that answers it, and
that is the whole of the chunk-ordering rule. Awaiting the write opens the gap —
chunk 1 arriving while chunk 0 is in the disk thread reads a position that has
not moved and is refused as out of order, so an upload would fail on a slow disk
and nowhere else. Verified, not assumed: without the lock the new ordering test
refuses three chunks of four.
So the check, the write and the advance are one critical section again, under a
lock held **per group**. Not per session: `partial_uploads` lives in the group
context so a reconnecting client finds its upload where it left it, which means
two sessions of one member share the position of one `.part` file. Arrival order
is preserved by construction — the dispatcher creates one task per message as it
arrives, tasks start in creation order, and the lock is the first thing each one
waits on, so its waiters queue in arrival order too.
`_do_file_upload` is a coroutine now, which is why forty-two test call sites gain
an `await`. Their outcomes are unchanged, file by file, against the run before
the change.
`test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`,
imported rather than defined there. It now asks for the ones written in the
module, which is what its own docstring means; all forty-three operations are
still checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The streaming path mapped 0:a:0 unconditionally, so a dubbed film played in
whichever language was muxed first and the others were unreachable. The node
now enumerates the tracks in stream_init and honours audio_track in
stream_req; switching is the seek path, since one ffmpeg carries one track.
MNP 3.2, additive: the player draws its selector from the node's own list and
never from a version number, so an older node is never asked for a track it
would ignore. MNP_MIN_SUPPORTED does not move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`IndexProgress` said "scanning, this many bytes of that many" and nothing
more. A group's roots are walked one after another, so a second directory
added during a large scan showed as the bar jumping back to 0 %. It now also
carries the root being walked and its position in the roots table, the kind
of walk (scan, rescan, reconcile, watch), file counts, and the roots
waiting for the scan lock in order: queued by the initial scan, by a
retarget, and by a plug; dropped when a root is removed.
`GET /api/index-status` answers for every group at once, including a group
still in its initial scan, so a client can show indexing on any page. It
names roots: loopback only, like `current_dir`.
`index_progress` and the handshake ack gain the same counters, still naming
nothing (decision D3): the root is a position in the roots table the member
already opened from the sealed index, and the queue is a count. The pusher
keeps speaking while a root only waits for the lock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A watchdog burst wrote the indexer's single `progress` directly. A file
dropped into a folder during a large scan added its size to the scan's
total, then cleared `scanning` when its own hash finished, so the progress
went blank with hours of hashing left. Bursts now keep their own counters,
shown only while no whole-root walk runs.
A plug rescan took no scan lock and walked its root beside an added root's
scan, both resetting the same counters and reading the drive in turn. It
now waits for the lock, and skips the rescan if the root was ejected or
removed while it waited, since the rescan drops the entries before it walks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A root added from the client arrives over MNP, and _retarget_indexer
started the daemon's reload with the session's own _spawn. When that
session closed - a client reconnecting 47 s into the scan of a 900 GB
root - shutdown_tasks() cancelled the reload mid-scan, and the reload
queued behind it, without a line in the log. The new root was in
node.toml and in the indexer's set but never in the group's context; the
lock was free and nothing retried, so the node served the old roots table
for hours while reconcile hashed the whole drive as missed events. One
loopback reload fixed the live node in 9 ms.
_reload_config now runs the work in a node-owned task and awaits it
through asyncio.shield, so a caller that goes away only stops waiting; a
cancelled reload is logged. plug_root does the same for its rescan, which
drops the root's entries before walking the disk and so left the root
empty when its admin op's session closed.
The existing MNP test replaced _spawn with a list and could not cancel
anything; the new tests close the session for real and fail without this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding a large directory to a running group made the reload await the
scan of the new root before putting the new RootSet in the group's
context, holding _reload_lock the whole time. For the hours a large drive
takes to hash, the node served the old set:
- a file request under the new root got None from entry_abs_path and the
handler died on None.exists() without replying;
- a writable/removable toggle answered with the live table, still the
old one, so the directory vanished from the operator's settings;
- reconcile saw every file the scan had not reached as a missed event and
hashed it again on the same executor, rewriting progress under the scan.
retarget now applies the set, the roots table and the watcher first, and
with wait=False scans the added roots in the background; the daemon swaps
ctx["roots"] before calling it. A scan lock shared by the initial scan,
added-root scans and reconcile makes the reconcile loop sit out a running
scan without backing off. Every transport site that resolves an entry
answers ROOT_NOT_SERVED instead of crashing, and a delete keeps the entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
asyncio keeps only a weak reference to a task, so a coroutine started with
`asyncio.ensure_future(...)` whose result is discarded can be collected while
it is still running: the loop logs "Task was destroyed but it is pending!" and
the work simply does not happen. No error reaches the caller, and what is lost
is whatever that coroutine was in the middle of.
The node already had a guard for this, written after an abandoned stream task
lost a transcode slot for good — and it read one file, `webrtc_server.py`,
because that is where the defect was found. Outside that file there were
nineteen sites: the hub's `chat_notify` (a notification for every member of a
group), the indexer's debounce (every real-time index update), eleven in
`daemon.py` including the SIGHUP reload and each enrichment pass, two in
`ops.py`, and five in the loopback API.
`meshbay_common.background.spawn()` is the one door. It holds the task, drops
it when it finishes, and logs what it raised under the coroutine's own name —
an exception in a task nobody awaits was otherwise reported by asyncio at
collection time, out of context or not at all. A peer session's `_spawn` stays
as it is: that one can also *cancel* what it holds, which a module-level holder
cannot, because a session ends and a process does not.
`test_background_tasks.py` walks every package's source and refuses a discarded
handle. It parses rather than greps, so an assignment, a comprehension or an
await is not mistaken for one, and it was checked against a deliberate
reintroduction. A guard that stops at the edge of the file where the bug was
found is a guard against that bug, not against its class.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`_register_uploader` walked the index for the entry it had just written, at a
moment when no such entry can exist: the file was a `.part` until the rename on
the line above, which is not indexable, and the watchdog that will index it
debounces for two seconds and then hashes. The walk matched nothing, silently,
so every uploaded file in every group was owned by nobody — and `file_delete`
refuses a caller with no admin authority when the entry records no uploader, so
a member could not delete what they had just sent. MESHBAY_DESIGN.md §5.4
grants that to any non-revoked device of the uploading account.
The record is now written when the last chunk lands (`indexer.record_upload`)
and the entry is stamped from it in `_hash_or_cached`, the one funnel every
entry passes through — initial scan, watchdog, reconcile and replug alike. It
lives in the index cache rather than on the entry alone, because the index is
rebuilt from disk at every start and an owner the node forgets on restart is a
right quietly taken away. It is validated against a live `stat()`, so whatever
later occupies that path inherits nothing; and `_rescan_root`'s carry-over no
longer copies over it, or memory would beat the durable record.
§5.4 also claimed ownership was *provable* — a transcript the uploader signs,
stored with the entry. No such signature has ever existed; `meshbay:upload:v1`
in the code is the groupbox purpose that seals the envelope. The section now
states what the code does, and the transcript is an open item in §15.3.
`test_upload_attribution.py` drives the real handler and a real indexer across
that seam. Against the previous source its two positive cases fail on the
property, not on a missing method — an upload, then a rebuild from disk, then
a different file at the same path inheriting nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
e1dbdf0 made a replugged root re-enrich, which was correct and not enough:
the operator still watched their albums vanish. Measured on the reported
library with a cold metadata cache, the node broadcast twice — the first
delta stripped every album, the second put them back 14 seconds later.
Fourteen seconds of "no music found" is the bug, whatever happens after.
An entry's id is its content hash, so an entry that comes back under the
same id, name and path is the same bytes in the same place and everything
enrichment derived from it still holds. `_rescan_root` now carries those
fields across the drop-and-rescan that `reconcile` and `plug_root` share.
Re-enrichment stays as the fallback for what genuinely changed: a
different id is different content, and a different name or path can change
the folder and filename fallbacks that artist, album, display_title and
track_no rest on, so those entries are still handed to the daemon through
`rescanned_ids`.
`uploader_id`/`uploader_pk` ride along. They are the same shape of field —
set once on an entry, readable from nowhere on disk — and they decide who
may delete the file, so losing them to a replug quietly took a right away.
Verified on the running node: one broadcast 550ms after the plug, carrying
the albums, and no metadata lookups at all.
The tests now assert the field on the entry rather than a call to an
enricher. Counting calls is what let the previous version of this file pass
while the operator still saw an empty tab.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reported live: a removable root ejected from Files and plugged back in
returned with its files and without its albums. Music showed "no music
found" and stayed there through a force reload — the loss was on the
node, not in the client.
`plug_root` drops the root's entries and rescans, which is right; the
drive may have changed while it was away. What comes back is a bare
IndexEntry: `_hash_or_cached` fills id/name/path/size/type and nothing
else. Every enrichment field goes with the old object, and the Music tag
fields are cached nowhere by design (enrich_audio.py re-reads them so a
rename can re-derive the filename fallback), so re-enrichment is the only
way back.
Two gates then made sure it never ran:
* enrichment is scheduled for `delta.additions`, and ejecting broadcasts
nothing, so `_last_broadcast_snapshot` still held those ids — the
rebuilt entries diffed as updates, not additions;
* `_enrich_new_*_entries` skips anything in `_enriched_attempted`, which
is only discarded for `delta.deletions` — and dropping and rescanning
inside one call broadcasts no deletion either.
A restart cleared both, since an empty snapshot makes every entry an
addition. Nothing short of one did.
The indexer now records the ids it rebuilt and the daemon drains them at
broadcast time: their "already attempted" mark is discarded and they
rejoin the entries offered to the three enrichment passes. Not
Music-specific — Videos lost durations and titles and Photos lost
thumbnails the same way; Music is just where an untagged file has no
album to file itself under, so the app goes empty rather than plain.
`reconcile()` does the same drop-and-rescan when a root reappears on its
own, so a USB drive that fell off and re-mounted hit this with nobody
touching the UI. Covered too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review of the Phase 1 commit found the RO/RW model sound but three paths
unfinished, each of which broke the flow the phase exists to deliver. Plus
29 test failures it introduced and no coverage for anything it added.
Uploads went to the wrong directory. The node read a `root` field on
file_upload that no client ever sent, so every upload landed in the first
writable root while the Files toolbar offered its button based on the root
being browsed — with two writable roots, uploading from one wrote into the
other. Files now names the root it is showing; Chat names one chosen in the
shell (an operator-configured directory arrives in Phase 2); the node refuses
an unknown name rather than falling back, and refuses read-only and ejected
roots by code.
Shared directories were unreachable on the web. The table read its roots
only from the loopback API, which resolves to "not available" in a browser,
so the section rendered for nobody there — while the Uploads controls it
replaced had worked — and the transport.updateRoot/ejectRoot/plugRoot methods
beside it were dead. MNP is now the path, loopback the fallback for a local
node with no live connection, and adding a root over MNP takes a typed path
since no web page can browse a remote disk.
Ejecting updated nobody's screen. transport.js resolves an admin ack against
the pending request and returns, which is right for every op whose caller
knows the value it chose; the root acks carry state only the node can compute,
so the operator who clicked Eject was the one client that never saw it happen.
And the ejected flag reached roster.db but was never read back, so a restart
undid it and the next scan read an empty mount point as an erased library.
Also: the member-upload endpoint answered 200 and did nothing (removed); the
wizard ignored the first root's RW switch; reload compared roots on name and
path, so editing writable in node.toml did nothing; the table had no path
column, which is the only thing separating two libraries sharing a basename;
apps_enabled normalisation differed between the two sides of a signed subject.
Tests: eject/plug, per-root upload refusal and the node.toml rewrite had no
coverage at all. test_member_upload_policy.py is replaced by
test_root_writable_policy.py — it tested a removed feature — and every
property worth keeping from it moved rather than being dropped.
Docs: draft-v6 structural decision 9 is annotated as superseded (the operator
can no longer have a directory only they may write to — a real capability
removed, flagged rather than hidden), the man page documents the root verb and
the RO/RW fields, and refactor-groups.md §7b records what the plan got wrong.
Suite: 41 failures before, 13 after — all 13 pre-existing on main.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Replace the upload boolean with per-root writable/removable/ejected flags.
Backend: new ops (update_root, eject_root, plug_root), MNP 1.1 protocol
messages, live RootSet updates so API always reflects current state, CLI
root subcommand (add/remove/set/list/eject/plug).
Frontend: SharedDirectoriesTable with optimistic toggle switches, eject/plug
in Files and Settings, upload gated on root.writable, ejected-root filtering
in all media apps, updated Create Group wizard, 10-locale i18n.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
content
The real-time watchdog path lacked the duplicate-content guard that
reconciliation already had. When a file with identical content appeared
(e.g. browser download appending " (2)"), add_entry overwrote the
original's index entry — making it vanish from the file list despite
still being on disk. Now check get_entry before adding, matching the
reconciliation logic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Files above 40 MB are no longer read in full. Instead, blake3 hashes
45 MB of samples (first 20 MB + last 20 MB + 5 MB at 50% offset).
Files at or below 40 MB are unchanged (full read, hash_version 1).
A new `hash_version` field on IndexEntry (default 1) travels on the
wire and through the cache so both versions coexist without breaking
existing nodes or clients.
The IndexCache auto-migrates its schema on open (ALTER TABLE), so no
manual step is required on upgrade. A standalone migration script is
available in QE/migration/ for operators who want to preview or force
a full re-hash.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
| |
Platform directories, signal handling, chmod guards, ffmpeg discovery,
and platform-conditional CLI messages — all testable on Linux.
See docs/WINDOWS-PORT.md §5 for the plan these implement.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`file_chunk` and `index_sync` were each built twice, once per transport, and
the two copies did not agree. WebRTC sent binary, unsigned chunks carrying a
`file_id`; QUIC sent base64 fields, two BLAKE3 hashes, a per-chunk Ed25519
signature and no `file_id`. `index_sync` was plain entries on one transport
and a `GroupIndex.serialize()` envelope on the other. One message type, two
shapes, one consumer each, and nothing that failed when they drifted — finding
C6 one size down, in the two places the handshake unification did not reach.
Phase 9.15 moved WebRTC to the binary format and dropped the per-chunk
signature; the QUIC encoder was never brought along. It is dropped here rather
than reintroduced: the AES-GCM tag authenticates the ciphertext under a
GEK-derived key, and since C3 the node authenticates itself once in the
handshake instead of once per megabyte.
`meshbay_common.protocol` now owns the chunk codec (`chunk_ciphertext`,
`file_chunk_wire`, `file_chunk_plaintext`) and `meshbay_node/transport/wire.py`
the index builder, which also absorbs the delta the daemon used to hand-build.
`test_transport_wire_parity.py` fails if either server grows its own copy back.
`ChunkRequest`/`ChunkResponse` are deleted. `ChunkResponse` described the QUIC
half while reading like the contract for both, which is what made the fork hard
to see at all.
BREAKING CHANGE: MNP 0.15 changes the encoding of `file_chunk` and `index_sync`
on the QUIC transport. The WebRTC shapes are byte for byte unchanged and no
QUIC client ships, which is why this is a MINOR bump; a deployed QUIC peer
would have made it MAJOR.
Also fixes a test fixture that put a `Path` where the daemon puts a `RootSet`.
Nothing caught it: the old QUIC index handler never touched `roots`, and
`entry_abs_path` fell through `Path.resolve(strict=...)`, reading the virtual
path as a truthy flag and returning the right file by accident.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Real franchise / show / release-group names had crept back into test
fixtures, code comments, a docstring and docs/mediacenter.md while fixing
the saga-match and misclassification bugs. Replace them all with invented
placeholders ("Some Saga", "A Different Show") and shape descriptions
("a franchise-origin film", "a 3-season show"). Behaviour and assertions
unchanged; 738 node tests still pass.
Record the rule in CLAUDE.md so it stops recurring.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Every "<Saga> Episode <N> - <subtitle>" file in a numbered franchise
resolved to the series' first entry (a real, older film). `sequel_variants`
stripped "Episode <N>" and offered the bare "<Saga>" as a candidate query;
that matches the first film's original_title at ratio 1.0 and beat PASS 1's
correct-but-lower hit. A franchise's bare name is very often a real,
different film.
When a Part/Episode/Chapitre/… keyword carried the index, sequel_variants
no longer emits the bare base — only "<base> <digit>" and "<base> <roman>".
Without a keyword ("<Franchise> 3") the bare base is still offered, so that
fix is untouched. Verified live against TMDB: the franchise's episodes each
resolve to their own entry; the earlier numbered-sequel, two-part-film and
franchise-subtitle regressions all hold.
docs/mediacenter.md §10.3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found on demo35: "Some.Film.2017.MULTI.108.grp.mkv" — the release name's
"1080p" truncated to "108" — makes guessit read S01E08, so enrich.py's
flat-library branch (elif ep.episode is not None) filed a standalone film
as a series. "Fix match" then only offered TV results for the phantom
show, so there was no way out from the UI.
- title_parse.has_episode_marker(): true only for an explicit SxxExx /
1x08 / "Episode N" / "Season N" token, not a bare 3-4 digit run.
- enrich.py: the flat-library branch now needs ep.season AND ep.episode,
plus either a real marker or the absence of a "(2019)"-style year.
Every genuine flat-dumped episode in the corpus carries a marker, so
real shows are untouched; the same misparse on "1280" ("...2013.1280...")
is covered too.
docs/mediacenter.md §10.2. Known gap left open: no operator control over
the movie/show kind itself — a "this is a movie / a show" toggle would.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sequel_variants
V8: the TV/show branch of _tmdb_search used the old "first candidate over
0.6 wins" shape. It now shares one _tmdb_ladder helper with the movie
branch — score every candidate query, keep the best, fast-path a
confident primary hit. A year lifted off the show's folder name
(title_parse.year_in, e.g. "Some.Show.2022.S01") rescues a sub-0.6 hit
that lands on the exact year. title_parse.clean_query de-dots a
folder-derived title without naive_title's extension-stripping trap.
V9: _best_match gains an optional `year`. When the top result is not a
confident textual hit (ratio < 0.6) and a year was requested, a
different result of that exact release year is preferred — TMDB already
year-filtered the search, so this is a hard corroboration, not the fuzzy
re-rank §3.3 warns against. A confident top hit is never overridden.
search_movie/search_tv forward the year.
V10: sequel_variants widened — trailing Roman→digit as well as
digit→Roman, spelled-out indices (one..twelve / un..douze / ordinals),
and a "Part N" / "Chapitre N" wrapper. Still empty for a trailing word
that is not an index or a 4-digit year.
V11: when the primary hit is already decent (>= 0.6) and there is nothing
more specific to try (no alternative_title, no sequel variant — only a
punctuation restatement left), the ladder returns without the extra
requests. The clean-title common case is back to one call.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A batch of wrong poster-grid matches found live on a real library
(2026-08-29): a two-volume film's second part matched the first; a
numbered sequel matched a same-year making-of documentary; several
entries of one franchise matched a single early entry whose localized
TMDB title is the franchise name; one matched nothing. One mechanism:
_tmdb_search returned the first candidate query whose title-similarity
ratio merely cleared 0.6, before alternative_title / the Roman-numeral
variant was ever tried.
Matching:
- title_parse: fold guessit's volume/part number back into display_title
so the parts of a multi-part film stay distinct in the query, the card
and the override.
- _tmdb_search: keep a strong PASS 1 fast path (ratio >= 0.85, one
request), otherwise score every candidate query and pick the best. A
year-exact rescue lifts a sub-0.6 top hit to the confidence floor only
when TMDB's own year-filtered result lands exactly on the filename's
year. No local re-ranking of any single result list; no tmdb.py change.
Fix match / rematch:
- _admin_exec_tmdb_override: a movie override touches its own file only
(guessit gives a whole franchise one display_title); a show override
still fans out. Corrected files are marked in media_cache.tmdb_override.
- media_cache: tmdb_override table; clear_file_tmdb / clear_tmdb_matches
drop auto-resolved matches while sparing manual corrections.
- ops.rematch_video + `meshbay-node video rematch` (loopback endpoint +
CLI verb): re-resolve a group's video matches after a matcher fix.
file_tmdb is keyed by content hash and otherwise only pruned on
deletion, so nothing dislodged a cached match before.
- a rename now drops the stale auto match too (daemon
_reenrich_renamed_video_entries).
UI:
- VideoDetailModal shows the source filename and resolved TMDB id; an
unmatched poster gets a badge (3 new video.* i18n keys x 10 locales).
So a wrong match can actually be identified before hitting Fix match.
docs/mediacenter.md 10.1 records this and the V8-V13 follow-up backlog
(show-branch ladder, year-aware _best_match, wider sequel_variants, the
0.6-0.85 extra calls, movie grid merge, per-card rematch).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
merge various-artists compilations into one album
Two real, confirmed bugs in a large flat music library:
- enrich_audio.py's sibling-cover fallback assumed one folder is one
release. A large flat "chart ranking" folder mixing dozens of unrelated
artists carried several distinct WMP AlbumArt-cache guids (one per
original album a track was ripped from), and the fallback picked
whichever one WMP had copied to Folder.jpg — attaching one unrelated
release's cover to every other track in the folder. Now refuses to pick
a cover at all once 2+ distinct guids show up, rather than guess.
- music-app.js's groupMusicEntries grouped by artist first, album second,
so a various-artists compilation (many genuinely different per-track
artists, one shared album tag, no album-artist tag at all — a real
~20-track soundtrack rip has exactly this shape) could never be
recognized as one release: every track landed alone in its own artist's
bucket and got folded into a singleton pile. Now detects an album key
shared across 2+ distinct artist keys and merges those tracks into one
compilation card under a "Various" heading instead.
Both verified against real, previously-affected files and live in the
browser: the shared wrong cover is gone, and the compilation renders as
one card with all its tracks in order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found live: a real show organized its first season as "House.of.the.Dragon.
S01E01...mkv" inside a folder named "S1" (the show name embedded in every
filename, so grouping worked by guessit's own title alone), but its later
seasons as "S02E01. Episode's Own Title.mkv" inside "S2"/"S3" — no show
name in any filename at all, relying entirely on the folder. Those seasons
showed up as loose individual entries instead of grouped under the show:
season_from_folder_name only recognized "season"/"saison"/"livre" as full
words, so "S2" didn't register as a season folder at all, and the
ancestor-based show-grouping fix (previous commits) never triggered for
those seasons.
A bare "S" + 1-2 digits as the *whole* folder name is now recognized too —
anchored to the entire name so it can't match some unrelated folder that
merely starts with "s" followed by digits.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
episodes
1. A season spanning more than one folder (a per-book Bonus folder nested
inside every numbered season) got the same synthetic episode numbers
handed out again in each folder independently — three unrelated
Specials all showing up as "S0E01". _synthetic_episode_number now ranks
across the whole show for the target season, not just one file's own
folder.
2. guessit reads a bare 3-digit leading episode number as a concatenated
season+episode guess rather than a plain episode number — "100" parses
as season=1, episode=0, not episode=100, with nothing in its output
distinguishing that from a real 2-digit episode. A season-like ancestor
already overrides guessit's season (previous commit); this applies the
same fix to episode via a direct regex on the leading number, capped at
3 digits so a leading year (4 digits) is never misread the same way.
Both confirmed against the real library this whole fix was found on:
per-book Bonus features across two books no longer collide, and a real
100th-episode file now resolves to episode 100 instead of 0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
guessit title
The mechanism itself was wrong, not just the TMDB matching: a bare episode
numbering convention with no show name in the filename at all
(`001 Episode's Own Title.ext`, no SxxExx, no show prefix — entirely
ordinary on its own) makes guessit invent a "title" from whatever text
follows the number. That text is the individual episode's own name, and
differs for every episode in the folder — trusting it, as the code did,
groups nothing together at all: every episode became its own single-
episode "show", searched against TMDB by that one-off title alone.
Whenever a season-like ancestor folder exists (a numbered season, or
Specials/Bonus/Extras -> season 0), its own root folder now names the show
unconditionally — never a per-file guessit title, which cannot tell a
show's real name from an individual episode's one-off name when the
filename carries no reliable ShowName/SxxExx structure. The walk continues
past *every* consecutive season-like ancestor, not just the first: a
per-season Bonus folder (Show/Season N/Bonus/file.ext) is nested two
levels inside the show, both "Bonus" and "Season N" season-like on their
own, and stopping at the first would hand back "Season N" as the show's
name instead of "Show".
Also adds "livre" ("book") to the season-word vocabulary (§3.4) — some
shows number their seasons that way (Roman numerals) rather than
"season"/"saison".
Supersedes _title_from_show_siblings from the previous commit (removed):
that fallback assumed a per-file title could still be trusted often
enough to be worth borrowing from a sibling season folder — this fix
means it never needs trusting in the first place once a season-like
ancestor exists.
Verified directly against the real library this was found on, not just
synthetic tests: every sample file (a numbered episode, a Book/Bonus
feature, a Book-root "making of") now resolves to the show's real name.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
unrelated standalone movies
Found live: a "Specials" folder full of one-off-named bonus episodes had
every file appear as its own poster, matched against TMDB by its own
title, because guessit finds no season/episode grammar at all in a
filename with no SxxExx of its own — so the classification (episode vs
standalone movie), based solely on that, fell to the movie branch. Real,
unrelated films happened to share several of those one-off titles and
matched confidently, one per Special, cluttering the Videos view with
dozens of wrong posters instead of grouping under the show.
An ancestor folder saying this is part of a show — a numbered season, or
Specials/Bonus/Extras -> season 0 — is now trusted over the filename
having no SxxExx of its own. The show's name can't come from this file's
own guessit title (that's the bug) or from siblings in the same Specials
folder (every one of them has the same gap) — it's borrowed from the
show's ordinary season folders next door, which do carry it in the usual
ShowName.SxxExx shape (_title_from_show_siblings). A synthetic, stable
episode number (alphabetical rank among the folder's video files) stands
in for the real one nothing in a Specials folder provides.
Generic, not specific to a folder literally named "Specials": the same
fallback fires for any season-like ancestor folder whose files lack
per-file episode grammar, numbered seasons included (covered by a
dedicated test).
Also hardens _title_from_siblings to require the sibling's own episode
number too, not just a title — otherwise it would borrow one Special's
one-off title as if it were representative, on a folder like this one.
Existing already-indexed entries will need a rescan (restart the node) to
be re-enriched under this logic — nothing re-derives them on its own.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
IndexEntry.path is the *folder* a file is in (indexer.py's
_virtual_dir docstring: "the directory a file appears in"), not the
file itself. GroupIndex.get_entry_by_path() treated it as if it named
one file, and every one of its four callers did too:
_do_music_meta_request, _do_media_meta_request, _do_tmdb_override, and
_admin_exec_tmdb_override. Any two files sharing a folder — an album is
one folder with many tracks, a season is one folder with many episodes
— collided: a lookup by path silently returned whichever entry the
index happened to iterate to first, regardless of which file the
client actually asked about.
Found live (2026-08-25): three unrelated albums ("High Tone - Various",
two "Le Peuple de l'Herbe" albums) all showed the same MusicBrainz
cover, because all their representative tracks happened to sit in one
"high_tone" folder alongside a track that legitimately matched that
cover. A force-reload didn't help — the bug is server-side, not a
stale client state.
Fixed by keying these four request/response pairs by `file_id` (the
entry's own content hash — already unique, already how every other
lookup in the system identifies a file) instead of `path`, both in the
wire messages (music_meta_req/resp, media_meta_req/resp, tmdb_override)
and in music-app.js/video-app.js's own hooks. GroupIndex.get_entry_by_path
is now unused and removed — GroupIndex.get_entry(file_id) already did
the right thing.
No test previously exercised either handler with two entries sharing a
folder — the only existing coverage (test_tmdb_override_policy.py) gave
each entry its own folder, so the bug never had a chance to show up.
Added that scenario there and in two new test files, all confirmed
failing against the pre-fix code before being confirmed green against
the fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An operator routinely shares the same physical folder into more than one
group (a music library, a Séries drive) — IndexCache used to be opened
once per group (data_dir/{group_id}/index_cache.db), so the second group
to reference an already-fully-hashed multi-terabyte folder paid the same
full content read the first one did. IndexCache itself carried no
group_id in its schema; only daemon.py's wiring did. Now one instance,
opened once at startup (data_dir/index_cache.db), shared by every group's
DirectoryIndexer.
Confirmed against a real deployment (2026-08-25/26): a group sharing an
already-indexed folder with an existing group indexes it instantly, with
zero rehashing.
Also fixes a related cross-group correctness gap found during this work:
media_cache.db (thumbnails, TMDB/MusicBrainz metadata — already node-wide,
untouched by this change) was pruned for a file the moment it left *one*
group's index, even if another group's index still held the same content
hash — forcing a redundant re-fetch/re-probe/re-thumbnail for a group that
never actually lost anything. Prune now runs only once no group's index
references the file_id any more.
Adds a node admin UI action ("Maintenance" card, prune-index-cache) to
drop cache rows that no longer belong to any group's roots — skips
anything under a root that is merely temporarily unavailable (indexer.py's
"a root that goes away freezes, never empties" rule extends to this
cache too, or a reconnected drive would pay a full rehash for no reason).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
scanning progress
Two real-world bugs found together while testing multi-root group
creation:
- CreateGroupWizard only sent the enabled-apps PUT when the operator had
*unchecked* something, assuming "every box left checked" already matched
the node's own default (Roster.DEFAULT_APPS = chat, files). It doesn't —
so leaving every app checked, the common case, silently left
Videos/Music/Photos disabled on the node. Now sent unconditionally.
- The wizard's "add extra roots" step never polled index-status, so once
step 3 (which only watches the first/upload root) finished, the
progress bar froze while the node kept scanning the remaining roots for
minutes, unwatched. Added waitForRootsIndexed (platform.js), mirroring
waitForGroupHosted's own race handling.
That fix exposed a deeper one: indexer.py's _scan_root() only flipped
`progress.scanning` on *after* walking the directory and stat()-ing every
file — both off-loop, but slow enough on a large root that a poller's
grace period (waitForRootsIndexed's 5s) could expire before ever
observing `scanning: true` (confirmed against production logs: a GEK-init
step fired 5.058s after a root started scanning, matching the grace
period almost exactly). The stat() pass was also a synchronous loop
directly on the asyncio event loop — blocking the whole daemon (WebRTC,
chat, admin UI) for as long as it took on a root with many files. Both
fixed: `scanning` now flips on before the walk starts, and stat()-ing is
now off-loop too (_size_files).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only thumbnail bytes were ever durable in media_cache.db — every other
derived field (photo width/height/EXIF, video ffprobe duration/dims,
audio cover art) lived solely on the in-memory GroupIndex entry, so a
node restart re-decoded every photo through Pillow, re-ran ffprobe on
every video, and re-scanned for every album cover from scratch, even
though the answers already sat in the cache.
Adds photo_meta and video_meta tables (content-only fields, keyed by
file_id) and checks them before doing the expensive work. Audio gets no
new table: mutagen reads tags and duration in one inseparable call, so
caching duration alone buys nothing — instead cover-art extraction alone
is skipped via a new skip_cover flag when a cached cover already exists.
Deliberately excluded from all three caches: anything derived from the
filename or folder path (video display_title/season/episode via guessit,
audio artist/album folder-fallback) — those must keep being recomputed
fresh so a rename/move is still correctly re-derived by the existing
_reenrich_renamed_*_entries mechanisms, instead of silently handing back
a stale parse under the new name/location.
Regression tests prove cache reuse by deleting the source file (or cover)
between two enrichment runs, and prove rename/move correctness survives
the new cache by renaming/moving to a path that never exists on disk.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new group application (docs/apps.md's plug-in mechanism), following the
plan in docs/photos.md. Unlike Videos/Music: several photo roots per group
instead of one (photo_roots is a set, one signed op replaces it whole),
a single album-grid view with no third-party matching step, and per-photo
info read from the file's own EXIF at index time — no metadata service,
no credential, no outbound network call at all.
Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera`
on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`.
Node: roster.py stores photo_roots as a group_settings entry (JSON list,
same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the
whole set in one op, same pattern as apps_enabled; a new PhotoEnricher
(indexer/enrich_photo.py) runs Pillow in its own small bounded pool,
separate from the video/audio pools, producing a resized thumbnail plus
the two EXIF fields — never GPS, checked by a grep-based regression test.
Client: photos-app.js — one album card per directory containing images,
a per-album photo grid, and a lightbox with next/previous (keyboard and
buttons), zoom in/out/fit/100% starting from the actual on-screen fit
percentage, and a "zip this album" button reusing files-app.js's own zip
mechanism (lifted into file-utils.js's downloadDirectory so both call the
same implementation). group-settings.js gets an add/remove multi-root
picker, distinct from Videos/Music's single-value one.
Bugs found and fixed before this ever shipped, worth keeping the story of:
- enrich_photo.py read width/height from the raw image *before* applying
EXIF orientation correction, and read DateTimeOriginal off the plain
0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which
Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict
round-trips through Pillow either way, which is exactly what would have
hidden both bugs; the regression test builds EXIF with piexif instead,
matching what real hardware produces.
- photos-app.js's album grouping stripped a trailing path segment from
entry.path under the assumption it still carried a filename — it
doesn't (files-app.js's own convention: e.path is already the
containing directory), so every album collapsed one level into its
parent. Found live against a real multi-folder library.
- transport.js's ADMIN_OP_TYPES allowlist (already the fix for an
identical bug on video_root/apps_enabled, see 4783d81) was missing
photo_roots: its admin_challenge matched no pending request and was
silently dropped, so saving a photo root just timed out after 30s with
no error.
- daemon.py pruned a thumbnail when its file left the index (root removed
or reconfigured) but never forgot the content hash was "already
attempted" — the same bytes reappearing under a renamed/relocated root
(an operator's real workflow) were then permanently skipped, forever,
with nothing to indicate why. Discarding the attempt alongside the
cache entry on prune is what makes pruning actually reversible.
- packages/meshbay-client's app:// protocol handler served every file
with no Cache-Control header, so Chromium was free to serve a stale
cached copy indefinitely — none of several `npm run sync-ui` + reload
cycles during development actually picked up the new code until the
renderer's disk cache was cleared by hand. Now sends Cache-Control:
no-store.
- the lightbox's zoomed image used flex centering (align-items/
justify-content: center) combined with overflow: auto — a well-known
trap where the browser centers overflowing content by shifting it, and
the leading half of that overflow (here, the top of a zoomed photo)
sits outside what the scrollport can actually reach. Reported live as
"unusable". Fixed by switching to top/left alignment once zoomed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL
|
| |
|
|
|
|
|
|
|
|
| |
The "P.H. Theme" failure investigated earlier turned out to be a genuinely
corrupt 1256-byte source file with no audio stream at all, just an ID3
tag — a real, if rare, corruption pattern worth guarding against directly
rather than only handling gracefully at playback time. Scoped to audio
only, applied wherever a file actually gets hashed/typed (fresh scan and
the cache-miss rehash path alike) — a tiny file of any other type is still
indexed normally.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A real-library scan turned up 250 .wma and 23 .mpc files that the indexer
was silently classifying as "other" — genuinely lost from the Music app,
not a consolidation-rule artifact (checked separately: the grouping logic
itself drops nothing). Both are now indexed as audio and tagged properly:
- WMA has no mutagen "easy" wrapper, so the generic tag reader was reading
nothing from it at all. Reads the real ASF keys directly instead
(Title/Author/WM-AlbumTitle/WM-TrackNumber), confirmed against a real
sample file before writing the mapping.
- Musepack's format auto-detection is unreliable enough (misidentified a
real .mpc as MP3 in spot checks) that it now always opens by its own
class instead of guessing from content.
- Filters out another placeholder value found along the way: a French
ripping tool's auto-generated "Album inconnu (<timestamp>)".
Neither format decodes natively in a browser's <audio> element, so this
gets them correctly visible, tagged, and covered — not yet playable
in-browser. That would need server-side transcoding, deliberately left
out of this change.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Album-grid readability, part two:
- groupMusicEntries (music-app.js): an album bucket left with exactly
one track - a real album tag, but only one song from it, not the
whole release - clutters the grid the same way an untagged loose
track does. Both kinds now fold into one "<artist> - Various" tile
per artist, unless there is only one leftover track overall, where
relabeling buys nothing and the track keeps its own name (or the
generic placeholder, if it never had one).
- foldKey also normalizes "&" vs "and" ("Artist & The Band" / "Artist
and The Band" is one act, tagged both ways across different rips of
the same catalogue) alongside the existing case/whitespace fold.
- music-player.js: a close button pauses and tears the player down;
an unmount cleanup effect (pause, revoke every cached blob URL)
fires either way, whether that's the close button or the shell
tearing the bar down on its own. A "current queue" button opens an
overlay listing the whole playing queue with the current track
highlighted, click any to jump to it - works identically regardless
of how the queue was built (an album, the consolidated misc bucket,
a single standalone track), since it only ever reads the player's
own live tracks/order/pos.
- group-page.js: this component is not remounted when switching to a
*different* group on the same /group/:id route (only the groupId
prop changes) - so without an explicit reset, music from one group
would carry into the next one opened. Resets musicQueue to null on
groupId change; a tab switch inside one group still leaves it alone.
- Scrubbed real artist/band names that had leaked into code comments
and test fixtures (enrich_audio.py's docstrings, several
test_enrich_audio.py assertions, a music-app.js comment) - replaced
with generic placeholders, no behavioural change.
- i18n: music.various, music.player_close, music.player_queue,
music.queue_title added across all ten locales.
Client-side only except none of this touches the node at all. npm run
sync-ui re-run. Full suite: 1129 passed, no regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Music grouping was measured against a real ~5700-file library and came
back worse than a plain file listing. Root cause: the artist/album
ancestor walk always climbed exactly two levels (parent = album,
grandparent = artist) with no idea where the group's own shared root
was. Any file in a flat top-level folder — common here: bare
`Artist/track.mp3`, no album subfolder at all — had its "grandparent"
resolve to the root directory's own name, so the artist got replaced
by the share's name. Measured: 289 of 5664 tracks across 41 real,
unrelated artists (Ben Harper, Dire Straits, Jimi Hendrix, Janis
Joplin, ...) collapsed into one fake artist this way — the single
biggest bucket in the whole library, ahead of every real one.
- `_artist_album_from_ancestors` now takes the entry's own root
boundary (daemon.py resolves it via `RootSet.split`) and refuses to
read it as a name. A file sitting in a top-level folder — genuinely
ambiguous, artist or a standalone album/compilation — is handled by
`_split_top_level_folder`: split on "Artist - Album" when the
(cleaned) folder name has that shape, otherwise the whole name
becomes the artist alone, the more common real case here.
- `_clean_tag` treats known tagger placeholders ("No Artist", a French
tool's "Nouvel artiste (334)") as absent rather than a real value —
they were just as truthy as a real name and were locking out the
fallback that would have done better. "Various Artists" is kept, a
real compilation credit rather than a placeholder.
- A `title` tag that's the bare filename copied verbatim (track number
included — found live on a whole CD-single) is stripped through the
same prefix rule the filename parser already used
(`title_parse.strip_track_prefix`), since a tag normally wins over
the parsed title.
- Cover art: only 11% of a 400-file sample had embedded art (expected
for this era of rip), but 267 loose cover images sit beside the
tracks across the library (Windows Media Player's `Folder.jpg`/
`AlbumArt_{guid}_*.jpg`, manual `cover.jpg`) and were never looked
at. `_find_sibling_cover` checks the track's own folder before
giving up — measured coverage 11% -> 26% on the same library, zero
network calls.
`_enriched_attempted` is in-memory and resets on restart, so a node
restart is enough to re-run enrichment over an already-scanned library
with the fixed logic — no rescan flag, no cache to clear by hand.
22 tests in test_enrich_audio.py (11 new), including the exact
regression case end to end through the real pool. Full suite: 1129
passed, no regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements the node half of docs/musicbay.md against MNP 0.8:
- IndexEntry gains artist/album/track_no (reuses duration/thumb_hash/
display_title, already generic). New musicbrainz_config/_enabled and
music_meta_req/_resp message pairs, mirroring the TMDB shape.
- title_parse.parse_track_filename: track-number-prefix + title parsing,
fallback-only (embedded tags are the primary source, unlike Videos).
- indexer.enrich_audio.AudioEnricher: mutagen-based tag/embedded-cover
extraction through its own bounded pool (asyncio.to_thread, no
subprocess — no ffmpeg-shaped deadlock risk). Gated on "music" in a
group's enabled_apps rather than a video_root-style scoped folder.
- musicbrainz.py: MusicBrainzClient — no API key (unlike TMDB), just a
self-imposed ~1 req/s pace and a configurable, non-default User-Agent
contact string; inert (no calls at all) when no contact is configured,
never sends an unidentified client.
- media_cache.py: file_mbid/mbid_meta tables alongside the existing TMDB
ones, cover art reusing the thumbs table via a synthetic
musicbrainz:{mbid} id, pruned on file deletion.
- roster.py/ops.py/webrtc_server.py: musicbrainz_contact (node-wide) and
musicbrainz_enabled (per-group, from the start) as signed operator
settings, ALLOWED_APPS gains "music", _do_music_meta_request resolves
and caches a release-level MusicBrainz match per (artist, album).
- daemon.py: AudioEnricher/MusicBrainzClient wired alongside the video
ones; a group's existing library is swept when "music" is newly
enabled (no video_root equivalent — see musicbay.md §2.1).
41 new tests (musicbrainz.py against a mocked transport, admin-op policy
for both new settings, media_cache round-trip/pruning, enrich_audio
end-to-end against real ffmpeg-generated MP3s). Full suite (common +
node + hub): 1116 passed, no regressions.
Client-side (music-app.js, persistent player bar) not started yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three bugs found live testing the Videos app against a real HEVC/EAC3 show,
plus a design change requested afterward:
- Streaming always did "-c:v copy", which faithfully reports a source's real
hev1 codec string but is unplayable in a browser with no HEVC decoder
(most Chrome/Linux builds). The node now transcodes to H264 whenever the
probed codec is browser-incompatible (media_probe.py's new
BROWSER_INCOMPATIBLE_VIDEO_CODECS), with a `transcode_incompatible_video`
node.toml opt-out for operators who know their viewers already decode it.
- Dropping a whole season into an already-watched folder gave no scanning
indicator and no progress bar: IndexProgress was only ever updated by the
two bulk scan paths, never by the real-time per-file watchdog path
(_schedule_update/_debounce/_update_entry). That path now accounts a
"burst" the same way, without double-counting a file rewritten mid-debounce.
- A stray literal "0" rendered in the video detail modal when there was no
TMDB match (`meta.confidence` is 0, and `0 && x` renders "0" in JSX/htm,
not nothing) — `confident` is now a real boolean.
- Whether TMDB is used at all moves from a node-wide setting to per-group
(OP_TMDB_ENABLED/tmdb_enabled/tmdb_enabled_ack, scoped like OP_VIDEO_ROOT):
an operator running a real media-library group alongside test/demo groups
on one node wants outbound TMDB traffic for the one that needs it, not all
of them. The custom API token and query language stay node-wide, one
shared credential/cache (tmdb_config/OP_TMDB_CONFIG, unchanged reasoning).
MNP_VERSION 0.6 -> 0.7, additive.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements docs/mediacenter.md: a "Videos" group application built on the
existing files index rather than a separate catalogue. On the node side,
new indexer enrichment (technical probe, filename/season parsing, thumbnail
generation) runs per-file once an operator has chosen a video_root for the
group, plus a TMDB client for on-demand poster/metadata lookups (never
client-side, thumbnails delivered over the existing chunk path). On the hub
side, a new video-app.js renders a lazily-mounted poster grid or a
thumbnail-only flat list, with TMDB entirely optional per group.
Along the way: the global apps registry now drives Settings' default-tab
picker instead of a hardcoded list, and the video_root is configured from
group Settings (like uploads) rather than from Files, with the node
refusing to run any TMDB/thumbnail work until one is set.
Fixes several bugs found via live testing against a real library, notably
a race between two effects writing the same "image ready" state that could
leave a poster grid spinning forever on a same-tab revisit — see
mediacenter.md §5.4 for the full account of each one.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reconcile, and delta sync
Indexer performance work, in four parts:
- Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node
restart no longer re-hashes every file — measured at 23 minutes for a
114 GB library on a slow disk before this, near-instant after. Hashing
is deliberately kept sequential (max_workers=1): it was never actually
concurrent despite the pool size, and two interleaved reads seek-thrash
a spinning disk instead of going faster.
- Byte-based scan progress (IndexProgress), surfaced via the loopback
index-status route, the handshake ack, and a periodic INDEX_PROGRESS
push to connected peers — drives a progress bar in the Create Group
wizard and "add a directory" in Settings, and an animated presence dot.
Guaranteed to settle back to idle via try/finally and a final push on
the scanning->false transition.
- The reconcile backstop's directory walks now run in the executor
instead of blocking the daemon's event loop; its interval defaults to
10 min (was 60s) with adaptive backoff to 2h when nothing changes,
reset on a real change or a peer connecting, and is now a per-group
operator setting (signed op + group Settings UI).
- INDEX_DELTA wired up (protocol support existed, nothing called it):
_on_index_change now sends additions/deletions instead of rebuilding
the full entries list, coalesced over a short window so a burst of
file events produces one push, and the hub swarm registration for
public groups only (re-)registers newly added hashes.
Also fixes several bugs found while testing the above against real
libraries (a 114 GB and a 100+ GB group on a USB HDD):
- /api/reload blocked until the reload — including a brand-new group's
full initial scan — finished, which the Electron bridge's fixed 30s
call timeout turned into a hard failure on any real library. The route
now fires the reload without waiting (ops.start_reload), matching
add_root/remove_root's existing pattern; the wizard's own step order
was fixed to wait for the group to actually appear hosted before the
steps that need it (extra roots, GEK), with retries for the residual
race between that and the daemon's own bookkeeping.
- transport.js's hand-rolled msgpack codec had no case for uint64/int64
(0xcf/0xd3) and crashed decoding any message containing one — hit by
IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB.
Verified against real msgpack-encoded bytes from the Python side.
- chat_hist_resp, and this change's own index_progress and
set_scan_settings_ack pushes, were not routed by message type and
could be handed to an unrelated pending request by the transport's
"oldest pending" fallback, stalling it until its own 30s timeout and
corrupting whatever received the wrong reply in its place.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Node daemon no longer blocks startup on slow directory scans — initial
indexing runs in the background so the node reaches "running" immediately
after transports are up. Fixes the wizard failing to detect the node when
large USB/NAS roots take minutes to scan.
Also: wizard key-linking deadlock resolved (main.js links during poll),
invite form stays in DOM during reconnects (disabled instead of destroyed),
pairing code bridges to renderer, and firewall docs for LAN casting added.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage A — a group's content is a set of named roots
---------------------------------------------------
`shared_dir` becomes a list of {name, path, kind}. The name is the directory's
basename, derived once at add time and *stored*: recomputing it would
re-identify a whole library the day someone renames a folder on disk. Duplicate
names are refused case-insensitively and no root may contain another — both
compared with NFC folding, because most of these directories live on exFAT or
NTFS where `Films` and `films` are one directory.
Every index path carries its root name, in a one-root group as much as in a
five-root one. One path shape has to be got right once; two have to be kept
right for ever.
**A root that goes away freezes; it never empties.** Unmounting a volume makes
watchdog report every file under it as deleted, or presents an empty directory
to the next scan. Acting on either propagates deletions for a whole library to
every member, as though the owner had erased it. So a deletion is acted on only
once its root is confirmed readable, and availability is tracked per root — one
unplugged drive leaves the others serving. 12 tests, verified to fail against an
indexer without the check.
Events are not trusted to be complete either: ReadDirectoryChangesW drops them
under load and inotify on a FUSE mount misses changes made outside it. A
periodic reconciliation sweep is the only thing that recovers a missed event.
MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content
hash, a node id and an endpoint — no paths, no filenames — and private groups
register nothing (H7).
Stage B — one implementation behind every front door
----------------------------------------------------
C1 and C6 were both "a second path into the node with its own weaker
handshake". Two implementations of `revoke` with two authorization checks is
that shape one size down. `meshbay_node/ops.py` holds each operation once,
takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback
API is one `_op(...)` line per endpoint; the MNP handlers call the same
functions. test_ops.py asserts the shape rather than trusting it.
Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload`
(SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a
browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP
operations: rotation is the half of revocation that revocation cannot do, since
the ex-member holds the current key, and the node generates the replacement
with its own CSPRNG — no key material crosses the wire, which is what the C5b
rule is actually about.
Two bugs found by running it rather than by testing it
------------------------------------------------------
GroupIndex is keyed by **content hash**, so the same bytes at two paths are one
entry — which is also why a scan reports ten files and indexes nine.
Reconciliation compared paths, so it decided the second path was a missed event
every 60 s, rewrote the entry and pushed an index update to every connected
peer. Seen in a live node's log.
`meshbay-node reload` crashed on first use with `subprocess` unimported: the
module compiles fine, which is the "syntax, not names" trap already recorded for
the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be
added to the parser without an entry there.
Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the
wire carried "0.2" — harmless only because nothing imported it. And
_do_dir_create/_do_dir_delete referenced an undefined `filename` on their error
path.
740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Complete browser-based client: Preact SPA with login, group file browser,
encrypted download, video playback, group chat, i18n, and dark/light theme.
Browser connects P2P to nodes behind residential NAT via WebRTC DataChannel
(aiortc). Hub handles signaling only — all data flows E2E.
Performance: pipelined downloads (8-chunk sliding window), binary msgpack
wire format (no base64), redundant I/O elimination. Large file downloads
stream to disk via File System Access API (showSaveFilePicker).
Validated on SFR + Orange residential NATs, Chrome + Firefox, IPv4/IPv6.
132 tests passing. Deployed to meshbay.org + Orange node.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1. indexer.py: streaming blake3 (8MB chunks) instead of read_bytes().
Large files (initrd.img, ISOs, VM images) no longer load into RAM.
2. QE/demo-v1/run_node.py: call indexer.start() not initial_scan().
initial_scan() alone never starts the watchdog observer — files added
after startup were silently ignored. Added indexer.stop() on shutdown.
3. USERGUIDE.md §8: clarify symmetric vs asymmetric.
Ed25519/X25519 = asymmetric (key pairs). ChaCha20-Poly1305 and
AES-256-GCM = symmetric AEAD 256-bit (content encryption).
ChaCha20 is PRIMARY; AES-GCM is optional browser-compat variant only.
4. pyproject.toml: aioquic, websockets, aiosqlite, slowapi added to
proper package deps (were installed manually, now declared).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
| |
GroupIndex: msgpack→zstd→GEK-encrypt→sign for private groups,
plaintext+sign for public groups. DirectoryIndexer: watchdog-based
watcher, async initial scan via thread pool, on_change callback.
Delta support (diff between versions). 10/10 tests passing.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|