meshbay.git/packages/meshbay-node/src/meshbay_node/indexer, branch 0.7

fix(node): skip indexing audio files under 50KB, likely-corrupt source

2026-08-24T18:55:52Z

The "P.H. Theme" failure investigated earlier turned out to be a genuinely corrupt 1256-byte source file with no audio stream at all, just an ID3 tag — a real, if rare, corruption pattern worth guarding against directly rather than only handling gracefully at playback time. Scoped to audio only, applied wherever a file actually gets hashed/typed (fresh scan and the cache-miss rehash path alike) — a tiny file of any other type is still indexed normally.

feat(node): recognize WMA and Musepack as audio, read their real tag keys

2026-08-24T18:31:04Z

A real-library scan turned up 250 .wma and 23 .mpc files that the indexer was silently classifying as "other" — genuinely lost from the Music app, not a consolidation-rule artifact (checked separately: the grouping logic itself drops nothing). Both are now indexed as audio and tagged properly: - WMA has no mutagen "easy" wrapper, so the generic tag reader was reading nothing from it at all. Reads the real ASF keys directly instead (Title/Author/WM-AlbumTitle/WM-TrackNumber), confirmed against a real sample file before writing the mapping. - Musepack's format auto-detection is unreliable enough (misidentified a real .mpc as MP3 in spot checks) that it now always opens by its own class instead of guessing from content. - Filters out another placeholder value found along the way: a French ripping tool's auto-generated "Album inconnu ()". Neither format decodes natively in a browser's element, so this gets them correctly visible, tagged, and covered — not yet playable in-browser. That would need server-side transcoding, deliberately left out of this change.

feat(hub): consolidate loose tracks, "&"/"and" fold, player close/queue

2026-08-24T17:46:06Z

Album-grid readability, part two: - groupMusicEntries (music-app.js): an album bucket left with exactly one track - a real album tag, but only one song from it, not the whole release - clutters the grid the same way an untagged loose track does. Both kinds now fold into one " - Various" tile per artist, unless there is only one leftover track overall, where relabeling buys nothing and the track keeps its own name (or the generic placeholder, if it never had one). - foldKey also normalizes "&" vs "and" ("Artist & The Band" / "Artist and The Band" is one act, tagged both ways across different rips of the same catalogue) alongside the existing case/whitespace fold. - music-player.js: a close button pauses and tears the player down; an unmount cleanup effect (pause, revoke every cached blob URL) fires either way, whether that's the close button or the shell tearing the bar down on its own. A "current queue" button opens an overlay listing the whole playing queue with the current track highlighted, click any to jump to it - works identically regardless of how the queue was built (an album, the consolidated misc bucket, a single standalone track), since it only ever reads the player's own live tracks/order/pos. - group-page.js: this component is not remounted when switching to a *different* group on the same /group/:id route (only the groupId prop changes) - so without an explicit reset, music from one group would carry into the next one opened. Resets musicQueue to null on groupId change; a tab switch inside one group still leaves it alone. - Scrubbed real artist/band names that had leaked into code comments and test fixtures (enrich_audio.py's docstrings, several test_enrich_audio.py assertions, a music-app.js comment) - replaced with generic placeholders, no behavioural change. - i18n: music.various, music.player_close, music.player_queue, music.queue_title added across all ten locales. Client-side only except none of this touches the node at all. npm run sync-ui re-run. Full suite: 1129 passed, no regressions. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy

fix(node): stop inventing a fake artist from the shared root's name

2026-08-24T16:30:00Z

Music grouping was measured against a real ~5700-file library and came back worse than a plain file listing. Root cause: the artist/album ancestor walk always climbed exactly two levels (parent = album, grandparent = artist) with no idea where the group's own shared root was. Any file in a flat top-level folder — common here: bare `Artist/track.mp3`, no album subfolder at all — had its "grandparent" resolve to the root directory's own name, so the artist got replaced by the share's name. Measured: 289 of 5664 tracks across 41 real, unrelated artists (Ben Harper, Dire Straits, Jimi Hendrix, Janis Joplin, ...) collapsed into one fake artist this way — the single biggest bucket in the whole library, ahead of every real one. - `_artist_album_from_ancestors` now takes the entry's own root boundary (daemon.py resolves it via `RootSet.split`) and refuses to read it as a name. A file sitting in a top-level folder — genuinely ambiguous, artist or a standalone album/compilation — is handled by `_split_top_level_folder`: split on "Artist - Album" when the (cleaned) folder name has that shape, otherwise the whole name becomes the artist alone, the more common real case here. - `_clean_tag` treats known tagger placeholders ("No Artist", a French tool's "Nouvel artiste (334)") as absent rather than a real value — they were just as truthy as a real name and were locking out the fallback that would have done better. "Various Artists" is kept, a real compilation credit rather than a placeholder. - A `title` tag that's the bare filename copied verbatim (track number included — found live on a whole CD-single) is stripped through the same prefix rule the filename parser already used (`title_parse.strip_track_prefix`), since a tag normally wins over the parsed title. - Cover art: only 11% of a 400-file sample had embedded art (expected for this era of rip), but 267 loose cover images sit beside the tracks across the library (Windows Media Player's `Folder.jpg`/ `AlbumArt_{guid}_*.jpg`, manual `cover.jpg`) and were never looked at. `_find_sibling_cover` checks the track's own folder before giving up — measured coverage 11% -> 26% on the same library, zero network calls. `_enriched_attempted` is in-memory and resets on restart, so a node restart is enough to re-run enrichment over an already-scanned library with the fixed logic — no rescan flag, no cache to clear by hand. 22 tests in test_enrich_audio.py (11 new), including the exact regression case end to end through the real pool. Full suite: 1129 passed, no regressions. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy

feat(node): Music app node-side — indexing, MusicBrainz enrichment, protocol

2026-08-24T15:12:36Z

Implements the node half of docs/musicbay.md against MNP 0.8: - IndexEntry gains artist/album/track_no (reuses duration/thumb_hash/ display_title, already generic). New musicbrainz_config/_enabled and music_meta_req/_resp message pairs, mirroring the TMDB shape. - title_parse.parse_track_filename: track-number-prefix + title parsing, fallback-only (embedded tags are the primary source, unlike Videos). - indexer.enrich_audio.AudioEnricher: mutagen-based tag/embedded-cover extraction through its own bounded pool (asyncio.to_thread, no subprocess — no ffmpeg-shaped deadlock risk). Gated on "music" in a group's enabled_apps rather than a video_root-style scoped folder. - musicbrainz.py: MusicBrainzClient — no API key (unlike TMDB), just a self-imposed ~1 req/s pace and a configurable, non-default User-Agent contact string; inert (no calls at all) when no contact is configured, never sends an unidentified client. - media_cache.py: file_mbid/mbid_meta tables alongside the existing TMDB ones, cover art reusing the thumbs table via a synthetic musicbrainz:{mbid} id, pruned on file deletion. - roster.py/ops.py/webrtc_server.py: musicbrainz_contact (node-wide) and musicbrainz_enabled (per-group, from the start) as signed operator settings, ALLOWED_APPS gains "music", _do_music_meta_request resolves and caches a release-level MusicBrainz match per (artist, album). - daemon.py: AudioEnricher/MusicBrainzClient wired alongside the video ones; a group's existing library is swept when "music" is newly enabled (no video_root equivalent — see musicbay.md §2.1). 41 new tests (musicbrainz.py against a mocked transport, admin-op policy for both new settings, media_cache round-trip/pruning, enrich_audio end-to-end against real ffmpeg-generated MP3s). Full suite (common + node + hub): 1116 passed, no regressions. Client-side (music-app.js, persistent player bar) not started yet. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy

fix(node,hub): HEVC transcode fallback, live-add progress, per-group TMDB toggle

2026-08-24T13:57:41Z

Three bugs found live testing the Videos app against a real HEVC/EAC3 show, plus a design change requested afterward: - Streaming always did "-c:v copy", which faithfully reports a source's real hev1 codec string but is unplayable in a browser with no HEVC decoder (most Chrome/Linux builds). The node now transcodes to H264 whenever the probed codec is browser-incompatible (media_probe.py's new BROWSER_INCOMPATIBLE_VIDEO_CODECS), with a `transcode_incompatible_video` node.toml opt-out for operators who know their viewers already decode it. - Dropping a whole season into an already-watched folder gave no scanning indicator and no progress bar: IndexProgress was only ever updated by the two bulk scan paths, never by the real-time per-file watchdog path (_schedule_update/_debounce/_update_entry). That path now accounts a "burst" the same way, without double-counting a file rewritten mid-debounce. - A stray literal "0" rendered in the video detail modal when there was no TMDB match (`meta.confidence` is 0, and `0 && x` renders "0" in JSX/htm, not nothing) — `confident` is now a real boolean. - Whether TMDB is used at all moves from a node-wide setting to per-group (OP_TMDB_ENABLED/tmdb_enabled/tmdb_enabled_ack, scoped like OP_VIDEO_ROOT): an operator running a real media-library group alongside test/demo groups on one node wants outbound TMDB traffic for the one that needs it, not all of them. The custom API token and query language stay node-wide, one shared credential/cache (tmdb_config/OP_TMDB_CONFIG, unchanged reasoning). MNP_VERSION 0.6 -> 0.7, additive. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY

feat(node,hub): add Videos group app (poster grid, flat list, TMDB metadata)

2026-08-24T08:04:46Z

Implements docs/mediacenter.md: a "Videos" group application built on the existing files index rather than a separate catalogue. On the node side, new indexer enrichment (technical probe, filename/season parsing, thumbnail generation) runs per-file once an operator has chosen a video_root for the group, plus a TMDB client for on-demand poster/metadata lookups (never client-side, thumbnails delivered over the existing chunk path). On the hub side, a new video-app.js renders a lazily-mounted poster grid or a thumbnail-only flat list, with TMDB entirely optional per group. Along the way: the global apps registry now drives Settings' default-tab picker instead of a hardcoded list, and the video_root is configured from group Settings (like uploads) rather than from Files, with the node refusing to run any TMDB/thumbnail work until one is set. Fixes several bugs found via live testing against a real library, notably a race between two effects writing the same "image ready" state that could leave a poster grid spinning forever on a same-tab revisit — see mediacenter.md §5.4 for the full account of each one.

feat(node): persistent index cache, visible scan progress, adaptive reconcile, and delta sync

2026-08-23T19:55:20Z

Indexer performance work, in four parts: - Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node restart no longer re-hashes every file — measured at 23 minutes for a 114 GB library on a slow disk before this, near-instant after. Hashing is deliberately kept sequential (max_workers=1): it was never actually concurrent despite the pool size, and two interleaved reads seek-thrash a spinning disk instead of going faster. - Byte-based scan progress (IndexProgress), surfaced via the loopback index-status route, the handshake ack, and a periodic INDEX_PROGRESS push to connected peers — drives a progress bar in the Create Group wizard and "add a directory" in Settings, and an animated presence dot. Guaranteed to settle back to idle via try/finally and a final push on the scanning->false transition. - The reconcile backstop's directory walks now run in the executor instead of blocking the daemon's event loop; its interval defaults to 10 min (was 60s) with adaptive backoff to 2h when nothing changes, reset on a real change or a peer connecting, and is now a per-group operator setting (signed op + group Settings UI). - INDEX_DELTA wired up (protocol support existed, nothing called it): _on_index_change now sends additions/deletions instead of rebuilding the full entries list, coalesced over a short window so a burst of file events produces one push, and the hub swarm registration for public groups only (re-)registers newly added hashes. Also fixes several bugs found while testing the above against real libraries (a 114 GB and a 100+ GB group on a USB HDD): - /api/reload blocked until the reload — including a brand-new group's full initial scan — finished, which the Electron bridge's fixed 30s call timeout turned into a hard failure on any real library. The route now fires the reload without waiting (ops.start_reload), matching add_root/remove_root's existing pattern; the wizard's own step order was fixed to wait for the group to actually appear hosted before the steps that need it (extra roots, GEK), with retries for the residual race between that and the daemon's own bookkeeping. - transport.js's hand-rolled msgpack codec had no case for uint64/int64 (0xcf/0xd3) and crashed decoding any message containing one — hit by IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB. Verified against real msgpack-encoded bytes from the Python side. - chat_hist_resp, and this change's own index_progress and set_scan_settings_ack pushes, were not routed by message type and could be handed to an unrelated pending request by the transport's "oldest pending" fallback, stalling it until its own 30s timeout and corrupting whatever received the wrong reply in its place. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA

fix: first-run wizard reliability and node startup performance

2026-08-22T14:40:54Z

Node daemon no longer blocks startup on slow directory scans — initial indexing runs in the background so the node reaches "running" immediately after transports are up. Fixes the wizard failing to detect the node when large USB/NAS roots take minutes to scan. Also: wizard key-linking deadlock resolved (main.js links during poll), invite form stays in DOM during reconnects (disabled instead of destroyed), pairing code bridges to renderer, and firewall docs for LAN casting added. Co-Authored-By: Claude Opus 4.6

feat(node): several named roots per group, and one implementation per operation

2026-08-18T00:15:02Z

Stage A — a group's content is a set of named roots --------------------------------------------------- `shared_dir` becomes a list of {name, path, kind}. The name is the directory's basename, derived once at add time and *stored*: recomputing it would re-identify a whole library the day someone renames a folder on disk. Duplicate names are refused case-insensitively and no root may contain another — both compared with NFC folding, because most of these directories live on exFAT or NTFS where `Films` and `films` are one directory. Every index path carries its root name, in a one-root group as much as in a five-root one. One path shape has to be got right once; two have to be kept right for ever. **A root that goes away freezes; it never empties.** Unmounting a volume makes watchdog report every file under it as deleted, or presents an empty directory to the next scan. Acting on either propagates deletions for a whole library to every member, as though the owner had erased it. So a deletion is acted on only once its root is confirmed readable, and availability is tracked per root — one unplugged drive leaves the others serving. 12 tests, verified to fail against an indexer without the check. Events are not trusted to be complete either: ReadDirectoryChangesW drops them under load and inotify on a FUSE mount misses changes made outside it. A periodic reconciliation sweep is the only thing that recovers a missed event. MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content hash, a node id and an endpoint — no paths, no filenames — and private groups register nothing (H7). Stage B — one implementation behind every front door ---------------------------------------------------- C1 and C6 were both "a second path into the node with its own weaker handshake". Two implementations of `revoke` with two authorization checks is that shape one size down. `meshbay_node/ops.py` holds each operation once, takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback API is one `_op(...)` line per endpoint; the MNP handlers call the same functions. test_ops.py asserts the shape rather than trusting it. Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload` (SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP operations: rotation is the half of revocation that revocation cannot do, since the ex-member holds the current key, and the node generates the replacement with its own CSPRNG — no key material crosses the wire, which is what the C5b rule is actually about. Two bugs found by running it rather than by testing it ------------------------------------------------------ GroupIndex is keyed by **content hash**, so the same bytes at two paths are one entry — which is also why a scan reports ten files and indexes nine. Reconciliation compared paths, so it decided the second path was a missed event every 60 s, rewrote the entry and pushed an index update to every connected peer. Seen in a live node's log. `meshbay-node reload` crashed on first use with `subprocess` unimported: the module compiles fine, which is the "syntax, not names" trap already recorded for the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be added to the parser without an entry there. Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the wire carried "0.2" — harmless only because nothing imported it. And _do_dir_create/_do_dir_delete referenced an undefined `filename` on their error path. 740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment. Co-Authored-By: Claude Opus 5