| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Paste an http(s) link in a group's chat and it unfurls into an OpenGraph
card — title, description, site name, and image — the way WhatsApp/Signal/
Slack do it.
The fetch is the node's, never the browser's or the hub's. The browser
cannot: a strict img-src/connect-src and CORS block it, and a direct fetch
would leak every reader's IP to the linked host on each render. The hub must
not touch group content (draft-v6 §2.5). The node already fetches third-party
metadata for the Videos and Music apps, over the same authorised path.
Flow mirrors media_meta_req: the client sends `link_preview_req {url}`, the
node replies `link_preview_resp` with the card fields (or `ok: false`), and
any OG image is stored under its blake3 in the existing media_cache thumb
store — the client then fetches it via the normal file_req path, exactly like
a poster. Nothing durable is added: the card text lives in a bounded in-memory
TTL cache on the node (draft-v6 §2.7 — enrichment on demand, the asking device
caches), and MNP goes 0.11 → 0.12 (additive: an older node logs "unknown type"
and the client shows the bare link).
Because the URL is chosen by a *member* and triggers an outbound request from
the operator's machine, `linkpreview.safe_url` is an SSRF gate: http(s) only,
no credentials, and every resolved address must be globally routable — no
loopback, private, link-local, multicast or reserved range, cloud-metadata
included. Redirects are followed by hand so each hop is re-checked. Residual,
documented in the module: DNS rebinding between the check and connect, closed
properly by pinning the checked IP — a follow-up.
Also fixes a long-standing chat annoyance the preview cards made worse:
opening the Chat tab landed a screen or two above the newest message because
the scroll-to-bottom ran before attachment thumbnails and (now) preview cards
had loaded and grown the content. A ResizeObserver keeps the view pinned to
the bottom through late content growth, and does nothing once the reader
scrolls up.
Tests: test_linkpreview.py (the SSRF gate and the OpenGraph parse, incl.
redirect re-validation and image downscaling) and test_link_preview_request.py
(reply shape, the media_cache image round-trip, the result cache).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018gKJ85aZyvEwarXMFzFEwi
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
merge various-artists compilations into one album
Two real, confirmed bugs in a large flat music library:
- enrich_audio.py's sibling-cover fallback assumed one folder is one
release. A large flat "chart ranking" folder mixing dozens of unrelated
artists carried several distinct WMP AlbumArt-cache guids (one per
original album a track was ripped from), and the fallback picked
whichever one WMP had copied to Folder.jpg — attaching one unrelated
release's cover to every other track in the folder. Now refuses to pick
a cover at all once 2+ distinct guids show up, rather than guess.
- music-app.js's groupMusicEntries grouped by artist first, album second,
so a various-artists compilation (many genuinely different per-track
artists, one shared album tag, no album-artist tag at all — a real
~20-track soundtrack rip has exactly this shape) could never be
recognized as one release: every track landed alone in its own artist's
bucket and got folded into a singleton pile. Now detects an album key
shared across 2+ distinct artist keys and merges those tracks into one
compilation card under a "Various" heading instead.
Both verified against real, previously-affected files and live in the
browser: the shared wrong cover is gone, and the compilation renders as
one card with all its tracks in order.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found live: "fix match" appeared to work for two shows but not for a movie
whose own automatic search kept landing on the same wrong result. The
override only ever recorded the file->tmdb_id mapping — never the
metadata that id actually names. _do_media_meta_request's cache check
agrees the mapping is fresh (same media_type) but finds nothing under
that *new* id in tmdb_meta, since nothing had ever fetched it, and falls
through to a brand-new search using the file's own title — reproducing
the exact match the override was meant to replace.
This stayed invisible for the two shows only because their own title
happened to be enough for that fallback search to land on the right
answer anyway, entirely independent of whatever the override recorded —
never because the override was actually being honored. It surfaced on a
movie whose own title search kept landing on the same wrong match
regardless.
Fetches and stores the real metadata for the chosen tmdb_id up front (via
the existing _tmdb_build_meta, which needs only the id — no search result
object required), so a later lookup finds the override itself instead of
falling through to a search blind to it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Found live: a real show organized its first season as "House.of.the.Dragon.
S01E01...mkv" inside a folder named "S1" (the show name embedded in every
filename, so grouping worked by guessit's own title alone), but its later
seasons as "S02E01. Episode's Own Title.mkv" inside "S2"/"S3" — no show
name in any filename at all, relying entirely on the folder. Those seasons
showed up as loose individual entries instead of grouped under the show:
season_from_folder_name only recognized "season"/"saison"/"livre" as full
words, so "S2" didn't register as a season folder at all, and the
ancestor-based show-grouping fix (previous commits) never triggered for
those seasons.
A bare "S" + 1-2 digits as the *whole* folder name is now recognized too —
anchored to the entire name so it can't match some unrelated folder that
merely starts with "s" followed by digits.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
episodes
1. A season spanning more than one folder (a per-book Bonus folder nested
inside every numbered season) got the same synthetic episode numbers
handed out again in each folder independently — three unrelated
Specials all showing up as "S0E01". _synthetic_episode_number now ranks
across the whole show for the target season, not just one file's own
folder.
2. guessit reads a bare 3-digit leading episode number as a concatenated
season+episode guess rather than a plain episode number — "100" parses
as season=1, episode=0, not episode=100, with nothing in its output
distinguishing that from a real 2-digit episode. A season-like ancestor
already overrides guessit's season (previous commit); this applies the
same fix to episode via a direct regex on the leading number, capped at
3 digits so a leading year (4 digits) is never misread the same way.
Both confirmed against the real library this whole fix was found on:
per-book Bonus features across two books no longer collide, and a real
100th-episode file now resolves to episode 100 instead of 0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
guessit title
The mechanism itself was wrong, not just the TMDB matching: a bare episode
numbering convention with no show name in the filename at all
(`001 Episode's Own Title.ext`, no SxxExx, no show prefix — entirely
ordinary on its own) makes guessit invent a "title" from whatever text
follows the number. That text is the individual episode's own name, and
differs for every episode in the folder — trusting it, as the code did,
groups nothing together at all: every episode became its own single-
episode "show", searched against TMDB by that one-off title alone.
Whenever a season-like ancestor folder exists (a numbered season, or
Specials/Bonus/Extras -> season 0), its own root folder now names the show
unconditionally — never a per-file guessit title, which cannot tell a
show's real name from an individual episode's one-off name when the
filename carries no reliable ShowName/SxxExx structure. The walk continues
past *every* consecutive season-like ancestor, not just the first: a
per-season Bonus folder (Show/Season N/Bonus/file.ext) is nested two
levels inside the show, both "Bonus" and "Season N" season-like on their
own, and stopping at the first would hand back "Season N" as the show's
name instead of "Show".
Also adds "livre" ("book") to the season-word vocabulary (§3.4) — some
shows number their seasons that way (Roman numerals) rather than
"season"/"saison".
Supersedes _title_from_show_siblings from the previous commit (removed):
that fallback assumed a per-file title could still be trusted often
enough to be worth borrowing from a sibling season folder — this fix
means it never needs trusting in the first place once a season-like
ancestor exists.
Verified directly against the real library this was found on, not just
synthetic tests: every sample file (a numbered episode, a Book/Bonus
feature, a Book-root "making of") now resolves to the show's real name.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The real reason a node restart alone didn't fix already-indexed entries
after the previous commit's enrichment change: _do_media_meta_request
checked media_cache's file->tmdb mapping (keyed by content hash) and
trusted it unconditionally, before ever comparing it against the file's
*current* movie/show classification. A file whose season/episode changed
on a later scan — exactly what the Specials-folder fix does, for every
file it reclassifies from "movie" to "tv" — kept answering with its
stale, wrong-kind-of-match forever, since nothing about a reclassification
touches this cache or its key.
Now falls through to a fresh search whenever the cached media_type
disagrees with what the entry resolves to right now, rather than trusting
a mapping that predates the file's current classification.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
unrelated standalone movies
Found live: a "Specials" folder full of one-off-named bonus episodes had
every file appear as its own poster, matched against TMDB by its own
title, because guessit finds no season/episode grammar at all in a
filename with no SxxExx of its own — so the classification (episode vs
standalone movie), based solely on that, fell to the movie branch. Real,
unrelated films happened to share several of those one-off titles and
matched confidently, one per Special, cluttering the Videos view with
dozens of wrong posters instead of grouping under the show.
An ancestor folder saying this is part of a show — a numbered season, or
Specials/Bonus/Extras -> season 0 — is now trusted over the filename
having no SxxExx of its own. The show's name can't come from this file's
own guessit title (that's the bug) or from siblings in the same Specials
folder (every one of them has the same gap) — it's borrowed from the
show's ordinary season folders next door, which do carry it in the usual
ShowName.SxxExx shape (_title_from_show_siblings). A synthetic, stable
episode number (alphabetical rank among the folder's video files) stands
in for the real one nothing in a Specials folder provides.
Generic, not specific to a folder literally named "Specials": the same
fallback fires for any season-like ancestor folder whose files lack
per-file episode grammar, numbered seasons included (covered by a
dedicated test).
Also hardens _title_from_siblings to require the sibling's own episode
number too, not just a title — otherwise it would borrow one Special's
one-off title as if it were representative, on a folder like this one.
Existing already-indexed entries will need a rescan (restart the node) to
be re-enriched under this logic — nothing re-derives them on its own.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
IndexEntry.path is the *folder* a file is in (indexer.py's
_virtual_dir docstring: "the directory a file appears in"), not the
file itself. GroupIndex.get_entry_by_path() treated it as if it named
one file, and every one of its four callers did too:
_do_music_meta_request, _do_media_meta_request, _do_tmdb_override, and
_admin_exec_tmdb_override. Any two files sharing a folder — an album is
one folder with many tracks, a season is one folder with many episodes
— collided: a lookup by path silently returned whichever entry the
index happened to iterate to first, regardless of which file the
client actually asked about.
Found live (2026-08-25): three unrelated albums ("High Tone - Various",
two "Le Peuple de l'Herbe" albums) all showed the same MusicBrainz
cover, because all their representative tracks happened to sit in one
"high_tone" folder alongside a track that legitimately matched that
cover. A force-reload didn't help — the bug is server-side, not a
stale client state.
Fixed by keying these four request/response pairs by `file_id` (the
entry's own content hash — already unique, already how every other
lookup in the system identifies a file) instead of `path`, both in the
wire messages (music_meta_req/resp, media_meta_req/resp, tmdb_override)
and in music-app.js/video-app.js's own hooks. GroupIndex.get_entry_by_path
is now unused and removed — GroupIndex.get_entry(file_id) already did
the right thing.
No test previously exercised either handler with two entries sharing a
folder — the only existing coverage (test_tmdb_override_policy.py) gave
each entry its own folder, so the bug never had a chance to show up.
Added that scenario there and in two new test files, all confirmed
failing against the pre-fix code before being confirmed green against
the fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An operator routinely shares the same physical folder into more than one
group (a music library, a Séries drive) — IndexCache used to be opened
once per group (data_dir/{group_id}/index_cache.db), so the second group
to reference an already-fully-hashed multi-terabyte folder paid the same
full content read the first one did. IndexCache itself carried no
group_id in its schema; only daemon.py's wiring did. Now one instance,
opened once at startup (data_dir/index_cache.db), shared by every group's
DirectoryIndexer.
Confirmed against a real deployment (2026-08-25/26): a group sharing an
already-indexed folder with an existing group indexes it instantly, with
zero rehashing.
Also fixes a related cross-group correctness gap found during this work:
media_cache.db (thumbnails, TMDB/MusicBrainz metadata — already node-wide,
untouched by this change) was pruned for a file the moment it left *one*
group's index, even if another group's index still held the same content
hash — forcing a redundant re-fetch/re-probe/re-thumbnail for a group that
never actually lost anything. Prune now runs only once no group's index
references the file_id any more.
Adds a node admin UI action ("Maintenance" card, prune-index-cache) to
drop cache rows that no longer belong to any group's roots — skips
anything under a root that is merely temporarily unavailable (indexer.py's
"a root that goes away freezes, never empties" rule extends to this
cache too, or a reconnected drive would pay a full rehash for no reason).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
scanning progress
Two real-world bugs found together while testing multi-root group
creation:
- CreateGroupWizard only sent the enabled-apps PUT when the operator had
*unchecked* something, assuming "every box left checked" already matched
the node's own default (Roster.DEFAULT_APPS = chat, files). It doesn't —
so leaving every app checked, the common case, silently left
Videos/Music/Photos disabled on the node. Now sent unconditionally.
- The wizard's "add extra roots" step never polled index-status, so once
step 3 (which only watches the first/upload root) finished, the
progress bar froze while the node kept scanning the remaining roots for
minutes, unwatched. Added waitForRootsIndexed (platform.js), mirroring
waitForGroupHosted's own race handling.
That fix exposed a deeper one: indexer.py's _scan_root() only flipped
`progress.scanning` on *after* walking the directory and stat()-ing every
file — both off-loop, but slow enough on a large root that a poller's
grace period (waitForRootsIndexed's 5s) could expire before ever
observing `scanning: true` (confirmed against production logs: a GEK-init
step fired 5.058s after a root started scanning, matching the grace
period almost exactly). The stat() pass was also a synchronous loop
directly on the asyncio event loop — blocking the whole daemon (WebRTC,
chat, admin UI) for as long as it took on a root with many files. Both
fixed: `scanning` now flips on before the walk starts, and stat()-ing is
now off-loop too (_size_files).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
nothing
search_release() only ever tried a field-scoped exact-phrase Lucene query
(artist:"..." AND release:"..."). Verified live against musicbrainz.org:
any deviation from MusicBrainz's own spelling (a year suffix, an edition
tag, an artist credited under an older/aliased name) drops it to zero
results outright rather than a low-scored one, so the confidence check
never even ran — this is why well-recognized artists were still getting
almost no cover art. Add an unscoped loose-query fallback, escape Lucene
special characters in the interpolated tag text, and make the confidence
score consider the artist match too (not just the album title) now that
the fallback has no field scoping to rely on.
Also document MESHBAY_MUSICBRAINZ_CONTACT_DEFAULT in the systemd units
and man page, mirroring MESHBAY_TMDB_DEFAULT_TOKEN's precedent — never a
literal value in source, configured via node.env.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XSohfUQQiaE77qyFLgSv3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Only thumbnail bytes were ever durable in media_cache.db — every other
derived field (photo width/height/EXIF, video ffprobe duration/dims,
audio cover art) lived solely on the in-memory GroupIndex entry, so a
node restart re-decoded every photo through Pillow, re-ran ffprobe on
every video, and re-scanned for every album cover from scratch, even
though the answers already sat in the cache.
Adds photo_meta and video_meta tables (content-only fields, keyed by
file_id) and checks them before doing the expensive work. Audio gets no
new table: mutagen reads tags and duration in one inseparable call, so
caching duration alone buys nothing — instead cover-art extraction alone
is skipped via a new skip_cover flag when a cached cover already exists.
Deliberately excluded from all three caches: anything derived from the
filename or folder path (video display_title/season/episode via guessit,
audio artist/album folder-fallback) — those must keep being recomputed
fresh so a rename/move is still correctly re-derived by the existing
_reenrich_renamed_*_entries mechanisms, instead of silently handing back
a stale parse under the new name/location.
Regression tests prove cache reuse by deleting the source file (or cover)
between two enrichment runs, and prove rename/move correctness survives
the new cache by renaming/moving to a path that never exists on disk.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new group application (docs/apps.md's plug-in mechanism), following the
plan in docs/photos.md. Unlike Videos/Music: several photo roots per group
instead of one (photo_roots is a set, one signed op replaces it whole),
a single album-grid view with no third-party matching step, and per-photo
info read from the file's own EXIF at index time — no metadata service,
no credential, no outbound network call at all.
Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera`
on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`.
Node: roster.py stores photo_roots as a group_settings entry (JSON list,
same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the
whole set in one op, same pattern as apps_enabled; a new PhotoEnricher
(indexer/enrich_photo.py) runs Pillow in its own small bounded pool,
separate from the video/audio pools, producing a resized thumbnail plus
the two EXIF fields — never GPS, checked by a grep-based regression test.
Client: photos-app.js — one album card per directory containing images,
a per-album photo grid, and a lightbox with next/previous (keyboard and
buttons), zoom in/out/fit/100% starting from the actual on-screen fit
percentage, and a "zip this album" button reusing files-app.js's own zip
mechanism (lifted into file-utils.js's downloadDirectory so both call the
same implementation). group-settings.js gets an add/remove multi-root
picker, distinct from Videos/Music's single-value one.
Bugs found and fixed before this ever shipped, worth keeping the story of:
- enrich_photo.py read width/height from the raw image *before* applying
EXIF orientation correction, and read DateTimeOriginal off the plain
0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which
Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict
round-trips through Pillow either way, which is exactly what would have
hidden both bugs; the regression test builds EXIF with piexif instead,
matching what real hardware produces.
- photos-app.js's album grouping stripped a trailing path segment from
entry.path under the assumption it still carried a filename — it
doesn't (files-app.js's own convention: e.path is already the
containing directory), so every album collapsed one level into its
parent. Found live against a real multi-folder library.
- transport.js's ADMIN_OP_TYPES allowlist (already the fix for an
identical bug on video_root/apps_enabled, see 4783d81) was missing
photo_roots: its admin_challenge matched no pending request and was
silently dropped, so saving a photo root just timed out after 30s with
no error.
- daemon.py pruned a thumbnail when its file left the index (root removed
or reconfigured) but never forgot the content hash was "already
attempted" — the same bytes reappearing under a renamed/relocated root
(an operator's real workflow) were then permanently skipped, forever,
with nothing to indicate why. Discarding the attempt alongside the
cache entry on prune is what makes pruning actually reversible.
- packages/meshbay-client's app:// protocol handler served every file
with no Cache-Control header, so Chromium was free to serve a stale
cached copy indefinitely — none of several `npm run sync-ui` + reload
cycles during development actually picked up the new code until the
renderer's disk cache was cleared by hand. Now sends Cache-Control:
no-store.
- the lightbox's zoomed image used flex centering (align-items/
justify-content: center) combined with overflow: auto — a well-known
trap where the browser centers overflowing content by shifting it, and
the leading half of that overflow (here, the top of a zoomed photo)
sits outside what the scrollport can actually reach. Reported live as
"unusable". Fixed by switching to top/left alignment once zoomed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Major finding: entry.id is a content hash, so the exact same physical
file — the same MP3, byte-for-byte — indexed into two different groups
(a shared library reused across several demo/test groups, or genuinely
the same folder shared into two groups) produces the *same id* in both.
_enriched_attempted was a single flat set of bare ids shared across every
group this node hosts. The moment one group's copy got enriched, every
other group's otherwise-identical copy read as "already attempted" and
was skipped forever — nothing else ever revisits an id once it's in this
set. That group's Music tab (or Videos tab, same bug, same set) showed
every affected file at duration 0 with no artist/album/thumbnail,
permanently, no matter how long you waited or how many times you
reloaded — group A having been enriched first was enough to silently
starve every later group of the same content.
Now keyed by (group_id, entry.id) throughout — the enrichment gate, the
sweep, and the rename re-enrichment path, for both video and audio (they
already shared the one set, and the collision risk is identical for
both). New regression test constructs two groups with byte-identical
audio content and confirms both enrich independently.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Every existing audio_root test either called ops.set_audio_root directly
or mocked out _issue_admin_challenge — none of them exercised real
signature verification, _do_admin_response, or the shared groups_ctx/
roster wiring _run_op depends on. Worth ruling out a break somewhere in
that real path specifically: a report described a save that looked like
it worked (the Music tab showed content right after) not surviving a
reload.
Drives the real _do_audio_root -> admin_challenge -> sign -> _do_admin_response
-> _admin_exec_audio_root path with a genuine Ed25519 operator key, then
opens a *separate* Roster instance against the same db file — the direct
question a "worked, then reverted" report raises: does the value actually
land durably, in a form any later connection reads back correctly. It
does; this passes. The one thing missing from the session fixture to get
this far was peer-registry self-registration (a real session adds itself
on handshake completion — without it, the final ack has nowhere to go,
including back to the requester).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
musicbay.md's original call — Music needs no root, tag reads are cheap so
just cover the whole shared tree — didn't hold up against a real messy
library: everything under every shared folder got mixed together with no
way to scope Music down to an actual music collection. This adds an
audio_root setting, symmetric to video_root in every respect: signed
operator op (audio_root/audio_root_ack, MNP bumped to 0.10), validated
against a real directory in the group's own roots before a signature is
even asked for, gates tag/cover enrichment exactly like video_root gates
ffprobe/TMDB (nothing runs until it's set, only files under it once it
is), and a set/change fires a one-off sweep of whatever the folder already
contains.
The old trigger — sweep everything the instant "music" joins enabled_apps
— is gone along with the root-less design it belonged to; setting
audio_root is now the trigger, mirroring set_video_root's
enrich_video_root_fn exactly.
Test coverage mirrors the video_root suite: policy (refuse before a
signature round trip, accept/store correctly) and the enrichment gate
itself (nothing without a root, only files under it, sweep on set).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Tagging and covers for these two formats landed already, but neither one
decodes in any mainstream browser's <audio> element at all — a real
library scan turned up 273 such files that would show up correctly in the
Music app and then simply fail on click. This closes that gap: the node
transcodes to AAC/M4A on request (a one-shot whole-file conversion, not
live-piped like video's fMP4 segments — an audio file is small enough that
streaming it buys nothing), caches the result under its own content hash
the same way a TMDB poster or a MusicBrainz cover is cached, and serves it
back through the ordinary file_req/chunk path. That path used to assume
anything in the media cache was thumbnail-sized (single chunk, always);
generalized it to slice a cached blob the same way a real file on disk
gets sliced, since a transcoded track can be several MB.
New MNP pair (`audio_transcode_req`/`_resp`, version bump to 0.9), shares
its concurrency cap with video's transcode pool rather than getting its
own — both are real ffmpeg processes on the same node. Every other audio
format is untouched: this only fires for .wma/.mpc, the two extensions
that need it.
|
| |
|
|
|
|
|
|
|
|
| |
The "P.H. Theme" failure investigated earlier turned out to be a genuinely
corrupt 1256-byte source file with no audio stream at all, just an ID3
tag — a real, if rare, corruption pattern worth guarding against directly
rather than only handling gracefully at playback time. Scoped to audio
only, applied wherever a file actually gets hashed/typed (fresh scan and
the cache-miss rehash path alike) — a tiny file of any other type is still
indexed normally.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A real-library scan turned up 250 .wma and 23 .mpc files that the indexer
was silently classifying as "other" — genuinely lost from the Music app,
not a consolidation-rule artifact (checked separately: the grouping logic
itself drops nothing). Both are now indexed as audio and tagged properly:
- WMA has no mutagen "easy" wrapper, so the generic tag reader was reading
nothing from it at all. Reads the real ASF keys directly instead
(Title/Author/WM-AlbumTitle/WM-TrackNumber), confirmed against a real
sample file before writing the mapping.
- Musepack's format auto-detection is unreliable enough (misidentified a
real .mpc as MP3 in spot checks) that it now always opens by its own
class instead of guessing from content.
- Filters out another placeholder value found along the way: a French
ripping tool's auto-generated "Album inconnu (<timestamp>)".
Neither format decodes natively in a browser's <audio> element, so this
gets them correctly visible, tagged, and covered — not yet playable
in-browser. That would need server-side transcoding, deliberately left
out of this change.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Album-grid readability, part two:
- groupMusicEntries (music-app.js): an album bucket left with exactly
one track - a real album tag, but only one song from it, not the
whole release - clutters the grid the same way an untagged loose
track does. Both kinds now fold into one "<artist> - Various" tile
per artist, unless there is only one leftover track overall, where
relabeling buys nothing and the track keeps its own name (or the
generic placeholder, if it never had one).
- foldKey also normalizes "&" vs "and" ("Artist & The Band" / "Artist
and The Band" is one act, tagged both ways across different rips of
the same catalogue) alongside the existing case/whitespace fold.
- music-player.js: a close button pauses and tears the player down;
an unmount cleanup effect (pause, revoke every cached blob URL)
fires either way, whether that's the close button or the shell
tearing the bar down on its own. A "current queue" button opens an
overlay listing the whole playing queue with the current track
highlighted, click any to jump to it - works identically regardless
of how the queue was built (an album, the consolidated misc bucket,
a single standalone track), since it only ever reads the player's
own live tracks/order/pos.
- group-page.js: this component is not remounted when switching to a
*different* group on the same /group/:id route (only the groupId
prop changes) - so without an explicit reset, music from one group
would carry into the next one opened. Resets musicQueue to null on
groupId change; a tab switch inside one group still leaves it alone.
- Scrubbed real artist/band names that had leaked into code comments
and test fixtures (enrich_audio.py's docstrings, several
test_enrich_audio.py assertions, a music-app.js comment) - replaced
with generic placeholders, no behavioural change.
- i18n: music.various, music.player_close, music.player_queue,
music.queue_title added across all ten locales.
Client-side only except none of this touches the node at all. npm run
sync-ui re-run. Full suite: 1129 passed, no regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Music grouping was measured against a real ~5700-file library and came
back worse than a plain file listing. Root cause: the artist/album
ancestor walk always climbed exactly two levels (parent = album,
grandparent = artist) with no idea where the group's own shared root
was. Any file in a flat top-level folder — common here: bare
`Artist/track.mp3`, no album subfolder at all — had its "grandparent"
resolve to the root directory's own name, so the artist got replaced
by the share's name. Measured: 289 of 5664 tracks across 41 real,
unrelated artists (Ben Harper, Dire Straits, Jimi Hendrix, Janis
Joplin, ...) collapsed into one fake artist this way — the single
biggest bucket in the whole library, ahead of every real one.
- `_artist_album_from_ancestors` now takes the entry's own root
boundary (daemon.py resolves it via `RootSet.split`) and refuses to
read it as a name. A file sitting in a top-level folder — genuinely
ambiguous, artist or a standalone album/compilation — is handled by
`_split_top_level_folder`: split on "Artist - Album" when the
(cleaned) folder name has that shape, otherwise the whole name
becomes the artist alone, the more common real case here.
- `_clean_tag` treats known tagger placeholders ("No Artist", a French
tool's "Nouvel artiste (334)") as absent rather than a real value —
they were just as truthy as a real name and were locking out the
fallback that would have done better. "Various Artists" is kept, a
real compilation credit rather than a placeholder.
- A `title` tag that's the bare filename copied verbatim (track number
included — found live on a whole CD-single) is stripped through the
same prefix rule the filename parser already used
(`title_parse.strip_track_prefix`), since a tag normally wins over
the parsed title.
- Cover art: only 11% of a 400-file sample had embedded art (expected
for this era of rip), but 267 loose cover images sit beside the
tracks across the library (Windows Media Player's `Folder.jpg`/
`AlbumArt_{guid}_*.jpg`, manual `cover.jpg`) and were never looked
at. `_find_sibling_cover` checks the track's own folder before
giving up — measured coverage 11% -> 26% on the same library, zero
network calls.
`_enriched_attempted` is in-memory and resets on restart, so a node
restart is enough to re-run enrichment over an already-scanned library
with the fixed logic — no rescan flag, no cache to clear by hand.
22 tests in test_enrich_audio.py (11 new), including the exact
regression case end to end through the real pool. Full suite: 1129
passed, no regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements the node half of docs/musicbay.md against MNP 0.8:
- IndexEntry gains artist/album/track_no (reuses duration/thumb_hash/
display_title, already generic). New musicbrainz_config/_enabled and
music_meta_req/_resp message pairs, mirroring the TMDB shape.
- title_parse.parse_track_filename: track-number-prefix + title parsing,
fallback-only (embedded tags are the primary source, unlike Videos).
- indexer.enrich_audio.AudioEnricher: mutagen-based tag/embedded-cover
extraction through its own bounded pool (asyncio.to_thread, no
subprocess — no ffmpeg-shaped deadlock risk). Gated on "music" in a
group's enabled_apps rather than a video_root-style scoped folder.
- musicbrainz.py: MusicBrainzClient — no API key (unlike TMDB), just a
self-imposed ~1 req/s pace and a configurable, non-default User-Agent
contact string; inert (no calls at all) when no contact is configured,
never sends an unidentified client.
- media_cache.py: file_mbid/mbid_meta tables alongside the existing TMDB
ones, cover art reusing the thumbs table via a synthetic
musicbrainz:{mbid} id, pruned on file deletion.
- roster.py/ops.py/webrtc_server.py: musicbrainz_contact (node-wide) and
musicbrainz_enabled (per-group, from the start) as signed operator
settings, ALLOWED_APPS gains "music", _do_music_meta_request resolves
and caches a release-level MusicBrainz match per (artist, album).
- daemon.py: AudioEnricher/MusicBrainzClient wired alongside the video
ones; a group's existing library is swept when "music" is newly
enabled (no video_root equivalent — see musicbay.md §2.1).
41 new tests (musicbrainz.py against a mocked transport, admin-op policy
for both new settings, media_cache round-trip/pruning, enrich_audio
end-to-end against real ffmpeg-generated MP3s). Full suite (common +
node + hub): 1116 passed, no regressions.
Client-side (music-app.js, persistent player bar) not started yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three bugs found live testing the Videos app against a real HEVC/EAC3 show,
plus a design change requested afterward:
- Streaming always did "-c:v copy", which faithfully reports a source's real
hev1 codec string but is unplayable in a browser with no HEVC decoder
(most Chrome/Linux builds). The node now transcodes to H264 whenever the
probed codec is browser-incompatible (media_probe.py's new
BROWSER_INCOMPATIBLE_VIDEO_CODECS), with a `transcode_incompatible_video`
node.toml opt-out for operators who know their viewers already decode it.
- Dropping a whole season into an already-watched folder gave no scanning
indicator and no progress bar: IndexProgress was only ever updated by the
two bulk scan paths, never by the real-time per-file watchdog path
(_schedule_update/_debounce/_update_entry). That path now accounts a
"burst" the same way, without double-counting a file rewritten mid-debounce.
- A stray literal "0" rendered in the video detail modal when there was no
TMDB match (`meta.confidence` is 0, and `0 && x` renders "0" in JSX/htm,
not nothing) — `confident` is now a real boolean.
- Whether TMDB is used at all moves from a node-wide setting to per-group
(OP_TMDB_ENABLED/tmdb_enabled/tmdb_enabled_ack, scoped like OP_VIDEO_ROOT):
an operator running a real media-library group alongside test/demo groups
on one node wants outbound TMDB traffic for the one that needs it, not all
of them. The custom API token and query language stay node-wide, one
shared credential/cache (tmdb_config/OP_TMDB_CONFIG, unchanged reasoning).
MNP_VERSION 0.6 -> 0.7, additive.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
wizard polish
Two operator-facing fixes for a real 3-season show whose automatic TMDB
match was wrong at the show level: per-season overview/air_date tabs in the
detail modal (falling back to the show-level text when a season's own is
empty), and a "Fix match…" search-and-correct affordance that re-resolves
every file sharing the corrected show's display_title. New signed op
OP_TMDB_OVERRIDE and two read-only pairs (season_meta_req/resp,
tmdb_search_req/resp), MNP_VERSION 0.5 -> 0.6.
Also: the create-group wizard gets a spinning indexing indicator and an
app-selection step, group settings default the TMDB language to the
operator's own locale (never as a global default), and a file renamed
mid-session now re-triggers title parsing instead of being silently
skipped by the enrichment dedup guard.
Fixes two bugs found during this work: the search overlay's z-index lost
to the base video-overlay class and rendered invisibly, and season_meta's
own empty overview didn't fall back to the show-level one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements docs/mediacenter.md: a "Videos" group application built on the
existing files index rather than a separate catalogue. On the node side,
new indexer enrichment (technical probe, filename/season parsing, thumbnail
generation) runs per-file once an operator has chosen a video_root for the
group, plus a TMDB client for on-demand poster/metadata lookups (never
client-side, thumbnails delivered over the existing chunk path). On the hub
side, a new video-app.js renders a lazily-mounted poster grid or a
thumbnail-only flat list, with TMDB entirely optional per group.
Along the way: the global apps registry now drives Settings' default-tab
picker instead of a hardcoded list, and the video_root is configured from
group Settings (like uploads) rather than from Files, with the node
refusing to run any TMDB/thumbnail work until one is set.
Fixes several bugs found via live testing against a real library, notably
a race between two effects writing the same "image ready" state that could
leave a poster grid spinning forever on a same-tab revisit — see
mediacenter.md §5.4 for the full account of each one.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
MSE only decodes AAC/Opus, so copying a source's real audio codec left
non-AAC files silently unplayable in-browser (E-AC-3 additionally made
ffmpeg itself refuse to write the fragmented MP4 header). Audio is now
always transcoded to AAC and downmixed to stereo — multichannel AAC is
accepted by ffprobe/VLC but silently rejected by some browsers' MSE
decoder once real fragments are appended, which forces the SourceBuffer
out of its MediaSource with no explicit error. Video stays copy-only.
Also: report a clear client-side error instead of a bare STREAM_END when
ffmpeg exits nonzero before producing any output, add video-element/
MediaSource error logging on the client for the next time this class of
bug needs diagnosing, and fix a hub test that had grown too broad a scan
window after an earlier, unrelated transport.js change.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reconcile, and delta sync
Indexer performance work, in four parts:
- Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node
restart no longer re-hashes every file — measured at 23 minutes for a
114 GB library on a slow disk before this, near-instant after. Hashing
is deliberately kept sequential (max_workers=1): it was never actually
concurrent despite the pool size, and two interleaved reads seek-thrash
a spinning disk instead of going faster.
- Byte-based scan progress (IndexProgress), surfaced via the loopback
index-status route, the handshake ack, and a periodic INDEX_PROGRESS
push to connected peers — drives a progress bar in the Create Group
wizard and "add a directory" in Settings, and an animated presence dot.
Guaranteed to settle back to idle via try/finally and a final push on
the scanning->false transition.
- The reconcile backstop's directory walks now run in the executor
instead of blocking the daemon's event loop; its interval defaults to
10 min (was 60s) with adaptive backoff to 2h when nothing changes,
reset on a real change or a peer connecting, and is now a per-group
operator setting (signed op + group Settings UI).
- INDEX_DELTA wired up (protocol support existed, nothing called it):
_on_index_change now sends additions/deletions instead of rebuilding
the full entries list, coalesced over a short window so a burst of
file events produces one push, and the hub swarm registration for
public groups only (re-)registers newly added hashes.
Also fixes several bugs found while testing the above against real
libraries (a 114 GB and a 100+ GB group on a USB HDD):
- /api/reload blocked until the reload — including a brand-new group's
full initial scan — finished, which the Electron bridge's fixed 30s
call timeout turned into a hard failure on any real library. The route
now fires the reload without waiting (ops.start_reload), matching
add_root/remove_root's existing pattern; the wizard's own step order
was fixed to wait for the group to actually appear hosted before the
steps that need it (extra roots, GEK), with retries for the residual
race between that and the daemon's own bookkeeping.
- transport.js's hand-rolled msgpack codec had no case for uint64/int64
(0xcf/0xd3) and crashed decoding any message containing one — hit by
IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB.
Verified against real msgpack-encoded bytes from the Python side.
- chat_hist_resp, and this change's own index_progress and
set_scan_settings_ack pushes, were not routed by message type and
could be handed to an unrelated pending request by the transport's
"oldest pending" fallback, stalling it until its own 30s timeout and
corrupting whatever received the wrong reply in its place.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GroupPage's 6620-line app.js carried Chat and Files wedged in directly, with
no way to add another group-level app without touching the shell itself. It
is now app.js (routing, non-group pages) plus nine focused files — apps.js
(the registry), chat-app.js, files-app.js, video-player.js, group-page.js
(the shell), group-settings.js, hub-client.js, icon.js and file-utils.js —
with docs/apps.md as the checklist for adding one (Videos/Music/Photos are
sketched there, not built).
Node side gained the matching enablement mechanism, mirroring
member_upload exactly: a roster setting, a signed apps_enabled op enforced
by _has_admin_authority, exposed in the handshake ack. Operators toggle
applications per group from Settings, which also gained a small reorder:
Invite, Pairing, Applications, Shared directories, Uploads, danger zone,
Your devices, Members.
Two bugs surfaced during the split, both missing an import across the new
file boundary and invisible to node --check or a module-load probe since
they only throw when the code path actually runs:
- group-page.js called onRefreshAuth on a stale-token handshake rejection,
but app.js never imported refreshAccessToken from hub-client.js — so a
brand new member (including a group's own creator) hit "Not a member of
this group" and the retry silently failed, throwing before it could
refresh the token.
- chat-app.js called getLocale() for message timestamps without importing
it from i18n.js. Opening Chat on a group with real messages threw mid-
render; uncaught, that appears to wedge Preact's render scheduler, so
every button on the page stopped responding until reload.
Caught the second class of bug with a proper no-undef audit across all
split files (a temporarily installed ESLint 9, since the system one is too
old to parse this codebase's syntax) rather than trusting grep. 827 tests
pass; 6 new ones cover the apps_enabled policy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a status panel at the top of the Node page — always visible, even
before an MNP connection exists — showing the meshbay-node systemd
unit's own state (via `systemctl --user show`, main process only) with
Start/Stop/Restart controls. This is the piece the rest of the page
cannot provide: it has to work while the daemon is stopped or crash-
looping, which the MNP-based sections require the daemon to already
answer.
While touching node lifecycle: `reload` and `restart-daemon` in the
CLI shelled out to pgrep + SIGTERM/SIGHUP and respawned the process by
hand, logging to a hardcoded /tmp path. That pattern already SIGHUPed
a developer's own running node by accident once (see the old
test_cli_dispatch.py comment). Both now delegate to
`systemctl --user reload|restart meshbay-node`, which the unit already
supports correctly (ExecReload=, Restart=on-failure).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Node daemon no longer blocks startup on slow directory scans — initial
indexing runs in the background so the node reaches "running" immediately
after transports are up. Fixes the wizard failing to detect the node when
large USB/NAS roots take minutes to scan.
Also: wizard key-linking deadlock resolved (main.js links during poll),
invite form stays in DOM during reconnects (disabled instead of destroyed),
pairing code bridges to renderer, and firewall docs for LAN casting added.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- test_transport_contracts: CreateGroupPage was refactored into a
routing wrapper; assertions now read CreateGroupFormSimple
- test_task_lifetime: _spawn now uses an _on_done wrapper instead of
a bare self._tasks.discard callback; assertion checks both parts
- test_video_buffer_ceiling: target the real updateend handler, not
the settled() utility; add awaitingInitRef to the MSE harness scope
- test_video_seek: silence debug console.log in window_leak harness
so it does not pollute the JSON output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
- `init` now writes config AND creates the keystore (interactive password
or unlock.key/env var). Idempotent: skips either step if already done.
- `status` uses load_keystore instead of load_or_create_keystore — a
read-only command should never silently create identity keys.
- Stub getpass in test_cli_dispatch to prevent test hangs when no
keystore exists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Create Group wizard (Electron-only) consolidates 6 steps across 4 interfaces
into a single multi-step page: group creation on hub, node attachment, root
selection via folder picker, GEK initialization, and auto-pairing — all in one
flow. Browser SPA keeps its current behavior unchanged.
Public group support (Option A — GEK for all groups):
- All groups have GEK regardless of visibility; open-join groups auto-admit
via TOFU when join_policy is "open"
- Key rotation blocked for public groups (API guard + UI hidden)
- Hub signaling allows WebRTC offers for nodes hosting open-join groups even
when the caller isn't a member yet
- attach_group writes join_policy to node.toml
- Daemon loads GEK for all groups, not just private ones
- Known-device path in join_request now auto-admits to open-join groups
Node loopback API bridge (Electron IPC):
- node:detect, node:call, node:pairing-code IPC handlers in main process
- Renderer never sees tokens, paths, or keys (session token = physical access)
- platform.js node namespace for UI consumption
- Loopback endpoints: roots CRUD, member-upload toggle, reload
Bug fixes:
- Root change detection: removed premature ctx["roots"] updates from add_root
and remove_root that prevented indexer retarget on reload
- Duplicate offline message: global fallback now gated on !group
- Signaling membership check: fallback to open-join groups for non-members
Sidebar groups sorted by last_activity_at (most recent first):
- New Group.last_activity_at column with Alembic migration
- POST /v1/groups/{id}/activity endpoint, called on connect and chat send
- Client-side sort + throttled hub updates (1/min)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Node admin panel (NodePage) now covers every CLI operation over MNP:
group attach/detach, roster, member unpin, GEK rotate, denylist, reload.
Daemon hot-loads new groups and tears down removed ones on config reload
instead of requiring a full restart. Group attach/detach via MNP or
local API triggers an automatic reload so the group is live immediately.
Fixed GroupPage hang on first visit to a newly created group: the JWT
issued at login didn't include the new group, the node rejected with
not_a_member, and the token-refresh path returned without re-triggering
the connect effect (Boolean(token) didn't change). Now bumps retryKey
after a successful refresh so the effect re-runs with the fresh token.
NodePage marks groups hosted by the node but absent from the hub with a
"not on hub" badge so stale groups are visible and easy to remove.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A group where every member may add files stays the default. Some groups want a
library the operator curates, and until now the only way to get one was to
designate no upload root at all — which refuses the operator too.
**The node enforces it; the interface merely stops offering it.** The Upload
button in the Files toolbar and the paperclip in the chat composer both
disappear, which is a courtesy to the people who are not trying. The control is
`_do_file_upload` refusing with `member_upload_off`, so a member on an old tab,
or one speaking MNP directly, gets the same answer. There is a test for each,
and the enforcement test is in the node package rather than beside the UI one so
nobody reads the hidden button as the mechanism.
**Changing it is a signed operator instruction** — `OP_MEMBER_UPLOAD`, on the
same path as removing a member. An unsigned one would let any member turn it
back on and make the setting a suggestion. The transcript's subject is `on` or
`off`: what the operator is shown before signing has to name the outcome, not
the operation.
**It lives on the node**, in a new `group_settings` table in `roster.db`. Not
the hub, which has no business deciding who may write to someone else's disk.
Not `node.toml` either: that file is hand-written and full of comments recording
decisions, `ops.py` appends to it rather than round-tripping it through a
writer, and a setting toggled from a panel must not rewrite the operator's file
or need a restart. The value is cached in the group context because the upload
path is synchronous, and the signed operation updates both — storing it without
applying it would make the panel say one thing while the node did another.
**Absent means allowed**, at every layer: no row in the table, no key in the
context, no field in `handshake_ack`. An older node and an older client both
behave exactly as before, and upgrading never silently closes a group. Each of
those three has its own test, because they fail independently.
The operator is always exempt — otherwise turning it off locks them out of their
own node with a config file and a restart as the only way back. `is_node_admin`
was being computed in two places by then and is now one function, since two
copies of "is this the operator" is how the ack and the gate come to disagree.
A change reaches everyone already connected via `member_upload_ack`, so the
button goes without a reconnection. That message is both a broadcast and the
reply to the request that caused it, which is why the client does not return
early on it.
Docs updated for a cold start: draft-v6 §2.1b and change 9, a new "Where Phase
13 stands" section in CLAUDE.md recording what is built, deployed and still
missing, the module map row, and desktop-client-v1 §10b on the Settings tab and
where group settings live.
883 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage D, and the honest half of it.
D1 — the seam (done, and verified)
----------------------------------
`static/platform.js`. `HUB` becomes `platform.hubBase()` and the transport is
built with the same base, so one address has one source. In a browser it returns
'' and every path stays relative to the origin that served the page — the
acceptance criterion for this split was "the browser SPA behaves identically",
and it does. `platform.js` joins `_ASSETS`, or a change to it would not move the
content hash and a cached browser would never ask for it.
D2 — the shell (written, never launched)
-----------------------------------------
**There is no npm on this machine. Electron was never installed and
`packages/meshbay-client/` has not been run once.** That is stated here rather
than discovered later.
What is there: a main process serving the packaged interface over a privileged
`app://` scheme (`secure` and `standard` are not cosmetic — without them the
service worker refuses to register and streamed downloads break silently), a
preload exposing an enumerated bridge that never passes a filesystem path, a
window with `sandbox`, `contextIsolation` and no node integration, navigation
away from the package refused, and a CSP where the hub is reachable over
connect-src and is not a script source. The hub address arrives as a process
argument because `platform.hubBase()` runs before anything can await.
`test_desktop_shell.py` pins each of those by reading the source — the treatment
`test_downloads.py` already gives the three browser save paths. It catches a
property being removed and proves nothing about the application running. Two
were checked by breaking them.
The interface is *copied* into the package by `build/sync-ui.js` from the hub's
static directory, and `ui/` is gitignored: a silent fork is the only real way to
end up maintaining the interface twice.
D3 — partial
------------
The bridge, and the part worth having now: safeStorage's backend is reported
rather than assumed. On Linux it falls back to a fixed key when no keyring is
running, silently — someone who believes the OS is holding their keys is told
when it is not. The native key lifecycle belongs with D4 and needs a running
application to mean anything.
D8 — partial, and a real defect found
--------------------------------------
`meshbay-node.spec` installed the SYSTEM template — the one carrying `User=%i` —
into `%{_userunitdir}`. A user unit already runs as its owner and cannot carry
`User=`; systemd refuses the file, so the packaged unit could never have
started. Nothing noticed because nobody had built and installed the RPM.
Two units now: the template to `%{_unitdir}`, and a new `meshbay-node-user.service`
that a person enables themselves without a password — which is what lets the
desktop client install a node without asking for one. It carries ExecReload, so
`meshbay-node reload` does not have to stop a service somebody is streaming from,
and documents the drop-in for a drive outside the home, RequiresMountsFor
included.
798 tests pass; e2e.py still passes end to end. Nothing here was built or
launched: no npm, no rpmbuild.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage C. Identity keys are per node, so a browser and a desktop client are two
keys on one account there — and the node refused the second where it accepted
the first. Without this, an account created natively could never be opened in a
browser without an operator code per node, and "a native client must not prevent
web use" would have been dead on arrival.
Device linking (node)
---------------------
`identities` is keyed by `(user_id, pk_ed25519)` instead of `user_id` alone. The
old shape did `INSERT OR REPLACE`, so a second device overwrote the first
silently; SQLite cannot change a primary key in place, so the table is rebuilt.
Existing pins are carried over — verified against a live roster with 10 of them,
nobody re-pairs.
A new device files a request bound by `sha256(code ‖ its own keys)`, and a key
the node **already pinned** countersigns it. The hub cannot: it has stored no
user keys since 2026-08-14, which is what makes this safe to do without an
operator in the loop.
**The code never reaches the node.** It lists this account's pending requests
with their stored hashes; the approver recomputes and keeps the match. A node
offering fabricated keys would have to produce a hash over a code it has never
seen. Nothing rests on a human comparing digits — that ritual was dropped in
12.1 as "correct, unusable as the default" and must not return by the back door.
The design document had the approver look a request up *by* its hash, which is
circular: computing it needs the keys being asked about. Corrected in both.
Revocation marks rather than deletes, because a deleted row is a key the node
would happily pin again — which is the laptop somebody just reported lost. Your
last device cannot be revoked: coming back would need an operator's code.
Hub — the only change in the whole plan
---------------------------------------
`POST /v1/users/auth` signs in with a device Ed25519 key, on the same pattern as
`/v1/nodes/auth`, plus `/v1/users/devices` to register, list and retire. New
`user_devices` table with an Alembic migration, because `create_all()` is not
one.
This is **not** the key directory that was H3, and the tests say so: nothing
reads it but the hub, no group key is ever wrapped for one, and it is a
different key from the per-node identities. What it does cost is metadata — the
hub now knows how many devices an account has and when each last signed in.
Also `client.minimum` / `client.recommended` in `GET /v1/hub/version`: an
installed client meets a newer hub the day the interface ships in a package, and
that is cheap now and awkward to retrofit.
Browser
-------
The `key_changed` refusal becomes `unknown_device` and offers a linking code
instead of telling someone to find their operator. The Members panel lists this
account's devices here, approves one by code, and retires one.
773 tests pass. `e2e.py` gained a step that links a device end to end against
the live deployment — file, list, recompute, countersign, then open the group
with the new keys and no code — and it also gained `recv_type`, because a step
that assumes the next message is its own answer reads an ack left by the step
before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage A — a group's content is a set of named roots
---------------------------------------------------
`shared_dir` becomes a list of {name, path, kind}. The name is the directory's
basename, derived once at add time and *stored*: recomputing it would
re-identify a whole library the day someone renames a folder on disk. Duplicate
names are refused case-insensitively and no root may contain another — both
compared with NFC folding, because most of these directories live on exFAT or
NTFS where `Films` and `films` are one directory.
Every index path carries its root name, in a one-root group as much as in a
five-root one. One path shape has to be got right once; two have to be kept
right for ever.
**A root that goes away freezes; it never empties.** Unmounting a volume makes
watchdog report every file under it as deleted, or presents an empty directory
to the next scan. Acting on either propagates deletions for a whole library to
every member, as though the owner had erased it. So a deletion is acted on only
once its root is confirmed readable, and availability is tracked per root — one
unplugged drive leaves the others serving. 12 tests, verified to fail against an
indexer without the check.
Events are not trusted to be complete either: ReadDirectoryChangesW drops them
under load and inotify on a FUSE mount misses changes made outside it. A
periodic reconciliation sweep is the only thing that recovers a missed event.
MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content
hash, a node id and an endpoint — no paths, no filenames — and private groups
register nothing (H7).
Stage B — one implementation behind every front door
----------------------------------------------------
C1 and C6 were both "a second path into the node with its own weaker
handshake". Two implementations of `revoke` with two authorization checks is
that shape one size down. `meshbay_node/ops.py` holds each operation once,
takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback
API is one `_op(...)` line per endpoint; the MNP handlers call the same
functions. test_ops.py asserts the shape rather than trusting it.
Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload`
(SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a
browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP
operations: rotation is the half of revocation that revocation cannot do, since
the ex-member holds the current key, and the node generates the replacement
with its own CSPRNG — no key material crosses the wire, which is what the C5b
rule is actually about.
Two bugs found by running it rather than by testing it
------------------------------------------------------
GroupIndex is keyed by **content hash**, so the same bytes at two paths are one
entry — which is also why a scan reports ten files and indexes nine.
Reconciliation compared paths, so it decided the second path was a missed event
every 60 s, rewrote the entry and pushed an index update to every connected
peer. Seen in a live node's log.
`meshbay-node reload` crashed on first use with `subprocess` unimported: the
module compiles fine, which is the "syntax, not names" trap already recorded for
the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be
added to the parser without an entry there.
Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the
wire carried "0.2" — harmless only because nothing imported it. And
_do_dir_create/_do_dir_delete referenced an undefined `filename` on their error
path.
740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
says who stopped
Bounding the client's read-ahead changed what a transcode slot is. It used to
be a burst — the browser took segments as fast as it could append them, so a
slot came back within the minute whatever the length of the film. Now it is
held for as long as someone is watching, so the cap counts simultaneous
viewers, and two of them meant the third was refused for the next hour and a
half.
The right number depends on the machine, so it belongs to the operator:
`[node] max_concurrent_streams` in node.toml, or
MESHBAY_MAX_CONCURRENT_STREAMS. Default 8 — one ffmpeg per viewer, remuxing
rather than encoding, idle on a pipe for most of the film. Zero, a negative
number, a non-number and a bool are refused with a warning naming the setting:
`Semaphore(0)` is not "no limit", it is a node where no video ever plays and
nothing says why, and TOML `true` would have become 1 by way of `int()`.
A stream also ends on the peer's silence now rather than on its stinginess. A
viewer buffered well ahead deliberately grants nothing for minutes, and the
old budget accumulated over the whole wait, so a keepalive that granted no
credit could not keep a paused film alive.
The rest is diagnosis, which is what this cost. `client_diag` carries the
player's own view — readyState, refused appends, buffered ranges, the video
element's error — into the node's log at DEBUG, next to the node's view of the
same stream. It is the only window into a phone, and every field is
stringified and cut short because all of it is peer-controlled. The node also
logs the first keepalive, which distinguishes a paced client from an unpaced
one at a glance, and progress every hundred segments, whose last line says
where a stream stopped and which side stopped it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reported from a phone: play a video, close the viewer, open another — the second
hangs and the third is refused. Three separate causes, found by instrumenting
rather than guessing, after two fixes that addressed real but different bugs.
A task nobody holds can be collected mid-flight. asyncio keeps only a weak
reference, so `ensure_future` with the result discarded may be garbage-collected
while running — "Task was destroyed but it is pending!" — and `_stream_video`
never reached the exit of its `async with sem`. `_spawn` holds every background
task; all nineteen call sites go through it.
Losing the peer must stop its work. The connectionstatechange handler popped the
session from a dict and nothing else, so a closed tab went on transcoding for
the full 120 s credit timeout. Measured in the log: 91 s of ffmpeg after the
connection closed. `shutdown_tasks()` now runs on the way out, and the credit
wait checks the channel before sleeping and polls in slices instead of once.
And `await proc.wait()` after `kill()` still deadlocks. ffmpeg outruns a
credit-paced viewer and fills the stdout pipe; stop reading it and the transport
cannot finish closing, SIGKILL or not. Measured against the live node with a
169 MB video, closing the viewer after 20 segments and asking for the next one:
15.1 s then "Server busy" before, 0.1 s / 0.0 s / 0.0 s after.
Chunk replies wait for room on the channel. Eight megabyte-sized chunks answered
as they arrived queued 8 MB with nothing watching — measured at 7.3 MB of
bufferedAmount in milliseconds. Fine on a LAN, minutes of head-of-line delay on
a busy link.
Upload names accept any script. The rule was ASCII-only, so `été.txt` was
refused — and so was `rapport (1).pdf`, which is the form `_free_name` produces
itself, meaning the node rejected names it had chosen. Widened to Unicode with
the C5a and H2 protections intact, plus a refusal of names that lie about
themselves: trailing space or dot, and the right-to-left override. Errors now
name the file, so one bad name no longer fails every upload in flight.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`get_messages` pages forward from the oldest message. That is the right shape
for "what happened since I last looked" and the wrong one for opening a
conversation, and the browser asked it for `since=0, limit=200` — so a group
with more than two hundred messages showed its first two hundred and the
exchange anyone came for was unreachable. Demonstrated on 300 messages: the
newest was simply absent from the answer.
`get_recent` and `get_before` page backwards, cursored on the row id rather than
the timestamp. Nothing makes a `time.time()` float unique, and a cursor on a
value two rows can share eventually skips a message or repeats it.
PING/PONG covers liveness on an already-open channel: a DataChannel whose peer
vanished without closing still reads as connected, and nothing noticed until a
real request hung. It is not a discovery mechanism — opening a connection to
ping costs a full ICE/DTLS handshake, measured at 0.6-7 s across two ISPs — so
presence in the group list comes from the hub's registry instead.
Both additions are backward compatible: an 0.1 peer sends no `before` and is
answered with the newest page, which is what it wanted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Removing a member.** The owner can do it from the Members tab, and it
is two halves in the order that fails safe: the node stops serving the
group key first (an operator-signed request, so a paired browser only),
then the hub drops the membership row. The other order would leave
someone able to reach a node that still serves them.
It is a membership, not an account. The user row is never written: their
other groups, their files and their pinned identity survive, because one
group's owner must not be able to erase someone from the hub. It is also
per group — a node hosting two loses them from one — and it does not take
back the key they already unwrapped, which is what rotating the GEK is
for. The confirmation and the panel both say so.
**Downloads and streaming through the disk, in both browsers.** The audit
this started as found two ways to put gigabytes in a tab.
Firefox and Safari have no File System Access API, so every download
there was collected in memory. A service worker fixes it: the page keeps
the writable half of a transferred stream, the worker answers a made-up
URL with the readable half and a Content-Disposition header, and the
browser writes it to disk as it arrives, with real backpressure. The
worker caches nothing and falls through on every request that is not one
of these downloads. A zip announces no Content-Length, since the archive
is larger than the files in it and a length we miss truncates the file.
Video was worse and affected both browsers. The node pushed ffmpeg's
whole output as fast as it was produced while the player consumed a
segment at a time, so the queue held the film — and appending all of it
hit the SourceBuffer's cap, where the handler logged the error and
dropped the segment, leaving a hole in the middle of the film with
nothing to show for it. Streaming is credit-based now, 24 segments of
256 KB in flight, verified against the live node: three credits, three
segments, then silence until more are granted. The player evicts what is
more than a minute behind the playhead and retries a refused segment
rather than dropping it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two things a Files panel needs and did not have.
**Removing a directory** is privileged, where creating one is not: it
acts on a name other members are using, on the operator's disk. It is
refused unless the directory is empty, and that rule is the safety
property — whatever the browser sends, this cannot destroy content. The
check runs twice, once before the challenge and once after the signature
comes back, because a file can land during the round trip. A file also
accepts its uploader's key; a directory has no uploader, so only the
operator's key will do.
**Downloading a folder** produces a zip built in the browser, written
straight to disk as the chunks arrive. An archive of a group folder is
routinely tens of gigabytes, so nothing is held: peak memory is one chunk
plus a small record per file. The node is not involved at all — it serves
the same encrypted chunks as any other download, holds no temporary
files, and cannot be asked to compress anything.
zipstream.js is store-only. Group content is video and images, already
compressed, so deflate would spend CPU on every byte to save nothing, in
the thread that is also decrypting. Sizes and CRCs go in a data
descriptor after each file because a stream cannot seek back to patch a
header, and zip64 kicks in per entry past 4 GiB and for the archive
itself. Because none of that can be checked from the Python side of the
house, test_zipstream.py runs the real module under Node and reads what
it produces with zipfile — CRCs, UTF-8 names, zip64 records and all. The
archives also pass `unzip -t`.
Firefox and Safari have no File System Access API, so there is nowhere to
stream to: the fallback builds the archive in memory and says so, with
the size, before starting rather than after failing.
One mistake worth recording: the first version of deleteDirectory passed
the node's own answer as the value to check the challenge against, which
turns the comparison into a tautology. It checks the path we asked for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Attaching a group to a node meant hand-editing node.toml with a UUID
copied from a browser URL, restarting, and knowing that gek-init exists.
Nothing in the CLI said so, and on a node reached over SSH there is no
paste buffer to carry a UUID across in the first place.
meshbay-node group add grenet --dir ~/grenet-share
The name is resolved against the operator's groups on the hub by the
daemon, which is the process holding the session. The [[groups]] block is
appended to node.toml as text rather than round-tripped through a TOML
writer: the file is hand-written and its comments explain decisions worth
keeping. The directory is created, and the command says what remains —
restart, then gek-init for that group.
It refuses a name it cannot find by printing the groups it can, with
their ids. That listing is the useful half of the answer and it was
missing everywhere: _daemon_api now renders an `available` list from any
endpoint that offers one.
The key is per group and pairing is not, which is the part that reads as
a gap until it is written down: one paired browser covers every group the
node hosts, while each group's key admits only its own members. §4 of the
user guide now says all three of those in one place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A description could only be set the moment a group was created, so every
group made before anyone thought of one stayed blank for good. The owner
can now edit it from the group's page, and PATCH /v1/groups/{id} takes it.
That endpoint takes the description and nothing else, deliberately. The
name, the visibility and the join policy are the terms members joined on;
a private group that can quietly become public is not the group they
agreed to be in. Changing those needs a decision about who gets told, not
a field on a form — there is a test saying so.
Separately, the legacy operator key is gone. `admin_pk_ed25519` in
node.toml named the operator before the roster existed and was kept so
that an existing deployment would keep working; nothing uses it, and a
second source of node authority is not something to carry around out of
politeness. Authority is the roster, read fresh on every check.
It is removed rather than ignored: a config that still names the key gets
a warning at startup pointing at the file. Dropping it in silence would
refuse invites and file deletion with a signature error that looks like a
bug somewhere else — which is exactly how finding M3 presented.
Two tests were verifying admin operations by naming a key in the context,
which was the legacy path. They now pair an operator into a roster, the
way an operator does. The authority test anchored on the deleted function
and passed vacuously once it disappeared; it states the invariant against
the verifier and the daemon instead.
Also defined .btn-secondary, used in four places and styled in none.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Correction to the previous commit. Uploads went wherever the member happened to
be looking, which spreads chat attachments through the tree and makes the
destination a client-supplied path — surface that had to be defended. Everything
a member sends now lands in `uploads/` at the root of the shared directory:
visible, one place, easy for the operator to look into or empty.
Chat attachments go there too, so the separate out-of-tree thumbs directory is
not needed and is not built. They were already ordinary uploads; now they are
ordinary uploads that land somewhere sensible.
The destination is chosen by the node, so a client naming somewhere else changes
nothing — the traversal surface simply is not there on this path. safe_subdir()
remains for dir_create, where the path genuinely does come from the client, and
keeps its tests.
One shared directory means name collisions are ordinary rather than adversarial:
every camera produces IMG_1234.jpg. The node finds a free name — "IMG_1234 (2).jpg"
— and reports it in the ack, because a chat message has to point at the file that
was actually written and not at someone else's. Nothing is ever replaced, which
is the property the per-user quarantine existed for (C5a) and the one the tests
assert; they fail if the free-name search is removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The per-user quarantine is gone. `.uploads/{user_id}/` was the fix for C5a, and
it worked, but it made the shared directory something nobody could organise:
every file landed under a uuid nobody recognises. Files now go where the member
is looking, most often the root.
What the quarantine actually bought is kept, and is now what the tests assert
rather than the location:
- an existing file is never replaced. That was the real defect — overwriting a
file also made the attacker its recorded uploader, and therefore able to
delete it through the uploader path
- the name allowlist is unchanged
- the destination is confined under the shared root
That last one is new surface: the directory arrives from the client. safe_subdir()
is the single place that decides, with two independent guards — every segment
against the name allowlist, and the resolved result under the root — because one
of them will eventually be refactored by someone who does not know why it is
there. Ten traversal cases are covered, and they fail if both guards go.
Also adds `dir_create` (any member may organise a shared directory; audited like
anything that writes to the operator's disk) and makes the node report its real
directory list in index_sync — folders were inferred from file paths, so a new
empty one, or one that had been emptied, simply did not exist as far as the UI
was concerned.
Two C5a tests changed their assertions deliberately, as C5b's did before: they
encoded the quarantine path, which is the thing being removed. The property they
existed for is asserted more directly than before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
One keypair was copied to every node its owner joined, so cracking the bundle on
any single node yielded the identity used on all of them: their content on other
operators' machines, and the ability to sign as them anywhere. That lateral reach
was the part of C4 worth attacking.
Each node now gets its own keypair, generated the first time its owner joins it
and left with that node alone. An operator who cracks what sits on their own disk
holds a key that is a stranger to every other node — and on their own node, one
that unlocks nothing they did not already hold: they serve the content, the index
and every byte of it by design.
Nothing changes for the user. A first contact with a node already needed that
operator's code, and the key is created in the same step; a second browser still
recovers it from the node with the passphrase alone. Two operators can also no
longer tell they host the same person by comparing keys.
BREAKING, and deliberately without a compatibility path — the deployment is wiped
for the next demo:
- users.pk_ed25519 / pk_x25519 dropped (migration a7c31f9e40b2)
- registration no longer sends or stores a key
- PUT /v1/users/me/keys and regenerateKeys() gone; rotation is now
`member unpin` plus a fresh code, decided on the machine that pinned it
- /pubkeys returns an account id and a node's linking key. It was the directory
H3 read, and nothing wraps for it any more
- the pk_user JWT claim is gone
That last one closed a live defect the inventory turned up: the node recorded
pk_user as the uploader's identity and authorized deletion against it, so a hub
issuing a token naming its own key could delete anyone's uploads on any node.
Attribution now uses the key the node itself pinned.
A simplification falls out. Registration generates nothing, so a scripted signup
is a real account: `demo.py bootstrap` takes a wiped hub and node to a working
demo with no browser, which was impossible while keys were born in one.
Also fixes, found by running it on a wiped deployment: the key handed back on a
join now belongs to the group the connection is for, not the group named in the
invitation — an operator pairs node-wide but redeems the code while opening a
group, and expects to read it.
Tests: 343, including the two that state the property — a key pinned by one node
is refused at another, and someone else's code does not admit it. Verified end to
end against a wiped hub and node: bootstrap, pair, invite, join, download,
stream, second browser, revoke.
Design: docs/per-node-identity-v1.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Both found by deploying the thing and running the workflow end to end. Neither
was reachable from the test suite, for the same reason in each case: the tests
knew something a real client cannot.
1. A first-time joiner had no way to learn node_pk.
join_request signs a transcript naming the node, and the node key was only
sent in handshake_ack — which an invited member cannot reach, having no GEK to
prove. joinGroup() therefore threw "handshake incomplete" and the browser path
for an invited member was broken. Every test built the transcript from a node
key it already had, so nothing noticed.
The challenge now carries node_pk. It is unverified at that point and never a
substitute for the ack: the ack still proves possession and signs the
transcript, the client checks the two values match and refuses a peer that
changed identity mid-handshake, and TOFU pinning is unchanged. A wrong value
only makes our own verification fail.
test_invite_then_join_delivers_the_gek now takes the key from the challenge
instead of from sk_node, so it proves a real client can learn it.
2. The roster pinned everyone without a name.
`_do_join_request` took the username from the session, which takes it from the
JWT — and the hub puts no username claim in a token. So identities were pinned
with an empty name and `member revoke <name>` could never match: the live node
answered "known: , ,". Invitations now carry the name (new invites.username
column, with a migration for the roster DBs already out there), and the CLI
resolves a name through the daemon: its own roster first, the hub as fallback
for identities pinned before this.
The harness that found them is QE/deploy/e2e.py — gitignored with the rest of
QE/, so it is not in this commit. It does the SPA's job in Python against the
live deployment: hub login, WebRTC via hub signaling, the unified handshake,
joining with a code, index, chunk download and MSE segments.
Verified against meshbay.org and the local node: an account registered from
scratch is invited by code, receives the group key wrapped for a key it proved it
holds, downloads and decrypts a file, streams 5 encrypted fMP4 segments,
reconnects with no code, and is refused after `member revoke`. The node audit log
shows invite_create → join_pinned(via=code) → gek_wrapped → handshake, then
join_no_gek once revoked.
Tests: 232 node+common.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|