| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`search_listed` is a per-group setting on the node, changed by a signed
operator op and carried in the sealed handshake ack. Search reads it after
the handshake and stops there: no index is fetched, cached or merged, in any
of the four views, and the page says how many groups it left out. The switch
is a "Search" section in the group's settings, shown to the operator.
Absent means listed, at every layer: roster default, ack default, and the
client only drops a group on an explicit `false` — so an upgrade or an older
node removes nothing from anyone's Search.
It is a listing preference and protects nothing: the node serves the same
index to Search and to the group page and cannot tell them apart, every
member lists the group by opening it, and a client that ignores the flag
lists it in Search too. Design §9.11 says so, so it is never described as
private. The cost is one handshake per unlisted group, because only the node
knows the setting.
Search also took `nodes[0]` twice — for the index and for the pooled
connection — the defect 4cce50f fixed on the group page only. One
`connectToGroup` now walks the list the same way: a refusal about this
browser stops, `not_hosted` or a failed connection moves on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XuNrwLf5EFWCMHzfoEvnpm
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`video_root`, `audio_root` and `photo_roots` are gone — the messages, the
signed operations, the handlers, the `ops` wrappers, the three scalars on the
handshake ack, and the client's handlers for their acks. `app_directories`
does the same thing for every application, keyed by the app's own registry
name, and it is what the SPA has been sending.
The three were the same instruction three times, differing only in the key they
wrote and whether they carried a string or a list. That shape is what made
adding an application mean adding a message type, an op, a handler and a widget;
it also meant three validation paths, and the older ones validated nothing —
a typo was stored and then quietly matched no entry, an app showing an empty tab
with no way to tell "misconfigured" from "no files yet".
**What stays, and why.** `Roster.LEGACY_DIR_KEYS` still reads `video_root` and
friends out of `group_settings`: that is a key on an operator's disk, not on the
wire, and a node upgraded into this must find its own configuration. The Search
page still reads its own older cache keys, for the same reason — the cache
outlives a deploy. `CTX_ALIASES` keeps only `chat`, which is the one app whose
second name something still reads.
The two per-app policy test files go with the messages. What only they held —
the real challenge/response path from message to database, which no other test
exercises — is retargeted at `app_directories` in
`test_app_directories_signed.py`, and the handler's own refusals (unknown app,
malformed `directories`, nobody to authorize it) join `test_app_directories.py`.
Node and common suites 1368 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The group-wide `member_upload` switch is gone: the message, the signed
operation, the field on the handshake ack, the `upload` alias on every root in
the index payload, and the client's fallback path to it.
Whether a member may write has been a property of each root for a while, and
that is the model that survives: a single flag over the group cannot express
"this library is published read-only and that folder is a drop box", which is
the ordinary arrangement. What was left of the switch was a handler that logged
a deprecation and acted on nothing, and a client that read `ack.member_upload`
whenever the roots carried no `writable` — a second source for one question,
with whichever the code consulted first deciding it.
`roots.describe()` drops `upload` for the same reason: it was `writable` under
an older name, and two names for one boolean is one too many.
The paperclip now says "nowhere to write" rather than picking a root, in a group
that has none writable. That is the honest answer; the fallback picked whatever
came first and failed at send time.
Node suite 1215 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AsoWC3GmhNdwVFomW3QjH3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Step 3 of ~/next/improve-downloads.md. Step 2 built the pools with constants;
this gives them to the operator, in the two scopes they belong to.
**The pools are the machine's.** `[node] max_concurrent_downloads` and
`max_concurrent_uploads`, default 8, on the §2.11 pattern: node.toml for a
fresh install, a roster.db override for immediate effect, editable from the
Node page and from `meshbay-node transfers show|set`, applied live through the
one `set_capacity` step 1 fixed.
**The per-member cap is a group's.** How many transfers one member may run at
once here — on the node like every other group setting (not the hub, which
would have authority over someone else's disk; not node.toml, which is
hand-written and needs a restart), changed by a signed operator instruction
(`OP_TRANSFER_LIMITS`, subject "d=2,u=2" so what is signed names the outcome),
broadcast to the group, and read live by the pools.
That was the one thing step 2's shape could not express: `per_member` was a
single node-wide number. `group_limits` and `member_cap(kind, member)` make it
a lookup — the group's own value if it has one, the node's default otherwise —
and it is deliberately the only dimension that is not node-wide.
Three refusals, each with a test:
- **absent means the default (2), never "unlimited".** A group that predates
the setting coming back unlimited would leave the node-wide pool as the only
control, which is the situation slots exist to end;
- **zero is not "unlimited"**, and is not "this member may not transfer"
either: the floor is one everywhere, and the CLI says to revoke the member
instead;
- **an unreadable row reads as unset**, not as zero — the same discipline the
sealed messages follow, where a payload that does not open must never become
a default state on its own.
`handshake_ack` carries this member's own caps for this group, so the interface
can say "2 of your 2 slots are busy" instead of drawing a bare spinner. Absent
reads as "no limit known" and the hint is not drawn — never as "unlimited",
which would have the interface contradicting the node.
1164 node, 793 hub, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Chat messages are sealed with AES-256-GCM under a key derived per group, per
epoch, per *device*, and signed over the ciphertext with the device key the
node pinned. The node relays and archives; it cannot read a message.
There is no switch. MNP goes to 2.0 and MNP_MIN_SUPPORTED moves with it, so a
1.x peer is refused at the handshake with `version_too_old` rather than
admitted and then unable to speak. An opt-in flag was designed and rejected:
every node is a test node, so it would have bought nothing and left a plaintext
branch reachable — C6's lesson one feature later. A test reads the source and
refuses any code that consults a `chat_encrypted` setting.
Not Sender Keys, and `senderkeys.py` is now documented as unused. With
distribution under the group key and a node that serves history to devices
which were not present, the node must retain each chain's earliest key, and a
chain key at iteration i yields every message key from i on by pure HKDF —
forward secrecy is zero either way. What the ratchet was left buying was
stateful client code with silent failure modes, three of them reproduced: any
member could sign as any other, a second device dropped the first's chain, and
the skipped-key cache grew without bound. The reasoning is in
docs/chat-sender-keys.md, which is the specification and the decision record.
Epochs, not rotation: the epoch key is wrapped under the group key at delivery
and never stored under it, so `gek_rotate` is a re-wrap. A group-key-derived
archive key would have made every message ever sent unreadable on the first
`member unpin`, which is the documented step after removing a member. A new
epoch opens on member revoke/unpin, device revoke and `gek_rotate`; old epochs
are kept and still delivered, so history stays readable to everyone who could
already read it, and nothing anywhere deletes one.
Three prerequisites this needed, each a live defect on its own:
* The peer registry was keyed by user_id, so one account's second device
evicted the first and the broadcast skipped recipients by account — a
person's phone never saw what they typed on their laptop.
* The handshake authenticated an account, never a device. `device_hello`
(additive, signed, refused unless the key is a live device of this account in
the node's own roster) is what lets the node refuse a member claiming
somebody else's key.
* `_admin_exec_file_delete` authorized against the exact uploading key, so
device linking had already broken deleting your own file from your other
device. It now authorizes against any non-revoked device of `uploader_id`.
Found by driving the real panel over the real transport, not by reading source:
`chat_keys_resp` was routed by arrival order and handed to an unanswered
`media_meta_req` — the original frozen-tab defect in a message type that did
not exist when that probe was written. And `_asText` had been deleted with an
unrelated helper beside it; its only caller sits inside a promise the panel
catches, so every conversation rendered empty with nothing in the console.
Existing node data is migrated by QE/migration/migrate_chat_encryption.py
(not versioned, per the QE rule), run with the node stopped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`video_root` (a string), `audio_root` (a string) and `photo_roots` (a list)
said the same thing three ways: three roster accessors, three ops, three MNP
messages, three admin-op subjects. They become `set_app_directories(app_key,
paths)` and its single-directory wrapper, stored under `<app>_directories` and
keyed by the app's registry name — so an application can be added without
touching this layer, which is the whole claim of the plugin architecture.
The three old names still work. Their MNP messages are handled, and the roster
falls back to the old key when the new one is unset, so a node upgraded into
this keeps working with no migration step — the plan called for a script, and
a script nobody runs on the machine where it matters is worse than a fallback.
Two things are new rather than moved:
The paths are validated. The setters this replaces accepted anything, so a
typo — or a path left behind when a root was removed — was stored happily and
then matched no entry, leaving an app showing an empty tab with nothing to
distinguish "misconfigured" from "no files yet". Deliberately not
`RootSet.resolve()`: that also refuses a currently-unavailable root, and an
operator must be able to point an app at a library on a drive they ejected.
The legacy scalar is derived, never stored. `video_root` still rides on the
handshake ack for MNP 1.0 clients; kept as a second stored value it would
drift from the list within one run, which reads as "it works after a restart".
Also here: chat's own two settings (a directory, which must be on a read-write
root because it is a destination rather than a view, and a link-preview switch
gating the unfurl path — checked before the cache, or turning it off would
still serve every preview already fetched), the `app_directories`,
`chat_directory` and `chat_link_preview` MNP messages, the plural
`<app>_directories` on the handshake ack, and `music` as the app's one
identifier where storage said `audio` and the registry said `music`.
The Music enricher now resolves a boundary per configured directory rather
than one for the group: with several, a single boundary is wrong for all but
one of them, and for Music that is the difference between reading a folder as
an artist and reading it as a release.
Suite: 11 failures, all pre-existing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pvMdvLBG92jyhvD5pD6us
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Replace the upload boolean with per-root writable/removable/ejected flags.
Backend: new ops (update_root, eject_root, plug_root), MNP 1.1 protocol
messages, live RootSet updates so API always reflects current state, CLI
root subcommand (add/remove/set/list/eject/plug).
Frontend: SharedDirectoriesTable with optimistic toggle switches, eject/plug
in Files and Settings, upload gated on root.writable, ejected-root filtering
in all media apps, updated Create Group wizard, 10-locale i18n.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A one-click alternative to the full "Fix match" search-and-pick flow, and
reachable without SSH (`meshbay-node video rematch` clears a whole group).
- MNP 0.13: tmdb_rematch / tmdb_rematch_ack (additive — an older node
logs "unknown type", the button just does nothing). OP_TMDB_REMATCH,
signed like tmdb_override (media_cache is shared node-wide).
- media_cache.drop_tmdb_match(file_id): forgets the match AND the
override marker — deliberately stronger than clear_file_tmdb, since the
operator is explicitly asking for a fresh resolution.
- webrtc_server: _do_tmdb_rematch / _admin_exec_tmdb_rematch, dispatch +
admin-response routing, broadcasts tmdb_rematch_ack.
- transport.js: rematchTmdbMatch(fileId, signFn); 'tmdb_rematch' in the
admin-op allowlist; tmdb_rematch_ack handled like tmdb_override_ack.
- video-app.js: a "Re-match" button beside "Fix match" in the detail
modal (isNodeAdmin), then bumpMediaMetaGeneration(). video.rematch_one
key in all ten locales.
docs/mediacenter.md §10.1: V8–V13 marked done.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018BMLQjqFGCize2KtNBT79v
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Wizard (Electron):
- Auto-provisions node config (hub URL + username) from logged-in user
- node:start handles both cold start and restart of misconfigured daemon
- Waits for daemon to reach 'running', auto-links node key on hub
- probeNode accepts intermediate states for wizard progress feedback
Reset (meshbay-node reset):
- Unlinks node key from hub (DELETE /me/node_key, best-effort)
- Stops and disables daemon (systemctl --user disable --now)
- Erases ~/.config/meshbay, ~/.local/share/meshbay, ~/.local/state/meshbay
MusicBrainz contact:
- Resolved from owner's hub email instead of per-node roster config
- Removed musicbrainz_contact UI and WebRTC handshake field
- Removed set_musicbrainz_contact/musicbrainz_contact from roster
Node pairing:
- Added operator pairing banner on NodePage
- Added operator_paired flag to list_groups
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A new group application (docs/apps.md's plug-in mechanism), following the
plan in docs/photos.md. Unlike Videos/Music: several photo roots per group
instead of one (photo_roots is a set, one signed op replaces it whole),
a single album-grid view with no third-party matching step, and per-photo
info read from the file's own EXIF at index time — no metadata service,
no credential, no outbound network call at all.
Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera`
on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`.
Node: roster.py stores photo_roots as a group_settings entry (JSON list,
same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the
whole set in one op, same pattern as apps_enabled; a new PhotoEnricher
(indexer/enrich_photo.py) runs Pillow in its own small bounded pool,
separate from the video/audio pools, producing a resized thumbnail plus
the two EXIF fields — never GPS, checked by a grep-based regression test.
Client: photos-app.js — one album card per directory containing images,
a per-album photo grid, and a lightbox with next/previous (keyboard and
buttons), zoom in/out/fit/100% starting from the actual on-screen fit
percentage, and a "zip this album" button reusing files-app.js's own zip
mechanism (lifted into file-utils.js's downloadDirectory so both call the
same implementation). group-settings.js gets an add/remove multi-root
picker, distinct from Videos/Music's single-value one.
Bugs found and fixed before this ever shipped, worth keeping the story of:
- enrich_photo.py read width/height from the raw image *before* applying
EXIF orientation correction, and read DateTimeOriginal off the plain
0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which
Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict
round-trips through Pillow either way, which is exactly what would have
hidden both bugs; the regression test builds EXIF with piexif instead,
matching what real hardware produces.
- photos-app.js's album grouping stripped a trailing path segment from
entry.path under the assumption it still carried a filename — it
doesn't (files-app.js's own convention: e.path is already the
containing directory), so every album collapsed one level into its
parent. Found live against a real multi-folder library.
- transport.js's ADMIN_OP_TYPES allowlist (already the fix for an
identical bug on video_root/apps_enabled, see 4783d81) was missing
photo_roots: its admin_challenge matched no pending request and was
silently dropped, so saving a photo root just timed out after 30s with
no error.
- daemon.py pruned a thumbnail when its file left the index (root removed
or reconfigured) but never forgot the content hash was "already
attempted" — the same bytes reappearing under a renamed/relocated root
(an operator's real workflow) were then permanently skipped, forever,
with nothing to indicate why. Discarding the attempt alongside the
cache entry on prune is what makes pruning actually reversible.
- packages/meshbay-client's app:// protocol handler served every file
with no Cache-Control header, so Chromium was free to serve a stale
cached copy indefinitely — none of several `npm run sync-ui` + reload
cycles during development actually picked up the new code until the
renderer's disk cache was cleared by hand. Now sends Cache-Control:
no-store.
- the lightbox's zoomed image used flex centering (align-items/
justify-content: center) combined with overflow: auto — a well-known
trap where the browser centers overflowing content by shifting it, and
the leading half of that overflow (here, the top of a zoomed photo)
sits outside what the scrollport can actually reach. Reported live as
"unusable". Fixed by switching to top/left alignment once zoomed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
musicbay.md's original call — Music needs no root, tag reads are cheap so
just cover the whole shared tree — didn't hold up against a real messy
library: everything under every shared folder got mixed together with no
way to scope Music down to an actual music collection. This adds an
audio_root setting, symmetric to video_root in every respect: signed
operator op (audio_root/audio_root_ack, MNP bumped to 0.10), validated
against a real directory in the group's own roots before a signature is
even asked for, gates tag/cover enrichment exactly like video_root gates
ffprobe/TMDB (nothing runs until it's set, only files under it once it
is), and a set/change fires a one-off sweep of whatever the folder already
contains.
The old trigger — sweep everything the instant "music" joins enabled_apps
— is gone along with the root-less design it belonged to; setting
audio_root is now the trigger, mirroring set_video_root's
enrich_video_root_fn exactly.
Test coverage mirrors the video_root suite: policy (refuse before a
signature round trip, accept/store correctly) and the enrichment gate
itself (nothing without a root, only files under it, sweep on set).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements the node half of docs/musicbay.md against MNP 0.8:
- IndexEntry gains artist/album/track_no (reuses duration/thumb_hash/
display_title, already generic). New musicbrainz_config/_enabled and
music_meta_req/_resp message pairs, mirroring the TMDB shape.
- title_parse.parse_track_filename: track-number-prefix + title parsing,
fallback-only (embedded tags are the primary source, unlike Videos).
- indexer.enrich_audio.AudioEnricher: mutagen-based tag/embedded-cover
extraction through its own bounded pool (asyncio.to_thread, no
subprocess — no ffmpeg-shaped deadlock risk). Gated on "music" in a
group's enabled_apps rather than a video_root-style scoped folder.
- musicbrainz.py: MusicBrainzClient — no API key (unlike TMDB), just a
self-imposed ~1 req/s pace and a configurable, non-default User-Agent
contact string; inert (no calls at all) when no contact is configured,
never sends an unidentified client.
- media_cache.py: file_mbid/mbid_meta tables alongside the existing TMDB
ones, cover art reusing the thumbs table via a synthetic
musicbrainz:{mbid} id, pruned on file deletion.
- roster.py/ops.py/webrtc_server.py: musicbrainz_contact (node-wide) and
musicbrainz_enabled (per-group, from the start) as signed operator
settings, ALLOWED_APPS gains "music", _do_music_meta_request resolves
and caches a release-level MusicBrainz match per (artist, album).
- daemon.py: AudioEnricher/MusicBrainzClient wired alongside the video
ones; a group's existing library is swept when "music" is newly
enabled (no video_root equivalent — see musicbay.md §2.1).
41 new tests (musicbrainz.py against a mocked transport, admin-op policy
for both new settings, media_cache round-trip/pruning, enrich_audio
end-to-end against real ffmpeg-generated MP3s). Full suite (common +
node + hub): 1116 passed, no regressions.
Client-side (music-app.js, persistent player bar) not started yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KBi7ALLGfwcjBXt57yNMcy
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three bugs found live testing the Videos app against a real HEVC/EAC3 show,
plus a design change requested afterward:
- Streaming always did "-c:v copy", which faithfully reports a source's real
hev1 codec string but is unplayable in a browser with no HEVC decoder
(most Chrome/Linux builds). The node now transcodes to H264 whenever the
probed codec is browser-incompatible (media_probe.py's new
BROWSER_INCOMPATIBLE_VIDEO_CODECS), with a `transcode_incompatible_video`
node.toml opt-out for operators who know their viewers already decode it.
- Dropping a whole season into an already-watched folder gave no scanning
indicator and no progress bar: IndexProgress was only ever updated by the
two bulk scan paths, never by the real-time per-file watchdog path
(_schedule_update/_debounce/_update_entry). That path now accounts a
"burst" the same way, without double-counting a file rewritten mid-debounce.
- A stray literal "0" rendered in the video detail modal when there was no
TMDB match (`meta.confidence` is 0, and `0 && x` renders "0" in JSX/htm,
not nothing) — `confident` is now a real boolean.
- Whether TMDB is used at all moves from a node-wide setting to per-group
(OP_TMDB_ENABLED/tmdb_enabled/tmdb_enabled_ack, scoped like OP_VIDEO_ROOT):
an operator running a real media-library group alongside test/demo groups
on one node wants outbound TMDB traffic for the one that needs it, not all
of them. The custom API token and query language stay node-wide, one
shared credential/cache (tmdb_config/OP_TMDB_CONFIG, unchanged reasoning).
MNP_VERSION 0.6 -> 0.7, additive.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
wizard polish
Two operator-facing fixes for a real 3-season show whose automatic TMDB
match was wrong at the show level: per-season overview/air_date tabs in the
detail modal (falling back to the show-level text when a season's own is
empty), and a "Fix match…" search-and-correct affordance that re-resolves
every file sharing the corrected show's display_title. New signed op
OP_TMDB_OVERRIDE and two read-only pairs (season_meta_req/resp,
tmdb_search_req/resp), MNP_VERSION 0.5 -> 0.6.
Also: the create-group wizard gets a spinning indexing indicator and an
app-selection step, group settings default the TMDB language to the
operator's own locale (never as a global default), and a file renamed
mid-session now re-triggers title parsing instead of being silently
skipped by the enrichment dedup guard.
Fixes two bugs found during this work: the search overlay's z-index lost
to the base video-overlay class and rendered invisibly, and season_meta's
own empty overview didn't fall back to the show-level one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LAmyXtc6dAADsH23ydXQpY
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implements docs/mediacenter.md: a "Videos" group application built on the
existing files index rather than a separate catalogue. On the node side,
new indexer enrichment (technical probe, filename/season parsing, thumbnail
generation) runs per-file once an operator has chosen a video_root for the
group, plus a TMDB client for on-demand poster/metadata lookups (never
client-side, thumbnails delivered over the existing chunk path). On the hub
side, a new video-app.js renders a lazily-mounted poster grid or a
thumbnail-only flat list, with TMDB entirely optional per group.
Along the way: the global apps registry now drives Settings' default-tab
picker instead of a hardcoded list, and the video_root is configured from
group Settings (like uploads) rather than from Files, with the node
refusing to run any TMDB/thumbnail work until one is set.
Fixes several bugs found via live testing against a real library, notably
a race between two effects writing the same "image ready" state that could
leave a poster grid spinning forever on a same-tab revisit — see
mediacenter.md §5.4 for the full account of each one.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reconcile, and delta sync
Indexer performance work, in four parts:
- Persistent (path, size, mtime) -> hash cache (indexer/cache.py) so a node
restart no longer re-hashes every file — measured at 23 minutes for a
114 GB library on a slow disk before this, near-instant after. Hashing
is deliberately kept sequential (max_workers=1): it was never actually
concurrent despite the pool size, and two interleaved reads seek-thrash
a spinning disk instead of going faster.
- Byte-based scan progress (IndexProgress), surfaced via the loopback
index-status route, the handshake ack, and a periodic INDEX_PROGRESS
push to connected peers — drives a progress bar in the Create Group
wizard and "add a directory" in Settings, and an animated presence dot.
Guaranteed to settle back to idle via try/finally and a final push on
the scanning->false transition.
- The reconcile backstop's directory walks now run in the executor
instead of blocking the daemon's event loop; its interval defaults to
10 min (was 60s) with adaptive backoff to 2h when nothing changes,
reset on a real change or a peer connecting, and is now a per-group
operator setting (signed op + group Settings UI).
- INDEX_DELTA wired up (protocol support existed, nothing called it):
_on_index_change now sends additions/deletions instead of rebuilding
the full entries list, coalesced over a short window so a burst of
file events produces one push, and the hub swarm registration for
public groups only (re-)registers newly added hashes.
Also fixes several bugs found while testing the above against real
libraries (a 114 GB and a 100+ GB group on a USB HDD):
- /api/reload blocked until the reload — including a brand-new group's
full initial scan — finished, which the Electron bridge's fixed 30s
call timeout turned into a hard failure on any real library. The route
now fires the reload without waiting (ops.start_reload), matching
add_root/remove_root's existing pattern; the wizard's own step order
was fixed to wait for the group to actually appear hosted before the
steps that need it (extra roots, GEK), with retries for the residual
race between that and the daemon's own bookkeeping.
- transport.js's hand-rolled msgpack codec had no case for uint64/int64
(0xcf/0xd3) and crashed decoding any message containing one — hit by
IndexProgress.scanned_bytes/total_bytes for any group over ~4.3 GB.
Verified against real msgpack-encoded bytes from the Python side.
- chat_hist_resp, and this change's own index_progress and
set_scan_settings_ack pushes, were not routed by message type and
could be handed to an unrelated pending request by the transport's
"oldest pending" fallback, stalling it until its own 30s timeout and
corrupting whatever received the wrong reply in its place.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GroupPage's 6620-line app.js carried Chat and Files wedged in directly, with
no way to add another group-level app without touching the shell itself. It
is now app.js (routing, non-group pages) plus nine focused files — apps.js
(the registry), chat-app.js, files-app.js, video-player.js, group-page.js
(the shell), group-settings.js, hub-client.js, icon.js and file-utils.js —
with docs/apps.md as the checklist for adding one (Videos/Music/Photos are
sketched there, not built).
Node side gained the matching enablement mechanism, mirroring
member_upload exactly: a roster setting, a signed apps_enabled op enforced
by _has_admin_authority, exposed in the handshake ack. Operators toggle
applications per group from Settings, which also gained a small reorder:
Invite, Pairing, Applications, Shared directories, Uploads, danger zone,
Your devices, Members.
Two bugs surfaced during the split, both missing an import across the new
file boundary and invisible to node --check or a module-load probe since
they only throw when the code path actually runs:
- group-page.js called onRefreshAuth on a stale-token handshake rejection,
but app.js never imported refreshAccessToken from hub-client.js — so a
brand new member (including a group's own creator) hit "Not a member of
this group" and the retry silently failed, throwing before it could
refresh the token.
- chat-app.js called getLocale() for message timestamps without importing
it from i18n.js. Opening Chat on a group with real messages threw mid-
render; uncaught, that appears to wedge Preact's render scheduler, so
every button on the page stopped responding until reload.
Caught the second class of bug with a proper no-undef audit across all
split files (a temporarily installed ESLint 9, since the system one is too
old to parse this codebase's syntax) rather than trusting grep. 827 tests
pass; 6 new ones cover the apps_enabled policy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Node admin panel (NodePage) now covers every CLI operation over MNP:
group attach/detach, roster, member unpin, GEK rotate, denylist, reload.
Daemon hot-loads new groups and tears down removed ones on config reload
instead of requiring a full restart. Group attach/detach via MNP or
local API triggers an automatic reload so the group is live immediately.
Fixed GroupPage hang on first visit to a newly created group: the JWT
issued at login didn't include the new group, the node rejected with
not_a_member, and the token-refresh path returned without re-triggering
the connect effect (Boolean(token) didn't change). Now bumps retryKey
after a successful refresh so the effect re-runs with the fresh token.
NodePage marks groups hosted by the node but absent from the hub with a
"not on hub" badge so stale groups are visible and easy to remove.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A group where every member may add files stays the default. Some groups want a
library the operator curates, and until now the only way to get one was to
designate no upload root at all — which refuses the operator too.
**The node enforces it; the interface merely stops offering it.** The Upload
button in the Files toolbar and the paperclip in the chat composer both
disappear, which is a courtesy to the people who are not trying. The control is
`_do_file_upload` refusing with `member_upload_off`, so a member on an old tab,
or one speaking MNP directly, gets the same answer. There is a test for each,
and the enforcement test is in the node package rather than beside the UI one so
nobody reads the hidden button as the mechanism.
**Changing it is a signed operator instruction** — `OP_MEMBER_UPLOAD`, on the
same path as removing a member. An unsigned one would let any member turn it
back on and make the setting a suggestion. The transcript's subject is `on` or
`off`: what the operator is shown before signing has to name the outcome, not
the operation.
**It lives on the node**, in a new `group_settings` table in `roster.db`. Not
the hub, which has no business deciding who may write to someone else's disk.
Not `node.toml` either: that file is hand-written and full of comments recording
decisions, `ops.py` appends to it rather than round-tripping it through a
writer, and a setting toggled from a panel must not rewrite the operator's file
or need a restart. The value is cached in the group context because the upload
path is synchronous, and the signed operation updates both — storing it without
applying it would make the panel say one thing while the node did another.
**Absent means allowed**, at every layer: no row in the table, no key in the
context, no field in `handshake_ack`. An older node and an older client both
behave exactly as before, and upgrading never silently closes a group. Each of
those three has its own test, because they fail independently.
The operator is always exempt — otherwise turning it off locks them out of their
own node with a config file and a restart as the only way back. `is_node_admin`
was being computed in two places by then and is now one function, since two
copies of "is this the operator" is how the ack and the gate come to disagree.
A change reaches everyone already connected via `member_upload_ack`, so the
button goes without a reconnection. That message is both a broadcast and the
reply to the request that caused it, which is why the client does not return
early on it.
Docs updated for a cold start: draft-v6 §2.1b and change 9, a new "Where Phase
13 stands" section in CLAUDE.md recording what is built, deployed and still
missing, the module map row, and desktop-client-v1 §10b on the Settings tab and
where group settings live.
883 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stage A — a group's content is a set of named roots
---------------------------------------------------
`shared_dir` becomes a list of {name, path, kind}. The name is the directory's
basename, derived once at add time and *stored*: recomputing it would
re-identify a whole library the day someone renames a folder on disk. Duplicate
names are refused case-insensitively and no root may contain another — both
compared with NFC folding, because most of these directories live on exFAT or
NTFS where `Films` and `films` are one directory.
Every index path carries its root name, in a one-root group as much as in a
five-root one. One path shape has to be got right once; two have to be kept
right for ever.
**A root that goes away freezes; it never empties.** Unmounting a volume makes
watchdog report every file under it as deleted, or presents an empty directory
to the next scan. Acting on either propagates deletions for a whole library to
every member, as though the owner had erased it. So a deletion is acted on only
once its root is confirmed readable, and availability is tracked per root — one
unplugged drive leaves the others serving. 12 tests, verified to fail against an
indexer without the check.
Events are not trusted to be complete either: ReadDirectoryChangesW drops them
under load and inotify on a FUSE mount misses changes made outside it. A
periodic reconciliation sweep is the only thing that recovers a missed event.
MNP 0.2 → 0.3 (additive). The hub needs no change: SwarmSource carries a content
hash, a node id and an endpoint — no paths, no filenames — and private groups
register nothing (H7).
Stage B — one implementation behind every front door
----------------------------------------------------
C1 and C6 were both "a second path into the node with its own weaker
handshake". Two implementations of `revoke` with two authorization checks is
that shape one size down. `meshbay_node/ops.py` holds each operation once,
takes the daemon state, and knows nothing about HTTP, argv or MNP. The loopback
API is one `_op(...)` line per endpoint; the MNP handlers call the same
functions. test_ops.py asserts the shape rather than trusting it.
Phase 14 is finished on top of it — `group list`, `gek init|rotate`, `reload`
(SIGHUP), `denylist show|clear`, `file list|rm`. **No operator action requires a
browser any more.** Plus `gek_rotate` and `member_unpin` as operator-signed MNP
operations: rotation is the half of revocation that revocation cannot do, since
the ex-member holds the current key, and the node generates the replacement
with its own CSPRNG — no key material crosses the wire, which is what the C5b
rule is actually about.
Two bugs found by running it rather than by testing it
------------------------------------------------------
GroupIndex is keyed by **content hash**, so the same bytes at two paths are one
entry — which is also why a scan reports ten files and indexes nine.
Reconciliation compared paths, so it decided the second path was a missed event
every 60 s, rewrote the entry and pushed an index update to every connected
peer. Seen in a live node's log.
`meshbay-node reload` crashed on first use with `subprocess` unimported: the
module compiles fine, which is the "syntax, not names" trap already recorded for
the SPA. test_cli_dispatch.py now walks every verb and refuses to let one be
added to the parser without an entry there.
Also corrected: protocol.py declared a second MNP_VERSION of "0.1" while the
wire carried "0.2" — harmless only because nothing imported it. And
_do_dir_create/_do_dir_delete referenced an undefined `filename` on their error
path.
740 tests pass; QE/deploy/e2e.py passes end to end against the live deployment.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Removing a member.** The owner can do it from the Members tab, and it
is two halves in the order that fails safe: the node stops serving the
group key first (an operator-signed request, so a paired browser only),
then the hub drops the membership row. The other order would leave
someone able to reach a node that still serves them.
It is a membership, not an account. The user row is never written: their
other groups, their files and their pinned identity survive, because one
group's owner must not be able to erase someone from the hub. It is also
per group — a node hosting two loses them from one — and it does not take
back the key they already unwrapped, which is what rotating the GEK is
for. The confirmation and the panel both say so.
**Downloads and streaming through the disk, in both browsers.** The audit
this started as found two ways to put gigabytes in a tab.
Firefox and Safari have no File System Access API, so every download
there was collected in memory. A service worker fixes it: the page keeps
the writable half of a transferred stream, the worker answers a made-up
URL with the readable half and a Content-Disposition header, and the
browser writes it to disk as it arrives, with real backpressure. The
worker caches nothing and falls through on every request that is not one
of these downloads. A zip announces no Content-Length, since the archive
is larger than the files in it and a length we miss truncates the file.
Video was worse and affected both browsers. The node pushed ffmpeg's
whole output as fast as it was produced while the player consumed a
segment at a time, so the queue held the film — and appending all of it
hit the SourceBuffer's cap, where the handler logged the error and
dropped the segment, leaving a hole in the middle of the film with
nothing to show for it. Streaming is credit-based now, 24 segments of
256 KB in flight, verified against the live node: three credits, three
segments, then silence until more are granted. The player evicts what is
more than a minute behind the playhead and retries a refused segment
rather than dropping it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two things a Files panel needs and did not have.
**Removing a directory** is privileged, where creating one is not: it
acts on a name other members are using, on the operator's disk. It is
refused unless the directory is empty, and that rule is the safety
property — whatever the browser sends, this cannot destroy content. The
check runs twice, once before the challenge and once after the signature
comes back, because a file can land during the round trip. A file also
accepts its uploader's key; a directory has no uploader, so only the
operator's key will do.
**Downloading a folder** produces a zip built in the browser, written
straight to disk as the chunks arrive. An archive of a group folder is
routinely tens of gigabytes, so nothing is held: peak memory is one chunk
plus a small record per file. The node is not involved at all — it serves
the same encrypted chunks as any other download, holds no temporary
files, and cannot be asked to compress anything.
zipstream.js is store-only. Group content is video and images, already
compressed, so deflate would spend CPU on every byte to save nothing, in
the thread that is also decrypting. Sizes and CRCs go in a data
descriptor after each file because a stream cannot seek back to patch a
header, and zip64 kicks in per entry past 4 GiB and for the archive
itself. Because none of that can be checked from the Python side of the
house, test_zipstream.py runs the real module under Node and reads what
it produces with zipfile — CRCs, UTF-8 names, zip64 records and all. The
archives also pass `unzip -t`.
Firefox and Safari have no File System Access API, so there is nowhere to
stream to: the fallback builds the archive in memory and says so, with
the size, before starting rather than after failing.
One mistake worth recording: the first version of deleteDirectory passed
the node's own answer as the value to check the challenge against, which
turns the comparison into a tautology. It checks the path we asked for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The invite flow fetched the invitee's pk_x25519 from the hub and wrapped the
GEK for whatever came back (app.js:1466, and gek-init did the same server-side).
The hub is the key directory, so a hub answering with its own key was handed the
group key by an honest member following the protocol exactly. No forgery, no
injection, nothing for the client to notice. That was H3.
The fix is not safety numbers. Nobody reads the directory any more:
- the node holds the GEK and wraps it itself, on every connection, for the
X25519 key the joiner signed with their Ed25519 identity in one transcript
(meshbay:join:v1), so the identity key vouches for the encryption key;
- identities are bound to accounts by a one-time code the hub never sees —
40 bits, single use, one account, bounded per connection AND node-wide;
- the node's own roster decides who may receive the key. Hub membership lets
someone reach a node; it no longer gets them anything. A hub that invents an
account and mints it a token is answered not_authorized_for_group.
Safety numbers would have made substitution detectable by a human who checks, at
the moment there is nothing to check against — first contact. Removing the lookup
makes it impossible, and costs the user one code to pass along.
M3 falls out of the same work. The daemon auto-pinned its own keystore key as
admin_pk_ed25519 while the browser signs with the user identity key, so every
privileged operation failed closed with a signature error that looked like a bug
somewhere else; the demo only worked because a deploy script overwrote the value.
Authority now comes from the roster, established locally by `operator pair`.
Asking the hub for the operator's key — the obvious-looking fix — would have let
the hub install itself as node administrator.
BREAKING: gek_bundle_store is deleted, not gated. No member hands the node key
material at all, so C5b becomes structural rather than an authorization to check.
Existing stored bundles are still served, so current deployments keep working.
Also:
- join_policy (invite|open) is read from node.toml, never from the hub — a hub
able to declare a group open would be handed its key. Unknown group ⇒ invite.
- admin signatures are verified against the roster on every check, so unpinning
takes effect without a restart. admin_pk_ed25519 stays readable as legacy.
- two C5b tests were rewritten, deliberately: they asserted that
gek_bundle_store demanded an operator signature, and the message is gone. They
now assert the stronger property. The file says not to fix these tests, so
this is the record of why they changed.
- a slice-1 bug found while writing slice 2: connect() never passed skEdB64, so
pairing would have failed at runtime with no test able to catch it.
Tests: 152 node+common here, including an end-to-end DataChannel run where a
member who has never held the group key redeems a code in the pre-proof window
and receives the key wrapped for a key only they can open.
Design: docs/invite-pairing-v1.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
|
|
Phase 11.5 — findings H1, C5a, H2, C5b, H5 (see second-review.md).
Batched together because the node-side changes share webrtc_server.py and
cannot be separated into working commits.
H1 — cross-group chat leak. chat_store, the peer registry and the display-name
cache were read from the shared transport context, and daemon.py hoisted the
FIRST group's chat store onto it. On a node hosting several groups every
group's messages went to one database, chat_history served them back to members
of every other group, and chat broadcast reached all peers regardless of group.
All three now resolve through _group_ctx().
C5a — upload confinement. Uploads landed in the shared root under a
client-chosen name and overwrote whatever was there. Any member could destroy
the operator's files, and by becoming the recorded uploader of the replaced
file could then delete it through the uploader path, bypassing the Ed25519
admin challenge. Uploads now go to a per-user quarantine (.uploads/{user_id}/),
refuse to overwrite, and enforce chunk ordering, a filename allowlist and a
size cap.
H2 — stored XSS in the node admin UI. Filenames chosen by any group member were
interpolated raw into the localhost UI, which has no authentication, so script
execution there equals control of the node admin API. Now html.escape()
throughout, textContent in the audit table, plus CSP/nosniff/no-referrer. The
CSP contains exfiltration but cannot stop injected inline script — escaping is
the fix.
C5b — group key seizure. gek_bundle_store wrote whatever any member sent and
auto-activated bundles addressed to the node operator. The operator's X25519
public key is public (the node publishes it in handshake_ack), so any member
could wrap a key of their choosing for it and take over the group, locking
every legitimate member out. Storing now requires an operator signature and
_try_activate_gek is removed: nothing arriving over MNP can set a live GEK.
H5 — unbound signing oracle. The node challenged with 32 raw random bytes and
the client signed them blind, so a signature named no operation, subject, node
or time. New meshbay_common/adminop.py defines a length-prefixed,
domain-separated transcript; both sides build it independently and the client
refuses to sign when the announced op/subject do not match its request.
BREAKING: a group admin who does not operate the node can no longer store GEK
bundles on it. Invites must be performed by the node operator.
Adds tests/test_security_regressions.py. Verified against pre-fix source via
git stash. Three pre-existing tests asserted the vulnerable behaviour as a
feature and were inverted: gek auto-activation, and the transport-wide
chat_store in test_daemon.
Tests: 109 node, 132 hub+common.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|