meshbay.git/packages/meshbay-hub/tests/test_downloads.py, branch main

docs: cite MESHBAY_DESIGN.md and a section instead of the merged notes

2026-09-19T00:12:47Z

The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and deleted from the tree on 2026-09-11, but ~230 comments across the three packages still named them — usually written `docs/musicbay.md §3.2`, as though the file were still in docs/. A reader had to know §16 existed to resolve any of them. They now name the section directly. Every mapping comes from §16, the concordance, which already records where each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the Videos app and §6.5 where the subject is derived data, photos -> §9.9, auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 -> §3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys -> §4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section. Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`) are retargeted too: those collide with real section numbers in the design document, so leaving them would have been worse than the named form. Four cases the concordance does not cover, each decided rather than guessed: Sub-item references into documents that no longer exist — mediacenter's `§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and §9.2 do not reproduce. The module-level citation stays; the sub-item pointer is dropped. The V-findings keep their labels but lose the dead `§10.1/` prefix. §13.8 lists V1–V13 as per-application open items, which is not what the labels mean in these comments, so pointing them at §13.8 would have been a false citation. `apps.md §5`'s virtualization requirement has no counterpart anywhere in the design document. The requirement is stated in the comment itself, so the citation is dropped rather than aimed at a section that does not say it. Comments that attributed a *sentence* to an old note — musicbay's "several thousand files" example, its "what got measured" note, its measured ~11%/~26% cover-art figures, the "original no root, whole shared tree" call — state the fact without attribution now. §9.8 does not contain those sentences and citing it for them would have been wrong. CLAUDE.md's "a reference to a document that no longer exists" row now says the concordance is for git history and out-of-tree material; the code cites sections directly. Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox leg of test_sticky_header.py's browser harness, which is broken at the browser level on this machine — headless Firefox (snap) dies with `[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders nothing, and the probe exits `{"error": "no measurement"}` after its full 90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing here can affect it: every changed line in style.css is inside a comment. Also checked: ast.parse on every changed .py, `node --check` on every changed .js, the /* */ balance in style.css, and that no changed line exceeds the width its file already used. Co-Authored-By: Claude Opus 5

fix(spa): wake the download worker before handing it a stream

2026-09-09T10:43:08Z

Reported from Chrome: a download started while an upload was running took thirty seconds to begin, every time. The console named it exactly — /_mbdl/mtty5btz-sbmgdegx 404 () [MeshBay] the worker did not answer the download within 15s (attempt 1) A 404 from the hub means the request reached the *network*: the worker looked, found no entry for that id and let it through. So the worker was alive and controlling the page, and the message handing it the stream had simply never been processed. `pending` lives in the worker's memory, and a worker with nothing to do is terminated within tens of seconds. A WebRTC upload gives it no events at all, so minutes of uploading leave it dead; the stream posted to it is lost, silently, and the iframe then wakes it with nothing to find. `mbdl-ping` already existed for this exact reason -- sent every ten seconds *while* writing, because a streaming response does not count as activity. Nothing sent one before *starting*. So a download now wakes the worker and waits for the pong, and `sw.js` answers `mbdl-ready` once it has actually stored the entry, which the page waits for before navigating: confirmed rather than assumed. A worker that predates the ack sends nothing and the page navigates anyway, which is what it did before. This cause was measured and wrongly dismissed hours earlier, with an idle probe that made the worker work between its own attempts -- it never actually slept. A measurement that does not reproduce the conditions refutes nothing. The harness now models a worker that is asleep: a ping wakes it, and anything else posted while it sleeps is lost, which is what made the failure silent. `test_backpressure_is_real` read the first `worker.postMessage` in the function to check that the readable half is transferred rather than copied. The wake-up put a ping in front of it, so it began inspecting a call that carries only a port -- and kept passing. It now checks every post, each bounded by its own call, since the keep-alive ping transfers nothing at all. Same shape as the upload-seal contract this morning: a guard that reads "the first" stops guarding the moment something is inserted before it. Hub suite 851 passed. Both new cases checked against the unfixed source. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

feat(spa): pause and resume a download, in session

2026-09-09T08:48:49Z

Stage 7a of ~/next/improve-downloads.md: pausing within a session, on the targets that can actually do it. Resuming across a reload is 7b. A paused transfer holds **nothing**. Its slot goes back to the node the moment it stops and resuming rejoins the queue at the tail, because anything else lets one member close a node by pausing four downloads and going to lunch. So the lease is taken inside the run loop rather than before it, and pause is refused outright for a transfer that could not ask for another one. Resuming is exact rather than approximate: the pipeline stops between two chunks and never inside one, so what is on disk is always a whole number of chunks and `fromChunk` is a verified position. The failure mode being avoided is a file that looks complete and is quietly corrupt. The target has to survive it, so a pause no longer reaches the `abort()` that a failure does -- that would delete Electron's `.part` or the file just created in the granted folder, leaving nothing to continue. And the in-memory fallback keeps its accumulated chunks rather than starting a second array. The button is drawn only where the target says it can. A service-worker stream says no, in its own code and for its own reasons: the browser is already writing an HTTP response into its own download folder, not feeding it stalls that download where we cannot see or resume it, and an idle worker is terminated within seconds. Firefox and Safari therefore keep cancel and get no pause, which is the decision recorded in §6.5. Cancelling a paused transfer ends it. A paused run is parked on a promise; without waking it the row said "cancelled" over work that had not stopped and a target that was still open. Six cases, each checked against the unfixed source. Hub suite 842 passed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

fix(spa): the target queue must not be able to freeze a batch

2026-09-09T00:09:52Z

Four downloads on Firefox all sat at "preparing", with the node journal showing `d=0/8(q0) u=0/8(q0)` — not one transfer opened, so nothing had got past the client's target opening. Serialising those openings was new in d6c4808, and on Firefox it regressed what had always worked: four openings that ran at the same time began waiting on the slowest. `_targetQueue` is module-level and never reset, so an opening that never settles leaves the page unable to start any download again until it is reloaded. Two bounds, both narrowings of the queue rather than of any capability: Only an opening that could actually show a dialog joins it. Firefox and Safari have no `showSaveFilePicker` at all, so nothing there can race anything and the queue bought nothing while costing everything; they now bypass it entirely, which restores the previous behaviour by construction rather than by tuning. And no opening waits behind another for longer than TARGET_QUEUE_BUDGET_MS (90s) — generous enough never to cut in front of a real dialog, finite because the alternative is a download panel that only a reload can fix. Releasing early is safe: whatever is ahead is still the only unbatched opening, so the released one takes the streamed path and opens no second dialog. Measured on Firefox 154 against the deployed hub before writing any of this: `register` and `ready` return instantly, the page is controlled, and four serialised openings are served in 5-18 ms. The streamed path was never the delay; the queue was. Both new cases were checked against the unfixed source: without the bypass the peak concurrency is 1 instead of 4, and without the budget the stuck-opening case hangs. Hub suite 826 passed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

fix(spa): one save dialog per batch, not one per file

2026-09-08T23:54:10Z

Selecting four files on Chrome produced a Save As dialog for the first, then one for the second only after that file had finished, while the last two timed out; on a later attempt the three remaining transfers appeared frozen. Two things were going on. Opening the target inside `prepare` had removed the accidental serialisation that `for (…) await downloadFile(e)` used to provide, so `_openTargetInTurn` now queues the openings — but a queue whose head is an unanswered dialog is a head-of-line block, which is what the "freeze" was. The code already recovered from a picker with no gesture behind it by streaming instead, on the `SecurityError` Chrome throws. That branch was never reached: Chrome does not throw, it shows the dialog anyway and waits for a human. So anything that has to wait its turn is now marked `batched`, and a batched opening prefers the streamed path whatever the download mode says. The first file of a batch — the one actually holding the gesture — still gets its dialog, so the preference is honoured where it can be. For the rest there is no gesture left to spend and nothing is lost by streaming: the file still lands on disk, in the browser's own download folder. Only the choice of folder goes, and it was not on offer. If the worker does not answer, a batched download falls back to the dialog rather than failing. Also logs which path led to a dialog. A dialog is the one outcome nobody can diagnose after the fact — it looks the same whether it was asked for or fallen back to — and the report this fixes needed three test cycles to narrow. The two harnesses that lift `_openDownloadTarget` as text now route console.info to stderr, since they parse stdout as JSON. Measured against the deployed hub in Chrome 152: the streamed path serves the hidden iframe in 2-3 ms on a normal load, after a hard reload (via the `mbdl-claim` recovery already in `_claimController`), and twice in the same document. Hub suite 824 passed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

fix(hub): keep the download worker alive, and never hang on a dead sink

2026-09-08T20:53:53Z

A download froze part-way through, on Firefox, with an empty console and a node that stayed perfectly healthy. Three separate measurements cleared the node (615 MB pulled whole over MNP), the transport (three files interleaved on one connection, 1.5 GB, all whole) and the service worker (three concurrent 150 MB streams in real Firefox 154) — because none of them was wrong. The empty console was the evidence. `_sendAndWait` logs every timeout, so no chunk request had expired: the client was not waiting on the node. Of the three awaits left on that path only one was unbounded. **A service worker with no event for about thirty seconds is terminated**, and `respondWith(new Response(stream))` does not extend its life while the response is still being written. The reader vanished mid-file and `writable.write()` then never resolved and never rejected — no error, no log, no failed transfer, just a progress bar that stops. The first stress probe wrote 450 MB in two seconds and passed: fast enough to hide it entirely. Measured in Firefox 154, writing 1 MB every 2 s: without the ping it stalled at 17 MB after 59 s; with it, 40 MB in 80 s, complete. - the page pings the worker every 10 s while it writes, and the worker answers. Receiving a message is an event, and an event resets the timer; - that interval stops itself after two minutes with no write. A target can be opened and never written to — a transfer cancelled while it waits for a slot never runs, so nothing calls close() or abort() — and a timer nobody clears pings for the life of the page. It also kept the Node test process alive for ever, which is the same defect wearing a louder symptom; - `writable.write()` is bounded at 60 s and fails with a message naming the chunk. That does not fix whatever stopped a sink; it turns an unexplainable freeze into a failed transfer that says so, which is the difference between a mystery and a bug report. Also: `Content-Disposition` lost a filename to a single apostrophe. `encodeURIComponent` leaves `'` alone and `'` is the delimiter in RFC 5987's `filename*=''`, so the header became unparseable and Firefox named the file after the URL — 449 MB of film arrived complete as "mtsshk9w-ohqty535". `(`, `)` and `*` get the same treatment, and a plain ASCII `filename=` rides alongside. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

fix(hub): make the streamed download path reliable, and clean up after an abort

2026-09-08T11:20:29Z

On Firefox and Safari the service worker is the only unbounded way to write a download to disk: neither has the File System Access API, and OPFS is not a substitute — measured on Firefox 154, its quota is exactly 10% of the volume's size (389,233,459 bytes of a 3,892,334,592-byte volume, refused to the byte), which a film exceeds. So when this path declines, a large download has nowhere left to go, which makes its reliability a correctness property. Four ways it declined, all of them avoidable: - it was registered inside the first click on Download, so that click paid install, activate and claim while somebody watched a button do nothing; - `_swReady` cached a null for the life of the page. One slow first click left the tab unable to stream anything again, curable only by a reload nobody knew to do. Only a successful controller is remembered now; - control was waited for with a 3 s cap. It is 15 s, and a page that is active but not controlled asks the worker to claim again (`mbdl-claim`) instead of declaring the path unavailable; - a missed navigation gave up at once. It gets a second attempt with a fresh id and iframe, the failed one torn down completely first. Also closes a MessagePort leaked per download, and gives the reason a name (`lastStreamFailure`) so a refusal can say what happened. The timeouts became parameters: the defaults are the production values, no caller passes any, and the tests do not spend a minute waiting. `openTarget` gets an unrelated but adjacent fix, in the same file: it creates the destination with `getFileHandle({create: true})`, so an empty file exists before the first byte, and `abort()` leaves the target untouched — every cancelled download left a 0-byte file behind, and since `freeName` avoids collisions, three cancels left film.mkv, film (2).mkv and film (3).mkv, all empty. Its `abort()` now removes the entry. Safe here and only here, because `freeName` guarantees the name was not taken: the `showSaveFilePicker` path must not do the same, where the person may have picked an existing file whose contents `abort()` correctly preserves. Verified by hand in Chrome. test_streamed_download_reliability.py runs the real module under Node against a stubbed browser — it fails if the null is cached again. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

feat: add Photos group app

2026-08-25T09:46:17Z

A new group application (docs/apps.md's plug-in mechanism), following the plan in docs/photos.md. Unlike Videos/Music: several photo roots per group instead of one (photo_roots is a set, one signed op replaces it whole), a single album-grid view with no third-party matching step, and per-photo info read from the file's own EXIF at index time — no metadata service, no credential, no outbound network call at all. Protocol (meshbay-common, MNP 0.10 -> 0.11, additive): `taken_at`/`camera` on IndexEntry; `photo_roots`/`photo_roots_ack`; `OP_PHOTO_ROOTS`. Node: roster.py stores photo_roots as a group_settings entry (JSON list, same shape as enabled_apps); ops.py/webrtc_server.py validate and sign the whole set in one op, same pattern as apps_enabled; a new PhotoEnricher (indexer/enrich_photo.py) runs Pillow in its own small bounded pool, separate from the video/audio pools, producing a resized thumbnail plus the two EXIF fields — never GPS, checked by a grep-based regression test. Client: photos-app.js — one album card per directory containing images, a per-album photo grid, and a lightbox with next/previous (keyboard and buttons), zoom in/out/fit/100% starting from the actual on-screen fit percentage, and a "zip this album" button reusing files-app.js's own zip mechanism (lifted into file-utils.js's downloadDirectory so both call the same implementation). group-settings.js gets an add/remove multi-root picker, distinct from Videos/Music's single-value one. Bugs found and fixed before this ever shipped, worth keeping the story of: - enrich_photo.py read width/height from the raw image *before* applying EXIF orientation correction, and read DateTimeOriginal off the plain 0th-IFD Exif object — a real camera stores it in the Exif sub-IFD, which Pillow only exposes via get_ifd(Exif). A flat, hand-built EXIF dict round-trips through Pillow either way, which is exactly what would have hidden both bugs; the regression test builds EXIF with piexif instead, matching what real hardware produces. - photos-app.js's album grouping stripped a trailing path segment from entry.path under the assumption it still carried a filename — it doesn't (files-app.js's own convention: e.path is already the containing directory), so every album collapsed one level into its parent. Found live against a real multi-folder library. - transport.js's ADMIN_OP_TYPES allowlist (already the fix for an identical bug on video_root/apps_enabled, see 4783d81) was missing photo_roots: its admin_challenge matched no pending request and was silently dropped, so saving a photo root just timed out after 30s with no error. - daemon.py pruned a thumbnail when its file left the index (root removed or reconfigured) but never forgot the content hash was "already attempted" — the same bytes reappearing under a renamed/relocated root (an operator's real workflow) were then permanently skipped, forever, with nothing to indicate why. Discarding the attempt alongside the cache entry on prune is what makes pruning actually reversible. - packages/meshbay-client's app:// protocol handler served every file with no Cache-Control header, so Chromium was free to serve a stale cached copy indefinitely — none of several `npm run sync-ui` + reload cycles during development actually picked up the new code until the renderer's disk cache was cleared by hand. Now sends Cache-Control: no-store. - the lightbox's zoomed image used flex centering (align-items/ justify-content: center) combined with overflow: auto — a well-known trap where the browser centers overflowing content by shifting it, and the leading half of that overflow (here, the top of a zoomed photo) sits outside what the scrollport can actually reach. Reported live as "unusable". Fixed by switching to top/left alignment once zoomed. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01TiZG4AuSnxHohQMpwTHTyL

feat(hub): split the group UI into a pluggable "applications" architecture

2026-08-23T13:15:35Z

GroupPage's 6620-line app.js carried Chat and Files wedged in directly, with no way to add another group-level app without touching the shell itself. It is now app.js (routing, non-group pages) plus nine focused files — apps.js (the registry), chat-app.js, files-app.js, video-player.js, group-page.js (the shell), group-settings.js, hub-client.js, icon.js and file-utils.js — with docs/apps.md as the checklist for adding one (Videos/Music/Photos are sketched there, not built). Node side gained the matching enablement mechanism, mirroring member_upload exactly: a roster setting, a signed apps_enabled op enforced by _has_admin_authority, exposed in the handshake ack. Operators toggle applications per group from Settings, which also gained a small reorder: Invite, Pairing, Applications, Shared directories, Uploads, danger zone, Your devices, Members. Two bugs surfaced during the split, both missing an import across the new file boundary and invisible to node --check or a module-load probe since they only throw when the code path actually runs: - group-page.js called onRefreshAuth on a stale-token handshake rejection, but app.js never imported refreshAccessToken from hub-client.js — so a brand new member (including a group's own creator) hit "Not a member of this group" and the retry silently failed, throwing before it could refresh the token. - chat-app.js called getLocale() for message timestamps without importing it from i18n.js. Opening Chat on a group with real messages threw mid- render; uncaught, that appears to wedge Preact's render scheduler, so every button on the page stopped responding until reload. Caught the second class of bug with a proper no-undef audit across all split files (a temporarily installed ESLint 9, since the system one is too old to parse this codebase's syntax) rather than trusting grep. 827 tests pass; 6 new ones cover the apps_enabled policy. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_016SF6RKNBKg9qejmoMJ9ybA

feat(hub): chat, presence, a Profile page, and downloads that do not freeze

2026-08-16T13:29:11Z

Chat opens on the newest hundred messages, loads fifty older on demand with the reading position anchored — the distance from the *bottom*, since everything above the viewport just grew — and follows new messages only when the reader was already at the end. Day separators, sender grouping, an unread marker, and a jump-to-latest pill. Messages are keyed by id: index keys plus prepending makes Preact reuse the wrong bubbles. A presence dot per group in the sidebar, three states, each backed by something: the hub's registry, or a connection this browser made or failed to make. Never colour alone — red and green are the pair colour-blind readers cannot separate — so each dot carries a title and an aria-label. Profile is split out of Settings: identity, node link, pinned node identities and account deletion. Mixing them put an irreversible button two scrolls under a theme picker. The create-group page loses its centred 520 px card, which left 190 px of margin either side, and its two button panels become a radio group — a button conveys no chosen state to a screen reader, and side by side they read as two independent actions rather than one either/or. The Files toolbar shows its actions as icon buttons the moment Select is on, disabled when they do not apply rather than appearing and vanishing. On a phone the right-hand group could not wrap and ran 130 px off the screen. Streamed downloads no longer freeze after one chunk. `registration.active` says a worker exists, not that this page is controlled by it — and an uncontrolled page's requests never reach its fetch handler, so the worker took the stream and was never asked for it, leaving `writer.write()` waiting on backpressure that would never lift. The page now requires control and the worker confirms it actually served the request before the sink is trusted. Fixed on the way: `setActionsOpen` outlived the state it belonged to and threw on every Files action; the chat scrollbar stopped short of the bottom; the owner's row sat lower than the rest; About showed a version hardcoded two releases ago. Co-Authored-By: Claude Opus 5