meshbay.git/packages/meshbay-node/tests/test_disk_io_off_loop.py, branch 0.20

fix(node): index, serve and write files past Windows' MAX_PATH

2026-10-14T09:17:38Z

With the system's default setting (LongPathsEnabled off) every call given a plain path of 260 characters or more refuses it, so a deep library was partly invisible: the walk could not list such a directory and its files never reached the members. The node now hands each call that touches a group's disk the extended form (paths.long_path), and keeps the plain path in the index, the hash cache and the log. A directory is listed under its long form and its entries named from the plain one (indexer.walk_files, wire._walk_dirs). Covered: the walk, stats and hashing, reconcile and the watcher, the folder list members see, serving over WebRTC and QUIC, deletion, uploads and their recovery, folder creation and removal, audio tags and the cover beside a track, photos, and a show's episode numbering. ffmpeg extends a long path itself and keeps the plain one. long_path now normalises before prefixing (the extended form skips the system's normalisation) and starts at 248 characters, where creating a directory is already refused. test_long_paths.py runs with the setting off: 15 of its 17 tests fail without the change. Checked by hand in the installed app with files at 312 to 380 characters. Co-Authored-By: Claude Opus 5.5

perf(node): list a group's directories off the event loop

2026-10-07T19:31:10Z

Every full index walked all roots on the loop, and a node with several large roots stopped answering for seconds. Walk directories only, on the roots' disk thread; index_sync is spawned and still answers on failure. Co-Authored-By: Claude Opus 5.5

test: make both suites pass on Windows

2026-09-27T20:21:26Z

Most of these failed on Windows for reasons that had nothing to do with the code under test, which is how real Windows defects hid among them: - Read and write files as UTF-8, and talk to Node in UTF-8. read_text(), write_text() and subprocess text=True use the locale codepage, cp1252 on Windows: "é", "—" and "→" arrived as "?" or crashed, some sixty tests. Calls to PowerShell and schtasks are left alone -- they answer in the console codepage. - Import ESM harness modules by file URL (as_uri): a raw "C:\..." path is not a module specifier. - test_cli_golden: mask the tmp path in its JSON-escaped form, spell it the POSIX way, record on Linux, mask the protocol version (the recording had failed everywhere since the MNP 4.0 bump) and argparse's version-dependent quoting; point USERPROFILE at the tmp home, or `member invite` and `operator pair` wrote their codes into the developer's profile. - test_disk_io_off_loop: expect what a free loop can reach on the platform's timer, 15.6 ms on Windows, not an assumed 5 ms. - test_root_paths_are_operator_only: expect the OS's spelling of the path. - test_audio_meta_cache: find ffprobe with shutil.which. Node suite on Windows: 1489 passed, none failed. Hub suite: 3 failures left, all older than this change (two SQLite concurrency tests, one transfer resume). Co-Authored-By: Claude Opus 5.5

refactor(node): move uploads out of webrtc_server

2026-09-24T14:45:38Z

UploadMixin in transport/webrtc/upload_handlers.py, with the upload cap; _upload_chunk moved unchanged, _append_chunk joins webrtc/disk.py. Co-Authored-By: Claude Opus 5.5

refactor(node): move file browsing and folder/file ops out of webrtc_server

2026-09-24T14:45:38Z

FilesMixin in transport/webrtc/files.py; the blocking disk helpers join _locate in webrtc/disk.py, and the lease states go to webrtc/limits.py. Co-Authored-By: Claude Opus 5.5

refactor(node): move framing, shared limits and ffmpeg jobs out of webrtc_server

2026-09-24T14:45:37Z

transport/webrtc/channel.py, limits.py and media_tools.py, cut from webrtc_server.py as text; the facade imports them back. Co-Authored-By: Claude Opus 5.5

test: read the WebRTC transport's source as a set of files

2026-09-24T14:45:37Z

Source-reading tests take their text from node_source (node) and node_tree (hub): webrtc_server.py plus anything under transport/webrtc/, so a check for something's absence keeps reading the code it guards if that code moves. test_node_source_scope holds the boundary. Co-Authored-By: Claude Opus 5.5

fix(node): bound what a transcode may produce, and read ffmpeg's output off the loop

2026-09-19T11:24:53Z

Two things about the same three functions, which write to a temp file with ffmpeg and then read it back. **The read was on the event loop.** These files are ffmpeg's own, under `tempfile.mkstemp` on the system disk, so they are not a group root and there is no spun-down platter to serialise against — which is why they go through `asyncio.to_thread` and not `roots.off_disk`. But a whole transcode read inline is still tens of megabytes of blocking read while nothing else in the node is served. The AST guard now covers the module with no exemption at all, and a second check refuses a direct call to the reading helper: passed to `to_thread` it appears in the syntax tree as a name, called inline it appears as a call. **The audio transcode had no size ceiling**, where the subtitle path beside it has had one all along. The bound is the media cache's rather than memory's: `put_thumb` writes one SQLite row, and the store is 512 MB with least-recently-used eviction, sized for thumbnails, posters and short transcodes. At 192 kbit/s a three-hour source is ~260 MB — one row that evicts most of the cache to fit and is evicted again by the next few thumbnails. Not a size this store can hold usefully. 64 MiB, about forty-five minutes: past any track, any single piece, most sets. It takes away nothing that worked. `AUDIO_TRANSCODE_TIMEOUT_SECS` is 120, so a source long enough to reach this was already liable to be killed mid-transcode; what changes is that the refusal now names the limit it met and the size that met it. Serving audio of that length properly means streaming the conversion instead of buffering it, which is a different feature — recorded in §9.8 rather than left as an implied promise. The stat comes before the read, so an oversized result costs a stat rather than the read and the memory behind it. Twelve `test_sticky_header.py[firefox]` setup errors again: Firefox is still open on this machine, and its `[chrome]` half passes. Co-Authored-By: Claude Opus 5

fix(node): creating and removing a folder are syscalls too

2026-09-18T14:13:53Z

The last two handlers on the loop. Both were synchronous, so a member creating a folder on a root that had spun down held the node for the spin-up, exactly as a chunk read did. Where a check and an act belong together they are now one call rather than two awaits, and the single disk thread is what makes that atomic: `_mkdir_if_absent` so two members creating the same name cannot both find nothing there and have the second `mkdir` raise where a refusal was meant, and `_rmdir_if_empty` for the reason the caller already re-tested emptiness — the first test happened before a round trip to the operator's browser, and a file can land in between. Two awaits would reopen that window one size smaller. The guard is now the whole class rather than the calls that were fixed. It walks the module's syntax tree and fails on any filesystem call outside the handful of functions written to be run through `off_disk` — a new handler that stats a root inline would pass every measured test, because those exercise the handlers that exist today. Checked by putting a call back: it names the function and the line. It leaves ffmpeg's own scratch files out, listed rather than silently allowed: they are under `tempfile.mkstemp` on the system disk, not on a group root, so they are not what spins down — but they do read a whole transcode into memory from the loop, and the day that matters it is a different measurement from this one. Co-Authored-By: Claude Opus 5

fix(node): take the availability poll and every upload write off the loop

2026-09-18T13:59:24Z

The rest of AV9's disk half. Serving a file left the loop in the commit before this one; two paths were still on it. **The availability poll.** `RootSet.refresh_availability` stats every root, and eleven call sites reached it from `async def` — the reconcile loop among them, on a timer. On a sleeping disk that is a stall once per tick, and the stat is also what keeps the disk awake, so a node paid spin-up for a library nobody was reading. All eleven now go through `off_disk`, `Root.is_live` included. **The upload write.** `open`/`write`, and the resolve, the stat, the free-name search, the rename and the unlink around it. This one could not simply be awaited: the handler was synchronous, so nothing could come between the `chunk_index != state.next_index` check and the `advance` that answers it, and that is the whole of the chunk-ordering rule. Awaiting the write opens the gap — chunk 1 arriving while chunk 0 is in the disk thread reads a position that has not moved and is refused as out of order, so an upload would fail on a slow disk and nowhere else. Verified, not assumed: without the lock the new ordering test refuses three chunks of four. So the check, the write and the advance are one critical section again, under a lock held **per group**. Not per session: `partial_uploads` lives in the group context so a reconnecting client finds its upload where it left it, which means two sessions of one member share the position of one `.part` file. Arrival order is preserved by construction — the dispatcher creates one task per message as it arrives, tasks start in creation order, and the lock is the first thing each one waits on, so its waiters queue in arrival order too. `_do_file_upload` is a coroutine now, which is why forty-two test call sites gain an `await`. Their outcomes are unchanged, file by file, against the run before the change. `test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`, imported rather than defined there. It now asks for the ones written in the module, which is what its own docstring means; all forty-three operations are still checked. Co-Authored-By: Claude Opus 5