Merge branch 'fix/large-download-paths'

Concurrent-transfer limits, with the queue, the pause and the flag day.

A node now caps how many transfers it runs at once (8 downloads, 8 uploads,
node-wide) and how many one member may run in one group (2 by default,
operator-signed). Beyond that the node answers "queued" and the client waits its
turn, visibly, in the transfers panel — and a slot that frees starts whatever is
next, skipping past a member who is at their own cap rather than letting them
stall everyone behind them.

Browsing is never subject to a slot: not the poster grid, not the covers, not
opening a photo to look at it. That is structural — a transfer is what the
transfers widget shows — and the exemption is bounded rather than open, at two
files in flight per session, because an exemption with no bound is a leaseless
branch under another name.

Transfers can be cancelled, and now paused and resumed. A paused one holds
nothing: its slot goes back at once and resuming rejoins the queue at the tail.
Uploads survive the connection that started them and resume where the node
stopped, asked for inside the seal rather than on a clear message. What they
leave behind when they are abandoned is reaped, which closes a disk leak that
predates this work.

MNP 3.0 makes the lease compulsory and refuses 2.x at the handshake, with the
desktop client checking `client.minimum` before connecting so an un-updated one
says "update" instead of failing every connection in a protocol vocabulary.

Fourteen defects were found on the way, eight of them by a person clicking
Download and pasting a console — none of which 2075 tests could reach. Section
12 of ~/next/improve-downloads.md is that report, including the three this work
introduced itself and the one that turned out to be caused by an instruction to
hard-reload after each deployment.

Node suite 1209 passed, hub suite 866 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HCGdheDLxGReuKHga3BtST

1 files changed, 83 insertions, 9 deletions

diff --git a/packages/meshbay-hub/tests/test_zip_size_limit.py b/packages/meshbay-hub/tests/test_zip_size_limit.py
index 203c10c..9471b8a 100644
--- a/packages/meshbay-hub/tests/test_zip_size_limit.py
+++ b/packages/meshbay-hub/tests/test_zip_size_limit.py
@@ -6,11 +6,16 @@ folder as a zip" button — Files' single folder, Files' multi-folder selection,
 and the Photos album button (docs/photos.md §3) — so the limit is checked
 once, there, and holds for all of them.
 
-Two things are worth pinning. That an oversized folder is refused *before*
+Three things are worth pinning. That an oversized folder is refused *before*
 `_openDownloadTarget`, because a save dialog for an archive that will never be
-written is worse than no dialog at all. And that a folder at exactly the limit
+written is worse than no dialog at all. That a folder at exactly the limit
 still goes through, since an off-by-one here silently costs a whole megabyte
-of allowance and nobody would ever notice.
+of allowance and nobody would ever notice. And that the two limits in play do
+not contradict each other: ZIP_MAX_BYTES (512 MB) bounds the archive, while
+MEMORY_CEILING (100 MB, test_memory_ceiling.py) bounds what may be built in the
+page — so a 400 MB zip is allowed when there is somewhere to stream it and
+refused when the only route left is memory. The `confirm()` that offers the
+build-in-memory path therefore only ever appears below the ceiling.
 """
 
 import json
@@ -30,7 +35,7 @@ pytestmark = pytest.mark.skipif(
 MIB = 1024 * 1024
 
 
-def _run(total_bytes, tmp_path):
+def _run(total_bytes, tmp_path, picker=False):
     """
     Call downloadDirectory over one folder holding `total_bytes`, and report
     what it did: the errors it set, how many times it put a question to the
@@ -44,6 +49,7 @@ def _run(total_bytes, tmp_path):
     (tmp_path / "package.json").write_text('{"type":"module"}')
 
     script = tmp_path / "case.mjs"
+    picker_js = "true" if picker else "false"
     script.write_text(f"""
 const store = new Map();
 globalThis.localStorage = {{
@@ -54,17 +60,53 @@ globalThis.localStorage = {{
 // Node 22 defines `navigator` itself, so it is left alone; `window` is what
 // platform.js reaches for to decide it is not running in the desktop app.
 globalThis.window = globalThis;
-const out = {{ errors: [], started: 0, asked: 0 }};
+// stdout carries the outcome and nothing else, so file-utils' own logging goes
+// to stderr -- where it is still shown when a case fails. It logs before every
+// save dialog, which is exactly what this harness provokes.
+console.info = (...a) => console.error(...a);
+const out = {{ errors: [], started: 0, asked: 0, dropped: 0 }};
 // Reached only once the size check has passed: with no File System Access API
 // under Node, downloadDirectory falls through to its build-in-memory path and
 // asks first. Answering yes is what lets the at-the-limit case get as far as
 // starting a transfer, and `asked` is how the refusal proves it never did.
 globalThis.confirm = () => {{ out.asked += 1; return true; }};
+// With `picker`, the browser can stream to a file the person chooses, which is
+// the only legal route for an archive over MEMORY_CEILING. Never exercised —
+// the stubbed `transfers.start` below does not run the job — it just has to be
+// a target rather than null.
+if ({picker_js}) {{
+  window.showSaveFilePicker = async () => ({{
+    name: 'album.zip',
+    createWritable: async () => ({{ write: async () => {{}}, close: async () => {{}},
+                                   abort: async () => {{}} }}),
+  }});
+}}
 
 const M = await import('{(sandbox / "file-utils.js").as_posix()}');
 
-const transfers = {{ start: () => {{ out.started += 1; }} }};
-const transport = {{ connected: true }};
+// Faithful enough to the real store: it runs `prepare` and honours what it
+// returns. The target is opened there now — the row exists from the click and
+// the slow part happens behind it — so a stub that only counts calls would
+// never reach the size check this file is about.
+const transfers = {{ start: (opts) => {{
+  out.started += 1;
+  if (!opts.prepare) return;
+  Promise.resolve()
+    .then(() => opts.prepare())
+    .then((ready) => {{ if (ready === false) {{ out.started -= 1; out.dropped += 1; }} }})
+    .catch((e) => {{ out.started -= 1; out.errors.push(e.message); }});
+}} }};
+// A transport hands out transfer slots now (transfers.py's leases). The stub
+// grants at once, which is what a node with no caps does: what this file is
+// about is the archive limit, not the queue.
+const transport = {{
+  connected: true,
+  openTransfer: () => ({{
+    tr: 'stub', state: 'granted', ahead: 0,
+    acquire: () => Promise.resolve(),
+    release: () => {{}},
+  }}),
+}};
 // One file, in the folder itself — entriesUnder keys on `path`.
 const entries = [{{ id: 'f1', name: 'big.bin', path: 'album',
                    size: {total_bytes}, added_at: 0 }}];
@@ -73,6 +115,8 @@ await M.downloadDirectory(transfers, transport, null, entries, 'album', {{
   setError: (m) => out.errors.push(m),
 }});
 
+// `prepare` runs on a microtask, so let it.
+await new Promise(r => setTimeout(r, 10));
 out.limit = M.ZIP_MAX_BYTES;
 console.log(JSON.stringify(out));
 """, encoding="utf-8")
@@ -101,7 +145,37 @@ def test_an_oversized_folder_is_refused_before_anything_opens(tmp_path):
 
 
 def test_a_folder_exactly_at_the_limit_still_downloads(tmp_path):
-    """The bound is inclusive: `> ZIP_MAX_BYTES`, not `>=`."""
-    result = _run(512 * MIB, tmp_path)
+    """The bound is inclusive: `> ZIP_MAX_BYTES`, not `>=`.
+
+    Given somewhere to stream to, because 512 MB is five times MEMORY_CEILING
+    and building it in the page is no longer a route this code will take. That
+    is what the next test is about; this one is still only about the off-by-one.
+    """
+    result = _run(512 * MIB, tmp_path, picker=True)
     assert result["errors"] == []
     assert result["started"] == 1
+    assert result["asked"] == 0, "nothing is built in memory when it can stream"
+
+
+def test_a_zip_over_the_memory_ceiling_is_refused_when_nothing_streams(tmp_path):
+    """
+    Between the two limits — larger than the page may hold, smaller than the
+    archive limit — and no way to stream it. Before the ceiling existed this
+    asked "build it in memory?" and, on yes, held 400 MB in the tab.
+
+    The refusal names the memory ceiling, not the zip limit: quoting 512 MB at
+    someone whose folder is under 512 MB would be a message about the wrong
+    rule.
+    """
+    result = _run(400 * MIB, tmp_path)
+    assert result["started"] == 0
+    assert result["asked"] == 0, (
+        "the person must not be offered a build-in-memory path above the ceiling")
+    assert result["errors"] and "group.zip_too_large" not in result["errors"][0]
+
+
+def test_a_small_folder_may_still_be_built_in_memory(tmp_path):
+    """The floor is intact below the ceiling — that is what it is for."""
+    result = _run(4 * MIB, tmp_path)
+    assert result["errors"] == []
+    assert result["asked"] == 1 and result["started"] == 1