diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-09-16 17:24:10 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-09-16 17:24:10 +0200 |
| commit | 3d4080e9a1b79e99428a80391e224ea73cf4274d (patch) | |
| tree | 5c44081ca087d9442bc694492cf8d85595ba2584 | |
| parent | 1c6bc8d98bcb4674e4a874d37a61af7732a9402d (diff) | |
| download | meshbay-3d4080e9a1b79e99428a80391e224ea73cf4274d.tar.gz | |
playlists: drive the blob ops over a real DataChannel
The six MNP 3.1 ops were the only new ones never crossing a channel in a
test. Two cases on the existing aiortc harness: a round trip read back on
a second connection, and a 256 KB body. Found that the 1 MB body cap is
unreachable from a browser — docs/playlists.md §15.3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| -rw-r--r-- | docs/MESHBAY_NODE_PROTOCOL.md | 6 | ||||
| -rw-r--r-- | docs/playlists.md | 41 | ||||
| -rw-r--r-- | packages/meshbay-node/tests/test_webrtc_transport.py | 179 |
3 files changed, 221 insertions, 5 deletions
diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index 47c3636..2682820 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -654,6 +654,12 @@ already holds a bundle for, so this adds **no new trust boundary**. * **Caps refuse, never truncate**: 64 KB for the manifest, 1 MB for one body, 8 MB per account per node, each with a stated reason. A truncating cap loses tracks silently, which is the failure the design exists to prevent. + * The 1 MB body cap is **not the binding one**. A browser cannot send a frame + above the negotiated `max-message-size`, which aiortc fixes at **65 536**, + so a client tops out near 64 KB per write however high this cap is set — + the same constraint that keeps uploads chunking at 48 KB. Reads are not + limited that way: the node answers with a whole body of up to 1 MB. See + `docs/playlists.md` §15.3. * A `fetch` for a kind never written answers `null`, not an error: that is the ordinary state of a node the reader has just joined. * The node keeps **no history**. The client is the authority on which revision diff --git a/docs/playlists.md b/docs/playlists.md index d8eda18..488a2de 100644 --- a/docs/playlists.md +++ b/docs/playlists.md @@ -1184,17 +1184,26 @@ level of the named failure rather than the count. | 7 | The store: a stale node cannot lower the merged state; a deletion is not resurrected; an edit made elsewhere arrives with its body; a sync with nothing to do writes nothing; a deleted body is reclaimed; what leaves the browser is sealed | `test_playlist_store.py`, in Chrome against a stubbed node | | 8 | The menus: the toolbar's five verbs, naming a playlist in a field, Favourites offered first before it exists, adding an album from its cover, loading into the queue, the two-level tracklist, deleting after a confirm | `test_playlist_ui.py`, in Chrome | | 9 | An unreachable group is skipped **whole** — six dead tracks, one more than the bound | `test_music_queue.py` | +| 10 | The six ops across a **live DataChannel**: sealed bytes out and the same bytes back on a *second* connection, `req_id` on the reply, a listing that carries revisions and no payloads, an absence that is not an error, a delete that reclaims | `test_webrtc_transport.py` — real aiortc, real handshake, real frames | ### 15.2 Not checked, and what that costs These need things a test suite here does not have. They are listed so nobody reads §15.1 as more than it is. -1. **A round trip through a real node daemon.** The MNP handlers are driven - directly against a real `BundleStore`, and the store is checked across a - close and reopen — but not across a running daemon, a real DataChannel and a - real handshake. The likeliest thing to be wrong there is msgpack `bin` - framing for a payload of this size, which nothing here exercises. +1. ~~A round trip through a real node daemon.~~ **Closed**, and more cheaply + than this entry implied: `test_webrtc_transport.py` already stands up a real + `WebRTCTransport`, a real aiortc `RTCPeerConnection` and a real handshake for + two dozen other cases, and `test_keypair_bundle_store_and_fetch` is the same + shape as what was missing. The six `USER_BLOB_*` ops were simply the only new + ones in 3.1 that never got it. + + Measuring corrected this entry's own guess. A 256 KB blob arrives as **one** + application message of 262 213 bytes — SCTP fragments and reassembles beneath + us, so the four-byte length prefix is not doing the work credited to it here. + What the test pins is that nothing truncates or re-encodes a `bin` value of + that size. It also found the ceiling in §15.3, which is the part that + matters. 2. **Two browsers, one account, one node, converging** — and the same with the node stopped between the two edits. The merge is proved; the *plumbing* around it is proved against a stub that always behaves. Confirmed working in @@ -1211,6 +1220,28 @@ reads §15.1 as more than it is. ### 15.3 Still open +- **A body larger than about 64 KB cannot leave a browser.** aiortc advertises + `a=max-message-size:65536` in its SDP (`RTCSctpTransport.getCapabilities`, + hardcoded — checked by generating one), and a browser refuses a `send()` above + the negotiated value. The SPA builds one frame per message with no size check + (`transport.js:3016`), so a sealed body over that limit throws instead of + syncing. The codebase already lives under this ceiling, and asymmetrically: + `UPLOAD_CHUNK_SIZE` is 48 KB going up while `CHUNK_SIZE` is 1 MB coming down, + which is the same constraint seen from both ends. + + So `USER_BLOB_BODY_MAX = 1 MB` is a cap the client can never reach. The real + limit is about fifteen 4 KB pages — near **225 tracks in one playlist**, at the + ~270 bytes a track measures once compressed. Past that the push fails: + visibly, since the sync result is reported now, but in the browser's own + words. Reading is unaffected — a node answers with a whole body, and 256 KB is + measured coming back. + + Three ways out, none taken: cap the client with a sentence a reader can act on + (small, safe, keeps the limit); split a long body over several blob kinds (no + protocol change, but the kind namespace must then carry part numbers); or raise + what the node advertises (aiortc moved 256 KB both ways here despite + advertising 64 KB — one line, and a protocol change that belongs in its own + work, with uploads to re-examine alongside it). - **O7 is now decided, not deferred**: there is no `/playlists` page, and the Search page needed no change beyond passing `userId` — §11's requirement is met by `MusicApp` being mounted in both places. diff --git a/packages/meshbay-node/tests/test_webrtc_transport.py b/packages/meshbay-node/tests/test_webrtc_transport.py index 284c461..3010fe9 100644 --- a/packages/meshbay-node/tests/test_webrtc_transport.py +++ b/packages/meshbay-node/tests/test_webrtc_transport.py @@ -9,6 +9,7 @@ Uses local loopback (no STUN/ICE needed for localhost). import asyncio import base64 +from contextlib import asynccontextmanager import hashlib import hmac import os @@ -1770,3 +1771,181 @@ async def test_gek_bundle_fetch_not_found(sk_node, sk_hub, gek, shared_dir, tmp_ await bundle_store.close() await browser_pc.close() await transport.close_all() + + +# ── Per-account blobs, MNP 3.1 (docs/playlists.md §15.2) ───────────────────── +# +# `test_user_blob_mnp.py` drives these six handlers directly, which proves what +# they decide but nothing about how their payloads travel: it never builds a +# frame and never crosses a channel. A playlist body is the largest `bin` value +# this protocol carries after a file chunk, and the failure it would hit is +# silent — a playlist that does not come back reports no error, it is simply +# absent, which is exactly what was seen on a phone once already. + + +@asynccontextmanager +async def _user_blob_node(sk_node, sk_hub, gek, shared_dir, tmp_path): + """ + A node with a bundle store, and peers, torn down whatever happens. + + The `finally` is not tidiness. `BundleStore` runs an aiosqlite thread, and a + test that fails before closing it leaves that thread alive — the process + then hangs in `threading._shutdown`, *after* pytest has printed the failure + and the summary. Found by breaking the node on purpose to check these tests + catch it: they did, in 0.66s, and then the run never ended. A red suite is a + result; a stuck one is an outage. + """ + indexer = DirectoryIndexer(roots=one_root(shared_dir), group_id="g", + sk_node=sk_node, gek=gek) + await indexer.initial_scan() + + bundle_store = BundleStore(db_path=tmp_path / "bundles.db") + await bundle_store.open() + + transport = WebRTCTransport( + sk_node=sk_node, hub_pk_pem=_hub_pk_pem(sk_hub), gek=gek, + roots=one_root(shared_dir), index=indexer.index, + stun_servers=[], + ) + transport._ctx["bundle_store"] = bundle_store + + peers: list = [] + try: + yield transport, peers + finally: + for pc in peers: + await pc.close() + await bundle_store.close() + await transport.close_all() + + +@pytest.mark.asyncio +async def test_user_blobs_round_trip_over_a_live_datachannel( + sk_node, sk_hub, gek, shared_dir, tmp_path): + """ + Sealed bytes out, the same bytes back, across two connections. + + The blobs are `os.urandom`, deliberately: sealed output is incompressible + and uses every byte value, so anything that treats this as text — a UTF-8 + decode, msgpack `str` instead of `bin` — corrupts it. Bytes the node could + have round-tripped by accident would prove nothing. + + The second connection is the case this feature exists for: a device that was + not the one that wrote. + """ + async with _user_blob_node(sk_node, sk_hub, gek, shared_dir, tmp_path) as ( + transport, peers): + manifest = os.urandom(512) + # Bodies are padded to a multiple of 4 KB before sealing, so this is the + # smallest one a real client ever sends. + body = os.urandom(4096) + + pc1, ch1, q1 = await _setup_peer(transport, sk_hub, gek, "peer-blob-write") + peers.append(pc1) + + ch1.send(_pack({"type": MNP.USER_BLOB_STORE, "v": MNP_VERSION, + "kind": "playlists", "rev": 7, "blob_enc": manifest})) + ack = await asyncio.wait_for(q1.get(), timeout=5.0) + assert ack["type"] == "ack" + assert ack["detail"] == "user_blob_stored" + + ch1.send(_pack({"type": MNP.USER_BLOB_STORE, "v": MNP_VERSION, + "kind": "playlist:favorites", "rev": 3, "blob_enc": body})) + ack = await asyncio.wait_for(q1.get(), timeout=5.0) + assert ack["detail"] == "user_blob_stored" + + await pc1.close() + + # A second device, same account, which has never seen either blob. + pc2, ch2, q2 = await _setup_peer(transport, sk_hub, gek, "peer-blob-read") + peers.append(pc2) + + # What is here, and at what revision. This is the call that lets a fresh + # device discover kinds it cannot guess — they carry client-made ids — + # and it must not carry payloads. + ch2.send(_pack({"type": MNP.USER_BLOB_LIST, "v": MNP_VERSION})) + listing = await asyncio.wait_for(q2.get(), timeout=5.0) + assert listing["type"] == MNP.USER_BLOB_LIST_RESP + assert listing["blobs"] == [{"kind": "playlist:favorites", "rev": 3}, + {"kind": "playlists", "rev": 7}] + + # `req_id` is what lets the client match a reply to its request rather + # than to whatever arrives next; over a channel carrying several + # requests at once, arrival order is not an answer. + ch2.send(_pack({"type": MNP.USER_BLOB_FETCH, "v": MNP_VERSION, + "kind": "playlists", "req_id": "r-91"})) + resp = await asyncio.wait_for(q2.get(), timeout=5.0) + assert resp["type"] == MNP.USER_BLOB_RESP + assert resp.get("req_id") == "r-91" + assert resp["kind"] == "playlists" + assert resp["rev"] == 7 + assert isinstance(resp["blob_enc"], bytes), "came back as something else than bin" + assert resp["blob_enc"] == manifest + + ch2.send(_pack({"type": MNP.USER_BLOB_FETCH, "v": MNP_VERSION, + "kind": "playlist:favorites"})) + resp = await asyncio.wait_for(q2.get(), timeout=5.0) + assert resp["rev"] == 3 + assert resp["blob_enc"] == body + + # A kind never written is an absence, not an error: the ordinary state + # of a fresh node. + ch2.send(_pack({"type": MNP.USER_BLOB_FETCH, "v": MNP_VERSION, + "kind": "playlist:never-written"})) + resp = await asyncio.wait_for(q2.get(), timeout=5.0) + assert resp["rev"] is None + assert resp["blob_enc"] is None + + # And a delete reclaims it, which is how a tombstoned playlist stops + # costing an account its quota. + ch2.send(_pack({"type": MNP.USER_BLOB_DELETE, "v": MNP_VERSION, + "kind": "playlist:favorites"})) + ack = await asyncio.wait_for(q2.get(), timeout=5.0) + assert ack["detail"] == "user_blob_deleted" + + ch2.send(_pack({"type": MNP.USER_BLOB_FETCH, "v": MNP_VERSION, + "kind": "playlist:favorites"})) + resp = await asyncio.wait_for(q2.get(), timeout=5.0) + assert resp["blob_enc"] is None + + +@pytest.mark.asyncio +async def test_a_large_user_blob_round_trips_whole( + sk_node, sk_hub, gek, shared_dir, tmp_path): + """ + A quarter-megabyte blob, out and back, byte for byte. + + This is the risk §15.2 named, and measuring it corrected how it was + described: a 256 KB blob arrives as **one** application message of 262213 + bytes, not as fragments the four-byte length prefix reassembles. SCTP + fragments it and puts it back together underneath. What this pins is that + nothing in the node's own framing truncates or re-encodes a `bin` value of + that size. + + Note which direction is proved. A *browser* cannot send a frame this large + to this node: aiortc advertises `a=max-message-size:65536`, so Chrome + refuses anything above it — which is why uploads chunk at 48 KB + (`transport.js:41`). The store below is aiortc talking to aiortc and says + nothing about that ceiling. The fetch is the direction that matters here: a + node answers with a whole body, and a long playlist is the largest one. + """ + async with _user_blob_node(sk_node, sk_hub, gek, shared_dir, tmp_path) as ( + transport, peers): + # 256 KB: about nine hundred tracks at the 270 bytes a sealed track + # measures. + body = os.urandom(256 * 1024) + + pc, ch, q = await _setup_peer(transport, sk_hub, gek, "peer-blob-big") + peers.append(pc) + + ch.send(_pack({"type": MNP.USER_BLOB_STORE, "v": MNP_VERSION, + "kind": "playlist:long", "rev": 1, "blob_enc": body})) + ack = await asyncio.wait_for(q.get(), timeout=10.0) + assert ack["detail"] == "user_blob_stored", ack + + ch.send(_pack({"type": MNP.USER_BLOB_FETCH, "v": MNP_VERSION, + "kind": "playlist:long"})) + resp = await asyncio.wait_for(q.get(), timeout=10.0) + assert resp["type"] == MNP.USER_BLOB_RESP + assert len(resp["blob_enc"]) == len(body), "truncated" + assert resp["blob_enc"] == body |