diff options
32 files changed, 1773 insertions, 181 deletions
@@ -761,6 +761,59 @@ do. Read them before writing anything that touches the same mechanism. socket buffer gave 19.5 MB/s (`udp_buffer.py`); it needs `net.core.rmem_max` raised on a default machine +- **A transfer capped with a core busy: profile that process before tuning + the network.** The command-line client downloaded at 19 MB/s on loopback. + Its 4 MB receive buffer removed 3245 kernel drops and changed nothing else; + `top` then showed the client, not the node, at 100 %, and cProfile (a + wrapper script as `MESHBAY_CLI_E2E_BIN`, the e2e harness does the rest) put + two thirds of its time in aiortc's `InboundStream.pop_messages`: 153 million + steps for 400 MB, the reassembly walked from its start after every fragment. + Next, a SACK per packet cost the node more than sending. Both fixed in + `sctp_fix.py`: 39 MB/s, the node at 49. No amount of reading our code finds + a quadratic loop in a dependency; one profile does + +- **A ceiling that falls as the loop gets busier is not a CPU ceiling.** + Windows downloads stayed slow with no core saturated: the node at 45 % of a + core, 9 MB/s over eight connections, and an SCTP round trip of 745 ms on + loopback, which is a queue, not a wire. The ProactorEventLoop's datagram + transport sends one datagram per turn of the loop and reads one, each through + an overlapped operation (42 % of a connection process was + `IocpProactor.sendto`), so every other thing the loop does slows every + connection. Linux sends straight from the socket. `proactor_udp.py` does the + same on Windows; a benchmark only showed it once its loop had other work to + do, because an idle loop turns fast enough to hide it. When a transfer is slow, compare the round trip SCTP reports with the + real one before profiling: a large gap is a queue in this process + +- **A default of "off" ships a feature nobody has.** `connection_processes` + defaulted to 1, which starts no process: the multi-core path was measured at + 131 MB/s and reached by no node whose operator had not found the setting, so + on Windows a download over eight lanes landed on the node's one core and ran + at the speed of one connection. Worse, the Node page saved **every** setting + on each Save, so any operator who changed one thing had pinned all the others + at that day's defaults, in the roster and in `node.toml`. It sends only what + changed now, and `init` leaves the line commented. A test node that saved + its settings before this keeps `connection_processes = 1` until it is set + again on the Node page + +- **A reconnect let every message through, not just its handshake.** + `_inReconnectAttempt` opened `_sendAndWait` to all callers so the handshake + would not wait on itself; a chunk retry landing between the new channel's + open and the proof got "Handshake required", which no retry treats as a + dropped connection, and the download died with that text in the transfers + panel. Only the opening messages pass now (`SESSION_OPENING`, held to + `dispatch.py` by `test_reconnect_race.py`), and a replaced channel's late + `close` no longer clobbers the new one. `harness/reconnect_race_harness.mjs` + runs the real `connect()`, `_reconnectLoop` and pipeline against a node that + refuses before the proof — the first harness here that does; the earlier + ones stub `connect`, which is exactly where this lived + +- **A push handler that re-fetches the world.** The interactive CLI answered + every `index_delta` by fetching and re-parsing the whole index, on the event + loop that was receiving the download: 67 full indexes in 80 s while a node + enriched a library (one delta a second). It applies the delta in place + (`Tree.apply_delta`), and only a delta that adds or removes files asks again, + once per burst + ### More of the same, from four rounds of live testing - **A protocol harness cannot test the SPA.** Any second implementation of the diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 4e99994..4db06c4 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1486,9 +1486,12 @@ download land on different processes without the client doing anything. | 8 | 150 MB/s, the receiving browser being the limit | 21 % | Measured on loopback, a 1.5 GB file through the interface in a headless browser, -byte for byte. A process costs about 85 MB of memory. The default is 1, which -starts none and leaves the node exactly as it was; the operator raises it. A -change applies to new connections, and a connection already open stays where it +byte for byte. A process costs about 85 MB of memory. The default is one per +core, up to four: past four a single download gains nothing, and a process is +memory held for the life of the node. One starts none, and leaves every +connection, every lane of a download included, on the node's own core — the +case where Python's per-packet cost is the whole ceiling, and highest on Windows. +A change applies to new connections, and a connection already open stays where it is until it closes, in a process asked to leave or in the node. A process that dies takes its own connections with it, which their clients see as a dropped connection and reopen, and is replaced at the next offer. The loopback socket is @@ -1512,6 +1515,52 @@ machine nothing changes until the operator raises it; the node says so at startu The limit that remains is the node's own receive path, about 5 % of a core per MB/s. +**A receiver on aiortc reassembles in linear time and acknowledges every second +packet.** That is the command-line client in a download and the node in an +upload. aiortc 1.15 walks the reassembly from its start after every DATA chunk, +so a message of n fragments costs n²/2 steps — a 1 MiB `file_chunk` is about 870 +— and it acknowledges every packet, where RFC 4960 §6.2 asks for one SACK every +second packet; each SACK costs the sender a DTLS decryption and a parse on the +core that also sends. `sctp_fix.py` (the node's, and the client's copy) skips the +walk until a last fragment is in, and acknowledges every second packet of data, +at once on a gap, a duplicate or a packet carrying anything else, and within +200 ms otherwise. The client asks for a 4 MB receive buffer, as the node does. +Measured on loopback, 400 MB to the command-line client over one connection: 19 +MB/s with 3245 datagrams dropped by its kernel; none dropped with the buffer, and +still 20, the client at a full core; 28 with the reassembly fixed; 39 with the +acknowledgements, 49 by the node's own count of the transfer. An upload from the +client went from 25 to 29. + +**On Windows a datagram goes straight through the socket, in both directions.** +The node and its connection processes run on the ProactorEventLoop, the one that +can start ffmpeg there, and its datagram transport keeps one overlapped operation +per direction, starting the next from the completion callback of the one before, +on a later turn of the loop. A socket therefore sent one datagram per turn and +received one, and a turn lasts as long as everything else the loop does: the +other connections, the file, the hub. That is a ceiling per connection that +falls as the node gets busier, with cores to spare. With the loop otherwise busy +0.5 ms a turn: 1,928 datagrams sent a second, and 1,954 received with 87 % of +what arrived dropped by the kernel. In a node holding its connections itself, a +browser's download over eight of them ran at 9 MB/s with the node at 45 % of a +core and an SCTP round trip of 745 ms on loopback, and one of the eight sessions +was lost on the way. `proactor_udp.py` sends with a non-blocking `sendto` when +nothing is queued, and after a completed read takes whatever else the socket +holds before posting the next one; refusals and errors take the proactor's own +path. Measured on loopback on a two-core machine, 400 MB, with it: the +command-line client's download 27 to 33 MB/s, its upload 18 to 22 (23 with two +connection processes), the browser's download over eight connections 9 to 14 +before and 17 to 19 after, where the browser takes the other core. + +**A session is open from the end of its handshake, and a reconnect does not +shorten that.** Until connect() returns, a client sends only what the node takes +before the proof — `handshake`, `handshake_response`, `keypair_bundle_fetch`, +`gek_bundle_fetch`, `join_request`, `device_add_request` — and `device_hello`; +anything else waits for the session, or fails as a dropped connection, which a +download waits out. The node answers anything else with "Handshake required", a +refusal and not a dropped connection, so a chunk let through while a reconnect's +handshake runs ends the download the reconnect is for. The close event of a +channel a reconnect has replaced concerns nothing on the new one. + The lease module is free of asyncio and of the transport: it decides, and the caller does the I/O. A queue that reveals itself only through a DataChannel is a queue nobody can prove things about. @@ -2163,7 +2212,7 @@ Node page: | `max_concurrent_downloads` | 8 | node-wide download leases (§5.5) | | `max_concurrent_uploads` | 8 | node-wide upload leases. A separate pool from downloads, because the two cost different things and one queue for both makes each cap meaningless | | `max_upload_gb` | 8 | the largest single upload, §6.4. Fractions are allowed, and it is read per chunk so a change reaches an upload already running | -| `connection_processes` | 1 | how many processes carry the peer connections, §5.5. One is the node itself; above it, that many cores serve transfers. Bounded by the cores of the machine, which the Node page shows beside the field | +| `connection_processes` | one per core, up to 4 | how many processes carry the peer connections, §5.5. One is the node itself; above it, that many cores serve transfers. Bounded by the cores of the machine, which the Node page shows beside the field. `node.toml` as `init` writes it leaves the line commented, so the default reaches every node that has not chosen | | `transcode_incompatible_video` | true | whether browser-incompatible video is transcoded during streaming. Unlike remuxing this costs real CPU per viewer | `hardware_video_encode` (default true) governs whether that transcode may use the diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index 1808598..ab5ee8c 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -852,15 +852,16 @@ and running `meshbay-node reload`: | `max_concurrent_downloads` | 8 | node-wide download slots | | `max_concurrent_uploads` | 8 | node-wide upload slots | | `max_upload_gb` | 8 | the largest single file a member may send you | -| `connection_processes` | 1 | how many CPU cores carry members' connections | +| `connection_processes` | one per core, up to 4 | how many CPU cores carry members' connections | | `transcode_incompatible_video` | true | re-encode video no browser can play | | `hardware_video_encode` | true | use the GPU for that, if one actually works | -`connection_processes` is the one for speed. With 1, everything a node sends -goes through one core, about 35 MB/s shared between everyone downloading at -that moment. Raise it when several people download at once and the machine has -cores to spare: each one adds a core's worth, and about 85 MB of memory. It -applies to connections opened after the change. +`connection_processes` is the one for speed. Each process is a core's worth +of sending, shared between everyone downloading at that moment, and about 85 MB +of memory. With 1, everything a node sends goes through one core: about 35 MB/s +on Linux, much less on Windows. Lower it on a machine short of memory; raise it +above 4 only when many people download at once. It applies to connections +opened after the change. `max_concurrent_streams` is the one to think about: a stream costs a video process for as long as the film lasts, so this counts **viewers**, not diff --git a/packages/meshbay-cli/src/meshbay_cli/client/sctp_fix.py b/packages/meshbay-cli/src/meshbay_cli/client/sctp_fix.py new file mode 100644 index 0000000..62835d3 --- /dev/null +++ b/packages/meshbay-cli/src/meshbay_cli/client/sctp_fix.py @@ -0,0 +1,126 @@ +"""Fixes to aiortc's SCTP stack, applied on import (aiortc 1.15). + +The first two are the node's (`meshbay_node/transport/webrtc/sctp_fix.py`, where +why each is needed is written down); this is a copy because the client does not +depend on the node, and `meshbay-common` does not depend on aiortc (the hub uses +it without). `test_sctp_fix.py` in each package holds its copy to aiortc itself. + +In a download this side is the receiver of the node's 1 MiB `file_chunk` +messages, about 870 SCTP packets each, and two receiver costs set the ceiling. +Measured on loopback, 400 MB, one connection: + +- aiortc walked every fragment of the message in progress after each one + arrived, 153 million steps: two thirds of the client's CPU. Fixed by the + reassembly fix below — 20 MB/s became 28; +- it acknowledged every packet. RFC 4960 §6.2 asks for one SACK every second + packet (Chromium does that), and each SACK costs the node a DTLS decryption + and a parse on the one core that also sends: more of its time than sending + the data. One every second packet — 28 MB/s became 39, the node's own figure + for the transfer 49. + +A SACK still goes at once when something is out of order or duplicated (that is +the loss signal the sender is waiting for) or the packet carried anything other +than data; a lone packet is acknowledged within `SACK_DELAY_S`. +""" + +import asyncio +import contextlib + +from aiortc.rtcsctptransport import ( + SCTP_DATA_LAST_FRAG, + DataChunk, + InboundStream, + InitChunk, + RTCSctpTransport, + parse_packet, +) + +# RFC 4960 §6.2: at most 500 ms; 200 is what implementations use. +SACK_DELAY_S = 0.2 + + +def _flight_size_increase(self, chunk) -> None: + chunk._acked = False + self._flight_size += chunk._book_size + + +RTCSctpTransport._flight_size_increase = _flight_size_increase + + +_add_chunk = InboundStream.add_chunk +_pop_messages = InboundStream.pop_messages +_prune_chunks = InboundStream.prune_chunks + + +def add_chunk(self, chunk) -> None: + _add_chunk(self, chunk) + if chunk.flags & SCTP_DATA_LAST_FRAG: + self._last_frags = getattr(self, "_last_frags", 0) + 1 + + +def pop_messages(self): + # Only ever too high (a pruned last fragment), never too low: a message + # leaves the reassembly with exactly one last fragment, and only here. + if not getattr(self, "_last_frags", 0): + return + for message in _pop_messages(self): + self._last_frags -= 1 + yield message + + +def prune_chunks(self, tsn: int) -> int: + size = _prune_chunks(self, tsn) + self._last_frags = sum(1 for c in self.reassembly if c.flags & SCTP_DATA_LAST_FRAG) + return size + + +InboundStream.add_chunk = add_chunk +InboundStream.pop_messages = pop_messages +InboundStream.prune_chunks = prune_chunks + + +async def _handle_data(self, data: bytes) -> None: + """aiortc's own, down to the SACK at the end: that one may wait for a + second packet.""" + try: + _, _, verification_tag, chunks = parse_packet(data) + except ValueError: + return + if any(isinstance(x, InitChunk) for x in chunks): + assert len(chunks) == 1 + expected_tag = 0 + else: + expected_tag = self._local_verification_tag + if verification_tag != expected_tag: + return + for chunk in chunks: + await self._receive_chunk(chunk) + if not self._sack_needed: + return + self._sack_packets = getattr(self, "_sack_packets", 0) + 1 + if (self._sack_packets >= 2 or self._sack_misordered or self._sack_duplicates + or not all(isinstance(c, DataChunk) for c in chunks)): + await _sack_now(self) + elif getattr(self, "_sack_timer", None) is None: + self._sack_timer = self._loop.call_later(SACK_DELAY_S, _sack_later, self) + + +async def _sack_now(self) -> None: + timer = getattr(self, "_sack_timer", None) + if timer is not None: + timer.cancel() + self._sack_timer = None + self._sack_packets = 0 + if self._sack_needed and self._association_state != self.State.CLOSED: + # The DTLS transport may be gone under a timer that fired late. + with contextlib.suppress(ConnectionError): + await self._send_sack() + + +def _sack_later(self) -> None: + self._sack_timer = None + # Held: a task nobody references can be collected before it runs. + self._sack_task = asyncio.ensure_future(_sack_now(self)) + + +RTCSctpTransport._handle_data = _handle_data diff --git a/packages/meshbay-cli/src/meshbay_cli/client/session.py b/packages/meshbay-cli/src/meshbay_cli/client/session.py index d10acc8..b6fc9e2 100644 --- a/packages/meshbay-cli/src/meshbay_cli/client/session.py +++ b/packages/meshbay-cli/src/meshbay_cli/client/session.py @@ -63,6 +63,7 @@ from meshbay_common.handshake import ( from meshbay_common.join import join_transcript from ..errors import NODE, NODE_KEY, VERSION, CliError +from . import sctp_fix, udp_buffer # noqa: F401 (patch aiortc and aioice on import) from .account import Account from .keys import ( LEGACY_MAGIC, @@ -250,6 +251,13 @@ class GroupSession: raise CliError(NODE, "The group's index does not open under the group key; " "the session is ended.") from e + def open_delta(self, msg: dict) -> dict: + """A pushed `index_delta`, opened under the group key.""" + try: + return unseal(self.gek, PURPOSE_INDEX, "index_delta", self.group_id, msg) + except Exception as e: + raise CliError(NODE, "An index update does not open under the group key.") from e + async def close(self) -> None: with contextlib.suppress(Exception): await self.pc.close() diff --git a/packages/meshbay-cli/src/meshbay_cli/client/tree.py b/packages/meshbay-cli/src/meshbay_cli/client/tree.py index 6e537ef..dd3acc8 100644 --- a/packages/meshbay-cli/src/meshbay_cli/client/tree.py +++ b/packages/meshbay-cli/src/meshbay_cli/client/tree.py @@ -57,6 +57,39 @@ class Tree: dirs |= set(roots) return cls(roots, dirs, files) + def apply_delta(self, delta: dict) -> bool: + """ + Bring the tree up to an `index_delta`, in place. True when the delta + changed what the tree holds — a file added or removed, a root's state — + rather than only the fields of files already there. + + A delta carries no `dirs`, so an empty folder made elsewhere is not in + it: a structural change is where the caller asks for the whole index + again, once. The common delta is enrichment filling in a file's fields, + about one a second while a library is being enriched, and asking for + the whole index on each of those took the download's own event loop: + 67 full indexes in the 80 s of one download. + """ + updates = {e["id"]: e for e in delta.get("updates", [])} + gone = set(delta.get("deletions", [])) + for parent, entries in self.files.items(): + if gone or any(e["id"] in updates for e in entries): + self.files[parent] = [updates.get(e["id"], e) for e in entries + if e["id"] not in gone] + known = {e["id"] for entries in self.files.values() for e in entries} + added = [e for e in delta.get("additions", []) if e["id"] not in known] + for e in added: + self.files.setdefault(e["path"], []).append(e) + parts = e["path"].split("/") + for i in range(1, len(parts) + 1): + self.dirs.add("/".join(parts[:i])) + roots = {r["name"]: r for r in delta.get("roots", [])} + moved_roots = bool(roots) and roots != self.roots + if moved_roots: + self.roots = roots + self.dirs |= set(roots) + return bool(gone or added or moved_roots) + def root_available(self, path: str) -> bool: root = self.roots.get(path.split("/", 1)[0]) return bool(root.get("available", True)) if root else True diff --git a/packages/meshbay-cli/src/meshbay_cli/client/udp_buffer.py b/packages/meshbay-cli/src/meshbay_cli/client/udp_buffer.py new file mode 100644 index 0000000..c0045d8 --- /dev/null +++ b/packages/meshbay-cli/src/meshbay_cli/client/udp_buffer.py @@ -0,0 +1,18 @@ +"""A larger receive buffer on the client's ICE sockets, applied on import. + +In a download the client is the receiver, and it reads its socket from Python +— on the same event loop as the terminal interface — more slowly than a node +sends in bursts. aioice asks the kernel for 256 KB per socket; what does not fit +is dropped by this machine's kernel (`UdpRcvbufErrors`), the node's SCTP sender +takes each drop for congestion and halves its window. Measured on loopback, 400 +MB: 3245 drops and 19 MB/s with 256 KB. The node does the same for the same +reason in the other direction (`meshbay_node/transport/webrtc/udp_buffer.py`). + +Linux grants no more than `net.core.rmem_max`; Windows grants what is asked. +""" + +import aioice.turn + +WANTED = 4 * 1024 * 1024 + +aioice.turn.UDP_SOCKET_BUFFER_SIZE = WANTED diff --git a/packages/meshbay-cli/src/meshbay_cli/tui/screens.py b/packages/meshbay-cli/src/meshbay_cli/tui/screens.py index 39b5df3..b6fd347 100644 --- a/packages/meshbay-cli/src/meshbay_cli/tui/screens.py +++ b/packages/meshbay-cli/src/meshbay_cli/tui/screens.py @@ -41,6 +41,9 @@ from .panes import LocalPane, RemotePane from .widgets import Choice, TransferStrip PER_GROUP = 2 +# A burst of index updates that added or removed files is answered by one full +# index, this long after the last of them. +REFRESH_AFTER_DELTA_S = 2.0 class SignInScreen(Screen): @@ -118,6 +121,7 @@ class MainScreen(Screen): self._trees: dict[str, Tree] = {} self._gates: dict[str, asyncio.Semaphore] = {} self._watching: set[str] = set() + self._refreshes: dict[str, asyncio.TimerHandle] = {} def compose(self) -> ComposeResult: with Horizontal(): @@ -150,12 +154,36 @@ class MainScreen(Screen): self._trees[group["id"]] = Tree.from_index(await session.index()) if group["id"] not in self._watching: self._watching.add(group["id"]) - session.ch.on_push("index_delta", lambda _msg, g=group: self._changed(g)) + session.ch.on_push( + "index_delta", lambda msg, g=group, s=session: self._delta(g, s, msg)) return self._trees[group["id"]] def forget_tree(self, group: dict) -> None: self._trees.pop(group["id"], None) + def _delta(self, group: dict, session, msg: dict) -> None: + """A pushed index update, applied to the tree in place. Only one that + adds or removes something is drawn, and asks for the whole index — once + for a burst, not once per message.""" + tree = self._trees.get(group["id"]) + if tree is None: + return # the next look asks for the whole index anyway + try: + structural = tree.apply_delta(session.open_delta(msg)) + except CliError: + structural = True + if not structural: + return + pending = self._refreshes.pop(group["id"], None) + if pending is not None: + pending.cancel() + self._refreshes[group["id"]] = asyncio.get_running_loop().call_later( + REFRESH_AFTER_DELTA_S, self._refreshed, group) + + def _refreshed(self, group: dict) -> None: + self._refreshes.pop(group["id"], None) + self._changed(group) + def _changed(self, group: dict) -> None: """Something changed in the group: the index is asked for again, and the pane redrawn if it is showing that group.""" diff --git a/packages/meshbay-cli/tests/fake_node.py b/packages/meshbay-cli/tests/fake_node.py index c618ee4..47f82b3 100644 --- a/packages/meshbay-cli/tests/fake_node.py +++ b/packages/meshbay-cli/tests/fake_node.py @@ -79,6 +79,8 @@ class FakeNode: self.members: dict[str, Identity] = {} # sender → the device it signs with self.unresponsive = False # pings go unanswered, as a dead link's self.admin_ops: dict[str, dict] = {} + self.channels: list[FakeChannel] = [] + self.index_syncs = 0 # full indexes asked for def add(self, path: str, name: str, data: bytes, uploader_id: str = "") -> dict: fid = hashlib.blake2b(data + path.encode() + name.encode(), digest_size=32).hexdigest() @@ -92,9 +94,26 @@ class FakeNode: return entry def session(self) -> GroupSession: - return GroupSession(None, FakeChannel(self), GROUP_ID, "node-1", NODE_PK, self.gek, + ch = FakeChannel(self) + self.channels.append(ch) + return GroupSession(None, ch, GROUP_ID, "node-1", NODE_PK, self.gek, dict(self.ack), self.identity, self.nonce, self.role) + def push_delta(self, *, additions=(), updates=(), deletions=()) -> None: + """An `index_delta` to every session, sealed as the node seals one. The + entries are applied here first, so a later `index_sync` agrees.""" + for e in (*additions, *updates): + self.files[e["id"]] = e + for fid in deletions: + self.files.pop(fid, None) + wire = lambda es: [{k: v for k, v in e.items() if k != "data"} for e in es] # noqa: E731 + payload = {"base_version": 1, "version": 1, "additions": wire(additions), + "deletions": list(deletions), "updates": wire(updates)} + msg = {"type": "index_delta", "group_id": GROUP_ID, + **seal(self.gek, PURPOSE_INDEX, "index_delta", GROUP_ID, payload)} + for ch in self.channels: + ch._emit(msg) + def list_device(self, sender: str, device: Identity, *, countersigned_by: Identity | None = None, proof: bool = True) -> None: """A roster row for `sender`'s device: a root, or countersigned by another @@ -189,6 +208,7 @@ class FakeChannel: def _index_sync(self, msg): n = self.node + n.index_syncs += 1 payload = {"version": 1, "roots": n.roots, "dirs": sorted(n.dirs), "entries": [{k: v for k, v in e.items() if k != "data"} for e in n.files.values()]} diff --git a/packages/meshbay-cli/tests/test_e2e_local.py b/packages/meshbay-cli/tests/test_e2e_local.py index b131658..385693d 100644 --- a/packages/meshbay-cli/tests/test_e2e_local.py +++ b/packages/meshbay-cli/tests/test_e2e_local.py @@ -196,7 +196,14 @@ def _build(root: Path, procs: list) -> dict: _wait_http(f"http://127.0.0.1:{ui_port}/", timeout=90) # Attaching reloads the node by itself; the key needs the group loaded. - run_node("group", "add", group_id, "--dir", str(share)) + # The control API answers before the hub socket is up, later still now + # that the connection processes start first. + deadline = time.monotonic() + 90 + while "Hub not connected" in (out := _try(run_node, "group", "add", group_id, + "--dir", str(share))): + assert time.monotonic() < deadline, f"group add never succeeded: {out}" + time.sleep(1) + assert "meshbay-node (" not in out, out deadline = time.monotonic() + 90 while "GEK ready" not in (out := _try(run_node, "gek", "init", "--group", group_id)): assert time.monotonic() < deadline, f"gek init never succeeded: {out}" @@ -781,7 +788,11 @@ class _Chrome: target = next(p for p in pages if p["type"] == "page") break except (httpx.HTTPError, StopIteration): - assert time.monotonic() < deadline, "Chrome did not come up" + if time.monotonic() > deadline: + # Not left running: nobody holds this object to close it. + self.proc.kill() + self.proc.wait(10) + raise AssertionError("Chrome did not come up") from None time.sleep(0.2) self.ws = connect(target["webSocketDebuggerUrl"], max_size=None) self.n = 0 @@ -1375,17 +1386,18 @@ def test_tab_completes_remote_paths_from_the_real_node(world): assert complete("ls", "No such group/") == [] # nothing, quietly # bash itself: the function the script installs, on a word typed with its - # spaces escaped, as bash hands it over. - script = cli(world, "completion", "bash").stdout - escaped = group[:10].replace(" ", "\\ ") - bash = subprocess.run( - ["bash", "-c", script + "\nCOMP_WORDS=(meshbay-cli ls " + escaped + ")\n" - "COMP_CWORD=2\n_meshbay_cli\nprintf '%s\\n' \"${COMPREPLY[@]}\"\n"], - env={**os.environ, "MESHBAY_CLI_HOME": str(world["root"] / "cli-chat"), - "PYTHON_KEYRING_BACKEND": "keyring.backends.fail.Keyring", - "PATH": f"{BIN}{os.pathsep}{os.environ['PATH']}"}, - capture_output=True, text=True, encoding="utf-8", timeout=60) - assert bash.stdout.splitlines() == [group + "/"], (bash.stdout, bash.stderr) + # spaces escaped, as bash hands it over. On Windows, where Git's is on PATH. + if shutil.which("bash"): + script = cli(world, "completion", "bash").stdout + escaped = group[:10].replace(" ", "\\ ") + bash = subprocess.run( + ["bash", "-c", script + "\nCOMP_WORDS=(meshbay-cli ls " + escaped + ")\n" + "COMP_CWORD=2\n_meshbay_cli\nprintf '%s\\n' \"${COMPREPLY[@]}\"\n"], + env={**os.environ, "MESHBAY_CLI_HOME": str(world["root"] / "cli-chat"), + "PYTHON_KEYRING_BACKEND": "keyring.backends.fail.Keyring", + "PATH": f"{BIN}{os.pathsep}{os.environ['PATH']}"}, + capture_output=True, text=True, encoding="utf-8", timeout=60) + assert bash.stdout.splitlines() == [group + "/"], (bash.stdout, bash.stderr) # PowerShell 7, or Windows PowerShell 5.1, which decodes a program's output # with the console's code page unless told otherwise: an accented name is the diff --git a/packages/meshbay-cli/tests/test_interactive.py b/packages/meshbay-cli/tests/test_interactive.py index 36b7f47..9960533 100644 --- a/packages/meshbay-cli/tests/test_interactive.py +++ b/packages/meshbay-cli/tests/test_interactive.py @@ -444,3 +444,46 @@ async def test_put_sends_a_file_into_the_chat_folder_and_get_saves_one_here( await say("/get ../share/x") await _until(pilot, lambda: "name a file of the chat's folder" in _said(room)) await pilot.press("escape") + + +async def test_index_updates_are_applied_not_asked_for_again(world, monkeypatch): + """ + A library being enriched pushes an `index_delta` about once a second, each + one filling in a file's fields. The panes used to answer every one with the + whole index — 67 full indexes in the 80 s of one download on a real node, + on the event loop the download was being received on. Updates are applied + where they land; only a delta that adds or removes a file asks again, once + for a burst. + """ + from meshbay_cli.tui import screens + monkeypatch.setattr(screens, "REFRESH_AFTER_DELTA_S", 0.2) + hub, node, work = world + film = node.add("share", "film.mkv", b"F" * 5000) + assert await asyncio.to_thread(cli.main, ["--hub", "https://hub.test", "-u", USERNAME, + "--password", PASSPHRASE, "login"]) == 0 + app = _app() + async with app.run_test(size=(140, 40)) as pilot: + await _until(pilot, lambda: isinstance(app.screen, MainScreen)) + remote = app.screen.query_one(RemotePane) + await _open(pilot, remote, f"Family@{USERNAME}/") + await _open(pilot, remote, "share/") + await _until(pilot, lambda: "film.mkv" in _names(remote)) + asked = node.index_syncs + + for i in range(20): + node.push_delta(updates=[{**film, "duration": 100 + i}]) + await pilot.pause(0.6) + assert node.index_syncs == asked, "an update must not fetch the whole index" + tree = app.screen._trees["g1"] + assert tree.file("share/film.mkv")["duration"] == 119 + + extra = [node.add("share", f"new{i}.txt", b"n%d" % i) for i in range(3)] + node.files = {k: v for k, v in node.files.items() if v not in extra} + for e in extra: + node.push_delta(additions=[e]) + await _until(pilot, lambda: "new2.txt" in _names(remote)) + await pilot.pause(0.6) + assert node.index_syncs == asked + 1, "a burst of additions asks again once" + + node.push_delta(deletions=[film["id"]]) + await _until(pilot, lambda: "film.mkv" not in _names(remote)) diff --git a/packages/meshbay-cli/tests/test_sctp_fix.py b/packages/meshbay-cli/tests/test_sctp_fix.py new file mode 100644 index 0000000..04638db --- /dev/null +++ b/packages/meshbay-cli/tests/test_sctp_fix.py @@ -0,0 +1,89 @@ +"""The client's copy of the node's aiortc fixes (client/sctp_fix.py). + +The copy exists because neither package may depend on the other; it is held to +the node's by comparing the code itself, docstrings aside, and the two effects +that set a download's speed here are replayed against aiortc. +""" + +import ast +import asyncio +from pathlib import Path + +import pytest +from aiortc.rtcsctptransport import ( + SCTP_DATA_FIRST_FRAG, + SCTP_DATA_LAST_FRAG, + DataChunk, + InboundStream, + RTCSctpTransport, + serialize_packet, +) +from meshbay_cli.client import sctp_fix + +NODE_COPY = (Path(__file__).resolve().parents[2] / "meshbay-node" / "src" / "meshbay_node" + / "transport" / "webrtc" / "sctp_fix.py") + + +def _code(path: Path) -> str: + tree = ast.parse(path.read_text(encoding="utf-8")) + tree.body = tree.body[1:] # the module docstring + return ast.dump(tree) + + +@pytest.mark.skipif(not NODE_COPY.exists(), reason="not in the source tree") +def test_the_code_is_the_nodes(): + assert _code(Path(sctp_fix.__file__)) == _code(NODE_COPY) + + +def _chunks(n_fragments: int, count: int, tsn: int = 1000) -> list[DataChunk]: + out = [] + for seq in range(count): + for i in range(n_fragments): + c = DataChunk() + c.flags = ((SCTP_DATA_FIRST_FRAG if i == 0 else 0) + | (SCTP_DATA_LAST_FRAG if i == n_fragments - 1 else 0)) + c.tsn, c.stream_id, c.stream_seq, c.protocol = tsn, 1, seq, 53 + c.user_data = b"x" + tsn += 1 + out.append(c) + return out + + +def test_a_megabyte_is_walked_once(monkeypatch): + walks = [] + original = sctp_fix._pop_messages + monkeypatch.setattr(sctp_fix, "_pop_messages", + lambda s: (walks.append(1), original(s))[1]) + s = InboundStream() + got = 0 + for c in _chunks(870, 3): + s.add_chunk(c) + got += len(list(s.pop_messages())) + assert got == 3 and len(walks) == 3 + + +def test_every_second_packet_is_acknowledged(): + class _Dtls: + state = "connected" + + def _register_data_receiver(self, *a): + pass + + async def go(): + s = RTCSctpTransport(_Dtls()) + s._association_state = s.State.ESTABLISHED + s._last_received_tsn = 999 + sacks = [] + + async def send_sack(): + sacks.append(s._last_received_tsn) + s._sack_needed = False + + async def deliver(*_a): + pass + + s._send_sack, s._receive = send_sack, deliver + for c in _chunks(1, 6): + await s._handle_data(serialize_packet(1, 1, s._local_verification_tag, c)) + return sacks + assert asyncio.run(go()) == [1001, 1003, 1005] diff --git a/packages/meshbay-client/build/appx-extensions.xml b/packages/meshbay-client/build/appx-extensions.xml index db6e345..3326dd8 100644 --- a/packages/meshbay-client/build/appx-extensions.xml +++ b/packages/meshbay-client/build/appx-extensions.xml @@ -13,10 +13,3 @@ <uap5:ExecutionAlias Alias="meshbay-node.exe" /> </uap5:AppExecutionAlias> </uap5:Extension> - <!-- `meshbay-cli` in a terminal, the command-line client: the same alias - mechanism, for the same reason. --> - <uap5:Extension Category="windows.appExecutionAlias" Executable="app\resources\node-runtime\meshbay-cli.exe" EntryPoint="Windows.FullTrustApplication"> - <uap5:AppExecutionAlias> - <uap5:ExecutionAlias Alias="meshbay-cli.exe" /> - </uap5:AppExecutionAlias> - </uap5:Extension> diff --git a/packages/meshbay-client/build/appx-manifest.xml b/packages/meshbay-client/build/appx-manifest.xml index c097fdd..0542bd5 100644 --- a/packages/meshbay-client/build/appx-manifest.xml +++ b/packages/meshbay-client/build/appx-manifest.xml @@ -43,6 +43,27 @@ </uap:VisualElements> ${extensions} </Application> + <!-- `meshbay-cli` in a terminal, by the same alias mechanism as the node. + An application declares one execution alias, and meshbay-node.exe has + the app's (appx-extensions.xml): a second one there made makeappx + refuse the package. So the command-line client is an application of + its own, listed nowhere. --> + <Application Id="MeshBayCli" Executable="app\resources\node-runtime\meshbay-cli.exe" EntryPoint="Windows.FullTrustApplication"> + <uap:VisualElements + AppListEntry="none" + BackgroundColor="${backgroundColor}" + DisplayName="MeshBay command-line client" + Square150x150Logo="${square150x150Logo}" + Square44x44Logo="${square44x44Logo}" + Description="MeshBay command-line client" /> + <Extensions> + <uap5:Extension Category="windows.appExecutionAlias" Executable="app\resources\node-runtime\meshbay-cli.exe" EntryPoint="Windows.FullTrustApplication"> + <uap5:AppExecutionAlias> + <uap5:ExecutionAlias Alias="meshbay-cli.exe" /> + </uap5:AppExecutionAlias> + </uap5:Extension> + </Extensions> + </Application> </Applications> <!-- The node accepts connections from peers (WebRTC, QUIC, casting), and the command-line client does during a transfer (WebRTC). Rules declared diff --git a/packages/meshbay-hub/src/meshbay_hub/static/file-utils.js b/packages/meshbay-hub/src/meshbay_hub/static/file-utils.js index 585558f..ae6cd6f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/file-utils.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/file-utils.js @@ -349,9 +349,13 @@ async function _writeOrStall(writable, bytes, at) { } } +// "Transport closed" is not here: that is close(), somebody ending the +// connection on purpose. "DataChannel closed" is the channel going away under +// it, which is what a reconnect is for. function _isRetryableTransportError(err) { return err.name === 'TransportLostError' || err.message === 'Response timeout' + || err.message === 'DataChannel closed' || (err.message || '').startsWith('DataChannel not open'); } diff --git a/packages/meshbay-hub/src/meshbay_hub/static/node-page.js b/packages/meshbay-hub/src/meshbay_hub/static/node-page.js index bc22cbd..674779b 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/node-page.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/node-page.js @@ -501,7 +501,15 @@ export function NodePage({ groups, token, username }) { setSavingSettings(true); setActionMsg(''); try { - await nodeOp('setNodeSettings', { settings: editSettings }); + // What was changed, and nothing else. Sending the whole form stored every + // setting the operator had not touched as a choice, in the roster and in + // node.toml, so no later default ever reached that node again: one + // saved stream cap kept connection_processes at 1 for good. + const changed = Object.fromEntries(Object.entries(editSettings).filter( + ([k, v]) => JSON.stringify(v) !== JSON.stringify((nodeSettings || {})[k]))); + if (Object.keys(changed).length) { + await nodeOp('setNodeSettings', { settings: changed }); + } setNodeSettings({ ...editSettings }); setActionMsg(t('node.settings_saved')); } catch (err) { @@ -509,7 +517,7 @@ export function NodePage({ groups, token, username }) { } finally { setSavingSettings(false); } - }, [editSettings]); + }, [editSettings, nodeSettings]); const saveStun = useCallback(async () => { if (!editStun) return; diff --git a/packages/meshbay-hub/src/meshbay_hub/static/style.css b/packages/meshbay-hub/src/meshbay_hub/static/style.css index 627f2ca..de4f2ae 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/style.css +++ b/packages/meshbay-hub/src/meshbay_hub/static/style.css @@ -101,6 +101,14 @@ foreign highlight. Unrelated to real hover/focus — safe to kill outright. */ *, *::before, *::after { -webkit-tap-highlight-color: transparent; } +/* Every :hover rule in this file sits inside @media (hover: hover). A tap on a + touchscreen leaves the element in :hover until something else is tapped, + since no pointer ever moves away, so a tapped button kept its hover colour + (the music player's next/prev, the pager arrows, the nav hamburger: found + live on Android Chrome and in the app, 2026-08-26 and again 2026-10-16). + Fixing it button by button only moved the bug to the next one; scoping + them all is what test_hover_scoped.py holds. Real hover is unaffected. */ + body { font-family: system-ui, -apple-system, sans-serif; background: var(--bg-base); @@ -110,7 +118,7 @@ body { } a { color: var(--accent); text-decoration: none; } -a:hover { text-decoration: underline; } +@media (hover: hover) { a:hover { text-decoration: underline; } } /* ── Cross-group search page ──────────────────────────────────────────────── */ @@ -141,7 +149,7 @@ a:hover { text-decoration: underline; } background: none; border: none; padding: 2px; cursor: pointer; color: var(--text-dim); display: flex; align-items: center; } -.search-bar-clear:hover { color: var(--text); } +@media (hover: hover) { .search-bar-clear:hover { color: var(--text); } } .search-bar-clear .icon { width: 14px; height: 14px; } .search-progress { @@ -182,7 +190,7 @@ a:hover { text-decoration: underline; } display: flex; align-items: center; gap: 10px; padding: 8px 10px; border-radius: 6px; cursor: pointer; } -.search-media-row:hover { background: var(--bg-raised); } +@media (hover: hover) { .search-media-row:hover { background: var(--bg-raised); } } .search-media-icon { flex-shrink: 0; width: 32px; height: 32px; display: flex; align-items: center; justify-content: center; @@ -244,7 +252,7 @@ a:hover { text-decoration: underline; } font-size: 1.375em; text-decoration: none; } -.nav-brand:hover { text-decoration: none; } +@media (hover: hover) { .nav-brand:hover { text-decoration: none; } } /* The M is a picture standing in for a letter, so it is sized in `em` and follows the wordmark rather than being pinned to a pixel height. @@ -304,7 +312,7 @@ a:hover { text-decoration: underline; } font-size: 0.85em; transition: background 0.12s; } -.user-menu-trigger:hover { background: rgba(255, 255, 255, 0.15); } +@media (hover: hover) { .user-menu-trigger:hover { background: rgba(255, 255, 255, 0.15); } } .user-menu-avatar { width: 28px; height: 28px; @@ -368,7 +376,7 @@ a:hover { text-decoration: underline; } color: var(--text); font-size: 0.85em; cursor: pointer; text-align: left; } -.user-menu-item:hover { background: var(--bg-raised); } +@media (hover: hover) { .user-menu-item:hover { background: var(--bg-raised); } } .user-menu-logout { color: var(--error); } .umi-icon { width: 17px; height: 17px; flex-shrink: 0; color: var(--text-secondary); } @@ -387,7 +395,9 @@ a:hover { text-decoration: underline; } cursor: pointer; text-decoration: none; } -.nav-btn:hover { background: rgba(255, 255, 255, 0.18); text-decoration: none; } +@media (hover: hover) { + .nav-btn:hover { background: rgba(255, 255, 255, 0.18); text-decoration: none; } +} /* ── Layout ───────────────────────────────────────────────────────────────── */ @@ -426,7 +436,7 @@ a:hover { text-decoration: underline; } color: var(--text-dim); font-size: 0.75em; } -.sidebar-legal:hover { color: var(--accent); } +@media (hover: hover) { .sidebar-legal:hover { color: var(--accent); } } .register-legal { margin: 12px 0 0; @@ -477,7 +487,9 @@ a:hover { text-decoration: underline; } color: var(--text-dim); } .sidebar-item.active .sidebar-owner { color: inherit; opacity: 0.75; } -.sidebar-item:hover { background: var(--sidebar-hover); text-decoration: none; } +@media (hover: hover) { + .sidebar-item:hover { background: var(--sidebar-hover); text-decoration: none; } +} .sidebar-item.active { background: var(--accent); color: var(--accent-text); } .sidebar-empty { @@ -936,9 +948,9 @@ a:hover { text-decoration: underline; } font-size: 0.95em; font-weight: 600; } -.welcome-cta:hover { background: #15803d; text-decoration: none; } +@media (hover: hover) { .welcome-cta:hover { background: #15803d; text-decoration: none; } } .welcome-legal { color: var(--text-secondary); font-size: 0.9em; } -.welcome-legal:hover { color: var(--accent); } +@media (hover: hover) { .welcome-legal:hover { color: var(--accent); } } @media (max-width: 1000px) { .welcome { flex-direction: column; align-items: center; gap: 36px; } @@ -995,7 +1007,7 @@ input:focus { outline: none; border-color: var(--border-focus); } font-size: 1.15em; cursor: pointer; } -.pw-toggle:hover { background: none; color: var(--text); } +@media (hover: hover) { .pw-toggle:hover { background: none; color: var(--text); } } button { padding: 10px 20px; @@ -1007,7 +1019,7 @@ button { cursor: pointer; transition: background 0.15s; } -button:hover { background: var(--accent-hover); } +@media (hover: hover) { button:hover { background: var(--accent-hover); } } button:disabled { opacity: 0.5; cursor: not-allowed; } /* ── Alerts ───────────────────────────────────────────────────────────────── */ @@ -1066,10 +1078,12 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } color: var(--text); display: block; } -.group-card:hover { - border-color: var(--accent); - box-shadow: var(--shadow-lg); - text-decoration: none; +@media (hover: hover) { + .group-card:hover { + border-color: var(--accent); + box-shadow: var(--shadow-lg); + text-decoration: none; + } } .group-card h3 { font-size: 1em; margin-bottom: 4px; } @@ -1123,7 +1137,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } align-items: center; justify-content: center; } -.group-tab:hover { color: var(--text); background: none; } +@media (hover: hover) { .group-tab:hover { color: var(--text); background: none; } } .group-tab.active { color: var(--accent); border-bottom-color: var(--accent); @@ -1141,7 +1155,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } align-items: center; margin-left: auto; } -.group-mute-btn:hover { color: var(--text); } +@media (hover: hover) { .group-mute-btn:hover { color: var(--text); } } .group-mute-btn .icon { width: 20px; height: 20px; } /* ── Chat panel ──────────────────────────────────────────────────────────── */ @@ -1236,7 +1250,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } text-decoration: none; overflow: hidden; } -.chat-link-preview:hover { background: var(--bg-surface); } +@media (hover: hover) { .chat-link-preview:hover { background: var(--bg-surface); } } .clp-img { width: 72px; height: 72px; @@ -1356,9 +1370,11 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; transition: border-color 0.12s, color 0.12s, background 0.12s; } -.tb-btn:hover { border-color: var(--accent); color: var(--accent); } +@media (hover: hover) { .tb-btn:hover { border-color: var(--accent); color: var(--accent); } } .tb-btn:disabled { opacity: 0.45; cursor: not-allowed; } -.tb-btn:disabled:hover { border-color: var(--border); color: var(--text); } +@media (hover: hover) { + .tb-btn:disabled:hover { border-color: var(--border); color: var(--text); } +} .tb-btn .icon { width: 15px; height: 15px; } /* Square, for a toolbar button whose name is in its tooltip rather than beside it. The horizontal padding goes; the height does not, so it lines up @@ -1381,7 +1397,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } border-color: var(--accent); color: var(--accent-text); } -.tb-btn.primary:hover { background: var(--accent-hover); color: var(--accent-text); } +@media (hover: hover) { + .tb-btn.primary:hover { background: var(--accent-hover); color: var(--accent-text); } +} .tb-btn.active { background: color-mix(in srgb, var(--accent) 12%, var(--bg-surface)); border-color: var(--accent); @@ -1434,7 +1452,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } padding: 2px 4px; border-radius: 4px; } -.crumb:hover { background: var(--bg-raised); text-decoration: none; } +@media (hover: hover) { .crumb:hover { background: var(--bg-raised); text-decoration: none; } } .crumb-sep { color: var(--text-dim); } .crumb-refresh { background: none; @@ -1442,7 +1460,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } font: inherit; color: var(--text-secondary); } -.crumb-refresh:hover:not(:disabled) { color: var(--text); } +@media (hover: hover) { .crumb-refresh:hover:not(:disabled) { color: var(--text); } } .crumb-refresh:disabled { cursor: default; } .crumb-refresh.spinning .icon { animation: spin 0.8s linear infinite; } @@ -1477,7 +1495,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } } .file-table th.sortable { cursor: pointer; } -.file-table th.sortable:hover { color: var(--text); } +@media (hover: hover) { .file-table th.sortable:hover { color: var(--text); } } .file-row td { padding: 8px 12px; @@ -1485,7 +1503,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } vertical-align: middle; } -.file-row:hover { background: var(--bg-raised); } +@media (hover: hover) { .file-row:hover { background: var(--bg-raised); } } .dir-row { cursor: pointer; } .dir-row td { font-weight: 500; } @@ -1522,7 +1540,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } color: var(--text-dim); cursor: pointer; } -.file-loc:hover { color: var(--accent); text-decoration: underline; } +@media (hover: hover) { .file-loc:hover { color: var(--accent); text-decoration: underline; } } .file-size { white-space: nowrap; color: var(--text-secondary); } .file-type { color: var(--text-dim); } .file-date { white-space: nowrap; color: var(--text-dim); } @@ -1573,7 +1591,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; line-height: 1; } -.file-menu-btn:hover { background: var(--bg-raised); border-color: var(--accent); color: var(--accent); } +@media (hover: hover) { + .file-menu-btn:hover { background: var(--bg-raised); border-color: var(--accent); color: var(--accent); } +} .file-menu { position: absolute; @@ -1600,7 +1620,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; border-radius: 0; } -.file-menu button:hover { background: var(--bg-raised); } +@media (hover: hover) { .file-menu button:hover { background: var(--bg-raised); } } /* ── File link (clickable name) ─────────────────────────────────────────── */ @@ -1609,7 +1629,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; text-decoration: none; } -.file-link:hover { text-decoration: underline; } +@media (hover: hover) { .file-link:hover { text-decoration: underline; } } /* ── File preview overlay ───────────────────────────────────────────────── */ @@ -1665,7 +1685,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } flex-shrink: 0; color: var(--text-secondary); } -.chat-attach:hover { background: var(--bg-raised); color: var(--accent); } +@media (hover: hover) { .chat-attach:hover { background: var(--bg-raised); color: var(--accent); } } .chat-attachment { display: flex; @@ -1692,7 +1712,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } .file-menu button { display: flex; align-items: center; gap: 8px; } .file-menu button.danger { color: var(--error); } -.file-menu button.danger:hover { background: var(--error-bg); } +@media (hover: hover) { .file-menu button.danger:hover { background: var(--error-bg); } } .fmi { font-size: 0.9em; width: 16px; text-align: center; flex-shrink: 0; } /* ── Download button + progress ──────────────────────────────────────────── */ @@ -1711,7 +1731,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } justify-content: center; cursor: pointer; } -.dl-btn:hover { background: var(--bg-raised); border-color: var(--accent); } +@media (hover: hover) { + .dl-btn:hover { background: var(--bg-raised); border-color: var(--accent); } +} .dl-btn:disabled { opacity: 0.3; cursor: not-allowed; } @@ -1858,7 +1880,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } background: var(--accent); color: var(--accent-text); font-family: inherit; font-size: 0.83em; cursor: pointer; } -.app-save:hover:not(:disabled) { background: var(--accent-hover); } +@media (hover: hover) { .app-save:hover:not(:disabled) { background: var(--accent-hover); } } /* No transition on the enabled/disabled swap. It animates a colour change that coincides with a text-colour change, which reads as a flicker — and it made the state genuinely hard to observe: a measurement taken during it reports a @@ -2044,10 +2066,10 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } justify-content: center; flex-shrink: 0; } -.video-close:hover { background: rgba(255, 255, 255, 0.25); } +@media (hover: hover) { .video-close:hover { background: rgba(255, 255, 255, 0.25); } } .video-close.dl-active { background: rgba(34, 197, 94, 0.25); pointer-events: none; } .video-close.cast-active { background: rgba(59, 130, 246, 0.35); } -.video-close.cast-active:hover { background: rgba(59, 130, 246, 0.5); } +@media (hover: hover) { .video-close.cast-active:hover { background: rgba(59, 130, 246, 0.5); } } .cast-url-input { background: rgba(255, 255, 255, 0.08); border: 1px solid rgba(255, 255, 255, 0.15); border-radius: 4px; color: #e2e8f0; font-size: 0.75em; font-family: inherit; @@ -2058,9 +2080,11 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } .cast-wrapper { display: inline-block; position: relative; } .video-close:disabled { opacity: 0.4; cursor: default; } .video-close.cast-btn.casting { background: rgba(59, 130, 246, 0.35); } -.video-close.cast-btn.casting:hover { background: rgba(59, 130, 246, 0.5); } +@media (hover: hover) { + .video-close.cast-btn.casting:hover { background: rgba(59, 130, 246, 0.5); } +} .cast-picker-item.cast-menu-current { color: #94a3b8; cursor: default; } -.cast-picker-item.cast-menu-current:hover { background: none; } +@media (hover: hover) { .cast-picker-item.cast-menu-current:hover { background: none; } } /* The cast button in a page's toolbar (cast-session.js): the same panel as any other menu there, hung under the button rather than at a point. */ @@ -2069,7 +2093,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } /* From the music bar, at the bottom of the window: the menu opens upward. */ .cast-menu-up { top: auto !important; bottom: calc(100% + 4px); } .ctx-menu-item.cast-menu-current { color: var(--text-dim); cursor: default; } -.ctx-menu-item.cast-menu-current:hover { background: none; } +@media (hover: hover) { .ctx-menu-item.cast-menu-current:hover { background: none; } } .ctx-menu-item svg { width: 16px; height: 16px; flex-shrink: 0; } .photo-album-actions { display: flex; align-items: center; gap: 4px; } .cast-picker { @@ -2084,12 +2108,12 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } color: #e2e8f0; font-size: 0.82em; cursor: pointer; text-align: left; font-family: inherit; } -.cast-picker-item:hover { background: rgba(255, 255, 255, 0.09); } +@media (hover: hover) { .cast-picker-item:hover { background: rgba(255, 255, 255, 0.09); } } .cast-picker-item svg { width: 16px; height: 16px; flex-shrink: 0; } .cast-picker-scanning { color: #94a3b8; cursor: default; } -.cast-picker-scanning:hover { background: none; } +@media (hover: hover) { .cast-picker-scanning:hover { background: none; } } .cast-picker-empty { color: #64748b; cursor: default; font-style: italic; } -.cast-picker-empty:hover { background: none; } +@media (hover: hover) { .cast-picker-empty:hover { background: none; } } .cast-picker-sep { height: 1px; margin: 4px 10px; background: rgba(255,255,255,0.1); } .cast-status-label { color: #94a3b8; font-size: 0.78em; display: flex; align-items: center; @@ -2270,7 +2294,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } color: #e2e8f0; font-size: 44px; } -.video-remote-skip:hover:not(:disabled) { background: rgba(148, 163, 184, 0.12); } +@media (hover: hover) { + .video-remote-skip:hover:not(:disabled) { background: rgba(148, 163, 184, 0.12); } +} .video-remote-skip .icon { stroke-width: 1.4; } .video-remote-skip-n { position: absolute; @@ -2292,7 +2318,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } font-size: 34px; box-shadow: 0 8px 24px rgba(35, 177, 240, 0.35); } -.video-remote-main:hover:not(:disabled) { background: #4cc3f5; } +@media (hover: hover) { .video-remote-main:hover:not(:disabled) { background: #4cc3f5; } } .video-remote-main .icon { stroke-width: 2.6; } /* The triangle is a closed path: filled, it reads as play at a glance. */ .video-remote-main.is-paused .icon path { fill: currentColor; } @@ -2311,7 +2337,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } font-size: 0.9rem; cursor: pointer; } -.video-remote-stop:hover { border-color: #fca5a5; color: #fecaca; } +@media (hover: hover) { .video-remote-stop:hover { border-color: #fca5a5; color: #fecaca; } } /* Where the film was picked up again. Sits over the picture rather than pushing it down, and clears itself once the viewer is watching. */ @@ -2407,7 +2433,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; margin-right: 4px; } -.play-btn:hover { background: var(--bg-raised); border-color: var(--success); } +@media (hover: hover) { + .play-btn:hover { background: var(--bg-raised); border-color: var(--success); } +} .play-btn:disabled { opacity: 0.3; cursor: not-allowed; } /* ── Admin panel ─────────────────────────────────────────────────────────── */ @@ -2431,7 +2459,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } margin-bottom: -2px; transition: color 0.15s, border-color 0.15s; } -.admin-tab:hover { color: var(--text); } +@media (hover: hover) { .admin-tab:hover { color: var(--text); } } .admin-tab.active { color: var(--accent); border-bottom-color: var(--accent); @@ -2532,7 +2560,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } word-break: break-word; min-width: 0; } -.admin-table tr:hover { background: var(--bg-raised); } +@media (hover: hover) { .admin-table tr:hover { background: var(--bg-raised); } } .admin-table .admin-empty { text-align: center; padding: 24px; @@ -2642,9 +2670,9 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; white-space: nowrap; } -.admin-btn:hover { border-color: var(--accent); color: var(--accent); } +@media (hover: hover) { .admin-btn:hover { border-color: var(--accent); color: var(--accent); } } .admin-btn.danger { color: var(--error); } -.admin-btn.danger:hover { border-color: var(--error); } +@media (hover: hover) { .admin-btn.danger:hover { border-color: var(--error); } } .admin-btn:disabled { opacity: 0.4; cursor: not-allowed; } /* Used in four places and never defined, so it rendered as a bare browser @@ -2659,7 +2687,7 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: pointer; white-space: nowrap; } -.btn-secondary:hover { border-color: var(--border); color: var(--text); } +@media (hover: hover) { .btn-secondary:hover { border-color: var(--border); color: var(--text); } } .btn-secondary:disabled { opacity: 0.4; cursor: not-allowed; } /* The recovery screen's "e-mail it to myself": a mailto: link, shaped like the Copy button beside it. */ @@ -2683,7 +2711,7 @@ a.recovery-mail { font-size: 0.8em; cursor: pointer; } -.link-btn:hover { color: var(--accent); } +@media (hover: hover) { .link-btn:hover { color: var(--accent); } } /* The header is a flex row, so a textarea in it collapses to its content width unless told otherwise. */ @@ -2799,7 +2827,7 @@ a.transfer-name { padding: 0 2px; display: flex; } -.transfer-cancel:hover { color: var(--error); } +@media (hover: hover) { .transfer-cancel:hover { color: var(--error); } } /* Same shape as cancel, and beside it: pausing and cancelling are the two things a person does to a transfer, and one of them is not destructive. */ .transfer-pause { @@ -2810,7 +2838,7 @@ a.transfer-name { padding: 0 2px; display: flex; } -.transfer-pause:hover { color: var(--accent); } +@media (hover: hover) { .transfer-pause:hover { color: var(--accent); } } /* Not a button: there is nothing to click. Dimmer than the cancel beside it, and it carries its explanation in a tooltip rather than in the row, which would be four lines of prose in a panel that has none. */ @@ -3136,11 +3164,16 @@ a.transfer-name { border-radius: 6px; opacity: 0.75; } -.nav-tray:hover, .nav-tray:focus-visible { opacity: 1; background: rgba(255, 255, 255, 0.1); } +@media (hover: hover) { + .nav-tray:hover { + opacity: 1; + background: rgba(255, 255, 255, 0.1); + } +} /* ── Notification bell ───────────────────────────────────────────────────── */ @@ -3160,8 +3193,8 @@ a.transfer-name { padding: 4px; border-radius: 6px; } -.nav-search:hover, .nav-search[aria-current="page"] { background: rgba(255, 255, 255, 0.1); } +@media (hover: hover) { .nav-search:hover { background: rgba(255, 255, 255, 0.1); } } .notif-badge { position: absolute; top: -6px; @@ -3199,8 +3232,10 @@ a.transfer-name { cursor: pointer; border: 1px solid var(--border); } -.notif-item:hover { - background: var(--bg-raised); +@media (hover: hover) { + .notif-item:hover { + background: var(--bg-raised); + } } .notif-unread { /* `--primary` is not defined here either; it is `--accent`. */ @@ -3296,7 +3331,7 @@ a.transfer-name { cursor: pointer; font-size: 0.9em; } -.btn-danger:hover { filter: brightness(1.1); } +@media (hover: hover) { .btn-danger:hover { filter: brightness(1.1); } } .btn-danger:disabled { opacity: 0.5; cursor: not-allowed; } .notif-feed h3 { display: flex; align-items: center; justify-content: space-between; } @@ -3379,7 +3414,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } font-size: 0.8em; cursor: pointer; } -.chat-older-btn:hover:not(:disabled) { background: var(--bg-surface); color: var(--text); } +@media (hover: hover) { + .chat-older-btn:hover:not(:disabled) { background: var(--bg-surface); color: var(--text); } +} .chat-older-btn:disabled { opacity: 0.6; cursor: default; } .chat-older-icon { width: 13px; height: 13px; transform: rotate(180deg); } @@ -3435,7 +3472,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } cursor: pointer; box-shadow: var(--shadow-lg); } -.chat-jump:hover { color: var(--text); } +@media (hover: hover) { .chat-jump:hover { color: var(--text); } } .chat-jump.unread { background: var(--accent); color: var(--accent-text); @@ -3464,10 +3501,12 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } cursor: pointer; transition: background 0.12s, color 0.12s, border-color 0.12s; } -.tb-icon-btn:hover:not(:disabled) { - background: var(--bg-raised); - color: var(--text); - border-color: var(--accent); +@media (hover: hover) { + .tb-icon-btn:hover:not(:disabled) { + background: var(--bg-raised); + color: var(--text); + border-color: var(--accent); + } } /* Present but inert: the bar shows what exists here, and greys out what this selection cannot do. `not-allowed` rather than `default` so the reason is @@ -3478,10 +3517,12 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } } .tb-icon-btn .icon { width: 16px; height: 16px; } .tb-icon-btn.danger { color: var(--error); } -.tb-icon-btn.danger:hover:not(:disabled) { - background: var(--error-bg); - border-color: var(--error); - color: var(--error); +@media (hover: hover) { + .tb-icon-btn.danger:hover:not(:disabled) { + background: var(--error-bg); + border-color: var(--error); + color: var(--error); + } } @@ -3505,7 +3546,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } cursor: pointer; transition: border-color 0.12s, background 0.12s; } -.choice:hover { border-color: var(--accent); } +@media (hover: hover) { .choice:hover { border-color: var(--accent); } } .choice.selected { border-color: var(--accent); background: var(--bg-raised); } .choice:focus-within { outline: 2px solid var(--accent); outline-offset: 2px; } @@ -3538,7 +3579,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } font-weight: 600; cursor: pointer; } -.btn-primary:hover:not(:disabled) { background: var(--accent-hover); } +@media (hover: hover) { .btn-primary:hover:not(:disabled) { background: var(--accent-hover); } } .btn-primary:disabled { opacity: 0.5; cursor: not-allowed; } /* A root whose volume is absent. Its files stay listed — they are frozen, not @@ -3627,9 +3668,13 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } line-height: 1; vertical-align: middle; } .sdt-action-btn + .sdt-action-btn { margin-left: 6px; } -.sdt-action-btn:hover:not(:disabled) { background: var(--bg-raised); color: var(--text); } +@media (hover: hover) { + .sdt-action-btn:hover:not(:disabled) { background: var(--bg-raised); color: var(--text); } +} .sdt-action-btn:disabled { opacity: 0.3; cursor: default; } -.sdt-action-danger:hover:not(:disabled) { color: var(--error); border-color: var(--error); } +@media (hover: hover) { + .sdt-action-danger:hover:not(:disabled) { color: var(--error); border-color: var(--error); } +} .sdt-action-btn .icon { width: 1em; height: 1em; display: block; } .sdt-dir-name { display: inline-flex; align-items: center; gap: 6px; font-weight: 500; } .sdt-dir-name .icon { width: 16px; height: 16px; flex-shrink: 0; } @@ -3640,7 +3685,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } padding: 2px 6px; cursor: pointer; font-size: 0.85em; color: var(--text-dim); line-height: 1; } -.root-eject-btn:hover { background: var(--bg-raised); } +@media (hover: hover) { .root-eject-btn:hover { background: var(--bg-raised); } } .root-eject-btn { margin-left: 8px; } /* An ejected root in the Files table: still listed, deliberately — its files are frozen, not gone — but not somewhere you can walk into. */ @@ -4187,7 +4232,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } transition: background 0.12s, color 0.12s; } .tb-typefilter button + button { border-left: 1px solid var(--border); } -.tb-typefilter button:hover { color: var(--accent); } +@media (hover: hover) { .tb-typefilter button:hover { color: var(--accent); } } .tb-typefilter button.active { background: var(--accent); color: var(--accent-text); @@ -4213,7 +4258,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } border: 1px solid var(--border); transition: border-color 0.12s, transform 0.12s; } -.video-card:hover { border-color: var(--accent); transform: translateY(-2px); } +@media (hover: hover) { + .video-card:hover { border-color: var(--accent); transform: translateY(-2px); } +} .video-poster { width: 100%; @@ -4321,7 +4368,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } } .video-detail .video-title { color: var(--text); } .video-detail .video-close { background: var(--bg-surface); color: var(--text); } -.video-detail .video-close:hover { background: var(--border); } +@media (hover: hover) { .video-detail .video-close:hover { background: var(--border); } } .video-detail-body { padding: 16px 20px; @@ -4386,7 +4433,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } font: inherit; cursor: pointer; } -.video-overview-toggle:hover { text-decoration: underline; } +@media (hover: hover) { .video-overview-toggle:hover { text-decoration: underline; } } .video-overview-toggle.standalone { float: none; display: inline-block; @@ -4464,7 +4511,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } text-align: left; margin-bottom: 4px; } -.video-episode-row:hover { border-color: var(--accent); color: var(--accent); } +@media (hover: hover) { + .video-episode-row:hover { border-color: var(--accent); color: var(--accent); } +} /* The episode last started from this modal, so the one to watch next is the row under it. A border and a tint, not a colour change of the text alone, so it does not read as the hover state left behind. */ @@ -4508,7 +4557,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } border-radius: 6px; cursor: pointer; } -.video-flat-row:hover { background: var(--bg-raised); } +@media (hover: hover) { .video-flat-row:hover { background: var(--bg-raised); } } .video-flat-thumb-slot { width: 64px; height: 40px; flex-shrink: 0; } .video-flat-thumb { @@ -4560,7 +4609,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } cursor: pointer; text-align: left; } -.video-season-trigger:hover { border-color: var(--accent); color: var(--accent); } +@media (hover: hover) { + .video-season-trigger:hover { border-color: var(--accent); color: var(--accent); } +} .video-season-current { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } .video-season-caret { width: 14px; height: 14px; flex-shrink: 0; color: var(--text-dim); } @@ -4597,7 +4648,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } text-align: left; cursor: pointer; } -.video-season-option:hover { background: var(--bg-raised); } +@media (hover: hover) { .video-season-option:hover { background: var(--bg-raised); } } .video-season-option.active { color: var(--accent); } .video-season-option-count { color: var(--text-dim); font-size: 0.85em; flex-shrink: 0; } @@ -4634,7 +4685,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } text-align: left; cursor: pointer; } -.video-search-result:hover:not(:disabled) { border-color: var(--accent); background: var(--bg-raised); } +@media (hover: hover) { + .video-search-result:hover:not(:disabled) { border-color: var(--accent); background: var(--bg-raised); } +} .video-search-result:disabled { opacity: 0.5; cursor: not-allowed; } .video-search-result-thumb { width: 46px; height: 68px; flex-shrink: 0; } .video-search-result-poster { @@ -4681,7 +4734,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } border: 1px solid var(--border); transition: border-color 0.12s, transform 0.12s; } -.music-card:hover { border-color: var(--accent); transform: translateY(-2px); } +@media (hover: hover) { + .music-card:hover { border-color: var(--accent); transform: translateY(-2px); } +} .music-cover { width: 100%; @@ -4740,7 +4795,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } } .music-detail .video-title { color: var(--text); } .music-detail .video-close { background: var(--bg-surface); color: var(--text); } -.music-detail .video-close:hover { background: var(--border); } +@media (hover: hover) { .music-detail .video-close:hover { background: var(--border); } } .music-queue-now-playing { padding: 10px 20px; @@ -4814,7 +4869,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } text-align: left; cursor: pointer; } -.music-track-row:hover { background: var(--bg-raised); border-color: var(--border); } +@media (hover: hover) { + .music-track-row:hover { background: var(--bg-raised); border-color: var(--border); } +} .music-track-row.active { border-color: var(--accent); color: var(--accent); } .music-track-no { width: 24px; flex-shrink: 0; color: var(--text-dim); text-align: right; } .music-track-title { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } @@ -4955,7 +5012,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } color: var(--text-dim); cursor: pointer; } -.index-dock-hide:hover { background: var(--bg-surface); } +@media (hover: hover) { .index-dock-hide:hover { background: var(--bg-surface); } } .index-dock-hide .icon { width: 16px; height: 16px; } @media (max-width: 640px) { .index-dock-head { flex-direction: column; gap: 2px; } @@ -5009,12 +5066,6 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } cursor: pointer; position: relative; } -/* (hover: hover) — a tap on a touchscreen fires a synthetic hover that - never gets a "pointer left" to end it, so next/prev/shuffle stayed - highlighted as if still being pressed until some other element was - tapped (found live on mobile Chrome, 2026-08-26; not reproducible on - desktop, which actually has a mouse to move away with). Scoped to - devices that can genuinely hover — real hover behaviour is unaffected. */ @media (hover: hover) { .music-player-btn:hover { background: var(--bg-surface); color: var(--text); } } @@ -5140,7 +5191,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } align-items: center; justify-content: center; } -.photo-icon-btn:hover { color: var(--text); background: var(--bg-surface); } +@media (hover: hover) { + .photo-icon-btn:hover { color: var(--text); background: var(--bg-surface); } +} .photo-icon-btn-icon { width: 22px; height: 22px; } /* .tb-btn's own chevron is drawn pointing down; rotated here to read as @@ -5165,7 +5218,9 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } border: 1px solid var(--border); transition: border-color 0.12s, transform 0.12s; } -.photo-album-card:hover { border-color: var(--accent); transform: translateY(-2px); } +@media (hover: hover) { + .photo-album-card:hover { border-color: var(--accent); transform: translateY(-2px); } +} .photo-album-cover { width: 100%; @@ -5205,7 +5260,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } background: var(--bg-raised); border: 1px solid var(--border); } -.photo-tile:hover { border-color: var(--accent); } +@media (hover: hover) { .photo-tile:hover { border-color: var(--accent); } } .photo-tile { position: relative; } /* A video among the photos: a play mark and its length over the thumbnail. */ .photo-tile-video { @@ -5227,7 +5282,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } border: 0; background: rgba(0, 0, 0, 0.6); color: #fff; cursor: pointer; display: flex; align-items: center; justify-content: center; } -.photo-lightbox-play:hover { background: var(--accent); } +@media (hover: hover) { .photo-lightbox-play:hover { background: var(--accent); } } .photo-lightbox-play .icon { width: 28px; height: 28px; } .photo-tile-thumb { width: 100%; @@ -5282,7 +5337,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } align-items: center; justify-content: center; } -.photo-nav:hover { background: rgba(255, 255, 255, 0.22); } +@media (hover: hover) { .photo-nav:hover { background: rgba(255, 255, 255, 0.22); } } .photo-nav:disabled { opacity: 0.3; cursor: default; } .photo-nav-icon { width: 20px; height: 20px; } .photo-nav-prev-icon { transform: rotate(90deg); } @@ -5342,10 +5397,12 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } padding: 5px; border-radius: 4px; } -.photo-zoom-controls .photo-icon-btn:hover { background: rgba(255, 255, 255, 0.15); } +@media (hover: hover) { + .photo-zoom-controls .photo-icon-btn:hover { background: rgba(255, 255, 255, 0.15); } +} .photo-zoom-controls .photo-icon-btn.active { background: rgba(255, 255, 255, 0.18); } .photo-zoom-controls .photo-icon-btn:disabled { opacity: 0.35; cursor: default; } -.photo-zoom-controls .photo-icon-btn:disabled:hover { background: none; } +@media (hover: hover) { .photo-zoom-controls .photo-icon-btn:disabled:hover { background: none; } } .photo-zoom-controls .photo-icon-btn-icon { width: 18px; height: 18px; } .photo-icon-btn-text { font-size: 0.72em; font-weight: 600; padding: 5px 8px; } .photo-zoom-percent { @@ -5625,7 +5682,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } text-align: left; cursor: pointer; } -.ctx-menu-item:hover:not(:disabled) { background: var(--sidebar-hover); } +@media (hover: hover) { .ctx-menu-item:hover:not(:disabled) { background: var(--sidebar-hover); } } .ctx-menu-item:disabled { color: var(--text-dim); cursor: default; } .ctx-menu-item.danger { color: var(--error); } @@ -5673,10 +5730,12 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } transition: opacity 0.12s, background 0.12s; } .ctx-dots svg { width: 18px; height: 18px; } -.ctx-dots:hover { background: var(--sidebar-hover); color: var(--text); } +@media (hover: hover) { .ctx-dots:hover { background: var(--sidebar-hover); color: var(--text); } } .ctx-dots:focus-visible { opacity: 1; outline: 2px solid var(--border-focus); } -.music-card:hover .ctx-dots, -.music-track-row:hover .ctx-dots { opacity: 1; } +@media (hover: hover) { + .music-card:hover .ctx-dots, + .music-track-row:hover .ctx-dots { opacity: 1; } +} @media (hover: none), (pointer: coarse) { .ctx-dots { opacity: 1; } @@ -5691,7 +5750,7 @@ h2 .gn-owner, h3 .gn-owner { font-size: 0.55em; } color: #fff; backdrop-filter: blur(2px); } -.music-card-dots:hover { background: rgba(0, 0, 0, 0.75); color: #fff; } +@media (hover: hover) { .music-card-dots:hover { background: rgba(0, 0, 0, 0.75); color: #fff; } } /* A flat folder row expands on click; its menu button sits in the same row and must not take the whole width with it. */ diff --git a/packages/meshbay-hub/src/meshbay_hub/static/transport.js b/packages/meshbay-hub/src/meshbay_hub/static/transport.js index a07f431..15c7b09 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/transport.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/transport.js @@ -447,6 +447,16 @@ const LEASE_WATCHDOG_MS = 60000; // one sender: 1 connection 5 to 6 MB/s, 4 about 19, 8 between 27 and 34, and // 16 no more than that (the node's sender is one Python process). const DOWNLOAD_LANES = 8; + +// What a connection sends while it is opening its session, before the node +// knows who is on it: the node answers anything else with "Handshake +// required" (webrtc/dispatch.py), and the same list is what it accepts there. +// While a reconnect runs these are the only messages that go out; everything +// else waits for the session (see `_sendAndWait`). +const SESSION_OPENING = new Set([ + 'handshake', 'handshake_response', 'keypair_bundle_fetch', 'gek_bundle_fetch', + 'join_request', 'device_add_request', 'device_hello', +]); // A lane left unused this long is closed: each one is a peer connection the // node holds open. const LANE_IDLE_MS = 30000; @@ -594,6 +604,10 @@ class MeshBayTransport { this._lanesTimer = 0; this._laneRetryAt = 0; this._lane = false; + // True from the end of connect() — the node has authenticated this side — + // until the connection is lost. Not `_connected`, which is true from the + // moment the channel opens, a whole handshake earlier. + this._sessionUp = false; // Set once close() runs — stops the automatic reconnect from firing on a // connection the caller tore down on purpose (leaving the group, page // unload), which would otherwise race back in right as everything else @@ -810,6 +824,7 @@ class MeshBayTransport { // reconnect that never got as far as announcing a device still looked, to // the composer, exactly like one that had. this._setDevicePk('', 'new connection'); + this._sessionUp = false; this._pc = new RTCPeerConnection({ iceServers: await iceServers() }); this._channel = this._pc.createDataChannel('mnp', { ordered: true }); @@ -836,15 +851,23 @@ class MeshBayTransport { channelReady.catch(() => { /* the awaiter below reports it */ }); this._channel.onmessage = (event) => this._onMessage(event.data); + const channel = this._channel; this._channel.onclose = (ev) => { - console.warn('[MeshBay] DataChannel closed', this._channel?.readyState, ev); + console.warn('[MeshBay] DataChannel closed', channel.readyState, ev); trace('channel_close', { - readyState: this._channel?.readyState, + readyState: channel.readyState, pc: this._pc?.connectionState, ice: this._pc?.iceConnectionState, }); - this._connected = false; if (channelReject) channelReject(new Error('DataChannel closed')); + // A reconnect closes the old channel before it opens the next, and the + // event comes when it comes: from a channel that is no longer this + // connection's, it is news about nothing. Acted on, it marked the new + // session closed and failed whatever it had in flight, its own + // handshake included. + if (channel !== this._channel) return; + this._connected = false; + this._sessionUp = false; for (const [, p] of this._pending) p.reject(new Error('DataChannel closed')); this._pending.clear(); }; @@ -876,6 +899,7 @@ class MeshBayTransport { // sit out its own timeout instead of failing fast. if (pc.connectionState === 'failed' && pc === this._pc && !this._closed) { this._connected = false; + this._sessionUp = false; this._reconnect(); const err = new Error('WebRTC connection lost'); err.name = 'TransportLostError'; @@ -1303,6 +1327,7 @@ class MeshBayTransport { console.warn('[MeshBay] device_hello failed — chat will not work:', e); }); + this._sessionUp = true; return ack; } @@ -1757,6 +1782,7 @@ class MeshBayTransport { if (this._channel) this._channel.close(); if (this._pc) this._pc.close(); this._connected = false; + this._sessionUp = false; for (const [, p] of this._pending) p.reject(new Error('Transport closed')); this._pending.clear(); } @@ -1823,12 +1849,25 @@ class MeshBayTransport { async _sendAndWait(obj, timeoutMs = 30000) { // A reconnect already in flight (see _reconnectLoop) means the channel - // this would send on is the one just declared dead. `_inReconnectAttempt` - // excludes the handshake connect() itself makes while reconnecting — that - // call runs *inside* this same _reconnectPromise, which cannot resolve - // until it returns, so waiting on it here would just be waiting on - // itself for the full 6s, on every step of the handshake, every time. - if (!this._inReconnectAttempt) await this.waitForReconnect(6000); + // this would send on is the one just declared dead. The handshake that + // connect() sends while reconnecting runs *inside* that same + // _reconnectPromise, which cannot resolve until it returns, so waiting on + // it here would be waiting on itself — those messages go straight out. + // + // Only those. This used to let *every* message through while the attempt + // ran, so a download's chunk asked for again in that window went out on a + // channel that had opened and not finished its handshake: the node + // answered "Handshake required", which no retry treats as a dropped + // connection, and a download was failed by the reconnect meant to save it. + const opening = SESSION_OPENING.has(obj.type); + if (!(opening && this._inReconnectAttempt)) await this.waitForReconnect(6000); + if (!opening && !this._sessionUp) { + // No session to send it on: what the caller sees on a dropped connection, + // and what the download path waits out (`_fetchChunkResilient`). + const err = new Error('WebRTC connection lost'); + err.name = 'TransportLostError'; + throw err; + } return new Promise((resolve, reject) => { const id = this._seqId++; const timeout = setTimeout(() => { diff --git a/packages/meshbay-hub/tests/harness/reconnect_race_harness.mjs b/packages/meshbay-hub/tests/harness/reconnect_race_harness.mjs new file mode 100644 index 0000000..0f1cd97 --- /dev/null +++ b/packages/meshbay-hub/tests/harness/reconnect_race_harness.mjs @@ -0,0 +1,230 @@ +// Run the SHIPPED download over the SHIPPED reconnect, against a node that +// refuses what comes before the handshake, and see whether the download lives. +// +// transport.js is executed whole — connect(), _reconnectLoop, _sendAndWait, +// _dispatch and the framing are the shipped code — and the pipeline is lifted +// out of file-utils.js as text. What is modelled is the environment: a clock, +// a peer connection that opens its channel some time after the answer, a hub +// that answers the offer and the token, and a node that answers anything but +// the opening messages with "Handshake required" until the proof is in, as +// webrtc/dispatch.py does. Cryptography is not under test here: the checks +// that would refuse a peer are stubbed to accept. +// +// The scenario is the one that failed live: the node stops answering, the +// chunks in flight time out, and their retries come due while the reconnect's +// new channel is open and its handshake is not finished. +// +// Usage: node reconnect_race_harness.mjs <file-utils.js> <transport sources> <json> +import { readFileSync } from 'fs'; +import { delimiter } from 'path'; + +const utils = readFileSync(process.argv[2], 'utf8'); +const transportSrc = process.argv[3].split(delimiter) + .map((p) => readFileSync(p, 'utf8')).join('\n'); +const { + chunks = 400, + chunkMs = 100, + hangAt = 5000, // the node stops answering the first connection + failAfter = 30200, // ...and the peer connection reports `failed` this much later + openMs = 100, // answer → channel open + rttMs = 300, // one request → its reply, during the handshake + lateCloseMs = 0, // >0: the replaced channel's close event fires this late + runUntil = 300000, +} = JSON.parse(process.argv[4] || '{}'); + +for (const level of ['log', 'warn', 'error', 'info', 'debug']) { + console[level] = (...args) => process.stderr.write(args.join(' ') + '\n'); +} +process.on('unhandledRejection', () => {}); + +// ── The clock ──────────────────────────────────────────────────────────────── + +let now = 0; +let nextId = 1; +let timers = []; +globalThis.setTimeout = (fn, ms) => { + const t = { at: now + (ms || 0), fn, id: nextId++ }; + timers.push(t); + return t.id; +}; +globalThis.setInterval = (fn, ms) => { + const t = { at: now + ms, fn, id: nextId++, every: ms }; + timers.push(t); + return t.id; +}; +globalThis.clearTimeout = (id) => { timers = timers.filter((t) => t.id !== id); }; +globalThis.clearInterval = globalThis.clearTimeout; +Date.now = () => now; +const flush = async () => { for (let i = 0; i < 50; i++) await Promise.resolve(); }; +async function run(until) { + await flush(); + while (timers.length) { + const due = timers.reduce((a, b) => (b.at < a.at ? b : a)); + if (due.at > until) break; + if (due.every) due.at += due.every; + else timers = timers.filter((t) => t !== due); + now = Math.max(now, due.every ? due.at - due.every : due.at); + due.fn(); + await flush(); + } + now = until; +} + +// ── The page around the transport ──────────────────────────────────────────── + +globalThis.window = globalThis; +globalThis.addEventListener = () => {}; +globalThis.removeEventListener = () => {}; +globalThis.location = { hash: '' }; +globalThis.document = { + addEventListener() {}, removeEventListener() {}, visibilityState: 'visible', +}; +globalThis.localStorage = { getItem: () => null, setItem() {}, removeItem() {} }; + +const b64 = (u8) => Buffer.from(u8).toString('base64'); +window.MeshBayCrypto = { + b64encode: (u8) => b64(u8), + b64decode: (s) => new Uint8Array(Buffer.from(String(s), 'base64')), + verifyNodeSignature: async () => true, + challengeTranscript: () => new Uint8Array(1), + webrtcBinding: () => new Uint8Array(1), + handshakeProof: async () => new Uint8Array([7]), + handshakeTranscript: () => new Uint8Array(1), + constantTimeEqual: () => true, + openGroup: async () => window.__codec.msgpack_encode({ chat_epoch: 1 }), + decryptChunkBin: async (_k, _f, index) => new Uint8Array([index & 255]), +}; + +const SDP = 'v=0\r\na=fingerprint:sha-256 AA:BB\r\n'; +globalThis.fetch = async (url) => ({ + ok: true, status: 200, headers: { get: () => null }, + json: async () => (url.endsWith('/mnp-token') ? { mnp_token: 't' } : { sdp: SDP }), +}); + +// ── The node ───────────────────────────────────────────────────────────────── + +const OPENING = new Set(['handshake', 'handshake_response', 'keypair_bundle_fetch', + 'gek_bundle_fetch', 'join_request', 'device_add_request', 'device_hello']); +let refusedBeforeHandshake = 0; +let served = 0; +const pcs = []; + +class NodeSession { + constructor(channel) { this.channel = channel; this.authed = false; this.hung = false; } + reply(obj, ms) { + setTimeout(() => { + if (this.hung || this.channel.readyState !== 'open') return; + const body = window.__codec.msgpack_encode(obj); + const frame = new Uint8Array(4 + body.byteLength); + new DataView(frame.buffer).setUint32(0, body.byteLength, false); + frame.set(body, 4); + this.channel.onmessage({ data: frame.buffer }); + }, ms); + } + receive(frame) { + if (this.hung) return; + const msg = window.__codec.msgpack_decode(frame.slice(4)); + const req_id = msg.req_id; + if (!this.authed && !OPENING.has(msg.type)) { + refusedBeforeHandshake += 1; + return this.reply({ type: 'error', detail: 'Handshake required', req_id }, 1); + } + switch (msg.type) { + case 'handshake': + return this.reply({ type: 'handshake_challenge', v: '99.0', v_min: '0.0', + nonce: b64(new Uint8Array(32)), node_pk: 'pk', sig: 's', req_id }, rttMs); + case 'handshake_response': + setTimeout(() => { this.authed = true; }, rttMs); + return this.reply({ type: 'handshake_ack', proof: 'AA==', node_pk: 'pk', sig: 's', + nonce: 'n', ct: 'c', req_id }, rttMs); + case 'device_hello': + return this.reply({ type: 'device_hello_ack', req_id }, rttMs); + case 'file_req': + served += 1; + return this.reply({ type: 'file_chunk', file_id: msg.file_id, + chunk_index: msg.chunk_index, nonce: 1, ct: new Uint8Array([1]), req_id }, chunkMs); + default: + return undefined; + } + } +} + +class FakeChannel { + constructor() { this.readyState = 'connecting'; this.binaryType = 'arraybuffer'; } + send(frame) { + if (this.readyState !== 'open') throw new Error('InvalidStateError'); + this.session.receive(frame); + } + close() { + if (this.readyState === 'closed') return; + this.readyState = 'closed'; + if (this.onclose) setTimeout(() => this.onclose({}), lateCloseMs); + } +} + +class FakePC { + constructor() { + this.connectionState = 'new'; + this.iceConnectionState = 'new'; + this.iceGatheringState = 'complete'; + pcs.push(this); + } + createDataChannel() { this.channel = new FakeChannel(); return this.channel; } + async createOffer() { return { type: 'offer', sdp: SDP }; } + async setLocalDescription(d) { this.localDescription = d; } + async setRemoteDescription() { + setTimeout(() => { + if (this.connectionState === 'closed') return; + this.channel.session = new NodeSession(this.channel); + this.channel.readyState = 'open'; + this.connectionState = 'connected'; + if (this.channel.onopen) this.channel.onopen(); + }, openMs); + } + close() { + this.connectionState = 'closed'; + if (this.channel) this.channel.close(); + } +} +globalThis.RTCPeerConnection = FakePC; + +// ── The shipped transport and pipeline ─────────────────────────────────────── + +new Function(`${transportSrc}\nwindow.__codec = { msgpack_encode, msgpack_decode };`)(); +const Transport = window.MeshBayTransport; + +const constant = (name) => utils.match(new RegExp(`^const ${name} = [^;]+;`, 'm'))[0]; +const from = utils.indexOf('const CHUNK_RETRY_ATTEMPTS'); +const to = utils.indexOf('async function downloadEntry'); +const { pipelinedDownload } = new Function('window', 't', ` + ${constant('PIPELINE_WINDOW')} + ${constant('LANE_MIN_CHUNKS')} + ${utils.slice(from, utils.lastIndexOf('/**', to))} + return { pipelinedDownload };`)(window, (key) => key); + +const identity = { pkEdB64: 'ed', signAs: async () => 'sig', native: false }; +const main = new Transport('https://hub.invalid', 'token'); +let outcome = 'pending'; +let reconnected = 0; +main.addReconnectListener(() => { reconnected += 1; }); +const written = []; +const writable = { write: async (bytes) => { written.push(bytes[0]); } }; + +main.connect('node', 'token', 'group', new Uint8Array(32), identity, null, 'u', 'uid', + undefined, undefined, '', 'pk') + .then(() => pipelinedDownload(main, 'gek', 'f'.repeat(64), chunks, null, writable, null, '')) + .then(() => { outcome = 'done'; }, (err) => { outcome = `failed: ${err.message}`; }); + +await run(hangAt); +pcs[0].channel.session.hung = true; +await run(hangAt + failAfter); +pcs[0].connectionState = 'failed'; +pcs[0].onconnectionstatechange(); +await run(runUntil); + +process.stdout.write(JSON.stringify({ + outcome, reconnected, refusedBeforeHandshake, served, + written: written.length, + inOrder: written.every((b, i) => b === (i & 255)), + connections: pcs.length, +})); diff --git a/packages/meshbay-hub/tests/test_hover_scoped.py b/packages/meshbay-hub/tests/test_hover_scoped.py new file mode 100644 index 0000000..f28dacd --- /dev/null +++ b/packages/meshbay-hub/tests/test_hover_scoped.py @@ -0,0 +1,62 @@ +""" +Every `:hover` rule in style.css is inside `@media (hover: hover)`. + +A tap on a touchscreen puts the element in `:hover` and nothing takes it out +until another element is tapped, so a button tapped once kept its hover colour +as if still pressed. It was fixed for the music player's buttons alone, then +reported again on the pager arrows, the nav hamburger and the rest: a rule per +button only moves the bug to the next button. This holds the whole file. +""" + +from pathlib import Path + +import pytest + +STYLE = (Path(__file__).resolve().parents[1] / "src" / "meshbay_hub" + / "static" / "style.css") + +pytestmark = pytest.mark.skipif(not STYLE.exists(), + reason="the stylesheet is not in this checkout") + + +def _hover_rules(source: str) -> list[tuple[int, list[str], str]]: + """(line, enclosing at-rule preludes, selector) for each rule naming :hover.""" + found = [] + stack: list[str] = [] + start = 0 + i = 0 + while i < len(source): + if source.startswith("/*", i): + i = source.index("*/", i) + 2 + start = i + continue + c = source[i] + if c == "{": + prelude = source[start:i].strip() + if ":hover" in prelude and not prelude.startswith("@"): + found.append((source.count("\n", 0, i) + 1, list(stack), prelude)) + stack.append(prelude) + start = i + 1 + elif c == "}": + stack.pop() + start = i + 1 + elif c == ";": + start = i + 1 + i += 1 + return found + + +def test_every_hover_rule_is_scoped_to_devices_that_hover(): + rules = _hover_rules(STYLE.read_text(encoding="utf-8")) + assert rules, "no :hover rule found, did the parser break?" + loose = [f"line {line}: {sel}" for line, stack, sel in rules + if not any("(hover: hover)" in at for at in stack)] + assert not loose, ("wrap these in @media (hover: hover), a tap leaves them " + "stuck on a touchscreen:\n" + "\n".join(loose)) + + +def test_the_parser_catches_a_loose_rule(): + rules = _hover_rules("/* a { } */ .a:hover { color: red; }\n" + "@media (hover: hover) { .b:hover { color: red; } }") + assert [(sel, bool(stack)) for _, stack, sel in rules] == [ + (".a:hover", False), (".b:hover", True)] diff --git a/packages/meshbay-hub/tests/test_reconnect_race.py b/packages/meshbay-hub/tests/test_reconnect_race.py new file mode 100644 index 0000000..01c5182 --- /dev/null +++ b/packages/meshbay-hub/tests/test_reconnect_race.py @@ -0,0 +1,93 @@ +""" +A reconnect must not fail the download it is reconnecting for. + +Reported live on 0.20: a large download from a node under load stopped with +"Handshake required" in the transfers panel. `_sendAndWait` let every message +through while the reconnect's own connect() ran, because the handshake it sends +has to go through; so a chunk whose retry came due in that window went out on a +channel that had opened and not finished its handshake. The node answered +"Handshake required" (webrtc/dispatch.py), which no retry treats as a dropped +connection. Now only the opening messages pass, and anything else waits for the +session or fails as a dropped connection, which the download waits out. + +The replaced channel's close event is the second half: it reached the new +connection, marked it closed and failed what it had in flight. + +These run the shipped transport and pipeline against a node that refuses what +comes before the handshake: harness/reconnect_race_harness.mjs. +""" + +import json +import os +import re +import shutil +import subprocess +from pathlib import Path + +import pytest +from spa_source import classic_scripts, transport_source + +STATIC = Path(__file__).resolve().parents[1] / "src" / "meshbay_hub" / "static" +FILE_UTILS = STATIC / "file-utils.js" +HARNESS = Path(__file__).parent / "harness" / "reconnect_race_harness.mjs" +DISPATCH = (Path(__file__).resolve().parents[2] / "meshbay-node" / "src" / "meshbay_node" + / "transport" / "webrtc" / "dispatch.py") + +pytestmark = pytest.mark.skipif( + shutil.which("node") is None or not FILE_UTILS.exists(), + reason="node or the SPA sources are not available") + + +def _run(**cfg) -> dict: + transport = os.pathsep.join( + str(STATIC / name) for name in classic_scripts() if name.startswith("transport")) + proc = subprocess.run( + ["node", str(HARNESS), str(FILE_UTILS), transport, json.dumps(cfg)], + capture_output=True, text=True, encoding="utf-8", timeout=120) + assert proc.returncode == 0, proc.stderr + return json.loads(proc.stdout) + + +def test_retries_that_come_due_during_the_handshake_wait_for_it(): + # The chunks in flight time out just before the connection is declared + # failed, and their retries land while the new channel is handshaking. + out = _run() + assert out["reconnected"] == 1 and out["connections"] == 2 + assert out["refusedBeforeHandshake"] == 0, "a chunk was asked for before the handshake" + assert out["outcome"] == "done" + assert out["written"] == 400 and out["inOrder"] + + +@pytest.mark.parametrize("cfg", [ + {"failAfter": 20000}, # failure declared before anything times out + {"failAfter": 31000, "rttMs": 800}, # a slow handshake + {"openMs": 2500}, # a slow channel +]) +def test_the_download_survives_other_timings(cfg): + out = _run(**cfg) + assert out["refusedBeforeHandshake"] == 0 + assert out["outcome"] == "done" and out["written"] == 400 and out["inOrder"] + + +@pytest.mark.parametrize("late", [800, 1500, 3000]) +def test_the_replaced_channels_close_event_is_ignored(late): + out = _run(lateCloseMs=late) + assert out["outcome"] == "done" and out["written"] == 400 + + +def _node_opening_types() -> set[str]: + """What the node answers before the handshake, read from its dispatch.""" + from meshbay_common.protocol import MNP + src = DISPATCH.read_text(encoding="utf-8") + before = src[src.index("def _dispatch_message"):src.index("elif self._user_id is None")] + names = set(re.findall(r"MNP\.([A-Z_]+)", before)) + return {getattr(MNP, n) for n in names} + + +def test_the_opening_messages_are_the_ones_the_node_takes_before_the_handshake(): + m = re.search(r"const SESSION_OPENING = new Set\(\[(.*?)\]\);", transport_source(), re.S) + client = set(re.findall(r"'([a-z_]+)'", m.group(1))) + # device_hello is the one connect() sends after the proof and before it + # returns: the node takes it then, and the session is not up until it has. + assert client - _node_opening_types() == {"device_hello"} + assert _node_opening_types() <= client diff --git a/packages/meshbay-node/src/meshbay_node/config.py b/packages/meshbay-node/src/meshbay_node/config.py index c5596f1..d46ab8c 100644 --- a/packages/meshbay-node/src/meshbay_node/config.py +++ b/packages/meshbay-node/src/meshbay_node/config.py @@ -74,13 +74,14 @@ max_concurrent_uploads = 8 # the partial file is deleted. max_upload_gb = 8 -# How many processes carry the connections of members. At 1 the node does it +# How many processes carry the connections of members. Left unset: one per core +# of this machine, up to four, about 85 MB of memory each. At 1 the node does it # all in one process, on one core: about 35 MB/s summed over every transfer in -# progress. Above 1 the node starts that many processes for the connections and -# keeps the rest for itself, so transfers use that many cores, plus part of one -# for the node. Takes effect for new connections, without a restart. Not above -# the number of cores of this machine. -connection_processes = 1 +# progress on Linux, and much less on Windows. Above 1 the node starts that many +# processes for the connections and keeps the rest for itself, so transfers use +# that many cores, plus part of one for the node. Takes effect for new +# connections, without a restart. Not above the number of cores of this machine. +# connection_processes = 4 # HEVC sources have no browser decoder on most platforms, so streaming one is # transcoded to H264 rather than the usual free copy — real CPU per viewer. @@ -202,8 +203,9 @@ class NodeConfig: # §15.3), so this is what stands between a writable root and a full disk. max_upload_gb: float = 8.0 # How many processes hold the peer connections. One means this process - # holds them itself, as it always did. See transport/webrtc/workers.py. - connection_processes: int = 1 + # holds them itself. See transport/webrtc/workers.py, and + # `default_connection_processes` for why it is not one by default. + connection_processes: int = field(default_factory=lambda: default_connection_processes()) # HEVC (and any future codec in media_probe.py's # BROWSER_INCOMPATIBLE_VIDEO_CODECS) has no decoder in most browsers, so # streaming it needs a real re-encode to H264 rather than the usual free @@ -305,6 +307,23 @@ def max_processes() -> int: return max(1, os.cpu_count() or 1) +# Above four, a single download no longer gains (docs/MESHBAY_DESIGN.md §5.5: +# 131 MB/s at four, 150 at eight with the receiving browser the limit), and each +# process is 85 MB held for the life of the node. +DEFAULT_CONNECTION_PROCESSES_MAX = 4 + + +def default_connection_processes() -> int: + """ + One per core, up to four. It was 1, which starts no process at all: the + several-cores path was measured, shipped and reached by no node whose + operator had not found the setting, and the lanes of a download all landed + on the node's one core — on Windows, where Python's UDP costs most, a + download over eight connections was as slow as over one. + """ + return min(DEFAULT_CONNECTION_PROCESSES_MAX, max_processes()) + + def _positive(value: object, default: int, name: str) -> int: """A count that must be at least one, or the default with a word about it. diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc/core.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc/core.py index d197044..07820b2 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc/core.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc/core.py @@ -15,6 +15,7 @@ from meshbay_common.protocol import MNP from meshbay_node import ops from meshbay_node import transfers as transfers_mod from meshbay_node.transport.webrtc import ( + proactor_udp, # noqa: F401 (patches asyncio's Windows loop on import) sctp_fix, # noqa: F401 (patches aiortc on import) sctp_stats, # noqa: F401 (patches aiortc on import) udp_buffer, # noqa: F401 (patches aioice on import) diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc/proactor_udp.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc/proactor_udp.py new file mode 100644 index 0000000..38cb97c --- /dev/null +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc/proactor_udp.py @@ -0,0 +1,139 @@ +"""Datagrams moved straight through the socket on Windows' event loop, applied +on import. + +The node runs on the ProactorEventLoop, the one that can start ffmpeg there +(`platform.configure_event_loop`), and so do its connection processes. Its +datagram transport keeps one overlapped operation in each direction and starts +the next only from the completion callback of the one before, which runs on a +later turn of the loop. So a socket sends **one datagram per turn of the loop, +and receives one**, however many are waiting, and a turn lasts as long as +everything else the loop has to do: the other connections, reading the file, +the hub. The busier the node, the slower every connection, with cores to spare. + +Measured on loopback, 1200-byte datagrams, with the loop otherwise busy 0.5 ms +a turn: 1,928 sent a second (2.3 MB/s), and 1,954 received, the kernel dropping +87 % of what arrived; straight through the socket, 88,000 sent, and every +datagram received at the 20,000 a second it was sent at. In the node holding the +connections itself, a browser's download over eight connections ran at 9.1 MB/s +with the node at 45 % of a core and an SCTP round trip of 745 ms on loopback: +that is the queue in front of each socket. In a connection process serving the +command-line client, 42 % of the time was `IocpProactor.sendto`. Linux never +paid for the sending half: the selector loop's transport tries the socket first +and queues only what it refuses. + +Sending: a datagram goes out with a non-blocking `sendto` when nothing is queued +and no overlapped send is pending, so order is kept; anything else, the socket +refusing included, goes through the proactor's own queue as before. + +Receiving: once an overlapped read completes, its datagram is delivered, then +whatever else the socket holds is read with non-blocking `recvfrom`, and only +then is the next overlapped read posted. Never while one is pending, which +could hand a later datagram to the completion port ahead of the ones read here. +The overlapped path turns the ICMP "port unreachable" a send to a silent +address brings back into an empty read; read directly it is a +ConnectionResetError, and it is skipped the same way. + +No effect off Windows, where the class is never used. +""" + +import sys +from asyncio import exceptions, proactor_events + +_Transport = proactor_events._ProactorDatagramTransport +_proactor_sendto = _Transport.sendto + +# Datagrams read straight from the socket after one completion, at most: a +# socket under a flood still lets the loop turn. +DRAIN_MAX = 256 + + +def sendto(self, data, addr=None) -> None: + if (self._write_fut is None and not self._buffer and not self._conn_lost + and isinstance(data, (bytes, bytearray, memoryview)) and data + and (self._address is None or addr in (None, self._address))): + try: + if self._address is not None: + self._sock.send(data) + else: + self._sock.sendto(data, addr) + return + except (BlockingIOError, InterruptedError): + pass + except OSError as exc: + # What the overlapped path does with a failed send. + self._protocol.error_received(exc) + return + _proactor_sendto(self, data, addr) + + +def _deliver(self, data, addr) -> None: + """To the protocol, and whatever it raises reported as a callback's would + be: in CPython's order the next read was posted first, and an exception + here must not end the reading either.""" + try: + self._protocol.datagram_received(data, addr) + except (SystemExit, KeyboardInterrupt): + raise + except BaseException as exc: + self._loop.call_exception_handler({ + "message": "Exception in datagram_received", "exception": exc, + "transport": self, "protocol": self._protocol}) + + +def _drain(self) -> None: + for _ in range(DRAIN_MAX): + if self._conn_lost or self._closing: + return + try: + if self._address is not None: + data, addr = self._sock.recv(self.max_size), self._address + else: + data, addr = self._sock.recvfrom(self.max_size) + except (BlockingIOError, InterruptedError): + return + except ConnectionResetError: + continue + except OSError as exc: + # The overlapped read posted next meets it again, and reports it + # the proactor's way. + self._protocol.error_received(exc) + return + if data: + _deliver(self, data, addr) + + +def _loop_reading(self, fut=None) -> None: + """CPython's own (3.12), but a completed read is delivered and the socket + drained before the next overlapped read is posted, not after.""" + try: + if self._conn_lost: + return + assert self._read_fut is fut or (self._read_fut is None and self._closing) + self._read_fut = None + if fut is not None: + res = fut.result() + if self._closing: + return + data, addr = (res, self._address) if self._address is not None else res + if data: + _deliver(self, data, addr) + _drain(self) + if self._conn_lost: + return + if self._address is not None: + self._read_fut = self._loop._proactor.recv(self._sock, self.max_size) + else: + self._read_fut = self._loop._proactor.recvfrom(self._sock, self.max_size) + except OSError as exc: + self._protocol.error_received(exc) + except exceptions.CancelledError: + if not self._closing: + raise + else: + if self._read_fut is not None: + self._read_fut.add_done_callback(self._loop_reading) + + +if sys.platform == "win32": + _Transport.sendto = sendto + _Transport._loop_reading = _loop_reading diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc/sctp_fix.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc/sctp_fix.py index 0d45a7d..a7e6f86 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc/sctp_fix.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc/sctp_fix.py @@ -1,12 +1,13 @@ -"""A fix to aiortc's SCTP sender, applied on import (aiortc 1.15). +"""Two fixes to aiortc's SCTP stack, applied on import (aiortc 1.15). -aiortc counts the bytes in flight and sends nothing new while that count is at -or above the congestion window. A chunk the peer acknowledged in a gap block is -marked `_acked` and taken out of the count. If the T3 timer then expires, every -chunk still queued is sent again, that one included, and goes back into the -count with `_acked` still set. When the cumulative ack finally covers it, the -SACK handler skips it because it reads as acknowledged already, and its bytes -stay counted for the life of the association. +**The sender leaked bytes in flight.** aiortc counts the bytes in flight and +sends nothing new while that count is at or above the congestion window. A chunk +the peer acknowledged in a gap block is marked `_acked` and taken out of the +count. If the T3 timer then expires, every chunk still queued is sent again, +that one included, and goes back into the count with `_acked` still set. When +the cumulative ack finally covers it, the SACK handler skips it because it reads +as acknowledged already, and its bytes stay counted for the life of the +association. Each such loss leaks a chunk. On a lossy path (a phone on Wi-Fi) the leak grows past the window, which T3 itself keeps small. From then on the node sends @@ -14,14 +15,49 @@ nothing on that connection: nothing is outstanding, so no timer runs to unstick it, and the association still reads as established. The peer keeps talking and the node keeps answering into a queue that never moves. Live symptom: the photo backup stalled after a few dozen photos, its slot granted and the grant never -delivered (docs/MESHBAY_DESIGN.md §9.12). +delivered (docs/MESHBAY_DESIGN.md §9.12). The fix is the invariant the code +assumed: a chunk being sent again is in flight, so it is not acknowledged. -The fix is the invariant the code assumed: a chunk being sent again is in -flight, so it is not acknowledged. `test_sctp_fix.py` replays the sequence -against aiortc itself. +**The receiver reassembled in quadratic time.** After every DATA chunk, +`InboundStream.pop_messages` walks the reassembly list from its start, through +every fragment of the message still being assembled. A message of n fragments +costs n²/2 steps, and a 1 MiB `file_chunk` is about 870 fragments: measured on a +receiving aiortc, 153 million steps for 400 MB, two thirds of its CPU, and the +whole of why it could not take more than 20 MB/s on loopback. No message can +come out of a reassembly holding no last fragment, so the walk is skipped until +one is there; when one is, the walk is aiortc's own, unchanged. + +The node receives uploads in 48 KB messages, where the walk is short; the +command-line client receives 1 MiB ones and carries the same fix +(`meshbay_cli/client/sctp_fix.py`). + +**The receiver acknowledged every packet.** RFC 4960 §6.2 asks for one SACK +every second packet, and Chromium does that; aiortc sent one at the end of +every packet of data. Each SACK costs its receiver a DTLS decryption and a parse +on the one core that also does everything else — on a node sending a download +to the command-line client, more time than sending the data. In an upload the +node is the receiver and pays the other half: the encryption and the send of a +SACK per packet. A SACK still goes at once when something is out of order or +duplicated (the loss signal the sender is waiting for) or the packet carried +anything other than data; a lone packet is acknowledged within `SACK_DELAY_S`. + +`test_sctp_fix.py` replays all three against aiortc itself. """ -from aiortc.rtcsctptransport import RTCSctpTransport +import asyncio +import contextlib + +from aiortc.rtcsctptransport import ( + SCTP_DATA_LAST_FRAG, + DataChunk, + InboundStream, + InitChunk, + RTCSctpTransport, + parse_packet, +) + +# RFC 4960 §6.2: at most 500 ms; 200 is what implementations use. +SACK_DELAY_S = 0.2 def _flight_size_increase(self, chunk) -> None: @@ -30,3 +66,82 @@ def _flight_size_increase(self, chunk) -> None: RTCSctpTransport._flight_size_increase = _flight_size_increase + + +_add_chunk = InboundStream.add_chunk +_pop_messages = InboundStream.pop_messages +_prune_chunks = InboundStream.prune_chunks + + +def add_chunk(self, chunk) -> None: + _add_chunk(self, chunk) + if chunk.flags & SCTP_DATA_LAST_FRAG: + self._last_frags = getattr(self, "_last_frags", 0) + 1 + + +def pop_messages(self): + # Only ever too high (a pruned last fragment), never too low: a message + # leaves the reassembly with exactly one last fragment, and only here. + if not getattr(self, "_last_frags", 0): + return + for message in _pop_messages(self): + self._last_frags -= 1 + yield message + + +def prune_chunks(self, tsn: int) -> int: + size = _prune_chunks(self, tsn) + self._last_frags = sum(1 for c in self.reassembly if c.flags & SCTP_DATA_LAST_FRAG) + return size + + +InboundStream.add_chunk = add_chunk +InboundStream.pop_messages = pop_messages +InboundStream.prune_chunks = prune_chunks + + +async def _handle_data(self, data: bytes) -> None: + """aiortc's own, down to the SACK at the end: that one may wait for a + second packet.""" + try: + _, _, verification_tag, chunks = parse_packet(data) + except ValueError: + return + if any(isinstance(x, InitChunk) for x in chunks): + assert len(chunks) == 1 + expected_tag = 0 + else: + expected_tag = self._local_verification_tag + if verification_tag != expected_tag: + return + for chunk in chunks: + await self._receive_chunk(chunk) + if not self._sack_needed: + return + self._sack_packets = getattr(self, "_sack_packets", 0) + 1 + if (self._sack_packets >= 2 or self._sack_misordered or self._sack_duplicates + or not all(isinstance(c, DataChunk) for c in chunks)): + await _sack_now(self) + elif getattr(self, "_sack_timer", None) is None: + self._sack_timer = self._loop.call_later(SACK_DELAY_S, _sack_later, self) + + +async def _sack_now(self) -> None: + timer = getattr(self, "_sack_timer", None) + if timer is not None: + timer.cancel() + self._sack_timer = None + self._sack_packets = 0 + if self._sack_needed and self._association_state != self.State.CLOSED: + # The DTLS transport may be gone under a timer that fired late. + with contextlib.suppress(ConnectionError): + await self._send_sack() + + +def _sack_later(self) -> None: + self._sack_timer = None + # Held: a task nobody references can be collected before it runs. + self._sack_task = asyncio.ensure_future(_sack_now(self)) + + +RTCSctpTransport._handle_data = _handle_data diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc/worker.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc/worker.py index 832428a..c933aba 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc/worker.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc/worker.py @@ -224,9 +224,10 @@ async def _main() -> None: root.handlers[:] = [_Forward(writer.write)] root.setLevel(hello.get("level", logging.INFO)) - # Patches aiortc on import, as the node does for the connections it holds - # itself. + # Patches aiortc, and asyncio's Windows loop, on import, as the node does + # for the connections it holds itself. from meshbay_node.transport.webrtc import ( # noqa: F401 + proactor_udp, sctp_fix, sctp_stats, udp_buffer, diff --git a/packages/meshbay-node/tests/test_connection_processes.py b/packages/meshbay-node/tests/test_connection_processes.py index ab9dfd3..778da5a 100644 --- a/packages/meshbay-node/tests/test_connection_processes.py +++ b/packages/meshbay-node/tests/test_connection_processes.py @@ -13,6 +13,7 @@ not carry. """ import asyncio +import re import struct from pathlib import Path @@ -231,11 +232,21 @@ async def test_one_process_starts_none(): # ── the setting ───────────────────────────────────────────────────────────── -def test_the_default_is_one_process(tmp_path): - assert node_settings_defaults()["connection_processes"] == 1 +@pytest.mark.parametrize("cores, expected", [(1, 1), (2, 2), (4, 4), (28, 4)]) +def test_the_default_is_one_per_core_up_to_four(tmp_path, monkeypatch, cores, expected): + """It was 1, which starts none: the path was shipped and reached by no node + whose operator had not found the setting.""" + monkeypatch.setattr("meshbay_node.config.max_processes", lambda: cores) + assert node_settings_defaults()["connection_processes"] == expected conf = tmp_path / "node.toml" conf.write_text('[hub]\nurl = "https://example.invalid"\nusername = "op"\n') - assert load_config(conf).node.connection_processes == 1 + assert load_config(conf).node.connection_processes == expected + + +def test_a_new_node_toml_leaves_it_to_the_default(tmp_path): + from meshbay_node.config import EXAMPLE_CONFIG + assert not re.search(r"^connection_processes\s*=", EXAMPLE_CONFIG, re.M), ( + "a value written by `init` is a value no later default reaches") def test_node_toml_cannot_ask_for_more_than_the_cores(tmp_path): @@ -296,4 +307,5 @@ async def test_a_number_the_machine_cannot_honour_is_refused(state, monkeypatch, with pytest.raises(ops.OpError): await ops.set_node_settings(state, {"connection_processes": value}) assert state["webrtc"].asked == [] - assert (await ops.get_node_settings(state))["connection_processes"] == 1 + assert ((await ops.get_node_settings(state))["connection_processes"] + == state["config"].node.connection_processes) diff --git a/packages/meshbay-node/tests/test_packaging_win.py b/packages/meshbay-node/tests/test_packaging_win.py index 6ee4891..0110b6c 100644 --- a/packages/meshbay-node/tests/test_packaging_win.py +++ b/packages/meshbay-node/tests/test_packaging_win.py @@ -1497,6 +1497,27 @@ def test_msix_gives_the_cli_an_execution_alias_and_the_windows_it_needs(): assert 'xmlns:uap5="http://schemas.microsoft.com/appx/manifest/uap/windows10/5"' in xml +def test_msix_gives_the_command_line_client_an_alias_of_its_own_application(): + """An Application may declare one windows.appExecutionAlias extension. + meshbay-node.exe has the app's; a second one beside it for meshbay-cli.exe + made makeappx refuse the whole package ("must only be declared once"), at + the very end of a Store build. The client is an Application of its own, + kept out of the Start menu, with its alias and its firewall rule.""" + xml = MSIX_MANIFEST_XML.read_text(encoding="utf-8") + ext = MSIX_EXTENSIONS_XML.read_text(encoding="utf-8") + assert ext.count('Category="windows.appExecutionAlias"') == 1 + apps = re.findall(r"<Application\b.*?</Application>", xml, re.S) + for app in apps: + assert app.count('Category="windows.appExecutionAlias"') <= 1, app[:80] + [cli] = [a for a in apps if 'Executable="app\\resources\\node-runtime\\meshbay-cli.exe"' + in a.split(">", 1)[0]] + assert 'AppListEntry="none"' in cli + assert 'Alias="meshbay-cli.exe"' in cli + package_level = xml.split("</Applications>", 1)[1] + rule = package_level.split('Executable="app\\resources\\node-runtime\\meshbay-cli.exe"', 1) + assert len(rule) == 2 and 'IPProtocol="UDP"' in rule[1].split("</desktop2:", 1)[0] + + def test_the_node_runtime_has_a_windowless_daemon_beside_the_cli(): spec = (WIN / "meshbay-node.spec").read_text(encoding="utf-8") windowless = spec.split('name="meshbay-nodew"', 1) diff --git a/packages/meshbay-node/tests/test_proactor_udp.py b/packages/meshbay-node/tests/test_proactor_udp.py new file mode 100644 index 0000000..0b22b4e --- /dev/null +++ b/packages/meshbay-node/tests/test_proactor_udp.py @@ -0,0 +1,150 @@ +""" +On Windows the node's datagrams go straight through the socket, not one +overlapped operation per datagram and per turn of the loop (`proactor_udp.py`). + +Run on a real ProactorEventLoop, which only Windows has. +""" + +import asyncio +import socket +import sys +import time + +import pytest +from meshbay_node.transport.webrtc import proactor_udp + +pytestmark = pytest.mark.skipif(sys.platform != "win32", reason="the proactor is Windows'") + +N = 2000 + + +class _Collect(asyncio.DatagramProtocol): + def __init__(self, raise_on: int = -1): + self.got: list[int] = [] + self.raise_on = raise_on + self.all = asyncio.get_running_loop().create_future() + + def datagram_received(self, data, addr): + self.got.append(int.from_bytes(data[:4], "big")) + if len(self.got) == N and not self.all.done(): + self.all.set_result(None) + if len(self.got) == self.raise_on: + raise RuntimeError("a protocol's own bug") + + +class _Refusing(socket.socket): + """A socket that refuses a send every so often, as a full buffer does, and + can answer one read with the reset an ICMP port unreachable leaves. The + overlapped path works by handle and never calls these.""" + + def __init__(self, every: int = 0, reset_once: bool = False): + super().__init__(socket.AF_INET, socket.SOCK_DGRAM) + self.every, self.calls, self.reset_once = every, 0, reset_once + + def sendto(self, data, addr): + self.calls += 1 + if self.every and self.calls % self.every == 0: + raise BlockingIOError + return super().sendto(data, addr) + + def recvfrom(self, n): + if self.reset_once: + self.reset_once = False + raise ConnectionResetError(10054, "port unreachable") + return super().recvfrom(n) + + +def _busy_turns(ms: float): + """A loop with other work to do: every turn takes `ms` longer.""" + async def busy(): + while True: + end = time.perf_counter() + ms / 1000 + while time.perf_counter() < end: + pass + await asyncio.sleep(0) + return asyncio.ensure_future(busy()) + + +async def _exchange(*, refuse_every=0, reset_once=False, raise_on=-1, busy_ms=0.0): + """N datagrams from one proactor transport to another on loopback: what + arrived in what order, and how many overlapped sends and reads it took.""" + loop = asyncio.get_running_loop() + rsock = _Refusing(reset_once=reset_once) + rsock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 8 << 20) + rsock.bind(("127.0.0.1", 0)) + rtr, rx = await loop.create_datagram_endpoint(lambda: _Collect(raise_on), sock=rsock) + tsock = _Refusing(refuse_every) + tsock.bind(("127.0.0.1", 0)) + tx, _ = await loop.create_datagram_endpoint(asyncio.DatagramProtocol, sock=tsock) + counts = {"sendto": 0, "recvfrom": 0} + originals = {name: getattr(loop._proactor, name) for name in counts} + + def counting(name): + def call(*a, **kw): + counts[name] += 1 + return originals[name](*a, **kw) + return call + for name in counts: + setattr(loop._proactor, name, counting(name)) + errors = [] + loop.set_exception_handler(lambda _loop, ctx: errors.append(ctx)) + busy = _busy_turns(busy_ms) if busy_ms else None + try: + dest = rsock.getsockname() + for i in range(N): + tx.sendto(i.to_bytes(4, "big") * 50, dest) + if i % 50 == 49: + await asyncio.sleep(0) + await asyncio.wait_for(rx.all, 20) + return rx.got, counts, errors + finally: + if busy: + busy.cancel() + for name, fn in originals.items(): + setattr(loop._proactor, name, fn) + tx.close() + rtr.close() + + +def _run(**kw): + return asyncio.run(_exchange(**kw), loop_factory=asyncio.ProactorEventLoop) + + +def test_the_transport_is_patched(): + cls = asyncio.proactor_events._ProactorDatagramTransport + assert cls.sendto is proactor_udp.sendto + assert cls._loop_reading is proactor_udp._loop_reading + + +def test_datagrams_go_straight_out_in_order(): + got, counts, _ = _run() + assert got == list(range(N)) + assert counts["sendto"] == 0 + + +def test_what_the_socket_refuses_is_queued_and_order_is_kept(): + """A refusal sends that datagram the overlapped way, and the ones after it + wait behind it rather than overtake it.""" + got, counts, _ = _run(refuse_every=97) + assert got == list(range(N)) + assert counts["sendto"] >= N // 97 + + +def test_a_busy_loop_reads_everything_waiting_in_order(): + """Unpatched, this loop read one datagram a turn: about 2,000 a second + whatever was waiting, and the kernel dropped the rest.""" + got, counts, _ = _run(busy_ms=2) + assert got == list(range(N)) + assert counts["recvfrom"] < N // 10 + + +def test_a_port_unreachable_on_a_direct_read_is_skipped(): + got, _, errors = _run(reset_once=True, busy_ms=2) + assert got == list(range(N)) + assert not errors + + +def test_a_protocol_that_raises_does_not_end_the_reading(): + got, _, errors = _run(raise_on=10) + assert got == list(range(N)) + assert [str(e["exception"]) for e in errors] == ["a protocol's own bug"] diff --git a/packages/meshbay-node/tests/test_sctp_fix.py b/packages/meshbay-node/tests/test_sctp_fix.py index 0b2536a..c3fe622 100644 --- a/packages/meshbay-node/tests/test_sctp_fix.py +++ b/packages/meshbay-node/tests/test_sctp_fix.py @@ -1,4 +1,5 @@ -"""aiortc's SCTP sender must not leak bytes in flight (see sctp_fix.py). +"""aiortc's SCTP stack as sctp_fix.py leaves it: no bytes leaked in flight, +reassembly in linear time, one SACK every second packet. Replayed against aiortc's own RTCSctpTransport: three chunks sent, the first lost and the other two acknowledged in a gap block, T3 expires and they are @@ -8,10 +9,20 @@ enough of those stop the node from sending anything on that connection. """ import asyncio - -from aiortc.rtcsctptransport import RTCSctpTransport, SackChunk +import random import meshbay_node.transport.webrtc.core # noqa: F401 (applies the fix) +from aiortc.rtcsctptransport import ( + SCTP_DATA_FIRST_FRAG, + SCTP_DATA_LAST_FRAG, + SCTP_DATA_UNORDERED, + DataChunk, + InboundStream, + RTCSctpTransport, + SackChunk, + serialize_packet, +) +from meshbay_node.transport.webrtc import sctp_fix class _Dtls: @@ -57,3 +68,137 @@ def test_nothing_outstanding_means_nothing_in_flight(): s = asyncio.run(_replay()) assert not s._sent_queue assert s._flight_size == 0 + + +# ── Reassembly (see sctp_fix.py) ───────────────────────────────────────────── + + +def _fragments(messages: list[int], *, unordered: bool = False, tsn: int = 1000): + """DATA chunks for messages of the given fragment counts, in TSN order.""" + out = [] + for seq, n in enumerate(messages): + for i in range(n): + c = DataChunk() + c.flags = ((SCTP_DATA_FIRST_FRAG if i == 0 else 0) + | (SCTP_DATA_LAST_FRAG if i == n - 1 else 0) + | (SCTP_DATA_UNORDERED if unordered else 0)) + c.tsn, c.stream_id, c.stream_seq, c.protocol = tsn, 1, seq, 53 + c.user_data = f"{seq}.{i};".encode() + tsn += 1 + out.append(c) + return out + + +class _Original: + """aiortc's own reassembly, untouched, for the same arrivals.""" + + def __init__(self): + self.reassembly, self.sequence_number = [], 0 + + def add(self, chunk): + sctp_fix._add_chunk(self, chunk) + return list(sctp_fix._pop_messages(self)) + + +def test_reassembly_gives_what_aiortc_gives_for_any_arrival_order(): + rng = random.Random(7) + for trial in range(300): + messages = [rng.randint(1, 12) for _ in range(rng.randint(1, 8))] + chunks = _fragments(messages, unordered=trial % 3 == 0) + # Reordered within a short window, as a network reorders. + for i in range(len(chunks)): + j = min(len(chunks) - 1, i + rng.randint(0, 4)) + chunks[i], chunks[j] = chunks[j], chunks[i] + fixed, original = InboundStream(), _Original() + got, want = [], [] + for c in chunks: + fixed.add_chunk(c) + got.append(list(fixed.pop_messages())) + want.append(original.add(c)) + assert got == want, (trial, messages) + + +def test_a_large_message_is_walked_once_not_once_per_fragment(monkeypatch): + walks = [] + original = sctp_fix._pop_messages + + def counted(stream): + walks.append(len(stream.reassembly)) + return original(stream) + + monkeypatch.setattr(sctp_fix, "_pop_messages", counted) + s = InboundStream() + out = [] + for c in _fragments([1000, 1000]): + s.add_chunk(c) + out += list(s.pop_messages()) + assert len(out) == 2 + assert walks == [1000, 1000], "one walk per message, when its last fragment is in" + + +def test_pruning_a_last_fragment_does_not_stall_the_next_message(): + s = InboundStream() + first, second = _fragments([3]), _fragments([2], tsn=1003) + second[0].stream_seq = second[1].stream_seq = 1 + for c in first[1:]: # the first fragment never comes + s.add_chunk(c) + assert not list(s.pop_messages()) + s.prune_chunks(first[-1].tsn) # abandoned (FORWARD TSN) + s.sequence_number = 1 + for c in second: + s.add_chunk(c) + assert [m[2] for m in s.pop_messages()] == [b"0.0;0.1;"] + + +# ── Delayed acknowledgement (see sctp_fix.py) ──────────────────────────────── + +def _receiver(): + s = RTCSctpTransport(_Dtls()) + s._association_state = s.State.ESTABLISHED + s._last_received_tsn = 999 + sacks = [] + + async def send_sack(): + sacks.append(s._last_received_tsn) + s._sack_needed = False + s._sack_duplicates.clear() + + async def deliver(*_a): + pass + + s._send_sack = send_sack + s._receive = deliver + return s, sacks + + +def _packet(s, chunk) -> bytes: + return serialize_packet(5000, 5000, s._local_verification_tag, chunk) + + +def test_one_sack_every_second_packet_of_data(): + async def go(): + s, sacks = _receiver() + for c in _fragments([1] * 10): + await s._handle_data(_packet(s, c)) + return sacks + assert asyncio.run(go()) == [1001, 1003, 1005, 1007, 1009] + + +def test_a_gap_is_acknowledged_at_once(): + async def go(): + s, sacks = _receiver() + chunks = _fragments([1] * 4) + await s._handle_data(_packet(s, chunks[0])) + await s._handle_data(_packet(s, chunks[2])) # 1001 missing + return sacks + assert asyncio.run(go()) == [1000], "the gap is the loss signal the sender waits for" + + +def test_a_lone_packet_is_acknowledged_within_the_delay(): + async def go(): + s, sacks = _receiver() + await s._handle_data(_packet(s, _fragments([1])[0])) + assert sacks == [] + await asyncio.sleep(sctp_fix.SACK_DELAY_S + 0.1) + return sacks + assert asyncio.run(go()) == [1000] diff --git a/packaging/build/build-node.sh b/packaging/build/build-node.sh index c7e7577..74d1d40 100755 --- a/packaging/build/build-node.sh +++ b/packaging/build/build-node.sh @@ -53,7 +53,7 @@ mkdir -p "$ROOT/opt/meshbay-node/share" # must read it, and it is the same token in every copy of the package. No # token, no build: an empty one goes unnoticed. DEFAULT_ENV="$REPO/QE/default.env" -TOKEN_LINE="$(head -n 1 "$DEFAULT_ENV" 2>/dev/null | tr -d '\r')" +TOKEN_LINE="$(head -n 1 "$DEFAULT_ENV" 2>/dev/null | tr -d '\r' || true)" if ! [[ "$TOKEN_LINE" =~ ^MESHBAY_TMDB_DEFAULT_TOKEN=eyJ[A-Za-z0-9_.-]+$ ]]; then echo "!! $DEFAULT_ENV missing, or its first line is not MESHBAY_TMDB_DEFAULT_TOKEN=eyJ... (no BOM)" >&2 exit 1 diff --git a/packaging/win/README.md b/packaging/win/README.md index 89185fe..959e651 100644 --- a/packaging/win/README.md +++ b/packaging/win/README.md @@ -37,7 +37,7 @@ way — not LocalSystem/NetworkService, which would have none of this. `build/installer.nsh` also adds `…\resources\node-runtime` to the **per-user** `Path` (`HKCU\Environment`) so `meshbay-node` and `meshbay-cli` work in a terminal, and takes it back out on uninstall. The Store package declares an -execution alias for each instead (`build/appx-extensions.xml`). New shells only — a `WM_SETTINGCHANGE` broadcast nudges +execution alias for each instead (the node's in `build/appx-extensions.xml`; the client's on an application of its own in `build/appx-manifest.xml`, since an application declares one alias). New shells only — a `WM_SETTINGCHANGE` broadcast nudges open ones. It uses stock `WordFunc.nsh` (the `EnVar` plugin is not in electron-builder's NSIS bundle). |