diff options
| -rw-r--r-- | CLAUDE.md | 12 | ||||
| -rw-r--r-- | docs/MESHBAY_DESIGN.md | 20 |
2 files changed, 32 insertions, 0 deletions
@@ -772,6 +772,18 @@ do. Read them before writing anything that touches the same mechanism. `sctp_fix.py`: 39 MB/s, the node at 49. No amount of reading our code finds a quadratic loop in a dependency; one profile does +- **A ceiling that falls as the loop gets busier is not a CPU ceiling.** + Windows downloads stayed slow with no core saturated: the node at 45 % of a + core, 9 MB/s over eight connections, and an SCTP round trip of 745 ms on + loopback, which is a queue, not a wire. The ProactorEventLoop's datagram + transport sends one datagram per turn of the loop and reads one, each through + an overlapped operation (42 % of a connection process was + `IocpProactor.sendto`), so every other thing the loop does slows every + connection. Linux sends straight from the socket. `proactor_udp.py` does the + same on Windows; a benchmark only showed it once its loop had other work to + do, because an idle loop turns fast enough to hide it. When a transfer is slow, compare the round trip SCTP reports with the + real one before profiling: a large gap is a queue in this process + - **A default of "off" ships a feature nobody has.** `connection_processes` defaulted to 1, which starts no process: the multi-core path was measured at 131 MB/s and reached by no node whose operator had not found the setting, so diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index e10209a..4db06c4 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1531,6 +1531,26 @@ still 20, the client at a full core; 28 with the reassembly fixed; 39 with the acknowledgements, 49 by the node's own count of the transfer. An upload from the client went from 25 to 29. +**On Windows a datagram goes straight through the socket, in both directions.** +The node and its connection processes run on the ProactorEventLoop, the one that +can start ffmpeg there, and its datagram transport keeps one overlapped operation +per direction, starting the next from the completion callback of the one before, +on a later turn of the loop. A socket therefore sent one datagram per turn and +received one, and a turn lasts as long as everything else the loop does: the +other connections, the file, the hub. That is a ceiling per connection that +falls as the node gets busier, with cores to spare. With the loop otherwise busy +0.5 ms a turn: 1,928 datagrams sent a second, and 1,954 received with 87 % of +what arrived dropped by the kernel. In a node holding its connections itself, a +browser's download over eight of them ran at 9 MB/s with the node at 45 % of a +core and an SCTP round trip of 745 ms on loopback, and one of the eight sessions +was lost on the way. `proactor_udp.py` sends with a non-blocking `sendto` when +nothing is queued, and after a completed read takes whatever else the socket +holds before posting the next one; refusals and errors take the proactor's own +path. Measured on loopback on a two-core machine, 400 MB, with it: the +command-line client's download 27 to 33 MB/s, its upload 18 to 22 (23 with two +connection processes), the browser's download over eight connections 9 to 14 +before and 17 to 19 after, where the browser takes the other core. + **A session is open from the end of its handshake, and a reconnect does not shorten that.** Until connect() returns, a client sends only what the node takes before the proof — `handshake`, `handshake_response`, `keypair_bundle_fetch`, |