From 2ef0c215ea4af421a4b22ab543a5999320998f68 Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Sun, 11 Oct 2026 20:47:42 +0200 Subject: fix(node): a receive buffer that holds an upload's window Uploads ran at 6.5 MB/s because the kernel dropped datagrams at the node's 256 KB socket. Asking for 4 MB gave 19.5 MB/s, no drop. Co-Authored-By: Claude Opus 5.5 --- docs/MESHBAY_DESIGN.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'docs/MESHBAY_DESIGN.md') diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 233bcc8..9c63c52 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1465,6 +1465,19 @@ TCP because that is the one transport the event loop offers identically on every platform; a process proves it is the one the node started by returning 32 random bytes handed to it on its standard input. +**Uploads met the same wall from the other side.** A browser 22 ms away uploaded +at 6.5 MB/s, with no core busy on either machine and no loss in the node's log. +The loss was on the node's machine: its kernel dropped 2460 of 264000 datagrams at +the socket (`UdpRcvbufErrors`), because aioice asks for a 256 KB receive buffer, +the node reads its sockets from Python, and the browser keeps 1.5 MB of an upload +in flight. Its SCTP sender took each drop for congestion. The node now asks for +4 MB per socket (`transport/webrtc/udp_buffer.py`): the same upload ran at 19.3 and +19.7 MB/s with no datagram dropped. The kernel grants no more than +`net.core.rmem_max`, which many distributions leave at 208 KB, and on such a +machine nothing changes until the operator raises it; the node says so at startup. +The limit that remains is the node's own receive path, about 5 % of a core per +MB/s. + The lease module is free of asyncio and of the transport: it decides, and the caller does the I/O. A queue that reveals itself only through a DataChannel is a queue nobody can prove things about. -- cgit v1.2.3