diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-10-11 16:52:48 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-10-11 16:52:48 +0200 |
| commit | aca58f66df5c21be71a7159620d0b50bb969199f (patch) | |
| tree | 28bcb0e834d9c05a25b37e56ad71c424924d93fc /docs/MESHBAY_DESIGN.md | |
| parent | 9d38418817d1894b31c93ece0113e503d82167cd (diff) | |
| download | meshbay-aca58f66df5c21be71a7159620d0b50bb969199f.tar.gz | |
feat: spread a large download over eight connections
One WebRTC connection carries 5 to 6 MB/s whatever the line can do.
A leased download of 64 MB or more now opens seven more to its node,
and a lease serves the other connections of its device in its group.
Measured through the whole stack: 5 MB/s before, 24 after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'docs/MESHBAY_DESIGN.md')
| -rw-r--r-- | docs/MESHBAY_DESIGN.md | 45 |
1 files changed, 43 insertions, 2 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 3820587..81e2e33 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1285,7 +1285,9 @@ decisions: - **`tr` is drawn by the client.** Re-opening after a reconnect with the same `tr` is idempotent, so a reconnect cannot charge a member twice for one transfer. - **A lease is scoped to the connection**, never to the account. It dies with the - session, which makes the primary reclaim deterministic. + session, which makes the primary reclaim deterministic. Other connections of the + same device to the same group may read and write under it, and that is all they + may do (see "One transfer, several connections" below). - **A lease covers a job, not a file.** A directory zip is dozens of files and one lease. - **Nothing is persisted.** A restart drops every session anyway; a lease that @@ -1335,7 +1337,8 @@ The rule stated as a sentence someone can check by reading: **Which of the two a request is, is the node's answer and not the client's.** `tr` is drawn by the client, so it is a claim: the node resolves it against its own record and serves the chunk as a leased transfer only when that record is a -**granted** lease **of this connection**. A lease it holds but has not granted is +**granted** lease **of this connection, or of another connection of the same +device to the same group**. A lease it holds but has not granted is refused with a code — a member reading while queued is the cap not applying — and an id it has no record of is treated as leaseless and bounded below, because that is also what a reconnect looks like from here, where the session's leases died @@ -1402,6 +1405,33 @@ neither; `transport/webrtc/sctp_stats.py` counts them). A window that stays at a few hundred KB with `loss` climbing is a transfer held back by the path, and before this line the journal could not say so. +**One transfer, several connections.** One WebRTC connection carries 5 to 6 MB/s +at 22 ms of round trip on a line that carries 47 over TCP, and less as the round +trip grows. A browser receives on a UDP socket whose kernel buffer is 64 KB, about +56 packets; a burst overflows it, the packets are dropped on the receiving +machine, and the node's SCTP sender halves its window at every loss and regains +1200 bytes per round trip. Softer loss handling, pacing and a shorter RTO were +each measured and changed nothing that mattered. The ceiling is per connection, +so a client spreads a large download over several: 4 connections measured about +19 MB/s and 8 between 27 and 34, where the node's sender, one Python process, +is the next ceiling. Through the whole stack (the interface in a headless +browser on a server 22 ms away, a 213 MB file on a node in service), one +connection gave 5.1 and 5.3 MB/s and eight gave 24 once the lanes had landed, +three seconds in. They have to be separate peer connections. Several data +channels on one share its socket, its association and its window. + +The lease stays what it was. The connection that opened it holds it, is told its +state, closes it, and takes it away by ending. What changes is who may move bytes +under it: **a connection of the same device, as the same member of the same +group**, presenting the same `tr`. The slot was charged to that member when the +lease was granted, so the extra connections cost nobody anything they had not +already been granted, and a lease that is queued is refused on all of them alike. +A different device of the same account is not a sibling, and neither is a +connection that has not got as far as a pinned key. Nothing is added to the wire: +there is no new message and no new field, so a node that predates this treats a +sibling's `tr` as one it does not know and serves it under the leaseless ceiling, +which bounds how many files a connection reads at once and not how much of one. + The lease module is free of asyncio and of the transport: it decides, and the caller does the I/O. A queue that reveals itself only through a DataChannel is a queue nobody can prove things about. @@ -2636,6 +2666,17 @@ Specific rules the download path is built on: receiving a message is the event that resets the timer. - **Every await on a download path is bounded and says which chunk it gave up on.** An unbounded await is a freeze nobody can report. +- **A leased download of 64 MB or more runs over eight connections** (§5.5): + `MeshBayTransport.lanes` opens seven beside the one the group already has, each + the same connection again (same node, group, identity and device), and + `pipelinedDownload` asks for chunk *i* on lane *i* modulo their number, two in + flight per lane, and writes in order as before. The download starts at once on + the main connection and widens as lanes land. A lane that fails a request is + closed and its chunk asked for again on the main connection, which keeps the + retries and the reconnect; a lane never reconnects. Lanes idle for 30 s are + closed, and one that could not be opened is not tried again for 30 s, because + the question is asked at every chunk. Previews and playback hold no lease and + open none. - **A filename in a content-disposition header needs both forms.** The RFC 5987 encoding uses `'` as a delimiter and the standard escaper does not escape it, so a plain ASCII fallback rides alongside — the next surprise loses accents rather than |