From 176b5ed563f7022009b70751e62b7ee7901e6da0 Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Sun, 11 Oct 2026 19:09:04 +0200 Subject: feat(node): carry connections on several cores connection_processes (default 1) starts that many processes for the WebRTC stack; sessions and keys stay in the node. Set from node.toml or the Node page, applied to new connections. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 12 + docs/MESHBAY_DESIGN.md | 39 +- docs/USERGUIDE.md | 7 + .../src/meshbay_hub/static/locales/de.js | 2 + .../src/meshbay_hub/static/locales/en.js | 2 + .../src/meshbay_hub/static/locales/es.js | 2 + .../src/meshbay_hub/static/locales/fr.js | 2 + .../src/meshbay_hub/static/locales/it.js | 2 + .../src/meshbay_hub/static/locales/ja.js | 2 + .../src/meshbay_hub/static/locales/nl.js | 2 + .../src/meshbay_hub/static/locales/pl.js | 2 + .../src/meshbay_hub/static/locales/pt-BR.js | 2 + .../src/meshbay_hub/static/locales/zh-CN.js | 2 + .../src/meshbay_hub/static/node-page.js | 10 + .../tests/test_node_page_connection_processes.py | 38 ++ packages/meshbay-node/src/meshbay_node/config.py | 20 ++ packages/meshbay-node/src/meshbay_node/daemon.py | 10 + .../meshbay-node/src/meshbay_node/ops/settings.py | 36 +- packages/meshbay-node/src/meshbay_node/roster.py | 2 + .../src/meshbay_node/transport/webrtc/channel.py | 3 + .../meshbay_node/transport/webrtc/sctp_stats.py | 3 + .../src/meshbay_node/transport/webrtc/worker.py | 245 +++++++++++++ .../src/meshbay_node/transport/webrtc/workers.py | 392 +++++++++++++++++++++ .../src/meshbay_node/transport/webrtc_server.py | 63 +++- .../tests/test_connection_processes.py | 281 +++++++++++++++ .../meshbay-node/tests/test_transfer_settings.py | 2 +- 26 files changed, 1167 insertions(+), 16 deletions(-) create mode 100644 packages/meshbay-hub/tests/test_node_page_connection_processes.py create mode 100644 packages/meshbay-node/src/meshbay_node/transport/webrtc/worker.py create mode 100644 packages/meshbay-node/src/meshbay_node/transport/webrtc/workers.py create mode 100644 packages/meshbay-node/tests/test_connection_processes.py diff --git a/CLAUDE.md b/CLAUDE.md index 24f024b..460a709 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -737,6 +737,18 @@ do. Read them before writing anything that touches the same mechanism. with one connection and with eight, so only a browser on another machine measures it +- **Measure which machine is the limit before adding cores to one of them.** + "Two clients will share the node's one core" was the reasoning; measured, two + browsers downloading at once left the node at 60 % of a core and saturated + the four cores of the machine receiving. The node's ceiling is real (one + aiortc process, about 35 MB/s over all its connections) but it is the third + fast client that meets it, not the second. What moved to other processes is + only the stack below the data channel (`connection_processes`, + `transport/webrtc/workers.py`, `docs/MESHBAY_DESIGN.md` §5.5), because that + is where the CPU went: with four processes the node proper used 18 % of a + core at 131 MB/s. Splitting there also means no key leaves the node's own + process + ### More of the same, from four rounds of live testing - **A protocol harness cannot test the SPA.** Any second implementation of the diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 81e2e33..233bcc8 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1432,6 +1432,39 @@ there is no new message and no new field, so a node that predates this treats a sibling's `tr` as one it does not know and serves it under the leaseless ceiling, which bounds how many files a connection reads at once and not how much of one. +**Several processes for the connections.** aiortc is Python: one process sends +about 35 MB/s summed over every connection it holds, on one core, and a loop that +busy stops answering the connections that are not transferring. Two browsers +downloading at once from a server 22 ms away received 12 to 13 MB/s each where +one alone received 21, with the node at 60 % of a core and the receiving machine +saturated; a third fast client is where the node itself runs out. With +`connection_processes` above 1 (§6.8) the node starts that many processes and +gives each new peer connection to the least loaded +(`transport/webrtc/workers.py`). A process runs ICE, DTLS and SCTP and nothing +else: no node key, no group key, no index, no session. What it exchanges with the +node is what the data channel carries, framed on a loopback socket, and the +session stands on a stand-in for the peer connection that offers the few things +a session reads from one, so no handler knows which it was given. The lanes of a +download land on different processes without the client doing anything. + +| Processes | One download, same machine | CPU of the node proper | +|---|---|---| +| 1 (the node holds them) | 40 MB/s | all of it | +| 2 | 90 MB/s | 11 % of a core | +| 4 | 131 MB/s | 18 % | +| 8 | 150 MB/s, the receiving browser being the limit | 21 % | + +Measured on loopback, a 1.5 GB file through the interface in a headless browser, +byte for byte. A process costs about 85 MB of memory. The default is 1, which +starts none and leaves the node exactly as it was; the operator raises it. A +change applies to new connections, and a connection already open stays where it +is until it closes, in a process asked to leave or in the node. A process that +dies takes its own connections with it, which their clients see as a dropped +connection and reopen, and is replaced at the next offer. The loopback socket is +TCP because that is the one transport the event loop offers identically on every +platform; a process proves it is the one the node started by returning 32 random +bytes handed to it on its standard input. + The lease module is free of asyncio and of the transport: it decides, and the caller does the I/O. A queue that reveals itself only through a DataChannel is a queue nobody can prove things about. @@ -1998,7 +2031,7 @@ a real capability reduction, not an oversight. ### 6.8 Node settings -Eight `[node] `settings affect what the node does rather than how it starts, and +Nine `[node] `settings affect what the node does rather than how it starts, and their value is invisible until something goes wrong — so they are surfaced on the Node page: @@ -2011,6 +2044,7 @@ Node page: | `max_concurrent_downloads` | 8 | node-wide download leases (§5.5) | | `max_concurrent_uploads` | 8 | node-wide upload leases. A separate pool from downloads, because the two cost different things and one queue for both makes each cap meaningless | | `max_upload_gb` | 8 | the largest single upload, §6.4. Fractions are allowed, and it is read per chunk so a change reaches an upload already running | +| `connection_processes` | 1 | how many processes carry the peer connections, §5.5. One is the node itself; above it, that many cores serve transfers. Bounded by the cores of the machine, which the Node page shows beside the field | | `transcode_incompatible_video` | true | whether browser-incompatible video is transcoded during streaming. Unlike remuxing this costs real CPU per viewer | `hardware_video_encode` (default true) governs whether that transcode may use the @@ -4392,6 +4426,9 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | **"Copy stream URL" after picking a receiver casts to that receiver** | The player keeps the last device chosen, so the copy-only path reconnects it instead of only starting the relay | | **A folder renamed on the node's disk empties until the next reconciliation** | Seen live (2026-10-10, ntfs3): renaming a 2,441-file folder inside a root made the watcher drop every entry under the old path and announce none under the new one, so the group read as empty, and the loop was busy long enough for the hub's keepalive to time out. A restart (full scan) brought them back. A directory moved within a root should rescan its new subtree at once, not wait for the periodic scan | | **A photo deleted from the group comes back after a reinstall** | The backup's memory of what it sent is the phone's ledger (§9.12); a reinstall or cleared data empties it, and the folder no longer holds the deleted photo, so it is sent again. Clean fix: the node keeps a tombstone per folder on `file_delete` (name, size, SHA-256 taken before deleting), the index carries them, and the backup records a match as sent. Not done: a protocol field and a node table for a rare case | +| **`connection_processes` has no CLI verb** | Set from the Node page or `node.toml`; `meshbay-node` has no command for it, unlike the transfer caps | +| **Connection processes are untested on Windows and in a frozen build** | The frozen node starts them as `meshbay-node webrtc-worker`; written, never run there | +| **A connection process does not slow a peer the node cannot keep up with** | What a process receives is forwarded to the node unbounded. An upload is bounded by its own rate, 5 to 6 MB/s a connection, so it would take a node stalled for seconds to matter | | **The home page says "No groups yet" when the hub cannot be reached** | An unreachable hub reads as an account with no groups, rather than as an error | | **`meshbay-node init` says "Settings → Link Node"** | The control is on the Profile page, as QUICKSTART says | | **C4** for accounts with browser access | Closed against operators by the pepper; **open against an active hub**, which holds the pepper and can fetch a bundle with a token it mints — the adversary T3 already concedes for browsers (§3.7) | diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index bc16ba6..5649b30 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -745,9 +745,16 @@ and running `meshbay-node reload`: | `max_concurrent_downloads` | 8 | node-wide download slots | | `max_concurrent_uploads` | 8 | node-wide upload slots | | `max_upload_gb` | 8 | the largest single file a member may send you | +| `connection_processes` | 1 | how many CPU cores carry members' connections | | `transcode_incompatible_video` | true | re-encode video no browser can play | | `hardware_video_encode` | true | use the GPU for that, if one actually works | +`connection_processes` is the one for speed. With 1, everything a node sends +goes through one core, about 35 MB/s shared between everyone downloading at +that moment. Raise it when several people download at once and the machine has +cores to spare: each one adds a core's worth, and about 85 MB of memory. It +applies to connections opened after the change. + `max_concurrent_streams` is the one to think about: a stream costs a video process for as long as the film lasts, so this counts **viewers**, not requests. diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js index 17b8c2b..d8381e1 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js @@ -985,6 +985,8 @@ export default { 'node.setting_max_uploads': 'Max. gleichzeitige Uploads', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Größter einzelner Upload', + 'node.setting_connection_processes': 'CPU-Kerne für Verbindungen', + 'node.setting_unit_of': 'von {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js index a4e07a1..07c885d 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js @@ -1135,6 +1135,8 @@ export default { 'node.setting_max_uploads': 'Max concurrent uploads', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Largest single upload', + 'node.setting_connection_processes': 'CPU cores for connections', + 'node.setting_unit_of': 'of {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js index e8ea3e7..a6fb30f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js @@ -979,6 +979,8 @@ export default { 'node.setting_max_uploads': 'Subidas simultáneas máximas', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Subida individual máxima', + 'node.setting_connection_processes': 'Núcleos de CPU para las conexiones', + 'node.setting_unit_of': 'de {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js index 0d927ae..c38be75 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js @@ -982,6 +982,8 @@ export default { 'node.setting_max_uploads': 'Téléversements simultanés max', 'node.setting_transcode': 'Transcoder les vidéos incompatibles', 'node.setting_max_upload': 'Taille maximale d\'un envoi', + 'node.setting_connection_processes': 'Cœurs CPU pour les connexions', + 'node.setting_unit_of': 'sur {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'heures', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js index e2c6df3..05f903e 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js @@ -981,6 +981,8 @@ export default { 'node.setting_max_uploads': 'Caricamenti simultanei massimi', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Caricamento singolo massimo', + 'node.setting_connection_processes': 'Core CPU per le connessioni', + 'node.setting_unit_of': 'su {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js index 9aa4833..bd63b4f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js @@ -969,6 +969,8 @@ export default { 'node.setting_max_uploads': '同時アップロードの上限', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': '1ファイルあたりの上限', + 'node.setting_connection_processes': '接続に使う CPU コア数', + 'node.setting_unit_of': '/ {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js index 863f7c9..eb6c6c2 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js @@ -983,6 +983,8 @@ export default { 'node.setting_max_uploads': 'Max. gelijktijdige uploads', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Grootste enkele upload', + 'node.setting_connection_processes': 'CPU-kernen voor verbindingen', + 'node.setting_unit_of': 'van {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js index 2dc2e74..1b97dae 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js @@ -1003,6 +1003,8 @@ export default { 'node.setting_max_uploads': 'Maks. równoczesnych wysyłek', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Największa pojedyncza wysyłka', + 'node.setting_connection_processes': 'Rdzenie CPU dla połączeń', + 'node.setting_unit_of': 'z {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js index 6c4f03b..a680cc8 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js @@ -980,6 +980,8 @@ export default { 'node.setting_max_uploads': 'Máximo de envios simultâneos', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': 'Maior envio individual', + 'node.setting_connection_processes': 'Núcleos de CPU para as conexões', + 'node.setting_unit_of': 'de {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js index 8899ed0..ae16098 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js @@ -957,6 +957,8 @@ export default { 'node.setting_max_uploads': '最大同时上传数', 'node.setting_transcode': 'Transcode incompatible video', 'node.setting_max_upload': '单个上传的最大大小', + 'node.setting_connection_processes': '用于连接的 CPU 核心数', + 'node.setting_unit_of': '/ {n}', 'node.setting_unit_gb': 'GB', 'node.setting_unit_hours': 'hours', 'node.setting_unit_minutes': 'min', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/node-page.js b/packages/meshbay-hub/src/meshbay_hub/static/node-page.js index 56166c6..88167d3 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/node-page.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/node-page.js @@ -1187,6 +1187,16 @@ export function NodePage({ groups, token, username }) { ${t('node.setting_unit_gb')} +