From a1aaf31a27d1c1b65efc3c6a25fc6cc8771578ea Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Mon, 14 Sep 2026 02:29:13 +0200 Subject: fix(hub): Argon2 runs off the event loop, on exactly one worker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One derivation is 256 MB and a quarter to half a second of CPU (240 ms here, 485 ms on meshbay.org). All eleven call sites — sign-in, registration, the two rehashes, passphrase change, reset and account deletion — ran it inline in an async handler, so every one stopped the whole hub for that long: no request served, no node socket read, no offer relayed. Measured on a local hub during eight concurrent sign-ins, the worst `/v1/health` response went from 232 ms to 10 ms; the sign-ins themselves take the same time. It could not simply go to a thread pool. Two concurrent `lanes=4` derivations deadlock inside OpenSSL and never return, at no CPU — reproduced on cryptography 50.0.x / OpenSSL 4.0.x both locally and on meshbay.org, while `lanes=1` does not. `lanes` is part of every stored hash, so it is not ours to change, and inline on the loop two derivations could never overlap, which is the only reason production never hung. So `auth.hash_password_off_loop` / `verify_password_off_loop` hand the work to a dedicated executor with exactly one worker. Not a semaphore around `to_thread`: a cancelled request would release its permit while its thread was still deriving, and the next derivation would start beside it. One worker also bounds Argon2's memory to one derivation whatever the number of callers. `test_argon2_off_loop.py` reads every module for a direct call, pins the single worker, runs four derivations and four sign-ins concurrently to completion, and checks the loop keeps turning during a derivation; each fails with its guard removed. CLAUDE.md and AV9 state the rule and the trap. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01LcF3QKWii7uQ2kSyXErzCt --- docs/MESHBAY_DESIGN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/MESHBAY_DESIGN.md') diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 6b1fd05..6615f66 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -2666,7 +2666,7 @@ had already been asked. | **AV6** | **A relay proves possession of its approved key.** A public key is not a password, and the register call is unauthenticated by design — it is not a user — so the proof is the only thing standing between a stranger and where nodes send relayed traffic | | **AV7** | **A node bounds how many peers it holds and how long an unproven one lasts.** The hub's cap is per calling account, which is a limit on each member and not on the machine, so without this an operator's exposure grew with the size of their groups | | **AV8** | **One account cannot make the hub mail another at will.** The invitation email's subject comes from the group row, never from the request, and the endpoint is metered | -| **AV9** | **No mail is sent from the event loop.** `smtplib` is synchronous and waits up to ten seconds; called from an async handler that wait is the whole instance's, not one request's. Every send goes through `mail.send_off_loop` | +| **AV9** | **No mail is sent from the event loop.** `smtplib` is synchronous and waits up to ten seconds; called from an async handler that wait is the whole instance's, not one request's. Every send goes through `mail.send_off_loop`. **Argon2 is held to the same rule**: every derivation runs on one dedicated worker thread (`auth.*_off_loop`), never on the loop and never two at a time, because two concurrent `lanes=4` derivations deadlock in OpenSSL | | **AV10** | **Every path that makes the hub send mail is metered, per account.** A rate limit that counts by IP bounds a caller, not an inbox. Changing one's address mails an arbitrary stranger, so it carries a cooldown *and* a daily ceiling; a reset request and a registration resend carry cooldowns | | **AV13** | **The mail server is not a relay, and `mail.py` is where that is decided.** Every message passes one function; `purpose` is keyword-required and checked against a closed list, so a helper that names anything else does not send and one that names nothing is a TypeError. Under it sit a bound per **recipient** — the thing a person being mail-bombed actually experiences, unmoved by which account, address or endpoint asks — and an instance-wide hourly ceiling, because registration is open and "per account" is a bound an attacker buys more of | | **AV11** | **A namespace a client writes into is closed, and its rows are capped.** The preference key space is an allow-list plus `default_tab:` checked as a group id, the value is length-bounded, and the row count per account is bounded | -- cgit v1.2.3