fix(hub): Search reaches each group with one offer, and a busy hub is not a dead node0.15

The other half of the 4G failure. The page negotiated every group twice: the
sweep opened a connection, read the index and closed it, then the warm-up
opened the same group again. The sweep, the warm-up and the tiles each had a
concurrency ceiling of their own, and together they went past what the hub
admits per account. Whatever the hub refused was then reported as "node
unreachable" and remembered as down, which put that group last next time.

- Every connection goes through ConnectionPool, which holds the page's one
  ceiling (six at once, sized for twenty groups on a phone) and keeps what
  the sweep opened for the tiles. A visit costs one offer per group. A
  refresh costs none for a connection that answers a four-second ping, and
  a connection that died while the phone slept is replaced, not waited on.
- A connection whose index is being read is held against eviction. With
  more groups than the pool keeps, it was otherwise the least recently used
  one.
- Negotiations still under way when the page closes close what they get,
  and a sweep cut short that way remembers nobody as down.
- transport.js sends an offer again on 429, 502 or 503, honouring
  Retry-After, with jittered waits of about twenty seconds at worst. Search
  counts each retry as progress. A 404, 403 or 504 still fails at once, so
  a dead node costs no time.

The fan-out tests assumed a ceiling of three and were re-measured: four dead
groups of twelve now hold nothing back, even on a first visit. The pool and
the retry run as shipped code, lifted as text, against a fake clock. Each
guard was checked by removing it and seeing its test fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

1 files changed, 37 insertions, 3 deletions

diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md
index 325a3c5..2e46d1a 100644
--- a/docs/MESHBAY_DESIGN.md
+++ b/docs/MESHBAY_DESIGN.md
@@ -1770,6 +1770,29 @@ share an active group with the target node**. Otherwise any authenticated user
 could make a third party's machine allocate peer connections on demand (**H6**).
 The address in a NAT-punch request must match the caller's source address.
 
+**The offer ceilings bound a member and never an ordinary one** (**AV30**). They
+are sized against an account with twenty groups on three devices, because a
+refused offer is indistinguishable, to the reader, from a node that is down:
+
+- **Per account and per node, a budget**: a burst of 120 offers, refilled at two a
+  second (`OFFER_BURST`, `OFFER_REFILL_PER_S`). This is what bounds a member's cost
+  to one machine, and it spares the others — an account that has spent it on one
+  node still reaches every other. It is counted by account, not by address,
+  because a mobile carrier puts many subscribers behind one IPv4 address.
+- **Per account, 32 offers pending** at once, across devices and nodes — an offer
+  is pending for the node's round trip, or up to the fifteen-second answer timeout
+  when a connected node is silent.
+- **Per address and per node, 600 a minute**: a coarse guard in front of
+  authentication, and nothing more.
+- **Every refusal carries `Retry-After`, and a client sends the same offer again**
+  on 429, 502 or 503 — the hub busy or restarting — with growing, jittered waits
+  (`postOffer` in `transport.js`). Any other failure, 404 for a node that is not
+  connected above all, fails at once, so a dead node costs no time.
+
+The node bounds its own total separately (`MAX_PEER_SESSIONS` in
+`webrtc_server.py`), which is the limit that protects the machine whatever the
+number of members.
+
 **A node registered for no group shares one with nobody**, and is refused rather
 than exempted. Written as "check membership if the node claims any group", the
 rule skipped itself — membership, group status and the public-group gate
@@ -2731,11 +2754,21 @@ still lets silent nodes hold every place at once. The ordering is advisory —
 storage that refuses, a private window or a first visit all fall back to the
 hub's own order, which costs that one sweep and then corrects itself.
 
+**Search reaches each group with one connection, through one pool.** The sweep,
+the warm-up and a tile asking for its group all go through `ConnectionPool`,
+which negotiates at most six at once (`MAX_IN_FLIGHT`) and keeps what the sweep
+opened for the tiles, up to its size; a group past that connects when a tile of
+it scrolls into view. So a visit costs one offer per group, a refresh costs none
+for a connection that answers a short ping, and a connection whose index is being
+read is never the one evicted. Separate ceilings for each of these add up behind
+each other's backs, past what the hub admits per account (§7.2).
+
 **Search dials many nodes at once, so its deadline measures stalling rather than
 elapsed time.** Opening one group from the sidebar waits out the transport's own
-budgets; Search opens up to a poolful, three at a time, and waits for the slowest
-of each batch — so the deadline it imposes is what a page of unreachable groups
-costs in spinner, and raising it to accommodate a slow link taxes every dead one.
+budgets; in Search a dead group holds one of the pool's places for the deadline —
+so the deadline is what a page of unreachable groups costs in spinner, and
+raising it to accommodate a slow link taxes every dead one. A hub that refuses an
+offer for load is answering, and each retry counts as progress.
 A node that is not there reports no progress and fails in one window. A node that
 answers ICE, then opens a channel, buys another window at each step, to a ceiling,
 because a deadline that only ever resets has none. A flat budget cannot tell a
@@ -3119,6 +3152,7 @@ had already been asked.
 | **AV27** | **A free-text third-party search is bounded per member and per node** (§6.5). `tmdb_search_req` spends the *operator's* credential, which TMDB rates and the whole group's automatic matching depends on, so one member holding a search box degrades the library for everyone. Per member and not per connection — three tabs is one person — and kept in the group context so a reconnect does not reset it. The refusal is an error, because an empty result list is what "no such film" looks like |
 | **AV29** | **An invitation link is bounded on both halves and its mail on the sender** (§3.4, §7.3). Twenty outstanding per group on the node (bearer codes) and on the hub (tickets); and because a link mail reaches an address the hub has no relationship with, at the request of anyone who owns a group, it is counted **per sending account per day** (`mail.invite_link_daily_cap`, 10), under the recipient and instance bounds and outside the recovery reserve (`invite_link` is not a recovery purpose) |
 | **AV28** | **How many node keys one account may announce is bounded** (§7.2). Each is a row plus an IP-log row under a one-year retention, so an account in a loop writes a year of storage on the operator's disk having paid only for signatures. Proof of possession (**M8**) settles whose key it is and not how many. Counted only where a row is added: re-announcing a key already held keeps working at the ceiling, or a node that reached it could never refresh its address again |
+| **AV30** | **What one member's offers cost a node is bounded per account and per node, and the bound admits the heaviest ordinary account** (§7.2). Each offer makes the node allocate a peer connection. A budget of 120 per node refilled at two a second bounds a member there without touching their other nodes, and it is counted by account because a mobile carrier shares one IPv4 address among many subscribers. Pending offers are capped at 32 per account. Both refusals carry `Retry-After` and the client retries them, because a refused offer otherwise reads as a node that is down |
 
 ### 13.6 Chat design findings