aboutsummaryrefslogtreecommitdiffstats
path: root/docs/MESHBAY_DESIGN.md
Commit message (Collapse)AuthorAgeFilesLines
* fix(node): the node's own controls take no authority from a hub tokenChristophe Besson12 days1-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `_is_node_admin()` is `self._user_id == node_user_id`, and `_user_id` is the `sub` of a JWT the hub issued. Six node-wide controls were gated on that alone: `node_status` — which lists every group on the machine with each root's **absolute path** — plus `node_settings_set`, `roster_read`, `denylist_read`, `denylist_clear` and `node_reload`. So the answer to "are you the operator of this node" was "the hub says so", which NS4 and M3 rule out in as many words: operator authority comes from the node's roster and from nowhere else, and asking the hub is how the hub installs itself as node administrator. The reach is bounded — a completed handshake also needs the group key — but an active hub obtains one legitimately in an open-join group, which §3.5 concedes, and from there it could read the operator's directory layout or clear the denylist, which is the persisted revocation H4 exists to keep. `_operator_device()` requires both halves now: the account is the one the node belongs to, *and* the device on this connection has proved a key the roster holds as an operator. `device_hello` is signed over a transcript naming the node, the group and this connection's nonce, and `operator_pks()` is rebuilt from the roster on each call, so an unpinned browser and a revoked one are both refused at once. The hub holds no user keys and cannot countersign a device. Keeping the account check as well is deliberate: dropping it would widen these node-wide controls to any paired operator of any group on the machine, which is a separate decision. `_is_node_admin()` stays as what it is in the handshake ack — a hint telling a client whether to offer the Node page — and says so. Nothing changes for a paired operator: `device_hello` runs unconditionally after the ack, and anyone using the Node page's controls is already paired, since `root_add` and every other signed op has always verified against `operator_pks()`. A browser that never paired now reads nothing there, which is the state in which it could already write nothing. test_node_status.py's fixture set the account and not the device, which is how it went on passing; it now wires the device the way `device_hello` leaves it. The adversary itself is in test_security_regressions.py — a token naming the owner's account with no proved device, which the previous source answered with `node_status_ack`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
* fix(node): a transfer id names a lease, or it names nothingChristophe Besson12 days1-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `_do_file_request` read `tr` as a boolean. Present meant "this is a leased transfer, skip the leaseless ceiling", and nothing asked whether this node had ever granted such a lease — `slots.touch(tr)` was called beside it and its answer, `False` if it is not granted, was discarded. So any non-empty string bought the whole library with no ceiling of any kind: not the per-member cap, not the node-wide one, not the leaseless bound that exists to bound a client claiming to be browsing. The queue held only the clients that chose to wait. `_lease_of` decides it now, and the three answers differ on purpose: - **granted**, and of *this* session — served, and touched so the sweeper does not reclaim a transfer that is plainly moving. The session is checked as well as the id, because touching another connection's lease refreshed its idle timer. - **queued** — refused with `lease_not_granted`, on the upload path too, before anything reaches the operator's disk. A member reading while queued is the cap not applying. - **unknown** — bounded by the leaseless ceiling rather than refused. That is also what a reconnect looks like from here, where the session's leases died with the old connection and the client is re-opening them, and it leaves the residual §5.5 already states: a client that lies gets that bound's worth of files at a time, not the group. Noted once per connection so the residual is visible rather than merely documented. Nothing changes for the shipped client: the transfer store awaits `lease.acquire()` before it reads a byte, so the refused case is one it never enters. §5.5 gains a paragraph saying the node decides which of the two a request is — the document described the accounting without ever saying it was enforced, which is how it came not to be. `test_lease_enforcement.py` drives the real handlers over a real index; six of its nine cases fail against the previous source, each on the property. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
* fix(node): an uploaded file records who sent itChristophe Besson12 days1-6/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `_register_uploader` walked the index for the entry it had just written, at a moment when no such entry can exist: the file was a `.part` until the rename on the line above, which is not indexable, and the watchdog that will index it debounces for two seconds and then hashes. The walk matched nothing, silently, so every uploaded file in every group was owned by nobody — and `file_delete` refuses a caller with no admin authority when the entry records no uploader, so a member could not delete what they had just sent. MESHBAY_DESIGN.md §5.4 grants that to any non-revoked device of the uploading account. The record is now written when the last chunk lands (`indexer.record_upload`) and the entry is stamped from it in `_hash_or_cached`, the one funnel every entry passes through — initial scan, watchdog, reconcile and replug alike. It lives in the index cache rather than on the entry alone, because the index is rebuilt from disk at every start and an owner the node forgets on restart is a right quietly taken away. It is validated against a live `stat()`, so whatever later occupies that path inherits nothing; and `_rescan_root`'s carry-over no longer copies over it, or memory would beat the durable record. §5.4 also claimed ownership was *provable* — a transcript the uploader signs, stored with the entry. No such signature has ever existed; `meshbay:upload:v1` in the code is the groupbox purpose that seals the envelope. The section now states what the code does, and the transcript is an open item in §15.3. `test_upload_attribution.py` drives the real handler and a real indexer across that seam. Against the previous source its two positive cases fail on the property, not on a missing method — an upload, then a rebuild from disk, then a different file at the same path inheriting nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
* fix(packaging): the hub unit can start, and nothing carries the migration pathChristophe Besson13 days1-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `ExecStartPre` ran `alembic -c /opt/meshbay-hub/migrations/alembic.ini upgrade head`. The build does stage that file, so the path existed and the contents were wrong: `alembic.ini` resolves `script_location` with `%(here)s`, so the copy pointed at `/opt/meshbay-hub/migrations/src/meshbay_hub/db/migrations` — which nothing installs, because the migrations ship inside `meshbay_hub`, in the shared venv. `ExecStartPre` failing stops the unit. A hub installed from the RPM or the DEB could not start at all, and nothing noticed because the one live deployment was assembled by hand — the same shape as the node unit that carried `User=` into the user unit directory. The same `%(here)s` trap was already found once on the server, where a stray `alembic.ini` resolved to a month-old snapshot of the tree. Twice is a trap rather than an accident, so the fix is that the path is no longer written down anywhere: `meshbay-hub migrate` asks the installed package where its own migrations are, which is correct for the RPM, the DEB, a venv and a checkout. The build stages no `alembic.ini`; the repo keeps its own for `alembic revision` and for deploy scripts that already work. `env.py` now prefers a URL the caller resolved over re-reading the environment itself, so `migrate --config` connects with exactly the string the server will — one resolution, not two that agree until they do not. Six tests, three of which fail against the unit as it was. They read the directives rather than the file, because searching the whole thing finds the comment explaining a directive and calls that the directive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* feat(hub): the mail state is a panel section, and a ceiling falling is an eventChristophe Besson13 days1-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | The figure was a line beside the settings form, which is where it is changed and not where it is watched. It sits with the other live figures under Statistics now — four cards and, above them, a banner saying which of the two ceilings has fallen. The two states are not the same to whoever is reading: one means newcomers are turned away, the other means somebody locked out of their account cannot get back in. The settings block keeps a line pointing at it. And an operator no longer has to be looking. When a global ceiling is reached the administrators are notified — in `mail.py`, in its own session, never raising, because this runs while a request is being refused and an alert that fails must not turn a refusal into a 500. Once per hour, keyed on a row rather than a flag in memory: a flood is what spends the budget, so one alert per refusal would bury the message under its own cause, and a hub that is refusing mail is a hub somebody is about to restart. `/v1/admin/mail` gains `general_exhausted` and `all_exhausted` rather than leaving the panel to compare two numbers. Labels in all ten catalogues; `.warn-msg` for the middle state, on the `--warn` token both themes already define. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* docs: AV16–AV17, and why node --check is not the checkChristophe Besson13 days1-0/+2
| | | | | | | | | | | | | | AV16 the allowance that survives a restart and that an operator can see and change; AV17 the single worker. And a lesson that cost a full suite run: `node --check` reports success on a module-syntax error. An unclosed `.map(` inside a tagged template came back clean four times. `test_spa_syntax.py` says exactly this in its own docstring — copying to `.mjs` is what forces the module parser — and the reflex still reached for `node --check`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* docs: AV13–AV15, and two lessons from how they were foundChristophe Besson13 days1-2/+15
| | | | | | | | | | | | | | | | | | | | | | | AV13 the mail gate, AV14 the MHP audience and the hub's own identity, AV15 the shape check on the unauthenticated blocklist endpoints. §13.5b also records the auto-block threshold as an open decision rather than a defect. Two lessons, both about tests. A fixture narrower than production tests the fixture — and when it writes down *why* it has to be narrow, that is a bug report nobody filed. Two test helpers described the federation defects accurately, in comments, while routing around them. And a global's state is not a given, in a test least of all. One assertion — that a refusal never logs the address — took three attempts, each passing alone and failing in the full run, for three different reasons: caplog saw nothing because the app configures logging, then a handler saw nothing because an earlier test raised the logger's level, then because `disabled` was left True. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* docs: AV9–AV12, and the two shapes that keep producing themChristophe Besson13 days1-1/+21
| | | | | | | | | | | | | | | | The register grows by the four findings from users.py, and gains the pattern underneath several of them: a limit written on one of several equivalent paths (three endpoints send mail, one had no limit; every list in admin.py is bounded, the two outside it were not; the group claim was bounded at registration and not on reload), and a bound that counts the wrong thing (a per-IP rate limit bounds a caller, never the mailbox that receives what they cause). Also records that admin.py was read under the lens and needed nothing, so the next reader does not re-derive that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* docs: availability between members is a finding categoryChristophe Besson13 days1-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | The first three reviews asked who can read what, who can impersonate whom, and what a hostile node can forge. None asked what a legitimate but misconfigured or careless member costs everyone else — which is the question a group platform lives on, because every member was invited by someone who trusted them and none of them is an attacker. C2 had asked "can a node claim a group its owner is not in?" and the answer was correctly no. Nobody had asked what happens when a node claims one its owner *is* in but does not host, which is how a group went dark for all of its members with its real host online throughout. §13.5b is the register, AV1 to AV8. The lens, for anything reviewed from here: a participant supplies input; if anyone other than the sender bears the cost, there is a ceiling to write, and it goes on every path that writes the state. CLAUDE.md gets the working rule and the incident as a lesson, and the stale path to sync-ui.js corrected — it lives under scripts/, not build/. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* fix: an empty group claim is a claim on nothingChristophe Besson13 days1-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A node that hosts no groups sends no `group_ids` on its hub socket, and the hub resolved the claim with `set(claimed_groups or authorized)` — so "I host nothing" arrived as "I host every group this account belongs to", other members' included. Such a node can serve none of them: it holds no GEK, and its own handshake refuses them with "Group not hosted on this node". `/v1/groups/{id}/nodes` answers in registration order and `_node_groups` is in-memory, so which node a client was sent to depended on who reconnected first after a hub restart. GroupPage took `nodes[0]` with no fallback. On 2026-09-11 a hub deploy at 20:14 reshuffled the registry, a second member's unconfigured node won the race, and a group stopped opening for everyone in it with its only real host online throughout. Any member could take one of their groups down, by accident, by leaving an empty node running. Four changes, because no one of them is sufficient: - the hub never widens an absent claim, and `update_groups` goes through the same ceiling as registration — it assigned its list verbatim, so the bound that makes C2 hold at authentication was one message wide - the node states the empty set rather than omitting the field - the refusal carries `not_hosted`, so a client can tell "try the next node" from "you, here, must do something first" - GroupPage walks the list instead of indexing into it The three lines involved date from 13, 20 and 23 August and each is defensible alone. The defect is in the seam, which is where the last two also were: a falsy empty collection must never mean "unspecified". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T4YmK41VsEURWFdop4EEeT
* fix(hub): an administrator can erase an account that owns groupsChristophe Besson2026-09-111-5/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An administrator's deletion answered 409 for any account owning a group, so an erasure ordered by an authority had to wait on the person it was about. It now deletes the account's groups with it, then pushes a signed revocation for the account and for each group to every connected node: an access token already issued stays valid on a node until it expires, and the revocation is what makes the nodes refuse the account and close the groups' sessions now. The action is written to the IP log, and the confirmation dialog says the groups go too, in all ten catalogues. The owner's own deletion is unchanged: refused while they own groups, which they can hand over first (CGU 3.4, privacy statement). Deleting a group had three partial cascades. The owner's route left email_verifications behind, and the cleanup of unhosted groups left notifications, invitations and reports - each an IntegrityError on PostgreSQL, invisible on SQLite, which does not enforce foreign keys by default. db/purge.py is now the one implementation: it finds every table referencing groups.id from the schema, deletes the group's rows and detaches content reports, which are evidence and outlive the group. test_group_purge.py turns foreign-key enforcement on for its connection, seeds every referencing table, and fails without the fix on all three routes. MESHBAY_DESIGN.md 7.7 states the rule, and now lists the device keys and swarm sources that e3c68b3 erases. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9MCBBWSm9GhBESmqzJxNy
* fix(hub): the asset fingerprint covers every file under static/Christophe Besson2026-09-111-3/+4
| | | | | | | | | | | | | | | | | | | Everything under static/ is served at /a/<hash>/ with a year's `immutable`, but the hash was computed from a hand-kept list of 43 top-level modules. The ten catalogues and vendor/ were not on it, nor was anything the guarding test could see: it globbed *.js at the top level only. A change confined to the catalogues therefore kept the hash, and a phone went on showing a heading that had been rewritten and deployed - pull-to-refresh fetched the no-store shell, which was current, and never refetched en.js at a URL that had not moved. The fingerprint now hashes every file under static/, path and content, so a change, a rename or a new file moves the version with nothing to register. _ASSETS is gone, and CLAUDE.md, MESHBAY_DESIGN.md 9.4 step 6, assets/brand/README.md and docs/playlists.md no longer ask for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9MCBBWSm9GhBESmqzJxNy
* docs: remove the documents MESHBAY_DESIGN.md replacesChristophe Besson2026-09-111-4/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Twenty-four files, about 17 000 lines: the two architecture drafts, the three security reviews, eleven design notes, the roadmap, the decisions file, the v1–v4 archive, the deprecated user guide and the stale quickstart. Their content is in MESHBAY_DESIGN.md, and git history holds the originals. The reason to delete rather than keep bannered: a document that is superseded but present still gets read, and a reader cannot always tell which of two accounts of one mechanism is the live one. That was the argument for retiring the user guide rather than repairing it, and it applies to the whole set. What made this safe is the concordance. Roughly 290 comments and docstrings cite these files by section — `musicbay.md §6`, `mediacenter.md §5.5`, `draft-v6 §2.11` — and section 16 maps every one onto its replacement, so not a single comment needs editing to stay followable. It now says plainly that the files are gone and where to recover them, and it gained rows for the three reviews (their findings are section 13), and for the two guides. Four kept documents pointed into the set and were repointed first: `playlists.md` (nine references — it is a live proposal and must not dangle), `WINDOWS-PORT.md`, and CLAUDE.md's example. No dangling reference remains outside section 16. Two files were dropped from the list after checking what they hold. `HTTPS.md` is an operational runbook — Caddy, certificate renewal, DNS, troubleshooting — and MESHBAY_DESIGN.md deliberately covers no operations, so nothing would replace it; the versioned Caddyfile is the config, not the procedure. `cast-smart-tv.md` is the plan for the unbuilt DLNA phase of a feature whose first two phases ship, and section 11.4 summarises it in four lines rather than carrying the SSDP/UPnP work. There is no user guide now, and section 0.1 says so rather than leaving a reader to discover it. Suites green: 2258 passed, 4 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVoHVCcfBqud6ZjG4db3y7
* docs(guide): mark USERGUIDE.md deprecatedChristophe Besson2026-09-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is too far out of date to be worth repairing. It describes identity keys derived from a username and password, one `shared_dir` per group with an `uploads/` quarantine, ChaCha20 as the content cipher, a hub that stores users' public keys and the wrapped group keys, and a member wrapping that key for another member — which is finding H3, in the section that explains why the hub cannot read your files. The banner lists each of those against what is actually true, so that no section below it is mistaken for current, and points at MESHBAY_DESIGN.md and MESHBAY_NODE_PROTOCOL.md instead. It also records what the document predates entirely: encrypted chat, the sealed index and upload path, transfer leases, device linking, the application framework. Repairing it section by section is refused deliberately. Enough of it is wrong that a reader cannot tell the sound parts from the stale ones, which is worse than having no guide, and fixing one section leaves exactly that problem in place. The previous commit — which translated two French passages and corrected the errors immediately around them — is dropped for the same reason: it made a small part of a misleading document accurate, which makes the whole harder to distrust, not easier. There is no replacement user guide today. That gap is real and is better stated than papered over. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVoHVCcfBqud6ZjG4db3y7
* fix(node): the handshake ack dropped one app's directoriesChristophe Besson2026-09-101-1/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ack was assembled from its own tuple of application names, a copy of the daemon's `APP_DIR_KEYS`, and the two had drifted: the copy was missing `helloworld`. So the reference application — the one that exists to prove a new application needs no special-casing — was the single application whose configured folders never reached a client, which made the plugin claim false exactly where it is demonstrated. Fixed by removing the copy rather than syncing it. The ack now emits whatever `<app>_directories` the group context carries, and `_app_directories_ctx` is the only thing that puts one there, so the two cannot disagree again. The transport names an application in one place, `ALLOWED_APPS`, which is enforcement rather than a directory list. The client had the same fault one layer up: `group-page.js` read three names by hand from the ack while the live-update path beside it was already generic. It derives the map from the ack's own keys now, so the fix reaches the settings pane instead of stopping at the wire. A first attempt moved the list to `roster.py`, where directory *storage* lives, and `test_helloworld_proves_the_plugin_claim.py` refused it: the roster, the ops, the config and the root set must name no application at all. That test is the architecture's own guard and it was right — the list belongs on the daemon, which is what wires a group's context, and everything downstream is derived from it. Two new tests, both verified to fail against the previous shape: the ack carries an application the node names nowhere else, and the ack keeps no list of its own. `test_the_lists_are_read_under_one_name_each` now asserts the shell names no application rather than that it names exactly three. Two stale comments went with it — the ack's, which described scalars removed in 07ff8b4, and the client's, which said those scalars still rode the wire for MNP 1.0 peers that can no longer connect. Full suite: 2258 passed, 4 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVoHVCcfBqud6ZjG4db3y7
* docs: MESHBAY_DESIGN.md — one reference for the designChristophe Besson2026-09-101-0/+2628
Thirty documents under docs/ described this system between them: two architecture drafts, three security reviews, eleven design notes, a roadmap and a decisions file. Reading any one of them meant following cross-references into four others, and several were flatly wrong — the keystore KDF, the protocol version, and four features whose headers still said "not implemented" months after they shipped. This is the synthesis. It states design rather than history: a section says why the node wraps the group key itself, not which finding made it necessary. Development history, spikes and reversed directions are gone. The security findings survive as section 13, where each label names the invariant it stands for today rather than the defect it was reported as. Two things it is careful about, because hundreds of code comments depend on them. Every short label — C1, H3, NS6, T3, C5b, W2, E9, F1 — is defined in section 13, including the three colliding namespaces (each review numbered its findings from C1, and the code means the second review's). And section 16 maps every "<doc> section n" reference the code makes onto its replacement, so no comment has to be edited to stay resolvable. transfers-v1.md comes in from outside the tree with it. The lease design is section 5.5; what a synthesis cannot carry is that document's failure-mode analysis — every way a slot can be lost, every way a client can be left waiting — and what a live pass found after the work was called done. Every claim was checked against the code rather than the drafts. Suites green: 2256 passed, 4 skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YVoHVCcfBqud6ZjG4db3y7