summaryrefslogtreecommitdiffstats
path: root/packages/meshbay-node
Commit message (Collapse)AuthorAgeFilesLines
* refactor(node): move the session core out of webrtc_serverChristophe Besson30 hours3-322/+342
| | | | | | | | SessionCore in transport/webrtc/core.py, last among the bases: state, the data channel, task ownership, the peer registry, sending and teardown. The facade keeps _dispatch_message and WebRTCTransport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move operator authority and the admin response out of ↵Christophe Besson30 hours2-307/+318
| | | | | | | | | webrtc_server AdminMixin in transport/webrtc/admin.py: who the operator is, the signed challenge, signature checks, and _do_admin_response moved unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the operator's group controls out of webrtc_serverChristophe Besson30 hours3-368/+383
| | | | | | | | GroupOpsMixin in transport/webrtc/group_ops.py: member revocation and unpinning, the group key rotation, apps and their directories with the allow-list, and Search listing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the operator's node controls out of webrtc_serverChristophe Besson30 hours2-563/+588
| | | | | | | | NodeOpsMixin in transport/webrtc/node_ops.py: status and settings, roster and denylist, roots, hosted groups, reload, scan pacing and transfer limits, with their class-level bounds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): section headers follow the code they introduceChristophe Besson30 hours3-21/+16
| | | | | | | | Three headers stayed in webrtc_server when their sections moved; they now sit above the same methods in blobs.py and admission.py. Two that were only a title over what is now a whole module are dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the MNP handshake out of webrtc_serverChristophe Besson30 hours3-355/+373
| | | | | | | HandshakeMixin in transport/webrtc/handshake.py: challenge, proof, channel binding, and the sealed configuration a peer receives once admitted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs(node): two stale comments in webrtc_serverChristophe Besson30 hours1-6/+2
| | | | | | | | The peer-session ceiling no longer quotes the hub's old per-account number, and the note on a single uploads/ directory goes: its constant left with f3fb449, when an upload started landing in the folder it was sent to. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move invitations, joins and device linking out of webrtc_serverChristophe Besson30 hours2-847/+864
| | | | | | | AdmissionMixin in transport/webrtc/admission.py, with the invite handle pattern and the join attempt bounds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move uploads out of webrtc_serverChristophe Besson30 hours7-416/+422
| | | | | | | UploadMixin in transport/webrtc/upload_handlers.py, with the upload cap; _upload_chunk moved unchanged, _append_chunk joins webrtc/disk.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move transfer slots and leases out of webrtc_serverChristophe Besson30 hours3-262/+285
| | | | | | | TransferMixin in transport/webrtc/transfer_handlers.py, with the sweep interval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move file browsing and folder/file ops out of webrtc_serverChristophe Besson30 hours7-473/+502
| | | | | | | FilesMixin in transport/webrtc/files.py; the blocking disk helpers join _locate in webrtc/disk.py, and the lease states go to webrtc/limits.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move group chat out of webrtc_serverChristophe Besson30 hours4-654/+670
| | | | | | | | ChatMixin in transport/webrtc/chat.py: sealed messages, history, epoch keys, link previews with their cache and rate bounds, and the operator's chat ops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move per-account blobs and key bundles out of webrtc_serverChristophe Besson30 hours3-244/+262
| | | | | | | BlobsMixin in transport/webrtc/blobs.py, with the blob caps and the kind pattern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move video streaming out of webrtc_serverChristophe Besson30 hours12-652/+663
| | | | | | | StreamingMixin in transport/webrtc/apps/streaming.py: stream credit, handover, the transcode slots and _stream_video_inner, moved unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the Videos catalogue handlers out of webrtc_serverChristophe Besson30 hours3-716/+739
| | | | | | | | VideoMetaMixin in transport/webrtc/apps/video_meta.py: media and season meta, TMDB search and its per-member bound, posters, and the operator's TMDB ops. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the MusicBrainz switch to the Music mixinChristophe Besson30 hours2-38/+40
| | | | | | An operator op that exists for one app lives with that app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the Music handlers out of webrtc_serverChristophe Besson30 hours3-192/+212
| | | | | | | MusicMixin in transport/webrtc/apps/music.py: tags and cover art, and the audio transcode with the extension list that gates it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move the subtitle handler out of webrtc_serverChristophe Besson30 hours5-176/+199
| | | | | | | SubtitlesMixin in transport/webrtc/apps/subtitles.py; _locate, which it shares with the files, music and streaming handlers, in webrtc/disk.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(node): move framing, shared limits and ffmpeg jobs out of webrtc_serverChristophe Besson30 hours6-343/+383
| | | | | | | transport/webrtc/channel.py, limits.py and media_tools.py, cut from webrtc_server.py as text; the facade imports them back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(node): record what the session does with every message it can be sentChristophe Besson30 hours2-0/+24943
| | | | | | | | A characterisation test over every MNP type, peer state and message shape, and every signed operation's answer: the reply, the audit, the handler started. It pins dispatch order and the admin table so they can be moved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(node): a patched module attribute must be read where it is patchedChristophe Besson30 hours1-0/+118
| | | | | | | A function reads its globals from the module that defines it, so patching a name a module only re-exports changes nothing and the test passes anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test: read the WebRTC transport's source as a set of filesChristophe Besson30 hours11-93/+160
| | | | | | | | | Source-reading tests take their text from node_source (node) and node_tree (hub): webrtc_server.py plus anything under transport/webrtc/, so a check for something's absence keeps reading the code it guards if that code moves. test_node_source_scope holds the boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(node): a connection that ends leaves its group's peer setChristophe Besson30 hours2-0/+70
| | | | | | | | | | connectionstatechange dropped a closed or failed session from the transport and stopped its tasks, but only close() took it out of the group's peer set, and nothing on that path called it. Every broadcast then went to closed channels, and each reconnect left a dead session held until restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(hub): offer ceilings that bound a member without failing an ordinary oneChristophe Besson46 hours1-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | A phone on 4G found one of five groups missing from Search on half its visits, and all but one on a fifth of them. Every offer that reached the node was answered in about a second; the hub refused the others with 429 — 93 of them in half an hour on meshbay.org, all from that phone, all to nodes that were up. Two ceilings did it. Three pending offers per account was exactly what one Search page dialled at once, so the first tile or reconnection beside the sweep was refused. Thirty offers a minute per address and per node was spent by four or five reloads of a page whose groups share one node. Sized now against an account with twenty groups on three devices: - 32 offers pending per account, across devices and nodes. - A budget per account and per node, a burst of 120 refilled at two a second. This is what bounds a member's cost to one machine (H6), and it leaves their other nodes alone. Counted by account, because a mobile carrier puts many subscribers behind one IPv4 address. - 600 a minute per address and per node, as a coarse guard in front of authentication only. Both refusals carry Retry-After, which the browser now honours. The node's own ceiling on peer sessions is unchanged; its comment no longer quotes the old per-account number. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(node): member invite --link and member cancel in the CLIChristophe Besson2 days6-6/+274
| | | | | | | | | The CLI makes both halves itself — the node's code, then the hub's ticket bound to the address — and prints the link; a refused ticket takes the code back, and cancel takes back both. The CLI never asks the hub to mail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docs: drop the invitation-link plan from the treeChristophe Besson2 days5-5/+5
| | | | | | | | The plan was a working document; what it decided lives in MESHBAY_DESIGN.md and MESHBAY_NODE_PROTOCOL.md. Code and tests now cite those instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(node): invitation links — a code bound to no account until redeemedChristophe Besson2 days7-25/+695
| | | | | | | | | | New invite kind "link": member of one group, once, never operator, not spendable by an active member, capped at 20 per group, cancellable by handle. Signed ops invite_link_create / invite_cancel, loopback routes, and the known-device join path now accepts a link code. Adds the plan, docs/invite-links.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat: the node signs its handshake challenge (MNP 3.4)Christophe Besson2 days4-4/+109
| | | | | | | | | node_pk in handshake_challenge is now signed over the channel binding and both nonces, so a client can check the node key before a join rather than only at the ack. Both transports; the browser and the QUIC client refuse a wrong signature and treat an absent one as an older node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* perf(hub): bound video read-ahead by bytes, not by secondsChristophe Besson5 days1-2/+10
| | | | | | | | | | A browser limits bytes, so a bound in seconds had to be sized for the highest-bitrate file and every ordinary one then held a fraction of what the same buffer would have taken. Floored at the old 90 s so nothing pulls less than before, and walked up rather than declared. A refused append now waits for an eviction instead of retrying on every tick. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: removing someone who never redeemed their invitationChristophe Besson5 days4-6/+149
| | | | | | | | | | A member row appears only when a code is consumed, so revoking someone invited to the wrong group was refused for having no row — and the node's refusal aborted the browser's removal before its hub half, leaving them a member everywhere with a live code. Revoking now cancels unredeemed codes for that group, and a node refusal no longer cancels the hub removal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: bump version to 0.15.0Christophe Besson6 days2-2/+2
| | | | Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Merge origin/main: the tree-wide ruff pass beside the Music reconnect workChristophe Besson6 days100-532/+356
|\
| * refactor(node): delete the revocation subscriber H4 replacedChristophe Besson6 days1-149/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `RevocationSubscriber` kept its revocations in two in-memory sets. That is the defect **H4** names: revocations that live only in memory mean a node restart silently un-revokes everyone, and group revocations were dropped entirely. The fix was the persisted `Denylist`, fed from `hub_client.maintain_ws` through `daemon.py`'s `on_revocation`, which verifies the token against the hub key and calls `deny_user` / `deny_group`. So this is not merely unused code. It is the version the register records as wrong, left beside the one that replaced it — and its docstring still opens with "Usage in daemon:" and three lines of instructions for wiring it up, which is the worst shape dead code takes: it reads as the way to do the thing. **L7** is about exactly this, and the index cache removed two days ago was the same shape. Certainty before deleting, since that was the condition: no import of it anywhere, static or dynamic; no entry point; nothing in packaging, the systemd units, the man page, QE or the documentation; and no test covers it. Every module in the three packages still imports, ruff passes, and the suite is 2893 — unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| * style: the 98 ruff could not fix, so the linter is a signal againChristophe Besson6 days15-43/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pass before this applied ruff's own fixes. These are the ones needing a decision, and the point of doing them is that `ruff check .` now passes: a linter reporting 98 known-acceptable findings reports nothing, because the next real one arrives invisible. **Lines over 100 (70).** Mostly wrapped where they stood. Two exceptions: the aligned trailing comments in `protocol.py`'s message table were shortened rather than wrapped, because wrapping one row of a table breaks the table; and in `models.py` the column comments moved above their columns for the same reason. **Imports below the first statement (14).** `csam.py` kept its FastAPI imports under a section header halfway down the file; two node tests had a constant and a `pytestmark` wedged between two import blocks. Moved, not suppressed. **Bindings nothing reads (4).** Three in tests, where the call stays and only the name goes — `_user(client, "listener")` is there to create the user, not to return one. The fourth was in `revocation.py` and was not a lint finding at all: `_connect_and_listen` opened an httpx stream to the WebSocket URL, did `pass`, and then opened the real connection through the `websockets` library. One pointless request per connect, left over from before that library was used directly. Removed, and `httpx` with it. **`l` as a name (4)**, **semicolons (6)** in the POC spikes, and the rest. 2893 passed, the same count as the two commits before it. `meshbay_node/revocation.py` is worth a decision separately: 154 lines that nothing imports, superseded by `hub_client.maintain_ws`'s `on_revocation`. This commit only stopped it failing the linter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| * style: ruff's own fixes, mechanically appliedChristophe Besson6 days96-348/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `ruff check .` had gone unrun long enough to report 568 errors, which is the same as having no linter: the next real finding would have been invisible in the noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the 98 it cannot fix are the next commit. What actually changed: import sorting (225), imports nobody used (87, none of them a re-export — no `__init__.py` is touched, which was the one way this could have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69) and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12 this project requires, `Optional[X]` to `X | None` (24), and f-strings with nothing to interpolate (19). Checked rather than assumed: every module in the three packages still imports, and the suite is 2893 passed — the same count, test for test, as the merge before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | fix(node): the Music app's tags survive a restartChristophe Besson6 days3-31/+365
|/ | | | | | | | | | | | | | | media_cache held video_meta, photo_meta and thumbs; the audio tags lived only in the in-memory IndexEntry. So every start re-read every audio file the node serves, and until that pass landed it served an index with no artist on any track — one the Music app cannot group. Over a real 6176-file library the pass costs 27.8s cold and 6.4s from audio_meta. Only what the bytes decided is stored. The filename and folder fallbacks still run live, or a renamed file would get the old name's answer; the sibling-cover scan reads the folder, so it stays live too; and a read that failed is not cached, or one bad read becomes permanent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Merge origin/main: the operator's upload ceiling beside the disk-thread workChristophe Besson6 days10-73/+482
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One conflict, in §15.3's open list, and it was two changes agreeing rather than disagreeing: this side removed the rows for the third-party search bound and the node-announcement bound because both are now built (AV27, AV28), while the other side kept them and added a new one. Resolved by keeping what is genuinely still open — per-device revocation having no CLI — and leaving the two closed. `webrtc_server.py` merged without conflict but the two sides met inside one function: `_upload_chunk` gained the operator's `max_upload_gb` ceiling from there and the per-group lock and `off_disk` calls from here. Read back rather than trusted: the operator's ceiling now sits inside the critical section that keeps chunk ordering, and the unlink beside it goes to the disk thread with everything else. 2893 passed. The twelve `test_sticky_header.py[firefox]` setup errors are the open Firefox on this machine, as before.
| * fix(node): node.toml's transfer pools reach the transportChristophe Besson7 days5-65/+184
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The daemon built the defaults dict for `Roster.node_settings` by hand and left out `max_concurrent_downloads` and `max_concurrent_uploads`. Absent from the dict, both resolved to None, were assigned back onto the config, and the transport skipped them — so node.toml was parsed, validated, and then replaced by `transfers.py`'s own 8. Invisible to anyone who left the value at 8, which is the value the template suggests. There were three copies of that dict and they all disagreed: node_status' was missing those two and `max_upload_gb` besides. One builder now, `config.node_settings_defaults`, and the resolver's key list is a class attribute the tests hold it to, along with the writer's. 1429 passed against a baseline of 1423; the two new behavioural tests fail with the builder reverted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| * feat(node): per-file upload ceiling is an operator setting, default 8 GBChristophe Besson7 days9-13/+303
| | | | | | | | | | | | | | | | | | | | | | Was a 4 GB constant in webrtc_server.py, the same on a small board and on a machine holding a library. Now max_upload_gb in node.toml, on the Node page and via `meshbay-node transfers max-size`, read from the transport context per chunk so a change reaches an upload already running. MESHBAY_DESIGN.md §6.4; §15.3 records a defect found beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | fix(node): the node decides which files it will transcode, not the playerChristophe Besson6 days2-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `BROWSER_INCOMPATIBLE_AUDIO_EXTS` named the two formats no browser decodes and was read by nothing. The player asked for `.wma` and `.mpc` and no others, and `music-player.js` called itself "kept in sync with the node's" constant — so the entire restriction lived in the caller, and a member's own message is not the caller. What that let through: this converts a whole file and holds a transcode slot shared with video streaming while it runs. One message naming a two-hour film spends minutes of the operator's CPU and a slot every other viewer is queued behind. `AUDIO_TRANSCODE_MAX_BYTES` catches the result, after the work is done; only the extension catches the work. The client comment now says what it is — an optimisation that saves asking for a refusal — rather than implying it is the rule. Two tests: a film is refused before ffmpeg is reached, and a WMA still passes, because a gate that refuses everything is not a gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | fix(node): bound what a transcode may produce, and read ffmpeg's output off ↵Christophe Besson6 days2-15/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the loop Two things about the same three functions, which write to a temp file with ffmpeg and then read it back. **The read was on the event loop.** These files are ffmpeg's own, under `tempfile.mkstemp` on the system disk, so they are not a group root and there is no spun-down platter to serialise against — which is why they go through `asyncio.to_thread` and not `roots.off_disk`. But a whole transcode read inline is still tens of megabytes of blocking read while nothing else in the node is served. The AST guard now covers the module with no exemption at all, and a second check refuses a direct call to the reading helper: passed to `to_thread` it appears in the syntax tree as a name, called inline it appears as a call. **The audio transcode had no size ceiling**, where the subtitle path beside it has had one all along. The bound is the media cache's rather than memory's: `put_thumb` writes one SQLite row, and the store is 512 MB with least-recently-used eviction, sized for thumbnails, posters and short transcodes. At 192 kbit/s a three-hour source is ~260 MB — one row that evicts most of the cache to fit and is evicted again by the next few thumbnails. Not a size this store can hold usefully. 64 MiB, about forty-five minutes: past any track, any single piece, most sets. It takes away nothing that worked. `AUDIO_TRANSCODE_TIMEOUT_SECS` is 120, so a source long enough to reach this was already liable to be killed mid-transcode; what changes is that the refusal now names the limit it met and the size that met it. Serving audio of that length properly means streaming the conversion instead of buffering it, which is a different feature — recorded in §9.8 rather than left as an implied promise. The stat comes before the read, so an oversized result costs a stat rather than the read and the memory behind it. Twelve `test_sticky_header.py[firefox]` setup errors again: Firefox is still open on this machine, and its `[chrome]` half passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | fix: the two ceilings §13.5b was still missing, as AV27 and AV28Christophe Besson6 days3-0/+251
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | **A free-text TMDB search spends the operator's credential.** TMDB rates it, and the automatic matching every member sees runs on the same one, so a member holding down the search box — or a script doing it — degrades the library for everyone and empties a quota the operator pays for. The handler had no ceiling of any kind, where link previews beside it carry two. §6.5's standing rule is a bound and a named adversary in the same commit; this arrived with neither. Per member and not per connection, unlike link previews: three tabs is one person, and a ceiling a tab can multiply is not a ceiling. Kept in the group context, so a reconnect does not reset it — a client that drops its channel between searches would otherwise have no ceiling at all. The node-wide window stays too, because the two answer different questions: one keeps a member from spending everyone's quota, the other keeps a roomful of them from doing it together. Ten a minute each, thirty for the node — a search every six seconds, sustained, is past what anyone types. The refusal is an error rather than an empty list. An empty list is what "no such film" looks like, and telling somebody their film is unknown when the node simply declined to ask is a worse answer than the truth; `video-app.js` already puts `detail` on screen. **How many node keys one account may announce.** Each is a row in `nodes` plus a row in the IP log, and the IP log is kept for a year, so an account in a loop writes a year of storage on the operator's disk having paid only for signatures. M8 settled whose key it is and said nothing about how many. Ten: a node is a machine left running, and an account wanting an eleventh *identity* rather than an eleventh machine is the case this refuses. Counted only where a row is added. Applied to every announce it would freeze the address of every node an account already runs the moment it reached the limit, and a node that cannot re-announce is unreachable after its ISP renumbers it — an outage caused by the protection. There is a test for exactly that. Both tests are two accounts, per §13.5b: a ceiling one person can exhaust for another is not a ceiling but a queue, and a ceiling shared between accounts would let one member stop every other from bringing a machine online. Checked by removing each ceiling: seven tests fail. `test_season_and_search_requests.py` built its session without a `_user_id`, which production guarantees — `_dispatch_message` refuses every message until the handshake settles it. The fixture was narrower than the node, so it could not exercise a per-member bound at all; it has one now. Twelve `test_sticky_header.py[firefox]` setup errors in a full run here: Firefox is open on this machine, the trap CLAUDE.md describes, and its twelve `[chrome]` tests covering the same geometry pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | docs: cite MESHBAY_DESIGN.md and a section instead of the merged notesChristophe Besson7 days45-228/+253
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and deleted from the tree on 2026-09-11, but ~230 comments across the three packages still named them — usually written `docs/musicbay.md §3.2`, as though the file were still in docs/. A reader had to know §16 existed to resolve any of them. They now name the section directly. Every mapping comes from §16, the concordance, which already records where each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the Videos app and §6.5 where the subject is derived data, photos -> §9.9, auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 -> §3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys -> §4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section. Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`) are retargeted too: those collide with real section numbers in the design document, so leaving them would have been worse than the named form. Four cases the concordance does not cover, each decided rather than guessed: Sub-item references into documents that no longer exist — mediacenter's `§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and §9.2 do not reproduce. The module-level citation stays; the sub-item pointer is dropped. The V-findings keep their labels but lose the dead `§10.1/` prefix. §13.8 lists V1–V13 as per-application open items, which is not what the labels mean in these comments, so pointing them at §13.8 would have been a false citation. `apps.md §5`'s virtualization requirement has no counterpart anywhere in the design document. The requirement is stated in the comment itself, so the citation is dropped rather than aimed at a section that does not say it. Comments that attributed a *sentence* to an old note — musicbay's "several thousand files" example, its "what got measured" note, its measured ~11%/~26% cover-art figures, the "original no root, whole shared tree" call — state the fact without attribution now. §9.8 does not contain those sentences and citing it for them would have been wrong. CLAUDE.md's "a reference to a document that no longer exists" row now says the concordance is for git history and out-of-tree material; the code cites sections directly. Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox leg of test_sticky_header.py's browser harness, which is broken at the browser level on this machine — headless Firefox (snap) dies with `[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders nothing, and the probe exits `{"error": "no measurement"}` after its full 90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing here can affect it: every changed line in style.css is inside a comment. Also checked: ast.parse on every changed .py, `node --check` on every changed .js, the /* */ balance in style.css, and that no changed line exceeds the width its file already used. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* | docs: drop the references to design notes that live outside the repoChristophe Besson7 days1-21/+18
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Seventeen comments across the Windows packaging targets pointed at C:\Users\admin\devel\light-client.md and msix-installer.md -- absolute paths on one developer's machine, unreadable to anyone else who clones this repository and unverifiable by any test here. The surrounding prose already carried the substance in every case, so these are removals, not rewrites, with two exceptions where the pointer was doing real work: electron-builder.msix.yml's header told the reader to go read §4 and §8 first. It now states the fact directly (an AppX/MSIX install never elevates, by design, so the package carries none of installer.nsh's elevation logic) and says the open items are called out at each declaration below -- which they already were, at `capabilities` and `customExtensionsPath`. The two "msix-installer.md §8" citations become "an open item" / "the other open item", beside the description of the item that was already there. Paragraphs the removals left ragged are re-wrapped. Verified: node suite 1403 pass / 4 skip (71 of them test_packaging_win.py), both electron-builder configs still parse as YAML, node --check on main.js. The .ps1 edits are inside <# #> comment headers plus one deleted Write-Host in a block that keeps two others; no pwsh on this machine to parse them. Not touched, and much larger: ~230 comments elsewhere in the codebase cite per-feature design notes (musicbay.md, mediacenter.md, auth-confirm.md and twenty more) that were merged into docs/MESHBAY_DESIGN.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): creating and removing a folder are syscalls tooChristophe Besson7 days5-53/+122
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The last two handlers on the loop. Both were synchronous, so a member creating a folder on a root that had spun down held the node for the spin-up, exactly as a chunk read did. Where a check and an act belong together they are now one call rather than two awaits, and the single disk thread is what makes that atomic: `_mkdir_if_absent` so two members creating the same name cannot both find nothing there and have the second `mkdir` raise where a refusal was meant, and `_rmdir_if_empty` for the reason the caller already re-tested emptiness — the first test happened before a round trip to the operator's browser, and a file can land in between. Two awaits would reopen that window one size smaller. The guard is now the whole class rather than the calls that were fixed. It walks the module's syntax tree and fails on any filesystem call outside the handful of functions written to be run through `off_disk` — a new handler that stats a root inline would pass every measured test, because those exercise the handlers that exist today. Checked by putting a call back: it names the function and the line. It leaves ffmpeg's own scratch files out, listed rather than silently allowed: they are under `tempfile.mkstemp` on the system disk, not on a group root, so they are not what spins down — but they do read a whole transcode into memory from the loop, and the day that matters it is a different measurement from this one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): take the availability poll and every upload write off the loopChristophe Besson7 days12-103/+247
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rest of AV9's disk half. Serving a file left the loop in the commit before this one; two paths were still on it. **The availability poll.** `RootSet.refresh_availability` stats every root, and eleven call sites reached it from `async def` — the reconcile loop among them, on a timer. On a sleeping disk that is a stall once per tick, and the stat is also what keeps the disk awake, so a node paid spin-up for a library nobody was reading. All eleven now go through `off_disk`, `Root.is_live` included. **The upload write.** `open`/`write`, and the resolve, the stat, the free-name search, the rename and the unlink around it. This one could not simply be awaited: the handler was synchronous, so nothing could come between the `chunk_index != state.next_index` check and the `advance` that answers it, and that is the whole of the chunk-ordering rule. Awaiting the write opens the gap — chunk 1 arriving while chunk 0 is in the disk thread reads a position that has not moved and is refused as out of order, so an upload would fail on a slow disk and nowhere else. Verified, not assumed: without the lock the new ordering test refuses three chunks of four. So the check, the write and the advance are one critical section again, under a lock held **per group**. Not per session: `partial_uploads` lives in the group context so a reconnecting client finds its upload where it left it, which means two sessions of one member share the position of one `.part` file. Arrival order is preserved by construction — the dispatcher creates one task per message as it arrives, tasks start in creation order, and the lock is the first thing each one waits on, so its waiters queue in arrival order too. `_do_file_upload` is a coroutine now, which is why forty-two test call sites gain an `await`. Their outcomes are unchanged, file by file, against the run before the change. `test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`, imported rather than defined there. It now asks for the ones written in the module, which is what its own docstring means; all forty-three operations are still checked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): serve a file from a disk thread, never from the event loopChristophe Besson7 days5-34/+338
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A root that has spun down, or that lives on a network mount, answers its first syscall in seconds rather than microseconds. Made from the event loop, that stalls the whole node: no other group is served, no stream is fed, no chat message is delivered and the hub socket is not read, for as long as the platter takes to come back. It was found from the other end — a client's connection attempt timing out on a node with one member, while the disk woke up — and it is AV9's lesson with the disk in the place of the mail server. Every filesystem call on a group's content now goes through `off_disk`, onto a thread that belongs to that group's root set. The stat goes with the read: a stat is what *wakes* a sleeping disk, so offloading only the read would move the stall rather than remove it, and the read would then find the disk already awake. `_locate` is the one place allowed to call `entry_abs_path`, which is `Path.resolve()` and therefore syscalls too. Five handlers touched: a chunk request, an audio transcode, a subtitle request, a video stream and a delete. The delete became a coroutine, which its one caller already was. One worker per root set, not a pool and not one for the node. One worker for the same reason the indexer's executor has one — two interleaved reads of a spinning drive seek-thrash rather than go faster — and it keeps two threads from being inside the same file at once, which is what makes the `seek`/`read` pair safe without a lock. Per root set, because a node serves several groups and their roots are not all on the same volume: a single worker would put one group's sleeping USB drive in front of another's SSD, which is this symptom one level down. The thread is created on the first read, so a group nobody downloads from never starts one, and the daemon stops them all on the way out. The tests measure rather than read: a ticker counts its own wake-ups beside a request made slow on purpose, and a handler that blocks the loop takes every one of them with it. Put either call back inline and both tests report zero wake-ups, which was checked before they were trusted. QUIC still reads on its loop. Its handler is synchronous by construction, no client speaks it, and its distance from parity is already recorded in the design document (§15.3, L3); moving it is part of bringing it to parity, not of this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat: decode and re-encode video on the GPU where there is oneChristophe Besson7 days6-22/+677
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A 1080p film is decoded by whoever watches it and re-encoded by the node when no browser can decode the source. Both were on the CPU, and on an Atom or Celeron mini-PC neither reaches real time — which is what `transcode_incompatible_video` exists to refuse. This adds the mechanism that makes refusing it unnecessary. Node — `hwaccel.py`: VA-API on Linux, Quick Sync or NVENC on Windows, established by encoding 1080p and reading the file back with ffprobe. Nothing is accepted that does not produce the exact profile and level `stream_init` announces, since the client checks that string before it trusts a byte: an encoder that wrote another level would make the node's own codec string a lie, and ffmpeg takes `-level 4.1` and `-level 41` from h264_qsv without saying which it understood. Three modes per stream — hardware decode and encode, hardware encode alone, libx264 — demoted per source codec, because a GPU that decodes HEVC may have no decoder for MPEG-4 Part 2 and only asking it finds out. A mode that fails is detected on an empty stdout before `stream_init` goes out, so the viewer sees one working stream and never an error. Client — Chromium ships VA-API off on Linux. It is enabled where a render node and a driver are present, then verified through `navigator.mediaCapabilities`: a no moves to the next GL backend on the next launch and an exhausted list drops the switches, so a renamed feature cannot pass for a feature that is on and `--ignore-gpu-blocklist` cannot survive on a machine it did not help. Feature lists now merge rather than overwrite — `appendSwitch` replaces the value, and a second caller would have silently cancelled the mDNS switch aiortc depends on. Packaging — the drivers are weak dependencies on all four formats, so a machine without a GPU installs exactly as before. `dpkg -i` and `rpm -ivh` ignore weak deps; `packaging/README.md` now says so. Windows needs no driver: the bundled ffmpeg already carries h264_qsv and h264_nvenc, and a re-pin that dropped them would cost every low-power Windows node its hardware encoding silently. AMD on Windows (AMF) and macOS (VideoToolbox) are named gaps, not oversights: neither could be tried anywhere in this project, and both re-encode in software as before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(node): size the subtitle budget to the file, and extract each track onceChristophe Besson8 days2-41/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Verifying the feature across a library turned up two faults, one of which broke it outright. **The extraction budget was a constant and the cost is not.** Extracting demuxes the whole container, so the file sets the price: measured at 9.8 s per GB on a library held on an external disk — 36 s for a 3.9 GB title, 71 s for a 7.3 GB one. Against a flat 60 s that worked on most of a library and failed on the big films, which to whoever is watching one is indistinguishable from a broken feature. `Avatar.The.Way.of.Water` timed out every time. The budget is now three times the measured rate per gigabyte, floored at the old 60 s and capped so no container can pin a transcode slot indefinitely. The client's own timeout follows it: the node always answers, so that one is a backstop against a silent peer rather than a deadline for the work. **Two clicks ran two extractions.** The cache is consulted on the way in, so a second request arriving while the first was still running missed it and ran its own — seen in the log as two identical extractions of one 4.3 GB file overlapping, each holding a transcode slot and reading the file end to end. Latecomers now wait on the answer the first is producing. The in-flight entry is registered *before* the first await, not after. The first version registered it after the probe, two concurrent requests both got past the check while neither had registered, and the test for it failed — which is the only reason this note can be written from the right side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
* fix(node): measure where a seek lands instead of predicting itChristophe Besson8 days2-86/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous fix read ffprobe's key frames and took the last one at or before the request. It was wrong twice, and a viewer felt the difference: subtitles went from 5 s early to 2–3 s late. Matroska's Cues index only some keyframes, so an index seek backs off to an indexed one that the frame list does not single out. And the landing point moves with **which streams are mapped**, because the container is positioned where every mapped stream has data — on the reported title, a seek to 4913.7 s landed at 4909.863 with video alone and at 4907.236 with the second audio track mapped beside it. The frame scan gave the first number; the stream delivered the second; the gap was 2.65 s, and the measured audio displacement in the served stream was 2.65 s. So the node asks ffmpeg instead: the same seek, the same mapping, one copied frame under `-copyts`, and the answer read back off the result. 0.06–0.07 s, cheaper than the scan it replaces. The `-ss` argument stays at the request, so the bytes served are exactly the ones served before — only the number naming them changes. The probe runs after the audio track is resolved, because it cannot be right before that is known. An answer after the request, or further before it than any real keyframe gap, is discarded in favour of the old label: a number wrong by seconds beats a fabricated one. Found by decoding the served stream and locating its first frame in the source, which put it at 4907.213 s against an announced 4909.863 s. The test does the same thing rather than comparing the announced number against a second reading of the same probe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc