| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
GroupOpsMixin in transport/webrtc/group_ops.py: member revocation and
unpinning, the group key rotation, apps and their directories with the
allow-list, and Search listing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
NodeOpsMixin in transport/webrtc/node_ops.py: status and settings, roster
and denylist, roots, hosted groups, reload, scan pacing and transfer
limits, with their class-level bounds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
Three headers stayed in webrtc_server when their sections moved; they now
sit above the same methods in blobs.py and admission.py. Two that were only
a title over what is now a whole module are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
HandshakeMixin in transport/webrtc/handshake.py: challenge, proof, channel
binding, and the sealed configuration a peer receives once admitted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
The peer-session ceiling no longer quotes the hub's old per-account number,
and the note on a single uploads/ directory goes: its constant left with
f3fb449, when an upload started landing in the folder it was sent to.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
AdmissionMixin in transport/webrtc/admission.py, with the invite handle
pattern and the join attempt bounds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
UploadMixin in transport/webrtc/upload_handlers.py, with the upload cap;
_upload_chunk moved unchanged, _append_chunk joins webrtc/disk.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
TransferMixin in transport/webrtc/transfer_handlers.py, with the sweep
interval.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
FilesMixin in transport/webrtc/files.py; the blocking disk helpers join
_locate in webrtc/disk.py, and the lease states go to webrtc/limits.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
ChatMixin in transport/webrtc/chat.py: sealed messages, history, epoch
keys, link previews with their cache and rate bounds, and the operator's
chat ops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
BlobsMixin in transport/webrtc/blobs.py, with the blob caps and the kind
pattern.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
StreamingMixin in transport/webrtc/apps/streaming.py: stream credit,
handover, the transcode slots and _stream_video_inner, moved unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
VideoMetaMixin in transport/webrtc/apps/video_meta.py: media and season
meta, TMDB search and its per-member bound, posters, and the operator's
TMDB ops.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
| |
An operator op that exists for one app lives with that app.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
MusicMixin in transport/webrtc/apps/music.py: tags and cover art, and the
audio transcode with the extension list that gates it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
SubtitlesMixin in transport/webrtc/apps/subtitles.py; _locate, which it
shares with the files, music and streaming handlers, in webrtc/disk.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
| |
transport/webrtc/channel.py, limits.py and media_tools.py, cut from
webrtc_server.py as text; the facade imports them back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
| |
connectionstatechange dropped a closed or failed session from the
transport and stopped its tasks, but only close() took it out of the
group's peer set, and nothing on that path called it. Every broadcast
then went to closed channels, and each reconnect left a dead session
held until restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A phone on 4G found one of five groups missing from Search on half its
visits, and all but one on a fifth of them. Every offer that reached the node
was answered in about a second; the hub refused the others with 429 — 93 of
them in half an hour on meshbay.org, all from that phone, all to nodes that
were up. Two ceilings did it. Three pending offers per account was exactly
what one Search page dialled at once, so the first tile or reconnection beside
the sweep was refused. Thirty offers a minute per address and per node was
spent by four or five reloads of a page whose groups share one node.
Sized now against an account with twenty groups on three devices:
- 32 offers pending per account, across devices and nodes.
- A budget per account and per node, a burst of 120 refilled at two a
second. This is what bounds a member's cost to one machine (H6), and it
leaves their other nodes alone. Counted by account, because a mobile
carrier puts many subscribers behind one IPv4 address.
- 600 a minute per address and per node, as a coarse guard in front of
authentication only.
Both refusals carry Retry-After, which the browser now honours. The node's
own ceiling on peer sessions is unchanged; its comment no longer quotes the
old per-account number.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
| |
The plan was a working document; what it decided lives in
MESHBAY_DESIGN.md and MESHBAY_NODE_PROTOCOL.md. Code and tests now
cite those instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
| |
New invite kind "link": member of one group, once, never operator, not
spendable by an active member, capped at 20 per group, cancellable by
handle. Signed ops invite_link_create / invite_cancel, loopback routes,
and the known-device join path now accepts a link code. Adds the plan,
docs/invite-links.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
| |
node_pk in handshake_challenge is now signed over the channel binding
and both nonces, so a client can check the node key before a join
rather than only at the ack. Both transports; the browser and the QUIC
client refuse a wrong signature and treat an absent one as an older node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
| |
A browser limits bytes, so a bound in seconds had to be sized for the
highest-bitrate file and every ordinary one then held a fraction of what
the same buffer would have taken. Floored at the old 90 s so nothing
pulls less than before, and walked up rather than declared. A refused
append now waits for an eviction instead of retrying on every tick.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pass before this applied ruff's own fixes. These are the ones needing a
decision, and the point of doing them is that `ruff check .` now passes: a
linter reporting 98 known-acceptable findings reports nothing, because the next
real one arrives invisible.
**Lines over 100 (70).** Mostly wrapped where they stood. Two exceptions: the
aligned trailing comments in `protocol.py`'s message table were shortened rather
than wrapped, because wrapping one row of a table breaks the table; and in
`models.py` the column comments moved above their columns for the same reason.
**Imports below the first statement (14).** `csam.py` kept its FastAPI imports
under a section header halfway down the file; two node tests had a constant and
a `pytestmark` wedged between two import blocks. Moved, not suppressed.
**Bindings nothing reads (4).** Three in tests, where the call stays and only the
name goes — `_user(client, "listener")` is there to create the user, not to
return one. The fourth was in `revocation.py` and was not a lint finding at all:
`_connect_and_listen` opened an httpx stream to the WebSocket URL, did `pass`,
and then opened the real connection through the `websockets` library. One
pointless request per connect, left over from before that library was used
directly. Removed, and `httpx` with it.
**`l` as a name (4)**, **semicolons (6)** in the POC spikes, and the rest.
2893 passed, the same count as the two commits before it.
`meshbay_node/revocation.py` is worth a decision separately: 154 lines that
nothing imports, superseded by `hub_client.maintain_ws`'s `on_revocation`. This
commit only stopped it failing the linter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`ruff check .` had gone unrun long enough to report 568 errors, which is the
same as having no linter: the next real finding would have been invisible in the
noise. This is the 521 it fixes by itself, in 173 files, and nothing else — the
98 it cannot fix are the next commit.
What actually changed: import sorting (225), imports nobody used (87, none of
them a re-export — no `__init__.py` is touched, which was the one way this could
have broken an import elsewhere), `datetime.timezone.utc` to `datetime.UTC` (69)
and `asyncio.TimeoutError` to `TimeoutError` (18), both plain aliases on the 3.12
this project requires, `Optional[X]` to `X | None` (24), and f-strings with
nothing to interpolate (19).
Checked rather than assumed: every module in the three packages still imports,
and the suite is 2893 passed — the same count, test for test, as the merge
before it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
One conflict, in §15.3's open list, and it was two changes agreeing rather than
disagreeing: this side removed the rows for the third-party search bound and the
node-announcement bound because both are now built (AV27, AV28), while the other
side kept them and added a new one. Resolved by keeping what is genuinely still
open — per-device revocation having no CLI — and leaving the two closed.
`webrtc_server.py` merged without conflict but the two sides met inside one
function: `_upload_chunk` gained the operator's `max_upload_gb` ceiling from
there and the per-group lock and `off_disk` calls from here. Read back rather
than trusted: the operator's ceiling now sits inside the critical section that
keeps chunk ordering, and the unlink beside it goes to the disk thread with
everything else.
2893 passed. The twelve `test_sticky_header.py[firefox]` setup errors are the
open Firefox on this machine, as before.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Was a 4 GB constant in webrtc_server.py, the same on a small board and on
a machine holding a library. Now max_upload_gb in node.toml, on the Node
page and via `meshbay-node transfers max-size`, read from the transport
context per chunk so a change reaches an upload already running.
MESHBAY_DESIGN.md §6.4; §15.3 records a defect found beside it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
`BROWSER_INCOMPATIBLE_AUDIO_EXTS` named the two formats no browser decodes and
was read by nothing. The player asked for `.wma` and `.mpc` and no others, and
`music-player.js` called itself "kept in sync with the node's" constant — so the
entire restriction lived in the caller, and a member's own message is not the
caller.
What that let through: this converts a whole file and holds a transcode slot
shared with video streaming while it runs. One message naming a two-hour film
spends minutes of the operator's CPU and a slot every other viewer is queued
behind. `AUDIO_TRANSCODE_MAX_BYTES` catches the result, after the work is done;
only the extension catches the work.
The client comment now says what it is — an optimisation that saves asking for a
refusal — rather than implying it is the rule.
Two tests: a film is refused before ffmpeg is reached, and a WMA still passes,
because a gate that refuses everything is not a gate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
the loop
Two things about the same three functions, which write to a temp file with ffmpeg
and then read it back.
**The read was on the event loop.** These files are ffmpeg's own, under
`tempfile.mkstemp` on the system disk, so they are not a group root and there is
no spun-down platter to serialise against — which is why they go through
`asyncio.to_thread` and not `roots.off_disk`. But a whole transcode read inline is
still tens of megabytes of blocking read while nothing else in the node is served.
The AST guard now covers the module with no exemption at all, and a second check
refuses a direct call to the reading helper: passed to `to_thread` it appears in
the syntax tree as a name, called inline it appears as a call.
**The audio transcode had no size ceiling**, where the subtitle path beside it
has had one all along. The bound is the media cache's rather than memory's:
`put_thumb` writes one SQLite row, and the store is 512 MB with least-recently-used
eviction, sized for thumbnails, posters and short transcodes. At 192 kbit/s a
three-hour source is ~260 MB — one row that evicts most of the cache to fit and is
evicted again by the next few thumbnails. Not a size this store can hold usefully.
64 MiB, about forty-five minutes: past any track, any single piece, most sets.
It takes away nothing that worked. `AUDIO_TRANSCODE_TIMEOUT_SECS` is 120, so a
source long enough to reach this was already liable to be killed mid-transcode;
what changes is that the refusal now names the limit it met and the size that met
it. Serving audio of that length properly means streaming the conversion instead
of buffering it, which is a different feature — recorded in §9.8 rather than left
as an implied promise.
The stat comes before the read, so an oversized result costs a stat rather than
the read and the memory behind it.
Twelve `test_sticky_header.py[firefox]` setup errors again: Firefox is still open
on this machine, and its `[chrome]` half passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
**A free-text TMDB search spends the operator's credential.** TMDB rates it, and
the automatic matching every member sees runs on the same one, so a member
holding down the search box — or a script doing it — degrades the library for
everyone and empties a quota the operator pays for. The handler had no ceiling of
any kind, where link previews beside it carry two. §6.5's standing rule is a bound
and a named adversary in the same commit; this arrived with neither.
Per member and not per connection, unlike link previews: three tabs is one
person, and a ceiling a tab can multiply is not a ceiling. Kept in the group
context, so a reconnect does not reset it — a client that drops its channel
between searches would otherwise have no ceiling at all. The node-wide window
stays too, because the two answer different questions: one keeps a member from
spending everyone's quota, the other keeps a roomful of them from doing it
together. Ten a minute each, thirty for the node — a search every six seconds,
sustained, is past what anyone types.
The refusal is an error rather than an empty list. An empty list is what "no such
film" looks like, and telling somebody their film is unknown when the node simply
declined to ask is a worse answer than the truth; `video-app.js` already puts
`detail` on screen.
**How many node keys one account may announce.** Each is a row in `nodes` plus a
row in the IP log, and the IP log is kept for a year, so an account in a loop
writes a year of storage on the operator's disk having paid only for signatures.
M8 settled whose key it is and said nothing about how many. Ten: a node is a
machine left running, and an account wanting an eleventh *identity* rather than
an eleventh machine is the case this refuses.
Counted only where a row is added. Applied to every announce it would freeze the
address of every node an account already runs the moment it reached the limit,
and a node that cannot re-announce is unreachable after its ISP renumbers it —
an outage caused by the protection. There is a test for exactly that.
Both tests are two accounts, per §13.5b: a ceiling one person can exhaust for
another is not a ceiling but a queue, and a ceiling shared between accounts would
let one member stop every other from bringing a machine online. Checked by
removing each ceiling: seven tests fail.
`test_season_and_search_requests.py` built its session without a `_user_id`,
which production guarantees — `_dispatch_message` refuses every message until the
handshake settles it. The fixture was narrower than the node, so it could not
exercise a per-member bound at all; it has one now.
Twelve `test_sticky_header.py[firefox]` setup errors in a full run here: Firefox
is open on this machine, the trap CLAUDE.md describes, and its twelve `[chrome]`
tests covering the same geometry pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The per-feature design notes were merged into docs/MESHBAY_DESIGN.md and
deleted from the tree on 2026-09-11, but ~230 comments across the three
packages still named them — usually written `docs/musicbay.md §3.2`, as
though the file were still in docs/. A reader had to know §16 existed to
resolve any of them. They now name the section directly.
Every mapping comes from §16, the concordance, which already records where
each old section landed: musicbay -> §9.8, mediacenter -> §9.7 for the
Videos app and §6.5 where the subject is derived data, photos -> §9.9,
auth-confirm -> §3.6, refactoring-search -> §9.11, invite-pairing-v1 ->
§3.4, per-node-identity-v1 -> §3.2, captcha -> §7.7, chat-sender-keys ->
§4.5, apps/refactor-groups -> §9.1–§9.4, desktop-client-v1 per section.
Bare citations of the same documents (`draft-v6 §2.11`, `§4.8`, `§3.4`)
are retargeted too: those collide with real section numbers in the design
document, so leaving them would have been worse than the named form.
Four cases the concordance does not cover, each decided rather than guessed:
Sub-item references into documents that no longer exist — mediacenter's
`§3.3 row 4`, `§3.4b/c`, `apps.md §3b` — name rows and sub-items §9.7 and
§9.2 do not reproduce. The module-level citation stays; the sub-item
pointer is dropped.
The V-findings keep their labels but lose the dead `§10.1/` prefix.
§13.8 lists V1–V13 as per-application open items, which is not what the
labels mean in these comments, so pointing them at §13.8 would have been
a false citation.
`apps.md §5`'s virtualization requirement has no counterpart anywhere in
the design document. The requirement is stated in the comment itself, so
the citation is dropped rather than aimed at a section that does not say
it.
Comments that attributed a *sentence* to an old note — musicbay's "several
thousand files" example, its "what got measured" note, its measured
~11%/~26% cover-art figures, the "original no root, whole shared tree"
call — state the fact without attribution now. §9.8 does not contain those
sentences and citing it for them would have been wrong.
CLAUDE.md's "a reference to a document that no longer exists" row now says
the concordance is for git history and out-of-tree material; the code cites
sections directly.
Verified: 2851 passed, 4 skipped. The 12 errors in the run are the Firefox
leg of test_sticky_header.py's browser harness, which is broken at the
browser level on this machine — headless Firefox (snap) dies with
`[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer`, renders
nothing, and the probe exits `{"error": "no measurement"}` after its full
90s wait. Chrome runs the same 12 assertions in 3.2s and passes. Nothing
here can affect it: every changed line in style.css is inside a comment.
Also checked: ast.parse on every changed .py, `node --check` on every
changed .js, the /* */ balance in style.css, and that no changed line
exceeds the width its file already used.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The last two handlers on the loop. Both were synchronous, so a member creating a
folder on a root that had spun down held the node for the spin-up, exactly as a
chunk read did.
Where a check and an act belong together they are now one call rather than two
awaits, and the single disk thread is what makes that atomic: `_mkdir_if_absent`
so two members creating the same name cannot both find nothing there and have
the second `mkdir` raise where a refusal was meant, and `_rmdir_if_empty` for
the reason the caller already re-tested emptiness — the first test happened
before a round trip to the operator's browser, and a file can land in between.
Two awaits would reopen that window one size smaller.
The guard is now the whole class rather than the calls that were fixed. It walks
the module's syntax tree and fails on any filesystem call outside the handful of
functions written to be run through `off_disk` — a new handler that stats a root
inline would pass every measured test, because those exercise the handlers that
exist today. Checked by putting a call back: it names the function and the line.
It leaves ffmpeg's own scratch files out, listed rather than silently allowed:
they are under `tempfile.mkstemp` on the system disk, not on a group root, so
they are not what spins down — but they do read a whole transcode into memory
from the loop, and the day that matters it is a different measurement from this
one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The rest of AV9's disk half. Serving a file left the loop in the commit before
this one; two paths were still on it.
**The availability poll.** `RootSet.refresh_availability` stats every root, and
eleven call sites reached it from `async def` — the reconcile loop among them, on
a timer. On a sleeping disk that is a stall once per tick, and the stat is also
what keeps the disk awake, so a node paid spin-up for a library nobody was
reading. All eleven now go through `off_disk`, `Root.is_live` included.
**The upload write.** `open`/`write`, and the resolve, the stat, the free-name
search, the rename and the unlink around it. This one could not simply be
awaited: the handler was synchronous, so nothing could come between the
`chunk_index != state.next_index` check and the `advance` that answers it, and
that is the whole of the chunk-ordering rule. Awaiting the write opens the gap —
chunk 1 arriving while chunk 0 is in the disk thread reads a position that has
not moved and is refused as out of order, so an upload would fail on a slow disk
and nowhere else. Verified, not assumed: without the lock the new ordering test
refuses three chunks of four.
So the check, the write and the advance are one critical section again, under a
lock held **per group**. Not per session: `partial_uploads` lives in the group
context so a reconnecting client finds its upload where it left it, which means
two sessions of one member share the position of one `.part` file. Arrival order
is preserved by construction — the dispatcher creates one task per message as it
arrives, tasks start in creation order, and the lock is the first thing each one
waits on, so its waiters queue in arrival order too.
`_do_file_upload` is a coroutine now, which is why forty-two test call sites gain
an `await`. Their outcomes are unchanged, file by file, against the run before
the change.
`test_ops.py` asked which public coroutines `ops` exposes and got `off_disk`,
imported rather than defined there. It now asks for the ones written in the
module, which is what its own docstring means; all forty-three operations are
still checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A root that has spun down, or that lives on a network mount, answers its first
syscall in seconds rather than microseconds. Made from the event loop, that
stalls the whole node: no other group is served, no stream is fed, no chat
message is delivered and the hub socket is not read, for as long as the platter
takes to come back. It was found from the other end — a client's connection
attempt timing out on a node with one member, while the disk woke up — and it is
AV9's lesson with the disk in the place of the mail server.
Every filesystem call on a group's content now goes through `off_disk`, onto a
thread that belongs to that group's root set. The stat goes with the read: a stat
is what *wakes* a sleeping disk, so offloading only the read would move the stall
rather than remove it, and the read would then find the disk already awake.
`_locate` is the one place allowed to call `entry_abs_path`, which is
`Path.resolve()` and therefore syscalls too.
Five handlers touched: a chunk request, an audio transcode, a subtitle request, a
video stream and a delete. The delete became a coroutine, which its one caller
already was.
One worker per root set, not a pool and not one for the node. One worker for the
same reason the indexer's executor has one — two interleaved reads of a spinning
drive seek-thrash rather than go faster — and it keeps two threads from being
inside the same file at once, which is what makes the `seek`/`read` pair safe
without a lock. Per root set, because a node serves several groups and their
roots are not all on the same volume: a single worker would put one group's
sleeping USB drive in front of another's SSD, which is this symptom one level
down. The thread is created on the first read, so a group nobody downloads from
never starts one, and the daemon stops them all on the way out.
The tests measure rather than read: a ticker counts its own wake-ups beside a
request made slow on purpose, and a handler that blocks the loop takes every one
of them with it. Put either call back inline and both tests report zero wake-ups,
which was checked before they were trusted.
QUIC still reads on its loop. Its handler is synchronous by construction, no
client speaks it, and its distance from parity is already recorded in the design
document (§15.3, L3); moving it is part of bringing it to parity, not of this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A 1080p film is decoded by whoever watches it and re-encoded by the node
when no browser can decode the source. Both were on the CPU, and on an
Atom or Celeron mini-PC neither reaches real time — which is what
`transcode_incompatible_video` exists to refuse. This adds the mechanism
that makes refusing it unnecessary.
Node — `hwaccel.py`: VA-API on Linux, Quick Sync or NVENC on Windows,
established by encoding 1080p and reading the file back with ffprobe.
Nothing is accepted that does not produce the exact profile and level
`stream_init` announces, since the client checks that string before it
trusts a byte: an encoder that wrote another level would make the node's
own codec string a lie, and ffmpeg takes `-level 4.1` and `-level 41`
from h264_qsv without saying which it understood. Three modes per
stream — hardware decode and encode, hardware encode alone, libx264 —
demoted per source codec, because a GPU that decodes HEVC may have no
decoder for MPEG-4 Part 2 and only asking it finds out. A mode that
fails is detected on an empty stdout before `stream_init` goes out, so
the viewer sees one working stream and never an error.
Client — Chromium ships VA-API off on Linux. It is enabled where a
render node and a driver are present, then verified through
`navigator.mediaCapabilities`: a no moves to the next GL backend on the
next launch and an exhausted list drops the switches, so a renamed
feature cannot pass for a feature that is on and `--ignore-gpu-blocklist`
cannot survive on a machine it did not help. Feature lists now merge
rather than overwrite — `appendSwitch` replaces the value, and a second
caller would have silently cancelled the mDNS switch aiortc depends on.
Packaging — the drivers are weak dependencies on all four formats, so a
machine without a GPU installs exactly as before. `dpkg -i` and
`rpm -ivh` ignore weak deps; `packaging/README.md` now says so. Windows
needs no driver: the bundled ffmpeg already carries h264_qsv and
h264_nvenc, and a re-pin that dropped them would cost every low-power
Windows node its hardware encoding silently.
AMD on Windows (AMF) and macOS (VideoToolbox) are named gaps, not
oversights: neither could be tried anywhere in this project, and both
re-encode in software as before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Verifying the feature across a library turned up two faults, one of which
broke it outright.
**The extraction budget was a constant and the cost is not.** Extracting
demuxes the whole container, so the file sets the price: measured at 9.8 s per
GB on a library held on an external disk — 36 s for a 3.9 GB title, 71 s for a
7.3 GB one. Against a flat 60 s that worked on most of a library and failed on
the big films, which to whoever is watching one is indistinguishable from a
broken feature. `Avatar.The.Way.of.Water` timed out every time. The budget is
now three times the measured rate per gigabyte, floored at the old 60 s and
capped so no container can pin a transcode slot indefinitely. The client's
own timeout follows it: the node always answers, so that one is a backstop
against a silent peer rather than a deadline for the work.
**Two clicks ran two extractions.** The cache is consulted on the way in, so a
second request arriving while the first was still running missed it and ran
its own — seen in the log as two identical extractions of one 4.3 GB file
overlapping, each holding a transcode slot and reading the file end to end.
Latecomers now wait on the answer the first is producing.
The in-flight entry is registered *before* the first await, not after. The
first version registered it after the probe, two concurrent requests both got
past the check while neither had registered, and the test for it failed —
which is the only reason this note can be written from the right side.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The previous fix read ffprobe's key frames and took the last one at or before
the request. It was wrong twice, and a viewer felt the difference: subtitles
went from 5 s early to 2–3 s late.
Matroska's Cues index only some keyframes, so an index seek backs off to an
indexed one that the frame list does not single out. And the landing point
moves with **which streams are mapped**, because the container is positioned
where every mapped stream has data — on the reported title, a seek to 4913.7 s
landed at 4909.863 with video alone and at 4907.236 with the second audio
track mapped beside it. The frame scan gave the first number; the stream
delivered the second; the gap was 2.65 s, and the measured audio displacement
in the served stream was 2.65 s.
So the node asks ffmpeg instead: the same seek, the same mapping, one copied
frame under `-copyts`, and the answer read back off the result. 0.06–0.07 s,
cheaper than the scan it replaces. The `-ss` argument stays at the request, so
the bytes served are exactly the ones served before — only the number naming
them changes. The probe runs after the audio track is resolved, because it
cannot be right before that is known.
An answer after the request, or further before it than any real keyframe gap,
is discarded in favour of the old label: a number wrong by seconds beats a
fabricated one.
Found by decoding the served stream and locating its first frame in the
source, which put it at 4907.213 s against an announced 4909.863 s. The test
does the same thing rather than comparing the announced number against a
second reading of the same probe.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A cached track replied without leaving a line, so the journal showed nothing
for a viewer who had asked for subtitles and been served instantly. That made
"no request in the log" look like evidence nobody had asked — which it was not
entitled to be, and which cost a wrong conclusion about whether a fix had been
exercised at all.
The request is logged on arrival now, before the cache is consulted, and a
cache hit says so with the size it served. Second time in this feature that a
silent success was read as an absence; the first was an attach-time state that
was correct while the screen stayed empty.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Copied video can only start on a keyframe, so `-ss t` with `-c:v copy`
delivers the keyframe at or before `t`. The node reported `t` anyway and the
client builds `SourceBuffer.timestampOffset` out of that number, so everything
downstream believed the picture stood a few seconds further along than it did.
That was a wrong label while only the scrubber read it — it is recorded as
such, low priority, in the design document. Subtitles made it a wrong answer:
their cues carry the source's own absolute timestamps, so the mismatch put
every line on screen before it was spoken. Reported from real use on an H264
title, where seeking to 600 s, 2650 s and 5000 s lands on keyframes 0.82 s,
1.56 s and 4.64 s earlier.
The copy path now resolves the request to that keyframe, seeks to it, and
reports it. The bytes delivered do not change — ffmpeg lands on the same frame
either way — only the number that names them. The look-up reads the thirty
seconds before the request and cost 0.12–0.51 s on that title, which is the
price already costed in §15.3 and never paid. Re-encoded video is untouched:
it can begin exactly where it is asked to, and does.
Two details worth their lines. The keyframe is passed back to ffmpeg at six
decimals, because rounding a keyframe's own timestamp down puts it before the
frame it names and selects the previous one — the same fault again, smaller.
And the ffprobe window ends past the request, since an interval whose end is
the request never emits a keyframe sitting exactly on it: the resume position
is the one place a viewer asks for the same instant twice, and it would have
been answered a whole GOP early.
The test decodes the first frame served and matches it against the source
frame at the position announced, rather than comparing `start` to an expected
number — both sides of that comparison would be reading the same ffprobe and
would agree by construction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reported as "I click a subtitle and nothing appears", on three films. Nothing
was broken. The track selected was the container's forced track, which carries
signage and foreign dialogue only: measured on the film in question, 30 cues
and 77 seconds of text across 2h32 — 0.8% of the running time, against 1559
cues and 41.8% for the full track sitting beside it under the same language
tag. At all three positions tested there was genuinely no cue to show; the
full track would have shown one at two of them.
So the defect is that the menu could not say which was which. The label used
the container's title tag, which said "Forced" on that film and says nothing
at all on most, and no other field was carried. The disposition is the half
that is always there: `probe_video` now reads `forced` and `hearing_impaired`,
`stream_init` carries them, and the label states them in the reader's own
language rather than repeating an English word a muxer happened to type.
The node fixture grows a forced track with no title, because a title would
let the old code pass. The label harness's `t` stub took a parameters object
unconditionally and threw on a key that has none — a fixture narrower than
production, fixed here rather than worked around.
Also removes the activeCues probe that found this. It answered its question:
mode showing, cues 30, active 0, none due at that instant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two faults, both found while a hang could not be diagnosed from the node's
own journal.
`except Exception` does not catch `CancelledError`, which derives from
BaseException. A cancelled extraction therefore sent no reply and logged
nothing at all, leaving the client waiting on something nothing would ever
answer — the one shape that cannot be reported by whoever hits it. It now
answers, logs, and re-raises so the cancellation still propagates.
And the handler logged nothing on any path, so the journal could not even say
whether a request had arrived. It now names the file, the track, the file size
and the free slot count on entry, the moment the transcode slot is taken, and
the duration and byte count on the way out — the size and duration because the
extraction is a whole-file demux and its cost is set by the file, not by the
subtitle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
MSE decodes no in-band text track, so a subtitle cannot ride inside the
fragmented MP4 the player is fed. The node extracts one track whole, converts
it to WebVTT and caches it under its own hash; the client pulls that blob
through the ordinary file_req/chunk path and hangs a <track> on the video
element — the same indirection as a TMDB poster or an audio transcode, which
is what makes a film's subtitles extracted once in the life of the file rather
than once per viewing. Whole-file also makes the cues absolute, so a seek and
an audio-language change both leave the track untouched.
**The ordinal counts every subtitle stream, including the ones never listed.**
Only text codecs are offered: a bitmap track (PGS, VOBSUB — about a fifth of a
real library) has no path to WebVTT without OCR, and one extracted anyway
yields a header with no cues, which is a menu entry that shows nothing and
reports no error. Numbering the survivors of that filter would give a
PGS/SRT/SRT file the ordinals 0 and 1 for its text tracks and `-map 0:s:0`
would then extract the PGS — the same trap `AudioTrack.ordinal` exists for,
one level deeper. A fixture whose first subtitle stream cannot be decoded pins
it, and the handler checks membership of the probed list, never a range.
Additive and MINOR: the selector is drawn from `subtitle_tracks` in the node's
own `stream_init` and from no version number, so `subtitle_req` is never sent
to a peer that would not answer it. The floor stays at 3.0.
Also here: a failed extraction never touches playback, a superseded reply
cannot install its blob over a newer choice, and `_languageName` is shared
with the audio labels — lifted by both label harnesses, since a lift that
names one function stops covering the rule the moment logic moves out of it.
Tests: 9 node (tracks told apart by the words in the extracted cues, not by
tags), 10 client. Full suite green: 1545 node/common, 1252 hub.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-ss before -i cannot trim copied video, which must begin on a keyframe, but
accurate_seek did trim the re-encoded audio to the exact request. Every seek
on a copied stream therefore opened with a GOP-wide hole in the audio and ran
a GOP out of sync afterwards — 9.979s on a real film with a 10s keyframe
interval. Accurate seeking is now off wherever video is copied, and stays on
where it is re-encoded, which is the only path that could already begin where
it was asked to.
Every timestamp was correct throughout, which is why nothing caught it; the
tests assert on decoded audio and on frames compared against the source.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The streaming path mapped 0:a:0 unconditionally, so a dubbed film played in
whichever language was muxed first and the others were unreachable. The node
now enumerates the tracks in stream_init and honours audio_track in
stream_req; switching is the seek path, since one ffmpeg carries one track.
MNP 3.2, additive: the player draws its selector from the node's own list and
never from a version number, so an older node is never asked for a track it
would ignore. MNP_MIN_SUPPORTED does not move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reported from a phone: signing in with the same account showed no
playlists. syncWith was called from exactly one place in the interface,
so creating a playlist, deleting one, removing a track and saving the
queue all wrote to IndexedDB and stopped there. The store pushes itself
now, coalesced, so a new mutation cannot forget to.
Silence was the real defect. The node audited only successes, so a
refusal left no trace and user_blob_list none at all; the background
push swallowed its reason; the interface said nothing. All three report
now, and "Sync now" says what happened either way.
An unreadable blob on a node was treated as a fetch failure and returned
before the push — permanent, once the node held anything. It is an
absence: the client is the authority, and it gets overwritten.
A sign-in reconciles whatever this browser already holds, a pending push
is flushed when the page goes away, and a push that did not land is
retried once.
no_key is spelled out: a client that signs in with its remembered device
key only ever has a bundle key persisted before the playlist subkey
existed, and an AES handle is non-extractable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
One row per playlist plus a manifest, so starring a track rewrites that
playlist rather than the whole collection. blob_enc is a BLOB, not
base64 TEXT: these run to hundreds of kilobytes.
user_id comes from the session and never from the message; kind is
validated against a pattern; every cap refuses with a stated reason
rather than truncating.
Additive, so MNP_MIN_SUPPORTED does not move — a 3.0 node answers
"unknown message type" and the client writes to the next one it reaches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`search_listed` is a per-group setting on the node, changed by a signed
operator op and carried in the sealed handshake ack. Search reads it after
the handshake and stops there: no index is fetched, cached or merged, in any
of the four views, and the page says how many groups it left out. The switch
is a "Search" section in the group's settings, shown to the operator.
Absent means listed, at every layer: roster default, ack default, and the
client only drops a group on an explicit `false` — so an upgrade or an older
node removes nothing from anyone's Search.
It is a listing preference and protects nothing: the node serves the same
index to Search and to the group page and cannot tell them apart, every
member lists the group by opening it, and a client that ignores the flag
lists it in Search too. Design §9.11 says so, so it is never described as
private. The cost is one handshake per unlisted group, because only the node
knows the setting.
Search also took `nodes[0]` twice — for the index and for the pooled
connection — the defect 4cce50f fixed on the group page only. One
`connectToGroup` now walks the list the same way: a refusal about this
browser stops, `not_hosted` or a failed connection moves on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XuNrwLf5EFWCMHzfoEvnpm
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
`IndexProgress` said "scanning, this many bytes of that many" and nothing
more. A group's roots are walked one after another, so a second directory
added during a large scan showed as the bar jumping back to 0 %. It now also
carries the root being walked and its position in the roots table, the kind
of walk (scan, rescan, reconcile, watch), file counts, and the roots
waiting for the scan lock in order: queued by the initial scan, by a
retarget, and by a plug; dropped when a root is removed.
`GET /api/index-status` answers for every group at once, including a group
still in its initial scan, so a client can show indexing on any page. It
names roots: loopback only, like `current_dir`.
`index_progress` and the handshake ack gain the same counters, still naming
nothing (decision D3): the root is a position in the roots table the member
already opened from the sealed index, and the queue is a count. The pusher
keeps speaking while a root only waits for the lock.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding a large directory to a running group made the reload await the
scan of the new root before putting the new RootSet in the group's
context, holding _reload_lock the whole time. For the hours a large drive
takes to hash, the node served the old set:
- a file request under the new root got None from entry_abs_path and the
handler died on None.exists() without replying;
- a writable/removable toggle answered with the live table, still the
old one, so the directory vanished from the operator's settings;
- reconcile saw every file the scan had not reached as a missed event and
hashed it again on the same executor, rewriting progress under the scan.
retarget now applies the set, the roots table and the watcher first, and
with wait=False scans the added roots in the background; the daemon swaps
ctx["roots"] before calling it. A scan lock shared by the initial scan,
added-root scans and reconcile makes the reconcile loop sit out a running
scan without backing off. Every transport site that resolves an entry
answers ROOT_NOT_SERVED instead of crashing, and a delete keeps the entry.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T6jPTeocXA1BePekdsgPya
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A chat message is the plainest member-supplied write there is: the node stores
it in `chat.db`, where nothing expires it — retention is a manual command
(§6.6) — relays it to every other connected member, and has the hub write a
notification for every member of the group. Nothing bounded any of it. The only
ceiling was the DataChannel frame, 64 MB once the handshake is done, so one
member in a loop filled the operator's disk and saturated everyone else's
connection, and the node's answer to each message was `ack`.
Uploads, the other member-supplied write, have carried a filename allowlist,
strict chunk ordering, a no-overwrite rule and a 4 GB cap since C5a — because
somebody asked what one member costs the others on that path. Nobody had asked
it on this one.
Two bounds, for the two halves of the question: **64 KB of ciphertext** for what
one message may cost, and **60 a minute per account per group** for how often
one member may impose it. Both are checked before anything is stored or relayed;
a refusal names itself and is audited, so "why is my disk full" has an answer.
The rate is keyed by account, not by connection: a second tab does not make
anyone type faster, and keying on the session would hand a script one budget per
socket it opens.
No node-wide ceiling beside it, deliberately. The link-preview limiter has one
because a preview spends the *node's* egress and its third-party quota, which is
one shared thing; a chat message spends the sender's own group, and a node-wide
ceiling would let a busy group silence a quiet one — this same defect one level
up. The last test in the new file is that property: a member at their limit has
not spent anybody else's.
Two things stay open on purpose and are named rather than quietly done:
retention still keeps everything, because a default that deletes people's
history is not a review's call; and the composer still offers to send an
oversized message, so this is §6.4's pattern with only the node half built.
§6.6 gains the rule, §13.5b the label — AV20, with AV21–AV23 registering the
three fixes this week that closed the same kind of gap elsewhere.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UMxEQadpzPkYLFf5CYKhpW
|