diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-09-21 00:21:50 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-09-21 00:21:50 +0200 |
| commit | f7d33c299ae0d0c4f406779b8f5324d80637adc8 (patch) | |
| tree | 92102776658fce04451cff90dd616cb4ee26030f | |
| parent | 375ad7d0435a176ad593a32045a5f0182a36d505 (diff) | |
| download | meshbay-f7d33c299ae0d0c4f406779b8f5324d80637adc8.tar.gz | |
perf(hub): bound video read-ahead by bytes, not by seconds
A browser limits bytes, so a bound in seconds had to be sized for the
highest-bitrate file and every ordinary one then held a fraction of what
the same buffer would have taken. Floored at the old 90 s so nothing
pulls less than before, and walked up rather than declared. A refused
append now waits for an eviction instead of retrying on every tick.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| -rw-r--r-- | docs/MESHBAY_DESIGN.md | 36 | ||||
| -rw-r--r-- | packages/meshbay-hub/src/meshbay_hub/static/video-player.js | 153 | ||||
| -rw-r--r-- | packages/meshbay-hub/tests/harness/mse_harness.mjs | 56 | ||||
| -rw-r--r-- | packages/meshbay-hub/tests/test_video_buffer_ceiling.py | 231 | ||||
| -rw-r--r-- | packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py | 12 |
5 files changed, 460 insertions, 28 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 555d0f1..892b747 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -2099,6 +2099,39 @@ MSE string) fed to a source buffer, with the node holding one slot per viewer. driven by a clock and by playback and **never by arriving data**. Granting a credit per append pulls at network speed, fills the buffer ceiling, and then goes silent for the length of the accumulated balance. +- **The read-ahead bound is a byte budget, stated in seconds.** What a browser + limits is bytes, so a bound written in seconds has to be sized for the + highest-bitrate file in the library and every ordinary file then holds a + fraction of what the same browser would have taken — measured against a + 100 MB ceiling, a 1.0 Mbit/s film reached its ninety seconds on 20.7 MB and a + 2.3 Mbit/s one on 45.6 MB. On a mobile network that unspent ceiling is the + whole of the margin there is for a dead spot. The budget is therefore in + bytes, converted with the file's own average bitrate, and the fixed bound is + its **floor and never its ceiling**: nothing pulls less than it used to. +- **A ceiling that varies by engine and device is walked up to, not declared.** + The two ways to find a limit nobody publishes are to overshoot it or to + approach it, and overshooting costs a refused append every time it is tried: + a flat budget against a smaller ceiling measured 1386 refusals on a film that + had none. So the budget never exceeds one step past the most read-ahead that + SourceBuffer has actually accepted, and a refusal takes the ceiling down + below where it broke and leaves it there — backing off and climbing again is + a sawtooth with a refused append at every peak. +- **The budget only grows while the film is playing.** What it buys is a film + that survives a dead spot, and a film nobody has started cannot be + interrupted. Left to grow on the loading screen it filled a small engine's + whole buffer — 40 MB where the fixed bound took 8 — with film on a connection + that is paying for it, and reached a ceiling the fixed bound never came near. + Autoplay is blocked on a phone more often than not, so that is the ordinary + first state of every playback. Pausing does not shrink it: the lead already + paid for is the viewer's. +- **An append refused for quota is not retried until something frees room.** It + will be refused again for as long as the answer has not changed, and the + pipeline's clock calls the append path several times a second: the retry + alone cost minutes of refusals while the film played far enough for eviction + to have anything to drop. Eviction is the only thing that gives a + SourceBuffer room back, so its success is what lifts the hold — not a timer, + and not the playhead, which moves long before the answer changes. This cannot + deadlock: a full buffer is one holding film the playhead has not reached. - **Credit follows the buffer, decided in one place.** The append path grants nothing, because the buffer's update event fires for evictions too — crediting from it pays the node for the player's own housekeeping. @@ -3136,6 +3169,9 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | The exact-hash content check | Structural, not functional (§7.5) | | **QUIC** | Off by default, and **not at parity**: it serves the index and file chunks with no transfer lease, no leaseless ceiling and no root-availability check, does its file I/O on the event loop, and returns exception text to the peer (**L3**). No client speaks it. Either it comes to parity or it goes; until then §5.1's "chat is the only gap" is the one sentence here that overstates the code | | **The relay registry** | **Closed in the code**: `relay.RELAYS_ENABLED` is False and every `/v1/relays` route answers 503, as federation does. Nothing in the tree calls them, node or client, and §11.1 measured two ISPs with no TURN relay needed. Kept code that nothing calls is what **L7** says not to keep; it stays only as the proof-of-possession design (**AV6**) until a node needs a relay or it is deleted | +| **A very high bitrate wedges the player against a small buffer ceiling** | Where even the *floor* read-ahead does not fit — ninety seconds plus the minute kept behind, at the file's bitrate, above what the engine will hold — the film stalls: measured on the harness at 9.3 Mbit/s against a 100 MB ceiling, 100.8 s of film played in 900 s of wall clock. **Predates the byte budget and is unchanged by it**, to the tenth of a second; what the budget did change there is the refusal count, 1560 → 2. The fix is not a bound at all, it is a second stage of buffer outside the SourceBuffer, which means gating the append path — the riskiest change in this area and not one to make alongside another | +| **A reconnection throws the read-ahead away** | `addReconnectListener` resumes with a seek to the playhead, and `reinitAt` empties the SourceBuffer, so a transport drop costs every second of buffer that was the point of building it. Harmless while the read-ahead was ninety seconds and the buffer was empty by the time the link returned; with a budget of minutes it is the thing that stops the budget paying. Resuming at the *end* of what is buffered instead is the fix, and it touches the seek path, which is why it is here and not in the same commit | +| **The reconnect backoff only wakes on `visibilitychange`** | So a tab that stays visible through an outage — which is what a screen wake lock guarantees while a film is playing — waits out the full backoff, up to 30 s, after the network is already back. Nothing listens for `online` | | **Per-device revocation has no CLI** | A device is revoked over MNP (`roster.revoke_device`), from a device the node has already pinned. On a headless node the operator's only lever is `member unpin`, which removes **every** device of that account — so the per-device control the roster is built around is reachable from an interface and from nowhere else. §6.7 listed a `meshbay-node member device list\|revoke` verb that was never written, and that listing is how this was found: `USERGUIDE.md` was the first document written by reading the CLI rather than this specification, and the verb it copied out did not run | | **Migrations run on SQLite only** | The chain reaches head and agrees with the models there (§12), which is not where it ships. **The exposure is one revision deep, not the whole chain**: every revision behind the first packaged release was development that no installation ever ran, so nothing replays them on PostgreSQL. What is unguarded is the *next* migration — a default, an index type or a constraint PostgreSQL refuses reaches a deploy without the suite saying so | | **A node key is read from the terminal or the desktop client, never a browser** | **Accepted.** `meshbay-node status` on the node's own machine and Node → Overview in the desktop client are the two places the key can be read; the Node page is Electron-only, because `platform.node` resolves to "not available" without the bridge, and no hub route exposes the key. The create-group wizard links it automatically over that same bridge, so the manual paste in **Profile → Link Node** exists for the operator who runs the node from a terminal and the hub from a browser — who has a terminal by definition. Anyone linking a node is already at a shell prompt, so a browser-reachable copy would buy nothing and widen what the hub knows about the node | diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js index 66099de..0fed88c 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js @@ -10,13 +10,44 @@ import * as platform from './platform.js'; // Seconds of already-watched video kept in the SourceBuffer, and the queue depth // past which we start making room before being forced to. const BUFFER_BEHIND_S = 60; -// How far past the playhead we are willing to pull. The browser caps a video -// SourceBuffer at a few hundred megabytes and refuses the append that goes -// past, so "as fast as the network allows" is not a strategy for a film: the -// node remuxes with `-c copy`, so a 500 MB file puts 500 MB on the wire, and a -// ten-megabit second fills the ceiling in the first minute. Buffering by time -// rather than by bytes keeps a two-hour film and a two-minute clip alike. +// How far past the playhead we are willing to pull, at the least. The browser +// caps a video SourceBuffer at a few hundred megabytes and refuses the append +// that goes past, so "as fast as the network allows" is not a strategy for a +// film: the node remuxes with `-c copy`, so a 500 MB file puts 500 MB on the +// wire, and a ten-megabit second fills the ceiling in the first minute. +// Buffering by time rather than by bytes keeps a two-hour film and a +// two-minute clip alike. const BUFFER_AHEAD_S = 90; +// …but a bound written in seconds has to be sized for the highest-bitrate file +// in the library, and every ordinary file then holds a fraction of what the +// same browser would have accepted. Measured on harness/mse_harness.mjs +// against a 100 MB ceiling: a 1.0 Mbit/s film reached its ninety seconds on +// 20.7 MB, and a 2.3 Mbit/s one on 45.6 MB — four fifths and a half of that +// ceiling left unused, which is the whole of the margin a phone on a mobile +// network has to ride out a dead spot on. +// +// The quantity the browser limits is BYTES, so that is what is budgeted here +// and converted to seconds with the file's own average bitrate. It can only +// ever RAISE the bound: `BUFFER_AHEAD_S` stays as the floor, so a file whose +// bitrate spends the budget in under ninety seconds keeps exactly the bound it +// had before and nothing that worked pulls less than it used to. +// +// **The budget is discovered, not declared.** The real ceiling is per-engine +// and per-device and cannot be known up front, and there are only two ways to +// find it: overshoot it, or walk up to it. Overshooting costs a refused append +// every time it is attempted — a flat 48 MB budget against a 40 MB ceiling was +// measured here at 1386 refusals on a film that had none before it, which is a +// regression however well the player recovers from each one. So the budget +// starts at zero, which `aheadLimit()` reads as the fixed bound that shipped +// before, and thereafter is never more than one step past the most read-ahead +// this SourceBuffer has actually been seen to accept. +const BUFFER_AHEAD_STEP_BYTES = 4 * 1024 * 1024; +// Where the walk stops even if nothing ever refuses. Memory a phone has to +// find, and past this the gain is a read-ahead nobody needs. +const BUFFER_AHEAD_MAX_BYTES = 48 * 1024 * 1024; +// A sanity bound on what the budget may convert to. A file whose size or +// duration is wrong must not be able to ask for the whole film. +const BUFFER_AHEAD_MAX_S = 600; // While we deliberately hold credit back, the node must still hear from us: its // own stall timeout is two minutes, and a paused film is not a gone viewer. const CREDIT_KEEPALIVE_MS = 20000; @@ -285,6 +316,17 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { const appendingRef = useRef(false); const endedRef = useRef(false); const durationRef = useRef(0); + // Bytes of film per second of film, the byte budget the read-ahead is + // currently allowed, and the ceiling that budget may grow to. Zero bitrate + // means "not known yet" — before `stream_init` there is no duration — and a + // zero budget means "nothing learnt yet"; `aheadLimit()` reads either as the + // fixed `BUFFER_AHEAD_S` bound, which is what the player always had. + const bitrateRef = useRef(0); + const aheadBytesRef = useRef(0); + const aheadCapRef = useRef(BUFFER_AHEAD_MAX_BYTES); + // True while an append is known to be refusable and nothing has freed room + // since. See `flushQueue`. + const quotaHoldRef = useRef(false); // Segments the node is allowed to have in flight but has not sent yet, and // when we last said anything to it at all. const outstandingRef = useRef(0); @@ -414,6 +456,9 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { const start = range ? range[0] : sb.buffered.start(0); if (keepFrom - start < 10) return false; sb.remove(start, keepFrom); + // Room is about to appear, which is the one event that makes an append + // refused for quota worth trying again — see `flushQueue`. + quotaHoldRef.current = false; return true; } catch { return false; @@ -428,6 +473,21 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { return Math.max(0, range[1] - v.currentTime); }, [currentRange]); + /** + * Seconds past the playhead this film may be pulled to, right now. + * + * The byte budget is what the browser actually limits; the seconds are what + * the player can compare `bufferedAhead()` against. `BUFFER_AHEAD_S` is the + * floor and never the ceiling — see the constants — so this only ever gives + * a file more than the fixed bound did, never less. + */ + const aheadLimit = useCallback(() => { + const rate = bitrateRef.current; + if (!(rate > 0)) return BUFFER_AHEAD_S; + return Math.max(BUFFER_AHEAD_S, + Math.min(BUFFER_AHEAD_MAX_S, aheadBytesRef.current / rate)); + }, []); + const flushQueue = useCallback(() => { const sb = sbRef.current; if (!sb || appendingRef.current || sb.updating) return; @@ -437,6 +497,17 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { } return; } + // An append refused for quota that could evict nothing will be refused + // again until something frees room, and pump() calls this on a clock: the + // retry cost a refused append several times a second for as long as it + // took the film to play far enough for eviction to have anything to drop — + // minutes of it, measured on the harness. `evictBehind` is the only thing + // that ever gives a SourceBuffer room back, so its success is what lifts + // this rather than a timer or the playhead, which moves long before the + // answer changes. It cannot deadlock: a buffer is full of film the + // playhead has not reached, so playback always advances into it and + // eviction always comes. + if (quotaHoldRef.current) return; appendingRef.current = true; const chunk = queueRef.current[0]; try { @@ -446,10 +517,21 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { appendingRef.current = false; if (e.name === 'QuotaExceededError') { quotaRef.current += 1; + // This refusal is the browser stating where its own ceiling is — the + // one number the budget could not be given up front. Take the ceiling + // down below where it broke and leave it there, rather than backing + // off and letting the walk in `pump()` climb to the same place again: + // that sawtooth is a refused append every time round. `aheadLimit()` + // floors at `BUFFER_AHEAD_S`, so the worst this converges to is the + // fixed bound that shipped before. + aheadCapRef.current = Math.max( + 0, aheadBytesRef.current - BUFFER_AHEAD_STEP_BYTES); + aheadBytesRef.current = aheadCapRef.current; // The segment stays at the head of the queue and is tried again once // there is room. Dropping it — which is what this used to do — leaves a // hole in the middle of the film and no error anywhere. if (!evictBehind()) { + quotaHoldRef.current = true; console.warn('[MSE] buffer full and nothing to evict yet'); } return; @@ -494,7 +576,32 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { if (endedRef.current && queueRef.current.length === 0) return; if (!transport || !transport.connected) return; - if (bufferedAhead() > BUFFER_AHEAD_S + // Walk the budget up to one step past the most read-ahead this + // SourceBuffer has been seen to hold without refusing an append — never + // further, so an engine with a small ceiling is overshot by one step + // instead of by the whole budget, and only once, because the refusal takes + // `aheadCapRef` down below where it broke. Read-ahead rather than total + // occupancy, because read-ahead is what this bound governs and what + // eviction leaves alone. + // + // Only while the film is actually playing. What the budget buys is a film + // that survives a dead spot, and a film nobody has started cannot be + // interrupted: growing it there filled a small browser's whole buffer for + // a film left on the loading screen — 40 MB and nine minutes of read-ahead + // where the fixed bound took 8 MB — on a mobile connection paying for it. + // Autoplay is blocked on a phone more often than not, so that is the + // ordinary case and not a corner. Pausing does not *shrink* the budget: + // the lead already paid for is the viewer's. + const vp = videoRef.current; + const rate = (vp && !vp.paused) ? bitrateRef.current : 0; + if (rate > 0) { + aheadBytesRef.current = Math.min( + aheadCapRef.current, + Math.max(aheadBytesRef.current, + bufferedAhead() * rate + BUFFER_AHEAD_STEP_BYTES)); + } + + if (bufferedAhead() > aheadLimit() || queueRef.current.length > QUEUE_HIGH_WATER) { // Far enough ahead. Grant nothing, but do not go silent: two minutes of // silence is how the node decides nobody is watching, and pausing a film @@ -517,7 +624,7 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { lastPokeRef.current = Date.now(); transport.grantStreamCredit(room); } - }, [evictBehind, flushQueue, bufferedAhead]); + }, [evictBehind, flushQueue, bufferedAhead, aheadLimit]); useEffect(() => { let cancelled = false; @@ -538,6 +645,15 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { lastPokeRef.current = Date.now(); quotaRef.current = 0; stalledRef.current = false; + // The same rule: a new stream starts from a known state. Both are per-film + // — the bitrate obviously, and the budget because what a browser refused + // for a nine-megabit film says nothing about the next one. A budget left + // at the floor by the film before would silently cap every film after it + // for the life of the page. + bitrateRef.current = 0; + aheadBytesRef.current = 0; + aheadCapRef.current = BUFFER_AHEAD_MAX_BYTES; + quotaHoldRef.current = false; // The same shape again, and the seek refs are worse than the others. // Switching film while a seek was in flight leaves `awaitingInit` true, // and only reinitAt() ever lowers it — which the next film does not go @@ -683,6 +799,12 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { appendingRef.current = false; endedRef.current = false; quotaRef.current = 0; + // The buffer this was holding against has just been emptied, so there is + // room again — and a seek *backwards* would otherwise put the playhead + // below the held position and stop every append for the rest of the + // film. The budget is deliberately kept: what this browser refused is + // still true of it after a seek. + quotaHoldRef.current = false; awaitingInitRef.current = false; seekTargetRef.current = start; console.log('[seek] reinitAt done, start:', start, 'outstanding:', outstandingRef.current, 'queue:', queueRef.current.length); @@ -772,6 +894,15 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { } durationRef.current = msg.duration || 0; + // What the read-ahead budget is spent at. The file's own average, not + // the stream's: on the copy path they are the same thing, and where + // the node re-encodes it sends fewer bytes per second than the source + // holds — so this errs towards buffering less than the budget allows, + // never more. A film whose size or duration is missing keeps the fixed + // bound, which is what `aheadLimit()` does with a zero here. + bitrateRef.current = (entry.size > 0 && durationRef.current > 0) + ? entry.size / durationRef.current + : 0; // Recorded before either branch below: both restart the relay, and the // subtitle sent with it has to be shifted by *this* start, not the one // the previous stream had. @@ -1071,6 +1202,12 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { t.sendStreamDiag({ t: +v.currentTime.toFixed(1), ahead: +bufferedAhead().toFixed(1), + // What the read-ahead is allowed to be for this film, and the budget + // it came from. `ahead` on its own cannot say whether a short buffer + // is the gate holding or the network not keeping up, and the node's + // log is the only window there is into a phone. + limit: +aheadLimit().toFixed(0), + budgetMB: +(aheadBytesRef.current / 1048576).toFixed(1), ranges: ranges.trim(), ready: v.readyState, // 0 = nothing, 4 = enough to play through paused: v.paused, diff --git a/packages/meshbay-hub/tests/harness/mse_harness.mjs b/packages/meshbay-hub/tests/harness/mse_harness.mjs index 91018ef..f0fd04c 100644 --- a/packages/meshbay-hub/tests/harness/mse_harness.mjs +++ b/packages/meshbay-hub/tests/harness/mse_harness.mjs @@ -35,7 +35,8 @@ const useCallback = (fn) => fn; // be written in. Extracting into an order of our own would quietly repair a // hook declared before its own dependency — a real fault, which reached // production once, and which the harness is otherwise well placed to catch. -const src = ['currentRange', 'bufferedAhead', 'evictBehind', 'flushQueue', 'pump'] +const src = ['currentRange', 'bufferedAhead', 'aheadLimit', 'evictBehind', + 'flushQueue', 'pump'] .sort((a, b) => app.indexOf(`const ${a} = useCallback(`) - app.indexOf(`const ${b} = useCallback(`)) .map(grab).join('\n'); @@ -45,14 +46,21 @@ const CAP = capMB * 1048576; const BITRATE = fileMB * 1048576 / durationS; // Read from app.js too, so a change to the constants is a change to the test. +// Arithmetic is allowed because some of them are written as one (`48 * 1024 * +// 1024`), and refusing anything that is not arithmetic keeps this a reader +// rather than an evaluator of whatever happens to be on the line. const constOf = (name) => { - const m = app.match(new RegExp(`const ${name} = (\\d+)`)); + const m = app.match(new RegExp(`const ${name} = ([^;]+);`)); if (!m) throw new Error(`${name} not found`); - return Number(m[1]); + if (!/^[\d\s.*/+-]+$/.test(m[1])) throw new Error(`${name} is not a number`); + return Number(new Function(`return (${m[1]});`)()); }; const BUFFER_BEHIND_S = constOf('BUFFER_BEHIND_S'); const STREAM_WINDOW = constOf('STREAM_WINDOW'); const BUFFER_AHEAD_S = constOf('BUFFER_AHEAD_S'); +const BUFFER_AHEAD_STEP_BYTES = constOf('BUFFER_AHEAD_STEP_BYTES'); +const BUFFER_AHEAD_MAX_BYTES = constOf('BUFFER_AHEAD_MAX_BYTES'); +const BUFFER_AHEAD_MAX_S = constOf('BUFFER_AHEAD_MAX_S'); const QUEUE_HIGH_WATER = constOf('QUEUE_HIGH_WATER'); const CREDIT_KEEPALIVE_MS = constOf('CREDIT_KEEPALIVE_MS'); @@ -110,7 +118,11 @@ const sb = { }; let pendingRemoveEvents = 0; -const video = { currentTime: 0 }; +// `paused` is part of the element the player reads, not decoration: the +// read-ahead budget only grows while the film is running, and a fake video +// that never reports being paused would exercise the growing branch in a run +// whose whole point is that nobody pressed play. +const video = { currentTime: 0, paused: !playing }; const sbRef = { current: sb }, videoRef = { current: video }; const msRef = { current: { readyState: 'open', endOfStream() {} } }; const queueRef = { current: [] }; @@ -118,6 +130,13 @@ const appendingRef = { current: false }, endedRef = { current: false }; const outstandingRef = { current: 0 }, lastPokeRef = { current: 0 }; const quotaRef = { current: 0 }, stalledRef = { current: false }; const awaitingInitRef = { current: false }; +// The film's own average bitrate, as `stream_init` gives the player, and the +// budget state the read-ahead walks up from. Both start where the component +// starts them. +const bitrateRef = { current: BITRATE }; +const aheadBytesRef = { current: 0 }; +const aheadCapRef = { current: BUFFER_AHEAD_MAX_BYTES }; +const quotaHoldRef = { current: false }; const transportRef = { current: { connected: true, @@ -132,12 +151,16 @@ const fns = new Function( 'sbRef,videoRef,msRef,queueRef,appendingRef,endedRef,outstandingRef,' + 'lastPokeRef,transportRef,BUFFER_BEHIND_S,BUFFER_AHEAD_S,QUEUE_HIGH_WATER,' + 'CREDIT_KEEPALIVE_MS,STREAM_WINDOW,quotaRef,stalledRef,awaitingInitRef,' + - 'console,useCallback', - src + '\n return {currentRange, bufferedAhead, evictBehind, flushQueue, pump};' + 'bitrateRef,aheadBytesRef,aheadCapRef,quotaHoldRef,' + + 'BUFFER_AHEAD_STEP_BYTES,' + + 'BUFFER_AHEAD_MAX_BYTES,BUFFER_AHEAD_MAX_S,console,useCallback', + src + '\n return {currentRange, bufferedAhead, aheadLimit, evictBehind,' + + ' flushQueue, pump};' )(sbRef, videoRef, msRef, queueRef, appendingRef, endedRef, outstandingRef, lastPokeRef, transportRef, BUFFER_BEHIND_S, BUFFER_AHEAD_S, QUEUE_HIGH_WATER, CREDIT_KEEPALIVE_MS, STREAM_WINDOW, quotaRef, stalledRef, awaitingInitRef, - console, useCallback); + bitrateRef, aheadBytesRef, aheadCapRef, quotaHoldRef, BUFFER_AHEAD_STEP_BYTES, + BUFFER_AHEAD_MAX_BYTES, BUFFER_AHEAD_MAX_S, console, useCallback); // The player's own `updateend` listener, transcribed — the one part of the // component that is a listener rather than a callback, and the place the @@ -151,6 +174,15 @@ credit = STREAM_WINDOW; outstandingRef.current = STREAM_WINDOW; let wall = 0; const TICK = 0.05; +// Bytes the link has carried but not yet spent on a whole segment. This used +// to be re-created inside the loop and thrown away at the end of every tick, +// so a link slower than one segment per tick — 5 MB/s at this resolution, +// which is every mobile network there is — delivered *nothing at all* and the +// run reported a player that had simply never been fed. Anything below about +// forty megabits was unmeasurable here, which is most of the cases worth +// measuring. Carried over instead, and capped at one window's worth, because +// a link cannot bank a burst larger than what the node may have in flight. +let netBudget = 0; while (wall < wallS) { wall += TICK; if (playing) { @@ -160,9 +192,9 @@ while (wall < wallS) { fns.pump(); // the 1 s timer and timeupdate while (pendingRemoveEvents > 0) { pendingRemoveEvents--; updateend(); } - let budget = netMBs * 1048576 * TICK; - while (credit > 0 && budget >= SEG) { - credit--; budget -= SEG; sent++; + netBudget = Math.min(netBudget + netMBs * 1048576 * TICK, SEG * STREAM_WINDOW); + while (credit > 0 && netBudget >= SEG) { + credit--; netBudget -= SEG; sent++; outstandingRef.current = Math.max(0, outstandingRef.current - 1); queueRef.current.push({ byteLength: SEG }); fns.flushQueue(); @@ -176,6 +208,10 @@ console.log(JSON.stringify({ heldInBufferMB: +(bytes / 1048576).toFixed(1), queueDepth: queueRef.current.length, bufferedAheadS: +fns.bufferedAhead().toFixed(1), + aheadLimitS: +fns.aheadLimit().toFixed(1), + budgetMB: +(aheadBytesRef.current / 1048576).toFixed(1), + budgetCapMB: +(aheadCapRef.current / 1048576).toFixed(1), + bitrateMbits: +(BITRATE * 8 / 1e6).toFixed(2), watchedS: +video.currentTime.toFixed(1), grants: granted, keepalives, diff --git a/packages/meshbay-hub/tests/test_video_buffer_ceiling.py b/packages/meshbay-hub/tests/test_video_buffer_ceiling.py index 59f97de..7dd9c81 100644 --- a/packages/meshbay-hub/tests/test_video_buffer_ceiling.py +++ b/packages/meshbay-hub/tests/test_video_buffer_ceiling.py @@ -20,6 +20,18 @@ it. Memory was bounded by the browser's ceiling rather than by anything we chose. Buffering by *time* past the playhead instead makes a two-hour film cost the same as a two-minute clip. +**And then a bound in seconds turned out to be the wrong unit.** The browser +limits BYTES, so ninety seconds had to be sized for the highest-bitrate file in +a library and every ordinary file held a fraction of what the same browser +would have taken: measured here against a 100 MB ceiling, a 1.0 Mbit/s film +reached its ninety seconds on 20.7 MB and a 2.3 Mbit/s one on 45.6 MB. On a +mobile network that unused ceiling is the whole of the margin there is for a +dead spot. The read-ahead is now a byte budget converted to seconds at the +file's own bitrate, floored at the fixed bound so nothing pulls less than it +used to, and *walked up* rather than declared — because the real ceiling is +per-engine and per-device, and the alternative to walking up to it is +overshooting it, which costs a refused append every time it is tried. + **The pipeline could not restart itself.** An append refused for quota fires no `updateend`. `updateend` was where credit was granted, so no credit went out; the node then sent nothing, so no segment arrived to call `flushQueue` again. @@ -65,6 +77,17 @@ def _player(app: str) -> str: return app[i:nxt if nxt > 0 else len(app)] +def _const(app: str, name: str) -> float: + """One of the player's numeric constants, arithmetic and all. + + Read out of the source rather than restated here, so a change to a constant + is a change to what these tests assert — the same rule the harness follows. + """ + m = re.search(rf"^const {name} = ([\d\s.*/+-]+);$", app, re.M) + assert m, f"{name} is not a numeric constant in video-player.js" + return float(eval(m.group(1), {"__builtins__": {}}, {})) # noqa: S307 + + # ── The shipped functions, run against a browser that has a ceiling ─────────── HARNESS = Path(__file__).parent / "harness" / "mse_harness.mjs" @@ -108,12 +131,23 @@ def test_the_ceiling_is_never_reached_while_watching(watched): assert watched["quotaRefusals"] == 0 -def test_the_read_ahead_is_bounded_by_the_playhead(idle, watched): - """What replaced "as fast as the network allows".""" - ahead = int(re.search(r"const BUFFER_AHEAD_S = (\d+)", APP.read_text()).group(1)) +def test_the_read_ahead_is_bounded_by_the_playhead(app, idle, watched): + """What replaced "as fast as the network allows". + + The bound is the one the player computed for this film, not the constant: + the constant is only its floor. Asserting against the constant would say + nothing about a film whose bitrate raises the bound above it, which is now + most of them. + """ for name, run in (("idle", idle), ("watching", watched)): - assert run["bufferedAheadS"] < ahead * 2, ( - f"{name}: {run['bufferedAheadS']}s buffered against a {ahead}s " + limit = run["aheadLimitS"] + assert limit >= _const(app, "BUFFER_AHEAD_S"), ( + f"{name}: the bound came out at {limit}s, below the fixed floor — " + "some film now buffers less than it did before the budget existed") + # One window of segments may land past the gate before it shuts, which + # is the flow control working, not the gate leaking. + assert run["bufferedAheadS"] < limit * 1.5, ( + f"{name}: {run['bufferedAheadS']}s buffered against a {limit}s " "bound — the gate is not holding") @@ -131,9 +165,190 @@ def test_a_watched_film_keeps_being_fed(watched): assert watched["removes"] > 0, "nothing was ever evicted behind the playhead" -def test_memory_stays_bounded_over_a_long_watch(watched): - assert watched["heldInBufferMB"] < 40, ( - f"holding {watched['heldInBufferMB']} MB — eviction is not keeping up") +def test_memory_stays_bounded_over_a_long_watch(app, watched): + """The budget is a bound on memory, so it is the one to assert against. + + What the player may hold is the read-ahead budget plus the minute it keeps + behind the playhead for a small seek back. Anything past that is eviction + failing to keep up, which is the fault this guards. + """ + mb = 1024 * 1024 + per_s = watched["bitrateMbits"] * 1e6 / 8 / mb + allowed = _const(app, "BUFFER_AHEAD_MAX_BYTES") / mb \ + + _const(app, "BUFFER_BEHIND_S") * per_s + # One window of segments may be in flight past the budget when the run ends. + allowed += 8 * 256 * 1024 / mb + assert watched["heldInBufferMB"] < allowed, ( + f"holding {watched['heldInBufferMB']} MB against {allowed:.1f} MB of " + "budget and eviction window — eviction is not keeping up") + + +# ── The byte budget, and what it must not cost ──────────────────────────────── + +# A library's worth of bitrates against one ceiling, and the same low-bitrate +# film against a ceiling small enough that the budget cannot have it. Sizes are +# a two-hour film at each rate; nothing here names a real title. +_BITRATE_SWEEP = [ + ("0.6 Mbit/s", {"fileMB": 493.5, "durationS": 7200, "capMB": 100}), + ("2.3 Mbit/s", {"fileMB": 2000, "durationS": 7200, "capMB": 100}), + ("4.7 Mbit/s", {"fileMB": 4000, "durationS": 7200, "capMB": 100}), + ("9.3 Mbit/s", {"fileMB": 8000, "durationS": 7200, "capMB": 100}), + ("0.6 Mbit/s, small ceiling", {"fileMB": 493.5, "durationS": 7200, "capMB": 40}), + ("2.3 Mbit/s, small ceiling", {"fileMB": 2000, "durationS": 7200, "capMB": 40}), +] + + +@pytest.fixture(scope="module") +def sweep(): + return {name: _harness(playing=True, netMBs=35, wallS=900, **cfg) + for name, cfg in _BITRATE_SWEEP} + + +def _fits_under_the_ceiling(app: str, name: str, run: dict) -> bool: + """Whether even the *floor* bound fits in this browser's buffer. + + Where it does not, the film was already unservable before the budget + existed — the fixed ninety seconds plus the minute kept behind is more than + the ceiling, so there is nowhere to put it. See MESHBAY_DESIGN.md §15.3. + """ + cap_mb = dict(_BITRATE_SWEEP)[name]["capMB"] + per_s = run["bitrateMbits"] * 1e6 / 8 / (1024 * 1024) + need = (_const(app, "BUFFER_AHEAD_S") + _const(app, "BUFFER_BEHIND_S")) * per_s + return need <= cap_mb + + +def test_no_film_buffers_less_than_the_fixed_bound_used_to(app, sweep): + """The floor, which is the whole no-regression claim. + + The budget may only ever raise the read-ahead. A film whose bitrate spends + it in under the fixed bound keeps that bound exactly, and nothing that + played before pulls less than it did. + """ + floor = _const(app, "BUFFER_AHEAD_S") + for name, run in sweep.items(): + assert run["aheadLimitS"] >= floor, ( + f"{name}: bound came out at {run['aheadLimitS']}s, under the {floor}s " + "floor — this film now buffers less than it did before") + + +def test_a_film_nobody_started_does_not_spend_the_budget(app): + """A film on the loading screen cannot be interrupted, so it needs no lead. + + Autoplay is blocked on a phone more often than not, so this is the ordinary + first state of every playback and not a corner. Left to grow there, the + budget filled a small browser's entire buffer — 40 MB against the 8 MB the + fixed bound took — with film on a mobile connection that is paying for it, + and reached the ceiling that the fixed bound never came near. + """ + idle_small = _harness(playing=False, fileMB=493.5, durationS=7200, + capMB=40, netMBs=35, wallS=900) + assert idle_small["aheadLimitS"] == _const(app, "BUFFER_AHEAD_S"), ( + f"an unstarted film is allowed {idle_small['aheadLimitS']}s of " + "read-ahead — the budget grows before anyone has pressed play") + assert not idle_small["hitCeiling"], ( + f"a film nobody started filled the buffer to the ceiling " + f"({idle_small['heldInBufferMB']} MB)") + assert idle_small["quotaRefusals"] == 0 + + +def test_an_ordinary_film_buffers_far_past_the_fixed_bound(app, sweep): + """And the point of the exercise. + + Most of a library is well under five megabits, and that is where the + unused ceiling was. A bound that never actually rises is a budget that + bought nothing. + """ + floor = _const(app, "BUFFER_AHEAD_S") + low = sweep["0.6 Mbit/s"] + assert low["aheadLimitS"] > floor * 3, ( + f"a 0.6 Mbit/s film is allowed {low['aheadLimitS']}s of read-ahead " + f"against a {floor}s floor — the byte budget is not being spent") + assert low["bufferedAheadS"] > floor * 3, ( + f"allowed {low['aheadLimitS']}s but only reached " + f"{low['bufferedAheadS']}s — something other than the bound is holding") + + +def test_finding_the_ceiling_costs_a_handful_of_refusals_not_thousands(sweep): + """The budget is walked up to, never overshot wholesale. + + A flat budget discovers a browser's ceiling by being refused at it, and + that refusal repeats: a 48 MB budget against a 40 MB ceiling was measured + at 1386 refused appends on a film that had none before it. Each one is a + thrown exception, a warning on a phone's console and an append's work + thrown away, so the count is the thing to bound. + """ + for name, run in sweep.items(): + assert run["quotaRefusals"] < 50, ( + f"{name}: {run['quotaRefusals']} appends refused for quota — the " + "budget is overshooting the ceiling rather than walking up to it") + + +def test_the_walk_settles_and_playback_survives_it(app, sweep): + """Discovering the ceiling must not cost the film. + + Every rate that was servable at all has to still be watchable while the + budget is being found, and the run has to end below the ceiling rather than + pinned against it. A rate whose *floor* bound does not fit the ceiling is + excluded, not because the budget is allowed to break it, but because it was + already broken: the 9.3 Mbit/s run plays the same 100.8 s with the budget + as it did without one, to the tenth of a second. That wedge is §15.3, not + this commit. + """ + for name, run in sweep.items(): + assert not run["hitCeiling"], f"{name}: finished pinned at the ceiling" + if not _fits_under_the_ceiling(app, name, run): + continue + assert run["watchedS"] > 600, ( + f"{name}: only {run['watchedS']}s played in 900s of wall clock — " + "playback did not survive the walk") + + +def test_a_refused_append_is_not_retried_until_something_frees_room(app): + """Why the walk is cheap. + + An append refused for quota with nothing to evict will be refused again + until room appears, and pump() calls flushQueue on a clock — so the retry + alone cost a refusal several times a second for as long as it took the film + to play far enough for eviction to have anything to drop. `evictBehind` is + the only thing that gives a SourceBuffer room back, so it is the only thing + that may lift the hold; a timer or the playhead would move long before the + answer changes. + """ + player = _player(app) + flush = player[player.index("const flushQueue = useCallback("):] + flush = flush[:flush.index("\n }, [")] + assert "if (quotaHoldRef.current) return;" in flush, ( + "flushQueue retries a refused append on every pump tick") + evict = player[player.index("const evictBehind = useCallback("):] + evict = evict[:evict.index("\n }, [")] + assert "quotaHoldRef.current = false" in evict, ( + "nothing lifts the hold when room is freed, so the first refusal stops " + "every append for the rest of the film") + # A seek backwards leaves the playhead behind everything buffered, and + # reinitAt empties the buffer outright — so the hold must not survive it. + reinit = player[player.index("const reinitAt = async ("):] + reinit = reinit[:reinit.index("\n };")] + assert "quotaHoldRef.current = false" in reinit, ( + "a hold set before a seek survives the buffer it was held against") + + +def test_the_budget_starts_from_nothing_on_every_new_stream(app): + """A new stream starts from a known state, and this is two more refs. + + What one browser refused for a nine-megabit film says nothing about the + next one, and a budget left at the floor by the film before would silently + cap every film after it for the life of the page. + """ + player = _player(app) + i = player.index("appendingRef.current = false;\n endedRef.current = false;") + reset = player[i:i + 2000] + for ref, value in (("bitrateRef", "0"), + ("aheadBytesRef", "0"), + ("aheadCapRef", "BUFFER_AHEAD_MAX_BYTES"), + ("quotaHoldRef", "false")): + assert f"{ref}.current = {value};" in reset, ( + f"{ref} is not reset at the start of a stream, so the next film " + "inherits what this one learnt") # ── The shape the fix depends on ────────────────────────────────────────────── diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py index e92ec13..a5e15df 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py @@ -836,11 +836,19 @@ class WebRTCPeerSession: # misbehaving — it is the only view of the browser there is # when the browser is a phone. else: + # `ahead` on its own cannot say whether a short buffer is + # the player's own gate holding or the network failing to + # keep up, and those two want opposite answers. `limit` is + # what the gate is set to for this film and `budget` the + # byte budget it was derived from, so the three read as one + # sentence. log.debug( - "stream: client t=%ss ahead=%ss ready=%s paused=%s " + "stream: client t=%ss ahead=%ss/%ss budget=%sMB " + "ready=%s paused=%s " "stalled=%s q=%s inflight=%s appending=%s updating=%s " "quota=%s ms=%s err=%s ranges=[%s] (sent=%d)", - _f("t"), _f("ahead"), _f("ready"), _f("paused"), + _f("t"), _f("ahead"), _f("limit"), _f("budgetMB"), + _f("ready"), _f("paused"), _f("stalled"), _f("q"), _f("inflight"), _f("appending"), _f("updating"), _f("quota"), _f("ms"), _f("err", 80), _f("ranges", 120), self._stream_segments) |