From 677775266217b08044fe734be78f0e1bc874d708 Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Thu, 17 Sep 2026 11:23:36 +0200 Subject: fix(node): a seek left the audio a GOP behind the picture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit -ss before -i cannot trim copied video, which must begin on a keyframe, but accurate_seek did trim the re-encoded audio to the exact request. Every seek on a copied stream therefore opened with a GOP-wide hole in the audio and ran a GOP out of sync afterwards — 9.979s on a real film with a 10s keyframe interval. Accurate seeking is now off wherever video is copied, and stays on where it is re-encoded, which is the only path that could already begin where it was asked to. Every timestamp was correct throughout, which is why nothing caught it; the tests assert on decoded audio and on frames compared against the source. Co-Authored-By: Claude Opus 5 --- docs/MESHBAY_DESIGN.md | 15 +++++++++++++++ docs/MESHBAY_NODE_PROTOCOL.md | 5 ++++- 2 files changed, 19 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index e7d4a76..ef65995 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -2031,6 +2031,21 @@ MSE string) fed to a source buffer, with the node holding one slot per viewer. - **Seeking restarts the source with an index seek before the input**, clamped away from the end and echoed back; the client supplies the timestamp offset, because copying timestamps does not preserve position. +- **A seek trims what it can, and what it can differs per stream — which is a + desync, not an inconvenience.** Accurate seeking cannot trim copied video, which + must begin on a keyframe, but it does trim re-encoded audio to the exact request. + The output then carries video from the keyframe and audio from the request, a + whole GOP apart, with a hole between them. So **accurate seeking is off wherever + video is copied and on wherever it is re-encoded**, which is the only place the + two can agree on where to begin. + + > **Every timestamp was correct while this was happening.** First PTS per stream, + > durations, spans, and the browser's own audio/video delta through MediaSource + > all agreed, because both streams genuinely sat where the container said. Only + > the *content* at a given instant was displaced. A property that every + > timestamp-shaped check confirms is not thereby true: this one needed the output + > decoded and compared against the source, frame against frame and envelope + > against envelope. - **The viewer picks the audio track, and picking one is a seek.** One ffmpeg carries one audio track, so there is nothing to switch inside a running stream: the node is asked again at the current position and the source buffer is reset diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index 19cced9..d4b9552 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1636,7 +1636,9 @@ array while MediaSource consumes it a segment at a time. | | ffprobe: codec, duration, the | | audio tracks | | spawn ffmpeg - | | -ss before -i (index seek) + | | -ss before -i (index seek), + | | -noaccurate_seek when the + | | video is copied | | video: copy, or libx264 when | | the browser cannot decode | | audio: always AAC, 2 ch, @@ -1667,6 +1669,7 @@ array while MediaSource consumes it a segment at a time. | `n == 0` | keepalive: a viewer buffered 90 s ahead grants nothing and must still be able to say it is there | | Concurrent transcodes | 8 node-wide, semaphore on the transport context | | Seeking | a new `stream_req` with `start`; the previous stream is retired first, ffmpeg respawned with `-ss` | +| Accurate seek | **off when the video is copied, on when it is re-encoded.** Copied video has to begin on a keyframe and cannot be trimmed to the request; re-encoded audio can, and is. Leaving both at the default put a whole GOP of silence at the head of every seek and left sound and picture a GOP apart — with correct timestamps throughout, so nothing downstream could detect it | | `start` in `stream_init` | the value actually used — seeking lands on the keyframe at or before the request, and the client adds it back as `SourceBuffer.timestampOffset` | | `audio_tracks` in `stream_init` | every audio track: `i` (the **audio ordinal**, what `-map 0:a:` takes, never the container stream index), `lang`, `title`, `codec`, `ch`. Empty for a file with no audio | | `audio_track` in `stream_req` | which ordinal to map. Absent, out of range or malformed is the first track | -- cgit v1.2.3