From ad4ca3229002997934ccb5b2eaeb553c13b8888f Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Thu, 17 Sep 2026 13:39:23 +0200 Subject: feat: embedded subtitles in the video player (MNP 3.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MSE decodes no in-band text track, so a subtitle cannot ride inside the fragmented MP4 the player is fed. The node extracts one track whole, converts it to WebVTT and caches it under its own hash; the client pulls that blob through the ordinary file_req/chunk path and hangs a on the video element — the same indirection as a TMDB poster or an audio transcode, which is what makes a film's subtitles extracted once in the life of the file rather than once per viewing. Whole-file also makes the cues absolute, so a seek and an audio-language change both leave the track untouched. **The ordinal counts every subtitle stream, including the ones never listed.** Only text codecs are offered: a bitmap track (PGS, VOBSUB — about a fifth of a real library) has no path to WebVTT without OCR, and one extracted anyway yields a header with no cues, which is a menu entry that shows nothing and reports no error. Numbering the survivors of that filter would give a PGS/SRT/SRT file the ordinals 0 and 1 for its text tracks and `-map 0:s:0` would then extract the PGS — the same trap `AudioTrack.ordinal` exists for, one level deeper. A fixture whose first subtitle stream cannot be decoded pins it, and the handler checks membership of the probed list, never a range. Additive and MINOR: the selector is drawn from `subtitle_tracks` in the node's own `stream_init` and from no version number, so `subtitle_req` is never sent to a peer that would not answer it. The floor stays at 3.0. Also here: a failed extraction never touches playback, a superseded reply cannot install its blob over a newer choice, and `_languageName` is shared with the audio labels — lifted by both label harnesses, since a lift that names one function stops covering the rule the moment logic moves out of it. Tests: 9 node (tracks told apart by the words in the extracted cues, not by tags), 10 client. Full suite green: 1545 node/common, 1252 hub. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc --- docs/MESHBAY_NODE_PROTOCOL.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) (limited to 'docs/MESHBAY_NODE_PROTOCOL.md') diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index d4b9552..64f32e3 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1645,7 +1645,8 @@ array while MediaSource consumes it a segment at a time. | | -map 0:a: | | frag_keyframe+empty_moov |<- stream_init {v, file_id, codec, duration, start, | - | audio_tracks[], audio_track} ------------------| + | audio_tracks[], audio_track, | + | subtitle_tracks[]} ----------------------------| | | | check MediaSource.isTypeSupported(codec) | | | @@ -1676,6 +1677,11 @@ array while MediaSource consumes it a segment at a time. | `audio_track` in `stream_init` | the ordinal actually used, for the same reason `start` is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. `null` when the file has no audio | | Changing track | a new `stream_req` at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream | | Capability discovery | **the list, not the version.** A client draws its selector from `audio_tracks` and sends `audio_track` only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language | +| `subtitle_tracks` in `stream_init` | the subtitle tracks that can be shown: `i` (the **subtitle ordinal**, what `-map 0:s:` takes, counted over *every* subtitle stream including the ones absent from this list), `lang`, `title`, `codec`. Empty for a file with no convertible subtitles | +| Which subtitle tracks are listed | text codecs only (subrip, ass, mov_text, …). Bitmap streams (PGS, VOBSUB — about a fifth of a real library) have no WebVTT without OCR, and one extracted anyway yields a header with no cues: a track that appears in the menu and shows nothing. A file whose subtitles are all bitmap reports none, exactly like a file with none | +| Why the ordinal is not the list position | the two differ whenever a bitmap stream precedes a text one. Renumbering the survivors would map `0:s:0` to the stream that cannot be decoded — which is an empty WebVTT, not an error | +| Fetching a track | `subtitle_req {file_id, track}` → `subtitle_resp {file_id, track, hash, size, mime}`; the blob is pulled by `hash` over `file_req`, the same indirection as a poster or an audio transcode. Extracted whole-file, converted to WebVTT, cached under the file's own id — so a film is extracted once, not once per viewing | +| Subtitles and seeking | nothing. The cues carry the source's absolute timestamps, so a seek and an audio-language change both leave the client's `` untouched | An ffmpeg failure before any output produces `error: Could not stream this file`; stderr stays server-side, where it belongs — it names paths on the operator's disk and @@ -1856,6 +1862,7 @@ The corresponding *settings* are signed operations (§10.4). | `tmdb_search_req {media_type, query}` | `tmdb_search_resp {results: [{id, title, year, poster}]}` | candidates for a human to pick from; never collapsed to one guess | | `music_meta_req {file_id}` | `music_meta_resp {file_id, ...}` | MusicBrainz; cover art cached like a poster | | `audio_transcode_req {file_id}` | `audio_transcode_resp {file_id, hash, size, mime}` | WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by `hash` over `file_req` | +| `subtitle_req {file_id, track}` | `subtitle_resp {file_id, track, hash, size, mime}` | MSE decodes no in-band text track, so a subtitle travels beside the stream. `track` is the ordinal from `stream_init.subtitle_tracks` and is echoed back, because one film's two tracks are exactly the pair that can be in flight together. Fetch the result by `hash` over `file_req` | Every one of these is keyed by the entry's **`file_id`**, never by a path: a path names the folder a file is in, so two files in one folder — any multi-episode season — would @@ -1935,6 +1942,7 @@ it back (§3.5). | `tmdb_search_req` / `_resp` | C→N / N→C | auth | candidate matches for an operator | | `music_meta_req` / `_resp` | C→N / N→C | auth | MusicBrainz metadata for one file | | `audio_transcode_req` / `_resp` | C→N / N→C | auth | browser-playable copy of a WMA/MPC file | +| `subtitle_req` / `_resp` | C→N / N→C | auth | one embedded subtitle track as WebVTT, by cache hash | | `ping` / `pong` | C→N / N→C | auth | liveness on an open channel | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | | `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | @@ -2004,8 +2012,8 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`3.2`**; oldest peer -accepted: **`3.0`** — 3.1 and 3.2 are both additive, so the floor does not move with +MNP versions independently of the package version. Current: **`3.3`**; oldest peer +accepted: **`3.0`** — 3.1, 3.2 and 3.3 are all additive, so the floor does not move with them. The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; -- cgit v1.2.3