From ad4ca3229002997934ccb5b2eaeb553c13b8888f Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Thu, 17 Sep 2026 13:39:23 +0200 Subject: feat: embedded subtitles in the video player (MNP 3.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MSE decodes no in-band text track, so a subtitle cannot ride inside the fragmented MP4 the player is fed. The node extracts one track whole, converts it to WebVTT and caches it under its own hash; the client pulls that blob through the ordinary file_req/chunk path and hangs a on the video element — the same indirection as a TMDB poster or an audio transcode, which is what makes a film's subtitles extracted once in the life of the file rather than once per viewing. Whole-file also makes the cues absolute, so a seek and an audio-language change both leave the track untouched. **The ordinal counts every subtitle stream, including the ones never listed.** Only text codecs are offered: a bitmap track (PGS, VOBSUB — about a fifth of a real library) has no path to WebVTT without OCR, and one extracted anyway yields a header with no cues, which is a menu entry that shows nothing and reports no error. Numbering the survivors of that filter would give a PGS/SRT/SRT file the ordinals 0 and 1 for its text tracks and `-map 0:s:0` would then extract the PGS — the same trap `AudioTrack.ordinal` exists for, one level deeper. A fixture whose first subtitle stream cannot be decoded pins it, and the handler checks membership of the probed list, never a range. Additive and MINOR: the selector is drawn from `subtitle_tracks` in the node's own `stream_init` and from no version number, so `subtitle_req` is never sent to a peer that would not answer it. The floor stays at 3.0. Also here: a failed extraction never touches playback, a superseded reply cannot install its blob over a newer choice, and `_languageName` is shared with the audio labels — lifted by both label harnesses, since a lift that names one function stops covering the rule the moment logic moves out of it. Tests: 9 node (tracks told apart by the words in the extracted cues, not by tags), 10 client. Full suite green: 1545 node/common, 1252 hub. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01UGY17EPph5LsLzePPXhUVc --- docs/MESHBAY_DESIGN.md | 14 +- docs/MESHBAY_NODE_PROTOCOL.md | 14 +- .../meshbay-common/src/meshbay_common/__init__.py | 20 +- .../meshbay-common/src/meshbay_common/protocol.py | 7 + .../meshbay-hub/src/meshbay_hub/static/icon.js | 5 + .../src/meshbay_hub/static/locales/de.js | 4 + .../src/meshbay_hub/static/locales/en.js | 4 + .../src/meshbay_hub/static/locales/es.js | 4 + .../src/meshbay_hub/static/locales/fr.js | 4 + .../src/meshbay_hub/static/locales/it.js | 4 + .../src/meshbay_hub/static/locales/ja.js | 4 + .../src/meshbay_hub/static/locales/nl.js | 4 + .../src/meshbay_hub/static/locales/pl.js | 4 + .../src/meshbay_hub/static/locales/pt-BR.js | 4 + .../src/meshbay_hub/static/locales/zh-CN.js | 4 + .../src/meshbay_hub/static/transport.js | 37 +++ .../src/meshbay_hub/static/video-player.js | 176 ++++++++++- .../meshbay-hub/tests/test_video_audio_track.py | 6 + packages/meshbay-hub/tests/test_video_subtitles.py | 232 +++++++++++++++ .../meshbay-node/src/meshbay_node/media_probe.py | 57 ++++ .../src/meshbay_node/transport/webrtc_server.py | 165 +++++++++++ .../tests/test_stream_subtitle_tracks.py | 327 +++++++++++++++++++++ 22 files changed, 1088 insertions(+), 12 deletions(-) create mode 100644 packages/meshbay-hub/tests/test_video_subtitles.py create mode 100644 packages/meshbay-node/tests/test_stream_subtitle_tracks.py diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 92f56ef..817afaa 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1132,13 +1132,16 @@ it**. > the branches come back.** **The floor is not the current version, and MINOR additions are why.** It is -`MNP_MIN_SUPPORTED` in `handshake.py`, it equals the last MAJOR, and 3.1 and 3.2 -have both been added above it without moving it. So a peer can be reachable and +`MNP_MIN_SUPPORTED` in `handshake.py`, it equals the last MAJOR, and 3.1, 3.2 and +3.3 have all been added above it without moving it. So a peer can be reachable and still not do something the current version can, and the client has to cope with that — **by reading the peer's own answer, never by comparing version numbers**. 3.2's audio tracks are the worked example: the node lists them in `stream_init`, the client draws its selector from that list, and a node that sends no list gets no -selector. That is not the branch the box above refuses. The branch it refuses is a +selector. 3.3's subtitles repeat it exactly, and add the case where the list is +narrower than the file: only the tracks the node can convert to WebVTT are named, +so what the menu offers is what will actually appear, and a track's ordinal is +therefore not its position in the list. That is not the branch the box above refuses. The branch it refuses is a flag the client sets from a version it parsed; this is the node stating what it has, in the same message the feature already needed, and it takes exactly one path per peer because the peer said which. @@ -2940,7 +2943,8 @@ The hub, the node daemon, both transports, the unified handshake, admission and pairing, device linking with member-visible evidence, per-node identity, named roots with RO/RW and eject/plug, the indexer with partial hashing, uploads, the sealed index and sealed upload path, encrypted chat with epochs, video streaming with -seeking, transfer leases with queueing, pause and resume, the group-application +seeking, audio-language and subtitle selection, transfer leases with queueing, +pause and resume, the group-application framework with Chat, Files, Videos, Music and Photos, cross-group search with source merging, the operator CLI and loopback control API, the desktop client through its identity and download stages, @@ -2956,6 +2960,8 @@ account recovery, and the Windows port through packaging. | **Stage D11** | Windows clean-machine install verification and code signing | | **Stage D12** | Release key, signed repositories, updates through the OS | | — | DLNA/UPnP casting (§11.4) | +| — | **Subtitles on a Chromecast.** The client shows them; the relay re-serves the same fMP4 bytes the browser gets, and a `` in the renderer's DOM never leaves the machine. The route is an out-of-band `/subs.vtt` on the relay plus `tracks` on `player.load()`, and its one unknown — whether the receiver's CORS requirement can be met by the relay — cannot be settled by reading code | +| — | **Bitmap subtitles** (PGS, VOBSUB — about a fifth of the embedded streams). No WebVTT without OCR; they are not listed rather than listed and blank. Burn-in covers them and costs `-c:v copy`, which is what the eight-slot sizing assumes never happens | | — | Delegation (§3.4) | | — | Tier 3 roster attestation (§3.3) | | — | Playlists (§9.10) | diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index d4b9552..64f32e3 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1645,7 +1645,8 @@ array while MediaSource consumes it a segment at a time. | | -map 0:a: | | frag_keyframe+empty_moov |<- stream_init {v, file_id, codec, duration, start, | - | audio_tracks[], audio_track} ------------------| + | audio_tracks[], audio_track, | + | subtitle_tracks[]} ----------------------------| | | | check MediaSource.isTypeSupported(codec) | | | @@ -1676,6 +1677,11 @@ array while MediaSource consumes it a segment at a time. | `audio_track` in `stream_init` | the ordinal actually used, for the same reason `start` is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. `null` when the file has no audio | | Changing track | a new `stream_req` at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream | | Capability discovery | **the list, not the version.** A client draws its selector from `audio_tracks` and sends `audio_track` only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language | +| `subtitle_tracks` in `stream_init` | the subtitle tracks that can be shown: `i` (the **subtitle ordinal**, what `-map 0:s:` takes, counted over *every* subtitle stream including the ones absent from this list), `lang`, `title`, `codec`. Empty for a file with no convertible subtitles | +| Which subtitle tracks are listed | text codecs only (subrip, ass, mov_text, …). Bitmap streams (PGS, VOBSUB — about a fifth of a real library) have no WebVTT without OCR, and one extracted anyway yields a header with no cues: a track that appears in the menu and shows nothing. A file whose subtitles are all bitmap reports none, exactly like a file with none | +| Why the ordinal is not the list position | the two differ whenever a bitmap stream precedes a text one. Renumbering the survivors would map `0:s:0` to the stream that cannot be decoded — which is an empty WebVTT, not an error | +| Fetching a track | `subtitle_req {file_id, track}` → `subtitle_resp {file_id, track, hash, size, mime}`; the blob is pulled by `hash` over `file_req`, the same indirection as a poster or an audio transcode. Extracted whole-file, converted to WebVTT, cached under the file's own id — so a film is extracted once, not once per viewing | +| Subtitles and seeking | nothing. The cues carry the source's absolute timestamps, so a seek and an audio-language change both leave the client's `` untouched | An ffmpeg failure before any output produces `error: Could not stream this file`; stderr stays server-side, where it belongs — it names paths on the operator's disk and @@ -1856,6 +1862,7 @@ The corresponding *settings* are signed operations (§10.4). | `tmdb_search_req {media_type, query}` | `tmdb_search_resp {results: [{id, title, year, poster}]}` | candidates for a human to pick from; never collapsed to one guess | | `music_meta_req {file_id}` | `music_meta_resp {file_id, ...}` | MusicBrainz; cover art cached like a poster | | `audio_transcode_req {file_id}` | `audio_transcode_resp {file_id, hash, size, mime}` | WMA/Musepack decode in no mainstream browser; the node transcodes once to AAC/M4A and caches it. Fetch the result by `hash` over `file_req` | +| `subtitle_req {file_id, track}` | `subtitle_resp {file_id, track, hash, size, mime}` | MSE decodes no in-band text track, so a subtitle travels beside the stream. `track` is the ordinal from `stream_init.subtitle_tracks` and is echoed back, because one film's two tracks are exactly the pair that can be in flight together. Fetch the result by `hash` over `file_req` | Every one of these is keyed by the entry's **`file_id`**, never by a path: a path names the folder a file is in, so two files in one folder — any multi-episode season — would @@ -1935,6 +1942,7 @@ it back (§3.5). | `tmdb_search_req` / `_resp` | C→N / N→C | auth | candidate matches for an operator | | `music_meta_req` / `_resp` | C→N / N→C | auth | MusicBrainz metadata for one file | | `audio_transcode_req` / `_resp` | C→N / N→C | auth | browser-playable copy of a WMA/MPC file | +| `subtitle_req` / `_resp` | C→N / N→C | auth | one embedded subtitle track as WebVTT, by cache hash | | `ping` / `pong` | C→N / N→C | auth | liveness on an open channel | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | | `member_unpin` / `_ack` | C→N / N→C | signed | forget a pinned identity | @@ -2004,8 +2012,8 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`3.2`**; oldest peer -accepted: **`3.0`** — 3.1 and 3.2 are both additive, so the floor does not move with +MNP versions independently of the package version. Current: **`3.3`**; oldest peer +accepted: **`3.0`** — 3.1, 3.2 and 3.3 are all additive, so the floor does not move with them. The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; diff --git a/packages/meshbay-common/src/meshbay_common/__init__.py b/packages/meshbay-common/src/meshbay_common/__init__.py index c4c4bf4..c5b0279 100644 --- a/packages/meshbay-common/src/meshbay_common/__init__.py +++ b/packages/meshbay-common/src/meshbay_common/__init__.py @@ -201,5 +201,23 @@ __version__ = "0.14.0" # compatibility switch 3.0 refused, because there is no second branch on the # node: a 3.2 node always enumerates, always honours what it is asked for, and # always says which track it used. `MNP_MIN_SUPPORTED` does not move. -MNP_VERSION = "3.2" +# +# **3.3 (2026-09-17): the viewer picks a subtitle track.** +# +# `stream_init` carries `subtitle_tracks` (ordinal, language, title, codec); +# `subtitle_req` asks for one by ordinal and `subtitle_resp` names a cache hash +# to pull through the ordinary `file_req`/`file_chunk` path, exactly as +# `audio_transcode_resp` does. Subtitles travel beside the stream because MSE +# decodes no in-band text track, and whole-file because the extraction is then +# absolute in time: a seek re-extracts nothing and the `` is untouched. +# +# **Additive, MINOR, and for once nothing can be served wrongly.** An +# `audio_track` a 3.2 node ignores is the wrong language played in silence; +# `subtitle_req` has no such failure — a node too old to know the message +# answers `unknown message type` and the client shows no subtitles, which is +# what it showed before. The selector is still drawn from `subtitle_tracks` in +# the node's own `stream_init` and from no version number, so the request is +# never sent to a peer that could not answer it. `MNP_MIN_SUPPORTED` does not +# move. +MNP_VERSION = "3.3" MHP_VERSION = "0.1" diff --git a/packages/meshbay-common/src/meshbay_common/protocol.py b/packages/meshbay-common/src/meshbay_common/protocol.py index 8ad36e6..c3cb099 100644 --- a/packages/meshbay-common/src/meshbay_common/protocol.py +++ b/packages/meshbay-common/src/meshbay_common/protocol.py @@ -191,6 +191,13 @@ class MNP: # TMDB poster or a MusicBrainz cover. AUDIO_TRANSCODE_REQ = "audio_transcode_req" # client → node: transcode this file id AUDIO_TRANSCODE_RESP = "audio_transcode_resp" # node → client: cache hash/size/mime + # One embedded subtitle track, extracted whole-file to WebVTT and cached + # under its own hash — same shape as the audio transcode above, and for + # the same reason: the result is small, identical every time, and MSE + # cannot decode an in-band text track, so it has to travel beside the + # stream rather than inside it. Whole-file, so a seek re-extracts nothing. + SUBTITLE_REQ = "subtitle_req" # client → node: file id + track ordinal + SUBTITLE_RESP = "subtitle_resp" # node → client: cache hash/size/mime # Device linking. A new device files a request bound to a code it displays; # an already-pinned device of the same account approves it. Neither the hub # nor the node can produce the countersignature. diff --git a/packages/meshbay-hub/src/meshbay_hub/static/icon.js b/packages/meshbay-hub/src/meshbay_hub/static/icon.js index 306158a..94a82fb 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/icon.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/icon.js @@ -100,6 +100,11 @@ const ICON_PATHS = { // overlapping outlines read as a tangle at 20 px. 'speech-pair': ['M11 2.5H4A1.5 1.5 0 0 0 2.5 4v4A1.5 1.5 0 0 0 4 9.5h1V12l3-2.5h3A1.5 1.5 0 0 0 12.5 8V4A1.5 1.5 0 0 0 11 2.5z', 'M13 12.5h7A1.5 1.5 0 0 1 21.5 14v4a1.5 1.5 0 0 1-1.5 1.5h-1V22l-3-2.5h-3A1.5 1.5 0 0 1 11.5 18v-4a1.5 1.5 0 0 1 1.5-1.5z'], + // A frame with two lines at its foot — the conventional subtitle glyph, and + // the reason `speech-pair` above is not a frame: the two sit side by side in + // the player's toolbar and have to be told apart at a glance. + subtitles: ['M3.5 5.5h17a1 1 0 0 1 1 1v11a1 1 0 0 1-1 1h-17a1 1 0 0 1-1-1v-11a1 1 0 0 1 1-1z', + 'M6 14.5h7', 'M15.5 14.5h2.5'], image: ['M5 3.5h14a1.5 1.5 0 0 1 1.5 1.5v14a1.5 1.5 0 0 1-1.5 1.5H5a1.5 1.5 0 0 1-1.5-1.5V5a1.5 1.5 0 0 1 1.5-1.5z', 'M7 9.5a1.5 1.5 0 1 0 3 0 1.5 1.5 0 0 0-3 0', 'M20.5 15l-5-5-9.5 9.5'], diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js index 806fa38..dd53bcf 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js @@ -251,6 +251,10 @@ export default { 'video.from_start': "Von vorn beginnen", 'video.audio_track': 'Tonspur', 'video.audio_track_n': 'Spur {n}', + 'video.subtitles': 'Untertitel', + 'video.subtitles_off': 'Aus', + 'video.subtitle_track_n': 'Spur {n}', + 'video.err_subtitle': 'Diese Untertitelspur konnte nicht geladen werden', 'video.close': 'Schließen (Esc)', 'preview.pdf_fallback': 'Dieser Browser zeigt das PDF nicht direkt an. Laden Sie es ' + 'stattdessen herunter — entschlüsselt wurde es ohnehin hier.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js index 92114c9..e9f3d8d 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js @@ -250,6 +250,10 @@ export default { 'video.from_start': "Start from the beginning", 'video.audio_track': 'Audio track', 'video.audio_track_n': 'Track {n}', + 'video.subtitles': 'Subtitles', + 'video.subtitles_off': 'Off', + 'video.subtitle_track_n': 'Track {n}', + 'video.err_subtitle': 'Could not load this subtitle track', 'video.close': 'Close (Esc)', 'preview.pdf_fallback': 'This browser will not display the PDF inline. Download it instead — it was decrypted here either way.', 'preview.too_large': 'This file is {size}, more than this page can hold in memory ({limit}). Download it instead — a download is written straight to disk.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js index d9b8f63..68fbe10 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js @@ -249,6 +249,10 @@ export default { 'video.from_start': "Empezar desde el principio", 'video.audio_track': 'Pista de audio', 'video.audio_track_n': 'Pista {n}', + 'video.subtitles': 'Subtítulos', + 'video.subtitles_off': 'Ninguno', + 'video.subtitle_track_n': 'Pista {n}', + 'video.err_subtitle': 'No se pudo cargar esta pista de subtítulos', 'video.close': 'Cerrar (Esc)', 'preview.pdf_fallback': 'Este navegador no mostrará el PDF integrado. Descárguelo ' + 'en su lugar — en cualquier caso se descifró aquí.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js index d507d28..679a458 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js @@ -250,6 +250,10 @@ export default { 'video.from_start': "Reprendre depuis le début", 'video.audio_track': 'Piste audio', 'video.audio_track_n': 'Piste {n}', + 'video.subtitles': 'Sous-titres', + 'video.subtitles_off': 'Aucun', + 'video.subtitle_track_n': 'Piste {n}', + 'video.err_subtitle': 'Impossible de charger cette piste de sous-titres', 'video.close': 'Fermer (Échap)', 'preview.pdf_fallback': 'Ce navigateur n’affichera pas le PDF directement. ' + 'Téléchargez-le plutôt — il a été déchiffré ici dans les deux cas.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js index b6c68dd..9a34c93 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js @@ -250,6 +250,10 @@ export default { 'video.from_start': "Riparti dall'inizio", 'video.audio_track': 'Traccia audio', 'video.audio_track_n': 'Traccia {n}', + 'video.subtitles': 'Sottotitoli', + 'video.subtitles_off': 'Nessuno', + 'video.subtitle_track_n': 'Traccia {n}', + 'video.err_subtitle': 'Impossibile caricare questa traccia di sottotitoli', 'video.close': 'Chiudi (Esc)', 'preview.pdf_fallback': 'Questo browser non mostrerà il PDF nella pagina. Lo scarichi ' + 'invece — in ogni caso è stato decifrato qui.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js index 99e2e1f..1d58e2c 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js @@ -247,6 +247,10 @@ export default { 'video.from_start': "最初から再生する", 'video.audio_track': '音声トラック', 'video.audio_track_n': 'トラック {n}', + 'video.subtitles': '字幕', + 'video.subtitles_off': 'なし', + 'video.subtitle_track_n': 'トラック {n}', + 'video.err_subtitle': 'この字幕トラックを読み込めませんでした', 'video.close': '閉じる（Esc）', 'preview.pdf_fallback': 'このブラウザーはページ内に PDF を表示しません。' + 'ダウンロードしてご覧ください。いずれにせよ復号はここで行われています。', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js index 269fa70..5bef61f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js @@ -251,6 +251,10 @@ export default { 'video.from_start': "Vanaf het begin afspelen", 'video.audio_track': 'Audiospoor', 'video.audio_track_n': 'Spoor {n}', + 'video.subtitles': 'Ondertiteling', + 'video.subtitles_off': 'Geen', + 'video.subtitle_track_n': 'Spoor {n}', + 'video.err_subtitle': 'Dit ondertitelspoor kon niet worden geladen', 'video.close': 'Sluiten (Esc)', 'preview.pdf_fallback': 'Deze browser toont de PDF niet in de pagina zelf. Download ' + 'hem in plaats daarvan — ontsleuteld werd hij hoe dan ook hier.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js index 363296c..8989880 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js @@ -256,6 +256,10 @@ export default { 'video.from_start': "Odtwórz od początku", 'video.audio_track': 'Ścieżka dźwiękowa', 'video.audio_track_n': 'Ścieżka {n}', + 'video.subtitles': 'Napisy', + 'video.subtitles_off': 'Brak', + 'video.subtitle_track_n': 'Ścieżka {n}', + 'video.err_subtitle': 'Nie udało się wczytać tej ścieżki napisów', 'video.close': 'Zamknij (Esc)', 'preview.pdf_fallback': 'Ta przeglądarka nie wyświetli pliku PDF na stronie. Proszę ' + 'go pobrać — i tak został odszyfrowany tutaj.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js index 2a90956..f1b8643 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js @@ -251,6 +251,10 @@ export default { 'video.from_start': "Começar do início", 'video.audio_track': 'Faixa de áudio', 'video.audio_track_n': 'Faixa {n}', + 'video.subtitles': 'Legendas', + 'video.subtitles_off': 'Nenhuma', + 'video.subtitle_track_n': 'Faixa {n}', + 'video.err_subtitle': 'Não foi possível carregar esta faixa de legendas', 'video.close': 'Fechar (Esc)', 'preview.pdf_fallback': 'Este navegador não exibirá o PDF na própria página. Baixe ' + 'o arquivo — de todo modo ele foi descriptografado aqui.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js index e3aa440..91da464 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js @@ -245,6 +245,10 @@ export default { 'video.from_start': "从头开始播放", 'video.audio_track': '音轨', 'video.audio_track_n': '音轨 {n}', + 'video.subtitles': '字幕', + 'video.subtitles_off': '关闭', + 'video.subtitle_track_n': '字幕 {n}', + 'video.err_subtitle': '无法加载此字幕轨道', 'video.close': '关闭（Esc）', 'preview.pdf_fallback': '此浏览器不会在页面内显示该 PDF。请改为下载——无论如何它都已在本地解密。', 'preview.too_large': '该文件为 {size}，超出本页面可在内存中保存的上限（{limit}）。请改为下载——下载会直接写入磁盘。', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/transport.js b/packages/meshbay-hub/src/meshbay_hub/static/transport.js index 262b00e..63c71ff 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/transport.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/transport.js @@ -1606,6 +1606,25 @@ class MeshBayTransport { return msg; } + /** + * One embedded subtitle track, extracted node-side to WebVTT. Returns + * `{ hash, size, mime, track }` — the *cache* hash to pull through the + * ordinary file_req/chunk path, the same indirection as an audio transcode + * or a TMDB poster, and cached node-side under the file's own id so a film + * is extracted once rather than once per viewing. + * + * `track` is the ordinal the node published in `stream_init.subtitle_tracks` + * and is passed back untouched: it counts every subtitle stream in the + * container, including the bitmap ones that are never listed, so it is not + * a position in the list this client received. + */ + async requestSubtitle(fileId, track) { + const msg = await this._sendAndWait( + { type: 'subtitle_req', v: '0.9', file_id: fileId, track }, 90000); + if (msg.type === 'error') throw new Error(msg.detail); + return msg; + } + /** * Whether MusicBrainz lookups run for this group at all — per-group from * the start (docs/musicbay.md §3.2/§6). Signed like setTmdbEnabled. @@ -2968,6 +2987,12 @@ class MeshBayTransport { // Same reordering hazard as media_meta_req: the player prefetches // the next track while the current one may still be transcoding. : obj.type === 'audio_transcode_req' ? `audio_transcode:${obj.file_id}` + // The track ordinal is part of the key, not just the file id: a + // viewer who opens the menu and picks a second language before the + // first extraction has answered has two of these in flight for the + // same film, and the one that arrives first is not necessarily the + // one that was asked for first. + : obj.type === 'subtitle_req' ? `subtitle:${obj.file_id}:${obj.track}` // Two-step admin-op flow (_authorizeAdminOp) — see ADMIN_OP_TYPES' // own comment for the race this closes. The initial request and // the admin_response that follows it are keyed the same way @@ -3399,6 +3424,18 @@ class MeshBayTransport { return; } + // Keyed on file *and* track — see the `subtitle_req` key above. The node + // echoes `track` back for exactly this: without it a reply could only be + // matched to the film, and the two tracks of one film are precisely the + // pair that can be in flight together. + if (msg.type === 'subtitle_resp') { + const key = `subtitle:${msg.file_id}:${msg.track}`; + for (const [, handler] of this._pending) { + if (handler._key === key) { handler.resolve(msg); return; } + } + return; + } + // Same reasoning as media_meta_resp: keyed, not arrival-order, and // "nobody's waiting any more" must not fall through either. if (msg.type === 'season_meta_resp') { diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js index 23b0699..5725fc4 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js @@ -3,7 +3,7 @@ import { } from './vendor/htm-preact.js'; import { t, getLocale } from './i18n.js'; import { Icon } from './icon.js'; -import { formatSize } from './file-utils.js'; +import { formatSize, CHUNK_SIZE, pipelinedDownload } from './file-utils.js'; import { loadAuth } from './hub-client.js'; import * as platform from './platform.js'; @@ -68,8 +68,8 @@ const _ISO639 = { * otherwise indistinguishable in the menu — which is common, since a stereo * downmix usually sits beside the surround track it came from. */ -function audioTrackLabel(track) { - const code = (track.lang || '').toLowerCase(); +function _languageName(lang) { + const code = (lang || '').toLowerCase(); let name = null; const iso = _ISO639[code] || (code.length === 2 ? code : null); if (iso) { @@ -84,6 +84,11 @@ function audioTrackLabel(track) { } catch { /* no Intl.DisplayNames, or a code it does not know */ } } if (!name && code && code !== 'und') name = code; + return name; +} + +function audioTrackLabel(track) { + let name = _languageName(track.lang); if (!name) name = t('video.audio_track_n', { n: track.i + 1 }); // Two tracks in the same language are one menu entry repeated without // this, and a library where a stereo downmix sits beside the surround @@ -96,6 +101,21 @@ function audioTrackLabel(track) { return detail ? `${name} — ${detail}` : name; } +/** + * What to call one subtitle track. + * + * Same shape as `audioTrackLabel`, minus the channel layout, which subtitles + * have no equivalent of. The container's title still wins where there is one: + * "Forced", "SDH" and "Signs & Songs" are all the same language tag as the + * ordinary track they sit beside, and picking the wrong one of those is the + * difference between a full translation and three lines in a whole film. + */ +function subtitleTrackLabel(track) { + const name = _languageName(track.lang) + || t('video.subtitle_track_n', { n: track.i + 1 }); + return track.title ? `${name} — ${track.title}` : name; +} + function _mseSupported(codec) { if (!window.MediaSource) return false; const mime = `video/mp4; codecs="${codec}"`; @@ -218,6 +238,25 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { // Read inside the effect's closures, which are built once and would // otherwise capture the first track forever. const audioTrackRef = useRef(null); + // Subtitles. The node lists only the tracks it can turn into WebVTT, so an + // empty list means "nothing showable here" whatever the container holds, + // and draws no selector — the same discovery-from-the-answer shape as the + // audio tracks above. `null` is off, and off is where a film opens. + // + // None of this is torn down by a seek or a language change: the extraction + // is whole-file, so the cues are absolute and the outlives every + // restart of the MediaSource underneath it. + const [subtitleTracks, setSubtitleTracks] = useState([]); + const [subtitleTrack, setSubtitleTrack] = useState(null); + const [subtitleMenuOpen, setSubtitleMenuOpen] = useState(false); + const [subtitleUrl, setSubtitleUrl] = useState(null); + const [subtitleBusy, setSubtitleBusy] = useState(false); + const [subtitleError, setSubtitleError] = useState(false); + const subtitleUrlRef = useRef(null); + // Two extractions can be in flight when the viewer changes their mind, and + // the first one asked for is not necessarily the first one answered. Only + // the newest request may install its blob. + const subtitleGenRef = useRef(0); const [castActive, setCastActive] = useState(false); const [castUrl, setCastUrl] = useState(null); const [castPickerOpen, setCastPickerOpen] = useState(false); @@ -629,6 +668,12 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { // was drawn falls back to the first track, and the selector must show // the truth rather than the request. setAudioTracks(Array.isArray(msg.audio_tracks) ? msg.audio_tracks : []); + // Re-stated on every stream_init, including the ones a seek and an + // audio-language change produce. Deliberately does not touch + // `subtitleTrack` or the blob: the cues are absolute, so the track + // showing before the restart is still the right one after it. + setSubtitleTracks( + Array.isArray(msg.subtitle_tracks) ? msg.subtitle_tracks : []); if (Number.isInteger(msg.audio_track)) { audioTrackRef.current = msg.audio_track; setAudioTrack(msg.audio_track); @@ -1025,6 +1070,80 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { return () => window.removeEventListener('keydown', onKey); }, [onClose]); + /** + * Show one subtitle track, or none. + * + * The node extracts the whole track to WebVTT and caches it under its own + * hash; what comes back here is that hash, pulled through the ordinary + * chunk path like any other file. So this is slow exactly once per film per + * track, and instant every time after — including in a later sitting, which + * is the part a per-seek extraction could never have given. + * + * A failure here never touches playback. Subtitles are an addition to a + * film that is already running, and taking the film down because a text + * track could not be read would be a worse answer than no subtitles. + */ + const selectSubtitle = useCallback(async (track) => { + const gen = ++subtitleGenRef.current; + if (subtitleUrlRef.current) { + URL.revokeObjectURL(subtitleUrlRef.current); + subtitleUrlRef.current = null; + } + setSubtitleUrl(null); + setSubtitleError(false); + if (track === null) { + setSubtitleTrack(null); + setSubtitleBusy(false); + return; + } + const transport = transportRef.current; + if (!transport) return; + setSubtitleTrack(track.i); + setSubtitleBusy(true); + try { + const info = await transport.requestSubtitle(entry.id, track.i); + const chunks = await pipelinedDownload( + transport, gekRef.current, info.hash, Math.ceil(info.size / CHUNK_SIZE)); + const url = URL.createObjectURL( + new Blob(chunks, { type: info.mime || 'text/vtt' })); + // Someone changed their mind while this was in flight. Dropping the blob + // rather than installing it is the whole point of the generation: the + // reply that arrives last is not the choice that was made last. + if (subtitleGenRef.current !== gen) { URL.revokeObjectURL(url); return; } + subtitleUrlRef.current = url; + setSubtitleUrl(url); + } catch (err) { + if (subtitleGenRef.current !== gen) return; + console.warn('[MeshBay] subtitle track', track.i, 'failed:', err); + setSubtitleTrack(null); + setSubtitleError(true); + } finally { + if (subtitleGenRef.current === gen) setSubtitleBusy(false); + } + }, [entry, transportRef, gekRef]); + + // A added to a media element after it started playing is not shown + // by the `default` attribute — that one is read when the element is first + // parsed, and by then this track did not exist. The mode has to be set on + // the live TextTrack, which only appears once the element has adopted the + // child preact just rendered. + useEffect(() => { + const v = videoRef.current; + if (!v) return; + for (let i = 0; i < v.textTracks.length; i++) { + v.textTracks[i].mode = subtitleUrl ? 'showing' : 'disabled'; + } + }, [subtitleUrl]); + + useEffect(() => { + return () => { + if (subtitleUrlRef.current) { + URL.revokeObjectURL(subtitleUrlRef.current); + subtitleUrlRef.current = null; + } + }; + }, []); + return html`
{ if (e.target.classList.contains('video-overlay')) onClose(); @@ -1065,6 +1184,46 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { `}
`} + ${subtitleTracks.length > 0 && html` +
+ + ${subtitleMenuOpen && html` +
+ + ${subtitleTracks.map((track) => html` + + `)} + ${subtitleError && html` +
+ ${t('video.err_subtitle')} +
+ `} +
+ `} +
+ `} ${platform.capabilities.lanCast && html`