diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-09-17 10:12:31 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-09-17 10:12:31 +0200 |
| commit | a98445ce246509b0d2600df14907f72b448a6d10 (patch) | |
| tree | 023e1c09d8fa42f67f5da8ff32a22f271359d843 | |
| parent | f4fd6db8faa15bf02f38a14b652cc46936f8d6bf (diff) | |
| download | meshbay-a98445ce246509b0d2600df14907f72b448a6d10.tar.gz | |
feat: let the viewer pick the audio track
The streaming path mapped 0:a:0 unconditionally, so a dubbed film played in
whichever language was muxed first and the others were unreachable. The node
now enumerates the tracks in stream_init and honours audio_track in
stream_req; switching is the seek path, since one ffmpeg carries one track.
MNP 3.2, additive: the player draws its selector from the node's own list and
never from a version number, so an older node is never asked for a track it
would ignore. MNP_MIN_SUPPORTED does not move.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
22 files changed, 860 insertions, 63 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index a079e50..e7d4a76 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1102,19 +1102,30 @@ client on reload; the desktop client ships its own UI, which is why connecting and says "this version can no longer connect" rather than showing a handshake refusal nobody can act on. -**The floor being the current version is what keeps capability branches out of the -client.** `MNP_MIN_SUPPORTED` equals `MNP_VERSION`, so `check_version` refuses -every older peer at the handshake — which means **every capability is true of every -peer the client can reach**, and there is nothing to test for. An upload is sealed -or it is not sent; a transfer has a real lease or it does not run; there is one -app-directories op and no wrappers behind it. The client records the version its -peer declared, for diagnostics, and **branches on none of it**. +**Every *requirement* is true of every peer the client can reach.** The floor moves +with each MAJOR, so `check_version` refuses at the handshake any peer that cannot +meet one: an upload is sealed or it is not sent; a transfer has a real lease or it +does not run; there is one app-directories op and no wrappers behind it. The client +records the version its peer declared, for diagnostics, and **branches on none of +it**. > A capability flag on a peer whose floor already guarantees the capability is a > branch that can only ever take one path — until somebody lowers the floor, at > which point it silently takes the other. **A field kept "just in case" is how > the branches come back.** +**The floor is not the current version, and MINOR additions are why.** It is +`MNP_MIN_SUPPORTED` in `handshake.py`, it equals the last MAJOR, and 3.1 and 3.2 +have both been added above it without moving it. So a peer can be reachable and +still not do something the current version can, and the client has to cope with +that — **by reading the peer's own answer, never by comparing version numbers**. +3.2's audio tracks are the worked example: the node lists them in `stream_init`, +the client draws its selector from that list, and a node that sends no list gets no +selector. That is not the branch the box above refuses. The branch it refuses is a +flag the client sets from a version it parsed; this is the node stating what it +has, in the same message the feature already needed, and it takes exactly one path +per peer because the peer said which. + Where a break leaves data behind, a migration runs with the node stopped, backs the database up first and is idempotent. But **a migration that has to be remembered is a migration that does not happen**, so anything that *can* be a @@ -2020,6 +2031,14 @@ MSE string) fed to a source buffer, with the node holding one slot per viewer. - **Seeking restarts the source with an index seek before the input**, clamped away from the end and echoed back; the client supplies the timestamp offset, because copying timestamps does not preserve position. +- **The viewer picks the audio track, and picking one is a seek.** One ffmpeg + carries one audio track, so there is nothing to switch inside a running stream: + the node is asked again at the current position and the source buffer is reset + the way any seek resets it. It costs nothing extra — audio is transcoded on every + stream anyway — and mapping the first track unconditionally, which is what this + replaced, made a dubbed library playable in one language only. The node reports + the tracks it found and the one it used; **the client draws its selector from + that list and from no version number**, which is what keeps the addition MINOR. - **Losing a peer must stop its work, not merely forget it** — anything holding a resource is shut down on the way out, or a closed tab transcodes for the length of the credit timeout. diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index 2682820..19cced9 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1,6 +1,6 @@ # MeshBay Node Protocol (MNP) -**Wire version:** `3.1` — `meshbay_common/__init__.py` (`MNP_VERSION`) +**Wire version:** `3.2` — `meshbay_common/__init__.py` (`MNP_VERSION`) **Oldest peer accepted:** `3.0` — `handshake.py` (`MNP_MIN_SUPPORTED`) **Normative implementation:** `meshbay-common` (`protocol.py`, `handshake.py`, `groupbox.py`, `chatbox.py`, `adminop.py`, `join.py`, `device.py`, `crypto.py`, @@ -1627,19 +1627,23 @@ array while MediaSource consumes it a segment at a time. ``` C N - |-- stream_req {v, file_id, start, credits} -------->| + |-- stream_req {v, file_id, start, credits, | + | audio_track?} -------------------------------->| | | retire this session's previous | | stream (a second request | | means the first is over) | | acquire a transcode slot (8) - | | ffprobe: codec, duration, audio + | | ffprobe: codec, duration, the + | | audio tracks | | spawn ffmpeg | | -ss before -i (index seek) | | video: copy, or libx264 when | | the browser cannot decode - | | audio: always AAC, 2 ch + | | audio: always AAC, 2 ch, + | | -map 0:a:<audio_track> | | frag_keyframe+empty_moov - |<- stream_init {v, file_id, codec, duration, start}-| + |<- stream_init {v, file_id, codec, duration, start, | + | audio_tracks[], audio_track} ------------------| | | | check MediaSource.isTypeSupported(codec) | | | @@ -1664,6 +1668,11 @@ array while MediaSource consumes it a segment at a time. | Concurrent transcodes | 8 node-wide, semaphore on the transport context | | Seeking | a new `stream_req` with `start`; the previous stream is retired first, ffmpeg respawned with `-ss` | | `start` in `stream_init` | the value actually used — seeking lands on the keyframe at or before the request, and the client adds it back as `SourceBuffer.timestampOffset` | +| `audio_tracks` in `stream_init` | every audio track: `i` (the **audio ordinal**, what `-map 0:a:<n>` takes, never the container stream index), `lang`, `title`, `codec`, `ch`. Empty for a file with no audio | +| `audio_track` in `stream_req` | which ordinal to map. Absent, out of range or malformed is the first track | +| `audio_track` in `stream_init` | the ordinal actually used, for the same reason `start` is reported: a list drawn before the file was replaced on disk can name a track that is no longer there, and the client must show what is playing rather than what it asked for. `null` when the file has no audio | +| Changing track | a new `stream_req` at the current position, exactly like a seek — one ffmpeg produces one audio track, so there is nothing to switch inside a running stream | +| Capability discovery | **the list, not the version.** A client draws its selector from `audio_tracks` and sends `audio_track` only when it has one, so a node too old to enumerate is never asked for a track it would ignore and answer in the wrong language | An ffmpeg failure before any output produces `error: Could not stream this file`; stderr stays server-side, where it belongs — it names paths on the operator's disk and @@ -1992,8 +2001,9 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`3.1`**; oldest peer -accepted: **`3.0`** — 3.1 is additive, so the floor does not move with it. +MNP versions independently of the package version. Current: **`3.2`**; oldest peer +accepted: **`3.0`** — 3.1 and 3.2 are both additive, so the floor does not move with +them. The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; `MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about @@ -2009,6 +2019,14 @@ whether an older peer can still do anything useful: chat. A break that touches something every session depends on cannot be confined, and the honest form is to refuse at the handshake: **a stated refusal is a bug report, a feature that quietly does not work is a support case.** +* **Discovery from the answer, not from the version number.** 3.2's audio-track + selection is the shape to copy: the node lists the tracks in `stream_init`, and the + client sends `audio_track` only when it was given a list. A peer that ignores that + field would not degrade — it would serve a different language in silence, which is a + wrong answer and not a missing feature — and what keeps the change additive is that + no client can ever put an old node in that position. This is not the opt-in switch + I10 refuses: there is no second branch on the node, which always enumerates, always + honours the request and always reports the track it used. * **A requirement is breaking even when its messages are additive** (I10). New message types and a new optional field are additive on the wire; *requiring* them is not, and an opt-in switch that enforces the requirement only for peers that speak the new @@ -2183,7 +2201,7 @@ LP(x) = uint32be(len(x)) || x every field, no exceptions | Constant | Value | Source | |---|---|---| -| `MNP_VERSION` | `3.1` | `meshbay_common/__init__.py` | +| `MNP_VERSION` | `3.2` | `meshbay_common/__init__.py` | | `MNP_MIN_SUPPORTED` | `3.0` | `handshake.py` | | `NONCE_LEN` | 32 bytes (both handshake nonces) | `handshake.py` | | `ADMIN_CHALLENGE_TTL` | 120 s | `adminop.py` | diff --git a/packages/meshbay-common/src/meshbay_common/__init__.py b/packages/meshbay-common/src/meshbay_common/__init__.py index 80e6d92..c4c4bf4 100644 --- a/packages/meshbay-common/src/meshbay_common/__init__.py +++ b/packages/meshbay-common/src/meshbay_common/__init__.py @@ -181,5 +181,25 @@ __version__ = "0.14.0" # next one it reaches and keeps its own copy meanwhile (§6.4). `MNP_MIN_SUPPORTED` # does not move. That is the difference from 3.0, where the requirement — not the # messages — is what made it MAJOR. -MNP_VERSION = "3.1" +# **3.2 (2026-09-17): the viewer picks the audio track.** +# +# `stream_init` carries `audio_tracks` (ordinal, language, title, codec, +# channels) and the `audio_track` actually used; `stream_req` gains an optional +# `audio_track`. Until now the streaming path mapped `0:a:0` unconditionally, +# so a dubbed film played in whichever language was muxed first — across a real +# library that is one language, and the others were unreachable. +# +# **Additive, and MINOR because nothing is required, but the reason differs +# from 3.1's.** A playlist a node cannot store is a feature a client keeps to +# itself; an `audio_track` a node ignores is the *wrong language*, served +# silently, which is a wrong answer and not a missing one. What makes this +# MINOR anyway is that the client cannot get into that position: the selector +# is drawn from `audio_tracks` in the node's own `stream_init`, so a 3.1 node +# sends no list, the client shows no selector, and no `audio_track` is ever +# sent to a peer that would ignore it. **The capability is discovered from the +# answer, never from the version number** — and that is not the opt-in +# compatibility switch 3.0 refused, because there is no second branch on the +# node: a 3.2 node always enumerates, always honours what it is asked for, and +# always says which track it used. `MNP_MIN_SUPPORTED` does not move. +MNP_VERSION = "3.2" MHP_VERSION = "0.1" diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js index cb92b18..806fa38 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js @@ -249,6 +249,8 @@ export default { 'video.buffering': 'Wird gepuffert …', 'video.resumed_at': "Fortgesetzt bei {time}", 'video.from_start': "Von vorn beginnen", + 'video.audio_track': 'Tonspur', + 'video.audio_track_n': 'Spur {n}', 'video.close': 'Schließen (Esc)', 'preview.pdf_fallback': 'Dieser Browser zeigt das PDF nicht direkt an. Laden Sie es ' + 'stattdessen herunter — entschlüsselt wurde es ohnehin hier.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js index 3b84c3c..92114c9 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js @@ -248,6 +248,8 @@ export default { 'video.buffering': 'Buffering...', 'video.resumed_at': "Resumed at {time}", 'video.from_start': "Start from the beginning", + 'video.audio_track': 'Audio track', + 'video.audio_track_n': 'Track {n}', 'video.close': 'Close (Esc)', 'preview.pdf_fallback': 'This browser will not display the PDF inline. Download it instead — it was decrypted here either way.', 'preview.too_large': 'This file is {size}, more than this page can hold in memory ({limit}). Download it instead — a download is written straight to disk.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js index 145f464..d9b8f63 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js @@ -247,6 +247,8 @@ export default { 'video.buffering': 'Almacenando en búfer...', 'video.resumed_at': "Reanudado en {time}", 'video.from_start': "Empezar desde el principio", + 'video.audio_track': 'Pista de audio', + 'video.audio_track_n': 'Pista {n}', 'video.close': 'Cerrar (Esc)', 'preview.pdf_fallback': 'Este navegador no mostrará el PDF integrado. Descárguelo ' + 'en su lugar — en cualquier caso se descifró aquí.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js index a4f32de..d507d28 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js @@ -248,6 +248,8 @@ export default { 'video.buffering': 'Mise en mémoire tampon...', 'video.resumed_at': "Reprise à {time}", 'video.from_start': "Reprendre depuis le début", + 'video.audio_track': 'Piste audio', + 'video.audio_track_n': 'Piste {n}', 'video.close': 'Fermer (Échap)', 'preview.pdf_fallback': 'Ce navigateur n’affichera pas le PDF directement. ' + 'Téléchargez-le plutôt — il a été déchiffré ici dans les deux cas.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js index 1da0793..b6c68dd 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js @@ -248,6 +248,8 @@ export default { 'video.buffering': 'Buffering in corso...', 'video.resumed_at': "Ripreso da {time}", 'video.from_start': "Riparti dall'inizio", + 'video.audio_track': 'Traccia audio', + 'video.audio_track_n': 'Traccia {n}', 'video.close': 'Chiudi (Esc)', 'preview.pdf_fallback': 'Questo browser non mostrerà il PDF nella pagina. Lo scarichi ' + 'invece — in ogni caso è stato decifrato qui.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js index 971ecb2..99e2e1f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js @@ -245,6 +245,8 @@ export default { 'video.buffering': 'バッファリング中…', 'video.resumed_at': "{time} から再開しました", 'video.from_start': "最初から再生する", + 'video.audio_track': '音声トラック', + 'video.audio_track_n': 'トラック {n}', 'video.close': '閉じる（Esc）', 'preview.pdf_fallback': 'このブラウザーはページ内に PDF を表示しません。' + 'ダウンロードしてご覧ください。いずれにせよ復号はここで行われています。', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js index dae63d2..269fa70 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js @@ -249,6 +249,8 @@ export default { 'video.buffering': 'Bezig met bufferen...', 'video.resumed_at': "Hervat op {time}", 'video.from_start': "Vanaf het begin afspelen", + 'video.audio_track': 'Audiospoor', + 'video.audio_track_n': 'Spoor {n}', 'video.close': 'Sluiten (Esc)', 'preview.pdf_fallback': 'Deze browser toont de PDF niet in de pagina zelf. Download ' + 'hem in plaats daarvan — ontsleuteld werd hij hoe dan ook hier.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js index 639d518..363296c 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js @@ -254,6 +254,8 @@ export default { 'video.buffering': 'Buforowanie...', 'video.resumed_at': "Wznowiono od {time}", 'video.from_start': "Odtwórz od początku", + 'video.audio_track': 'Ścieżka dźwiękowa', + 'video.audio_track_n': 'Ścieżka {n}', 'video.close': 'Zamknij (Esc)', 'preview.pdf_fallback': 'Ta przeglądarka nie wyświetli pliku PDF na stronie. Proszę ' + 'go pobrać — i tak został odszyfrowany tutaj.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js index 371b84f..2a90956 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js @@ -249,6 +249,8 @@ export default { 'video.buffering': 'Armazenando em buffer...', 'video.resumed_at': "Retomado em {time}", 'video.from_start': "Começar do início", + 'video.audio_track': 'Faixa de áudio', + 'video.audio_track_n': 'Faixa {n}', 'video.close': 'Fechar (Esc)', 'preview.pdf_fallback': 'Este navegador não exibirá o PDF na própria página. Baixe ' + 'o arquivo — de todo modo ele foi descriptografado aqui.', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js index 96ba4fe..e3aa440 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js @@ -243,6 +243,8 @@ export default { 'video.buffering': '正在缓冲…', 'video.resumed_at': "已从 {time} 继续播放", 'video.from_start': "从头开始播放", + 'video.audio_track': '音轨', + 'video.audio_track_n': '音轨 {n}', 'video.close': '关闭（Esc）', 'preview.pdf_fallback': '此浏览器不会在页面内显示该 PDF。请改为下载——无论如何它都已在本地解密。', 'preview.too_large': '该文件为 {size}，超出本页面可在内存中保存的上限（{limit}）。请改为下载——下载会直接写入磁盘。', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/transport.js b/packages/meshbay-hub/src/meshbay_hub/static/transport.js index a6b3f52..262b00e 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/transport.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/transport.js @@ -2264,12 +2264,24 @@ class MeshBayTransport { * MediaSource consumes a segment at a time — which is fine for a clip and * fatal for anything worth streaming. */ - requestStream(fileId, credits = STREAM_CREDITS, start = 0) { + requestStream(fileId, credits = STREAM_CREDITS, start = 0, audioTrack = null) { // `start` is a seek: the node retires whatever this session was streaming // and spawns ffmpeg again from there. Omitted or zero is the film's // beginning, which is what an 0.1 node understands. - console.log('[stream] sending stream_req start:', start, 'credits:', credits); - this._send({ type: 'stream_req', v: '0.1', file_id: fileId, credits, start }); + // + // `audioTrack` is omitted entirely unless the caller has one, and the + // caller only has one because a `stream_init` listed the tracks. A node + // too old to enumerate them is therefore never sent a field it would + // ignore — which matters more here than it looks: ignoring it would not + // degrade the stream, it would serve a different language without saying + // so. + const req = { type: 'stream_req', v: '0.1', file_id: fileId, credits, start }; + if (Number.isInteger(audioTrack) && audioTrack >= 0) { + req.audio_track = audioTrack; + } + console.log('[stream] sending stream_req start:', start, 'credits:', credits, + 'audio_track:', req.audio_track ?? '-'); + this._send(req); } /** Room for `n` more segments. */ diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js index 4f8a4e4..4970262 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js @@ -1,7 +1,7 @@ import { html, useState, useEffect, useCallback, useRef, } from './vendor/htm-preact.js'; -import { t } from './i18n.js'; +import { t, getLocale } from './i18n.js'; import { Icon } from './icon.js'; import { formatSize } from './file-utils.js'; import { loadAuth } from './hub-client.js'; @@ -39,6 +39,63 @@ const RESUME_MIN_S = 30; const RESUME_MAX_FRACTION = 0.97; const QUEUE_HIGH_WATER = 12; +// ffprobe reports a container's language tag as ISO 639-2, and in either of +// its two variants for the dozen languages that have both — a bibliographic +// one (fre, ger, dut) and a terminological one (fra, deu, nld), with real +// files in this library using each. `Intl.DisplayNames` wants 639-1, so both +// variants are folded onto the same two-letter code here. Only what a media +// container actually carries is listed; anything unmapped falls through to the +// raw tag, which is more useful than "Unknown". +const _ISO639 = { + ara: 'ar', ben: 'bn', bul: 'bg', cat: 'ca', ces: 'cs', cze: 'cs', + chi: 'zh', dan: 'da', deu: 'de', dut: 'nl', ell: 'el', eng: 'en', + est: 'et', fas: 'fa', fin: 'fi', fra: 'fr', fre: 'fr', ger: 'de', + gle: 'ga', gre: 'el', heb: 'he', hin: 'hi', hrv: 'hr', hun: 'hu', + ice: 'is', ind: 'id', isl: 'is', ita: 'it', jpn: 'ja', kor: 'ko', + lav: 'lv', lit: 'lt', may: 'ms', msa: 'ms', nld: 'nl', nor: 'no', + per: 'fa', pol: 'pl', por: 'pt', ron: 'ro', rum: 'ro', rus: 'ru', + slk: 'sk', slo: 'sk', slv: 'sl', spa: 'es', srp: 'sr', swe: 'sv', + tam: 'ta', tha: 'th', tur: 'tr', ukr: 'uk', urd: 'ur', vie: 'vi', + zho: 'zh', +}; + +/** + * What to call one audio track, in the reader's language. + * + * The container's own `title` tag is preferred when there is one: a muxer that + * bothered to write "VFQ" or "Director's commentary" has said something the + * language code cannot, and two tracks tagged with the same language are + * otherwise indistinguishable in the menu — which is common, since a stereo + * downmix usually sits beside the surround track it came from. + */ +function audioTrackLabel(track) { + const code = (track.lang || '').toLowerCase(); + let name = null; + const iso = _ISO639[code] || (code.length === 2 ? code : null); + if (iso) { + try { + name = new Intl.DisplayNames([getLocale()], { type: 'language' }).of(iso); + // `Intl.DisplayNames` follows each locale's prose convention, which is + // lower case in French, Spanish and Italian among others. A menu entry + // is not prose, and "français" beside "AC3 5.1" reads like a bug. Only + // this branch needs it: a raw tag is a code and is shown as written, + // and the numbered fallback comes from the catalogues already cased. + if (name) name = name.charAt(0).toUpperCase() + name.slice(1); + } catch { /* no Intl.DisplayNames, or a code it does not know */ } + } + if (!name && code && code !== 'und') name = code; + if (!name) name = t('video.audio_track_n', { n: track.i + 1 }); + // Two tracks in the same language are one menu entry repeated without + // this, and a library where a stereo downmix sits beside the surround + // track it came from is the ordinary case. The container's title wins; the + // channel layout is the fallback, written in the "5.1" notation that needs + // no catalogue entry in any of the ten languages. + let detail = track.title; + if (!detail && track.ch > 2) detail = `${track.ch - 1}.1`; + else if (!detail && track.ch) detail = `${track.ch}.0`; + return detail ? `${name} — ${detail}` : name; +} + function _mseSupported(codec) { if (!window.MediaSource) return false; const mime = `video/mp4; codecs="${codec}"`; @@ -151,6 +208,16 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { // live; the render needs to reach it for "start from the beginning". const requestSeekRef = useRef(null); const [resumedFrom, setResumedFrom] = useState(0); + // The audio tracks this node reported for this file, and which one is + // playing. An empty list means either a file with one track or a node too + // old to enumerate them — both draw no selector, which is why nothing here + // needs to know which of the two it is. + const [audioTracks, setAudioTracks] = useState([]); + const [audioTrack, setAudioTrack] = useState(0); + const [audioMenuOpen, setAudioMenuOpen] = useState(false); + // Read inside the effect's closures, which are built once and would + // otherwise capture the first track forever. + const audioTrackRef = useRef(null); const [castActive, setCastActive] = useState(false); const [castUrl, setCastUrl] = useState(null); const [castPickerOpen, setCastPickerOpen] = useState(false); @@ -412,7 +479,7 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { outstandingRef.current = STREAM_WINDOW; setPhase('loading'); console.log('[seek] request', +target.toFixed(1), 'outstanding:', STREAM_WINDOW); - t.requestStream(entry.id, STREAM_WINDOW, target); + t.requestStream(entry.id, STREAM_WINDOW, target, audioTrackRef.current); }, SEEK_DEBOUNCE_MS); }; requestSeekRef.current = requestSeek; @@ -557,6 +624,16 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { const mime = `video/mp4; codecs="${msg.codec}"`; castCodecRef.current = msg.codec; + // What the node offered, and what it actually used — which is not + // always what was asked for: a file replaced on disk since the list + // was drawn falls back to the first track, and the selector must show + // the truth rather than the request. + setAudioTracks(Array.isArray(msg.audio_tracks) ? msg.audio_tracks : []); + if (Number.isInteger(msg.audio_track)) { + audioTrackRef.current = msg.audio_track; + setAudioTrack(msg.audio_track); + } + if (!window.MediaSource || !MediaSource.isTypeSupported(mime)) { setError(t('video.err_mse', { codec: msg.codec })); setPhase('error'); @@ -769,7 +846,7 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { outstandingRef.current = STREAM_WINDOW; const resumeAt = readResumePosition(entry.id); if (resumeAt) setResumedFrom(resumeAt); - transport.requestStream(entry.id, STREAM_WINDOW, resumeAt); + transport.requestStream(entry.id, STREAM_WINDOW, resumeAt, audioTrackRef.current); }; // Closing the tab, or backgrounding it on a phone, never runs a React @@ -954,6 +1031,40 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { }}> <div class="video-top-bar"> <span class="video-title">${entry.name} (${formatSize(entry.size)})</span> + ${audioTracks.length > 1 && html` + <div class="cast-wrapper" style="position:relative"> + <button class="video-close ${audioMenuOpen ? 'cast-active' : ''}" + onClick=${() => setAudioMenuOpen(!audioMenuOpen)} + title="${t('video.audio_track')}"> + <${Icon} name="volume" /></button> + ${audioMenuOpen && html` + <div class="cast-picker"> + ${audioTracks.map((track) => html` + <button class="cast-picker-item" onClick=${() => { + setAudioMenuOpen(false); + if (track.i === audioTrackRef.current) return; + // A different track is a different ffmpeg, so this is the + // seek path verbatim — and it has to be, because the new + // stream opens with an init segment the SourceBuffer can + // only accept after the abort/remove that `reinitAt` does. + // Resuming where the film already was is the whole point: + // the viewer changed language, not position. + audioTrackRef.current = track.i; + setAudioTrack(track.i); + const v = videoRef.current; + const seek = requestSeekRef.current; + if (v && seek) seek(v.currentTime); + }}> + ${track.i === audioTrack + ? html`<${Icon} name="check" />` + : html`<span style="display:inline-block;width:14px"></span>`} + ${' '}${audioTrackLabel(track)} + </button> + `)} + </div> + `} + </div> + `} ${platform.capabilities.lanCast && html` <div class="cast-wrapper" style="position:relative"> <button class="video-close ${castActive ? 'cast-active' : ''}" diff --git a/packages/meshbay-hub/tests/test_video_audio_track.py b/packages/meshbay-hub/tests/test_video_audio_track.py new file mode 100644 index 0000000..534da6d --- /dev/null +++ b/packages/meshbay-hub/tests/test_video_audio_track.py @@ -0,0 +1,213 @@ +""" +Choosing the audio track from the player. + +A dubbed film carries several audio tracks and the node used to map the first +one unconditionally, so it played in whichever language was muxed first. Three +things have to hold on this side: + +**The selector exists only where the node said there was something to choose.** +It is drawn from the `audio_tracks` list in `stream_init` and from nothing else +— there is no version check in the player — so a node too old to enumerate them +draws no selector and is never sent an `audio_track` it would ignore. Ignoring +it would not degrade the stream; it would serve a different language in +silence, which is the failure this shape exists to make impossible. + +**Changing track is the seek path.** One ffmpeg produces one audio track, so +there is nothing to switch inside a running stream: the node has to be asked +again, and the new stream opens with an init segment the SourceBuffer can only +accept after `reinitAt`'s abort/remove. + +**The label is derived, not translated.** ffprobe reports ISO 639-2 in either +of its two variants, and the ten catalogues have no room for a language list — +`Intl.DisplayNames` does that, from a fold that is real logic and is run here +rather than read. +""" + +import json +import shutil +import subprocess +from pathlib import Path + +import pytest + +STATIC = Path(__file__).resolve().parents[1] / "src" / "meshbay_hub" / "static" +APP = STATIC / "video-player.js" +TRANSPORT = STATIC / "transport.js" + +pytestmark = pytest.mark.skipif( + shutil.which("node") is None or not APP.exists(), + reason="node or the SPA sources are not available") + + +@pytest.fixture(scope="module") +def app(): + return APP.read_text() + + +def _player(app: str) -> str: + i = app.index("function VideoPlayer(") + nxt = app.find("\nfunction ", i + 1) + return app[i:nxt if nxt > 0 else len(app)] + + +def _lift(app: str, name: str) -> str: + """One top-level function, as text, for node to execute.""" + start = app.index(f"function {name}(") + depth, i, seen = 0, start, False + while i < len(app): + if app[i] == "{": + depth += 1 + seen = True + elif app[i] == "}": + depth -= 1 + if seen and depth == 0: + return app[start:i + 1] + i += 1 + raise AssertionError(f"{name} never closes") + + +# ── The label, run rather than read ─────────────────────────────────────────── + +def _label_cases(tmp_path, app, cases, locale="en"): + script = tmp_path / "label.mjs" + src = "\n".join([ + app[app.index("const _ISO639 = {"):app.index("};", app.index("const _ISO639 = {")) + 2], + _lift(app, "audioTrackLabel"), + ]) + script.write_text( + # The environment is modelled; the function under test is the shipped + # text above. `t` is only reached for a track with no usable language. + f"const getLocale = () => '{locale}';\n" + "const t = (k, p) => `${k}:${p.n}`;\n" + + src + + "\nconst out = JSON.parse(process.argv[2]).map(audioTrackLabel);\n" + "console.log(JSON.stringify(out));\n") + proc = subprocess.run( + ["node", str(script), json.dumps(cases)], + capture_output=True, text=True) + assert proc.returncode == 0, proc.stderr + return json.loads(proc.stdout) + + +def test_both_iso_639_2_variants_name_the_same_language(tmp_path, app): + """Real files in one library use each: `fre` and `fra` are both French, + `ger` and `deu` both German. A fold that covers only one variant leaves + half a collection labelled with a three-letter code.""" + out = _label_cases(tmp_path, app, [ + {"i": 0, "lang": "fre", "ch": 2}, + {"i": 1, "lang": "fra", "ch": 2}, + {"i": 2, "lang": "ger", "ch": 2}, + {"i": 3, "lang": "deu", "ch": 2}, + ]) + assert out[0] == out[1], f"fre and fra must agree: {out}" + assert out[2] == out[3], f"ger and deu must agree: {out}" + assert "French" in out[0] and "German" in out[2], out + + +def test_the_container_title_wins_over_the_language_name(tmp_path, app): + """Two tracks in one language are the same menu entry twice without it.""" + out = _label_cases(tmp_path, app, [ + {"i": 0, "lang": "fre", "title": "Surround", "ch": 6}, + {"i": 1, "lang": "fre", "title": "Stereo", "ch": 2}, + ]) + assert out[0] != out[1] + assert "Surround" in out[0] and "Stereo" in out[1] + + +def test_an_untitled_track_falls_back_to_its_channel_layout(tmp_path, app): + """Still two distinguishable entries, and "5.1" needs no catalogue entry.""" + out = _label_cases(tmp_path, app, [ + {"i": 0, "lang": "fre", "ch": 6}, + {"i": 1, "lang": "fre", "ch": 2}, + ]) + assert out[0] != out[1], f"two French tracks must not read alike: {out}" + assert "5.1" in out[0] and "2.0" in out[1], out + + +def test_an_untagged_track_is_numbered_not_called_unknown(tmp_path, app): + """`und`, or no tag at all — common enough in real files, and a number the + viewer can act on beats a word that tells them nothing.""" + out = _label_cases(tmp_path, app, [ + {"i": 0, "lang": "und", "ch": 2}, + {"i": 1, "ch": 2}, + ]) + assert out[0].startswith("video.audio_track_n:1"), out + assert out[1].startswith("video.audio_track_n:2"), out + + +def test_a_language_the_fold_does_not_know_shows_its_tag(tmp_path, app): + """More useful than "Unknown": the tag is what the file actually says.""" + out = _label_cases(tmp_path, app, [{"i": 0, "lang": "qaa", "ch": 2}]) + assert "qaa" in out[0], out + + +def test_a_language_name_is_cased_for_a_menu_not_for_prose(tmp_path, app): + """`Intl.DisplayNames` returns "français", which is right in a sentence and + wrong in a menu beside "AC3 5.1". The locales where this bites are the ones + this project ships most of.""" + out = _label_cases(tmp_path, app, [ + {"i": 0, "lang": "fre", "ch": 6}, + {"i": 1, "lang": "eng", "ch": 6}, + ], locale="fr") + assert out == ["Français — 5.1", "Anglais — 5.1"], out + + +# ── The capability gate ─────────────────────────────────────────────────────── + +def test_the_selector_is_drawn_from_the_nodes_list_and_no_version(app): + """The whole negotiation. A version comparison anywhere near this is the + bug: the floor is below the current version, so an older node is reachable, + and the only honest signal that it can switch track is that it enumerated.""" + player = _player(app) + assert "audioTracks.length > 1" in player, \ + "the selector must be gated on the list the node sent" + menu = player[player.index("audioTracks.length > 1"):] + menu = menu[:menu.index("platform.capabilities.lanCast")] + for forbidden in ("MNP_VERSION", "peerVersion", "3.2", "v_min"): + assert forbidden not in menu, ( + f"the selector branches on {forbidden}, not on what the node said") + + +def test_no_track_is_ever_sent_to_a_node_that_did_not_offer_one(): + """`requestStream` must omit the field rather than default it. + + A node that ignores `audio_track` does not degrade — it serves a different + language and says nothing. Sending 0 "harmlessly" to every node is what + makes that reachable, so the field has to be absent unless the caller was + given a real one. + """ + src = TRANSPORT.read_text() + fn = src[src.index("requestStream(fileId"):] + fn = fn[:fn.index("\n /** Room for")] + assert "Number.isInteger(audioTrack)" in fn, \ + "an omitted track must not become 0 on the wire" + assert "req.audio_track = audioTrack" in fn + assert "audio_track: audioTrack" not in fn, \ + "the field must be added conditionally, never built into the literal" + + +def test_changing_track_restarts_the_stream_where_the_film_already_was(app): + """Setting the ref alone changes nothing: one ffmpeg carries one audio + track, so the node has to be asked again — and at the current position, + because the viewer changed language and not place.""" + player = _player(app) + handler = player[player.index("audioTracks.length > 1"):] + handler = handler[:handler.index("platform.capabilities.lanCast")] + assert "audioTrackRef.current = track.i" in handler + assert "requestSeekRef.current" in handler, \ + "a track change must go through the seek path" + assert "seek(v.currentTime)" in handler, \ + "a track change must resume where the film already was" + + +def test_the_player_believes_the_node_about_which_track_is_playing(app): + """`stream_init` reports the track actually used, which is not always the + one asked for — a list drawn before the file was replaced on disk can name + a track that is gone, and the node falls back to the first. Showing the + request instead of the answer would tick the wrong entry for the rest of + the film.""" + player = _player(app) + assert "Number.isInteger(msg.audio_track)" in player + init = player[player.index("Number.isInteger(msg.audio_track)"):] + init = init[:init.index("}")] + assert "audioTrackRef.current = msg.audio_track" in init diff --git a/packages/meshbay-node/src/meshbay_node/indexer/enrich.py b/packages/meshbay-node/src/meshbay_node/indexer/enrich.py index 19ab9ce..6eeb1ef 100644 --- a/packages/meshbay-node/src/meshbay_node/indexer/enrich.py +++ b/packages/meshbay-node/src/meshbay_node/indexer/enrich.py @@ -238,11 +238,12 @@ class Enricher: duration = cached_meta["duration"] else: try: - _codec, duration, _has_audio, width, height, _raw = await asyncio.wait_for( + probe = await asyncio.wait_for( probe_video(str(file_path)), timeout=PROBE_TIMEOUT_SECS) + duration = probe.duration fields["duration"] = int(duration) if duration else None - fields["width"] = width - fields["height"] = height + fields["width"] = probe.width + fields["height"] = probe.height except Exception as e: log.warning("Probe failed for %s: %s", file_path, e) await self._media_cache.put_video_meta( diff --git a/packages/meshbay-node/src/meshbay_node/media_probe.py b/packages/meshbay-node/src/meshbay_node/media_probe.py index ad668fa..7858ebe 100644 --- a/packages/meshbay-node/src/meshbay_node/media_probe.py +++ b/packages/meshbay-node/src/meshbay_node/media_probe.py @@ -8,6 +8,7 @@ already imports from) can call it too without a circular import. import asyncio import json +from dataclasses import dataclass, field _H264_PROFILES = {"Baseline": "42", "Main": "4d", "High": "64", "High 10": "6e"} @@ -21,12 +22,46 @@ _H264_PROFILES = {"Baseline": "42", "Main": "4d", "High": "64", "High 10": "6e"} BROWSER_INCOMPATIBLE_VIDEO_CODECS = frozenset({"hevc"}) -async def probe_video( - path: str, -) -> tuple[str | None, float, bool, int | None, int | None, str | None]: +@dataclass(frozen=True) +class AudioTrack: """ - Probe video file with ffprobe, return (MSE codec string, duration, - has_audio, width, height, raw video codec name). + One selectable audio track. + + **`ordinal` is the position among the audio streams, not the container + stream index**, because that is what `-map 0:a:<n>` takes. A file whose + audio sits at container indices 1, 2 and 3 has ordinals 0, 1 and 2, and + mapping `0:a:1` on the container index would silently serve the third + track — the failure this field's name exists to prevent. + """ + ordinal: int + language: str | None + title: str | None + codec_name: str | None + channels: int | None + + +@dataclass +class VideoProbe: + """ + What one ffprobe call says about a video file. + + A dataclass rather than the tuple this used to return: the tuple had six + positional fields, `has_audio` sat third and `raw_codec_name` sixth, and + adding a seventh for the track list would have made every call site a + counting exercise. + """ + codec: str | None + duration: float + has_audio: bool + width: int | None + height: int | None + raw_codec_name: str | None + audio_tracks: list[AudioTrack] = field(default_factory=list) + + +async def probe_video(path: str) -> VideoProbe: + """ + Probe a video file with ffprobe. The audio half of the codec string is always "mp4a.40.2" (AAC-LC) or absent — never the source's real audio codec — because the streaming @@ -51,6 +86,13 @@ async def probe_video( H264. It answered it by refusing until 2026-09-09, which read to the operator as a broken file rather than as an unwired code path. + **Every audio track is reported, not just the first.** The streaming path + transcodes audio unconditionally, so serving the second track costs exactly + what serving the first costs and the choice is the viewer's to make; a + library of dubbed films is one where the first track is a language half the + group does not want. `has_audio` stays as the single question the muxing + decisions ask, and is now `bool(audio_tracks)`. + width/height come from the same ffprobe call (one extra `-show_entries` field, no second process spawn) — resolution is deliberately never guessed from the filename (docs/mediacenter.md §3.5). @@ -58,7 +100,9 @@ async def probe_video( from meshbay_node.platform import ffprobe_cmd proc = await asyncio.create_subprocess_exec( ffprobe_cmd(), "-v", "error", - "-show_entries", "stream=codec_name,profile,level,codec_type,width,height", + "-show_entries", + "stream=codec_name,profile,level,codec_type,width,height,channels", + "-show_entries", "stream_tags=language,title", "-show_entries", "format=duration", "-of", "json", path, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, @@ -69,9 +113,9 @@ async def probe_video( v_codec = "" raw_codec_name: str | None = None - has_audio = False width: int | None = None height: int | None = None + audio_tracks: list[AudioTrack] = [] for s in info.get("streams", []): if s.get("codec_type") == "video" and not v_codec: cn = s.get("codec_name", "") @@ -89,9 +133,26 @@ async def probe_video( width = s.get("width") height = s.get("height") elif s.get("codec_type") == "audio": - has_audio = True + tags = s.get("tags") or {} + audio_tracks.append(AudioTrack( + # Counted here, never read from `s["index"]` — see AudioTrack. + ordinal=len(audio_tracks), + language=(tags.get("language") or "").strip() or None, + title=(tags.get("title") or "").strip() or None, + codec_name=s.get("codec_name") or None, + channels=s.get("channels"), + )) - if not v_codec: - return None, duration, has_audio, width, height, raw_codec_name - codec = f"{v_codec},mp4a.40.2" if has_audio else v_codec - return codec, duration, has_audio, width, height, raw_codec_name + has_audio = bool(audio_tracks) + codec = None + if v_codec: + codec = f"{v_codec},mp4a.40.2" if has_audio else v_codec + return VideoProbe( + codec=codec, + duration=duration, + has_audio=has_audio, + width=width, + height=height, + raw_codec_name=raw_codec_name, + audio_tracks=audio_tracks, + ) diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py index 99ba3c9..b39941d 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py @@ -5948,11 +5948,14 @@ class WebRTCPeerSession: file_hash = bytes.fromhex(entry.id) try: - codec_str, duration, has_audio, _width, _height, raw_video_codec = \ - await _probe_video(str(file_path)) + probe = await _probe_video(str(file_path)) except Exception as e: self._send({"type": "error", "detail": f"Probe failed: {e}"}) return + codec_str = probe.codec + duration = probe.duration + has_audio = probe.has_audio + raw_video_codec = probe.raw_codec_name # No video stream at all is the only thing this path cannot serve, and # it is the only thing refused here. A source with no MSE codec string @@ -6051,8 +6054,23 @@ class WebRTCPeerSession: codec_str = "avc1.640029,mp4a.40.2" if has_audio else "avc1.640029" else: codec_args = ["-c:v", "copy"] + # Which audio track. A dubbed film carries several and the first one is + # not a neutral default — it is whatever the person who muxed the file + # happened to put first, which across a real library is overwhelmingly + # one language. Out of range falls back to the first rather than + # refusing: the client's list comes from a `stream_init` that may + # predate the file being replaced on disk, and a viewer who asked for + # the second track of a file that now has one wants the film, not an + # error. `stream_init` says which track was actually used, the same way + # it says which `start` was actually used and for the same reason. + try: + audio_track = int(msg.get("audio_track", 0) or 0) + except (TypeError, ValueError): + audio_track = 0 + if not 0 <= audio_track < len(probe.audio_tracks): + audio_track = 0 if has_audio: - map_args += ["-map", "0:a:0"] + map_args += ["-map", f"0:a:{audio_track}"] # Downmixed to stereo: a WEB-DL's 5.1 track becomes 6-channel AAC # with no "-ac", which ffprobe and VLC accept fine but which some # browsers' MSE decoder rejects outright once real fragments are @@ -6081,6 +6099,22 @@ class WebRTCPeerSession: # this is what the client adds back (`SourceBuffer.timestampOffset`) # to put the fragments where they belong on the timeline. "start": start, + # The track list is how a client discovers that this node can + # switch language at all — there is no version check anywhere in + # the player. A node that does not send it gets no selector, and + # the client then never sends `audio_track` to a peer that would + # ignore it and serve the wrong language without saying so. + "audio_tracks": [ + { + "i": tr.ordinal, + "lang": tr.language, + "title": tr.title, + "codec": tr.codec_name, + "ch": tr.channels, + } + for tr in probe.audio_tracks + ], + "audio_track": audio_track if has_audio else None, }) # A client that says nothing gets the old behaviour, which is why this @@ -6097,8 +6131,10 @@ class WebRTCPeerSession: self._stream_started_at = time.monotonic() self._stream_segments = 0 reason = "eof" - log.info("stream: stream_init sent file=%s paced=%s credits=%d start=%.1fs", - file_id[:12], paced, self._stream_credit, start) + log.info("stream: stream_init sent file=%s paced=%s credits=%d start=%.1fs " + "audio=%s/%d", + file_id[:12], paced, self._stream_credit, start, + audio_track if has_audio else "-", len(probe.audio_tracks)) try: while True: if paced and not await self._await_stream_credit(): diff --git a/packages/meshbay-node/tests/test_stream_audio_track_selection.py b/packages/meshbay-node/tests/test_stream_audio_track_selection.py new file mode 100644 index 0000000..53781f9 --- /dev/null +++ b/packages/meshbay-node/tests/test_stream_audio_track_selection.py @@ -0,0 +1,285 @@ +""" +The viewer picks which audio track is streamed. + +A dubbed film carries several audio tracks and the streaming path used to map +`0:a:0` unconditionally, so it played in whichever language happened to be +muxed first. Across a real library that is overwhelmingly one language, and the +others could not be reached at all. + +The tracks here are told apart by **amplitude**, not by their tags: each is the +same tone at a different volume, so an assertion about which track was served +is measured from the decoded audio of the reassembled stream and cannot be +satisfied by mapping the wrong one. Tags would only prove that the node copied +a string it was given. + +Like the other streaming tests, these spawn real ffmpeg/ffprobe against small +synthetic files rather than asserting against the source text. +""" + +import re +import shutil +import subprocess +from pathlib import Path + +import pytest +from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey + +from meshbay_common.crypto import generate_gek +from meshbay_common.webcrypto import chunk_key_aes, decrypt_chunk_aes +from meshbay_node.indexer.group_index import GroupIndex +from meshbay_node.transport.webrtc_server import WebRTCPeerSession, _probe_video + +from conftest import needs_subprocess, one_root + +_HAVE_FFMPEG = shutil.which("ffmpeg") and shutil.which("ffprobe") +pytestmark = [ + pytest.mark.asyncio, + pytest.mark.skipif(not _HAVE_FFMPEG, reason="ffmpeg/ffprobe not installed"), + needs_subprocess, +] + +# Each track's tone is attenuated by a different amount, far enough apart that +# an AAC round trip cannot blur one into another. Index in this list is the +# audio ordinal the node is asked for. +_TRACK_GAIN = [1.0, 0.1, 0.01] # 0 dB, -20 dB, -40 dB +_TRACK_LANG = ["fre", "eng", "spa"] +_TRACK_TITLE = ["Surround", "Original", None] + + +def _make_multitrack_clip(path: Path) -> None: + """~1s of H264 video plus three audio tracks at descending volumes. + + The video is muxed first, so the audio streams sit at container indices 1, + 2 and 3 while their audio *ordinals* are 0, 1 and 2 — the gap that + `-map 0:a:<n>` is indexed by and that a probe reading `s["index"]` would + get wrong. + """ + args = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-f", "lavfi", "-i", "testsrc=size=320x240:rate=25:duration=1"] + for gain in _TRACK_GAIN: + args += ["-f", "lavfi", + "-i", f"sine=frequency=440:duration=1:sample_rate=48000," + f"volume={gain}"] + args += ["-map", "0:v:0"] + for i in range(len(_TRACK_GAIN)): + args += ["-map", f"{i + 1}:a:0"] + args += ["-c:v", "libx264", "-preset", "ultrafast", "-c:a", "aac"] + for i, lang in enumerate(_TRACK_LANG): + args += [f"-metadata:s:a:{i}", f"language={lang}"] + if _TRACK_TITLE[i]: + args += [f"-metadata:s:a:{i}", f"title={_TRACK_TITLE[i]}"] + args.append(str(path)) + subprocess.run(args, check=True, capture_output=True) + + +def _session(tmp_path: Path, video_path: Path, gek: bytes): + import blake3 + file_bytes = video_path.read_bytes() + file_id = blake3.blake3(file_bytes).hexdigest() + + sk_node = Ed25519PrivateKey.generate() + index = GroupIndex(group_id="g" * 32, sk_node=sk_node, gek=gek) + from meshbay_common.protocol import IndexEntry + index.add_entry(IndexEntry( + id=file_id, name=video_path.name, path=video_path.parent.name, + size=len(file_bytes), type="video", added_at=0)) + + session = WebRTCPeerSession.__new__(WebRTCPeerSession) + session._ctx = { + "roots": one_root(video_path.parent), + "index": index, + "gek": gek, + "sk_node": sk_node, + "max_concurrent_streams": 4, + } + session._group_id = None + session._user_id = "tester" + session._stream_stopped = False + session._stream_keepalives = 0 + session.sent = [] + session._send = session.sent.append + session._audit = lambda *a, **k: None + return session, file_id + + +def _reassemble(sent: list[dict], gek: bytes, file_id: str) -> bytes: + file_hash = bytes.fromhex(file_id) + segments = sorted( + (m for m in sent if m.get("type") == "stream_data"), + key=lambda m: m["segment_index"]) + out = b"" + for m in segments: + key = chunk_key_aes(gek, file_hash, m["segment_index"]) + out += decrypt_chunk_aes(key, m["nonce"], m["ct"]) + return out + + +def _mean_volume_db(path: Path) -> float: + """What ffmpeg's volumedetect measures in the decoded audio.""" + proc = subprocess.run( + ["ffmpeg", "-hide_banner", "-i", str(path), "-af", "volumedetect", + "-f", "null", "-"], + capture_output=True, text=True) + m = re.search(r"mean_volume:\s*(-?\d+(?:\.\d+)?) dB", proc.stderr) + assert m, f"volumedetect said nothing usable: {proc.stderr[-400:]}" + return float(m.group(1)) + + +async def _stream(tmp_path: Path, clip: Path, msg_extra: dict): + gek = generate_gek() + session, file_id = _session(tmp_path, clip, gek) + await session._stream_video_inner( + {"file_id": file_id, "start": 0, "credits": 0, **msg_extra}) + errors = [m for m in session.sent if m.get("type") == "error"] + assert not errors, f"streaming must not fail: {errors}" + init = next(m for m in session.sent if m.get("type") == "stream_init") + return session, file_id, gek, init + + +async def _streamed_volume(tmp_path: Path, clip: Path, msg_extra: dict, tag: str): + session, file_id, gek, init = await _stream(tmp_path, clip, msg_extra) + out = tmp_path / f"out-{tag}.mp4" + out.write_bytes(_reassemble(session.sent, gek, file_id)) + return _mean_volume_db(out), init + + +async def test_probe_enumerates_every_track_by_ordinal_not_container_index(tmp_path): + """The trap this feature is one wrong line away from. + + `-map 0:a:1` counts audio streams; `s["index"]` counts every stream in the + container. With a video stream muxed first the two never agree, and a probe + reporting container indices would make the client ask for track 1 and be + served track 2 — silently, since both are real audio. + """ + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + probe = await _probe_video(str(clip)) + + assert [tr.ordinal for tr in probe.audio_tracks] == [0, 1, 2] + assert [tr.language for tr in probe.audio_tracks] == _TRACK_LANG + assert probe.has_audio is True + # The container really does disagree, or the assertion above proves nothing. + raw = subprocess.run( + ["ffprobe", "-v", "error", "-select_streams", "a", + "-show_entries", "stream=index", "-of", "csv=p=0", str(clip)], + check=True, capture_output=True, text=True) + assert [int(x) for x in raw.stdout.split()] == [1, 2, 3], ( + "the fixture must put audio at container indices that differ from the " + "ordinals, or it cannot tell the two apart") + + +async def test_probe_reports_the_title_tag_when_the_muxer_wrote_one(tmp_path): + """Two tracks in one language are one menu entry repeated without it.""" + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + probe = await _probe_video(str(clip)) + + assert [tr.title for tr in probe.audio_tracks] == _TRACK_TITLE + assert all(tr.channels == 1 for tr in probe.audio_tracks) + + +async def test_the_requested_audio_track_is_the_one_streamed(tmp_path): + """Measured from the decoded audio, not from a tag the node echoed back.""" + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + first, init_first = await _streamed_volume(tmp_path, clip, {"audio_track": 0}, "0") + third, init_third = await _streamed_volume(tmp_path, clip, {"audio_track": 2}, "2") + + assert init_first["audio_track"] == 0 + assert init_third["audio_track"] == 2 + # -40 dB of attenuation between them; anything under 15 dB of measured + # separation means the same track was served twice. + assert first - third > 15, ( + f"track 0 ({first} dB) and track 2 ({third} dB) decoded to the same " + "loudness, so the requested track was not the one mapped") + + +async def test_no_audio_track_asked_for_still_means_the_first(tmp_path): + """A client that says nothing gets exactly what it got before.""" + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + silent, init_silent = await _streamed_volume(tmp_path, clip, {}, "default") + explicit, _ = await _streamed_volume(tmp_path, clip, {"audio_track": 0}, "explicit") + + assert init_silent["audio_track"] == 0 + assert abs(silent - explicit) < 2 + + +async def test_an_out_of_range_track_falls_back_to_the_first_and_says_so(tmp_path): + """The client's list can predate the file being replaced on disk. + + A viewer who asked for the second track of a file that now has one wants + the film, not an error — and `stream_init` has to report the track actually + used, the same way it reports the `start` actually used. + """ + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + got, init = await _streamed_volume(tmp_path, clip, {"audio_track": 9}, "oob") + expected, _ = await _streamed_volume(tmp_path, clip, {"audio_track": 0}, "base") + + assert init["audio_track"] == 0, "stream_init must not echo the impossible request" + assert abs(got - expected) < 2 + + +async def test_a_malformed_track_number_is_not_an_error(tmp_path): + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + for bad in ("two", None, -1, 1.5): + _, _, _, init = await _stream(tmp_path, clip, {"audio_track": bad}) + assert init["audio_track"] in (0, 1), f"{bad!r} produced {init['audio_track']!r}" + + +async def test_stream_init_lists_the_tracks_for_the_client_to_choose_from(tmp_path): + """The list is the whole capability negotiation. + + The player draws its selector from this and from nothing else — there is no + version check in it — so a node that sends no list gets no selector and is + never sent an `audio_track` it would ignore and answer in the wrong + language. + """ + clip = tmp_path / "clip.mkv" + _make_multitrack_clip(clip) + + _, _, _, init = await _stream(tmp_path, clip, {}) + + assert [tr["i"] for tr in init["audio_tracks"]] == [0, 1, 2] + assert [tr["lang"] for tr in init["audio_tracks"]] == _TRACK_LANG + assert init["audio_tracks"][0]["title"] == "Surround" + assert init["audio_tracks"][2]["title"] is None + + +async def test_a_single_track_file_reports_a_list_of_one(tmp_path): + """Not an empty list: the player decides on length, and a one-entry list is + how it knows there is nothing to choose rather than nothing to report.""" + clip = tmp_path / "mono.mkv" + subprocess.run( + ["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-f", "lavfi", "-i", "testsrc=size=320x240:rate=25:duration=1", + "-f", "lavfi", "-i", "sine=frequency=440:duration=1:sample_rate=48000", + "-c:v", "libx264", "-preset", "ultrafast", "-c:a", "aac", str(clip)], + check=True, capture_output=True) + + _, _, _, init = await _stream(tmp_path, clip, {}) + + assert len(init["audio_tracks"]) == 1 + assert init["audio_track"] == 0 + + +async def test_a_file_with_no_audio_reports_no_track_at_all(tmp_path): + clip = tmp_path / "silent.mkv" + subprocess.run( + ["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-f", "lavfi", "-i", "testsrc=size=320x240:rate=25:duration=1", + "-c:v", "libx264", "-preset", "ultrafast", "-an", str(clip)], + check=True, capture_output=True) + + _, _, _, init = await _stream(tmp_path, clip, {"audio_track": 1}) + + assert init["audio_tracks"] == [] + assert init["audio_track"] is None diff --git a/packages/meshbay-node/tests/test_stream_audio_transcode.py b/packages/meshbay-node/tests/test_stream_audio_transcode.py index 959cc91..92518ba 100644 --- a/packages/meshbay-node/tests/test_stream_audio_transcode.py +++ b/packages/meshbay-node/tests/test_stream_audio_transcode.py @@ -150,15 +150,15 @@ async def test_probe_video_reports_aac_regardless_of_source_audio_codec(tmp_path clip = tmp_path / "clip.mkv" _make_clip(clip, acodec="eac3", channels=6) - codec, duration, has_audio, width, height, raw_codec = await _probe_video(str(clip)) + probe = await _probe_video(str(clip)) - assert has_audio is True - assert duration > 0 - assert codec is not None - assert "eac3" not in codec and "ec-3" not in codec - assert "mp4a.40.2" in codec - assert (width, height) == (320, 240) - assert raw_codec == "h264" + assert probe.has_audio is True + assert probe.duration > 0 + assert probe.codec is not None + assert "eac3" not in probe.codec and "ec-3" not in probe.codec + assert "mp4a.40.2" in probe.codec + assert (probe.width, probe.height) == (320, 240) + assert probe.raw_codec_name == "h264" async def test_probe_video_handles_no_audio_track(tmp_path): @@ -170,10 +170,11 @@ async def test_probe_video_handles_no_audio_track(tmp_path): check=True, capture_output=True, ) - codec, duration, has_audio, width, height, raw_codec = await _probe_video(str(clip)) + probe = await _probe_video(str(clip)) - assert has_audio is False - assert codec is not None and "," not in codec, \ + assert probe.has_audio is False + assert probe.audio_tracks == [] + assert probe.codec is not None and "," not in probe.codec, \ "no audio track must not produce a dangling ',' or a fake audio codec" - assert (width, height) == (320, 240) - assert raw_codec == "h264" + assert (probe.width, probe.height) == (320, 240) + assert probe.raw_codec_name == "h264" diff --git a/packages/meshbay-node/tests/test_stream_video_transcode.py b/packages/meshbay-node/tests/test_stream_video_transcode.py index 69f4c2d..b165af4 100644 --- a/packages/meshbay-node/tests/test_stream_video_transcode.py +++ b/packages/meshbay-node/tests/test_stream_video_transcode.py @@ -191,10 +191,10 @@ async def test_probe_video_reports_raw_codec_name_for_hevc(tmp_path): clip = tmp_path / "clip.mkv" _make_hevc_clip(clip) - codec, duration, has_audio, width, height, raw_codec = await _probe_video(str(clip)) + probe = await _probe_video(str(clip)) - assert raw_codec == "hevc" - assert codec is not None and codec.startswith("hev1.") + assert probe.raw_codec_name == "hevc" + assert probe.codec is not None and probe.codec.startswith("hev1.") @needs_mp3 @@ -206,13 +206,13 @@ async def test_probe_video_reports_no_codec_string_for_mpeg4(tmp_path): clip = tmp_path / "clip.avi" _make_mpeg4_clip(clip) - codec, duration, has_audio, width, height, raw_codec = await _probe_video(str(clip)) + probe = await _probe_video(str(clip)) - assert raw_codec == "mpeg4" - assert codec is None, ( + assert probe.raw_codec_name == "mpeg4" + assert probe.codec is None, ( "a codec string for MPEG-4 Part 2 would be one no browser can act on") - assert has_audio is True - assert (width, height) == (320, 240) + assert probe.has_audio is True + assert (probe.width, probe.height) == (320, 240) @needs_mp3 |