From d22ce0ba5aebfff5ffaea97982472f876af5da22 Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Fri, 18 Sep 2026 14:37:08 +0200 Subject: feat: decode and re-encode video on the GPU where there is one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A 1080p film is decoded by whoever watches it and re-encoded by the node when no browser can decode the source. Both were on the CPU, and on an Atom or Celeron mini-PC neither reaches real time — which is what `transcode_incompatible_video` exists to refuse. This adds the mechanism that makes refusing it unnecessary. Node — `hwaccel.py`: VA-API on Linux, Quick Sync or NVENC on Windows, established by encoding 1080p and reading the file back with ffprobe. Nothing is accepted that does not produce the exact profile and level `stream_init` announces, since the client checks that string before it trusts a byte: an encoder that wrote another level would make the node's own codec string a lie, and ffmpeg takes `-level 4.1` and `-level 41` from h264_qsv without saying which it understood. Three modes per stream — hardware decode and encode, hardware encode alone, libx264 — demoted per source codec, because a GPU that decodes HEVC may have no decoder for MPEG-4 Part 2 and only asking it finds out. A mode that fails is detected on an empty stdout before `stream_init` goes out, so the viewer sees one working stream and never an error. Client — Chromium ships VA-API off on Linux. It is enabled where a render node and a driver are present, then verified through `navigator.mediaCapabilities`: a no moves to the next GL backend on the next launch and an exhausted list drops the switches, so a renamed feature cannot pass for a feature that is on and `--ignore-gpu-blocklist` cannot survive on a machine it did not help. Feature lists now merge rather than overwrite — `appendSwitch` replaces the value, and a second caller would have silently cancelled the mDNS switch aiortc depends on. Packaging — the drivers are weak dependencies on all four formats, so a machine without a GPU installs exactly as before. `dpkg -i` and `rpm -ivh` ignore weak deps; `packaging/README.md` now says so. Windows needs no driver: the bundled ffmpeg already carries h264_qsv and h264_nvenc, and a re-pin that dropped them would cost every low-power Windows node its hardware encoding silently. AMD on Windows (AMF) and macOS (VideoToolbox) are named gaps, not oversights: neither could be tried anywhere in this project, and both re-encode in software as before. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 8 + docs/MESHBAY_DESIGN.md | 20 ++ packages/meshbay-client/src/main.js | 170 +++++++++- packages/meshbay-hub/tests/test_desktop_shell.py | 49 +++ packages/meshbay-node/src/meshbay_node/config.py | 17 + packages/meshbay-node/src/meshbay_node/daemon.py | 6 + packages/meshbay-node/src/meshbay_node/hwaccel.py | 368 +++++++++++++++++++++ .../src/meshbay_node/transport/webrtc_server.py | 89 +++-- packages/meshbay-node/tests/test_hwaccel.py | 171 ++++++++++ .../tests/test_stream_video_transcode.py | 48 +++ packaging/README.md | 30 ++ packaging/deb/meshbay-client/DEBIAN/control | 3 +- packaging/deb/meshbay-node/DEBIAN/control | 6 +- packaging/rpm/meshbay-client.spec | 7 + packaging/rpm/meshbay-node.spec | 6 + packaging/win/README.md | 9 +- packaging/win/fetch-ffmpeg.ps1 | 9 +- 17 files changed, 985 insertions(+), 31 deletions(-) create mode 100644 packages/meshbay-node/src/meshbay_node/hwaccel.py create mode 100644 packages/meshbay-node/tests/test_hwaccel.py diff --git a/CLAUDE.md b/CLAUDE.md index e576326..1f837eb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -185,6 +185,14 @@ These are about working on the tree rather than about the design: installed package where its own migrations are, which is right for a package, a venv and a checkout alike +- **A `DEBIAN/control` file has no comment syntax.** `dpkg-deb --build` refuses + the whole package — `field name '#' must be followed by colon` — so a `#` line + explaining why a dependency is weak turns an installable package into no + package at all. The `.spec` files take comments and the control files do not; + the reason for a `Recommends` goes in `packaging/README.md`. Found by building + a throwaway package, which takes seconds and is the only way to find it: no + test in the tree parses these files + - **`node --check` reports success on a module-syntax error, and a green one cost a full suite run.** An unclosed `.map(` inside a tagged template came back clean four times in a row. `test_spa_syntax.py` says this in its own diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 781438a..8c3f67a 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1926,6 +1926,14 @@ What running it establishes, and what each fact costs: - **Installation places files, never secrets.** No key generation in a package's post-install step or an installer custom action — a golden image would give every machine the same key. +- **Hardware video decoding is asked for and then verified.** Chromium ships VA-API + off on Linux, so the client enables it where a render node and a driver are + present, and then asks `navigator.mediaCapabilities` whether the codec the node + streams really decodes `powerEfficient`ly. A no moves to the next GL backend on + the next launch, and an exhausted list drops the switches — a feature name that a + Chromium release renamed must not pass for a feature that is on, and + `--ignore-gpu-blocklist` must not survive on a machine it did not help. The + package recommends the drivers; nothing requires them. Two guards apply to anything the hub can display **inside** the application, which is a phishing surface: plain text or a very restricted markup subset, never raw @@ -2034,6 +2042,18 @@ by any of this. **Video streaming** is fragmented-MP4 remux (or transcode where the codec has no MSE string) fed to a source buffer, with the node holding one slot per viewer. +- **The re-encode runs on the GPU wherever one works** — VA-API on Linux, Quick + Sync or NVENC on Windows. Which one is established by **encoding 1080p and reading + the file back**, never inferred from the hardware or from ffmpeg's encoder list, + and nothing is accepted that does not produce the exact profile and level + `stream_init` announces: an encoder that wrote another level would make the node's + own codec string a lie, and the client checks that string before it trusts a byte. + Every mode falls back to libx264 — per source codec, because a GPU that decodes + HEVC may have no decoder for MPEG-4 Part 2 and only asking it finds out. This is + what lets a low-power node keep `transcode_incompatible_video` on: the setting + refuses a capability, and hardware encoding is the mechanism that makes refusing + it unnecessary. + - **Flow control is a window, not a debt.** Read-ahead is bounded by *time past the playhead*, with a small window of segments in flight topped up as they land, driven by a clock and by playback and **never by arriving data**. Granting a diff --git a/packages/meshbay-client/src/main.js b/packages/meshbay-client/src/main.js index 122a302..755f1f7 100644 --- a/packages/meshbay-client/src/main.js +++ b/packages/meshbay-client/src/main.js @@ -41,6 +41,19 @@ const { pathToFileURL } = require('node:url'); // runs (`electron .`) consistent with it. app.commandLine.appendSwitch('class', 'MeshBay'); +// Chromium reads `--enable-features` and `--disable-features` as one +// comma-joined list each, and `appendSwitch` **replaces** that list rather +// than adding to it. Two call sites therefore cancel the first silently: no +// error, nothing in a log, just a feature that is quietly not on. The +// hardware-decode section further down is the second caller, so both go +// through this — and so must the third. +function addFeatures(kind, names) { + const current = app.commandLine.getSwitchValue(kind); + const merged = new Set(current ? current.split(',').filter(Boolean) : []); + for (const name of names) merged.add(name); + app.commandLine.appendSwitch(kind, [...merged].join(',')); +} + // Chromium publishes host candidates as random `.local` mDNS names rather // than as the real local IP. Every peer this client talks to is a node running // aiortc/aioice, and aioice has no mDNS resolver on any platform: it logs @@ -56,7 +69,7 @@ app.commandLine.appendSwitch('class', 'MeshBay'); // *client*; the platform that matters is the peer's, not ours, and the peer is // always a node. The trade is that our private address reaches the hub and the // node in the SDP — both the user's own infrastructure, not an arbitrary page. -app.commandLine.appendSwitch('disable-features', 'WebRtcHideLocalIpsWithMdns'); +addFeatures('disable-features', ['WebRtcHideLocalIpsWithMdns']); const UI_DIR = path.join(__dirname, '..', 'ui'); const SCHEME = 'app'; @@ -245,6 +258,159 @@ function writeConfig(next) { let config = readConfig(); +// ── Hardware video decoding (Linux) ───────────────────────────────────────── +// +// Chromium ships VA-API off on Linux; macOS and Windows decode H.264 in +// hardware without being asked, so everything here is Linux-only. +// +// It pays off exactly where it is needed. A 1080p H.264 film is the streaming +// path's normal case — the node hands the bytes over untouched (`-c:v copy`, +// webrtc_server.py) and all the decoding happens here — and every Intel iGPU +// of the last decade decodes that in fixed-function silicon, while an +// Atom/Celeron-class CPU cannot hold 25 fps at 1080p in software. +// +// Three decisions, written down because each replaces something a person +// would otherwise have to do by hand: +// +// - **Detected, not configured.** One package installs on every machine, so +// the switches cannot be chosen at build time and there is no second +// "low-power" package. The probe is two filesystem checks and runs before +// `whenReady`, because Chromium reads its command line exactly once. +// +// - **The feature name is not stable, so it is not trusted.** +// `VaapiVideoDecodeLinuxGL` has been renamed across Chromium releases, and +// `build-client.sh` deliberately rebuilds against the *latest* Electron +// every time — a name pinned here would stop matching one bump later with +// nothing failing to say so. Both known names are passed (Chromium ignores +// an unknown one) and the outcome is measured rather than assumed. +// +// - **The verdict is measured, remembered, and re-opened by a Chromium +// bump.** `measureVideoDecoding` asks the renderer whether the codec the +// node really streams decodes `powerEfficient`ly — Chromium's own answer to +// "is this hardware", which is what reading it out of devtools would have +// told you. A no moves to the next set of switches on the next launch; when +// the list runs out they are dropped altogether, so `--ignore-gpu-blocklist` +// cannot leave a machine with a broken GL stack paying for a feature it +// never got. +// +// `videoAcceleration` in config.json overrides the lot: "off" never applies +// anything, "on" applies it even where the probe finds no driver. + +const VAAPI_FEATURES = ['VaapiVideoDecodeLinuxGL', 'AcceleratedVideoDecodeLinuxGL']; + +// One per launch, in order, until the measurement says hardware. These are not +// a guess about *this* Chromium — they are the ways its GL backend has had to +// be named, and the machine picks which one is true for it. +const VAAPI_ATTEMPTS = [ + [], + [['use-gl', 'egl']], + [['use-gl', 'angle'], ['use-angle', 'gl']], +]; + +function vaapiDriverInstalled() { + const dirs = [process.env.LIBVA_DRIVERS_PATH, '/usr/lib/x86_64-linux-gnu/dri', + '/usr/lib/aarch64-linux-gnu/dri', '/usr/lib64/dri', '/usr/lib/dri']; + for (const dir of dirs) { + if (!dir) continue; + try { + if (fs.readdirSync(dir).some((f) => f.endsWith('_drv_video.so'))) return true; + } catch { /* no such directory on this distribution */ } + } + return false; +} + +// Readable *and* writable: VA-API maps buffers on the render node. On a +// desktop session logind grants that through an ACL rather than through group +// membership, so asking the kernel answers "can this process use it", which is +// the question — rather than "is this user in `render`", which is not. +function renderNode() { + try { + for (const name of fs.readdirSync('/dev/dri')) { + if (!name.startsWith('renderD')) continue; + const dev = `/dev/dri/${name}`; + try { + fs.accessSync(dev, fs.constants.R_OK | fs.constants.W_OK); + return dev; + } catch { /* another GPU may still be usable */ } + } + } catch { /* no /dev/dri at all */ } + return null; +} + +// Which attempt this launch is running, or null when no switches were applied +// — which is also what stops a second window measuring the same launch twice. +let vaapiAttempt = null; + +function configureVideoDecoding() { + if (process.platform !== 'linux') return; + const mode = config.videoAcceleration || 'auto'; + if (mode === 'off') return; + const forced = mode === 'on'; + if (!forced && !(renderNode() && vaapiDriverInstalled())) { + console.log('[video] no usable VA-API driver — decoding in software'); + return; + } + const probe = config.videoDecodeProbe || {}; + const known = probe.chrome === process.versions.chrome; + let attempt = (known && typeof probe.attempt === 'number') ? probe.attempt : 0; + if (attempt >= VAAPI_ATTEMPTS.length) { + if (!forced) { + console.log('[video] VA-API never took on this machine — decoding in software'); + return; + } + // "on" is the person overruling the measurement, so the list running out + // is not an answer here — it keeps the last set rather than climbing an + // index nothing will ever read again. + attempt = VAAPI_ATTEMPTS.length - 1; + } + addFeatures('enable-features', VAAPI_FEATURES); + app.commandLine.appendSwitch('ignore-gpu-blocklist'); + for (const [name, value] of VAAPI_ATTEMPTS[attempt]) + app.commandLine.appendSwitch(name, value); + vaapiAttempt = attempt; +} + +// `avc1.640029` is H.264 High 4.1 — what the node announces for a re-encode +// (webrtc_server.py) and the profile a copied 1080p film carries. Asking about +// the codec that is actually streamed is the point: a machine can decode +// H.264 in hardware and still answer no for HEVC, and the reverse. +const DECODE_PROBE = `navigator.mediaCapabilities.decodingInfo({ + type: 'media-source', + video: { + contentType: 'video/mp4; codecs="avc1.640029"', + width: 1920, height: 1080, bitrate: 5000000, framerate: 25, + }, +}).then((r) => !!r.powerEfficient).catch(() => false)`; + +async function measureVideoDecoding(win) { + const attempt = vaapiAttempt; + vaapiAttempt = null; + if (attempt === null) return; + let hardware; + try { + hardware = await win.webContents.executeJavaScript(DECODE_PROBE); + } catch { + return; // the window went away; the question is asked again next launch + } + const gpu = app.getGPUFeatureStatus().video_decode; + console.log(`[video] hardware decoding: ${hardware} ` + + `(attempt ${attempt}, gpu video_decode: ${gpu})`); + config = { + ...config, + videoDecodeProbe: { + chrome: process.versions.chrome, + attempt: hardware ? attempt : attempt + 1, + hardware, + gpu, + }, + }; + writeConfig(config); + if (!hardware && attempt + 1 < VAAPI_ATTEMPTS.length) + console.log('[video] another GL backend will be tried on the next launch'); +} + +configureVideoDecoding(); + // ── Secrets ───────────────────────────────────────────────────────────────── // // The OS keychain, through safeStorage. What it protects and what it does not @@ -482,6 +648,8 @@ function createWindow() { }); win.once('ready-to-show', () => win.show()); + // Once per launch, and only when the section above applied switches. + win.webContents.once('did-finish-load', () => { measureVideoDecoding(win); }); // Some Wayland compositors (observed under GNOME/Mutter on a VM with a // virtio-gpu device whose command-buffer creation fails) never schedule a // first paint for a surface that isn't mapped yet — but Electron won't map diff --git a/packages/meshbay-hub/tests/test_desktop_shell.py b/packages/meshbay-hub/tests/test_desktop_shell.py index 2e7ea82..5862ac3 100644 --- a/packages/meshbay-hub/tests/test_desktop_shell.py +++ b/packages/meshbay-hub/tests/test_desktop_shell.py @@ -536,3 +536,52 @@ def test_the_tray_is_created_at_launch(): assert bridge < launch, "ensureTray must come after registerBridge" assert launch < window, "the tray is created as part of startup" + + +# ── Chromium's command line ───────────────────────────────────────────────── + +def test_no_feature_list_is_ever_written_directly(): + """ + `--enable-features` and `--disable-features` are one comma-joined list + each, and `appendSwitch` **replaces** the value rather than appending to + it. A second direct call therefore cancels the first with no error and + nothing in a log — the symptom is a feature that is simply not on, which + is how the mDNS switch below and the VA-API switches could silently + cancel each other. Every caller goes through `addFeatures`, which merges. + """ + direct = re.findall(r"appendSwitch\(\s*['\"](?:enable|disable)-features", + _main()) + assert not direct, ( + "a feature list is being written with appendSwitch directly; it " + "overwrites whatever another line already put there — use addFeatures") + + +def test_the_mdns_concealment_is_still_disabled(): + """Chromium's `.local` host candidates are dropped by aiortc, which has no + mDNS resolver: concealing the local IP removes the only LAN-routable + candidate rather than degrading it.""" + assert "addFeatures('disable-features', ['WebRtcHideLocalIpsWithMdns'])" in _main() + + +def test_hardware_video_decoding_is_asked_for_and_then_verified(): + """ + Turning VA-API on is half the job. Chromium renames these features across + releases and the client is rebuilt against the newest Electron every time, + so a name that stops matching must not pass for a feature that is on: the + result is measured through `navigator.mediaCapabilities` and remembered, + and `--ignore-gpu-blocklist` is dropped again on a machine where the + measurement never says hardware. + """ + main = _main() + for feature in ("VaapiVideoDecodeLinuxGL", "AcceleratedVideoDecodeLinuxGL"): + assert feature in main, f"{feature} is no longer requested" + assert "ignore-gpu-blocklist" in main + assert "mediaCapabilities" in main and "powerEfficient" in main, ( + "the switches are applied but nothing checks whether they worked") + + +def test_hardware_decoding_can_be_turned_off_without_a_rebuild(): + """A machine whose GL stack misbehaves in a way the measurement does not + catch needs an answer that is not "reinstall": config.json, the same file + every other client setting lives in.""" + assert "config.videoAcceleration" in _main() diff --git a/packages/meshbay-node/src/meshbay_node/config.py b/packages/meshbay-node/src/meshbay_node/config.py index 7351b1c..086d800 100644 --- a/packages/meshbay-node/src/meshbay_node/config.py +++ b/packages/meshbay-node/src/meshbay_node/config.py @@ -72,6 +72,15 @@ max_concurrent_uploads = 8 # Set to false only if every viewer's client is known to decode HEVC itself. transcode_incompatible_video = true +# Use the GPU for that re-encode when there is one that works — VA-API here, +# Quick Sync or NVENC on Windows. Established by encoding 1080p and checking +# the result, never guessed from the hardware, and ignored where the test +# fails: this is on by default and the only reason to turn it off is a driver +# that misbehaves in a way the test does not catch. On an Intel mini-PC it is +# the difference between transcoding in real time and having to set +# transcode_incompatible_video = false. +hardware_video_encode = true + # ICE candidate gathering. By default, virtual/VPN interfaces (Tailscale, # libvirt, Docker) are auto-excluded — a STUN request that can't reach the # server holds the WebRTC answer for 5 seconds. Set this to restrict @@ -180,6 +189,12 @@ class NodeConfig: # that already decode the source codec directly, since transcoding costs # real CPU per concurrent viewer, unlike the copy path. transcode_incompatible_video: bool = True + # Whether that re-encode may run on the GPU. See hwaccel.py: the capability + # is established by encoding and reading the result back, so `true` here + # means "use it if it works", never "assume it does". Off returns the node + # to libx264 on every stream, which is where it was before hardware + # encoding existed. + hardware_video_encode: bool = True # ICE candidate gathering: which network interfaces to include or exclude. # By default, virtual and VPN interfaces (Tailscale, libvirt, Docker) are # auto-excluded because a STUN request that can't reach the server holds @@ -355,6 +370,8 @@ def load_config(path: Path = DEFAULT_CONFIG_PATH) -> Config: cfg.node.max_concurrent_uploads, "max_concurrent_uploads") cfg.node.transcode_incompatible_video = bool( nd.get("transcode_incompatible_video", cfg.node.transcode_incompatible_video)) + cfg.node.hardware_video_encode = bool( + nd.get("hardware_video_encode", cfg.node.hardware_video_encode)) ice_if = nd.get("ice_interfaces") if isinstance(ice_if, list): cfg.node.ice_interfaces = [str(s) for s in ice_if] diff --git a/packages/meshbay-node/src/meshbay_node/daemon.py b/packages/meshbay-node/src/meshbay_node/daemon.py index 666a5cc..fa9ef81 100644 --- a/packages/meshbay-node/src/meshbay_node/daemon.py +++ b/packages/meshbay-node/src/meshbay_node/daemon.py @@ -510,6 +510,12 @@ class NodeDaemon: denylist = self._denylist # 6. WebRTC transport (browser clients) + # + # The operator's answer about the GPU, before the first stream asks + # the question. The probe itself is lazy — it costs a test encode, + # and a node that never serves a video should never pay it. + from meshbay_node import hwaccel + hwaccel.set_enabled(self._config.node.hardware_video_encode) from meshbay_node.transport.ice_filter import install as install_ice_filter install_ice_filter( self._config.node.ice_interfaces or None, diff --git a/packages/meshbay-node/src/meshbay_node/hwaccel.py b/packages/meshbay-node/src/meshbay_node/hwaccel.py new file mode 100644 index 0000000..051f64c --- /dev/null +++ b/packages/meshbay-node/src/meshbay_node/hwaccel.py @@ -0,0 +1,368 @@ +""" +Hardware H.264 encoding for the streaming re-encode path. + +Streaming copies the video whenever a browser can decode it (`-c:v copy`), and +that path costs nothing. The re-encode path is the expensive one — HEVC, and +the codecs with no MSE string at all (Xvid, MPEG-2, VC-1) — and it is +`libx264 -preset veryfast`, one process per viewer, for the length of a film. +On a laptop that is fine. On an Atom or Celeron mini-PC it does not reach real +time at 1080p, which is why `transcode_incompatible_video` exists as an +operator opt-out: a node that cannot transcode says so instead of serving a +stream that stutters. + +**This module is the other answer to the same problem.** The GPU in every +machine of the last decade encodes H.264 in fixed-function silicon and does not +care how weak the CPU beside it is. Where that hardware is there and works, the +node re-encodes on it and the opt-out is not needed. + +Nothing here is configured by hand, and nothing is inferred from a CPU model or +from a driver's name. **The capability is established by encoding**: a listed +`h264_qsv` proves only that ffmpeg was built with it, and a render node proves +only that a GPU exists — neither says the driver on this machine can do the +work. Measured on the development VM, where `ffmpeg -encoders` lists every +VAAPI encoder and the virtio-gpu driver then fails to initialise: +`libva: virtio_gpu_drv_video.so init failed`. + +**The probe encodes 1080p and then reads the result back with ffprobe**, and +accepts nothing that is not High profile at level 4.1. That is not +belt-and-braces: `stream_init` announces `avc1.640029` and the client checks it +before it trusts a byte, so an encoder that quietly wrote a different level +would make the node a liar. It also means an argument spelled the way one +encoder wants and not another — `-level 4.1` against `-level 41` — is caught +here rather than by a viewer, which is why each candidate below may offer +several variants and the machine picks. + +Candidates, in the order they are tried: + + vaapi Linux, any GPU with a VA-API driver — Intel iGPU, AMD through mesa. + qsv Intel Quick Sync, which is how the same iGPU is reached on Windows. + nvenc NVIDIA, either platform. + +Two gaps, named rather than left to be discovered: **AMD on Windows** (AMF) and +**macOS** (VideoToolbox). Neither set of arguments could be tried anywhere in +this project, and MeshBay ships no macOS package at all; a node on either +re-encodes in software, exactly as every node did before this module existed. +Adding one is a `Candidate` and nothing else — the probe is what decides +whether it works, so a wrong guess costs a rejected candidate, not a broken +stream. + +Then three modes per stream, remembered per source codec: + + hw Hardware decode and encode. The whole pipeline on the GPU, which is + what makes 1080p HEVC playable on a machine that cannot decode it at + all in software. + hwenc Software decode, hardware encode. The fallback for a source the GPU + has no decoder for — iHD has none for MPEG-4 Part 2, so an Xvid .avi + lands here, and an SD Xvid decodes in software for nearly nothing. + sw libx264, the path that was always here. +""" + +from __future__ import annotations + +import asyncio +import logging +import os +import shutil +import sys +import tempfile +from dataclasses import dataclass + +from meshbay_node import platform + +log = logging.getLogger(__name__) + +HW = "hw" +HWENC = "hwenc" +SW = "sw" + +# How long one test encode may take before it is considered broken. A driver +# that deadlocks must not hold the first viewer's stream, and 20 s is far +# beyond the fraction of a second the probe needs when it works. +PROBE_TIMEOUT_SECS = 20 + +# What the probe encodes. 1080p because the level a film gets is the level that +# has to be checked: an encoder asked for 4.1 on a 320x240 clip may well write +# a lower one, and rejecting a good candidate for that would be the probe +# failing rather than the hardware. +PROBE_SIZE = "1920x1080" + +# The announced codec string is `avc1.640029` — High (profile_idc 100, no +# constraint flags) at level 4.1 (0x29) — and every encoder here is held to it. +_PROFILE_ARGS = ["-profile:v", "high", "-level", "4.1"] +_WANT_PROFILE = "High" +_WANT_LEVEL = 41 + +# -pix_fmt yuv420p: a 10-bit or 4:4:4 HEVC source (common for HDR WEB-DLs) +# fails "-profile:v high" outright otherwise — libx264's High profile is 8-bit +# 4:2:0 only. Downsampling loses nothing a browser could show anyway (MSE has +# no HDR path), which is also what each hardware filter below does on the GPU. +_SOFTWARE_ARGS = ["-c:v", "libx264", "-pix_fmt", "yuv420p", *_PROFILE_ARGS, + "-preset", "veryfast", "-crf", "21"] + + +@dataclass(frozen=True) +class Candidate: + """One way a machine might encode H.264 without its CPU.""" + + name: str + encoder: str + platforms: tuple[str, ...] + # `-hwaccel `, and the pixel format decoded frames stay in. + hwaccel: str + # Filters for frames already on the GPU, and for frames in system memory. + hw_filter: str + sw_filter: str + # Rate control and anything else, tried in order until one produces High + # at level 4.1. A later `-level` overrides the shared one, which is how a + # spelling one encoder rejects is offered without being imposed. + variants: tuple[tuple[str, tuple[str, ...]], ...] + # VA-API alone names a device; QSV and NVENC find their own. + needs_render_node: bool = False + + +CANDIDATES: tuple[Candidate, ...] = ( + Candidate( + name="vaapi", encoder="h264_vaapi", platforms=("linux",), + hwaccel="vaapi", + hw_filter="scale_vaapi=format=nv12", sw_filter="format=nv12,hwupload", + needs_render_node=True, + # CQP is the constant-quality mode every VA-API driver implements (iHD + # and i965 alike); ICQ and QVBR are not always there. Some Intel + # generations implement H.264 encoding only on the low-power (VDEnc) + # path and others only on the full one, which is a property of the chip + # and the driver — so it is measured rather than looked up. + variants=( + ("standard", ("-rc_mode", "CQP", "-qp", "23")), + ("low-power", ("-rc_mode", "CQP", "-qp", "23", "-low_power", "1")), + ), + ), + Candidate( + name="qsv", encoder="h264_qsv", platforms=("win32", "linux"), + hwaccel="qsv", + hw_filter="vpp_qsv=format=nv12", sw_filter="format=nv12", + # `-global_quality` is QSV's constant-quality knob. The second variant + # exists because h264_qsv has no `level` option of its own and takes + # the generic integer one, which may or may not read "4.1" as 41 — + # ffmpeg accepts both spellings without complaint, and only the encoded + # file says which one was understood. + variants=( + ("icq", ("-global_quality", "23")), + ("icq, integer level", ("-global_quality", "23", "-level", "41")), + ), + ), + Candidate( + name="nvenc", encoder="h264_nvenc", platforms=("win32", "linux"), + hwaccel="cuda", + hw_filter="scale_cuda=format=nv12", sw_filter="format=nv12", + variants=( + ("vbr", ("-rc", "vbr", "-cq", "23", "-b:v", "0")), + ), + ), +) + + +@dataclass(frozen=True) +class Encoder: + """A hardware encoder that has been seen to produce what the node claims.""" + + candidate: Candidate + variant: str + extra: tuple[str, ...] + device: str | None = None + + +_enabled = os.environ.get("MESHBAY_HW_VIDEO_ENCODE", "1") not in ("0", "false", "no") +_encoder: Encoder | None = None +_probed = False +_probe_lock: asyncio.Lock | None = None +# (source codec, mode) pairs that have failed once and are not tried again. +_demoted: set[tuple[str, str]] = set() + + +def set_enabled(enabled: bool) -> None: + """Operator switch (`node.toml`, `hardware_video_encode`).""" + global _enabled + _enabled = enabled + + +def render_node() -> str | None: + """The first DRM render node this process can actually open. + + Readable *and* writable, because VA-API maps buffers on it. On a desktop + session logind grants that through an ACL rather than through membership of + the `render` group, so asking the kernel answers "can this process use it" + — which is the question — rather than "is this user in a group", which is + not. + """ + try: + names = sorted(os.listdir("/dev/dri")) + except OSError: + return None + for name in names: + if not name.startswith("renderD"): + continue + dev = f"/dev/dri/{name}" + if os.access(dev, os.R_OK | os.W_OK): + return dev + return None + + +async def _run(args: list[str], timeout: int) -> tuple[int | None, str]: + proc = await asyncio.create_subprocess_exec( + *args, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) + try: + out, err = await asyncio.wait_for(proc.communicate(), timeout) + except TimeoutError: + proc.kill() + await proc.wait() + return None, "timed out" + text = (out or b"").decode("utf-8", "replace") or \ + (err or b"").decode("utf-8", "replace") + return proc.returncode, text.strip() + + +async def _test_encode(cand: Candidate, device: str | None, + extra: tuple[str, ...]) -> str | None: + """Encode 1080p and read the result back. Returns why it was rejected. + + `None` means it worked — the encoder exists, the driver initialises, and + the file it wrote really is High at level 4.1. + """ + ffmpeg = platform.ffmpeg_cmd() + if not (os.path.isabs(ffmpeg) or shutil.which(ffmpeg)): + return "ffmpeg not found" + fd, out = tempfile.mkstemp(suffix=".mp4", prefix="meshbay-hwprobe-") + os.close(fd) + try: + rc, err = await _run([ + ffmpeg, "-hide_banner", "-loglevel", "error", "-nostdin", "-y", + *device_args(cand, device), + "-f", "lavfi", "-i", f"testsrc=size={PROBE_SIZE}:rate=25:duration=0.2", + "-vf", cand.sw_filter, + "-c:v", cand.encoder, *_PROFILE_ARGS, *extra, + out, + ], PROBE_TIMEOUT_SECS) + if rc != 0: + # The *first* line: ffmpeg's last word is usually "nothing was + # written into the output file", which is the consequence. The + # cause is at the top — "Cannot load libcuda.so.1", "Error + # creating a MFX session", "libva ... init failed". + return (err.splitlines() or ["no output"])[0] + rc, info = await _run([ + platform.ffprobe_cmd(), "-v", "error", "-select_streams", "v:0", + "-show_entries", "stream=profile,level", "-of", "csv=p=0", out, + ], PROBE_TIMEOUT_SECS) + profile, _, level = info.partition(",") + if rc != 0 or profile.strip() != _WANT_PROFILE or level.strip() != str(_WANT_LEVEL): + return (f"produced {info or 'nothing'}, and the node announces " + f"{_WANT_PROFILE} at level {_WANT_LEVEL}") + return None + finally: + try: + os.unlink(out) + except OSError: + pass + + +async def encoder() -> Encoder | None: + """The working hardware encoder, established once per process.""" + global _encoder, _probed, _probe_lock + if not _enabled: + return None + if _probed: + return _encoder + if _probe_lock is None: + _probe_lock = asyncio.Lock() + async with _probe_lock: + if _probed: + return _encoder + _encoder = await _find_encoder() + _probed = True + return _encoder + + +async def _find_encoder() -> Encoder | None: + device = render_node() + for cand in CANDIDATES: + if sys.platform not in cand.platforms: + continue + if cand.needs_render_node and device is None: + continue + for variant, extra in cand.variants: + why = await _test_encode(cand, device, extra) + if why is None: + log.info("hwaccel: %s (%s) — video re-encoding runs on the GPU", + cand.encoder, variant) + return Encoder(candidate=cand, variant=variant, extra=extra, + device=device if cand.needs_render_node else None) + log.debug("hwaccel: %s (%s) rejected: %s", cand.encoder, variant, why) + log.info("hwaccel: no hardware H264 encoder works here — encoding in software") + return None + + +async def modes_for(source_codec: str | None) -> list[str]: + """Which re-encode modes to try for this source, best first. + + Always ends in `SW`: libx264 is the one that needs no hardware, so a plan + that ran out of hardware modes is still a plan that plays the film. + """ + enc = await encoder() + if enc is None: + return [SW] + key = source_codec or "?" + plan = [m for m in (HW, HWENC) if (key, m) not in _demoted] + plan.append(SW) + return plan + + +def demote(source_codec: str | None, mode: str, detail: str) -> None: + """This mode does not work for this source codec; stop trying it.""" + if mode == SW: + return + key = source_codec or "?" + if (key, mode) not in _demoted: + _demoted.add((key, mode)) + log.info("hwaccel: %s re-encoding does not work for %s on this machine " + "(%s) — not trying it again", mode, key, detail) + + +def device_args(cand: Candidate, device: str | None) -> list[str]: + """How this candidate is told which GPU to use, where it needs telling.""" + if cand.needs_render_node and device: + return ["-vaapi_device", device] + return [] + + +def input_args(mode: str, enc: Encoder | None) -> list[str]: + """ffmpeg options that must precede `-i`.""" + if enc is None or mode == SW: + return [] + cand = enc.candidate + if mode == HWENC: + return device_args(cand, enc.device) + # `-hwaccel_output_format` keeps decoded frames on the GPU rather than + # reading them back to system memory, which is the whole saving: a readback + # of every 1080p frame costs more on a weak machine than the encode it + # feeds. + args = ["-hwaccel", cand.hwaccel] + if enc.device: + args += ["-hwaccel_device", enc.device] + return args + ["-hwaccel_output_format", cand.hwaccel] + + +def codec_args(mode: str, enc: Encoder | None) -> list[str]: + """The `-vf`/`-c:v` half, where libx264's arguments used to be written.""" + if enc is None or mode == SW: + return list(_SOFTWARE_ARGS) + cand = enc.candidate + # The hardware filter converts a 10-bit HDR source to the 8-bit NV12 the + # encoder takes, on the GPU — the same downsampling the software path does + # with `-pix_fmt yuv420p`, and for the same reason. + filters = cand.hw_filter if mode == HW else cand.sw_filter + return ["-vf", filters, "-c:v", cand.encoder, *_PROFILE_ARGS, *enc.extra] + + +def _reset_for_tests() -> None: + global _encoder, _probed, _probe_lock + _encoder, _probed, _probe_lock = None, False, None + _demoted.clear() diff --git a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py index 540c174..d79674d 100644 --- a/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +++ b/packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py @@ -126,7 +126,7 @@ from meshbay_node.chat import FORMAT_SEALED_V1, ReplayedMessage from meshbay_node.transport.wire import index_sync_message from meshbay_node.indexer import GroupIndex from meshbay_node.indexer.indexer import DirectoryIndexer -from meshbay_node import linkpreview, ops, platform +from meshbay_node import hwaccel, linkpreview, ops, platform from meshbay_node import transfers as transfers_mod from meshbay_node import uploads as uploads_mod from meshbay_node.transfers import TransferSlots @@ -6265,20 +6265,33 @@ class WebRTCPeerSession: if transcode_video: log.info("stream: re-encoding %s (%s) to H264", entry.name, raw_video_codec) - # -pix_fmt yuv420p: a 10-bit or 4:4:4 HEVC source (common for HDR - # WEB-DLs) fails "-profile:v high" outright otherwise — libx264's - # High profile is 8-bit 4:2:0 only. Downsampling loses nothing a - # browser could show anyway (MSE/HTML5 video has no HDR path). - codec_args = ["-c:v", "libx264", "-pix_fmt", "yuv420p", - "-profile:v", "high", "-level", "4.1", - "-preset", "veryfast", "-crf", "21"] + # Where the re-encode runs, and with which arguments — both live + # in hwaccel.py now, including the 8-bit downsampling a 10-bit HDR + # source needs before either encoder will take it. `modes_for` has + # measured this machine by encoding on it and returns the ladder to + # try, always ending in libx264: a node with no usable VA-API does + # exactly what it did before this existed, and a Celeron with an + # iGPU stops being a machine where `transcode_incompatible_video` + # has to be turned off to keep streaming watchable. + modes = await hwaccel.modes_for(raw_video_codec) + hw = await hwaccel.encoder() # Must match "-profile:v high -level 4.1" byte-for-byte (avc1.) — the client checks this string with # MediaSource.isTypeSupported before trusting a single byte of the - # stream, so a mismatch here fails exactly the check this exists to pass. + # stream, so a mismatch here fails exactly the check this exists to + # pass. Both encoders get those two arguments, spelled the same way, + # from hwaccel._PROFILE_ARGS — one place, so they cannot drift. codec_str = "avc1.640029,mp4a.40.2" if has_audio else "avc1.640029" else: - codec_args = ["-c:v", "copy"] + modes, hw = [hwaccel.SW], None + + def video_args(mode: str) -> list[str]: + return (hwaccel.codec_args(mode, hw) if transcode_video + else ["-c:v", "copy"]) + + # The audio half does not change with the video encoder, and is never a + # copy — see _probe_video for why. + audio_args: list[str] = [] # Which audio track. A dubbed film carries several and the first one is # not a neutral default — it is whatever the person who muxed the file # happened to put first, which across a real library is overwhelmingly @@ -6302,7 +6315,7 @@ class WebRTCPeerSession: # appended — isTypeSupported() only checks the codec string, so # the failure doesn't surface until playback, as a SourceBuffer # forced out of its MediaSource with no further explanation. - codec_args += ["-c:a", "aac", "-ac", "2", "-b:a", "192k"] + audio_args = ["-c:a", "aac", "-ac", "2", "-b:a", "192k"] # Where that seek lands, measured with the mapping this stream will # use. It has to be here rather than beside `seek_args` above: the # landing point depends on which audio track is mapped, because the @@ -6316,16 +6329,45 @@ class WebRTCPeerSession: landed = await _seek_lands_at(file_path, requested, map_args) if landed is not None: start = landed - proc = await asyncio.create_subprocess_exec( - platform.ffmpeg_cmd(), "-hide_banner", "-loglevel", "error", - *seek_args, - "-i", str(file_path), - *map_args, - *codec_args, - "-movflags", "frag_keyframe+empty_moov+default_base_moof", - "-f", "mp4", "pipe:1", - stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, - ) + # One spawn per mode, and only ever more than one when hwaccel.py found + # a working GPU. **What a mode is tried against is the file itself**: + # a test encode proves the encoder, and nothing proves the GPU can + # decode *this* source until it is asked to — iHD has no MPEG-4 Part 2 + # decoder at all, so an Xvid .avi fails the full-hardware mode and + # nothing about the machine could have predicted it. + # + # The failure is silent and instant: ffmpeg writes its complaint to + # stderr and exits, so stdout reaches EOF with nothing on it. That is + # the signal read here, before `stream_init` is sent and therefore + # before the client has been told anything it would have to be told + # again. The first segment is kept and handed to the loop below rather + # than re-read, since the process it came from is still running. + # + # The last mode is spawned and trusted, which is what keeps the + # single-mode path — every node without a GPU, and every copied stream + # — byte-for-byte what it was: no extra read, no extra wait. + first_segment = b"" + for attempt, mode in enumerate(modes): + proc = await asyncio.create_subprocess_exec( + platform.ffmpeg_cmd(), "-hide_banner", "-loglevel", "error", + *hwaccel.input_args(mode, hw), + *seek_args, + "-i", str(file_path), + *map_args, + *video_args(mode), *audio_args, + "-movflags", "frag_keyframe+empty_moov+default_base_moof", + "-f", "mp4", "pipe:1", + stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, + ) + if attempt == len(modes) - 1: + break + first_segment = await proc.stdout.read(STREAM_SEGMENT_SIZE) + if first_segment: + break + err = (await proc.stderr.read()).decode("utf-8", "replace").strip() + await proc.wait() + hwaccel.demote(raw_video_codec, mode, + err.splitlines()[0] if err else "no output") self._send({ "type": MNP.STREAM_INIT, @@ -6404,7 +6446,10 @@ class WebRTCPeerSession: log.info("Stream stopped by peer=%s after %d segments", (self._user_id or "?")[:8], index) break - data = await proc.stdout.read(STREAM_SEGMENT_SIZE) + if first_segment: + data, first_segment = first_segment, b"" + else: + data = await proc.stdout.read(STREAM_SEGMENT_SIZE) if not data: break # Same derivation as a file chunk, indexed by segment: one diff --git a/packages/meshbay-node/tests/test_hwaccel.py b/packages/meshbay-node/tests/test_hwaccel.py new file mode 100644 index 0000000..a79d784 --- /dev/null +++ b/packages/meshbay-node/tests/test_hwaccel.py @@ -0,0 +1,171 @@ +""" +The GPU re-encoding plan, and the one thing that must stay true across both +encoders. + +`hwaccel.py` exists so that a low-power node can re-encode a browser-hostile +video at all — see its module docstring. What is tested here is the part that +holds whether or not the machine running the tests has a GPU: the shape of the +plan, what a demotion does, and the agreement between the codec string the node +announces and the arguments each encoder is actually given. + +The end-to-end fallback — a hardware mode that fails, the stream playing anyway +— is in `test_stream_video_transcode.py`, because it needs a real ffmpeg and a +real file. +""" + +import re +import sys +from pathlib import Path + +import pytest +from meshbay_node import hwaccel +from meshbay_node.transport import webrtc_server + +from conftest import needs_subprocess + +VAAPI = next(c for c in hwaccel.CANDIDATES if c.name == "vaapi") +FAKE = hwaccel.Encoder(candidate=VAAPI, variant="standard", extra=(), + device="/dev/dri/renderD128") + + +@pytest.fixture(autouse=True) +def _clean(): + hwaccel._reset_for_tests() + yield + hwaccel._reset_for_tests() + hwaccel.set_enabled(True) + + +async def test_no_working_encoder_means_the_software_path_and_nothing_else(monkeypatch): + """ + The guarantee this module owes every node that has no GPU, no driver, or a + driver that fails its test encode: one mode, the one that was always there. + A second mode would mean a second ffmpeg spawn and a second wait on a + machine that can never benefit from either. + """ + monkeypatch.setattr(hwaccel, "encoder", _fixed(None)) + + assert await hwaccel.modes_for("hevc") == [hwaccel.SW] + + +async def test_the_plan_always_ends_in_software(monkeypatch): + monkeypatch.setattr(hwaccel, "encoder", _fixed(FAKE)) + + assert await hwaccel.modes_for("hevc") == [hwaccel.HW, hwaccel.HWENC, hwaccel.SW] + + +async def test_a_demotion_is_remembered_for_that_source_codec_only(monkeypatch): + """ + iHD decodes HEVC and has no MPEG-4 Part 2 decoder at all, so "the GPU + cannot do this one" is a fact about the pair, not about the machine. + Demoting the machine would cost every HEVC film the acceleration that one + Xvid file proved was unavailable to *it*. + """ + monkeypatch.setattr(hwaccel, "encoder", _fixed(FAKE)) + hwaccel.demote("mpeg4", hwaccel.HW, "no decoder") + + assert await hwaccel.modes_for("mpeg4") == [hwaccel.HWENC, hwaccel.SW] + assert await hwaccel.modes_for("hevc") == [hwaccel.HW, hwaccel.HWENC, hwaccel.SW] + + +async def test_the_operator_switch_turns_off_the_probe_itself(monkeypatch): + """`hardware_video_encode = false` must cost no subprocess, not merely go + unused: the probe is a test encode, and a node whose operator said no + should never spawn it.""" + spawned = [] + monkeypatch.setattr(hwaccel, "_test_encode", + lambda *a, **k: spawned.append(a) or None) + hwaccel.set_enabled(False) + + assert await hwaccel.encoder() is None + assert await hwaccel.modes_for("hevc") == [hwaccel.SW] + assert spawned == [] + + +def test_every_encoder_produces_what_the_node_announces(): + """ + `stream_init` carries `avc1.640029` and the client puts it through + MediaSource.isTypeSupported before it trusts a byte of the stream. The + string is not a label chosen next to libx264 — it is a claim about the + bytes, and it has to stay true for whichever encoder produced them. + + So the claim is read out of the streaming code and decoded (profile_idc + 0x64 = 100 = High, level_idc 0x29 = 41 = 4.1) rather than restated here, + and every mode is checked against it. Changing the arguments of one encoder + without the other, or changing either without the announced string, fails. + """ + source = Path(webrtc_server.__file__).read_text(encoding="utf-8") + announced = set(re.findall(r"avc1\.([0-9a-f]{6})", source)) + assert announced == {"640029"}, ( + "the streaming path announces a codec string this test does not know " + f"how to check: {announced}") + + (code,) = announced + profile_idc, constraints, level_idc = (int(code[i:i + 2], 16) for i in (0, 2, 4)) + assert (profile_idc, constraints, level_idc) == (100, 0, 41), ( + "the announced string no longer says High/4.1, so the arguments below " + "are no longer what it claims") + + for mode in (hwaccel.SW, hwaccel.HW, hwaccel.HWENC): + args = hwaccel.codec_args(mode, FAKE) + assert args[args.index("-profile:v") + 1] == "high", \ + f"{mode} does not encode the High profile the node announces" + assert args[args.index("-level") + 1] == "4.1", \ + f"{mode} does not encode at the level the node announces" + + +def test_the_software_mode_never_touches_the_gpu(): + """The fallback has to be a real fallback: if `sw` carried a VA-API + argument, the mode that exists for machines where the GPU failed would + fail with it.""" + for args in (hwaccel.input_args(hwaccel.SW, FAKE), + hwaccel.codec_args(hwaccel.SW, FAKE)): + joined = " ".join(args) + assert "vaapi" not in joined and "hwupload" not in joined, joined + assert hwaccel.input_args(hwaccel.HW, None) == [] + assert "libx264" in hwaccel.codec_args(hwaccel.HW, None) + + +def _fixed(value): + async def _f(): + return value + return _f + + +@needs_subprocess +async def test_the_probe_accepts_only_what_the_node_announces(): + """The acceptance test is neither always true nor always false. + + Both halves are run with libx264, which is on any machine that can run + these tests, so what is being checked is the judgement itself rather than + anybody's GPU: the same encoder is accepted when it writes High at level + 4.1 and rejected when it writes level 3.1. Without this, an encoder whose + arguments were spelled in a way ffmpeg accepts but misreads would pass the + probe and then announce a level it had not produced. + """ + def _software(label: str, extra: tuple[str, ...]) -> hwaccel.Candidate: + return hwaccel.Candidate( + name=label, encoder="libx264", platforms=(sys.platform,), + hwaccel="none", hw_filter="null", sw_filter="format=yuv420p", + variants=((label, extra),)) + + right = _software("as announced", ("-preset", "veryfast", "-crf", "30")) + assert await hwaccel._test_encode(right, None, right.variants[0][1]) is None + + wrong = _software("a level lower", + ("-preset", "veryfast", "-crf", "30", "-level", "3.1")) + why = await hwaccel._test_encode(wrong, None, wrong.variants[0][1]) + assert why and "31" in why, ( + f"an encoder that wrote another level must be rejected, and say so: {why!r}") + + +@needs_subprocess +async def test_a_missing_encoder_is_rejected_rather_than_raised(): + """ffmpeg exits non-zero for an encoder it does not have; nothing here may + turn that into an exception, because it happens on every machine that lacks + one of the three candidates — which is most of them.""" + absent = hwaccel.Candidate( + name="not a thing", encoder="h264_definitely_not", platforms=(sys.platform,), + hwaccel="none", hw_filter="null", sw_filter="null", variants=(("x", ()),)) + + assert await hwaccel._test_encode(absent, None, ()) is not None diff --git a/packages/meshbay-node/tests/test_stream_video_transcode.py b/packages/meshbay-node/tests/test_stream_video_transcode.py index b165af4..0fd12c6 100644 --- a/packages/meshbay-node/tests/test_stream_video_transcode.py +++ b/packages/meshbay-node/tests/test_stream_video_transcode.py @@ -35,6 +35,7 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey from meshbay_common.crypto import generate_gek from meshbay_common.webcrypto import chunk_key_aes, decrypt_chunk_aes +from meshbay_node import hwaccel from meshbay_node.indexer.group_index import GroupIndex from meshbay_node.transport.webrtc_server import WebRTCPeerSession, _probe_video @@ -271,3 +272,50 @@ async def test_the_operator_can_refuse_the_re_encode_and_says_so(tmp_path): f"the refusal must point at the setting that caused it: {detail!r}") assert not [m for m in session.sent if m.get("type") == "stream_init"], ( "a stream_init went out with no codec string the client could check") + + +@needs_mp3 +async def test_a_hardware_encoder_that_does_not_work_costs_the_viewer_nothing( + tmp_path, monkeypatch): + """A VA-API mode that fails falls through to libx264 and the film plays. + + This is the failure that cannot be prevented by probing: `hwaccel.py` + proves the *encoder* with a test encode at startup, and nothing proves the + GPU can decode a particular source until it is asked to — iHD has no + MPEG-4 Part 2 decoder at all, which is exactly the file used here. + + The encoder is faked onto `/dev/null` rather than skipped for want of a + GPU, so this runs identically on a machine with one and on a machine + without: both hardware modes fail instantly, and what is tested is that + the viewer sees one working stream and not an error. The demotions are + checked too — a library of Xvid files must not pay for the first one's two + dead spawns again and again. + """ + clip = tmp_path / "clip.avi" + _make_mpeg4_clip(clip) + gek = generate_gek() + session, file_id = _session(clip, gek) + + hwaccel._reset_for_tests() + + async def _broken_encoder(): + vaapi = next(c for c in hwaccel.CANDIDATES if c.name == "vaapi") + return hwaccel.Encoder(candidate=vaapi, variant="standard", extra=(), + device="/dev/null") + + monkeypatch.setattr(hwaccel, "encoder", _broken_encoder) + try: + await session._stream_video_inner({"file_id": file_id, "start": 0, "credits": 0}) + + assert not [m for m in session.sent if m.get("type") == "error"], ( + "a hardware encoder that does not work must cost the viewer nothing") + remuxed = _reassemble(session.sent, gek, file_id) + out_path = tmp_path / "out.mp4" + out_path.write_bytes(remuxed) + assert _output_video_codec(out_path) == "h264", \ + "the software fallback must still deliver a playable H264 stream" + + assert ("mpeg4", hwaccel.HW) in hwaccel._demoted + assert ("mpeg4", hwaccel.HWENC) in hwaccel._demoted + finally: + hwaccel._reset_for_tests() diff --git a/packaging/README.md b/packaging/README.md index ff65488..b65d0ed 100644 --- a/packaging/README.md +++ b/packaging/README.md @@ -42,6 +42,36 @@ sudo rpm -ivh meshbay-hub-*.rpm meshbay-node-*.rpm sudo rpm -ivh meshbay-client-*.rpm ``` +### Hardware video acceleration is a weak dependency + +Both the node and the client use the GPU for video where one works — the node to +re-encode a browser-hostile codec (`hwaccel.py`: VA-API on Linux, Quick Sync or +NVENC on Windows), the client to decode H.264 (`main.js`; only on Linux, since +Chromium already decodes in hardware on Windows and macOS). Neither requires it, +and both establish it by measurement rather than by assuming the packaged driver +is present and functional, so a machine without a GPU installs and runs exactly +as before. + +Only Linux needs packaged drivers — on Windows the bundled ffmpeg carries its +own encoders and Chromium uses the OS's decoder. The drivers are therefore +`Recommends` (`va-driver-all` on Debian/Ubuntu; +`libva`, `intel-media-driver`, `mesa-va-drivers` on Fedora), which matters for +how the packages are installed: **`dpkg -i` and `rpm -ivh` both ignore weak +dependencies; `apt` and `dnf` install them.** On a low-power machine — an Intel +mini-PC serving and watching video is the case this exists for — use the +resolver, so the drivers come too: + +```bash +sudo apt install ./meshbay-common_*.deb ./meshbay-node_*.deb ./meshbay-client_*.deb +sudo dnf install ./meshbay-common-*.rpm ./meshbay-node-*.rpm ./meshbay-client-*.rpm +``` + +Nothing has to be enabled afterwards and there are no flags to set by hand. The +node logs which encoder it found at the first stream (`hwaccel:`), and the +client records what it measured in its own `config.json` — the file that +already remembers the hub address — under `videoDecodeProbe`, and +`videoAcceleration: "off"` in that same file turns it back off. + ## Architecture meshbay-common owns the shared Python virtual environment at diff --git a/packaging/deb/meshbay-client/DEBIAN/control b/packaging/deb/meshbay-client/DEBIAN/control index aeb5865..17711c8 100644 --- a/packaging/deb/meshbay-client/DEBIAN/control +++ b/packaging/deb/meshbay-client/DEBIAN/control @@ -7,7 +7,8 @@ Maintainer: MeshBay Team Homepage: https://meshbay.org Depends: libgtk-3-0, libnotify4, libnss3, libxss1, libxtst6, libatspi2.0-0, libdrm2, libgbm1, libasound2 -Recommends: meshbay-node +Recommends: meshbay-node, va-driver-all +Suggests: vainfo Description: MeshBay — peer-to-peer file sharing, streaming and group chat Desktop client for MeshBay. Connects to MeshBay nodes over WebRTC for file sharing, video streaming, and group chat. Includes Chromecast and diff --git a/packaging/deb/meshbay-node/DEBIAN/control b/packaging/deb/meshbay-node/DEBIAN/control index 3eda3fa..eb19eb7 100644 --- a/packaging/deb/meshbay-node/DEBIAN/control +++ b/packaging/deb/meshbay-node/DEBIAN/control @@ -6,7 +6,7 @@ Architecture: all Maintainer: MeshBay Team Homepage: https://meshbay.org Depends: meshbay-common (= __VERSION__) -Recommends: ffmpeg +Recommends: ffmpeg, va-driver-all Description: MeshBay Node — local file host, streaming server, and group daemon MeshBay Node indexes local directories and serves encrypted files to authenticated group members over WebRTC. Includes video streaming (fMP4 @@ -15,4 +15,6 @@ Description: MeshBay Node — local file host, streaming server, and group daemo . Installs node code into the shared venv at /opt/meshbay-common/venv/. Ships a default TMDB API token for the Videos app. - Runs as a systemd user service. + Runs as a systemd user service. Re-encodes video on the GPU where one + works (VA-API), which is what makes streaming an HEVC film from a + low-power machine possible at all. diff --git a/packaging/rpm/meshbay-client.spec b/packaging/rpm/meshbay-client.spec index 867205f..0816741 100644 --- a/packaging/rpm/meshbay-client.spec +++ b/packaging/rpm/meshbay-client.spec @@ -17,6 +17,13 @@ Requires: libdrm Requires: mesa-libgbm Requires: alsa-lib Recommends: meshbay-node +# Weak deps: an unresolvable one is skipped, so naming the drivers of both +# families costs nothing and covers Fedora and RHEL alike. See hwaccel.py +# (node) / main.js (client) — neither assumes any of this is installed. +Recommends: libva +Recommends: intel-media-driver +Recommends: mesa-va-drivers +Suggests: libva-utils %description Desktop client for MeshBay. Connects to MeshBay nodes over WebRTC for diff --git a/packaging/rpm/meshbay-node.spec b/packaging/rpm/meshbay-node.spec index 6ac13fb..1ba7f26 100644 --- a/packaging/rpm/meshbay-node.spec +++ b/packaging/rpm/meshbay-node.spec @@ -10,6 +10,12 @@ BuildArch: noarch Requires: meshbay-common = %{version} Recommends: ffmpeg +# Weak deps: an unresolvable one is skipped, so naming the drivers of both +# families costs nothing and covers Fedora and RHEL alike. See hwaccel.py +# (node) / main.js (client) — neither assumes any of this is installed. +Recommends: libva +Recommends: intel-media-driver +Recommends: mesa-va-drivers %description MeshBay Node indexes local directories and serves encrypted files to diff --git a/packaging/win/README.md b/packaging/win/README.md index b5ab1cd..0598193 100644 --- a/packaging/win/README.md +++ b/packaging/win/README.md @@ -98,8 +98,13 @@ That runs [`build-win.ps1`](build-win.ps1): **Bundled by default** — [`fetch-ffmpeg.ps1`](fetch-ffmpeg.ps1) downloads, checksum-verifies and stages it into `node-runtime/` on every build, ~161 MB. Not optional in practice: MeshBay transcodes browser-incompatible video to -H.264 (`-c:v libx264`, a real encode, not remux), and no LGPL-only ffmpeg -build includes an H.264 *encoder* — libx264 itself is GPL. Asking an end user +H.264 (a real encode, not remux), and no LGPL-only ffmpeg build includes an +H.264 *encoder* — libx264 itself is GPL. libx264 is also the floor rather +than the only path: this build carries `h264_qsv` and `h264_nvenc` as well, +which is what lets `hwaccel.py` move the encode onto an Intel or NVIDIA GPU +on Windows without a second download. **A re-pin that dropped those would +cost every low-power Windows node its hardware encoding, silently** — the +node would simply go back to libx264. Asking an end user to separately run `winget install ffmpeg` was considered and rejected: it needs network access and `winget`/App Installer present at that exact moment, and its failure mode is silent — video just does not stream, with nothing diff --git a/packaging/win/fetch-ffmpeg.ps1 b/packaging/win/fetch-ffmpeg.ps1 index 5f6123d..6ffbc51 100644 --- a/packaging/win/fetch-ffmpeg.ps1 +++ b/packaging/win/fetch-ffmpeg.ps1 @@ -3,9 +3,12 @@ Download, verify and stage the ffmpeg/ffprobe MeshBay bundles by default. .DESCRIPTION - MeshBay transcodes browser-incompatible video to H.264 (webrtc_server.py, - `-c:v libx264`) -- a real encode, not just remux/probe -- so only a GPL - ffmpeg build works here; libx264 is GPL, no LGPL-only build includes it. + MeshBay transcodes browser-incompatible video to H.264 (webrtc_server.py) + -- a real encode, not just remux/probe -- so only a GPL ffmpeg build works + here; libx264 is GPL, no LGPL-only build includes it. The gpl-shared + preset also carries h264_qsv and h264_nvenc, which hwaccel.py uses to put + that encode on the GPU where there is one; a build without them still + works and is simply slower, on the machines that can least afford it. Asking an end user to separately install ffmpeg (`winget install ffmpeg`) is not viable for a non-technical install: it needs network access and winget/App Installer present at that exact moment, and fails silently -- cgit v1.2.3