diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-08-11 16:47:01 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-08-11 16:47:01 +0200 |
| commit | e608b95bf1fe225915eaeafca2a933f687844733 (patch) | |
| tree | 255b28a2d667af0729bb7829d6307324c0119fce /devel-phases-next.md | |
| parent | fc509ae281c8cd0aa69870f6123a11e3519fb390 (diff) | |
| download | meshbay-e608b95bf1fe225915eaeafca2a933f687844733.tar.gz | |
feat: Phase 10c — MSE video streaming (real-time playback)
Replace download-then-play VideoPlayer with MSE (MediaSource Extensions)
streaming. Node remuxes to fMP4 via ffmpeg, probes codecs with ffprobe,
and sends encrypted segments over DataChannel. Browser decrypts and
appends to SourceBuffer — playback starts within seconds.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat (limited to 'devel-phases-next.md')
| -rw-r--r-- | devel-phases-next.md | 93 |
1 files changed, 91 insertions, 2 deletions
diff --git a/devel-phases-next.md b/devel-phases-next.md index 3e49f19..e423d83 100644 --- a/devel-phases-next.md +++ b/devel-phases-next.md @@ -1,6 +1,6 @@ # MeshBay — Next Implementation Phases -> Base: Phases 1–10b complete (except 10.9 → Phase 13). 166 tests. Web SPA + admin panel + self-service UI live on meshbay.org. +> Base: Phases 1–10c complete (except 10.9 → Phase 13). 167 tests. Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org. > Architecture reference: docs/meshbay-draft-v4.md > First security review: first-review.md (2026-08-10) @@ -189,7 +189,7 @@ ICE/STUN handles all tested NAT types automatically. path-based dedup (remove old entry before adding new). **Known remaining items for future phases:** -- True video streaming (MSE or Service Worker) — currently downloads full file first +- ~~True video streaming (MSE or Service Worker)~~ → Phase 10c (2026-08-11) - Multiple shared directories per node (UI + config) - Multi-node per user support @@ -405,6 +405,95 @@ Results link back to the group page. Accessible from sidebar. --- +## Phase 10c — MSE video streaming (real-time playback) + +Pending commit — 167 tests. + +**Objective:** replace the download-then-play video player with real-time +MSE (MediaSource Extensions) streaming. Playback starts within seconds +instead of waiting for the full file download. + +### Architecture + +``` +Browser Node + │ │ + ├── stream_req {file_id} ──────►│ + │ ├── ffprobe → codec info + │◄──── stream_init {codec,dur} ──┤ + │ ├── ffmpeg -c copy → fMP4 pipe + │◄──── stream_data {seg 0, ct} ──┤ (256 KB encrypted segments) + │◄──── stream_data {seg 1, ct} ──┤ + │ ... │ + │◄──── stream_end ───────────────┤ + │ │ + MediaSource → SourceBuffer │ + ├── appendBuffer(decrypted) │ + ├── video.play() after ~2-3s │ +``` + +**Key design decisions:** + +1. **Node-side remux via ffmpeg** — `ffmpeg -c copy -movflags frag_keyframe+empty_moov+default_base_moof -f mp4 pipe:1` remuxes any video format (MP4, MKV, AVI, WebM, MOV) into fragmented MP4 (fMP4) that MSE can consume. No transcoding — just remuxing. Near-zero CPU overhead. + +2. **Codec detection via ffprobe** — the node probes the video to determine the exact codec string for MSE SourceBuffer creation (e.g., `avc1.640028,mp4a.40.2` for H.264 High@4.0 + AAC-LC). This ensures the browser creates the correct decoder. + +3. **Same encryption model** — each 256 KB fMP4 segment is encrypted with AES-256-GCM using the same key derivation as file downloads (GEK + file_hash + segment_index → HKDF → chunk_key). E2E encryption is maintained. + +4. **Progressive SourceBuffer append** — the browser creates a MediaSource, opens a SourceBuffer with the probed codec, and appends decrypted segments as they arrive. SourceBuffer handles partial MP4 boxes internally. Playback starts after ~2-3 segments (~512 KB buffered). + +### Supported codecs + +| Codec | MSE string | Browser support | +|---|---|---| +| H.264 (AVC) | `avc1.PPCCLL` | Chrome, Firefox, Safari, Edge | +| H.265 (HEVC) | `hev1.1.6.L93.B0` | Safari, Chrome (partial) | +| VP9 | `vp09.00.10.08` | Chrome, Firefox | +| AV1 | `av01.0.01M.08` | Chrome, Firefox | +| AAC | `mp4a.40.2` | All | +| MP3 | `mp4a.6b` | All | +| Opus | `opus` | Chrome, Firefox | +| AC-3 | `ac-3` | Safari, Chrome | + +### New MNP message types + +| Type | Direction | Description | +|---|---|---| +| `stream_req` | client → node | Request MSE video stream for file_id | +| `stream_init` | node → client | Codec string + duration (probed via ffprobe) | +| `stream_data` | node → client | Encrypted fMP4 segment (256 KB, AES-GCM) | +| `stream_end` | node → client | End of stream signal | + +### Milestones + +| # | Component | Status | +|---|---|---| +| 10c.1 | MNP protocol: STREAM_REQUEST/INIT/DATA/END message types | ✅ | +| 10c.2 | Node: ffprobe codec detection + MSE codec string derivation | ✅ | +| 10c.3 | Node: ffmpeg fMP4 remux + encrypted segment streaming | ✅ | +| 10c.4 | Transport: event-based stream message dispatch | ✅ | +| 10c.5 | Browser: MSE VideoPlayer (MediaSource + SourceBuffer) | ✅ | +| 10c.6 | Tests: stream_request error handling | ✅ | + +### File changes + +**Modified:** +- `packages/meshbay-common/src/meshbay_common/protocol.py` — STREAM_REQUEST/INIT/DATA/END +- `packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py` — `_probe_video()`, `_stream_video()` handler +- `packages/meshbay-hub/src/meshbay_hub/static/transport.js` — `requestStream()`, stream event handlers +- `packages/meshbay-hub/src/meshbay_hub/static/app.js` — MSE-based VideoPlayer component +- `packages/meshbay-hub/src/meshbay_hub/static/style.css` — streaming progress bar +- `packages/meshbay-hub/src/meshbay_hub/static/i18n.js` — buffering/MSE error strings +- `packages/meshbay-node/tests/test_webrtc_transport.py` — stream_request error test + +### Known limitations (future work) + +- No seeking beyond buffered range (user must wait for data to arrive) +- No adaptive bitrate (single quality stream) +- Requires ffmpeg/ffprobe on the node (already a dependency for the old STREAM_SEGMENT handler) + +--- + ## Phase 11 — Android client MVP **Objective:** Android app for account creation, group browsing, file download, |