aboutsummaryrefslogtreecommitdiffstats
path: root/devel-phases-next.md
diff options
context:
space:
mode:
authorChristophe Besson <cbesson@gmail.com>2026-08-11 16:47:01 +0200
committerChristophe Besson <cbesson@gmail.com>2026-08-11 16:47:01 +0200
commite608b95bf1fe225915eaeafca2a933f687844733 (patch)
tree255b28a2d667af0729bb7829d6307324c0119fce /devel-phases-next.md
parentfc509ae281c8cd0aa69870f6123a11e3519fb390 (diff)
downloadmeshbay-e608b95bf1fe225915eaeafca2a933f687844733.tar.gz
feat: Phase 10c — MSE video streaming (real-time playback)
Replace download-then-play VideoPlayer with MSE (MediaSource Extensions) streaming. Node remuxes to fMP4 via ffmpeg, probes codecs with ffprobe, and sends encrypted segments over DataChannel. Browser decrypts and appends to SourceBuffer — playback starts within seconds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat (limited to 'devel-phases-next.md')
-rw-r--r--devel-phases-next.md93
1 files changed, 91 insertions, 2 deletions
diff --git a/devel-phases-next.md b/devel-phases-next.md
index 3e49f19..e423d83 100644
--- a/devel-phases-next.md
+++ b/devel-phases-next.md
@@ -1,6 +1,6 @@
# MeshBay — Next Implementation Phases
-> Base: Phases 1–10b complete (except 10.9 → Phase 13). 166 tests. Web SPA + admin panel + self-service UI live on meshbay.org.
+> Base: Phases 1–10c complete (except 10.9 → Phase 13). 167 tests. Web SPA + admin panel + self-service UI + MSE video streaming live on meshbay.org.
> Architecture reference: docs/meshbay-draft-v4.md
> First security review: first-review.md (2026-08-10)
@@ -189,7 +189,7 @@ ICE/STUN handles all tested NAT types automatically.
path-based dedup (remove old entry before adding new).
**Known remaining items for future phases:**
-- True video streaming (MSE or Service Worker) — currently downloads full file first
+- ~~True video streaming (MSE or Service Worker)~~ → Phase 10c (2026-08-11)
- Multiple shared directories per node (UI + config)
- Multi-node per user support
@@ -405,6 +405,95 @@ Results link back to the group page. Accessible from sidebar.
---
+## Phase 10c — MSE video streaming (real-time playback)
+
+Pending commit — 167 tests.
+
+**Objective:** replace the download-then-play video player with real-time
+MSE (MediaSource Extensions) streaming. Playback starts within seconds
+instead of waiting for the full file download.
+
+### Architecture
+
+```
+Browser Node
+ │ │
+ ├── stream_req {file_id} ──────►│
+ │ ├── ffprobe → codec info
+ │◄──── stream_init {codec,dur} ──┤
+ │ ├── ffmpeg -c copy → fMP4 pipe
+ │◄──── stream_data {seg 0, ct} ──┤ (256 KB encrypted segments)
+ │◄──── stream_data {seg 1, ct} ──┤
+ │ ... │
+ │◄──── stream_end ───────────────┤
+ │ │
+ MediaSource → SourceBuffer │
+ ├── appendBuffer(decrypted) │
+ ├── video.play() after ~2-3s │
+```
+
+**Key design decisions:**
+
+1. **Node-side remux via ffmpeg** — `ffmpeg -c copy -movflags frag_keyframe+empty_moov+default_base_moof -f mp4 pipe:1` remuxes any video format (MP4, MKV, AVI, WebM, MOV) into fragmented MP4 (fMP4) that MSE can consume. No transcoding — just remuxing. Near-zero CPU overhead.
+
+2. **Codec detection via ffprobe** — the node probes the video to determine the exact codec string for MSE SourceBuffer creation (e.g., `avc1.640028,mp4a.40.2` for H.264 High@4.0 + AAC-LC). This ensures the browser creates the correct decoder.
+
+3. **Same encryption model** — each 256 KB fMP4 segment is encrypted with AES-256-GCM using the same key derivation as file downloads (GEK + file_hash + segment_index → HKDF → chunk_key). E2E encryption is maintained.
+
+4. **Progressive SourceBuffer append** — the browser creates a MediaSource, opens a SourceBuffer with the probed codec, and appends decrypted segments as they arrive. SourceBuffer handles partial MP4 boxes internally. Playback starts after ~2-3 segments (~512 KB buffered).
+
+### Supported codecs
+
+| Codec | MSE string | Browser support |
+|---|---|---|
+| H.264 (AVC) | `avc1.PPCCLL` | Chrome, Firefox, Safari, Edge |
+| H.265 (HEVC) | `hev1.1.6.L93.B0` | Safari, Chrome (partial) |
+| VP9 | `vp09.00.10.08` | Chrome, Firefox |
+| AV1 | `av01.0.01M.08` | Chrome, Firefox |
+| AAC | `mp4a.40.2` | All |
+| MP3 | `mp4a.6b` | All |
+| Opus | `opus` | Chrome, Firefox |
+| AC-3 | `ac-3` | Safari, Chrome |
+
+### New MNP message types
+
+| Type | Direction | Description |
+|---|---|---|
+| `stream_req` | client → node | Request MSE video stream for file_id |
+| `stream_init` | node → client | Codec string + duration (probed via ffprobe) |
+| `stream_data` | node → client | Encrypted fMP4 segment (256 KB, AES-GCM) |
+| `stream_end` | node → client | End of stream signal |
+
+### Milestones
+
+| # | Component | Status |
+|---|---|---|
+| 10c.1 | MNP protocol: STREAM_REQUEST/INIT/DATA/END message types | ✅ |
+| 10c.2 | Node: ffprobe codec detection + MSE codec string derivation | ✅ |
+| 10c.3 | Node: ffmpeg fMP4 remux + encrypted segment streaming | ✅ |
+| 10c.4 | Transport: event-based stream message dispatch | ✅ |
+| 10c.5 | Browser: MSE VideoPlayer (MediaSource + SourceBuffer) | ✅ |
+| 10c.6 | Tests: stream_request error handling | ✅ |
+
+### File changes
+
+**Modified:**
+- `packages/meshbay-common/src/meshbay_common/protocol.py` — STREAM_REQUEST/INIT/DATA/END
+- `packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py` — `_probe_video()`, `_stream_video()` handler
+- `packages/meshbay-hub/src/meshbay_hub/static/transport.js` — `requestStream()`, stream event handlers
+- `packages/meshbay-hub/src/meshbay_hub/static/app.js` — MSE-based VideoPlayer component
+- `packages/meshbay-hub/src/meshbay_hub/static/style.css` — streaming progress bar
+- `packages/meshbay-hub/src/meshbay_hub/static/i18n.js` — buffering/MSE error strings
+- `packages/meshbay-node/tests/test_webrtc_transport.py` — stream_request error test
+
+### Known limitations (future work)
+
+- No seeking beyond buffered range (user must wait for data to arrive)
+- No adaptive bitrate (single quality stream)
+- Requires ffmpeg/ffprobe on the node (already a dependency for the old STREAM_SEGMENT handler)
+
+---
+
## Phase 11 — Android client MVP
**Objective:** Android app for account creation, group browsing, file download,