diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-10-05 01:16:12 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-10-05 01:16:12 +0200 |
| commit | 243bcf017b4957237b3ddf556bd0a37296eb8798 (patch) | |
| tree | 7d0820796737aad4dcfb89de7bec1e7271759af6 | |
| parent | ac8ed437659d765120d1b0cc9f83e942549b479f (diff) | |
| download | meshbay-243bcf017b4957237b3ddf556bd0a37296eb8798.tar.gz | |
feat(cast): photos on the TV, and a television chosen once for the session
A cast button in Videos' toolbar, at the top of Photos, in an album's bar and
in the lightbox, in a group and in Search alike. A television chosen there is
kept for the session: a film opened plays on it with the player as its remote
from the start, and a photo opened in the lightbox is shown on it, scaled to
1920x1080, upright, as JPEG. The lightbox gains a slideshow.
The relay serves one photo at /image behind the stream's token, on the desktop
and on Android; the shell, not the page, decides that the receiver loads it as
a picture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
31 files changed, 1203 insertions, 56 deletions
@@ -921,6 +921,16 @@ do. Read them before writing anything that touches the same mechanism. machine could. The receiver was half-crashed; a power cycle fixed it. A machine that reaches the receiver proves nothing about the phone +- **The default media receiver will not start a stream that arrives at real + time.** Casting the interface as live video, it stayed on BUFFERING until the + end of the file — with or without audio, at any bitrate. It started only with + an audio track *and* the first seconds delivered faster than real time, and + then kept that lead as five seconds of latency. A film meets both conditions + by construction, which is why films cast. Before that was found, a receiver in + a bad state had made single runs contradict each other for an hour: power-cycle + it, prove a real film casts, repeat every case, and run a known reference + first and last + **Corrections that used to live here** — `punch_nat()` is not a traversal stack, the node keystore's Argon2id parameters, what group chat actually uses, and what is sealed on the wire — are now design statements in `docs/MESHBAY_DESIGN.md` @@ -1023,6 +1033,7 @@ here are kept only where they are a rule about *editing* the code. | Where the hub is | `platform.js` — `hubBase()` | **the only file allowed to decide this** (§8.3) | | Downloads, decrypt pipeline | `file-utils.js`, `downloads.js`, `sw.js` | §8.5 | | Video player | `video-player.js` — `pump()` is the only place credit is granted | §8.5 | +| The television a session casts to, photos on it | `cast-session.js` | §11.4 | | Transfers widget | `transfers.js` | §5.5 | | Cross-group merge | `source-merge.js`, `group-name.js` | §9.11 | | Node page | `node-page.js` | §6.7 | @@ -1057,6 +1068,7 @@ when `ANDROID_HOME` is set (`test_android_shell.py`). | An invitation answered on the home page (accept, decline), in both engines | `packages/meshbay-hub/tests/harness/invitation_probe.py` | | A group owner's invited list and host approvals, in both engines | `packages/meshbay-hub/tests/harness/group_hosts_probe.py` | | A show's modal and the player, walked episode to episode | `packages/meshbay-hub/tests/harness/video_series_probe.py` | +| A television chosen once, then photos and a film sent to it, in both engines | `packages/meshbay-hub/tests/harness/cast_session_probe.py` | | A lazily loaded view whose module answers 404 (a tab older than the deploy) | `packages/meshbay-hub/tests/harness/lazy_failure_probe.py` | ## meshbay.org server (target state) diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 525c5c3..64364b0 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -3422,6 +3422,36 @@ lands the target is shown, not the old stream's position. Pausing the receiver pauses the local element too, which otherwise goes on fetching for nobody. The copy-URL cast has no receiver to read and keeps the ordinary player. +**A television is chosen for the session, not for one film** +(`cast-session.js`). Any cast button — Videos' toolbar, the Photos toolbar, a +photo album's bar, the lightbox, the player — sets it, and it is held in memory +only. A film opened +while one is set starts its cast as a restart does: pending until the first +segment, then the relay and `connect` (or `reload`, when the receiver is already +connected), with the player drawn as the remote from the start. *Stop casting* +clears it, from the player's remote or from any button. + +**A photo is served at `/image`, one at a time** (`cast:image`). The page scales +it to fit 1920×1080, applies its EXIF orientation and re-encodes it as JPEG +before it leaves: a camera original is too large for a receiver to decode in good +time, and some formats it does not decode at all. The relay starts for it if +nothing else has, serves it behind the stream's token with a versioned address +(a receiver handed the same URL twice shows what it already has), and accepts +only `image/jpeg`, `image/png` or `image/webp`. **The shell, not the page, says +what the receiver loads:** the relay's photo URL is loaded as a picture +(`streamType: NONE`, photo metadata, no subtitles), any other relay URL as the +stream, and on Android no URL but the relay's own is accepted at all. A photo +does not start the foreground service, and its load answers as soon as the +receiver accepts it: a photo never reaches PLAYING. Only the newest photo asked +for is sent; paging quickly shows where the reader stopped. + +**The interface itself is not cast.** The default media receiver plays what it +is given and renders nothing of its own, and a rendered interface sent to it as +live video does not work: it starts a progressive stream that arrives at real +time only when the stream carries audio and its first seconds arrive faster than +real time, and then holds that lead as latency — five seconds, measured. Showing +the application on a television needs a registered receiver of our own. + **Subtitles are rebased onto the relay's clock before they are sent.** The node extracts a track whole, so its cues carry the film's timeline, and the player can use them unchanged because its SourceBuffer is given `timestampOffset = diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index d6eab26..3b2f16c 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -352,7 +352,9 @@ An album browser and a player, over the folders the operator chose for it. ### Photos An album browser over the folders the operator chose for it, where **an album -is a folder**. Thumbnails come from the node, already rotated correctly. +is a folder**. Thumbnails come from the node, already rotated correctly. A +photo opens full size, with a slideshow button that moves on every five +seconds and stops at the album's last photo. **Location data is never shown.** Photos shows when a picture was taken and what took it, and no coordinates anywhere. Worth knowing, though: the @@ -392,15 +394,25 @@ resume. ### Casting to a TV -**The desktop and Android applications.** A film playing in the application can -be sent to a cast-capable TV or dongle on the same network: *Cast to device* in -the player, pick one from the list. On a phone, the film goes on playing silently -on the phone while the TV shows it, and the screen can be turned off. While a -TV plays the film, the player becomes its remote: the position shown is the -TV's, with play/pause, 30-second jumps and a slider to go anywhere in the film. -Devices appear as they answer, usually within a couple -of seconds; the search carries on for a few more, for a TV that is still -waking up. +**The desktop and Android applications.** Films and photos can be sent to a +cast-capable TV or dongle on the same network. The cast button is in the +Videos toolbar, at the top of Photos, in an album's bar and in the photo +viewer — in a group and in Search alike — and *Cast to device* is in the +player. Devices appear as +they answer, usually within a couple of seconds; the search carries on for a +few more, for a TV that is still waking up. + +**A TV, once chosen, stays chosen** until *Stop casting*, from any cast button +or from the player. Every film opened plays on it, and the player opens as its +remote: the position shown is the TV's, with play/pause, 30-second jumps and a +slider to go anywhere in the film. Every photo opened in the viewer is shown on +it, and a slideshow pages the TV along with the screen. Choosing a TV from an +album's bar opens its first photo. On a phone, a film goes on playing silently +on the phone while the TV shows it, and the screen can be turned off. + +Photos are sent scaled to the TV's screen, upright, as an ordinary picture. +The menus, covers and summaries stay on your phone or computer: the TV shows +what is playing, not the application. The application decrypts the film and relays it to the TV itself, over your LAN. The TV is not a group member and holds no key — which is also why the relay's @@ -981,7 +993,8 @@ Better to know now than to go looking for it: - **Hubs do not talk to each other yet.** Everyone in a group needs an account on the same hub. - **Casting reaches Chromecast devices** from the desktop and Android - applications. Support for other TV protocols is designed but not built. + applications, for films and photos. Music is not cast yet. Support for other + TV protocols is designed but not built. - **Some subtitle tracks cannot be shown** — the ones stored as images rather than text, roughly one embedded track in five. Displaying them would need text recognition. diff --git a/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js b/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js index 7ace33b..21d27df 100644 --- a/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js +++ b/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js @@ -151,6 +151,7 @@ push: (data) => pushSegment(data), stop: () => call('cast:stop'), subtitle: (sub) => call('cast:subtitle', sub), + image: (img) => call('cast:image', { bytes: base64Of(bytesOf(img.bytes)), type: img.type }), finish: () => call('cast:finish'), status: () => call('cast:status'), scan: () => call('cast:scan'), diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt index 74a086b..6be5d72 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt @@ -26,8 +26,8 @@ class CastChannels( fun handles(channel: String) = channel.startsWith("cast:") - /** What must reach the relay in the order the page sent it: start, pushes, subtitle, finish, stop. */ - fun ordered(channel: String) = channel in setOf("cast:start", "cast:push", "cast:subtitle", "cast:finish", "cast:stop") + /** What must reach the relay in the order the page sent it: start, pushes, subtitle, photo, finish, stop. */ + fun ordered(channel: String) = channel in setOf("cast:start", "cast:push", "cast:subtitle", "cast:image", "cast:finish", "cast:stop") fun call(channel: String, a: JSONArray): Any? = when (channel) { "cast:start" -> { @@ -38,6 +38,11 @@ class CastChannels( started } "cast:subtitle" -> relay.setSubtitle(a.optJSONObject(0)) + // No foreground service for a photo: the receiver has it once loaded. + "cast:image" -> { + val o = a.optJSONObject(0) ?: JSONObject() + relay.showImage(Base64.getDecoder().decode(o.optString("bytes", "")), o.optString("type", "")) + } "cast:push" -> { // the base64 path, for a WebView without ArrayBuffer messages relay.push(Base64.getDecoder().decode(a.optString(0, ""))); true } @@ -50,11 +55,13 @@ class CastChannels( "cast:devices" -> control.devices() "cast:chromecast:connect" -> { val o = a.optJSONObject(0) ?: JSONObject() - reported { control.connect(o.optString("deviceId", ""), relayUrl(o), subtitleOf(o)) } + val url = relayUrl(o) + reported { control.connect(o.optString("deviceId", ""), url, subtitleFor(url, o), typeOf(url)) } } "cast:chromecast:reload" -> { val o = a.optJSONObject(0) ?: JSONObject() - reported { control.reload(relayUrl(o), subtitleOf(o)) } + val url = relayUrl(o) + reported { control.reload(url, subtitleFor(url, o), typeOf(url)) } } "cast:chromecast:pause" -> control.pause() "cast:chromecast:play" -> control.play() @@ -79,19 +86,27 @@ class CastChannels( /** * The receiver is only ever pointed at this relay. A URL the page names is - * accepted when it is the relay's own, and refused otherwise — a page - * cannot use this application to make a television fetch anything else. + * accepted when it is the relay's own stream or photo, and refused + * otherwise — a page cannot use this application to make a television + * fetch anything else. */ private fun relayUrl(o: JSONObject): String { val asked = o.optString("mediaUrl", "") val ours = relay.url ?: throw Refused("The cast relay is not running") - if (asked != ours) throw Refused("Refused: not this relay's stream") - return ours + if (asked != ours && asked != relay.imageUrl) throw Refused("Refused: not this relay's stream") + return asked } - /** As main.js: no subtitle named means the relay's current one. */ - private fun subtitleOf(o: JSONObject): JSONObject? = - if (o.has("subtitle") && o.get("subtitle") != JSONObject.NULL) o.optJSONObject("subtitle") else relay.subtitleInfo() + /** The relay's photo is loaded as a picture, anything else as the stream; the page does not say which. */ + private fun typeOf(url: String): String = + if (url == relay.imageUrl) relay.imageType ?: "image/jpeg" else "video/mp4" + + /** As main.js: no subtitle named means the relay's current one; a photo has none. */ + private fun subtitleFor(url: String, o: JSONObject): JSONObject? = when { + url == relay.imageUrl -> null + o.has("subtitle") && o.get("subtitle") != JSONObject.NULL -> o.optJSONObject("subtitle") + else -> relay.subtitleInfo() + } /** * The Wi-Fi (or Ethernet) address, never the mobile network's: a TV cannot diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt index f574c89..56a1f5e 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt @@ -10,6 +10,7 @@ import androidx.mediarouter.media.MediaRouter import com.google.android.gms.cast.CastMediaControlIntent import com.google.android.gms.cast.MediaInfo import com.google.android.gms.cast.MediaLoadRequestData +import com.google.android.gms.cast.MediaMetadata import com.google.android.gms.cast.MediaStatus import com.google.android.gms.cast.MediaTrack import com.google.android.gms.cast.TextTrackStyle @@ -111,7 +112,7 @@ class CastControl(private val context: Context) { } /** Connect, launch the default receiver, load; answers once the receiver has. */ - fun connect(deviceId: String, mediaUrl: String, subtitle: JSONObject?): JSONObject { + fun connect(deviceId: String, mediaUrl: String, subtitle: JSONObject?, contentType: String = "video/mp4"): JSONObject { need() val session = onMain { val cast = CastContext.getSharedInstance(context) @@ -167,7 +168,7 @@ class CastControl(private val context: Context) { // device's name included: read it there, never from this worker. val name = deviceName ?: onMain { session.castDevice?.friendlyName } Log.i(TAG, "session up on '$name', loading the relay stream") - val state = load(session, mediaUrl, subtitle) + val state = load(session, mediaUrl, subtitle, contentType) watch(session) return JSONObject().put("deviceName", name ?: JSONObject.NULL).put("playerState", state) } @@ -200,18 +201,24 @@ class CastControl(private val context: Context) { watched = c } - fun reload(mediaUrl: String, subtitle: JSONObject?): JSONObject { + fun reload(mediaUrl: String, subtitle: JSONObject?, contentType: String = "video/mp4"): JSONObject { need() val session = onMain { CastContext.getSharedInstance(context).sessionManager.currentCastSession } ?: throw Refused("Not connected") - val state = load(session, mediaUrl, subtitle) + val state = load(session, mediaUrl, subtitle, contentType) watch(session) return JSONObject().put("playerState", state) } - private fun load(session: CastSession, mediaUrl: String, subtitle: JSONObject?): String { + private fun load(session: CastSession, mediaUrl: String, subtitle: JSONObject?, contentType: String): String { + val photo = contentType.startsWith("image/") val subUrl = subtitle?.optString("url", "")?.takeIf { it.isNotEmpty() } - val info = MediaInfo.Builder(mediaUrl) + val info = if (photo) { + // A picture, not a stream: nothing to seek, nothing to buffer. + MediaInfo.Builder(mediaUrl).setContentType(contentType) + .setStreamType(MediaInfo.STREAM_TYPE_NONE) + .setMetadata(MediaMetadata(MediaMetadata.MEDIA_TYPE_PHOTO)).build() + } else MediaInfo.Builder(mediaUrl) .setContentType("video/mp4") // LIVE: the relay has no beginning to seek back to — the film's own // timeline lives on this side, and a seek restarts the relay. @@ -258,6 +265,8 @@ class CastControl(private val context: Context) { throw IllegalStateException("The receiver did not load the stream") } if (!ok) throw IllegalStateException("The receiver refused the stream ($reason)") + // A photo has no playing state to wait for: loaded is shown. + if (photo) return onMain { stateName(session.remoteMediaClient?.playerState ?: MediaStatus.PLAYER_STATE_UNKNOWN) } // A load the receiver accepted is not a film on the screen: it answers // IDLE, then fetches the stream, and only then plays — or gives up. // So the answer waits for what the receiver actually did (measured diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt index 21328db..41bf4bd 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt @@ -82,6 +82,7 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir private val spoolSeq = AtomicLong() private class Subtitle(val vtt: ByteArray, val language: String, val label: String) + private class Image(val bytes: ByteArray, val type: String) @Volatile private var server: ServerSocket? = null @Volatile private var port = 0 @@ -92,6 +93,8 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir @Volatile private var headerReady = false @Volatile private var subtitle: Subtitle? = null @Volatile private var subtitleVersion = 0 + @Volatile private var image: Image? = null + @Volatile private var imageVersion = 0 /** How many times a receiver asked for the stream since the relay started. */ @Volatile var streamRequests = 0 private set @@ -113,6 +116,27 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir if (server == null || subtitle == null) null else "http://${hostPart()}:$port/subs.vtt?t=$token&v=$subtitleVersion" + /** Versioned like the subtitle: a receiver handed the same address twice shows what it has. */ + val imageUrl get() = + if (server == null || image == null) null + else "http://${hostPart()}:$port/image?t=$token&v=$imageVersion" + + val imageType get() = image?.type + + /** + * One photo, scaled and re-encoded by the page, served at `/image` until + * the next one. The relay starts for it when nothing else has. + */ + @Synchronized + fun showImage(bytes: ByteArray, type: String): JSONObject { + if (type !in IMAGE_TYPES) throw IllegalArgumentException("Not a picture a receiver shows: $type") + if (server == null) start(null, null) + image = Image(bytes, type) + imageVersion++ + log("image set: ${bytes.size} bytes, $type, v$imageVersion") + return JSONObject().put("url", imageUrl) + } + fun subtitleInfo(): JSONObject? = subtitle?.let { JSONObject().put("url", subtitleUrl).put("language", it.language).put("label", it.label) } @@ -145,6 +169,7 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir spoolDir.mkdirs() spoolDir.listFiles()?.forEach { it.delete() } subtitle = null + image = null setSubtitle(sub) var bound: ServerSocket? = null @@ -251,7 +276,7 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir clients.clear() server?.let { try { it.close() } catch (e: Exception) {} } server = null - port = 0; token = null; initSegment = null; subtitle = null + port = 0; token = null; initSegment = null; subtitle = null; image = null synchronized(ring) { ring.clear(); ringBytes = 0 } accum.reset() } @@ -291,6 +316,7 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir if (token == null || query["t"] != token) return respond(out, socket, 403, emptyMap()) when (path) { "/subs.vtt" -> serveSubtitle(out, socket) + "/image" -> serveImage(out, socket) "/stream.mp4" -> { streamRequests++; serveStream(out, socket) } else -> respond(out, socket, 404, emptyMap()) } @@ -308,6 +334,15 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir ), sub.vtt) } + private fun serveImage(out: OutputStream, socket: Socket) { + val img = image ?: return respond(out, socket, 404, CORS_HEADERS) + respond(out, socket, 200, CORS_HEADERS + mapOf( + "Content-Type" to img.type, + "Content-Length" to img.bytes.size.toString(), + "Cache-Control" to "no-store", + ), img.bytes) + } + private fun serveStream(out: OutputStream, socket: Socket) { // Same headers as the subtitle: a receiver given a side-loaded track // reads the media through the same CORS-checked path. @@ -393,6 +428,8 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir } const val RING_CAP = 64 + // The page re-encodes every photo to JPEG: a guard, not a menu. + val IMAGE_TYPES = setOf("image/jpeg", "image/png", "image/webp") const val RING_MAX_BYTES = 32L * 1024 * 1024 const val SPOOL_MAX_BYTES = 2L * 1024 * 1024 * 1024 private const val HEADER_WAIT_MS = 10_000L diff --git a/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt b/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt index 5d3125f..8013e29 100644 --- a/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt +++ b/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt @@ -158,6 +158,34 @@ class CastRelayTest { assertEquals(200, get(r.getString("url"), readBytes = 0).status) } + @Test fun `a photo starts the relay and is served behind the token, re-addressed when it changes`() { + val jpeg = byteArrayOf(0xFF.toByte(), 0xD8.toByte(), 1, 2, 3, 0xFF.toByte(), 0xD9.toByte()) + val first = relay.showImage(jpeg, "image/jpeg").getString("url") + assertTrue(relay.active) + val reply = get(first) + assertEquals(200, reply.status) + assertEquals("image/jpeg", reply.headers["content-type"]) + assertEquals("no-store", reply.headers["cache-control"]) + assertArrayEquals(jpeg, reply.body) + assertEquals(403, get(first.replace(Regex("t=[0-9a-f]+"), "t=x")).status) + // A receiver handed the same address twice shows what it already has. + val second = relay.showImage(byteArrayOf(9), "image/jpeg").getString("url") + assertNotEquals(first, second) + assertArrayEquals(byteArrayOf(9), get(second).body) + } + + @Test fun `only a picture is served as one`() { + val refused = try { relay.showImage(byteArrayOf(1), "text/html"); false } catch (e: IllegalArgumentException) { true } + assertTrue(refused) + assertNull(relay.imageUrl) + } + + @Test fun `a film started over a photo takes the photo down`() { + relay.showImage(byteArrayOf(1), "image/jpeg") + relay.start(null, null) + assertNull(relay.imageUrl) + } + @Test fun `the preflight is answered before the token is checked`() { val url = relay.start(null, null).getString("url") val reply = get(url.substringBefore("?"), method = "OPTIONS") diff --git a/packages/meshbay-client/src/cast-chromecast.js b/packages/meshbay-client/src/cast-chromecast.js index 55deb5b..4c8ddf0 100644 --- a/packages/meshbay-client/src/cast-chromecast.js +++ b/packages/meshbay-client/src/cast-chromecast.js @@ -41,12 +41,18 @@ const TEXT_TRACK_STYLE = Object.freeze({ }); /** - * What to hand `player.load()` for a stream, with or without subtitles. + * What to hand `player.load()` for a stream, with or without subtitles, or + * for a photo. * * `streamType: 'LIVE'` because the relay has no beginning to seek back to — * the film's own timeline lives on this side, and a seek restarts the relay. + * A photo is not a stream at all. */ -function mediaFor(mediaUrl, subtitle) { +function mediaFor(mediaUrl, subtitle, contentType) { + if (contentType && contentType.startsWith('image/')) { + return { contentId: mediaUrl, contentType, streamType: 'NONE', + metadata: { metadataType: 4 } }; // PhotoMediaMetadata + } const media = { contentId: mediaUrl, contentType: 'video/mp4', @@ -139,7 +145,7 @@ class CastChromecast { } } - async connect(deviceId, mediaUrl, subtitle) { + async connect(deviceId, mediaUrl, subtitle, contentType) { const device = this._devices.get(deviceId); if (!device) throw new Error(`Unknown device: ${deviceId}`); @@ -179,7 +185,7 @@ class CastChromecast { debug(`[cast-chromecast] loading with ${subtitle?.url ? 'subtitles' : 'no subtitles'}`); const status = await new Promise((resolve, reject) => { - player.load(mediaFor(mediaUrl, subtitle), loadOptionsFor(subtitle), + player.load(mediaFor(mediaUrl, subtitle, contentType), loadOptionsFor(subtitle), (err, s) => { if (err) return reject(err); resolve(s); @@ -191,12 +197,12 @@ class CastChromecast { return { deviceName: device.name, playerState: status.playerState }; } - async reload(mediaUrl, subtitle) { + async reload(mediaUrl, subtitle, contentType) { if (!this._player) throw new Error('Not connected'); debug(`[cast-chromecast] reloading stream on "${this._connectedDevice?.name}"` + ` with ${subtitle?.url ? 'subtitles' : 'no subtitles'}`); const status = await new Promise((resolve, reject) => { - this._player.load(mediaFor(mediaUrl, subtitle), loadOptionsFor(subtitle), + this._player.load(mediaFor(mediaUrl, subtitle, contentType), loadOptionsFor(subtitle), (err, s) => { if (err) return reject(err); resolve(s); diff --git a/packages/meshbay-client/src/cast-relay.js b/packages/meshbay-client/src/cast-relay.js index 58eb9ac..c5eadc7 100644 --- a/packages/meshbay-client/src/cast-relay.js +++ b/packages/meshbay-client/src/cast-relay.js @@ -15,6 +15,10 @@ * a media element, so unlike the stream it needs CORS headers to be readable * at all. * + * A photo is served at `/image`, one at a time: the page decrypts it, scales + * it to the television and hands it here, and the receiver loads it as a + * picture. The relay starts for it when it is not already running. + * * Mitigations: * · Bind to the LAN interface, never 0.0.0.0 * · Fixed port range (19550-19553), opened only during active cast @@ -50,6 +54,9 @@ const util = require('node:util'); const debug = util.debuglog('cast-relay'); const RING_CAP = 64; +// What the receiver is ever told a photo is. The page re-encodes every photo +// to JPEG, so this list is a guard, not a menu. +const IMAGE_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp']); const BACKPRESSURE_HIGH = 8 * 1024 * 1024; const MOOF = 0x6d6f6f66; const PORT_BASE = 19550; @@ -163,6 +170,8 @@ class CastRelay { this._bytesSent = 0; this._subtitle = null; this._subtitleVersion = 0; + this._image = null; + this._imageVersion = 0; } get active() { return this._server !== null; } @@ -185,6 +194,29 @@ class CastRelay { + `?t=${this._token}&v=${this._subtitleVersion}`; } + /** + * Where the current photo can be fetched, or null when there is none. + * Versioned for the same reason as the subtitle: a receiver handed the same + * address twice shows what it already has. + */ + get imageUrl() { + if (!this._server || !this._image) return null; + return `http://${this._lanIP}:${this._port}/image` + + `?t=${this._token}&v=${this._imageVersion}`; + } + + get imageType() { return this._image ? this._image.type : null; } + + /** Serve one photo, starting the relay if nothing else has. */ + async showImage({ bytes, type }) { + if (!IMAGE_TYPES.has(type)) throw new Error(`Not a picture a receiver shows: ${type}`); + if (!this._server) await this.start({}); + this._image = { bytes: Buffer.from(bytes), type }; + this._imageVersion++; + debug(`[cast-relay] image set: ${this._image.bytes.length} bytes, ${type}, v${this._imageVersion}`); + return { url: this.imageUrl }; + } + get subtitle() { if (!this._subtitle) return null; return { @@ -232,6 +264,7 @@ class CastRelay { this._chunkCount = 0; this._bytesSent = 0; this._subtitle = null; + this._image = null; this.setSubtitle(subtitle); const server = http.createServer((req, res) => this._handle(req, res)); @@ -323,6 +356,7 @@ class CastRelay { this._token = null; this._initSegment = null; this._subtitle = null; + this._image = null; this._ring = []; this._accum.reset(); this._fragCount = 0; @@ -366,6 +400,11 @@ class CastRelay { return; } + if (url.pathname === '/image') { + this._handleImage(res); + return; + } + if (url.pathname !== '/stream.mp4') { res.writeHead(404); res.end(); @@ -419,6 +458,22 @@ class CastRelay { res.end(this._subtitle.vtt); debug(`[cast-relay] subtitle served: ${this._subtitle.vtt.length} bytes`); } + + _handleImage(res) { + if (!this._image) { + res.writeHead(404, CORS_HEADERS); + res.end(); + return; + } + res.writeHead(200, { + ...CORS_HEADERS, + 'Content-Type': this._image.type, + 'Content-Length': this._image.bytes.length, + 'Cache-Control': 'no-store', + }); + res.end(this._image.bytes); + debug(`[cast-relay] image served: ${this._image.bytes.length} bytes`); + } } module.exports = CastRelay; diff --git a/packages/meshbay-client/src/main.js b/packages/meshbay-client/src/main.js index fa4f3ec..1c371f5 100644 --- a/packages/meshbay-client/src/main.js +++ b/packages/meshbay-client/src/main.js @@ -2333,14 +2333,27 @@ function registerBridge() { handle('cast:devices', async () => castChromecast.devices()); + // One photo at a time, scaled and re-encoded by the page. + handle('cast:image', async (_e, img) => { + return castRelay.showImage({ bytes: Buffer.from(img.bytes), type: img.type }); + }); + + // The relay's photo is loaded as a picture, anything else as the stream; + // the page does not get to say which. + const loadOf = (mediaUrl, subtitle) => ( + mediaUrl && mediaUrl === castRelay.imageUrl + ? { subtitle: null, contentType: castRelay.imageType } + : { subtitle: subtitle === undefined ? castRelay.subtitle : subtitle, + contentType: 'video/mp4' }); + handle('cast:chromecast:connect', async (_e, { deviceId, mediaUrl, subtitle }) => { - return castChromecast.connect(deviceId, mediaUrl, - subtitle === undefined ? castRelay.subtitle : subtitle); + const l = loadOf(mediaUrl, subtitle); + return castChromecast.connect(deviceId, mediaUrl, l.subtitle, l.contentType); }); handle('cast:chromecast:reload', async (_e, { mediaUrl, subtitle }) => { - return castChromecast.reload(mediaUrl, - subtitle === undefined ? castRelay.subtitle : subtitle); + const l = loadOf(mediaUrl, subtitle); + return castChromecast.reload(mediaUrl, l.subtitle, l.contentType); }); // The player becomes a remote while casting: these drive the receiver. diff --git a/packages/meshbay-client/src/preload.js b/packages/meshbay-client/src/preload.js index 3af6c10..f12d68e 100644 --- a/packages/meshbay-client/src/preload.js +++ b/packages/meshbay-client/src/preload.js @@ -178,6 +178,7 @@ contextBridge.exposeInMainWorld('meshbay', { push: (data) => ipcRenderer.invoke('cast:push', data), stop: () => ipcRenderer.invoke('cast:stop'), subtitle: (sub) => ipcRenderer.invoke('cast:subtitle', sub), + image: (img) => ipcRenderer.invoke('cast:image', img), finish: () => ipcRenderer.invoke('cast:finish'), status: () => ipcRenderer.invoke('cast:status'), scan: () => ipcRenderer.invoke('cast:scan'), diff --git a/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js b/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js new file mode 100644 index 0000000..007722c --- /dev/null +++ b/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js @@ -0,0 +1,191 @@ +import { html, useState, useEffect, useRef } from './vendor/htm-preact.js'; +import { t } from './i18n.js'; +import { Icon } from './icon.js'; +import * as platform from './platform.js'; + +// ── The television this session casts to ───────────────────────────────────── +// +// docs/MESHBAY_DESIGN.md §11.4. Chosen once, from any cast button — Videos' +// toolbar, a photo album's bar, the lightbox, the player — and from then on a +// film opened plays there with the player as its remote, and a photo opened +// in the lightbox is shown there. In memory only: a television chosen +// yesterday may be off today, and choosing it again is one tap. +// +// The receiver is the default media receiver, which shows a film or a +// picture and nothing else; the interface itself stays on this device. + +let current = null; +const listeners = new Set(); + +function notify() { + for (const fn of listeners) fn(current); +} + +export const castSession = { + get device() { return current; }, + get supported() { return Boolean(platform.capabilities.lanCast && platform.cast.available); }, + choose(device) { + current = device ? { id: device.id, name: device.name } : null; + notify(); + }, + /** Forget the television without touching it: whoever stopped it already has. */ + forget() { this.choose(null); }, + /** Stop casting: the television goes back to its idle screen. */ + async end() { + this.choose(null); + photoGen++; + await platform.cast.chromecastDisconnect().catch(() => {}); + await platform.cast.stop().catch(() => {}); + }, + subscribe(fn) { + listeners.add(fn); + return () => listeners.delete(fn); + }, +}; + +export function useCastDevice() { + const [device, setDevice] = useState(castSession.device); + useEffect(() => castSession.subscribe(setDevice), []); + return device; +} + +// ── A photo on the television ──────────────────────────────────────────────── +// +// Scaled to fit 1920×1080 and re-encoded as JPEG before it leaves: a camera's +// original is 20 MB and 6000 px wide, more than a receiver decodes in good +// time, and some are formats it does not decode at all. EXIF orientation is +// applied here, so a portrait photo arrives upright and the receiver centres +// it. Photos asked for while one is still on its way are skipped to the +// newest: paging quickly through an album shows where the reader stopped, +// not every photo passed on the way. + +const TV_WIDTH = 1920; +const TV_HEIGHT = 1080; +const JPEG_QUALITY = 0.9; + +async function scaledForTv(blob) { + const bitmap = await createImageBitmap(blob, { imageOrientation: 'from-image' }); + try { + const scale = Math.min(1, TV_WIDTH / bitmap.width, TV_HEIGHT / bitmap.height); + const canvas = document.createElement('canvas'); + canvas.width = Math.max(1, Math.round(bitmap.width * scale)); + canvas.height = Math.max(1, Math.round(bitmap.height * scale)); + canvas.getContext('2d').drawImage(bitmap, 0, 0, canvas.width, canvas.height); + const jpeg = await new Promise((resolve, reject) => canvas.toBlob( + (b) => (b ? resolve(b) : reject(new Error('JPEG encoding failed'))), + 'image/jpeg', JPEG_QUALITY)); + return new Uint8Array(await jpeg.arrayBuffer()); + } finally { + bitmap.close(); + } +} + +let photoGen = 0; +let photoChain = Promise.resolve(); + +/** Show `blob` on the session's television, if there is one. */ +export function showPhoto(blob) { + const gen = ++photoGen; + photoChain = photoChain.then(async () => { + const device = current; + if (gen !== photoGen || !device || !platform.cast.canShowImages) return; + const bytes = await scaledForTv(blob); + if (gen !== photoGen) return; + const shown = await platform.cast.image({ bytes, type: 'image/jpeg' }); + if (!shown || gen !== photoGen) return; + const status = await platform.cast.status(); + if (status && status.chromecast && status.chromecast.connected) { + await platform.cast.chromecastReload({ mediaUrl: shown.url }); + } else { + await platform.cast.chromecastConnect({ deviceId: device.id, mediaUrl: shown.url }); + } + }).catch((err) => console.warn('[cast] photo not shown:', err)); + return photoChain; +} + +// ── The cast button ────────────────────────────────────────────────────────── +// +// One button, wherever it is put: with no television chosen it lists the +// receivers as they answer; with one, it says which and offers to stop. +// `overlay` is for the dark full-screen views (lightbox), `page` for a +// toolbar. `cls` and `iconCls` are the button's and its icon's own classes, +// so it matches its neighbours. +// `onChosen` runs after a television is picked, for the view that has +// something to show on it straight away. + +export function CastButton({ variant = 'page', cls = 'tb-btn tb-btn-icon', iconCls = '', onChosen }) { + const device = useCastDevice(); + const [open, setOpen] = useState(false); + const [devices, setDevices] = useState([]); + const [scanning, setScanning] = useState(false); + const wrapRef = useRef(null); + + useEffect(() => { + if (!open || device) return undefined; + setDevices([]); + setScanning(true); + return platform.cast.discover((list, still) => { + setDevices(list || []); + setScanning(still); + }); + }, [open, device]); + + // Closed by a press anywhere else, as any menu is. + useEffect(() => { + if (!open) return undefined; + const outside = (e) => { + if (wrapRef.current && !wrapRef.current.contains(e.target)) setOpen(false); + }; + document.addEventListener('pointerdown', outside); + return () => document.removeEventListener('pointerdown', outside); + }, [open]); + + if (!castSession.supported) return null; + + const overlay = variant === 'overlay'; + const menuCls = overlay ? 'cast-picker' : 'ctx-menu cast-menu'; + const itemCls = overlay ? 'cast-picker-item' : 'ctx-menu-item'; + const title = device ? `${t('cast.casting_to')} ${device.name}` : t('cast.start'); + + return html` + <div class="cast-wrapper" ref=${wrapRef}> + <button class="${cls} cast-btn ${device ? 'casting' : ''}" + onClick=${() => setOpen(!open)} title=${title} aria-label=${title} + aria-expanded=${open ? 'true' : 'false'}> + <${Icon} name="cast" cls=${iconCls} /></button> + ${open && html` + <div class=${menuCls} role="menu"> + ${device && html` + <div class="${itemCls} cast-menu-current"> + <${Icon} name="cast" /> ${t('cast.casting_to')} ${device.name} + </div> + <button class=${itemCls} role="menuitem" onClick=${() => { + setOpen(false); + castSession.end(); + }}> + <${Icon} name="close" /> ${t('cast.stop')} + </button> + `} + ${!device && devices.map((d) => html` + <button class=${itemCls} role="menuitem" key=${d.id} onClick=${() => { + setOpen(false); + castSession.choose(d); + if (onChosen) onChosen(d); + }}> + <${Icon} name="cast" /> ${d.name} + </button> + `)} + ${!device && scanning && html` + <div class="${itemCls} cast-picker-scanning"> + <span class="spinner" style="width:14px;height:14px"></span> + ${t('cast.scanning')} + </div> + `} + ${!device && !scanning && devices.length === 0 && html` + <div class="${itemCls} cast-picker-empty">${t('cast.no_devices')}</div> + `} + </div> + `} + </div> + `; +} diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js index bdc4dd5..c72de49 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/de.js @@ -377,6 +377,8 @@ export default { 'photo.prev': 'Vorherige (←)', 'photo.next': 'Nächste (→)', 'photo.zip_album': 'Album als ZIP herunterladen', + 'photo.slideshow': 'Diashow', + 'photo.slideshow_stop': 'Diashow beenden', 'photo.zoom_in': 'Vergrößern', 'photo.zoom_out': 'Verkleinern', 'photo.zoom_fit_title': 'An Fenster anpassen', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js index 658f388..a799b34 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/en.js @@ -375,6 +375,8 @@ export default { 'photo.prev': 'Previous (←)', 'photo.next': 'Next (→)', 'photo.zip_album': 'Download album as zip', + 'photo.slideshow': 'Slideshow', + 'photo.slideshow_stop': 'Stop slideshow', 'photo.zoom_in': 'Zoom in', 'photo.zoom_out': 'Zoom out', 'photo.zoom_fit_title': 'Fit to window', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js index 9e73017..8e49240 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/es.js @@ -375,6 +375,8 @@ export default { 'photo.prev': 'Anterior (←)', 'photo.next': 'Siguiente (→)', 'photo.zip_album': 'Descargar álbum como zip', + 'photo.slideshow': 'Presentación', + 'photo.slideshow_stop': 'Detener la presentación', 'photo.zoom_in': 'Acercar', 'photo.zoom_out': 'Alejar', 'photo.zoom_fit_title': 'Ajustar a la ventana', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js index 5d18d41..1432f11 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/fr.js @@ -376,6 +376,8 @@ export default { 'photo.prev': 'Précédent (←)', 'photo.next': 'Suivant (→)', 'photo.zip_album': "Télécharger l'album en zip", + 'photo.slideshow': 'Diaporama', + 'photo.slideshow_stop': 'Arrêter le diaporama', 'photo.zoom_in': 'Zoomer', 'photo.zoom_out': 'Dézoomer', 'photo.zoom_fit_title': 'Ajuster à la fenêtre', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js index c4d2acc..579f8da 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/it.js @@ -376,6 +376,8 @@ export default { 'photo.prev': 'Precedente (←)', 'photo.next': 'Successiva (→)', 'photo.zip_album': "Scarica l'album come zip", + 'photo.slideshow': 'Presentazione', + 'photo.slideshow_stop': 'Interrompi la presentazione', 'photo.zoom_in': 'Ingrandisci', 'photo.zoom_out': 'Riduci', 'photo.zoom_fit_title': 'Adatta alla finestra', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js index fe52bfa..cea7bfc 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/ja.js @@ -373,6 +373,8 @@ export default { 'photo.prev': '前へ（←）', 'photo.next': '次へ（→）', 'photo.zip_album': 'アルバムをzipでダウンロード', + 'photo.slideshow': 'スライドショー', + 'photo.slideshow_stop': 'スライドショーを停止', 'photo.zoom_in': '拡大', 'photo.zoom_out': '縮小', 'photo.zoom_fit_title': 'ウィンドウに合わせる', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js index 90b71f0..007154a 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/nl.js @@ -377,6 +377,8 @@ export default { 'photo.prev': 'Vorige (←)', 'photo.next': 'Volgende (→)', 'photo.zip_album': 'Album downloaden als zip', + 'photo.slideshow': 'Diavoorstelling', + 'photo.slideshow_stop': 'Diavoorstelling stoppen', 'photo.zoom_in': 'Inzoomen', 'photo.zoom_out': 'Uitzoomen', 'photo.zoom_fit_title': 'Passend maken', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js index 8c31cf3..7969cd0 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pl.js @@ -388,6 +388,8 @@ export default { 'photo.prev': 'Poprzednie (←)', 'photo.next': 'Następne (→)', 'photo.zip_album': 'Pobierz album jako zip', + 'photo.slideshow': 'Pokaz slajdów', + 'photo.slideshow_stop': 'Zatrzymaj pokaz slajdów', 'photo.zoom_in': 'Powiększ', 'photo.zoom_out': 'Pomniejsz', 'photo.zoom_fit_title': 'Dopasuj do okna', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js index 2f951ee..52d9d4c 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/pt-BR.js @@ -377,6 +377,8 @@ export default { 'photo.prev': 'Anterior (←)', 'photo.next': 'Próxima (→)', 'photo.zip_album': 'Baixar álbum como zip', + 'photo.slideshow': 'Apresentação de slides', + 'photo.slideshow_stop': 'Parar a apresentação', 'photo.zoom_in': 'Aumentar zoom', 'photo.zoom_out': 'Diminuir zoom', 'photo.zoom_fit_title': 'Ajustar à janela', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js index ea02545..3739dcf 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/locales/zh-CN.js @@ -370,6 +370,8 @@ export default { 'photo.prev': '上一张（←）', 'photo.next': '下一张（→）', 'photo.zip_album': '将相册下载为 zip', + 'photo.slideshow': '幻灯片放映', + 'photo.slideshow_stop': '停止幻灯片放映', 'photo.zoom_in': '放大', 'photo.zoom_out': '缩小', 'photo.zoom_fit_title': '适应窗口', diff --git a/packages/meshbay-hub/src/meshbay_hub/static/photos-app.js b/packages/meshbay-hub/src/meshbay_hub/static/photos-app.js index 73b8c15..00bc288 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/photos-app.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/photos-app.js @@ -9,6 +9,7 @@ import { import { transfers } from './transfers.js'; import { MediaThumb, LazyTile } from './media-tiles.js'; import { SourceTag } from './group-name.js'; +import { CastButton, castSession, useCastDevice, showPhoto } from './cast-session.js'; // ── Photos ─────────────────────────────────────────────────────────────────── // @@ -21,6 +22,8 @@ import { SourceTag } from './group-name.js'; // keyboard arrows, EXIF info when present) and a "zip this album" button // that reuses Files' own zip mechanism unchanged (file-utils.js's // downloadDirectory, lifted out of files-app.js for exactly this reuse). +// With a television chosen (cast-session.js), the photo the lightbox shows +// is the photo the television shows, and a slideshow pages through both. function underAnyPhotoRoot(entry, photoRoots) { const p = entry.path || ''; @@ -133,6 +136,10 @@ function PhotoTile({ entry, transportRef, gekRef, onOpen }) { // _thumbBlobCache — clicking back and forth between two photos decrypts // each once, not once per visit. const _fullBlobCache = new Map(); +// The same photos as blobs, for the television: a cast scales the picture +// itself, and reading it back through its URL would be a fetch the page's +// content policy has no reason to allow. +const _fullBlobs = new Map(); // Zoom is only ever meaningful for the lightbox's own full-resolution // image — nowhere else in the app shows one, so there is nothing to gate @@ -142,6 +149,10 @@ const ZOOM_STEP = 25; const ZOOM_MIN = 25; const ZOOM_MAX = 400; +// How long a slideshow stays on a photo, counted from when it has loaded: +// a slow node must not skip a photo nobody saw. +const SLIDESHOW_MS = 5000; + function Lightbox({ photos, index, transportRef, gekRef, onClose, onNav }) { const entry = photos[index]; const tRef = entry._tRef || transportRef; @@ -155,7 +166,9 @@ function Lightbox({ photos, index, transportRef, gekRef, onClose, onNav }) { // same way the browser renders the <img> itself. const [zoomPercent, setZoomPercent] = useState(null); const [naturalSize, setNaturalSize] = useState(null); + const [slideshow, setSlideshow] = useState(false); const slotRef = useRef(null); + const castDevice = useCastDevice(); useEffect(() => { const cached = _fullBlobCache.get(entry.id); @@ -171,8 +184,10 @@ function Lightbox({ photos, index, transportRef, gekRef, onClose, onNav }) { const chunks = await pipelinedDownload( transport, gRef.current, entry.id, totalChunks); if (cancelled) return; - const url = URL.createObjectURL(new Blob(chunks)); + const blob = new Blob(chunks); + const url = URL.createObjectURL(blob); _fullBlobCache.set(entry.id, url); + _fullBlobs.set(entry.id, blob); setBlobUrl(url); } catch { /* leave the placeholder — a transient fetch failure isn't fatal, next/close still work */ @@ -187,6 +202,23 @@ function Lightbox({ photos, index, transportRef, gekRef, onClose, onNav }) { // level chosen for one picture saying nothing about the next. useEffect(() => { setZoomPercent(null); setNaturalSize(null); }, [entry.id]); + // The television follows the lightbox, from the moment one is chosen. + useEffect(() => { + const blob = _fullBlobs.get(entry.id); + if (castDevice && blobUrl && blob) showPhoto(blob); + }, [castDevice, blobUrl, entry.id]); + + // Next photo once this one has been on screen for SLIDESHOW_MS; the last + // photo ends the slideshow rather than wrapping round. + useEffect(() => { + if (!slideshow || !blobUrl) return undefined; + const id = setTimeout(() => { + if (index < photos.length - 1) onNav(1); + else setSlideshow(false); + }, SLIDESHOW_MS); + return () => clearTimeout(id); + }, [slideshow, blobUrl, index, photos.length, onNav]); + useEffect(() => { const onKey = (e) => { if (e.key === 'Escape') onClose(); @@ -245,6 +277,13 @@ function Lightbox({ photos, index, transportRef, gekRef, onClose, onNav }) { <button class="photo-icon-btn photo-icon-btn-text ${zoomPercent === 100 ? 'active' : ''}" disabled=${!blobUrl} onClick=${zoomActual} title=${t('photo.zoom_100')}>100%</button> </div> + <button class="video-close ${slideshow ? 'cast-active' : ''}" + disabled=${photos.length < 2} + onClick=${() => setSlideshow(!slideshow)} + title=${slideshow ? t('photo.slideshow_stop') : t('photo.slideshow')} + aria-pressed=${slideshow ? 'true' : 'false'}> + <${Icon} name=${slideshow ? 'pause' : 'play'} /></button> + <${CastButton} variant="overlay" cls="video-close" /> <button class="video-close" onClick=${onClose} title=${t('video.close')}> <${Icon} name="close" /></button> </div> @@ -300,8 +339,14 @@ function AlbumView({ album, entries, transportRef, gekRef, setError, onBack, rea ${year && html`<span class="photo-album-heading-year">${year}</span>`} </div> </div> - ${!readOnly && html`<button class="photo-icon-btn" onClick=${zip} title=${t('photo.zip_album')}> - <${Icon} name="archive" cls="photo-icon-btn-icon" /></button>`} + <div class="photo-album-actions"> + ${/* Choosing a television from here shows the album on it: the first + photo opens, since nothing is on screen yet to send. */''} + <${CastButton} cls="photo-icon-btn" iconCls="photo-icon-btn-icon" + onChosen=${() => setLightboxIndex((i) => (i === null ? 0 : i))} /> + ${!readOnly && html`<button class="photo-icon-btn" onClick=${zip} title=${t('photo.zip_album')}> + <${Icon} name="archive" cls="photo-icon-btn-icon" /></button>`} + </div> </div> <div class="photo-grid"> ${album.photos.map((e, i) => html` @@ -353,17 +398,19 @@ function PhotosApp({ <p class="page-message">${t('photo.no_roots_configured')}</p> `} ${status === 'connected' && (photoDirectories || []).length > 0 && !openAlbum && html` - ${/* The filter is the only thing in it, so under `hideFilter` there is - no toolbar rather than an empty one — an empty band still pins, - and would hold a strip of the page open under the search field - for nothing. */ - !hideFilter && html` + ${/* The filter and the cast button are all it holds, so with neither — + Search hides the filter, a browser cannot cast — there is no + toolbar rather than an empty one: an empty band still pins, and + would hold a strip of the page open under the search field for + nothing. */ + (!hideFilter || castSession.supported) && html` <div class="photo-toolbar"> - <div class="tb-search"> + <${CastButton} /> + ${!hideFilter && html`<div class="tb-search"> <${Icon} name="search" /> <input type="text" placeholder="${t('group.filter')}" value=${filter} onInput=${(e) => setFilter(e.target.value)} /> - </div> + </div>`} </div> `} ${filteredAlbums.length === 0 && html` diff --git a/packages/meshbay-hub/src/meshbay_hub/static/platform.js b/packages/meshbay-hub/src/meshbay_hub/static/platform.js index 0174edc..3a0c628 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/platform.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/platform.js @@ -460,6 +460,15 @@ export const cast = { if (!bridge || !bridge.cast || !bridge.cast.subtitle) return null; return bridge.cast.subtitle(sub); }, + // A photo for the receiver: `{ bytes, type }`, already scaled to the + // television. Absent from an older shell, so callers check `canShowImages`. + get canShowImages() { + return Boolean(bridge && bridge.cast && bridge.cast.image); + }, + async image(img) { + if (!this.canShowImages) return null; + return bridge.cast.image(img); + }, async finish() { if (!bridge || !bridge.cast) return false; return bridge.cast.finish(); diff --git a/packages/meshbay-hub/src/meshbay_hub/static/style.css b/packages/meshbay-hub/src/meshbay_hub/static/style.css index e1e152e..9be414f 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/style.css +++ b/packages/meshbay-hub/src/meshbay_hub/static/style.css @@ -2043,7 +2043,21 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } cursor: text; outline: none; } .cast-url-input:focus { border-color: rgba(59, 130, 246, 0.6); } -.cast-wrapper { display: inline-block; } +.cast-wrapper { display: inline-block; position: relative; } +.video-close:disabled { opacity: 0.4; cursor: default; } +.video-close.cast-btn.casting { background: rgba(59, 130, 246, 0.35); } +.video-close.cast-btn.casting:hover { background: rgba(59, 130, 246, 0.5); } +.cast-picker-item.cast-menu-current { color: #94a3b8; cursor: default; } +.cast-picker-item.cast-menu-current:hover { background: none; } + +/* The cast button in a page's toolbar (cast-session.js): the same panel as + any other menu there, hung under the button rather than at a point. */ +.cast-btn.casting { color: var(--accent); border-color: var(--accent); } +.ctx-menu.cast-menu { position: absolute; top: calc(100% + 4px); right: 0; z-index: 150; } +.ctx-menu-item.cast-menu-current { color: var(--text-dim); cursor: default; } +.ctx-menu-item.cast-menu-current:hover { background: none; } +.ctx-menu-item svg { width: 16px; height: 16px; flex-shrink: 0; } +.photo-album-actions { display: flex; align-items: center; gap: 4px; } .cast-picker { position: absolute; top: calc(100% + 4px); right: 0; z-index: 100; background: rgba(28, 28, 32, 0.97); border: 1px solid rgba(255, 255, 255, 0.14); diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-app.js b/packages/meshbay-hub/src/meshbay_hub/static/video-app.js index 4757395..aa1edf2 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-app.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-app.js @@ -7,6 +7,7 @@ import { formatSize } from './file-utils.js'; import { SourceTag } from './group-name.js'; import { usePager, Pager, pageSizeFrom } from './pager.js'; import { LazyTile, MediaThumb } from './media-tiles.js'; +import { CastButton } from './cast-session.js'; // ── Videos ─────────────────────────────────────────────────────────────────── // @@ -1083,6 +1084,8 @@ function VideoApp({ </button> </div> <${Pager} pager=${pager} /> + ${/* The television a film opened from here plays on (cast-session.js). */''} + <${CastButton} /> ${!hideFilter && html`<div class="tb-search"> <${Icon} name="search" /> <input type="text" placeholder="${t('group.filter')}" diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js index 2c424fb..ed1f18c 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js @@ -6,6 +6,7 @@ import { Icon } from './icon.js'; import { formatSize, CHUNK_SIZE, pipelinedDownload } from './file-utils.js'; import { loadAuth } from './hub-client.js'; import * as platform from './platform.js'; +import { castSession } from './cast-session.js'; // Seconds of already-watched video kept in the SourceBuffer, and the queue depth // past which we start making room before being forced to. @@ -484,17 +485,22 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { // Where the stream the node is sending begins, in film time. The same number // the SourceBuffer gets as its `timestampOffset`. const streamStartRef = useRef(0); - const [castActive, setCastActive] = useState(false); + // A television already chosen this session (cast-session.js) takes the film + // from its first segment: the cast starts as a restart would, pending until + // the init segment arrives, and the player opens as its remote. Read once — + // a television chosen while this film plays is chosen here, by the picker. + const [autoCast] = useState(() => (castSession.supported ? castSession.device : null)); + const [castActive, setCastActive] = useState(Boolean(autoCast)); const [castUrl, setCastUrl] = useState(null); const [castPickerOpen, setCastPickerOpen] = useState(false); const [castDevices, setCastDevices] = useState([]); const [castScanning, setCastScanning] = useState(false); - const [castDeviceName, setCastDeviceName] = useState(null); - const castActiveRef = useRef(false); + const [castDeviceName, setCastDeviceName] = useState(autoCast ? autoCast.name : null); + const castActiveRef = useRef(Boolean(autoCast)); const castCodecRef = useRef(null); const initSegmentRef = useRef(null); - const castRestartPendingRef = useRef(false); - const castDeviceRef = useRef(null); + const castRestartPendingRef = useRef(Boolean(autoCast)); + const castDeviceRef = useRef(autoCast); const castRestartGenRef = useRef(0); // The receiver as last polled: { playerState, position } with the position // on the film's timeline, or null before it has answered. @@ -1758,7 +1764,10 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { return () => { stopped = true; clearInterval(id); }; }, [remoteMode]); + // Stopping here is stopping for the session: the next film or photo stays + // on this device until a television is chosen again. const stopCast = async () => { + castSession.forget(); await platform.cast.chromecastDisconnect().catch(() => {}); await platform.cast.stop(); setCastActive(false); castActiveRef.current = false; @@ -1896,6 +1905,7 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { ${castDevices.map(d => html` <button class="cast-picker-item" onClick=${() => { setCastPickerOpen(false); + castSession.choose(d); setCastDeviceName(d.name); setCastActive(true); castActiveRef.current = true; castRestartPendingRef.current = true; diff --git a/packages/meshbay-hub/tests/harness/cast_session_probe.py b/packages/meshbay-hub/tests/harness/cast_session_probe.py new file mode 100644 index 0000000..2904860 --- /dev/null +++ b/packages/meshbay-hub/tests/harness/cast_session_probe.py @@ -0,0 +1,393 @@ +#!/usr/bin/env python3 +""" +A television chosen once, then photos and a film sent to it. + +Casting a photo or opening a film on a television chosen elsewhere is a claim +about four pieces at once: the cast button (cast-session.js), the lightbox +that follows it, the player that opens as a remote, and the native bridge the +page talks to. This renders the shipped `GroupPage` against a stub node and a +stub bridge that records what the page asks of it, and walks it as a reader +would: open an album, choose the television, page on, start a slideshow, +then open a film from Videos and stop casting from its remote. + +A second case renders the same page with no bridge at all — a browser — where +no cast button may appear. + + cast_session_probe.py Chrome, then Firefox +""" + +import http.server +import json +import shutil +import socketserver +import subprocess +import sys +import tempfile +import threading +import time +from pathlib import Path + +STATIC = Path(__file__).resolve().parents[2] / "src" / "meshbay_hub" / "static" +PORT = 8763 +RECORDS = [] +FINISHED = threading.Event() +socketserver.TCPServer.allow_reuse_address = True + +FRAME = r"""<!doctype html><html><head><meta charset=utf-8> +<link rel="stylesheet" href="/style.css"></head><body> +<nav class="nav"><div class="nav-left"><a class="nav-brand" href="#/">MeshBay</a></div></nav> +<div class="layout"><main class="main"><div id="root"></div></main></div> +<script> +// A module that fails to load never reaches its own catch: say so. +const report = (error) => parent.postMessage({ error }, '*'); +addEventListener('error', (e) => report( + `${e.message || 'load error'} ${e.filename || (e.target && e.target.src) || ''}` + + `:${e.lineno || ''}`), true); +addEventListener('unhandledrejection', + (e) => report(String((e.reason && e.reason.stack) || e.reason))); +const WITH_BRIDGE = !location.search.includes('nobridge'); +const ENTRIES = []; +for (let i = 1; i <= 3; i++) { + ENTRIES.push({ id: 'p' + i, name: 'photo' + i + '.png', path: 'photos/Trip', + type: 'image', size: 1000, added_at: 1750000000 + i }); +} +ENTRIES.push({ id: 'film', name: 'A.Film.mkv', display_title: 'A Film', + path: 'videos/films', type: 'video', duration: 6000, size: 1024, added_at: 1750000100 }); + +const ACK = { + is_node_admin: false, enabled_apps: ['video', 'photo'], + tmdb_enabled: true, tmdb_language: 'en-US', + video_directories: ['videos'], music_directories: [], photo_directories: ['photos'], +}; + +// A camera-sized picture, made here: 4000×3000, which the television must +// receive as 1440×1080. +let PHOTO = null; +async function photoBytes() { + if (PHOTO) return PHOTO; + const c = document.createElement('canvas'); + c.width = 4000; c.height = 3000; + const g = c.getContext('2d'); + g.fillStyle = '#3a7'; g.fillRect(0, 0, 4000, 3000); + const blob = await new Promise((r) => c.toBlob(r, 'image/png')); + PHOTO = new Uint8Array(await blob.arrayBuffer()); + return PHOTO; +} + +window.MeshBayCrypto = { + importGEK: async () => 'gek', + b64encode: (x) => String(x), + decryptChunkBin: async (_g, _f, _i, _n, ct) => ct, +}; + +window.MeshBayTransport = function () { + const self = { + connected: false, memberRole: 'member', supportsAppOps: true, + sessionKeys: null, gekRaw: 'k', + newNodeBundle: null, newNodeBundleRecovery: null, + async connect() { self.connected = true; return ACK; }, + async fetchIndex() { + return { entries: ENTRIES, dirs: ['photos', 'videos'], + roots: [{ name: 'photos', available: true, writable: false, removable: false }, + { name: 'videos', available: true, writable: false, removable: false }] }; + }, + async fetchMediaMeta() { return { confidence: 0 }; }, + // One chunk per photo: every photo is the same picture. + async fetchChunk() { return { nonce: 'n', ct: await photoBytes() }; }, + addReconnectListener() { return () => {}; }, + close() {}, + }; + return new Proxy(self, { + get(target, prop) { + if (prop in target) return target[prop]; + if (typeof prop === 'string' && prop.startsWith('on')) return undefined; + if (typeof prop === 'symbol') return undefined; + return () => new Promise(() => {}); + }, + set(target, prop, value) { target[prop] = value; return true; }, + }); +}; + +// The native side, as the Electron preload and the Android shim present it, +// recording every cast call. +window.CALLS = []; +if (WITH_BRIDGE) { + let connected = false, v = 0; + const rec = (name, arg) => { CALLS.push({ name, arg }); }; + window.meshbay = { + hubBase: () => '', + setLocale: async () => true, + capabilities: { lanCast: true, nodeAdmin: false, localFolders: false, nativeSave: false }, + cast: { + start: async (o) => { + rec('start', o && { codec: o.codec }); + return { url: 'http://tv/stream.mp4?t=x' }; + }, + push: async () => true, + stop: async () => { rec('stop'); return true; }, + subtitle: async () => null, + finish: async () => true, + image: async (img) => { + const bmp = await createImageBitmap(new Blob([img.bytes], { type: img.type })); + rec('image', { type: img.type, width: bmp.width, height: bmp.height, + jpeg: img.bytes[0] === 0xff && img.bytes[1] === 0xd8 }); + return { url: 'http://tv/image?t=x&v=' + (++v) }; + }, + status: async () => ({ active: true, url: null, chromecast: { connected } }), + scan: async () => { rec('scan'); return true; }, + devices: async () => ({ devices: [{ id: 'tv1', name: 'Salon' }], scanning: false }), + chromecastConnect: async (o) => { + rec('connect', o); connected = true; + return { deviceName: 'Salon' }; + }, + chromecastReload: async (o) => { rec('reload', o); return { playerState: 'IDLE' }; }, + chromecastDisconnect: async () => { rec('disconnect'); connected = false; return true; }, + chromecastPause: async () => ({}), + chromecastPlay: async () => ({}), + }, + }; +} +</script> +<script type="module"> +import { html, render } from '/vendor/htm-preact.js'; +import { initLocale } from '/i18n.js'; +import { GroupPage } from '/group-page.js'; + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +const $ = (sel) => document.querySelector(sel); +const $$ = (sel) => [...document.querySelectorAll(sel)]; +async function until(pred, what, tries = 200) { + for (let i = 0; i < tries; i++) { if (pred()) return; await sleep(50); } + throw new Error('timed out waiting for ' + what); +} +const calls = (name) => CALLS.filter((c) => c.name === name); +// Files is always first; then the enabled apps in their registry order. +const videosTab = () => $$('.group-tab')[1]; +const count = () => (($('.photo-lightbox-count') || {}).textContent || '').trim(); + +(async () => { + const out = { bridge: !!window.meshbay }; + try { + await initLocale(); + render(html`<${GroupPage} groupId="g1" token="t" username="me" userId="u1" + group=${{ id: 'g1', name: 'a group', owner_username: 'me', is_admin: false }} + userPrefs=${{ default_tab: 'photo', media_page_size: '50' }} />`, + document.getElementById('root')); + + await until(() => $('.photo-album-card'), 'the album card'); + out.landingButton = !!$('.photo-toolbar .cast-btn'); + $('.photo-album-card').click(); + await until(() => $('.photo-album-bar'), 'the album'); + out.albumButton = !!$('.photo-album-bar .cast-btn'); + + if (!window.meshbay) { + videosTab().click(); + await until(() => $('.video-toolbar'), 'the videos toolbar'); + out.videosButton = !!$('.video-toolbar .cast-btn'); + parent.postMessage(out, '*'); + return; + } + + // Choose the television from the album: the first photo opens and goes. + $('.photo-album-bar .cast-btn').click(); + await until(() => $$('.cast-menu .ctx-menu-item').some((b) => b.textContent.includes('Salon')), + 'the receiver in the menu'); + $$('.cast-menu .ctx-menu-item').find((b) => b.textContent.includes('Salon')).click(); + await until(() => $('.photo-lightbox'), 'the lightbox to open on choosing'); + await until(() => calls('connect').length === 1, 'the first photo to reach the television'); + out.first = { count: count(), image: calls('image')[0], connect: calls('connect')[0].arg, + lightboxCasting: !!$('.photo-lightbox .cast-btn.casting') }; + + // Next photo: the receiver is connected, so it is reloaded. + $('.photo-nav-next').click(); + await until(() => calls('reload').length === 1, 'the second photo'); + out.second = { count: count(), reload: calls('reload')[0].arg, images: calls('image').length }; + + // Slideshow: on by itself to the last photo, then off. + const play = $$('.photo-lightbox .video-top-bar .video-close') + .find((b) => b.getAttribute('aria-pressed') !== null); + play.click(); + await until(() => count() === '3 / 3', 'the slideshow to reach the last photo', 160); + await until(() => play.getAttribute('aria-pressed') === 'false', 'the slideshow to end', 160); + out.slideshow = { count: count(), images: calls('image').length, + reloads: calls('reload').length }; + + // Close the lightbox; Videos shows the same television; a film opens as its remote. + $$('.photo-lightbox .video-top-bar .video-close').pop().click(); + await until(() => !$('.photo-lightbox'), 'the lightbox to close'); + videosTab().click(); + await until(() => $('.video-toolbar'), 'the videos toolbar'); + out.videosCasting = !!$('.video-toolbar .cast-btn.casting'); + await until(() => $('.video-card'), 'the film card'); + $('.video-card').click(); + await until(() => $('.video-detail .admin-btn'), 'the film modal'); + $('.video-detail .admin-btn').click(); + await until(() => $('.video-remote'), 'the player as a remote'); + out.remote = { device: $('.video-remote-device').textContent.trim(), + pickerOpen: !!$('.video-player-overlay .cast-picker') }; + + // Stop from the remote: forgotten for the session. + $$('.video-remote button').pop().click(); + await until(() => !$('.video-remote'), 'the remote to go'); + $$('.video-player-overlay .video-top-bar .video-close').pop().click(); + await until(() => !$('.video-player-overlay'), 'the player to close'); + await sleep(100); + out.afterStop = { videosCasting: !!$('.video-toolbar .cast-btn.casting'), + disconnects: calls('disconnect').length }; + parent.postMessage(out, '*'); + } catch (err) { + parent.postMessage({ ...out, calls: CALLS, error: String((err && err.stack) || err) }, '*'); + } +})(); +</script></body></html>""" + +# Firefox headless runs a page only while taking its screenshot, which is taken +# once the page has loaded: an image that does not answer until the case has +# reported keeps it loading (the same hold as invitation_probe.py). +HOLD_TAG = '<img src="/hold" style="position:fixed;left:-4px;top:-4px;width:1px">' + +PAGE = r"""<!doctype html><html><head><meta charset=utf-8></head> +<body style="margin:0"> +<iframe src="/case{query}" style="width:1100px;height:800px;border:0"></iframe> +<script> +addEventListener('message', (e) => fetch('/log', { method: 'POST', body: JSON.stringify(e.data) })); +</script>{hold}</body></html>""" + + +class H(http.server.BaseHTTPRequestHandler): + def log_message(self, *a): + pass + + def do_POST(self): + length = int(self.headers.get("Content-Length") or 0) + if self.path == "/log": + RECORDS.append(json.loads(self.rfile.read(length).decode())) + FINISHED.set() + else: + self.rfile.read(length) + self.send_response(204) + self.end_headers() + + def _send(self, body: bytes, ctype: str) -> None: + self.send_response(200) + self.send_header("Content-Type", ctype) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def do_GET(self): + path, _, query = self.path.partition("?") + if path == "/hold": + FINISHED.wait(90) + self._send(b"", "image/gif") + elif path == "/": + q = f"?{query.replace('engine=firefox', '').strip('&')}" if query else "" + hold = HOLD_TAG if "engine=firefox" in query else "" + page = PAGE.replace("{query}", q if q != "?" else "").replace("{hold}", hold) + self._send(page.encode(), "text/html; charset=utf-8") + elif path == "/case": + self._send(FRAME.encode(), "text/html; charset=utf-8") + elif path == "/v1/groups/g1/nodes": + self._send(b'{"nodes": [{"node_id": "n1"}]}', "application/json") + else: + asset = (STATIC / path.lstrip("/")).resolve() + if not str(asset).startswith(str(STATIC)) or not asset.is_file(): + self.send_response(404) + self.end_headers() + return + self._send(asset.read_bytes(), + "text/css" if asset.suffix == ".css" + else "text/javascript" if asset.suffix == ".js" + else "application/octet-stream") + + +def browse(engine: str, query: str) -> dict | None: + RECORDS.clear() + FINISHED.clear() + sep = "&" if query else "?" + url = f"http://127.0.0.1:{PORT}/{query}{sep + 'engine=firefox' if engine == 'firefox' else ''}" + # Firefox is a snap here, and needs its profile under $HOME (invitation_probe.py). + snap = Path.home() / "snap" / "firefox" / "common" + parent = (str(snap if snap.is_dir() else Path.home()) if engine == "firefox" else None) + with tempfile.TemporaryDirectory(ignore_cleanup_errors=True, prefix="meshbay-probe-", + dir=parent) as profile: + if engine == "chrome": + cmd = ["google-chrome", "--headless=new", "--disable-gpu", "--no-sandbox", + f"--user-data-dir={profile}", "--window-size=1100,900", url] + else: + cmd = ["firefox", "--headless", "--profile", profile, "--window-size=1100,900", + "--screenshot", str(Path(profile) / "shot.png"), url] + proc = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + deadline = time.time() + 90 + while not RECORDS and time.time() < deadline: + time.sleep(0.2) + proc.terminate() + try: + proc.wait(timeout=20) + except subprocess.TimeoutExpired: + proc.kill() + proc.wait() + return RECORDS[0] if RECORDS else None + + +def check(engine: str, cast: dict, plain: dict) -> list[str]: + bad = [] + if "error" in cast: + return [f"{engine}: {cast['error']}"] + if not cast["landingButton"]: + bad.append("no cast button on the Photos landing") + if not cast["albumButton"]: + bad.append("no cast button in the album bar") + first = cast["first"] + if first["count"] != "1 / 3": + bad.append(f"choosing a television did not open the first photo: {first['count']}") + img = first["image"]["arg"] + if not (img["type"] == "image/jpeg" and img["jpeg"] and img["width"] == 1440 + and img["height"] == 1080): + bad.append(f"the photo did not leave as a 1440x1080 JPEG: {img}") + if first["connect"] != {"deviceId": "tv1", "mediaUrl": "http://tv/image?t=x&v=1"}: + bad.append(f"connect: {first['connect']}") + if not first["lightboxCasting"]: + bad.append("the lightbox's cast button does not say it is casting") + if cast["second"]["count"] != "2 / 3" or cast["second"]["reload"] != { + "mediaUrl": "http://tv/image?t=x&v=2"}: + bad.append(f"next photo: {cast['second']}") + if cast["slideshow"]["count"] != "3 / 3" or cast["slideshow"]["images"] != 3: + bad.append(f"slideshow: {cast['slideshow']}") + if not cast["videosCasting"]: + bad.append("Videos does not show the television chosen in Photos") + if cast["remote"]["device"] != "Salon" or cast["remote"]["pickerOpen"]: + bad.append(f"the film did not open as the television's remote: {cast['remote']}") + if cast["afterStop"]["videosCasting"] or cast["afterStop"]["disconnects"] < 1: + bad.append(f"stopping from the remote: {cast['afterStop']}") + if "error" in plain: + bad.append(f"no bridge: {plain['error']}") + elif plain["landingButton"] or plain["albumButton"] or plain["videosButton"]: + bad.append(f"a cast button without a bridge: {plain}") + return [f"{engine}: {b}" for b in bad] + + +def main() -> int: + failures = [] + with socketserver.ThreadingTCPServer(("127.0.0.1", PORT), H) as srv: + threading.Thread(target=srv.serve_forever, daemon=True).start() + for engine in ("chrome", "firefox"): + if shutil.which("google-chrome" if engine == "chrome" else engine) is None: + print(f"{engine}: not installed, skipped") + continue + cast = browse(engine, "") + plain = browse(engine, "?nobridge") + if cast is None or plain is None: + failures.append(f"{engine}: the page never reported") + continue + print(f"{engine}:", json.dumps(cast, indent=1)) + failures += check(engine, cast, plain) + for f in failures: + print("FAIL", f, file=sys.stderr) + if not failures: + print("OK") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/packages/meshbay-hub/tests/test_android_cast.py b/packages/meshbay-hub/tests/test_android_cast.py index bc5264e..b537df7 100644 --- a/packages/meshbay-hub/tests/test_android_cast.py +++ b/packages/meshbay-hub/tests/test_android_cast.py @@ -103,7 +103,7 @@ def test_the_receiver_is_pointed_at_the_relay_and_nothing_else(): def test_relay_calls_keep_their_order(): """The page does not await each push; on a pool they could overtake.""" channels = _read(CAST / "CastChannels.kt") - assert '"cast:start", "cast:push", "cast:subtitle", "cast:finish", "cast:stop"' in channels + assert '"cast:start", "cast:push", "cast:subtitle", "cast:image", "cast:finish", "cast:stop"' in channels bridge = _read(SRC / "bridge" / "Bridge.kt") assert "Executors.newSingleThreadExecutor()" in bridge assert "(if (ordered) serial else work).execute" in bridge diff --git a/packages/meshbay-hub/tests/test_cast_photos.py b/packages/meshbay-hub/tests/test_cast_photos.py new file mode 100644 index 0000000..699e663 --- /dev/null +++ b/packages/meshbay-hub/tests/test_cast_photos.py @@ -0,0 +1,240 @@ +""" +Photos on a cast receiver, and the television a session casts to. + +A photo takes a different road to the television than a film: no stream, one +picture at a time, served by the relay at `/image` and loaded as a picture by +the default media receiver. The page scales and re-encodes it first +(cast-session.js), so what reaches the relay is a JPEG the receiver can show. + +The first half runs the desktop relay itself, as test_cast_subtitles.py does: +it is plain Node with no dependencies. The photo is behind the same token as +the stream, its address changes with every photo — a receiver handed the same +URL twice shows what it already has — and only a picture is served as one. + +The second half holds the wiring: the receiver is told a photo is a photo by +the shell, never by the page; Android accepts no URL but the relay's own; and +a television chosen anywhere is the one a film opens on, with the player as +its remote from the start. +""" + +import json +import re +import shutil +import subprocess +from pathlib import Path + +import pytest + +PACKAGES = Path(__file__).resolve().parents[2] +STATIC = PACKAGES / "meshbay-hub" / "src" / "meshbay_hub" / "static" +DESKTOP = PACKAGES / "meshbay-client" / "src" +RELAY = DESKTOP / "cast-relay.js" +CHROMECAST = DESKTOP / "cast-chromecast.js" +ANDROID_SRC = PACKAGES / "meshbay-android" / "app" / "src" / "main" / "kotlin" +CAST = ANDROID_SRC / "org" / "meshbay" / "client" / "cast" + + +def _read(path: Path) -> str: + return path.read_text(encoding="utf-8") + + +RELAY_SCRIPT = r""" +const http = require('node:http'); +const CastRelay = require(process.argv[2]); +const get = (url) => new Promise((resolve, reject) => { + http.get(url, (res) => { + const parts = []; + res.on('data', (c) => parts.push(c)); + res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, + body: Buffer.concat(parts).toString('hex') })); + }).on('error', reject); +}); +(async () => { + const relay = new CastRelay(); + const out = {}; + out.activeBefore = relay.active; + try { + await relay.showImage({ bytes: Buffer.from('<p>'), type: 'text/html' }); + out.htmlRefused = false; + } catch { out.htmlRefused = true; } + out.activeAfterRefusal = relay.active; + + const jpeg = Buffer.from([0xff, 0xd8, 1, 2, 0xff, 0xd9]); + const first = await relay.showImage({ bytes: jpeg, type: 'image/jpeg' }); + out.activeAfterPhoto = relay.active; + out.firstUrl = first.url; + out.imageType = relay.imageType; + out.photo = await get(first.url); + out.badToken = (await get(first.url.replace(/t=[0-9a-f]+/, 't=x'))).status; + const second = await relay.showImage({ bytes: Buffer.from([9]), type: 'image/jpeg' }); + out.secondUrl = second.url; + out.second = await get(second.url); + + await relay.start({}); + out.imageAfterFilm = relay.imageUrl; + await relay.stop(); + out.imageAfterStop = relay.imageUrl; + console.log(JSON.stringify(out)); +})().catch((e) => { console.error(e); process.exit(1); }); +""" + + +@pytest.fixture(scope="module") +def relay_run(tmp_path_factory): + if shutil.which("node") is None or not RELAY.exists(): + pytest.skip("node or the desktop client sources are not available") + script = tmp_path_factory.mktemp("cp") / "relay.cjs" + script.write_text(RELAY_SCRIPT, encoding="utf-8") + proc = subprocess.run(["node", str(script), str(RELAY)], + capture_output=True, text=True, encoding="utf-8", timeout=30) + assert proc.returncode == 0, proc.stderr + return json.loads(proc.stdout) + + +def test_a_photo_starts_the_relay_and_is_served_behind_its_token(relay_run): + assert relay_run["activeBefore"] is False + assert relay_run["activeAfterPhoto"] is True + photo = relay_run["photo"] + assert photo["status"] == 200 + assert photo["headers"]["content-type"] == "image/jpeg" + assert photo["headers"]["cache-control"] == "no-store" + assert photo["body"] == "ffd80102ffd9" + assert relay_run["badToken"] == 403 + assert relay_run["imageType"] == "image/jpeg" + + +def test_every_photo_has_its_own_address(relay_run): + assert re.search(r"/image\?t=[0-9a-f]{32}&v=\d+$", relay_run["firstUrl"]) + assert relay_run["secondUrl"] != relay_run["firstUrl"] + assert relay_run["second"]["body"] == "09" + + +def test_only_a_picture_is_served_as_one(relay_run): + assert relay_run["htmlRefused"] is True + assert relay_run["activeAfterRefusal"] is False, "a refused photo starts nothing" + + +def test_a_film_or_a_stop_takes_the_photo_down(relay_run): + assert relay_run["imageAfterFilm"] is None + assert relay_run["imageAfterStop"] is None + + +def test_the_receiver_loads_a_photo_as_a_picture(): + if shutil.which("node") is None or not CHROMECAST.exists(): + pytest.skip("node or the desktop client sources are not available") + script = ( + "const Module = require('node:module');" + "const real = Module._load;" + "Module._load = function (r, ...a) {" + " if (r === 'bonjour-service') return { Bonjour: class {} };" + " if (r === 'castv2-client') return { Client: class {}, DefaultMediaReceiver: {} };" + " return real.call(this, r, ...a); };" + f"const {{ mediaFor }} = require({json.dumps(str(CHROMECAST))});" + "console.log(JSON.stringify({" + " photo: mediaFor('http://x/image?t=a&v=1', null, 'image/jpeg')," + " film: mediaFor('http://x/stream.mp4?t=a', null, 'video/mp4')," + " legacy: mediaFor('http://x/stream.mp4?t=a', null) }));") + proc = subprocess.run(["node", "-e", script], capture_output=True, text=True, + encoding="utf-8", timeout=30) + assert proc.returncode == 0, proc.stderr + out = json.loads(proc.stdout) + assert out["photo"]["contentType"] == "image/jpeg" + assert out["photo"]["streamType"] == "NONE" + assert "tracks" not in out["photo"] + assert out["film"] == out["legacy"], "a film is loaded exactly as before" + assert out["film"]["streamType"] == "LIVE" and out["film"]["contentType"] == "video/mp4" + + +def test_the_shell_not_the_page_says_what_the_receiver_loads(): + """The page names a URL; whether it is a photo is the relay's to say. A + page that could name the content type could have a television load the + stream URL as anything at all.""" + main = _read(DESKTOP / "main.js") + load_of = main.split("const loadOf = ", 1)[1].split("\n\n", 1)[0] + assert "mediaUrl === castRelay.imageUrl" in load_of + assert "contentType: castRelay.imageType" in load_of + assert "contentType: 'video/mp4'" in load_of + assert "subtitle: null" in load_of, "a photo carries no subtitle" + assert main.count("loadOf(mediaUrl, subtitle)") == 2, "connect and reload both" + + +def test_android_accepts_the_relays_photo_and_nothing_else(): + if not CAST.exists(): + pytest.skip("android sources not present") + channels = _read(CAST / "CastChannels.kt") + check = channels.split("private fun relayUrl(", 1)[1].split("\n }", 1)[0] + assert "asked != ours && asked != relay.imageUrl" in check + assert "if (url == relay.imageUrl) relay.imageType" in channels + assert "url == relay.imageUrl -> null" in channels, "a photo carries no subtitle" + # A photo is not a cast that has to survive the screen going off. + image = channels.split('"cast:image" ->', 1)[1].split('\n "', 1)[0] + assert "onCasting" not in image + relay = _read(CAST / "CastRelay.kt") + assert '"/image" -> serveImage(out, socket)' in relay + serve = relay.split("private fun serve(", 1)[1] + assert serve.index('query["t"] != token') < serve.index('"/image"'), "behind the token" + control = _read(CAST / "CastControl.kt") + assert "MediaInfo.STREAM_TYPE_NONE" in control + assert "MediaMetadata.MEDIA_TYPE_PHOTO" in control + # Loaded is shown: a photo never reaches PLAYING, and waiting for it + # would hold every photo for the full fifteen seconds. + load = control.split("private fun load(", 1)[1] + assert load.index("if (photo) return") < load.index("PLAY_WAIT_MS") + + +def test_a_television_chosen_anywhere_takes_the_next_film(): + """The player opens as the remote of a television chosen before it, the + cast pending until the first segment exactly as a restart is; choosing one + in the player keeps it for the session, and stopping there forgets it.""" + player = _read(STATIC / "video-player.js") + assert ("const [autoCast] = useState(() => " + "(castSession.supported ? castSession.device : null));") in player + assert "useState(Boolean(autoCast))" in player + assert "castActiveRef = useRef(Boolean(autoCast))" in player + assert "castRestartPendingRef = useRef(Boolean(autoCast))" in player + assert "castDeviceRef = useRef(autoCast)" in player + assert "useState(autoCast ? autoCast.name : null)" in player + stop = player.split("const stopCast = async () => {", 1)[1].split("};", 1)[0] + assert "castSession.forget()" in stop + assert "castSession.choose(d);" in player + + +def test_the_cast_button_is_where_the_media_are(): + """Videos' toolbar, the Photos landing and a photo album's bar are drawn by + the group page and the search page alike, so one button in each covers + both. On the landing the button keeps a toolbar Search would otherwise + hide.""" + videos = _read(STATIC / "video-app.js") + assert "<${CastButton} />" in videos.split('<div class="video-toolbar">', 1)[1] + photos = _read(STATIC / "photos-app.js") + landing = photos.split("function PhotosApp(", 1)[1] + assert "(!hideFilter || castSession.supported) && html`" in landing + assert "<${CastButton} />" in landing.split('<div class="photo-toolbar">', 1)[1] + album = photos.split("function AlbumView(", 1)[1] + assert "<${CastButton}" in album + lightbox = photos.split("function Lightbox(", 1)[1].split("function AlbumView(", 1)[0] + assert '<${CastButton} variant="overlay"' in lightbox + assert "if (castDevice && blobUrl && blob) showPhoto(blob);" in lightbox + # The group page draws them through the app registry, the search page + # imports them: the same components either way. + registry = _read(STATIC / "apps.js") + search = _read(STATIC / "search-page.js") + for page in (registry, search): + assert "VideoApp" in page and "PhotosApp" in page + + +def test_a_photo_leaves_scaled_and_upright(): + session = _read(STATIC / "cast-session.js") + assert "imageOrientation: 'from-image'" in session + assert "const TV_WIDTH = 1920;" in session and "const TV_HEIGHT = 1080;" in session + assert "Math.min(1, TV_WIDTH / bitmap.width, TV_HEIGHT / bitmap.height)" in session + assert "'image/jpeg'" in session + # Only the newest photo asked for is sent. + assert session.count("gen !== photoGen") >= 2 + + +def test_every_locale_names_the_slideshow(): + for f in sorted((STATIC / "locales").glob("*.js")): + text = f.read_text(encoding="utf-8") + for key in ("photo.slideshow", "photo.slideshow_stop"): + assert f"'{key}':" in text, f"{f.name} lacks {key}" |