diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-10-05 02:29:27 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-10-05 02:29:27 +0200 |
| commit | ee9798fe3b88face2fa34f14874770f509c788a5 (patch) | |
| tree | f63e9506882863a885a907796eb977ec69137718 | |
| parent | 243bcf017b4957237b3ddf556bd0a37296eb8798 (diff) | |
| download | meshbay-ee9798fe3b88face2fa34f14874770f509c788a5.tar.gz | |
feat(cast): music on the TV, the music bar as its remote
A cast button in Music's toolbar and in the music bar. With a television
chosen, each decrypted track goes to the relay with its cover — found as the
album card finds it — and plays there as music with its title, artist and
album; the bar's play, pause, seek, previous and next drive the receiver, its
clock is the receiver's, and the end of a track there moves the queue on.
A film or a photo taking the television pauses the bar; stopping the cast
carries the track on locally.
Photos and tracks now share one path: a whole file sent to the relay in
pieces (binary frames on Android, written to disk there), served at /file
with byte ranges and its cover at /cover, and loaded as what the relay says
it is. cast:image is gone; cast:chromecast:seek is new.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
24 files changed, 1331 insertions, 436 deletions
@@ -1033,7 +1033,7 @@ here are kept only where they are a rule about *editing* the code. | Where the hub is | `platform.js` — `hubBase()` | **the only file allowed to decide this** (§8.3) | | Downloads, decrypt pipeline | `file-utils.js`, `downloads.js`, `sw.js` | §8.5 | | Video player | `video-player.js` — `pump()` is the only place credit is granted | §8.5 | -| The television a session casts to, photos on it | `cast-session.js` | §11.4 | +| The television a session casts to; photos and music on it | `cast-session.js`, `music-player.js` | §11.4 | | Transfers widget | `transfers.js` | §5.5 | | Cross-group merge | `source-merge.js`, `group-name.js` | §9.11 | | Node page | `node-page.js` | §6.7 | @@ -1068,7 +1068,7 @@ when `ANDROID_HOME` is set (`test_android_shell.py`). | An invitation answered on the home page (accept, decline), in both engines | `packages/meshbay-hub/tests/harness/invitation_probe.py` | | A group owner's invited list and host approvals, in both engines | `packages/meshbay-hub/tests/harness/group_hosts_probe.py` | | A show's modal and the player, walked episode to episode | `packages/meshbay-hub/tests/harness/video_series_probe.py` | -| A television chosen once, then photos and a film sent to it, in both engines | `packages/meshbay-hub/tests/harness/cast_session_probe.py` | +| A television chosen once, then photos, a film and music sent to it, in both engines | `packages/meshbay-hub/tests/harness/cast_session_probe.py` | | A lazily loaded view whose module answers 404 (a tab older than the deploy) | `packages/meshbay-hub/tests/harness/lazy_failure_probe.py` | ## meshbay.org server (target state) diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 64364b0..5d33f5f 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -3423,27 +3423,49 @@ pauses the local element too, which otherwise goes on fetching for nobody. The copy-URL cast has no receiver to read and keeps the ordinary player. **A television is chosen for the session, not for one film** -(`cast-session.js`). Any cast button — Videos' toolbar, the Photos toolbar, a -photo album's bar, the lightbox, the player — sets it, and it is held in memory -only. A film opened +(`cast-session.js`). Any cast button — Videos', Photos' and Music's toolbars, a +photo album's bar, the lightbox, the video player, the music bar — sets it, and +it is held in memory only. A film opened while one is set starts its cast as a restart does: pending until the first segment, then the relay and `connect` (or `reload`, when the receiver is already connected), with the player drawn as the remote from the start. *Stop casting* clears it, from the player's remote or from any button. -**A photo is served at `/image`, one at a time** (`cast:image`). The page scales -it to fit 1920×1080, applies its EXIF orientation and re-encodes it as JPEG -before it leaves: a camera original is too large for a receiver to decode in good -time, and some formats it does not decode at all. The relay starts for it if -nothing else has, serves it behind the stream's token with a versioned address -(a receiver handed the same URL twice shows what it already has), and accepts -only `image/jpeg`, `image/png` or `image/webp`. **The shell, not the page, says -what the receiver loads:** the relay's photo URL is loaded as a picture -(`streamType: NONE`, photo metadata, no subtitles), any other relay URL as the -stream, and on Android no URL but the relay's own is accepted at all. A photo -does not start the foreground service, and its load answers as soon as the -receiver accepts it: a photo never reaches PLAYING. Only the newest photo asked -for is sent; paging quickly shows where the reader stopped. +**A photo or a music track is served whole at `/file`, one at a time** +(`cast:file:begin`, `:write`, `:end`), its cover at `/cover`. It crosses in +pieces — binary frames of a megabyte on Android — because a lossless track is +too large for one bridge message, and on Android it is written to the relay's +spool directory rather than held. The relay starts for it if nothing else has, +serves it behind the stream's token with a versioned address (a receiver handed +the same URL twice shows what it already has), honours byte ranges (a receiver +seeks in a track that way), and accepts only pictures and the audio types the +music player plays. **The shell, not the page, says what the receiver loads:** +the relay's file is loaded as what the relay was told it is — a photo as a +picture (`streamType: NONE`), a track as music (`BUFFERED`, with title, artist, +album and the relay's cover URL), at `startAt` seconds for a track picked up +mid-song — any other relay URL as the stream, and on Android no URL but the +relay's own is accepted at all. A file does not start the foreground service; a +photo's load answers as soon as the receiver accepts it, since a photo never +reaches PLAYING. + +**A photo is scaled before it leaves.** The page fits it to 1920×1080, applies +its EXIF orientation and re-encodes it as JPEG: a camera original is too large +for a receiver to decode in good time, and some formats it does not decode at +all. Only the newest photo asked for is sent; paging quickly shows where the +reader stopped. + +**With a television chosen, the music bar is its remote.** Each track, once +decrypted, goes to the relay with its cover instead of to the `<audio>` +element, which keeps the source for its duration only. Play, pause, seek +(`cast:chromecast:seek`) and previous drive the receiver; the bar's clock is the +receiver's, polled once a second; and the receiver's IDLE/FINISHED moves the +queue on as `ended` does, once per track, because the receiver keeps reporting +it until it is given something else. The session records which view the +television is showing (`castSession.owner`): a film or a photo opened meanwhile +takes it, the bar pauses and stops reading the receiver, and play gives it the +track back from where it was. A television chosen mid-track picks the track up +where it was; *Stop casting* carries it on locally from where the television +was. **The interface itself is not cast.** The default media receiver plays what it is given and renders nothing of its own, and a rendered interface sent to it as diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index 3b2f16c..97ea679 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -394,11 +394,11 @@ resume. ### Casting to a TV -**The desktop and Android applications.** Films and photos can be sent to a -cast-capable TV or dongle on the same network. The cast button is in the -Videos toolbar, at the top of Photos, in an album's bar and in the photo -viewer — in a group and in Search alike — and *Cast to device* is in the -player. Devices appear as +**The desktop and Android applications.** Films, photos and music can be sent +to a cast-capable TV or dongle on the same network. The cast button is in the +Videos and Music toolbars, at the top of Photos, in an album's bar, in the photo +viewer and in the music bar — in a group and in Search alike — and *Cast to +device* is in the video player. Devices appear as they answer, usually within a couple of seconds; the search carries on for a few more, for a TV that is still waking up. @@ -407,12 +407,19 @@ or from the player. Every film opened plays on it, and the player opens as its remote: the position shown is the TV's, with play/pause, 30-second jumps and a slider to go anywhere in the film. Every photo opened in the viewer is shown on it, and a slideshow pages the TV along with the screen. Choosing a TV from an -album's bar opens its first photo. On a phone, a film goes on playing silently -on the phone while the TV shows it, and the screen can be turned off. +album's bar opens its first photo. Music plays on it too, with the cover, title +and artist on the screen: the music bar becomes its remote — play, pause, the +slider, previous and next — and the queue moves on when the TV reaches the end +of a track. Choosing the TV in the middle of a song carries it on there; *Stop +casting* carries it on here. On a phone, a film goes on playing silently on the +phone while the TV shows it, and the screen can be turned off, as it can while +music plays on the TV. Photos are sent scaled to the TV's screen, upright, as an ordinary picture. The menus, covers and summaries stay on your phone or computer: the TV shows -what is playing, not the application. +what is playing, not the application. It plays it in the TV's standard cast +player, which shows its own name, *Default Media Receiver*, on the screen; a +player of MeshBay's own is not built. The application decrypts the film and relays it to the TV itself, over your LAN. The TV is not a group member and holds no key — which is also why the relay's @@ -993,8 +1000,8 @@ Better to know now than to go looking for it: - **Hubs do not talk to each other yet.** Everyone in a group needs an account on the same hub. - **Casting reaches Chromecast devices** from the desktop and Android - applications, for films and photos. Music is not cast yet. Support for other - TV protocols is designed but not built. + applications, for films, photos and music. Support for other TV protocols is + designed but not built. - **Some subtitle tracks cannot be shown** — the ones stored as images rather than text, roughly one embedded track in five. Displaying them would need text recognition. diff --git a/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js b/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js index 21d27df..b71b5e8 100644 --- a/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js +++ b/packages/meshbay-android/app/src/main/assets/bridge/meshbay-bridge.js @@ -50,6 +50,10 @@ // per 48 KB chunk). Without ArrayBuffer messages, base64 over JSON. const SAVE_WRITE = 1; const CAST_PUSH = 2; + const CAST_FILE = 3; + // A file crosses in pieces this size: big enough to be few, small enough + // that no one message is a burden to the WebView or the bridge. + const FILE_PIECE = 1024 * 1024; const bytesOf = (chunk) => (chunk instanceof Uint8Array ? chunk : ArrayBuffer.isView(chunk) ? new Uint8Array(chunk.buffer, chunk.byteOffset, chunk.byteLength) : new Uint8Array(chunk)); @@ -151,7 +155,19 @@ push: (data) => pushSegment(data), stop: () => call('cast:stop'), subtitle: (sub) => call('cast:subtitle', sub), - image: (img) => call('cast:image', { bytes: base64Of(bytesOf(img.bytes)), type: img.type }), + // A whole file — a photo, a track — in pieces: one message the size of a + // lossless track would be the WebView's to refuse. + file: async (f) => { + const bytes = bytesOf(f.bytes); + const cover = f.cover ? { bytes: base64Of(bytesOf(f.cover.bytes)), type: f.cover.type } : null; + await call('cast:file:begin', { type: f.type, size: bytes.length, cover, + title: f.title, artist: f.artist, album: f.album }); + for (let at = 0; at < bytes.length; at += FILE_PIECE) { + const piece = bytes.subarray(at, at + FILE_PIECE); + await (BINARY ? sendBinary(CAST_FILE, 0, piece) : call('cast:file:write', base64Of(piece))); + } + return call('cast:file:end'); + }, finish: () => call('cast:finish'), status: () => call('cast:status'), scan: () => call('cast:scan'), @@ -161,6 +177,7 @@ chromecastDisconnect: () => call('cast:chromecast:disconnect'), chromecastPause: () => call('cast:chromecast:pause'), chromecastPlay: () => call('cast:chromecast:play'), + chromecastSeek: (seconds) => call('cast:chromecast:seek', seconds), } } : {}), // Music is playing: keep the process, the Wi-Fi and the page awake with diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/bridge/Channels.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/bridge/Channels.kt index 7542fa1..f7094a0 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/bridge/Channels.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/bridge/Channels.kt @@ -63,6 +63,7 @@ class Channels( fun binary(frame: BinaryFrame): Any? = when (frame.channel) { BinaryFrame.SAVE_WRITE -> saves().write(frame.handle, frame.bytes, frame.offset, frame.length) BinaryFrame.CAST_PUSH -> (cast ?: throw Refused("Refused: no such channel")).push(frame.bytes, frame.offset, frame.length) + BinaryFrame.CAST_FILE -> (cast ?: throw Refused("Refused: no such channel")).writeFile(frame.bytes, frame.offset, frame.length) else -> throw Refused("Refused: no such channel") } @@ -72,7 +73,7 @@ class Channels( * they could overtake one another; these run on one thread, in arrival order. */ fun ordered(channel: String) = cast?.ordered(channel) == true - fun ordered(frame: BinaryFrame) = frame.channel == BinaryFrame.CAST_PUSH + fun ordered(frame: BinaryFrame) = frame.channel == BinaryFrame.CAST_PUSH || frame.channel == BinaryFrame.CAST_FILE private fun setLocale(code: String): String { // A code, never text, and only one the package has a catalogue for. diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt index 6be5d72..edaec67 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastChannels.kt @@ -26,8 +26,9 @@ class CastChannels( fun handles(channel: String) = channel.startsWith("cast:") - /** What must reach the relay in the order the page sent it: start, pushes, subtitle, photo, finish, stop. */ - fun ordered(channel: String) = channel in setOf("cast:start", "cast:push", "cast:subtitle", "cast:image", "cast:finish", "cast:stop") + /** What must reach the relay in the order the page sent it: start, pushes, subtitle, a file's pieces, finish, stop. */ + fun ordered(channel: String) = channel in setOf("cast:start", "cast:push", "cast:subtitle", + "cast:file:begin", "cast:file:write", "cast:file:end", "cast:finish", "cast:stop") fun call(channel: String, a: JSONArray): Any? = when (channel) { "cast:start" -> { @@ -38,11 +39,20 @@ class CastChannels( started } "cast:subtitle" -> relay.setSubtitle(a.optJSONObject(0)) - // No foreground service for a photo: the receiver has it once loaded. - "cast:image" -> { + // A whole file — a photo, a track — in pieces. No foreground service + // for one: a photo needs nothing once loaded, and music keeps the page + // alive through its own playback hold. + "cast:file:begin" -> { val o = a.optJSONObject(0) ?: JSONObject() - relay.showImage(Base64.getDecoder().decode(o.optString("bytes", "")), o.optString("type", "")) + val cover = o.optJSONObject("cover") + relay.beginFile(o.optString("type", ""), o.optLong("size", -1), + cover?.optString("bytes", "")?.takeIf { it.isNotEmpty() }?.let { Base64.getDecoder().decode(it) }, + cover?.optString("type", ""), o) } + "cast:file:write" -> { // the base64 path, for a WebView without ArrayBuffer messages + relay.writeFile(Base64.getDecoder().decode(a.optString(0, ""))) + } + "cast:file:end" -> relay.endFile() "cast:push" -> { // the base64 path, for a WebView without ArrayBuffer messages relay.push(Base64.getDecoder().decode(a.optString(0, ""))); true } @@ -56,15 +66,16 @@ class CastChannels( "cast:chromecast:connect" -> { val o = a.optJSONObject(0) ?: JSONObject() val url = relayUrl(o) - reported { control.connect(o.optString("deviceId", ""), url, subtitleFor(url, o), typeOf(url)) } + reported { control.connect(o.optString("deviceId", ""), url, subtitleFor(url, o), fileOf(url), startOf(url, o)) } } "cast:chromecast:reload" -> { val o = a.optJSONObject(0) ?: JSONObject() val url = relayUrl(o) - reported { control.reload(url, subtitleFor(url, o), typeOf(url)) } + reported { control.reload(url, subtitleFor(url, o), fileOf(url), startOf(url, o)) } } "cast:chromecast:pause" -> control.pause() "cast:chromecast:play" -> control.play() + "cast:chromecast:seek" -> control.seek(a.optDouble(0, 0.0)) "cast:chromecast:disconnect" -> control.disconnect() else -> throw Refused("Refused: no such channel") } @@ -84,26 +95,37 @@ class CastChannels( fun push(bytes: ByteArray, offset: Int, length: Int): Boolean { relay.push(bytes, offset, length); return true } + fun writeFile(bytes: ByteArray, offset: Int, length: Int): Boolean = relay.writeFile(bytes, offset, length) + /** * The receiver is only ever pointed at this relay. A URL the page names is - * accepted when it is the relay's own stream or photo, and refused + * accepted when it is the relay's own stream or file, and refused * otherwise — a page cannot use this application to make a television * fetch anything else. */ private fun relayUrl(o: JSONObject): String { val asked = o.optString("mediaUrl", "") val ours = relay.url ?: throw Refused("The cast relay is not running") - if (asked != ours && asked != relay.imageUrl) throw Refused("Refused: not this relay's stream") + if (asked != ours && asked != relay.fileUrl) throw Refused("Refused: not this relay's stream") return asked } - /** The relay's photo is loaded as a picture, anything else as the stream; the page does not say which. */ - private fun typeOf(url: String): String = - if (url == relay.imageUrl) relay.imageType ?: "image/jpeg" else "video/mp4" + /** + * The relay's file is loaded as what the relay says it is — its type, and + * the title, artist, album and cover it was given — and anything else as + * the stream (null here); the page does not say which. + */ + private fun fileOf(url: String): JSONObject? = + if (url != relay.fileUrl) null + else JSONObject().put("contentType", relay.fileType).put("meta", relay.fileMeta() ?: JSONObject()) + + /** Where in a track to begin; a film starts where its relay does. */ + private fun startOf(url: String, o: JSONObject): Double = + if (url == relay.fileUrl) maxOf(0.0, o.optDouble("startAt", 0.0)) else 0.0 - /** As main.js: no subtitle named means the relay's current one; a photo has none. */ + /** As main.js: no subtitle named means the relay's current one; a file has none. */ private fun subtitleFor(url: String, o: JSONObject): JSONObject? = when { - url == relay.imageUrl -> null + url == relay.fileUrl -> null o.has("subtitle") && o.get("subtitle") != JSONObject.NULL -> o.optJSONObject("subtitle") else -> relay.subtitleInfo() } diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt index 56a1f5e..0d18a90 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastControl.kt @@ -2,6 +2,7 @@ package org.meshbay.client.cast import android.content.Context import android.graphics.Color +import android.net.Uri import android.os.Handler import android.os.Looper import android.util.Log @@ -11,6 +12,8 @@ import com.google.android.gms.cast.CastMediaControlIntent import com.google.android.gms.cast.MediaInfo import com.google.android.gms.cast.MediaLoadRequestData import com.google.android.gms.cast.MediaMetadata +import com.google.android.gms.cast.MediaSeekOptions +import com.google.android.gms.common.images.WebImage import com.google.android.gms.cast.MediaStatus import com.google.android.gms.cast.MediaTrack import com.google.android.gms.cast.TextTrackStyle @@ -112,7 +115,7 @@ class CastControl(private val context: Context) { } /** Connect, launch the default receiver, load; answers once the receiver has. */ - fun connect(deviceId: String, mediaUrl: String, subtitle: JSONObject?, contentType: String = "video/mp4"): JSONObject { + fun connect(deviceId: String, mediaUrl: String, subtitle: JSONObject?, file: JSONObject? = null, startAt: Double = 0.0): JSONObject { need() val session = onMain { val cast = CastContext.getSharedInstance(context) @@ -168,7 +171,7 @@ class CastControl(private val context: Context) { // device's name included: read it there, never from this worker. val name = deviceName ?: onMain { session.castDevice?.friendlyName } Log.i(TAG, "session up on '$name', loading the relay stream") - val state = load(session, mediaUrl, subtitle, contentType) + val state = load(session, mediaUrl, subtitle, file, startAt) watch(session) return JSONObject().put("deviceName", name ?: JSONObject.NULL).put("playerState", state) } @@ -201,23 +204,41 @@ class CastControl(private val context: Context) { watched = c } - fun reload(mediaUrl: String, subtitle: JSONObject?, contentType: String = "video/mp4"): JSONObject { + fun reload(mediaUrl: String, subtitle: JSONObject?, file: JSONObject? = null, startAt: Double = 0.0): JSONObject { need() val session = onMain { CastContext.getSharedInstance(context).sessionManager.currentCastSession } ?: throw Refused("Not connected") - val state = load(session, mediaUrl, subtitle, contentType) + val state = load(session, mediaUrl, subtitle, file, startAt) watch(session) return JSONObject().put("playerState", state) } - private fun load(session: CastSession, mediaUrl: String, subtitle: JSONObject?, contentType: String): String { - val photo = contentType.startsWith("image/") + /** + * `file` is the relay's description of its file (type and what to show + * with it), or null for the stream. A photo is not a stream at all; a + * music track is an ordinary buffered file the receiver seeks in itself. + */ + private fun load(session: CastSession, mediaUrl: String, subtitle: JSONObject?, + file: JSONObject?, startAt: Double): String { + val contentType = file?.optString("contentType", "") ?: "video/mp4" + val photo = file != null && contentType.startsWith("image/") val subUrl = subtitle?.optString("url", "")?.takeIf { it.isNotEmpty() } val info = if (photo) { // A picture, not a stream: nothing to seek, nothing to buffer. MediaInfo.Builder(mediaUrl).setContentType(contentType) .setStreamType(MediaInfo.STREAM_TYPE_NONE) .setMetadata(MediaMetadata(MediaMetadata.MEDIA_TYPE_PHOTO)).build() + } else if (file != null) { + val meta = file.optJSONObject("meta") ?: JSONObject() + MediaInfo.Builder(mediaUrl).setContentType(contentType) + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setMetadata(MediaMetadata(MediaMetadata.MEDIA_TYPE_MUSIC_TRACK).apply { + putString(MediaMetadata.KEY_TITLE, meta.optString("title", "")) + putString(MediaMetadata.KEY_ARTIST, meta.optString("artist", "")) + putString(MediaMetadata.KEY_ALBUM_TITLE, meta.optString("album", "")) + meta.optString("cover", "").takeIf { it.isNotEmpty() && it != "null" } + ?.let { addImage(WebImage(Uri.parse(it))) } + }).build() } else MediaInfo.Builder(mediaUrl) .setContentType("video/mp4") // LIVE: the relay has no beginning to seek back to — the film's own @@ -243,6 +264,7 @@ class CastControl(private val context: Context) { } }.build() val request = MediaLoadRequestData.Builder().setMediaInfo(info).setAutoplay(true) + .apply { if (startAt > 0) setCurrentTime((startAt * 1000).toLong()) } .apply { if (subUrl != null) setActiveTrackIds(longArrayOf(TEXT_TRACK_ID)) }.build() val loaded = CountDownLatch(1) var ok = false @@ -295,6 +317,10 @@ class CastControl(private val context: Context) { /** The player as a remote: pause and play the receiver, answered once it has. */ fun pause(): JSONObject = command { it.pause() } fun play(): JSONObject = command { it.play() } + /** Where in a track the receiver should be; a film seeks by restarting the relay instead. */ + fun seek(seconds: Double): JSONObject = command { + it.seek(MediaSeekOptions.Builder().setPosition((maxOf(0.0, seconds) * 1000).toLong()).build()) + } private fun command(send: (com.google.android.gms.cast.framework.media.RemoteMediaClient) -> com.google.android.gms.common.api.PendingResult<com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult>): JSONObject { diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt index 41bf4bd..5b76f60 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/cast/CastRelay.kt @@ -82,7 +82,13 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir private val spoolSeq = AtomicLong() private class Subtitle(val vtt: ByteArray, val language: String, val label: String) - private class Image(val bytes: ByteArray, val type: String) + private class Cover(val bytes: ByteArray, val type: String) + /** A whole file on disk, served at `/file`: a photo, or a track with its cover. */ + private class FileSlot(val path: File, val size: Long, val type: String, val cover: Cover?, val meta: JSONObject) + private class Incoming(val slot: FileSlot) { + val out = java.io.FileOutputStream(slot.path) + var received = 0L + } @Volatile private var server: ServerSocket? = null @Volatile private var port = 0 @@ -93,8 +99,10 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir @Volatile private var headerReady = false @Volatile private var subtitle: Subtitle? = null @Volatile private var subtitleVersion = 0 - @Volatile private var image: Image? = null - @Volatile private var imageVersion = 0 + @Volatile private var file: FileSlot? = null + @Volatile private var fileVersion = 0 + private var incoming: Incoming? = null // guarded by this + private val fileSeq = AtomicLong() /** How many times a receiver asked for the stream since the relay started. */ @Volatile var streamRequests = 0 private set @@ -117,24 +125,74 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir else "http://${hostPart()}:$port/subs.vtt?t=$token&v=$subtitleVersion" /** Versioned like the subtitle: a receiver handed the same address twice shows what it has. */ - val imageUrl get() = - if (server == null || image == null) null - else "http://${hostPart()}:$port/image?t=$token&v=$imageVersion" + val fileUrl get() = + if (server == null || file == null) null + else "http://${hostPart()}:$port/file?t=$token&v=$fileVersion" + + val fileType get() = file?.type - val imageType get() = image?.type + /** What the receiver is told the file is: title, artist, album, and the cover's address. */ + fun fileMeta(): JSONObject? = file?.let { f -> + JSONObject(f.meta.toString()).put("cover", + if (f.cover == null) JSONObject.NULL else "http://${hostPart()}:$port/cover?t=$token&v=$fileVersion") + } /** - * One photo, scaled and re-encoded by the page, served at `/image` until - * the next one. The relay starts for it when nothing else has. + * A file arrives in three steps, so that a large one never crosses the + * bridge as one message: `beginFile` names it, `writeFile` appends, and + * `endFile` puts it up and answers its address. It is written to the spool + * directory rather than held: a long lossless track is a few hundred + * megabytes. The relay starts for it when nothing else has. */ @Synchronized - fun showImage(bytes: ByteArray, type: String): JSONObject { - if (type !in IMAGE_TYPES) throw IllegalArgumentException("Not a picture a receiver shows: $type") + fun beginFile(type: String, size: Long, cover: ByteArray?, coverType: String?, meta: JSONObject): Boolean { + if (type !in FILE_TYPES) throw IllegalArgumentException("Not something a receiver shows: $type") + if (size <= 0 || size > FILE_MAX_BYTES) throw IllegalArgumentException("Bad file size: $size") + if (cover != null && (coverType == null || !coverType.startsWith("image/") || coverType !in FILE_TYPES)) { + throw IllegalArgumentException("Bad cover type: $coverType") + } if (server == null) start(null, null) - image = Image(bytes, type) - imageVersion++ - log("image set: ${bytes.size} bytes, $type, v$imageVersion") - return JSONObject().put("url", imageUrl) + dropIncoming() + // At most the file being shown and the one arriving: anything older + // has been replaced on the television already. + spoolDir.listFiles()?.filter { it.name.startsWith("file-") && it != file?.path }?.forEach { it.delete() } + val text = { k: String -> meta.optString(k, "").take(500) } + val slot = FileSlot(File(spoolDir, "file-${fileSeq.incrementAndGet()}.bin"), size, type, + cover?.let { Cover(it, coverType!!) }, + JSONObject().put("title", text("title")).put("artist", text("artist")).put("album", text("album"))) + incoming = Incoming(slot) + return true + } + + @Synchronized + fun writeFile(data: ByteArray, offset: Int = 0, length: Int = data.size - offset): Boolean { + val f = incoming ?: throw IllegalStateException("No file is being sent") + if (f.received + length > f.slot.size) { dropIncoming(); throw IllegalStateException("More bytes than announced") } + f.out.write(data, offset, length) + f.received += length + return true + } + + @Synchronized + fun endFile(): JSONObject { + val f = incoming ?: throw IllegalStateException("No file is being sent") + incoming = null + f.out.close() + if (f.received != f.slot.size) { + f.slot.path.delete() + throw IllegalStateException("File cut short: ${f.received} of ${f.slot.size} bytes") + } + // The previous file stays on disk until the next one begins: a receiver + // may still be reading it. + file = f.slot + fileVersion++ + log("file set: ${f.slot.size} bytes, ${f.slot.type}, v$fileVersion") + return JSONObject().put("url", fileUrl) + } + + private fun dropIncoming() { + incoming?.let { try { it.out.close() } catch (e: Exception) {}; it.slot.path.delete() } + incoming = null } fun subtitleInfo(): JSONObject? = subtitle?.let { @@ -169,7 +227,8 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir spoolDir.mkdirs() spoolDir.listFiles()?.forEach { it.delete() } subtitle = null - image = null + dropIncoming() + file = null setSubtitle(sub) var bound: ServerSocket? = null @@ -276,7 +335,10 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir clients.clear() server?.let { try { it.close() } catch (e: Exception) {} } server = null - port = 0; token = null; initSegment = null; subtitle = null; image = null + port = 0; token = null; initSegment = null; subtitle = null + dropIncoming() + file = null + spoolDir.listFiles()?.filter { it.name.startsWith("file-") }?.forEach { it.delete() } synchronized(ring) { ring.clear(); ringBytes = 0 } accum.reset() } @@ -295,7 +357,12 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir socket.soTimeout = 15000 val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.ISO_8859_1)) val requestLine = reader.readLine() ?: return socket.close() - while (true) { val line = reader.readLine() ?: break; if (line.isEmpty()) break } + var range: String? = null + while (true) { + val line = reader.readLine() ?: break + if (line.isEmpty()) break + if (line.startsWith("range:", ignoreCase = true)) range = line.substringAfter(':').trim() + } socket.soTimeout = 0 val parts = requestLine.split(' ') val method = parts.getOrElse(0) { "" } @@ -316,7 +383,8 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir if (token == null || query["t"] != token) return respond(out, socket, 403, emptyMap()) when (path) { "/subs.vtt" -> serveSubtitle(out, socket) - "/image" -> serveImage(out, socket) + "/file" -> serveFile(out, socket, range) + "/cover" -> serveCover(out, socket) "/stream.mp4" -> { streamRequests++; serveStream(out, socket) } else -> respond(out, socket, 404, emptyMap()) } @@ -334,13 +402,52 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir ), sub.vtt) } - private fun serveImage(out: OutputStream, socket: Socket) { - val img = image ?: return respond(out, socket, 404, CORS_HEADERS) + /** The whole file, or the one range asked for: a receiver seeks in a track that way. */ + private fun serveFile(out: OutputStream, socket: Socket, range: String?) { + val f = file ?: return respond(out, socket, 404, CORS_HEADERS) + val total = f.size + val base = CORS_HEADERS + mapOf("Content-Type" to f.type, "Cache-Control" to "no-store", + "Accept-Ranges" to "bytes", "Connection" to "close") + var from = 0L + var to = total - 1 + var status = 200 + val m = range?.let { Regex("^bytes=(\\d*)-(\\d*)$").matchEntire(it) } + if (m != null && (m.groupValues[1].isNotEmpty() || m.groupValues[2].isNotEmpty())) { + val a = m.groupValues[1]; val b = m.groupValues[2] + from = if (a.isEmpty()) total - b.toLong() else a.toLong() + to = if (a.isEmpty() || b.isEmpty()) total - 1 else minOf(b.toLong(), total - 1) + from = maxOf(0L, from) + if (from > to || from >= total) { + head(out, 416, base + ("Content-Range" to "bytes */$total") + ("Content-Length" to "0")) + out.flush(); socket.close(); return + } + status = 206 + } + val length = to - from + 1 + head(out, status, base + ("Content-Length" to length.toString()) + + (if (status == 206) mapOf("Content-Range" to "bytes $from-$to/$total") else emptyMap())) + RandomAccessFile(f.path, "r").use { raf -> + raf.seek(from) + val block = ByteArray(BLOCK) + var left = length + while (left > 0) { + val n = raf.read(block, 0, minOf(BLOCK.toLong(), left).toInt()) + if (n <= 0) break + out.write(block, 0, n) + left -= n + } + } + out.flush() + socket.close() + } + + private fun serveCover(out: OutputStream, socket: Socket) { + val c = file?.cover ?: return respond(out, socket, 404, CORS_HEADERS) respond(out, socket, 200, CORS_HEADERS + mapOf( - "Content-Type" to img.type, - "Content-Length" to img.bytes.size.toString(), + "Content-Type" to c.type, + "Content-Length" to c.bytes.size.toString(), "Cache-Control" to "no-store", - ), img.bytes) + ), c.bytes) } private fun serveStream(out: OutputStream, socket: Socket) { @@ -428,8 +535,13 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir } const val RING_CAP = 64 - // The page re-encodes every photo to JPEG: a guard, not a menu. - val IMAGE_TYPES = setOf("image/jpeg", "image/png", "image/webp") + // What a receiver is ever handed as a file: the page re-encodes every + // photo to JPEG, and plays a track in the format it is stored in. + val FILE_TYPES = setOf("image/jpeg", "image/png", "image/webp", + "audio/mpeg", "audio/mp4", "audio/aac", "audio/flac", "audio/ogg", "audio/opus", + "audio/wav", "audio/webm") + // Only stops a page from filling the phone: a long lossless track is a few hundred MB. + const val FILE_MAX_BYTES = 1024L * 1024 * 1024 const val RING_MAX_BYTES = 32L * 1024 * 1024 const val SPOOL_MAX_BYTES = 2L * 1024 * 1024 * 1024 private const val HEADER_WAIT_MS = 10_000L @@ -450,8 +562,9 @@ class CastRelay(private val lanAddress: () -> InetAddress?, private val spoolDir "Access-Control-Expose-Headers" to "Content-Length, Content-Range", ) - private val REASONS = mapOf(200 to "OK", 204 to "No Content", 403 to "Forbidden", - 404 to "Not Found", 405 to "Method Not Allowed") + private val REASONS = mapOf(200 to "OK", 204 to "No Content", 206 to "Partial Content", + 403 to "Forbidden", 404 to "Not Found", 405 to "Method Not Allowed", + 416 to "Range Not Satisfiable") private fun randomToken(): String { val b = ByteArray(16).also { SecureRandom().nextBytes(it) } diff --git a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/save/SaveNames.kt b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/save/SaveNames.kt index 2414730..40250c7 100644 --- a/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/save/SaveNames.kt +++ b/packages/meshbay-android/app/src/main/kotlin/org/meshbay/client/save/SaveNames.kt @@ -59,6 +59,7 @@ class BinaryFrame(val id: Long, val channel: Int, val handle: Long, val bytes: B const val HEADER = 16 const val SAVE_WRITE = 1 const val CAST_PUSH = 2 + const val CAST_FILE = 3 private val MAGIC = byteArrayOf('M'.code.toByte(), 'B'.code.toByte(), 'B'.code.toByte(), '1'.code.toByte()) private fun u32(b: ByteArray, at: Int) = diff --git a/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt b/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt index 8013e29..14f49cf 100644 --- a/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt +++ b/packages/meshbay-android/app/src/test/kotlin/org/meshbay/client/CastRelayTest.kt @@ -158,9 +158,38 @@ class CastRelayTest { assertEquals(200, get(r.getString("url"), readBytes = 0).status) } - @Test fun `a photo starts the relay and is served behind the token, re-addressed when it changes`() { + private fun send(bytes: ByteArray, type: String, meta: JSONObject = JSONObject(), + cover: ByteArray? = null): String { + relay.beginFile(type, bytes.size.toLong(), cover, cover?.let { "image/jpeg" }, meta) + var at = 0 + while (at < bytes.size) { val n = minOf(3, bytes.size - at); relay.writeFile(bytes, at, n); at += n } + return relay.endFile().getString("url") + } + + private fun getRange(url: String, range: String): Reply { + val u = java.net.URI(url) + Socket(u.host, u.port).use { s -> + s.soTimeout = 5000 + s.getOutputStream().write("GET ${u.rawPath}?${u.rawQuery} HTTP/1.1\r\nHost: x\r\nRange: $range\r\n\r\n".toByteArray()) + val input = DataInputStream(s.getInputStream()) + val lines = ArrayList<String>() + val line = StringBuilder() + while (true) { + val c = input.read() + if (c == -1) break + if (c == '\n'.code) { val l = line.toString().trimEnd('\r'); if (l.isEmpty()) break; lines.add(l); line.clear() } + else line.append(c.toChar()) + } + val headers = lines.drop(1).associate { it.substringBefore(':').lowercase() to it.substringAfter(':').trim() } + val body = ByteArray(headers["content-length"]?.toInt() ?: 0) + input.readFully(body) + return Reply(lines[0].split(' ')[1].toInt(), headers, body) + } + } + + @Test fun `a file starts the relay and is served behind the token, re-addressed when it changes`() { val jpeg = byteArrayOf(0xFF.toByte(), 0xD8.toByte(), 1, 2, 3, 0xFF.toByte(), 0xD9.toByte()) - val first = relay.showImage(jpeg, "image/jpeg").getString("url") + val first = send(jpeg, "image/jpeg") assertTrue(relay.active) val reply = get(first) assertEquals(200, reply.status) @@ -169,21 +198,42 @@ class CastRelayTest { assertArrayEquals(jpeg, reply.body) assertEquals(403, get(first.replace(Regex("t=[0-9a-f]+"), "t=x")).status) // A receiver handed the same address twice shows what it already has. - val second = relay.showImage(byteArrayOf(9), "image/jpeg").getString("url") + val second = send(byteArrayOf(9), "image/jpeg") assertNotEquals(first, second) assertArrayEquals(byteArrayOf(9), get(second).body) } - @Test fun `only a picture is served as one`() { - val refused = try { relay.showImage(byteArrayOf(1), "text/html"); false } catch (e: IllegalArgumentException) { true } + @Test fun `a track is served with its cover, and by range`() { + val track = ByteArray(20) { it.toByte() } + val cover = byteArrayOf(0xFF.toByte(), 0xD8.toByte(), 7) + val url = send(track, "audio/flac", JSONObject().put("title", "A Song").put("artist", "Some Band"), cover) + assertEquals("audio/flac", relay.fileType) + val meta = relay.fileMeta()!! + assertEquals("A Song", meta.getString("title")) + assertArrayEquals(cover, get(meta.getString("cover")).body) + val part = getRange(url, "bytes=5-9") + assertEquals(206, part.status) + assertEquals("bytes 5-9/20", part.headers["content-range"]) + assertArrayEquals(track.copyOfRange(5, 10), part.body) + assertArrayEquals(track.copyOfRange(15, 20), getRange(url, "bytes=15-").body) + assertEquals(416, getRange(url, "bytes=50-60").status) + } + + @Test fun `only a whole picture or sound is served as one`() { + val refused = try { relay.beginFile("text/html", 1, null, null, JSONObject()); false } catch (e: IllegalArgumentException) { true } assertTrue(refused) - assertNull(relay.imageUrl) + assertNull(relay.fileUrl) + relay.beginFile("audio/mpeg", 10, null, null, JSONObject()) + relay.writeFile(byteArrayOf(1, 2, 3)) + val short = try { relay.endFile(); false } catch (e: IllegalStateException) { true } + assertTrue(short) + assertNull(relay.fileUrl) } - @Test fun `a film started over a photo takes the photo down`() { - relay.showImage(byteArrayOf(1), "image/jpeg") + @Test fun `a film started over a file takes the file down`() { + send(byteArrayOf(1), "image/jpeg") relay.start(null, null) - assertNull(relay.imageUrl) + assertNull(relay.fileUrl) } @Test fun `the preflight is answered before the token is checked`() { diff --git a/packages/meshbay-client/src/cast-chromecast.js b/packages/meshbay-client/src/cast-chromecast.js index 4c8ddf0..8c6c9c6 100644 --- a/packages/meshbay-client/src/cast-chromecast.js +++ b/packages/meshbay-client/src/cast-chromecast.js @@ -42,17 +42,30 @@ const TEXT_TRACK_STYLE = Object.freeze({ /** * What to hand `player.load()` for a stream, with or without subtitles, or - * for a photo. + * for a whole file the relay serves. * * `streamType: 'LIVE'` because the relay has no beginning to seek back to — * the film's own timeline lives on this side, and a seek restarts the relay. - * A photo is not a stream at all. + * A file is the relay's to describe (`file`: its type and what to show with + * it), never the page's: a photo is not a stream at all, and a music track is + * an ordinary buffered file the receiver seeks in itself. */ -function mediaFor(mediaUrl, subtitle, contentType) { - if (contentType && contentType.startsWith('image/')) { - return { contentId: mediaUrl, contentType, streamType: 'NONE', +function mediaFor(mediaUrl, subtitle, file) { + if (file && file.contentType.startsWith('image/')) { + return { contentId: mediaUrl, contentType: file.contentType, streamType: 'NONE', metadata: { metadataType: 4 } }; // PhotoMediaMetadata } + if (file) { + const meta = file.meta || {}; + return { + contentId: mediaUrl, contentType: file.contentType, streamType: 'BUFFERED', + metadata: { + metadataType: 3, // MusicTrackMediaMetadata + title: meta.title || '', artist: meta.artist || '', albumName: meta.album || '', + images: meta.cover ? [{ url: meta.cover }] : [], + }, + }; + } const media = { contentId: mediaUrl, contentType: 'video/mp4', @@ -73,11 +86,14 @@ function mediaFor(mediaUrl, subtitle, contentType) { return media; } -function loadOptionsFor(subtitle) { - return { +function loadOptionsFor(subtitle, startAt) { + const options = { autoplay: true, activeTrackIds: subtitle && subtitle.url ? [TEXT_TRACK_ID] : [], }; + // Where in a track to begin: a cast started mid-song carries on from there. + if (startAt > 0) options.currentTime = startAt; + return options; } class CastChromecast { @@ -145,7 +161,7 @@ class CastChromecast { } } - async connect(deviceId, mediaUrl, subtitle, contentType) { + async connect(deviceId, mediaUrl, subtitle, file, startAt) { const device = this._devices.get(deviceId); if (!device) throw new Error(`Unknown device: ${deviceId}`); @@ -185,7 +201,7 @@ class CastChromecast { debug(`[cast-chromecast] loading with ${subtitle?.url ? 'subtitles' : 'no subtitles'}`); const status = await new Promise((resolve, reject) => { - player.load(mediaFor(mediaUrl, subtitle, contentType), loadOptionsFor(subtitle), + player.load(mediaFor(mediaUrl, subtitle, file), loadOptionsFor(subtitle, startAt), (err, s) => { if (err) return reject(err); resolve(s); @@ -197,12 +213,12 @@ class CastChromecast { return { deviceName: device.name, playerState: status.playerState }; } - async reload(mediaUrl, subtitle, contentType) { + async reload(mediaUrl, subtitle, file, startAt) { if (!this._player) throw new Error('Not connected'); debug(`[cast-chromecast] reloading stream on "${this._connectedDevice?.name}"` + ` with ${subtitle?.url ? 'subtitles' : 'no subtitles'}`); const status = await new Promise((resolve, reject) => { - this._player.load(mediaFor(mediaUrl, subtitle, contentType), loadOptionsFor(subtitle), + this._player.load(mediaFor(mediaUrl, subtitle, file), loadOptionsFor(subtitle, startAt), (err, s) => { if (err) return reject(err); resolve(s); @@ -256,6 +272,16 @@ class CastChromecast { return { playerState: status && status.playerState }; } + /** Where in a track the receiver should be; a film seeks by restarting the relay instead. */ + async seek(seconds) { + if (!this._player) throw new Error('Not connected'); + const status = await new Promise((resolve, reject) => { + this._player.seek(Math.max(0, Number(seconds) || 0), (err, s) => (err ? reject(err) : resolve(s))); + }); + this._noteStatus(status); + return { playerState: status && status.playerState }; + } + async play() { if (!this._player) throw new Error('Not connected'); const status = await new Promise((resolve, reject) => { diff --git a/packages/meshbay-client/src/cast-relay.js b/packages/meshbay-client/src/cast-relay.js index c5eadc7..ed20b33 100644 --- a/packages/meshbay-client/src/cast-relay.js +++ b/packages/meshbay-client/src/cast-relay.js @@ -15,9 +15,11 @@ * a media element, so unlike the stream it needs CORS headers to be readable * at all. * - * A photo is served at `/image`, one at a time: the page decrypts it, scales - * it to the television and hands it here, and the receiver loads it as a - * picture. The relay starts for it when it is not already running. + * A whole file — a photo, or a music track with its cover at `/cover` — is + * served at `/file`, one at a time: the page decrypts it and hands it over in + * pieces (begin, writes, end), and the receiver loads it as a picture or as + * music. Ranges are honoured there, which is what a receiver seeks with. The + * relay starts for a file when it is not already running. * * Mitigations: * · Bind to the LAN interface, never 0.0.0.0 @@ -56,7 +58,14 @@ const debug = util.debuglog('cast-relay'); const RING_CAP = 64; // What the receiver is ever told a photo is. The page re-encodes every photo // to JPEG, so this list is a guard, not a menu. -const IMAGE_TYPES = new Set(['image/jpeg', 'image/png', 'image/webp']); +const FILE_TYPES = new Set([ + 'image/jpeg', 'image/png', 'image/webp', + 'audio/mpeg', 'audio/mp4', 'audio/aac', 'audio/flac', 'audio/ogg', 'audio/opus', + 'audio/wav', 'audio/webm', +]); +// A track is a few megabytes, a long lossless one a few hundred; this only +// stops a page from filling the machine's memory. +const FILE_MAX_BYTES = 1024 * 1024 * 1024; const BACKPRESSURE_HIGH = 8 * 1024 * 1024; const MOOF = 0x6d6f6f66; const PORT_BASE = 19550; @@ -170,8 +179,9 @@ class CastRelay { this._bytesSent = 0; this._subtitle = null; this._subtitleVersion = 0; - this._image = null; - this._imageVersion = 0; + this._file = null; + this._fileVersion = 0; + this._incoming = null; } get active() { return this._server !== null; } @@ -195,26 +205,67 @@ class CastRelay { } /** - * Where the current photo can be fetched, or null when there is none. + * Where the current file can be fetched, or null when there is none. * Versioned for the same reason as the subtitle: a receiver handed the same * address twice shows what it already has. */ - get imageUrl() { - if (!this._server || !this._image) return null; - return `http://${this._lanIP}:${this._port}/image` - + `?t=${this._token}&v=${this._imageVersion}`; + get fileUrl() { + if (!this._server || !this._file) return null; + return `http://${this._lanIP}:${this._port}/file` + + `?t=${this._token}&v=${this._fileVersion}`; } - get imageType() { return this._image ? this._image.type : null; } + get fileType() { return this._file ? this._file.type : null; } - /** Serve one photo, starting the relay if nothing else has. */ - async showImage({ bytes, type }) { - if (!IMAGE_TYPES.has(type)) throw new Error(`Not a picture a receiver shows: ${type}`); + /** What the receiver is told the file is: title, artist, album, cover. */ + get fileMeta() { + if (!this._file) return null; + const { title, artist, album } = this._file.meta; + const cover = this._file.cover + ? `http://${this._lanIP}:${this._port}/cover?t=${this._token}&v=${this._fileVersion}` + : null; + return { title, artist, album, cover }; + } + + /** + * A file arrives in three steps, so that a large one never has to cross in + * a single message: `beginFile` names it, `writeFile` appends, `endFile` + * puts it up and answers its address. The relay starts if nothing has. + */ + async beginFile({ type, size, cover, title, artist, album }) { + if (!FILE_TYPES.has(type)) throw new Error(`Not something a receiver shows: ${type}`); + if (!(size > 0 && size <= FILE_MAX_BYTES)) throw new Error(`Bad file size: ${size}`); + if (cover && !FILE_TYPES.has(cover.type)) throw new Error(`Bad cover type: ${cover.type}`); if (!this._server) await this.start({}); - this._image = { bytes: Buffer.from(bytes), type }; - this._imageVersion++; - debug(`[cast-relay] image set: ${this._image.bytes.length} bytes, ${type}, v${this._imageVersion}`); - return { url: this.imageUrl }; + const text = (v) => (typeof v === 'string' ? v.slice(0, 500) : ''); + this._incoming = { + type, size, parts: [], received: 0, + cover: cover && cover.type.startsWith('image/') + ? { bytes: Buffer.from(cover.bytes), type: cover.type } : null, + meta: { title: text(title), artist: text(artist), album: text(album) }, + }; + return true; + } + + writeFile(bytes) { + const f = this._incoming; + if (!f) throw new Error('No file is being sent'); + const buf = Buffer.from(bytes); + if (f.received + buf.length > f.size) throw new Error('More bytes than announced'); + f.parts.push(buf); + f.received += buf.length; + return true; + } + + endFile() { + const f = this._incoming; + this._incoming = null; + if (!f) throw new Error('No file is being sent'); + if (f.received !== f.size) throw new Error(`File cut short: ${f.received} of ${f.size} bytes`); + this._file = { bytes: Buffer.concat(f.parts), type: f.type, cover: f.cover, meta: f.meta }; + this._fileVersion++; + debug(`[cast-relay] file set: ${f.size} bytes, ${f.type}, v${this._fileVersion}`); + return { url: this.fileUrl }; } get subtitle() { @@ -264,7 +315,8 @@ class CastRelay { this._chunkCount = 0; this._bytesSent = 0; this._subtitle = null; - this._image = null; + this._file = null; + this._incoming = null; this.setSubtitle(subtitle); const server = http.createServer((req, res) => this._handle(req, res)); @@ -356,7 +408,8 @@ class CastRelay { this._token = null; this._initSegment = null; this._subtitle = null; - this._image = null; + this._file = null; + this._incoming = null; this._ring = []; this._accum.reset(); this._fragCount = 0; @@ -400,8 +453,13 @@ class CastRelay { return; } - if (url.pathname === '/image') { - this._handleImage(res); + if (url.pathname === '/file') { + this._handleFile(req, res); + return; + } + + if (url.pathname === '/cover') { + this._handleCover(res); return; } @@ -459,20 +517,58 @@ class CastRelay { debug(`[cast-relay] subtitle served: ${this._subtitle.vtt.length} bytes`); } - _handleImage(res) { - if (!this._image) { + /** + * The whole file, or the one range asked for: a receiver seeks in a track + * by asking for the bytes from there on. + */ + _handleFile(req, res) { + const f = this._file; + if (!f) { + res.writeHead(404, CORS_HEADERS); + res.end(); + return; + } + const total = f.bytes.length; + const base = { + ...CORS_HEADERS, + 'Content-Type': f.type, + 'Cache-Control': 'no-store', + 'Accept-Ranges': 'bytes', + }; + const m = /^bytes=(\d*)-(\d*)$/.exec(req.headers.range || ''); + if (m && (m[1] !== '' || m[2] !== '')) { + let from = m[1] === '' ? total - Number(m[2]) : Number(m[1]); + let to = m[1] === '' || m[2] === '' ? total - 1 : Math.min(Number(m[2]), total - 1); + from = Math.max(0, from); + if (from > to || from >= total) { + res.writeHead(416, { ...base, 'Content-Range': `bytes */${total}` }); + res.end(); + return; + } + res.writeHead(206, { ...base, 'Content-Range': `bytes ${from}-${to}/${total}`, + 'Content-Length': to - from + 1 }); + res.end(f.bytes.subarray(from, to + 1)); + return; + } + res.writeHead(200, { ...base, 'Content-Length': total }); + res.end(f.bytes); + debug(`[cast-relay] file served: ${total} bytes`); + } + + _handleCover(res) { + const c = this._file && this._file.cover; + if (!c) { res.writeHead(404, CORS_HEADERS); res.end(); return; } res.writeHead(200, { ...CORS_HEADERS, - 'Content-Type': this._image.type, - 'Content-Length': this._image.bytes.length, + 'Content-Type': c.type, + 'Content-Length': c.bytes.length, 'Cache-Control': 'no-store', }); - res.end(this._image.bytes); - debug(`[cast-relay] image served: ${this._image.bytes.length} bytes`); + res.end(c.bytes); } } diff --git a/packages/meshbay-client/src/main.js b/packages/meshbay-client/src/main.js index 1c371f5..a2a7a86 100644 --- a/packages/meshbay-client/src/main.js +++ b/packages/meshbay-client/src/main.js @@ -2333,29 +2333,31 @@ function registerBridge() { handle('cast:devices', async () => castChromecast.devices()); - // One photo at a time, scaled and re-encoded by the page. - handle('cast:image', async (_e, img) => { - return castRelay.showImage({ bytes: Buffer.from(img.bytes), type: img.type }); - }); + // A whole file for the receiver — a photo, a music track — in three steps, + // so that a large one never crosses in a single message. + handle('cast:file:begin', async (_e, f) => castRelay.beginFile(f || {})); + handle('cast:file:write', async (_e, data) => castRelay.writeFile(Buffer.from(data))); + handle('cast:file:end', async () => castRelay.endFile()); - // The relay's photo is loaded as a picture, anything else as the stream; - // the page does not get to say which. + // The relay's file is loaded as what the relay says it is, anything else as + // the stream; the page does not get to say which. const loadOf = (mediaUrl, subtitle) => ( - mediaUrl && mediaUrl === castRelay.imageUrl - ? { subtitle: null, contentType: castRelay.imageType } - : { subtitle: subtitle === undefined ? castRelay.subtitle : subtitle, - contentType: 'video/mp4' }); + mediaUrl && mediaUrl === castRelay.fileUrl + ? { subtitle: null, file: { contentType: castRelay.fileType, meta: castRelay.fileMeta } } + : { subtitle: subtitle === undefined ? castRelay.subtitle : subtitle, file: null }); - handle('cast:chromecast:connect', async (_e, { deviceId, mediaUrl, subtitle }) => { + handle('cast:chromecast:connect', async (_e, { deviceId, mediaUrl, subtitle, startAt }) => { const l = loadOf(mediaUrl, subtitle); - return castChromecast.connect(deviceId, mediaUrl, l.subtitle, l.contentType); + return castChromecast.connect(deviceId, mediaUrl, l.subtitle, l.file, l.file ? startAt : 0); }); - handle('cast:chromecast:reload', async (_e, { mediaUrl, subtitle }) => { + handle('cast:chromecast:reload', async (_e, { mediaUrl, subtitle, startAt }) => { const l = loadOf(mediaUrl, subtitle); - return castChromecast.reload(mediaUrl, l.subtitle, l.contentType); + return castChromecast.reload(mediaUrl, l.subtitle, l.file, l.file ? startAt : 0); }); + handle('cast:chromecast:seek', async (_e, seconds) => castChromecast.seek(seconds)); + // The player becomes a remote while casting: these drive the receiver. handle('cast:chromecast:pause', async () => castChromecast.pause()); handle('cast:chromecast:play', async () => castChromecast.play()); diff --git a/packages/meshbay-client/src/preload.js b/packages/meshbay-client/src/preload.js index f12d68e..0de76db 100644 --- a/packages/meshbay-client/src/preload.js +++ b/packages/meshbay-client/src/preload.js @@ -178,7 +178,18 @@ contextBridge.exposeInMainWorld('meshbay', { push: (data) => ipcRenderer.invoke('cast:push', data), stop: () => ipcRenderer.invoke('cast:stop'), subtitle: (sub) => ipcRenderer.invoke('cast:subtitle', sub), - image: (img) => ipcRenderer.invoke('cast:image', img), + // A file in pieces of a few megabytes, as the Android bridge sends it. + file: async (f) => { + const bytes = f.bytes; + await ipcRenderer.invoke('cast:file:begin', { + type: f.type, size: bytes.length, cover: f.cover || null, + title: f.title, artist: f.artist, album: f.album, + }); + for (let at = 0; at < bytes.length; at += 4 * 1024 * 1024) { + await ipcRenderer.invoke('cast:file:write', bytes.subarray(at, at + 4 * 1024 * 1024)); + } + return ipcRenderer.invoke('cast:file:end'); + }, finish: () => ipcRenderer.invoke('cast:finish'), status: () => ipcRenderer.invoke('cast:status'), scan: () => ipcRenderer.invoke('cast:scan'), @@ -188,6 +199,7 @@ contextBridge.exposeInMainWorld('meshbay', { chromecastDisconnect: () => ipcRenderer.invoke('cast:chromecast:disconnect'), chromecastPause: () => ipcRenderer.invoke('cast:chromecast:pause'), chromecastPlay: () => ipcRenderer.invoke('cast:chromecast:play'), + chromecastSeek: (seconds) => ipcRenderer.invoke('cast:chromecast:seek', seconds), }, // A sink that writes to disk as chunks arrive, never a buffer handed over at diff --git a/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js b/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js index 007722c..9f93919 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/cast-session.js @@ -11,10 +11,14 @@ import * as platform from './platform.js'; // in the lightbox is shown there. In memory only: a television chosen // yesterday may be off today, and choosing it again is one tap. // -// The receiver is the default media receiver, which shows a film or a -// picture and nothing else; the interface itself stays on this device. +// The receiver is the default media receiver, which shows one film, one +// picture or one track at a time and nothing else; the interface itself stays +// on this device. `owner` says which of the three the television is showing +// now ('video', 'photo', 'music'), so that the music bar stops treating the +// receiver as its own once a film or a photo has taken it. let current = null; +let owner = null; const listeners = new Set(); function notify() { @@ -23,7 +27,15 @@ function notify() { export const castSession = { get device() { return current; }, + get owner() { return owner; }, get supported() { return Boolean(platform.capabilities.lanCast && platform.cast.available); }, + /** Whether a whole file (a photo, a track) can be sent: an older shell cannot. */ + get canSendFiles() { return this.supported && platform.cast.canSendFiles; }, + /** The television now shows what `who` sends it. */ + claim(who) { + owner = who; + notify(); + }, choose(device) { current = device ? { id: device.id, name: device.name } : null; notify(); @@ -32,6 +44,7 @@ export const castSession = { forget() { this.choose(null); }, /** Stop casting: the television goes back to its idle screen. */ async end() { + owner = null; this.choose(null); photoGen++; await platform.cast.chromecastDisconnect().catch(() => {}); @@ -49,6 +62,27 @@ export function useCastDevice() { return device; } +// ── A file on the television ───────────────────────────────────────────────── +// +// A photo or a music track, whole, handed to the relay and loaded on the +// session's television — connected if it is not yet, reloaded if it is. The +// relay, not the page, tells the receiver what the file is; the page only +// names it (title, artist, album, cover) and says where in it to begin. + +export async function loadFile({ owner: who, startAt = 0, ...file }) { + const device = current; + if (!device || !castSession.canSendFiles) throw new Error('No television to cast to'); + castSession.claim(who); + const sent = await platform.cast.file(file); + if (!sent) throw new Error('The cast relay did not take the file'); + const status = await platform.cast.status(); + if (status && status.chromecast && status.chromecast.connected) { + await platform.cast.chromecastReload({ mediaUrl: sent.url, startAt }); + } else { + await platform.cast.chromecastConnect({ deviceId: device.id, mediaUrl: sent.url, startAt }); + } +} + // ── A photo on the television ──────────────────────────────────────────────── // // Scaled to fit 1920×1080 and re-encoded as JPEG before it leaves: a camera's @@ -87,18 +121,10 @@ let photoChain = Promise.resolve(); export function showPhoto(blob) { const gen = ++photoGen; photoChain = photoChain.then(async () => { - const device = current; - if (gen !== photoGen || !device || !platform.cast.canShowImages) return; + if (gen !== photoGen || !current || !castSession.canSendFiles) return; const bytes = await scaledForTv(blob); if (gen !== photoGen) return; - const shown = await platform.cast.image({ bytes, type: 'image/jpeg' }); - if (!shown || gen !== photoGen) return; - const status = await platform.cast.status(); - if (status && status.chromecast && status.chromecast.connected) { - await platform.cast.chromecastReload({ mediaUrl: shown.url }); - } else { - await platform.cast.chromecastConnect({ deviceId: device.id, mediaUrl: shown.url }); - } + await loadFile({ owner: 'photo', bytes, type: 'image/jpeg' }); }).catch((err) => console.warn('[cast] photo not shown:', err)); return photoChain; } @@ -111,9 +137,12 @@ export function showPhoto(blob) { // toolbar. `cls` and `iconCls` are the button's and its icon's own classes, // so it matches its neighbours. // `onChosen` runs after a television is picked, for the view that has -// something to show on it straight away. +// something to show on it straight away. `up` opens the menu above the +// button, for one at the bottom of the window. -export function CastButton({ variant = 'page', cls = 'tb-btn tb-btn-icon', iconCls = '', onChosen }) { +export function CastButton({ + variant = 'page', cls = 'tb-btn tb-btn-icon', iconCls = '', onChosen, up = false, +}) { const device = useCastDevice(); const [open, setOpen] = useState(false); const [devices, setDevices] = useState([]); @@ -143,7 +172,7 @@ export function CastButton({ variant = 'page', cls = 'tb-btn tb-btn-icon', iconC if (!castSession.supported) return null; const overlay = variant === 'overlay'; - const menuCls = overlay ? 'cast-picker' : 'ctx-menu cast-menu'; + const menuCls = `${overlay ? 'cast-picker' : 'ctx-menu cast-menu'}${up ? ' cast-menu-up' : ''}`; const itemCls = overlay ? 'cast-picker-item' : 'ctx-menu-item'; const title = device ? `${t('cast.casting_to')} ${device.name}` : t('cast.start'); diff --git a/packages/meshbay-hub/src/meshbay_hub/static/music-app.js b/packages/meshbay-hub/src/meshbay_hub/static/music-app.js index 67be344..f2c4374 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/music-app.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/music-app.js @@ -10,6 +10,7 @@ import { usePager, Pager, pageSizeFrom } from './pager.js'; import { Menu, MenuDots, useMenu } from './menu.js'; import { PlaylistMenuButton, NameModal, usePlaylists } from './playlist-menu.js'; import * as P from './playlists.js'; +import { CastButton } from './cast-session.js'; // -- Music -------------------------------------------------------------------- // @@ -747,6 +748,8 @@ function MusicApp({ lists=${playlists} reload=${reloadPlaylists} onPlayQueue=${onPlayQueue} onSync=${syncNow} />`} <${Pager} pager=${pager} /> + ${/* The television the music bar plays to (cast-session.js). */''} + <${CastButton} /> ${!hideFilter && html`<div class="tb-search"> <${Icon} name="search" /> <input type="text" placeholder="${t('group.filter')}" diff --git a/packages/meshbay-hub/src/meshbay_hub/static/music-player.js b/packages/meshbay-hub/src/meshbay_hub/static/music-player.js index e97ce00..4587990 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/music-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/music-player.js @@ -6,7 +6,8 @@ import { t } from './i18n.js'; import { Icon } from './icon.js'; import { useStickyBand } from './sticky.js'; import { CHUNK_SIZE, pipelinedDownload } from './file-utils.js'; -import { playback } from './platform.js'; +import { playback, cast as castBridge } from './platform.js'; +import { castSession, useCastDevice, loadFile, CastButton } from './cast-session.js'; /** * The Music app's persistent player bar (docs/MESHBAY_DESIGN.md §9.8). @@ -30,6 +31,14 @@ import { playback } from './platform.js'; * through `transport.requestAudioTranscode` first — a node-side, * cached-after-once AAC/M4A conversion — before the same download/blob path * runs. + * + * With a television chosen (cast-session.js), the same decrypted track is + * handed to the cast relay instead of the <audio> element, with its cover, + * and the bar becomes the receiver's remote: its play, pause, seek, previous + * and next drive the television, its clock is the television's, and the end + * of a track there moves the queue on as `ended` does here (docs/MESHBAY_DESIGN.md + * §11.4). The element keeps the track's source for its duration, and never + * plays while casting. */ const MIME_BY_EXT = { @@ -351,6 +360,64 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue } }, []); + // ── Casting ────────────────────────────────────────────────────────────── + const castDevice = useCastDevice(); + const remote = Boolean(castDevice) && castSession.canSendFiles; + const remoteRef = useRef(remote); + // Which track the television was last given, where it was when last + // polled, and which track's end has already moved the queue on: the + // receiver keeps reporting FINISHED until it is given something else. + const remoteTrackRef = useRef(null); + const remotePosRef = useRef(null); + const finishedRef = useRef(null); + // True while a track is on its way to the television: the receiver's + // status still describes the previous one until it lands. + const sendingRef = useRef(false); + + // The track's cover, for the television to show beside it — found as the + // album card finds it (music-app.js): this track's own, else another queued + // track of the same album that carries one, else the cover the node found + // for the album. The node answers that last question from its cache, and + // asks MusicBrainz only where the group has it on and the track is tagged. + // A cover that cannot be had is no reason not to play the track. + const fetchCover = useCallback(async (entry) => { + try { + const { transport, gek } = await getConnection(entry.groupId); + if (!transport || !transport.connected) return null; + const sibling = entry.album && tracks.find((tr) => tr.thumb_hash + && tr.groupId === entry.groupId && tr.album === entry.album && tr.artist === entry.artist); + let hash = entry.thumb_hash || (sibling && sibling.thumb_hash) || null; + if (!hash && transport.fetchMusicMeta) { + const meta = await transport.fetchMusicMeta(entry.id).catch(() => null); + hash = (meta && meta.cover_thumb_hash) || null; + } + if (!hash) return null; + const chunks = await pipelinedDownload(transport, gek, hash, 1); + return { bytes: new Uint8Array(await new Blob(chunks).arrayBuffer()), type: 'image/jpeg' }; + } catch { + return null; + } + }, [getConnection, tracks]); + + const sendToTv = useCallback(async (entry, startAt = 0) => { + const cached = blobCacheRef.current.get(entry.id); + if (!cached) throw new Error(t('music.err_playback')); + sendingRef.current = true; + try { + const cover = await fetchCover(entry); + remoteTrackRef.current = entry.id; + remotePosRef.current = startAt; + finishedRef.current = null; + await loadFile({ + owner: 'music', startAt, + bytes: new Uint8Array(await cached.blob.arrayBuffer()), type: cached.mime, cover, + title: entry.display_title || entry.name, artist: entry.artist || '', album: entry.album || '', + }); + } finally { + sendingRef.current = false; + } + }, [fetchCover]); + const fetchTrackBlob = useCallback(async (entry) => { const cached = blobCacheRef.current.get(entry.id); if (cached) return cached.url; @@ -395,7 +462,8 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue const chunks = await pipelinedDownload(transport, gek, downloadId, totalChunks); const blob = new Blob(chunks, { type: mime }); const url = URL.createObjectURL(blob); - blobCacheRef.current.set(entry.id, { url, order: blobInsertRef.current++ }); + // The blob itself too: a cast hands its bytes to the relay. + blobCacheRef.current.set(entry.id, { url, blob, mime, order: blobInsertRef.current++ }); evictOldBlobs(); return url; }, [getConnection, evictOldBlobs]); @@ -520,6 +588,22 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue if (!audio) return; audio.src = url; audio.currentTime = 0; + if (remoteRef.current) { + // To the television; the element keeps the source for its duration. + audio.pause(); + setCurrentTime(0); + try { + await sendToTv(currentTrack, 0); + } catch (castErr) { + castErr.isCast = true; + throw castErr; + } + if (loadTokenRef.current !== myToken) return; + consecutiveFailuresRef.current = 0; + setPlaying(true); + prefetchNext(pos); + return; + } await audio.play(); consecutiveFailuresRef.current = 0; setPlaying(true); @@ -529,6 +613,9 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue // Autoplay refused (no recent tap): the track is loaded and fine, it // waits for the play button. Not an error, and no reason to skip it. if (err && err.name === 'NotAllowedError') { setPlaying(false); prefetchNext(pos); return; } + // The television did not take it: that says nothing about the track, + // so the queue stays where it is, with the reason on screen. + if (err && err.isCast) { setError(err.message || String(err)); setPlaying(false); return; } if (err && err.isGroupDown && asleepOrJustWoke()) { // Our own connection is what went, not the group: keep the spinner // and this track, and try it again once we are back. @@ -557,6 +644,9 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue // stream). Both paths converge on the same bounded advance, so whichever // fires first, the queue moves on exactly once. const handleMediaError = useCallback(() => { + // Casting, the element only holds the source for its duration: whether it + // could decode it says nothing about what the television is playing. + if (remoteRef.current) return; const audio = audioRef.current; const mediaError = audio && audio.error; setError((mediaError && mediaError.message) || t('music.err_playback')); @@ -583,27 +673,111 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue if (order.length === 0) return; // A few seconds in: restart the current track, the way most players do, // rather than always jumping to the previous one. + if (remoteRef.current && castSession.owner === 'music' && (remotePosRef.current || 0) > 3) { + castBridge.chromecastSeek(0).catch(() => {}); + remotePosRef.current = 0; + setCurrentTime(0); + return; + } const audio = audioRef.current; - if (audio && audio.currentTime > 3) { audio.currentTime = 0; return; } + if (!remoteRef.current && audio && audio.currentTime > 3) { audio.currentTime = 0; return; } if (pos > 0) { skipTo(pos - 1); return; } if (repeat === 'all') { skipTo(order.length - 1); } }, [pos, order.length, repeat, skipTo]); const onEnded = useCallback(() => { if (repeat === 'one') { + if (remoteRef.current && currentTrack) { + sendToTv(currentTrack, 0).catch((e) => { setError(e.message || String(e)); setPlaying(false); }); + return; + } const audio = audioRef.current; if (audio) { audio.currentTime = 0; audio.play().catch(() => {}); } return; } skipNext(); - }, [repeat, skipNext]); + }, [repeat, skipNext, currentTrack, sendToTv]); + const onEndedRef = useRef(onEnded); + onEndedRef.current = onEnded; const togglePlaying = useCallback(() => { + if (remoteRef.current && currentTrack) { + // A film or a photo has the television: playing again gives it back + // the track, from where it was. + if (castSession.owner !== 'music') { + sendToTv(currentTrack, remotePosRef.current || 0) + .then(() => { setError(''); setPlaying(true); }) + .catch((e) => { setError(e.message || String(e)); setPlaying(false); }); + return; + } + (playing ? castBridge.chromecastPause() : castBridge.chromecastPlay()) + .then(() => setPlaying(!playing)).catch(() => {}); + return; + } const audio = audioRef.current; if (!audio) return; if (playing) { audio.pause(); setPlaying(false); } else { audio.play().then(() => { setError(''); setPlaying(true); }).catch(() => {}); } - }, [playing]); + }, [playing, currentTrack, sendToTv]); + + // A television chosen, or let go, in the middle of a track: the track goes + // on from where it was, on the other side. + useEffect(() => { + const was = remoteRef.current; + remoteRef.current = remote; + if (was === remote || !currentTrack) return; + const audio = audioRef.current; + if (remote) { + const at = audio ? audio.currentTime : 0; + const wasPlaying = playing; + if (audio) audio.pause(); + // Still loading: the load effect sends it once it has it. + if (!blobCacheRef.current.has(currentTrack.id)) return; + sendToTv(currentTrack, at) + .then(async () => { + if (!wasPlaying) await castBridge.chromecastPause().catch(() => {}); + setPlaying(wasPlaying); + }) + .catch((e) => { setError(e.message || String(e)); setPlaying(false); }); + } else { + const at = remotePosRef.current; + remoteTrackRef.current = null; + if (audio && at != null && isFinite(at)) audio.currentTime = at; + if (playing && audio) audio.play().catch(() => setPlaying(false)); + } + // eslint-disable-next-line + }, [remote]); + + // The television's clock and state, once a second, for as long as the + // track there is ours. + useEffect(() => { + if (!remote) return undefined; + let stopped = false; + const tick = async () => { + if (sendingRef.current || castSession.owner !== 'music' || !remoteTrackRef.current) return; + const s = await castBridge.status().catch(() => null); + const c = s && s.chromecast; + if (stopped || sendingRef.current || castSession.owner !== 'music' || !c) return; + if (!c.connected) { setPlaying(false); return; } + if (c.position != null) { remotePosRef.current = c.position; setCurrentTime(c.position); } + const state = c.playerState; + if (state === 'PLAYING' || state === 'BUFFERING') setPlaying(true); + else if (state === 'PAUSED') setPlaying(false); + else if (state === 'IDLE' && String(c.idleReason || '').toLowerCase() === 'finished' + && finishedRef.current !== remoteTrackRef.current) { + finishedRef.current = remoteTrackRef.current; + onEndedRef.current(); + } + }; + const id = setInterval(tick, 1000); + tick(); + // A film or a photo took the television: the track there has stopped. + // Not when casting stops altogether — the track then carries on here. + const unsubscribe = castSession.subscribe(() => { + if (castSession.device && castSession.owner !== 'music') setPlaying(false); + }); + return () => { stopped = true; clearInterval(id); unsubscribe(); }; + }, [remote]); // Reshuffling keeps the currently playing track in place — turning shuffle // on mid-album must not interrupt what's already playing. Which track that @@ -624,19 +798,36 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue }, []); const seek = useCallback((e) => { + const to = parseFloat(e.target.value); + if (remoteRef.current) { + if (castSession.owner !== 'music') return; + castBridge.chromecastSeek(to).catch(() => {}); + remotePosRef.current = to; + setCurrentTime(to); + return; + } const audio = audioRef.current; - if (audio && isFinite(audio.duration)) audio.currentTime = parseFloat(e.target.value); + if (audio && isFinite(audio.duration)) audio.currentTime = to; }, []); const handleClose = useCallback(() => { const audio = audioRef.current; if (audio) audio.pause(); + // The television stops with the bar; it stays chosen for what comes next. + if (remoteRef.current && castSession.owner === 'music') { + castSession.claim(null); + castBridge.chromecastDisconnect().catch(() => {}); + castBridge.stop().catch(() => {}); + } if (onClose) onClose(); }, [onClose]); if (!currentTrack) return null; const title = currentTrack.display_title || currentTrack.name; + // The element's, once it has read the file; the index's until then — and + // while casting, a file this device cannot decode still has a length. + const shownDuration = (isFinite(duration) && duration > 0) ? duration : (currentTrack.duration || 0); const repeatLabel = repeat === 'off' ? t('music.player_repeat_off') : repeat === 'all' ? t('music.player_repeat_all') : t('music.player_repeat_one'); @@ -678,17 +869,18 @@ function MusicPlayerBar({ getConnection, queue, onClose, userPrefs, onSaveQueue </div> <div class="music-player-seek"> <span class="music-player-time">${formatTime(currentTime)}</span> - <input type="range" min="0" max=${duration || 0} step="1" value=${currentTime} - disabled=${!duration} onInput=${seek} /> - <span class="music-player-time">${formatTime(duration)}</span> + <input type="range" min="0" max=${shownDuration} step="1" value=${currentTime} + disabled=${!shownDuration} onInput=${seek} /> + <span class="music-player-time">${formatTime(shownDuration)}</span> </div> <div class="music-player-volume"> <${Icon} name="volume" /> <input type="range" min="0" max="1" step="0.05" value=${volume} - title=${t('music.player_volume')} + title=${t('music.player_volume')} disabled=${remote} onInput=${(e) => setVolume(parseFloat(e.target.value))} /> </div> <div class="music-player-extra"> + <${CastButton} cls="music-player-btn" up /> <button class="music-player-btn" title=${t('music.player_queue')} onClick=${() => setShowQueue(true)}> <${Icon} name="menu" /> </button> diff --git a/packages/meshbay-hub/src/meshbay_hub/static/platform.js b/packages/meshbay-hub/src/meshbay_hub/static/platform.js index 3a0c628..28817c7 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/platform.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/platform.js @@ -460,14 +460,16 @@ export const cast = { if (!bridge || !bridge.cast || !bridge.cast.subtitle) return null; return bridge.cast.subtitle(sub); }, - // A photo for the receiver: `{ bytes, type }`, already scaled to the - // television. Absent from an older shell, so callers check `canShowImages`. - get canShowImages() { - return Boolean(bridge && bridge.cast && bridge.cast.image); + // A whole file for the receiver — a photo already scaled to the television, + // a music track — as `{ bytes, type, cover?: { bytes, type }, title?, + // artist?, album? }`; answers `{ url }` to load. Absent from an older + // shell, so callers check `canSendFiles`. + get canSendFiles() { + return Boolean(bridge && bridge.cast && bridge.cast.file); }, - async image(img) { - if (!this.canShowImages) return null; - return bridge.cast.image(img); + async file(f) { + if (!this.canSendFiles) return null; + return bridge.cast.file(f); }, async finish() { if (!bridge || !bridge.cast) return false; @@ -529,6 +531,15 @@ export const cast = { if (!this.canControl) return null; return bridge.cast.chromecastPlay(); }, + // Seeking inside a file the receiver plays (a track); a film seeks by + // restarting the relay instead. + get canSeek() { + return Boolean(bridge && bridge.cast && bridge.cast.chromecastSeek); + }, + async chromecastSeek(seconds) { + if (!this.canSeek) return null; + return bridge.cast.chromecastSeek(seconds); + }, }; /** diff --git a/packages/meshbay-hub/src/meshbay_hub/static/style.css b/packages/meshbay-hub/src/meshbay_hub/static/style.css index 9be414f..9383043 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/style.css +++ b/packages/meshbay-hub/src/meshbay_hub/static/style.css @@ -2054,6 +2054,8 @@ button:disabled { opacity: 0.5; cursor: not-allowed; } any other menu there, hung under the button rather than at a point. */ .cast-btn.casting { color: var(--accent); border-color: var(--accent); } .ctx-menu.cast-menu { position: absolute; top: calc(100% + 4px); right: 0; z-index: 150; } +/* From the music bar, at the bottom of the window: the menu opens upward. */ +.cast-menu-up { top: auto !important; bottom: calc(100% + 4px); } .ctx-menu-item.cast-menu-current { color: var(--text-dim); cursor: default; } .ctx-menu-item.cast-menu-current:hover { background: none; } .ctx-menu-item svg { width: 16px; height: 16px; flex-shrink: 0; } diff --git a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js index ed1f18c..37ea727 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/video-player.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/video-player.js @@ -1316,6 +1316,7 @@ function VideoPlayer({ entry, transportRef, gekRef, onClose, onDownload }) { }).then(async (result) => { if (castRestartGenRef.current !== gen) return; if (!result) return; + if (device || castSession.device) castSession.claim('video'); setCastUrl(result.url); const status = await platform.cast.status(); if (status && status.chromecast && status.chromecast.connected) { diff --git a/packages/meshbay-hub/tests/harness/cast_session_probe.py b/packages/meshbay-hub/tests/harness/cast_session_probe.py index 2904860..573dd95 100644 --- a/packages/meshbay-hub/tests/harness/cast_session_probe.py +++ b/packages/meshbay-hub/tests/harness/cast_session_probe.py @@ -40,9 +40,13 @@ FRAME = r"""<!doctype html><html><head><meta charset=utf-8> <script> // A module that fails to load never reaches its own catch: say so. const report = (error) => parent.postMessage({ error }, '*'); -addEventListener('error', (e) => report( - `${e.message || 'load error'} ${e.filename || (e.target && e.target.src) || ''}` - + `:${e.lineno || ''}`), true); +// Scripts only: the music case's stub track is three bytes no <audio> +// element can decode, which is expected and is not the page failing. +addEventListener('error', (e) => { + if (!(e instanceof ErrorEvent) && !(e.target && e.target.tagName === 'SCRIPT')) return; + report(`${e.message || 'load error'} ${e.filename || (e.target && e.target.src) || ''}` + + `:${e.lineno || ''}`); +}, true); addEventListener('unhandledrejection', (e) => report(String((e.reason && e.reason.stack) || e.reason))); const WITH_BRIDGE = !location.search.includes('nobridge'); @@ -111,6 +115,8 @@ window.MeshBayTransport = function () { // The native side, as the Electron preload and the Android shim present it, // recording every cast call. window.CALLS = []; +// What the stub receiver reports: the music case moves it along. +window.TV = { state: 'IDLE', position: 0, idle: null }; if (WITH_BRIDGE) { let connected = false, v = 0; const rec = (name, arg) => { CALLS.push({ name, arg }); }; @@ -127,13 +133,19 @@ if (WITH_BRIDGE) { stop: async () => { rec('stop'); return true; }, subtitle: async () => null, finish: async () => true, - image: async (img) => { - const bmp = await createImageBitmap(new Blob([img.bytes], { type: img.type })); - rec('image', { type: img.type, width: bmp.width, height: bmp.height, - jpeg: img.bytes[0] === 0xff && img.bytes[1] === 0xd8 }); - return { url: 'http://tv/image?t=x&v=' + (++v) }; + file: async (f) => { + let info = { type: f.type, size: f.bytes.length, title: f.title || null, + artist: f.artist || null, album: f.album || null, cover: !!f.cover }; + if (f.type.startsWith('image/')) { + const bmp = await createImageBitmap(new Blob([f.bytes], { type: f.type })); + info = { ...info, width: bmp.width, height: bmp.height, + jpeg: f.bytes[0] === 0xff && f.bytes[1] === 0xd8 }; + } + rec('file', info); + return { url: 'http://tv/file?t=x&v=' + (++v) }; }, - status: async () => ({ active: true, url: null, chromecast: { connected } }), + status: async () => ({ active: true, url: null, chromecast: { + connected, playerState: TV.state, position: TV.position, idleReason: TV.idle } }), scan: async () => { rec('scan'); return true; }, devices: async () => ({ devices: [{ id: 'tv1', name: 'Salon' }], scanning: false }), chromecastConnect: async (o) => { @@ -142,8 +154,9 @@ if (WITH_BRIDGE) { }, chromecastReload: async (o) => { rec('reload', o); return { playerState: 'IDLE' }; }, chromecastDisconnect: async () => { rec('disconnect'); connected = false; return true; }, - chromecastPause: async () => ({}), - chromecastPlay: async () => ({}), + chromecastPause: async () => { rec('pause'); TV.state = 'PAUSED'; return {}; }, + chromecastPlay: async () => { rec('play'); TV.state = 'PLAYING'; return {}; }, + chromecastSeek: async (t) => { rec('seek', t); TV.position = t; return {}; }, }, }; } @@ -161,12 +174,91 @@ async function until(pred, what, tries = 200) { throw new Error('timed out waiting for ' + what); } const calls = (name) => CALLS.filter((c) => c.name === name); +const files = () => calls('file').map((c) => c.arg); // Files is always first; then the enabled apps in their registry order. const videosTab = () => $$('.group-tab')[1]; const count = () => (($('.photo-lightbox-count') || {}).textContent || '').trim(); +// The music bar, with a television already chosen: two tracks of one album. +async function music() { + const { MusicPlayerBar } = await import('/music-player.js'); + const { castSession } = await import('/cast-session.js'); + const out = {}; + castSession.choose({ id: 'tv1', name: 'Salon' }); + // Which hashes were fetched, so a cover can be traced to where it came from. + window.FETCHED = []; + const transport = { connected: true, async waitForReconnect() {}, + async fetchChunk(id) { + FETCHED.push(id); + return { nonce: 'n', ct: new Uint8Array([1, 2, 3]) }; + }, + // The node's MusicBrainz answer: a cover for the other album only. + async fetchMusicMeta(id) { + CALLS.push({ name: 'musicMeta', arg: id }); + return id === 't3' ? { confidence: 1, cover_thumb_hash: 'mb-cover' } + : { confidence: 0 }; + } }; + const getConnection = async () => ({ transport, gek: 'gek' }); + // One has no cover of its own but Two, of the same album, has; Three is of + // another album whose only cover is the one the node found. + const track = (n, title, album, hash) => ({ id: 't' + n, name: title + '.mp3', size: 3, + groupId: 'g1', display_title: title, artist: 'Some Band', album, duration: 200, + thumb_hash: hash }); + const queue = { tracks: [track(1, 'One', 'An Album', null), track(2, 'Two', 'An Album', 'cover2'), + track(3, 'Three', 'Another Album', null)], startIndex: 0, nonce: 1 }; + render(html`<${MusicPlayerBar} getConnection=${getConnection} queue=${queue} + userPrefs=${{}} onClose=${() => {}} />`, document.getElementById('root')); + + await until(() => calls('connect').length === 1, 'the first track on the television'); + out.first = { file: files()[0], connect: calls('connect')[0].arg, + coverFetched: FETCHED.includes('cover2') }; + TV.state = 'PLAYING'; TV.position = 12; + await until(() => ($$('.music-player-time')[0] || {}).textContent === '0:12', + "the television's clock in the bar"); + out.barButton = !!$('.music-player-bar .cast-btn.casting'); + out.volumeDisabled = $('.music-player-volume input').disabled; + + $('.music-player-play').click(); + await until(() => calls('pause').length === 1, 'pause on the television'); + const slider = $('.music-player-seek input'); + slider.value = '30'; + slider.dispatchEvent(new Event('input', { bubbles: true })); + await until(() => calls('seek').length === 1, 'a seek on the television'); + out.seek = calls('seek')[0].arg; + + // The television finishes the track: the next one goes, once. + TV.state = 'IDLE'; TV.idle = 'FINISHED'; + await until(() => calls('reload').length === 1, 'the next track on the television'); + out.second = { file: files()[1], reload: calls('reload')[0].arg, + title: $('.music-player-title').textContent.trim() }; + TV.state = 'PLAYING'; TV.idle = null; TV.position = 1; + await sleep(2500); + out.filesAfterSettling = files().length; + + // Three: the cover comes from the node's answer for its album. + TV.state = 'IDLE'; TV.idle = 'FINISHED'; + await until(() => calls('reload').length === 2, 'the third track on the television'); + out.third = { file: files()[2], mbCover: FETCHED.includes('mb-cover'), + metaAsked: calls('musicMeta').map((c) => c.arg) }; + TV.state = 'PLAYING'; TV.idle = null; + + // Stopping the cast lets the television go. + await castSession.end(); + await sleep(200); + out.afterStop = { disconnects: calls('disconnect').length, + barCasting: !!$('.music-player-bar .cast-btn.casting') }; + parent.postMessage(out, '*'); +} + (async () => { const out = { bridge: !!window.meshbay }; + if (location.search.includes('music')) { + try { await initLocale(); await music(); } + catch (err) { + parent.postMessage({ calls: CALLS, error: String((err && err.stack) || err) }, '*'); + } + return; + } try { await initLocale(); render(html`<${GroupPage} groupId="g1" token="t" username="me" userId="u1" @@ -195,13 +287,13 @@ const count = () => (($('.photo-lightbox-count') || {}).textContent || '').trim( $$('.cast-menu .ctx-menu-item').find((b) => b.textContent.includes('Salon')).click(); await until(() => $('.photo-lightbox'), 'the lightbox to open on choosing'); await until(() => calls('connect').length === 1, 'the first photo to reach the television'); - out.first = { count: count(), image: calls('image')[0], connect: calls('connect')[0].arg, + out.first = { count: count(), image: files()[0], connect: calls('connect')[0].arg, lightboxCasting: !!$('.photo-lightbox .cast-btn.casting') }; // Next photo: the receiver is connected, so it is reloaded. $('.photo-nav-next').click(); await until(() => calls('reload').length === 1, 'the second photo'); - out.second = { count: count(), reload: calls('reload')[0].arg, images: calls('image').length }; + out.second = { count: count(), reload: calls('reload')[0].arg, images: files().length }; // Slideshow: on by itself to the last photo, then off. const play = $$('.photo-lightbox .video-top-bar .video-close') @@ -209,7 +301,7 @@ const count = () => (($('.photo-lightbox-count') || {}).textContent || '').trim( play.click(); await until(() => count() === '3 / 3', 'the slideshow to reach the last photo', 160); await until(() => play.getAttribute('aria-pressed') === 'false', 'the slideshow to end', 160); - out.slideshow = { count: count(), images: calls('image').length, + out.slideshow = { count: count(), images: files().length, reloads: calls('reload').length }; // Close the lightbox; Videos shows the same television; a film opens as its remote. @@ -341,16 +433,17 @@ def check(engine: str, cast: dict, plain: dict) -> list[str]: first = cast["first"] if first["count"] != "1 / 3": bad.append(f"choosing a television did not open the first photo: {first['count']}") - img = first["image"]["arg"] + img = first["image"] if not (img["type"] == "image/jpeg" and img["jpeg"] and img["width"] == 1440 and img["height"] == 1080): bad.append(f"the photo did not leave as a 1440x1080 JPEG: {img}") - if first["connect"] != {"deviceId": "tv1", "mediaUrl": "http://tv/image?t=x&v=1"}: + if first["connect"] != {"deviceId": "tv1", "mediaUrl": "http://tv/file?t=x&v=1", + "startAt": 0}: bad.append(f"connect: {first['connect']}") if not first["lightboxCasting"]: bad.append("the lightbox's cast button does not say it is casting") if cast["second"]["count"] != "2 / 3" or cast["second"]["reload"] != { - "mediaUrl": "http://tv/image?t=x&v=2"}: + "mediaUrl": "http://tv/file?t=x&v=2", "startAt": 0}: bad.append(f"next photo: {cast['second']}") if cast["slideshow"]["count"] != "3 / 3" or cast["slideshow"]["images"] != 3: bad.append(f"slideshow: {cast['slideshow']}") @@ -367,6 +460,39 @@ def check(engine: str, cast: dict, plain: dict) -> list[str]: return [f"{engine}: {b}" for b in bad] +def check_music(engine: str, m: dict) -> list[str]: + if "error" in m: + return [f"{engine}: music: {m['error']}"] + bad = [] + f = m["first"]["file"] + if not (f["type"] == "audio/mpeg" and f["title"] == "One" and f["artist"] == "Some Band" + and f["album"] == "An Album" and f["cover"] and m["first"]["coverFetched"]): + bad.append(f"the first track left without its album's cover: {m['first']}") + third = m["third"] + if not (third["file"]["title"] == "Three" and third["file"]["cover"] and third["mbCover"]): + bad.append(f"the node's cover did not reach the television: {third}") + if "t1" in third["metaAsked"] or "t2" in third["metaAsked"]: + bad.append(f"the node was asked for a cover a track already had: {third['metaAsked']}") + if m["first"]["connect"] != {"deviceId": "tv1", "mediaUrl": "http://tv/file?t=x&v=1", + "startAt": 0}: + bad.append(f"connect: {m['first']['connect']}") + if not m["barButton"]: + bad.append("the music bar's cast button does not say it is casting") + if not m["volumeDisabled"]: + bad.append("the volume slider still acts while casting") + if m["seek"] != 30: + bad.append(f"seek: {m['seek']}") + s = m["second"] + if s["file"]["title"] != "Two" or s["title"] != "Two" or s["reload"] != { + "mediaUrl": "http://tv/file?t=x&v=2", "startAt": 0}: + bad.append(f"the end of a track did not move the queue on: {s}") + if m["filesAfterSettling"] != 2: + bad.append(f"a finished track moved the queue on more than once: {m['filesAfterSettling']}") + if m["afterStop"]["disconnects"] < 1 or m["afterStop"]["barCasting"]: + bad.append(f"stopping: {m['afterStop']}") + return [f"{engine}: music: {b}" for b in bad] + + def main() -> int: failures = [] with socketserver.ThreadingTCPServer(("127.0.0.1", PORT), H) as srv: @@ -377,11 +503,13 @@ def main() -> int: continue cast = browse(engine, "") plain = browse(engine, "?nobridge") - if cast is None or plain is None: + music = browse(engine, "?music") + if cast is None or plain is None or music is None: failures.append(f"{engine}: the page never reported") continue - print(f"{engine}:", json.dumps(cast, indent=1)) + print(f"{engine}:", json.dumps({"photos-videos": cast, "music": music}, indent=1)) failures += check(engine, cast, plain) + failures += check_music(engine, music) for f in failures: print("FAIL", f, file=sys.stderr) if not failures: diff --git a/packages/meshbay-hub/tests/test_android_cast.py b/packages/meshbay-hub/tests/test_android_cast.py index b537df7..ebcdec4 100644 --- a/packages/meshbay-hub/tests/test_android_cast.py +++ b/packages/meshbay-hub/tests/test_android_cast.py @@ -103,12 +103,14 @@ def test_the_receiver_is_pointed_at_the_relay_and_nothing_else(): def test_relay_calls_keep_their_order(): """The page does not await each push; on a pool they could overtake.""" channels = _read(CAST / "CastChannels.kt") - assert '"cast:start", "cast:push", "cast:subtitle", "cast:image", "cast:finish", "cast:stop"' in channels + assert '"cast:start", "cast:push", "cast:subtitle",' in channels + assert ('"cast:file:begin", "cast:file:write", "cast:file:end", "cast:finish", "cast:stop"' + in channels) bridge = _read(SRC / "bridge" / "Bridge.kt") assert "Executors.newSingleThreadExecutor()" in bridge assert "(if (ordered) serial else work).execute" in bridge - assert "fun ordered(frame: BinaryFrame) = frame.channel == BinaryFrame.CAST_PUSH" in _read( - SRC / "bridge" / "Channels.kt") + assert ("fun ordered(frame: BinaryFrame) = frame.channel == BinaryFrame.CAST_PUSH" + " || frame.channel == BinaryFrame.CAST_FILE") in _read(SRC / "bridge" / "Channels.kt") def test_screen_off_survival_is_switched_on_only_while_casting_or_playing(): diff --git a/packages/meshbay-hub/tests/test_cast_files.py b/packages/meshbay-hub/tests/test_cast_files.py new file mode 100644 index 0000000..8f51d80 --- /dev/null +++ b/packages/meshbay-hub/tests/test_cast_files.py @@ -0,0 +1,372 @@ +""" +Whole files on a cast receiver — photos and music tracks — and the television +a session casts to. + +A file takes a different road to the television than a film: no stream, one +file at a time, handed to the relay in pieces (begin, writes, end), served at +`/file` with its cover at `/cover`, and loaded by the default media receiver +as a picture or as a music track. The page scales and re-encodes a photo +first (cast-session.js); a track goes as it is stored. + +The first half runs the desktop relay itself, as test_cast_subtitles.py does: +it is plain Node with no dependencies. The file is behind the same token as +the stream, its address changes with every file — a receiver handed the same +URL twice shows what it already has — ranges are honoured, because that is how +a receiver seeks in a track, and only a picture or a sound is served as one. + +The second half holds the wiring: what the receiver loads is the shell's to +say, never the page's; Android accepts no URL but the relay's own; a +television chosen anywhere is the one a film opens on, with the player as its +remote from the start; and the music bar drives the television once one is +chosen. +""" + +import json +import re +import shutil +import subprocess +from pathlib import Path + +import pytest + +PACKAGES = Path(__file__).resolve().parents[2] +STATIC = PACKAGES / "meshbay-hub" / "src" / "meshbay_hub" / "static" +DESKTOP = PACKAGES / "meshbay-client" / "src" +RELAY = DESKTOP / "cast-relay.js" +CHROMECAST = DESKTOP / "cast-chromecast.js" +ANDROID_MAIN = PACKAGES / "meshbay-android" / "app" / "src" / "main" +ANDROID_SRC = ANDROID_MAIN / "kotlin" +CLIENT = ANDROID_SRC / "org" / "meshbay" / "client" +CAST = CLIENT / "cast" +SHIM = ANDROID_MAIN / "assets" / "bridge" / "meshbay-bridge.js" + + +def _read(path: Path) -> str: + return path.read_text(encoding="utf-8") + + +RELAY_SCRIPT = r""" +const http = require('node:http'); +const CastRelay = require(process.argv[2]); +const get = (url, headers = {}) => new Promise((resolve, reject) => { + http.get(url, { headers }, (res) => { + const parts = []; + res.on('data', (c) => parts.push(c)); + res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, + body: Buffer.concat(parts).toString('hex') })); + }).on('error', reject); +}); +const refused = async (f) => { try { await f(); return false; } catch { return true; } }; +const send = async (relay, bytes, opts) => { + await relay.beginFile({ size: bytes.length, ...opts }); + for (let at = 0; at < bytes.length; at += 3) relay.writeFile(bytes.subarray(at, at + 3)); + return relay.endFile(); +}; +(async () => { + const relay = new CastRelay(); + const out = {}; + out.activeBefore = relay.active; + out.htmlRefused = await refused(() => relay.beginFile({ type: 'text/html', size: 3 })); + out.activeAfterRefusal = relay.active; + + const jpeg = Buffer.from([0xff, 0xd8, 1, 2, 0xff, 0xd9]); + const first = await send(relay, jpeg, { type: 'image/jpeg' }); + out.activeAfterPhoto = relay.active; + out.firstUrl = first.url; + out.photo = await get(first.url); + out.badToken = (await get(first.url.replace(/t=[0-9a-f]+/, 't=x'))).status; + out.photoMeta = relay.fileMeta; + + // A track, its cover and what to show with it; read back whole and by range. + const track = Buffer.from([...Array(20).keys()]); + const second = await send(relay, track, { + type: 'audio/flac', title: 'A Song', artist: 'Some Band', album: 'An Album', + cover: { bytes: Buffer.from([0xff, 0xd8, 7, 0xff, 0xd9]), type: 'image/jpeg' } }); + out.secondUrl = second.url; + out.trackType = relay.fileType; + out.trackMeta = relay.fileMeta; + out.whole = await get(second.url); + out.range = await get(second.url, { Range: 'bytes=5-9' }); + out.tail = await get(second.url, { Range: 'bytes=15-' }); + out.beyond = (await get(second.url, { Range: 'bytes=50-60' })).status; + out.cover = await get(relay.fileMeta.cover); + + // Fewer bytes than announced is not a file; more is refused as they come. + await relay.beginFile({ type: 'audio/mpeg', size: 10 }); + relay.writeFile(Buffer.from([1, 2, 3])); + out.shortRefused = await refused(() => relay.endFile()); + await relay.beginFile({ type: 'audio/mpeg', size: 2 }); + out.longRefused = await refused(() => relay.writeFile(Buffer.from([1, 2, 3]))); + out.stillServed = (await get(second.url)).status; + + await relay.start({}); + out.fileAfterFilm = relay.fileUrl; + await send(relay, jpeg, { type: 'image/jpeg' }); + await relay.stop(); + out.fileAfterStop = relay.fileUrl; + console.log(JSON.stringify(out)); +})().catch((e) => { console.error(e); process.exit(1); }); +""" + + +@pytest.fixture(scope="module") +def relay_run(tmp_path_factory): + if shutil.which("node") is None or not RELAY.exists(): + pytest.skip("node or the desktop client sources are not available") + script = tmp_path_factory.mktemp("cf") / "relay.cjs" + script.write_text(RELAY_SCRIPT, encoding="utf-8") + proc = subprocess.run(["node", str(script), str(RELAY)], + capture_output=True, text=True, encoding="utf-8", timeout=30) + assert proc.returncode == 0, proc.stderr + return json.loads(proc.stdout) + + +def test_a_file_starts_the_relay_and_is_served_behind_its_token(relay_run): + assert relay_run["activeBefore"] is False + assert relay_run["activeAfterPhoto"] is True + photo = relay_run["photo"] + assert photo["status"] == 200 + assert photo["headers"]["content-type"] == "image/jpeg" + assert photo["headers"]["cache-control"] == "no-store" + assert photo["body"] == "ffd80102ffd9" + assert relay_run["badToken"] == 403 + assert relay_run["photoMeta"]["cover"] is None + + +def test_every_file_has_its_own_address(relay_run): + assert re.search(r"/file\?t=[0-9a-f]{32}&v=\d+$", relay_run["firstUrl"]) + assert relay_run["secondUrl"] != relay_run["firstUrl"] + + +def test_a_track_is_served_with_its_cover_and_what_it_is(relay_run): + assert relay_run["trackType"] == "audio/flac" + meta = relay_run["trackMeta"] + assert (meta["title"], meta["artist"], meta["album"]) == ("A Song", "Some Band", "An Album") + assert re.search(r"/cover\?t=[0-9a-f]{32}&v=\d+$", meta["cover"]) + assert relay_run["cover"]["status"] == 200 + assert relay_run["cover"]["body"] == "ffd807ffd9" + assert relay_run["whole"]["body"] == bytes(range(20)).hex() + assert relay_run["whole"]["headers"]["accept-ranges"] == "bytes" + + +def test_a_receiver_seeks_in_a_track_by_range(relay_run): + r = relay_run["range"] + assert r["status"] == 206 + assert r["headers"]["content-range"] == "bytes 5-9/20" + assert r["body"] == bytes(range(5, 10)).hex() + tail = relay_run["tail"] + assert tail["status"] == 206 and tail["body"] == bytes(range(15, 20)).hex() + assert relay_run["beyond"] == 416 + + +def test_only_a_whole_picture_or_sound_is_served(relay_run): + assert relay_run["htmlRefused"] is True + assert relay_run["activeAfterRefusal"] is False, "a refused file starts nothing" + assert relay_run["shortRefused"] is True + assert relay_run["longRefused"] is True + assert relay_run["stillServed"] == 200, "a failed send leaves the shown file alone" + + +def test_a_film_or_a_stop_takes_the_file_down(relay_run): + assert relay_run["fileAfterFilm"] is None + assert relay_run["fileAfterStop"] is None + + +def test_the_receiver_loads_a_photo_as_a_picture_and_a_track_as_music(): + if shutil.which("node") is None or not CHROMECAST.exists(): + pytest.skip("node or the desktop client sources are not available") + script = ( + "const Module = require('node:module');" + "const real = Module._load;" + "Module._load = function (r, ...a) {" + " if (r === 'bonjour-service') return { Bonjour: class {} };" + " if (r === 'castv2-client') return { Client: class {}, DefaultMediaReceiver: {} };" + " return real.call(this, r, ...a); };" + f"const {{ mediaFor, loadOptionsFor }} = require({json.dumps(str(CHROMECAST))});" + "const meta = { title: 'A Song', artist: 'Some Band', album: 'An Album'," + " cover: 'http://x/cover?t=a&v=2' };" + "console.log(JSON.stringify({" + " photo: mediaFor('http://x/file?t=a&v=1', null," + " { contentType: 'image/jpeg', meta: { cover: null } })," + " track: mediaFor('http://x/file?t=a&v=2', null, { contentType: 'audio/flac', meta })," + " from: loadOptionsFor(null, 42.5)," + " film: mediaFor('http://x/stream.mp4?t=a', null, null)," + " filmOptions: loadOptionsFor(null, 0)," + " legacy: mediaFor('http://x/stream.mp4?t=a', null) }));") + proc = subprocess.run(["node", "-e", script], capture_output=True, text=True, + encoding="utf-8", timeout=30) + assert proc.returncode == 0, proc.stderr + out = json.loads(proc.stdout) + assert out["photo"]["contentType"] == "image/jpeg" + assert out["photo"]["streamType"] == "NONE" + assert "tracks" not in out["photo"] + track = out["track"] + assert track["contentType"] == "audio/flac" and track["streamType"] == "BUFFERED" + assert track["metadata"] == {"metadataType": 3, "title": "A Song", "artist": "Some Band", + "albumName": "An Album", + "images": [{"url": "http://x/cover?t=a&v=2"}]} + assert out["from"]["currentTime"] == 42.5 + assert "currentTime" not in out["filmOptions"] + assert out["film"] == out["legacy"], "a film is loaded exactly as before" + assert out["film"]["streamType"] == "LIVE" and out["film"]["contentType"] == "video/mp4" + + +def test_the_shell_not_the_page_says_what_the_receiver_loads(): + """The page names a URL; what it is, is the relay's to say. A page that + could name the content type could have a television load the stream as + anything at all.""" + main = _read(DESKTOP / "main.js") + load_of = main.split("const loadOf = ", 1)[1].split("\n\n", 1)[0] + assert "mediaUrl === castRelay.fileUrl" in load_of + assert "contentType: castRelay.fileType, meta: castRelay.fileMeta" in load_of + assert "subtitle: null" in load_of, "a file carries no subtitle" + assert main.count("loadOf(mediaUrl, subtitle)") == 2, "connect and reload both" + # A start position means something for a file only; a film starts where + # its relay does. + assert main.count("l.file ? startAt : 0") == 2 + + +def test_android_accepts_the_relays_file_and_nothing_else(): + if not CAST.exists(): + pytest.skip("android sources not present") + channels = _read(CAST / "CastChannels.kt") + check = channels.split("private fun relayUrl(", 1)[1].split("\n }", 1)[0] + assert "asked != ours && asked != relay.fileUrl" in check + assert 'put("contentType", relay.fileType).put("meta", relay.fileMeta()' in channels + assert "url == relay.fileUrl -> null" in channels, "a file carries no subtitle" + assert 'if (url == relay.fileUrl) maxOf(0.0, o.optDouble("startAt", 0.0)) else 0.0' in channels + # A file is not a cast that has to survive the screen going off. + begin = channels.split('"cast:file:begin" ->', 1)[1].split('"cast:file:end"', 1)[0] + assert "onCasting" not in begin + relay = _read(CAST / "CastRelay.kt") + assert '"/file" -> serveFile(out, socket, range)' in relay + assert '"/cover" -> serveCover(out, socket)' in relay + serve = relay.split("private fun serve(", 1)[1] + assert serve.index('query["t"] != token') < serve.index('"/file"'), "behind the token" + # On disk, never held: a long lossless track is a few hundred megabytes. + assert 'File(spoolDir, "file-${fileSeq.incrementAndGet()}.bin")' in relay + assert "java.io.FileOutputStream(slot.path)" in relay + control = _read(CAST / "CastControl.kt") + assert "MediaInfo.STREAM_TYPE_NONE" in control + assert "MediaMetadata.MEDIA_TYPE_PHOTO" in control + assert "MediaMetadata.MEDIA_TYPE_MUSIC_TRACK" in control + assert "MediaInfo.STREAM_TYPE_BUFFERED" in control + assert "setCurrentTime((startAt * 1000).toLong())" in control + assert "MediaSeekOptions.Builder().setPosition(" in control + # Loaded is shown: a photo never reaches PLAYING, and waiting for it + # would hold every photo for the full fifteen seconds. + load = control.split("private fun load(", 1)[1] + assert load.index("if (photo) return") < load.index("PLAY_WAIT_MS") + + +def test_a_file_crosses_the_android_bridge_in_binary_pieces(): + """One message the size of a lossless track would be the WebView's to + refuse; a file goes in pieces of a megabyte, as binary frames where the + WebView has them.""" + if not SHIM.exists(): + pytest.skip("android sources not present") + shim = _read(SHIM) + assert "const CAST_FILE = 3;" in shim + assert "const FILE_PIECE = 1024 * 1024;" in shim + assert ("BINARY ? sendBinary(CAST_FILE, 0, piece) : call('cast:file:write', base64Of(piece))" + in shim) + assert "const val CAST_FILE = 3" in _read(CLIENT / "save" / "SaveNames.kt") + channels = _read(CLIENT / "bridge" / "Channels.kt") + assert "BinaryFrame.CAST_FILE -> " in channels and ".writeFile(frame.bytes" in channels + + +def test_a_television_chosen_anywhere_takes_the_next_film(): + """The player opens as the remote of a television chosen before it, the + cast pending until the first segment exactly as a restart is; choosing one + in the player keeps it for the session, and stopping there forgets it.""" + player = _read(STATIC / "video-player.js") + assert ("const [autoCast] = useState(() => " + "(castSession.supported ? castSession.device : null));") in player + assert "useState(Boolean(autoCast))" in player + assert "castActiveRef = useRef(Boolean(autoCast))" in player + assert "castRestartPendingRef = useRef(Boolean(autoCast))" in player + assert "castDeviceRef = useRef(autoCast)" in player + assert "useState(autoCast ? autoCast.name : null)" in player + stop = player.split("const stopCast = async () => {", 1)[1].split("};", 1)[0] + assert "castSession.forget()" in stop + assert "castSession.choose(d);" in player + assert "castSession.claim('video')" in player + + +def test_the_cast_button_is_where_the_media_are(): + """Videos' toolbar, the Photos landing, a photo album's bar and Music's + toolbar are drawn by the group page and the search page alike, so one + button in each covers both. On the Photos landing the button keeps a + toolbar Search would otherwise hide. The music bar has its own, opening + upward from the bottom of the window.""" + videos = _read(STATIC / "video-app.js") + assert "<${CastButton} />" in videos.split('<div class="video-toolbar">', 1)[1] + photos = _read(STATIC / "photos-app.js") + landing = photos.split("function PhotosApp(", 1)[1] + assert "(!hideFilter || castSession.supported) && html`" in landing + assert "<${CastButton} />" in landing.split('<div class="photo-toolbar">', 1)[1] + album = photos.split("function AlbumView(", 1)[1] + assert "<${CastButton}" in album + lightbox = photos.split("function Lightbox(", 1)[1].split("function AlbumView(", 1)[0] + assert '<${CastButton} variant="overlay"' in lightbox + assert "if (castDevice && blobUrl && blob) showPhoto(blob);" in lightbox + music = _read(STATIC / "music-app.js") + assert "<${CastButton} />" in music.split('<div class="video-toolbar">', 1)[1] + bar = _read(STATIC / "music-player.js") + assert '<${CastButton} cls="music-player-btn" up />' in bar + # The group page draws them through the app registry, the search page + # imports them: the same components either way. + registry = _read(STATIC / "apps.js") + search = _read(STATIC / "search-page.js") + for page in (registry, search): + for app in ("VideoApp", "PhotosApp", "MusicApp"): + assert app in page + + +def test_the_music_bar_drives_the_television(): + """With a television chosen, the decrypted track goes to the relay instead + of the element, and the bar's controls and clock are the receiver's. The + receiver keeps saying FINISHED until it is given something else, so the + end of a track moves the queue on once; and a television a film or a + photo has taken is not the bar's to read.""" + bar = _read(STATIC / "music-player.js") + assert "const remote = Boolean(castDevice) && castSession.canSendFiles;" in bar + send = bar.split("const sendToTv = useCallback(", 1)[1].split("}, [fetchCover]);", 1)[0] + assert "owner: 'music', startAt," in send + assert "type: cached.mime, cover," in send + assert "sendingRef.current = true;" in send + load = bar.split("if (remoteRef.current) {\n // To the television", 1)[1] + assert "audio.pause();" in load.split("await audio.play();", 1)[0] + tick = bar.split("const tick = async () => {", 1)[1].split("const id = setInterval", 1)[0] + assert "castSession.owner !== 'music'" in tick + assert "sendingRef.current" in tick + assert "finishedRef.current !== remoteTrackRef.current" in tick + assert "String(c.idleReason || '').toLowerCase() === 'finished'" in tick + # The cover is found as the album card finds it, not only on the track. + cover = bar.split("const fetchCover = useCallback(", 1)[1] + cover = cover.split("}, [getConnection, tracks]);", 1)[0] + assert "entry.thumb_hash || (sibling && sibling.thumb_hash)" in cover + assert "transport.fetchMusicMeta(entry.id)" in cover + assert "meta.cover_thumb_hash" in cover + assert "castBridge.chromecastSeek(to)" in bar + assert "castBridge.chromecastPause() : castBridge.chromecastPlay()" in bar + # Stopping the cast carries the track on here; a film taking the + # television pauses the bar. + assert "if (castSession.device && castSession.owner !== 'music') setPlaying(false);" in bar + + +def test_a_photo_leaves_scaled_and_upright(): + session = _read(STATIC / "cast-session.js") + assert "imageOrientation: 'from-image'" in session + assert "const TV_WIDTH = 1920;" in session and "const TV_HEIGHT = 1080;" in session + assert "Math.min(1, TV_WIDTH / bitmap.width, TV_HEIGHT / bitmap.height)" in session + assert "await loadFile({ owner: 'photo', bytes, type: 'image/jpeg' });" in session + # Only the newest photo asked for is sent. + assert session.count("gen !== photoGen") >= 2 + + +def test_every_locale_names_the_slideshow(): + for f in sorted((STATIC / "locales").glob("*.js")): + text = f.read_text(encoding="utf-8") + for key in ("photo.slideshow", "photo.slideshow_stop"): + assert f"'{key}':" in text, f"{f.name} lacks {key}" diff --git a/packages/meshbay-hub/tests/test_cast_photos.py b/packages/meshbay-hub/tests/test_cast_photos.py deleted file mode 100644 index 699e663..0000000 --- a/packages/meshbay-hub/tests/test_cast_photos.py +++ /dev/null @@ -1,240 +0,0 @@ -""" -Photos on a cast receiver, and the television a session casts to. - -A photo takes a different road to the television than a film: no stream, one -picture at a time, served by the relay at `/image` and loaded as a picture by -the default media receiver. The page scales and re-encodes it first -(cast-session.js), so what reaches the relay is a JPEG the receiver can show. - -The first half runs the desktop relay itself, as test_cast_subtitles.py does: -it is plain Node with no dependencies. The photo is behind the same token as -the stream, its address changes with every photo — a receiver handed the same -URL twice shows what it already has — and only a picture is served as one. - -The second half holds the wiring: the receiver is told a photo is a photo by -the shell, never by the page; Android accepts no URL but the relay's own; and -a television chosen anywhere is the one a film opens on, with the player as -its remote from the start. -""" - -import json -import re -import shutil -import subprocess -from pathlib import Path - -import pytest - -PACKAGES = Path(__file__).resolve().parents[2] -STATIC = PACKAGES / "meshbay-hub" / "src" / "meshbay_hub" / "static" -DESKTOP = PACKAGES / "meshbay-client" / "src" -RELAY = DESKTOP / "cast-relay.js" -CHROMECAST = DESKTOP / "cast-chromecast.js" -ANDROID_SRC = PACKAGES / "meshbay-android" / "app" / "src" / "main" / "kotlin" -CAST = ANDROID_SRC / "org" / "meshbay" / "client" / "cast" - - -def _read(path: Path) -> str: - return path.read_text(encoding="utf-8") - - -RELAY_SCRIPT = r""" -const http = require('node:http'); -const CastRelay = require(process.argv[2]); -const get = (url) => new Promise((resolve, reject) => { - http.get(url, (res) => { - const parts = []; - res.on('data', (c) => parts.push(c)); - res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, - body: Buffer.concat(parts).toString('hex') })); - }).on('error', reject); -}); -(async () => { - const relay = new CastRelay(); - const out = {}; - out.activeBefore = relay.active; - try { - await relay.showImage({ bytes: Buffer.from('<p>'), type: 'text/html' }); - out.htmlRefused = false; - } catch { out.htmlRefused = true; } - out.activeAfterRefusal = relay.active; - - const jpeg = Buffer.from([0xff, 0xd8, 1, 2, 0xff, 0xd9]); - const first = await relay.showImage({ bytes: jpeg, type: 'image/jpeg' }); - out.activeAfterPhoto = relay.active; - out.firstUrl = first.url; - out.imageType = relay.imageType; - out.photo = await get(first.url); - out.badToken = (await get(first.url.replace(/t=[0-9a-f]+/, 't=x'))).status; - const second = await relay.showImage({ bytes: Buffer.from([9]), type: 'image/jpeg' }); - out.secondUrl = second.url; - out.second = await get(second.url); - - await relay.start({}); - out.imageAfterFilm = relay.imageUrl; - await relay.stop(); - out.imageAfterStop = relay.imageUrl; - console.log(JSON.stringify(out)); -})().catch((e) => { console.error(e); process.exit(1); }); -""" - - -@pytest.fixture(scope="module") -def relay_run(tmp_path_factory): - if shutil.which("node") is None or not RELAY.exists(): - pytest.skip("node or the desktop client sources are not available") - script = tmp_path_factory.mktemp("cp") / "relay.cjs" - script.write_text(RELAY_SCRIPT, encoding="utf-8") - proc = subprocess.run(["node", str(script), str(RELAY)], - capture_output=True, text=True, encoding="utf-8", timeout=30) - assert proc.returncode == 0, proc.stderr - return json.loads(proc.stdout) - - -def test_a_photo_starts_the_relay_and_is_served_behind_its_token(relay_run): - assert relay_run["activeBefore"] is False - assert relay_run["activeAfterPhoto"] is True - photo = relay_run["photo"] - assert photo["status"] == 200 - assert photo["headers"]["content-type"] == "image/jpeg" - assert photo["headers"]["cache-control"] == "no-store" - assert photo["body"] == "ffd80102ffd9" - assert relay_run["badToken"] == 403 - assert relay_run["imageType"] == "image/jpeg" - - -def test_every_photo_has_its_own_address(relay_run): - assert re.search(r"/image\?t=[0-9a-f]{32}&v=\d+$", relay_run["firstUrl"]) - assert relay_run["secondUrl"] != relay_run["firstUrl"] - assert relay_run["second"]["body"] == "09" - - -def test_only_a_picture_is_served_as_one(relay_run): - assert relay_run["htmlRefused"] is True - assert relay_run["activeAfterRefusal"] is False, "a refused photo starts nothing" - - -def test_a_film_or_a_stop_takes_the_photo_down(relay_run): - assert relay_run["imageAfterFilm"] is None - assert relay_run["imageAfterStop"] is None - - -def test_the_receiver_loads_a_photo_as_a_picture(): - if shutil.which("node") is None or not CHROMECAST.exists(): - pytest.skip("node or the desktop client sources are not available") - script = ( - "const Module = require('node:module');" - "const real = Module._load;" - "Module._load = function (r, ...a) {" - " if (r === 'bonjour-service') return { Bonjour: class {} };" - " if (r === 'castv2-client') return { Client: class {}, DefaultMediaReceiver: {} };" - " return real.call(this, r, ...a); };" - f"const {{ mediaFor }} = require({json.dumps(str(CHROMECAST))});" - "console.log(JSON.stringify({" - " photo: mediaFor('http://x/image?t=a&v=1', null, 'image/jpeg')," - " film: mediaFor('http://x/stream.mp4?t=a', null, 'video/mp4')," - " legacy: mediaFor('http://x/stream.mp4?t=a', null) }));") - proc = subprocess.run(["node", "-e", script], capture_output=True, text=True, - encoding="utf-8", timeout=30) - assert proc.returncode == 0, proc.stderr - out = json.loads(proc.stdout) - assert out["photo"]["contentType"] == "image/jpeg" - assert out["photo"]["streamType"] == "NONE" - assert "tracks" not in out["photo"] - assert out["film"] == out["legacy"], "a film is loaded exactly as before" - assert out["film"]["streamType"] == "LIVE" and out["film"]["contentType"] == "video/mp4" - - -def test_the_shell_not_the_page_says_what_the_receiver_loads(): - """The page names a URL; whether it is a photo is the relay's to say. A - page that could name the content type could have a television load the - stream URL as anything at all.""" - main = _read(DESKTOP / "main.js") - load_of = main.split("const loadOf = ", 1)[1].split("\n\n", 1)[0] - assert "mediaUrl === castRelay.imageUrl" in load_of - assert "contentType: castRelay.imageType" in load_of - assert "contentType: 'video/mp4'" in load_of - assert "subtitle: null" in load_of, "a photo carries no subtitle" - assert main.count("loadOf(mediaUrl, subtitle)") == 2, "connect and reload both" - - -def test_android_accepts_the_relays_photo_and_nothing_else(): - if not CAST.exists(): - pytest.skip("android sources not present") - channels = _read(CAST / "CastChannels.kt") - check = channels.split("private fun relayUrl(", 1)[1].split("\n }", 1)[0] - assert "asked != ours && asked != relay.imageUrl" in check - assert "if (url == relay.imageUrl) relay.imageType" in channels - assert "url == relay.imageUrl -> null" in channels, "a photo carries no subtitle" - # A photo is not a cast that has to survive the screen going off. - image = channels.split('"cast:image" ->', 1)[1].split('\n "', 1)[0] - assert "onCasting" not in image - relay = _read(CAST / "CastRelay.kt") - assert '"/image" -> serveImage(out, socket)' in relay - serve = relay.split("private fun serve(", 1)[1] - assert serve.index('query["t"] != token') < serve.index('"/image"'), "behind the token" - control = _read(CAST / "CastControl.kt") - assert "MediaInfo.STREAM_TYPE_NONE" in control - assert "MediaMetadata.MEDIA_TYPE_PHOTO" in control - # Loaded is shown: a photo never reaches PLAYING, and waiting for it - # would hold every photo for the full fifteen seconds. - load = control.split("private fun load(", 1)[1] - assert load.index("if (photo) return") < load.index("PLAY_WAIT_MS") - - -def test_a_television_chosen_anywhere_takes_the_next_film(): - """The player opens as the remote of a television chosen before it, the - cast pending until the first segment exactly as a restart is; choosing one - in the player keeps it for the session, and stopping there forgets it.""" - player = _read(STATIC / "video-player.js") - assert ("const [autoCast] = useState(() => " - "(castSession.supported ? castSession.device : null));") in player - assert "useState(Boolean(autoCast))" in player - assert "castActiveRef = useRef(Boolean(autoCast))" in player - assert "castRestartPendingRef = useRef(Boolean(autoCast))" in player - assert "castDeviceRef = useRef(autoCast)" in player - assert "useState(autoCast ? autoCast.name : null)" in player - stop = player.split("const stopCast = async () => {", 1)[1].split("};", 1)[0] - assert "castSession.forget()" in stop - assert "castSession.choose(d);" in player - - -def test_the_cast_button_is_where_the_media_are(): - """Videos' toolbar, the Photos landing and a photo album's bar are drawn by - the group page and the search page alike, so one button in each covers - both. On the landing the button keeps a toolbar Search would otherwise - hide.""" - videos = _read(STATIC / "video-app.js") - assert "<${CastButton} />" in videos.split('<div class="video-toolbar">', 1)[1] - photos = _read(STATIC / "photos-app.js") - landing = photos.split("function PhotosApp(", 1)[1] - assert "(!hideFilter || castSession.supported) && html`" in landing - assert "<${CastButton} />" in landing.split('<div class="photo-toolbar">', 1)[1] - album = photos.split("function AlbumView(", 1)[1] - assert "<${CastButton}" in album - lightbox = photos.split("function Lightbox(", 1)[1].split("function AlbumView(", 1)[0] - assert '<${CastButton} variant="overlay"' in lightbox - assert "if (castDevice && blobUrl && blob) showPhoto(blob);" in lightbox - # The group page draws them through the app registry, the search page - # imports them: the same components either way. - registry = _read(STATIC / "apps.js") - search = _read(STATIC / "search-page.js") - for page in (registry, search): - assert "VideoApp" in page and "PhotosApp" in page - - -def test_a_photo_leaves_scaled_and_upright(): - session = _read(STATIC / "cast-session.js") - assert "imageOrientation: 'from-image'" in session - assert "const TV_WIDTH = 1920;" in session and "const TV_HEIGHT = 1080;" in session - assert "Math.min(1, TV_WIDTH / bitmap.width, TV_HEIGHT / bitmap.height)" in session - assert "'image/jpeg'" in session - # Only the newest photo asked for is sent. - assert session.count("gen !== photoGen") >= 2 - - -def test_every_locale_names_the_slideshow(): - for f in sorted((STATIC / "locales").glob("*.js")): - text = f.read_text(encoding="utf-8") - for key in ("photo.slideshow", "photo.slideshow_stop"): - assert f"'{key}':" in text, f"{f.name} lacks {key}" |