diff options
13 files changed, 210 insertions, 22 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index b720623a..1480cd57 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -4179,6 +4179,15 @@ narrow bridge — and Android has all three: while Android will still ask, then the application's settings. Moving to SDK 37 also took the permission back from an installed application, which had it implied before: the first group opened after the update asks. + A phone connected on mobile data that walks onto the node's Wi-Fi without + the permission, in the application or in Chrome, loses its connection and + is refused on every reconnect attempt: each refusal reaches the group page + (`addReconnectFailureListener`), which says the same as at opening over the + view it had, while the reconnect goes on. The page's buttons ask that + reconnect to try now rather than tear it down, so a download or a film + under way resumes once the permission is granted or mobile data is back; + an attempt still waiting on ICE is given up for a new one, since Chrome + keeps refusing what it dropped when that attempt began. - **The page hears when the phone changes network** (`meshbay-default-network`, from the default network callback), mobile data to Wi-Fi or back, and checks the connection it holds (transport.js `onNetworkChange`). The event carries @@ -4819,7 +4828,6 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | **A queued transfer logs "no answer" every minute** | The lease watchdog re-asks after 60 s of silence whatever the last answer was, so a lease the node has answered `queued` and that has not moved prints "no answer for transfer ... asking again" once a minute for as long as it waits, which reads as a fault | | **"Prune stale entries" does not touch the index** | The Node page's maintenance button runs `prune_index_cache`, which drops rows of the hash cache and nothing else. An operator who sees a deleted file still listed reaches for it, and nothing changes: only the reconcile removes an index entry, and nothing lets the operator ask for one | | **Only the group page explains a refused local network** | `transport.js` names it (`local_network_blocked`, Android 17 and Chrome 155), and `group-page.js` says what to change. Search and the other screens that open a connection show it as a node that cannot be reached | -| **A reconnect refused the local network says nothing** | A phone connected on mobile data that walks onto the node's Wi-Fi without the permission loses the connection when Android drops mobile data, and `_reconnectLoop` retries with the reason in the console and nowhere else | --- diff --git a/packages/meshbay-hub/src/meshbay_hub/static/group-page.js b/packages/meshbay-hub/src/meshbay_hub/static/group-page.js index 45049a16..304a28fb 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/group-page.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/group-page.js @@ -247,6 +247,9 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, // In the Android application, when the local network was refused: whether // Android will still ask ("prompt") or only its settings can change it. const [lanBlocked, setLanBlocked] = useState(''); + // Set while the message is a reconnect refused the local network: the + // reconnect that gets through takes it away. + const lanLostRef = useRef(false); const transportRef = useRef(null); const gekRef = useRef(null); // One refresh per group: if a fresh token still says we are not a member, we @@ -394,6 +397,7 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, // Dropped by the teardown below, so a transport handed on to a running // download (`releaseWhenIdle`) stops driving a page that is gone. let offReconnect = null; + let offRefused = null; let deviceRetry = null; // The cache is written here and read only by the search page. It used to @@ -402,10 +406,27 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, // cache landing afterwards. Files shows what the node says, or says it // cannot reach the node. + // This device dropped the node's local addresses (transport.js + // localNetworkRefused): a setting to change here, not a node to wait for. + // Android 17 gates it per application, Chrome from 155. In the application, + // whether Android will still ask or only its settings can change it. + const explainLanRefusal = async () => { + if (!platform.localNetwork.available) { + return t(/Android/i.test(navigator.userAgent) + ? 'group.lan_blocked_android' : 'group.lan_blocked_browser'); + } + try { + const state = await platform.localNetwork.state(); + if (!cancelled) setLanBlocked(state === 'granted' ? '' : state); + } catch { /* the message stands alone */ } + return t('group.lan_blocked_app'); + }; + const connect = async () => { setStatus('discovering'); setError(''); setLanBlocked(''); + lanLostRef.current = false; // Belongs to the connection about to be made, not the group just left. setDeviceReady(false); gekRef.current = null; @@ -632,8 +653,24 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, // full fetch rather than a delta: this session was never told what it // missed, and a delta is computed against a snapshot only the node // has. + // Connected on mobile data, then on the node's Wi-Fi without the + // permission: every attempt is refused and the reconnect goes on, so a + // permission granted or mobile data again reaches the group by itself. + // Said while it lasts, over the view the group had. + offRefused = transport.addReconnectFailureListener(async (e) => { + if (cancelled || e.reason !== 'local_network_blocked') return; + const message = await explainLanRefusal(); + if (cancelled) return; + lanLostRef.current = true; + setError(message); + }); offReconnect = transport.addReconnectListener((reack) => { if (cancelled) return; + if (lanLostRef.current) { + lanLostRef.current = false; + setError(''); + setLanBlocked(''); + } (async () => { try { applyAck(reack); @@ -721,19 +758,9 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, if (err.reason === 'no_keys') setNeedsPass(true); // An identity sealed before the pepper: only the operator can clear it. if (err.reason === 'bundle_format_retired') err.message = t('group.bundle_format_retired'); - // This device dropped the node's local addresses (transport.js - // localNetworkRefused): a setting to change here, not a node to wait - // for. Android 17 gates it per application, Chrome from 155. - if (err.reason === 'local_network_blocked' && platform.localNetwork.available) { - err.message = t('group.lan_blocked_app'); - try { - const state = await platform.localNetwork.state(); - if (cancelled) return; - setLanBlocked(state === 'granted' ? '' : state); - } catch { /* the message stands alone */ } - } else if (err.reason === 'local_network_blocked') { - err.message = t(/Android/i.test(navigator.userAgent) - ? 'group.lan_blocked_android' : 'group.lan_blocked_browser'); + if (err.reason === 'local_network_blocked') { + err.message = await explainLanRefusal(); + if (cancelled) return; } // A key this node has never pinned, for an account it knows. The way in // is a device already trusted here, not an operator — which is the @@ -787,6 +814,7 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, return () => { cancelled = true; if (offReconnect) { offReconnect(); offReconnect = null; } + if (offRefused) { offRefused(); offRefused = null; } clearTimeout(deviceRetry); // Nothing will update this group's dock row once the page lets go of it. reportIndexPush(groupId, null); @@ -814,6 +842,14 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, // it; nothing in between should disturb a working connection. }, [groupId, Boolean(token), retryKey]); + // A connection still reconnecting is asked to try now rather than torn down: + // what it carries (a download, a film) goes on once it gets through. + const retry = useCallback(() => { + const transport = transportRef.current; + if (transport && transport.reconnecting) transport.retryNow(); + else setRetryKey((k) => k + 1); + }, []); + // A group that could not be reached is tried again when the device joins // another network: from the Wi-Fi to mobile data, or back, is what decides // whether the node can be reached, and in the Android application there is @@ -1061,12 +1097,12 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, <p class="settings-hint" style="margin-bottom:12px">${t('group.browser_access_off')}</p>`} ${error && html`<div class="error-msg" style="margin-bottom:12px">${error}${' '} <button class="admin-btn" style="margin-left:8px;font-size:0.9em" - onClick=${() => setRetryKey(k => k + 1)}>${t('group.retry')}</button> + onClick=${retry}>${t('group.retry')}</button> </div>`} ${lanBlocked === 'prompt' && html` <button class="admin-btn" style="margin-bottom:12px" onClick=${async () => { try { await platform.localNetwork.request(); } catch { /* retried either way */ } - setRetryKey(k => k + 1); + retry(); }}>${t('group.lan_allow')}</button>`} ${lanBlocked === 'denied' && html` <p class="settings-hint">${t('group.lan_settings_hint')}</p> @@ -1075,7 +1111,7 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs, const back = () => { if (document.visibilityState !== 'visible') return; document.removeEventListener('visibilitychange', back); - setRetryKey(k => k + 1); + retry(); }; document.addEventListener('visibilitychange', back); platform.localNetwork.openSettings().catch(() => { diff --git a/packages/meshbay-hub/src/meshbay_hub/static/transport.js b/packages/meshbay-hub/src/meshbay_hub/static/transport.js index 0b796cab..11088adc 100644 --- a/packages/meshbay-hub/src/meshbay_hub/static/transport.js +++ b/packages/meshbay-hub/src/meshbay_hub/static/transport.js @@ -747,6 +747,9 @@ class MeshBayTransport { // and a single setter meant the second to arrive silently replaced // the first, then cleared it on the way out. this._reconnectListeners = new Set(); + // Told of each attempt that failed, with its error: a reconnect refused + // the local network tries again for ever and says nothing otherwise. + this._reconnectFailureListeners = new Set(); this._onNeedToken = null; // Which device key THIS connection has identified itself to the node with. // Empty means "not identified": nothing can be sealed, so nothing can be @@ -762,6 +765,9 @@ class MeshBayTransport { // connect() like the diagnostic listener above it — this one has to // survive every reconnect attempt, not restart with each one. this._reconnectWakeResolve = null; + // Set by retryNow for the attempt it gave up: the next waits for nothing. + this._retryAtOnce = false; + this._abandonAttempt = null; this._onVisibilityWake = () => { if (document.visibilityState === 'visible') this._wakeReconnect(); }; @@ -823,6 +829,21 @@ class MeshBayTransport { } } + /** + * Tries again now: the person has just changed what made it fail. An + * attempt under way set up its connection before that change (a permission + * granted), and Chrome keeps refusing the addresses it dropped then until + * ICE gives up, half a minute: it is given up here, and the next starts at + * once. + */ + retryNow() { + if (this._inReconnectAttempt && this._abandonAttempt) { + this._retryAtOnce = true; + this._abandonAttempt(); + } + this._wakeReconnect(); + } + get connected() { return this._connected; } /** True while _reconnectLoop is running, backoff included. */ get reconnecting() { return !!this._reconnectPromise; } @@ -864,6 +885,17 @@ class MeshBayTransport { } /** + * Told of each automatic reconnect attempt that failed, with its error, while + * the reconnect goes on. A phone that leaves mobile data for the node's Wi-Fi + * without the local network permission is refused on every attempt + * (`reason` local_network_blocked), and only the page can say why. + */ + addReconnectFailureListener(fn) { + this._reconnectFailureListeners.add(fn); + return () => this._reconnectFailureListeners.delete(fn); + } + + /** * Told whenever this connection's device identity changes — including to * *nothing*, which is the case that mattered. * @@ -1007,6 +1039,8 @@ class MeshBayTransport { // DataChannel closed" on the console. Every failed reconnect attempt // printed one, which is noise in exactly the log a freeze gets read from. channelReady.catch(() => { /* the awaiter below reports it */ }); + // retryNow; a no-op once the channel is open. + this._abandonAttempt = () => channelReject(new Error('Tried again at once')); this._channel.onmessage = (event) => this._onMessage(event.data); const channel = this._channel; @@ -1698,7 +1732,9 @@ class MeshBayTransport { this._reconnectAttempts = 0; while (!this._closed) { this._reconnectAttempts += 1; - const delayMs = Math.min(30000, 1000 * 2 ** (this._reconnectAttempts - 1)); + const delayMs = this._retryAtOnce ? 0 + : Math.min(30000, 1000 * 2 ** (this._reconnectAttempts - 1)); + this._retryAtOnce = false; trace('reconnect_wait', { attempt: this._reconnectAttempts, delay_ms: delayMs }); // Interruptible: _wakeReconnect (fired on visibilitychange → visible) // resolves this immediately instead of waiting out the rest of a @@ -1752,6 +1788,11 @@ class MeshBayTransport { }); console.warn('[MeshBay] Reconnect attempt', this._reconnectAttempts, 'failed:', e.message); + for (const fn of [...this._reconnectFailureListeners]) { + try { fn(e); } catch (err) { + console.error('[MeshBay] reconnect failure listener threw:', err); + } + } // Loop again with a longer backoff — closing over `args`/`token` // freshly next time, in case the token was the actual problem. } diff --git a/packages/meshbay-hub/tests/harness/cast_session_probe.py b/packages/meshbay-hub/tests/harness/cast_session_probe.py index 573dd950..901d27cd 100644 --- a/packages/meshbay-hub/tests/harness/cast_session_probe.py +++ b/packages/meshbay-hub/tests/harness/cast_session_probe.py @@ -99,6 +99,7 @@ window.MeshBayTransport = function () { // One chunk per photo: every photo is the same picture. async fetchChunk() { return { nonce: 'n', ct: await photoBytes() }; }, addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/group_tab_probe.py b/packages/meshbay-hub/tests/harness/group_tab_probe.py index e23a9462..a430c625 100644 --- a/packages/meshbay-hub/tests/harness/group_tab_probe.py +++ b/packages/meshbay-hub/tests/harness/group_tab_probe.py @@ -73,6 +73,7 @@ window.MeshBayTransport = class { // unmount; a stub without this throws inside connect() and the page // renders its error state instead of a tab bar. addReconnectListener() { return () => {}; } + addReconnectFailureListener() { return () => {}; } // What the node should hold of our identity, settled after connecting. async settleNodeBundle() {} close() {} diff --git a/packages/meshbay-hub/tests/harness/music_grid_probe.py b/packages/meshbay-hub/tests/harness/music_grid_probe.py index 4f5b0ce7..1995b2a1 100644 --- a/packages/meshbay-hub/tests/harness/music_grid_probe.py +++ b/packages/meshbay-hub/tests/harness/music_grid_probe.py @@ -81,6 +81,7 @@ window.MeshBayTransport = function () { // Real, not left to the Proxy below: that would hand back a promise // where an unsubscribe belongs, and the page calls it on unmount. addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/music_queue_probe.py b/packages/meshbay-hub/tests/harness/music_queue_probe.py index 9be440b3..83c2972e 100755 --- a/packages/meshbay-hub/tests/harness/music_queue_probe.py +++ b/packages/meshbay-hub/tests/harness/music_queue_probe.py @@ -80,6 +80,7 @@ window.MeshBayTransport = function () { // Real, not left to the Proxy below: that would hand back a promise // where an unsubscribe belongs, and the page calls it on unmount. addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/music_untagged_probe.py b/packages/meshbay-hub/tests/harness/music_untagged_probe.py index f4c484c3..7af1efc7 100644 --- a/packages/meshbay-hub/tests/harness/music_untagged_probe.py +++ b/packages/meshbay-hub/tests/harness/music_untagged_probe.py @@ -110,6 +110,7 @@ window.MeshBayTransport = function () { async fetchChatHistory() { return { messages: [], hasMore: false }; }, async fetchLinkPreview() { return { ok: false }; }, addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/network_change_harness.mjs b/packages/meshbay-hub/tests/harness/network_change_harness.mjs index e0201125..a8bd0809 100644 --- a/packages/meshbay-hub/tests/harness/network_change_harness.mjs +++ b/packages/meshbay-hub/tests/harness/network_change_harness.mjs @@ -12,7 +12,8 @@ // // Usage: node network_change_harness.mjs <transport sources> <json> // {"scenario": "change-dead" | "change-alive" | "backoff" | "ice-dead" | "ice-alive" -// | "events" | "closed"} +// | "events" | "closed" | "refused" +// | "refused-during"} import { readFileSync } from 'fs'; import { delimiter } from 'path'; @@ -98,12 +99,21 @@ window.MeshBayCrypto = { }; const SDP = 'v=0\r\na=fingerprint:sha-256 AA:BB\r\n'; +const candidate = (address, kind) => `a=candidate:1 1 udp 2130706431 ${address} 50000 typ ${kind}\r\n`; +// On the node's Wi-Fi without the permission (test_local_network_refused.py): +// the phone's offer and the node's answer as measured, and the node's address +// on the phone's subnet missing from what the browser paired. +const WIFI_OFFER = SDP + candidate('192.168.1.13', 'host') + candidate('90.112.206.172', 'srflx'); +const NODE_ANSWER = SDP + candidate('192.168.1.22', 'host') + candidate('90.112.206.172', 'srflx'); +const KEPT_WHEN_REFUSED = ['90.112.206.172']; let offline = false; +let lanRefused = false; globalThis.fetch = async (url) => { if (offline) throw new TypeError('Failed to fetch'); return { ok: true, status: 200, headers: { get: () => null }, - json: async () => (url.endsWith('/mnp-token') ? { mnp_token: 't' } : { sdp: SDP }), + json: async () => (url.endsWith('/mnp-token') ? { mnp_token: 't' } + : { sdp: lanRefused ? NODE_ANSWER : SDP }), }; }; @@ -167,10 +177,30 @@ class FakePC { pcs.push(this); } createDataChannel() { this.channel = new FakeChannel(); return this.channel; } - async createOffer() { return { type: 'offer', sdp: SDP }; } + async createOffer() { return { type: 'offer', sdp: lanRefused ? WIFI_OFFER : SDP }; } async setLocalDescription(d) { this.localDescription = d; } - async getStats() { return new Map(); } + async getStats() { + const reports = new Map(); + if (this.refused) { + KEPT_WHEN_REFUSED.forEach((address, i) => { + reports.set(`R${i}`, { id: `R${i}`, type: 'remote-candidate', address }); + }); + } + return reports; + } async setRemoteDescription() { + if (lanRefused) { + // ICE gives up, and the channel goes with it, as Chrome reports it: + // there after thirty seconds, here after fifteen. + this.refused = true; + setTimeout(() => { + if (this.connectionState === 'closed') return; + this.iceConnectionState = 'failed'; + this.connectionState = 'failed'; + this.channel.close(); + }, 15000); + return; + } setTimeout(() => { if (this.connectionState === 'closed') return; this.channel.session = new NodeSession(this.channel); @@ -218,6 +248,8 @@ if (scenario === 'events') { const main = new Transport('https://hub.invalid', 'token'); let reconnectedAt = null; main.addReconnectListener(() => { if (reconnectedAt === null) reconnectedAt = now; }); + const failures = []; + main.addReconnectFailureListener((err) => { failures.push(err.reason || null); }); let connected = false; const identity = { pkEdB64: 'ed', signAs: async () => 'sig', native: false }; main.connect('node', 'token', 'group', new Uint8Array(32), identity, null, 'u', 'uid', @@ -252,6 +284,36 @@ if (scenario === 'events') { connection.dispatch('change'); out.backAt = now; await run(80000); + } else if (scenario === 'refused-during') { + // The same, with the permission granted while an attempt set up before it + // is still waiting on ICE: that attempt is given up, the next goes at once. + lanRefused = true; + first.channel.session.cut = true; + connection.type = 'wifi'; + connection.dispatch('change'); + await run(40000); + const attempts = pcs.length; + while (pcs.length === attempts) await run(now + 100); + await run(now + 2000); + lanRefused = false; + out.backAt = now; + main.retryNow(); + await run(now + 40000); + } else if (scenario === 'refused') { + // Connected on mobile data, then on the node's Wi-Fi without the local + // network permission: every attempt is refused, and says so. The + // permission is granted after a while, and the page asks to try now. + lanRefused = true; + first.channel.session.cut = true; + connection.type = 'wifi'; + connection.dispatch('change'); + await run(60000); + out.failures = [...failures]; + out.stillReconnecting = main.reconnecting; + lanRefused = false; + out.backAt = now; + main.retryNow(); + await run(80000); } else if (scenario === 'closed') { main.close(); connection.type = 'wifi'; diff --git a/packages/meshbay-hub/tests/harness/playlist_ui_probe.py b/packages/meshbay-hub/tests/harness/playlist_ui_probe.py index 6e3be1e2..0d6e4134 100644 --- a/packages/meshbay-hub/tests/harness/playlist_ui_probe.py +++ b/packages/meshbay-hub/tests/harness/playlist_ui_probe.py @@ -76,6 +76,7 @@ window.MeshBayTransport = function () { // Real, not left to the Proxy below: that would hand back a promise // where an unsubscribe belongs, and the page calls it on unmount. addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/sticky_header_probe.py b/packages/meshbay-hub/tests/harness/sticky_header_probe.py index 3e75494b..89fb2827 100755 --- a/packages/meshbay-hub/tests/harness/sticky_header_probe.py +++ b/packages/meshbay-hub/tests/harness/sticky_header_probe.py @@ -240,6 +240,7 @@ window.MeshBayTransport = function () { // Real, not left to the Proxy below: that would hand back a promise // where an unsubscribe belongs, and the page calls it on unmount. addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; return new Proxy(self, { diff --git a/packages/meshbay-hub/tests/harness/video_series_probe.py b/packages/meshbay-hub/tests/harness/video_series_probe.py index f161c305..2b9ade30 100644 --- a/packages/meshbay-hub/tests/harness/video_series_probe.py +++ b/packages/meshbay-hub/tests/harness/video_series_probe.py @@ -75,6 +75,7 @@ window.MeshBayTransport = function () { // what TMDB would have said. async fetchMediaMeta() { return { confidence: 0 }; }, addReconnectListener() { return () => {}; }, + addReconnectFailureListener() { return () => {}; }, close() {}, }; // Everything else the player asks for never answers: it sits on its diff --git a/packages/meshbay-hub/tests/test_network_change.py b/packages/meshbay-hub/tests/test_network_change.py index 2298e8e6..0d270ba7 100644 --- a/packages/meshbay-hub/tests/test_network_change.py +++ b/packages/meshbay-hub/tests/test_network_change.py @@ -88,3 +88,36 @@ def test_a_group_left_on_an_error_tries_again_when_the_network_changes(): block = re.search(r"onNetworkChange\(.*?\)", source, re.S) assert block, "the group page does not listen for another network" assert "setRetryKey" in source[block.start():block.start() + 200] + + +def test_a_reconnect_refused_the_local_network_says_so_and_goes_on(): + """Connected on mobile data, then on the node's Wi-Fi without the permission: + each attempt used to fail in silence, for ever. Each is now told to the page + with its reason, the reconnect goes on, and once the permission is granted + the page's "try now" reaches the group without waiting out the backoff.""" + out = _run("refused") + assert out["failures"] and set(out["failures"]) == {"local_network_blocked"}, out + assert out["stillReconnecting"], out + assert out["reconnectedAt"] - out["backAt"] < 3000, out + + +def test_try_now_gives_up_an_attempt_set_up_before_the_permission(): + """Measured on the Pixel: granted while an attempt was waiting on ICE, the + group came back 35 s later, once that attempt and the next backoff ran out.""" + out = _run("refused-during") + assert out["reconnectedAt"] - out["backAt"] < 2000, out + + +def test_the_group_page_explains_a_refused_reconnect_and_clears_it(): + source = (STATIC / "group-page.js").read_text(encoding="utf-8") + start = source.index("addReconnectFailureListener(") + refused = source[start:start + 400] + assert "local_network_blocked" in refused and "explainLanRefusal()" in refused + start = source.index("addReconnectListener(", start) + back = source[start:start + 300] + assert "lanLostRef.current" in back and "setError('')" in back + # The buttons ask the reconnect under way rather than tear it down. + assert "transport.retryNow()" in source + for button in ("group.retry", "group.lan_allow", "group.lan_open_settings"): + end = source.index(f"t('{button}')") + assert "retry()" in source[end - 700:end] or "onClick=${retry}" in source[end - 100:end], button |