aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorChristophe Besson <cbesson@gmail.com>2026-09-19 10:14:27 +0200
committerChristophe Besson <cbesson@gmail.com>2026-09-19 10:14:27 +0200
commite1bce3b8d5835c3c70208d39b5b2c66787625e15 (patch)
tree0541ee26df124093d964a92502e96cd17ade2548
parent20a824118c09af15d6c338db4c9480ffe5cbcdb6 (diff)
downloadmeshbay-e1bce3b8d5835c3c70208d39b5b2c66787625e15.tar.gz
fix(hub): stop keeping a copy of every group's file listing in the browser
`group_indexes` was an IndexedDB store holding a decrypted copy of each group's index — every file's name, path, size, hash and uploader — written on every index and on every delta, from three call sites. It was the cross-group search of Phase 10b: `doSearch` read `getAllCachedIndexes` and searched those records instead of dialling anything. On 2026-08-28 Search began dialling the nodes, and that commit removed the reader and left the writers. Since then the browser has gone on building a cleartext file listing that nothing consulted, that no sign-out removed — the key database is a different one — and that grew with every group ever opened. L7, at rest: kept code that nothing calls does not sit still. Drawing a group's files while its node is unreachable is the only thing such a cache buys, and it is not wanted: a listing that cannot be opened is worse than an honest absence. So there is nothing to read it with, and the writers go. The store stays in the schema and is emptied instead. Dropping it needs a version bump, a version bump is an upgrade another tab can block, and playlists share this database — so the tidier change is the one with a failure mode. `purgeGroupIndexCache` runs once per browser behind a flag, which clears what is already on people's machines; a browser that refuses storage simply runs it again, which is harmless because it is idempotent. Three guards, each checked by reintroducing the fault: only `openDB` and the purge may touch the store, the purge may only clear it, and the purge must actually be called at start-up — a purge nobody calls is the same defect wearing the opposite hat. `test_sticky_header.py[firefox]` reports twelve setup errors in a full run here. A Firefox instance is open on this machine, which is the trap CLAUDE.md describes; the same twelve appear with these changes stashed, and the `[chrome]` half of the same file, covering the same geometry, is clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
-rw-r--r--CLAUDE.md2
-rw-r--r--docs/MESHBAY_DESIGN.md8
-rw-r--r--packages/meshbay-hub/src/meshbay_hub/static/app.js18
-rw-r--r--packages/meshbay-hub/src/meshbay_hub/static/group-page.js10
-rw-r--r--packages/meshbay-hub/src/meshbay_hub/static/hub-client.js58
-rw-r--r--packages/meshbay-hub/src/meshbay_hub/static/search-page.js3
-rw-r--r--packages/meshbay-hub/tests/harness/search_fanout_harness.mjs5
-rw-r--r--packages/meshbay-hub/tests/test_no_index_cache.py98
-rw-r--r--packages/meshbay-hub/tests/test_search_unlisted.py10
9 files changed, 156 insertions, 56 deletions
diff --git a/CLAUDE.md b/CLAUDE.md
index 0199979..65ca668 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -831,7 +831,7 @@ here are kept only where they are a rule about *editing* the code.
| Shared settings widgets | `settings-ui.js`, `folder-tree.js` | **A pane must not import `group-settings.js`** — that is an import cycle, and it fails as a component that silently does not render |
| Transport, handshake, device hello, roster verify | `transport.js` | §5.2, §3.3 |
| Crypto | `crypto.js`, `keyderive.js` | §4 |
-| Hub session, token renewal, IndexedDB cache | `hub-client.js` | §3.1 |
+| Hub session, token renewal, IndexedDB (keys, playlists) | `hub-client.js` | §3.1. **No group index is stored**: the `group_indexes` store had no reader for weeks and is emptied on start-up (`purgeGroupIndexCache`) |
| Where the hub is | `platform.js` — `hubBase()` | **the only file allowed to decide this** (§8.3) |
| Downloads, decrypt pipeline | `file-utils.js`, `downloads.js`, `sw.js` | §8.5 |
| Video player | `video-player.js` — `pump()` is the only place credit is granted | §8.5 |
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md
index 03ae77e..3aeb660 100644
--- a/docs/MESHBAY_DESIGN.md
+++ b/docs/MESHBAY_DESIGN.md
@@ -2504,6 +2504,14 @@ Two rules for a new application here:
function. A copy keeps agreeing until one of them changes, and the symptom is a
show whose episodes stream from two different nodes.
+**A group's index is never kept in browser storage.** The browser holds keys and
+playlists; it does not hold a copy of what a group contains. Such a cache existed,
+for a cross-group search that read it instead of dialling, and it outlived that
+search by weeks — writing a cleartext file listing that nothing read and no
+sign-out removed. Its only remaining use would be to draw a group's files while
+its node is unreachable, and that is refused on its own merits: a listing that
+cannot be opened is worse than an honest absence.
+
**A group whose node is down is the normal case, and nothing waits for it.** A
node is a machine in somebody's house, so with a handful of groups one of them
is always off. Each index is drawn the moment it arrives rather than when its
diff --git a/packages/meshbay-hub/src/meshbay_hub/static/app.js b/packages/meshbay-hub/src/meshbay_hub/static/app.js
index 92dcb28..fbfb942 100644
--- a/packages/meshbay-hub/src/meshbay_hub/static/app.js
+++ b/packages/meshbay-hub/src/meshbay_hub/static/app.js
@@ -10,7 +10,7 @@ import * as downloads from './downloads.js';
import { Icon } from './icon.js';
import { formatSize } from './file-utils.js';
import {
- HUB, navigate, session, getCachedGroupIndex,
+ HUB, navigate, session, purgeGroupIndexCache,
_storeBundleKey, _loadBundleKey, _clearKeyDB,
loadAuth, saveAuth, setAuth, setAuthChangeListener, ensureFreshToken, hubFetch,
refreshAccessToken, logoutOnHub,
@@ -1252,7 +1252,23 @@ const trayLabels = () => ({
// Catalogues are fetched, so the first render waits for one: mounting earlier
// would paint the interface in English and then swap every string. initLocale()
// falls back to English rather than rejecting, so this cannot strand the page.
+// One sweep, once per browser, to remove what the cross-group search of 2026-08
+// left behind: a cleartext copy of every group's file listing that nothing has
+// read since, and that no sign-out removed. Guarded by a flag so it costs one
+// transaction ever rather than one per load; a browser that refuses storage
+// simply does it again, which is harmless.
+const PURGED_KEY = 'meshbay.indexcache.purged';
+const purgeOnce = () => {
+ try {
+ if (localStorage.getItem(PURGED_KEY)) return;
+ } catch { /* no storage: purge anyway, it is idempotent */ }
+ purgeGroupIndexCache().then(() => {
+ try { localStorage.setItem(PURGED_KEY, '1'); } catch { /* nothing to remember with */ }
+ });
+};
+
const mount = () => {
+ purgeOnce();
render(html`<${App} />`, document.getElementById('app'));
// Get the download worker registered and this page under its control now,
// rather than inside the first click on Download. On Firefox and Safari it is
diff --git a/packages/meshbay-hub/src/meshbay_hub/static/group-page.js b/packages/meshbay-hub/src/meshbay_hub/static/group-page.js
index 9a86f2a..26519da 100644
--- a/packages/meshbay-hub/src/meshbay_hub/static/group-page.js
+++ b/packages/meshbay-hub/src/meshbay_hub/static/group-page.js
@@ -6,7 +6,7 @@ import { Icon } from './icon.js';
import { transfers } from './transfers.js';
import { downloadEntry } from './file-utils.js';
import {
- HUB, session, cacheGroupIndex, hubFetch, ensureFreshToken,
+ HUB, session, hubFetch, ensureFreshToken,
_loadBundleKey, _loadRecoveryKey, _storeBundleKey,
} from './hub-client.js';
import { APPS, visibleApps } from './apps.js';
@@ -265,10 +265,6 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs,
setEntries(fresh);
if (indexMsg.dirs) setNodeDirs(indexMsg.dirs);
if (indexMsg.roots) setNodeRoots(indexMsg.roots);
- cacheGroupIndex(groupId, group ? group.name : groupId,
- group ? group.owner_username : null, fresh,
- { video: appDirs('video'), music: appDirs('music'),
- photo: appDirs('photo') });
}, [groupId, group, appDirs]);
// additions/deletions/updates (daemon.py _broadcast_index_change, once
@@ -296,10 +292,6 @@ function GroupPage({ groupId, group, token, username, userId, userPrefs,
const keptIds = new Set(updated.map((e) => e.id));
const additions = (deltaMsg.additions || []).filter((e) => !keptIds.has(e.id));
const fresh = updated.concat(additions);
- cacheGroupIndex(groupId, group ? group.name : groupId,
- group ? group.owner_username : null, fresh,
- { video: appDirs('video'), music: appDirs('music'),
- photo: appDirs('photo') });
return fresh;
});
}, [groupId, group, appDirs]);
diff --git a/packages/meshbay-hub/src/meshbay_hub/static/hub-client.js b/packages/meshbay-hub/src/meshbay_hub/static/hub-client.js
index f467720..ba91f00 100644
--- a/packages/meshbay-hub/src/meshbay_hub/static/hub-client.js
+++ b/packages/meshbay-hub/src/meshbay_hub/static/hub-client.js
@@ -81,42 +81,26 @@ function openDB() {
});
}
-async function cacheGroupIndex(groupId, groupName, groupOwner, entries, roots) {
- try {
- const db = await openDB();
- const tx = db.transaction(IDB_STORE, 'readwrite');
- tx.objectStore(IDB_STORE).put({
- groupId, groupName, groupOwner, entries, roots: roots || {},
- cachedAt: Date.now(),
- });
- await new Promise((r, rej) => { tx.oncomplete = r; tx.onerror = rej; });
- db.close();
- } catch { /* best-effort */ }
-}
-
-async function getCachedGroupIndex(groupId) {
- try {
- const db = await openDB();
- const tx = db.transaction(IDB_STORE, 'readonly');
- const req = tx.objectStore(IDB_STORE).get(groupId);
- const result = await new Promise((r, rej) => { req.onsuccess = () => r(req.result); req.onerror = rej; });
- db.close();
- return result || null;
- } catch { return null; }
-}
-
-async function getAllCachedIndexes() {
- try {
- const db = await openDB();
- const tx = db.transaction(IDB_STORE, 'readonly');
- const req = tx.objectStore(IDB_STORE).getAll();
- const result = await new Promise((r, rej) => { req.onsuccess = () => r(req.result); req.onerror = rej; });
- db.close();
- return result || [];
- } catch { return []; }
-}
-
-async function clearAllCachedIndexes() {
+// `group_indexes` held a decrypted copy of every group's index — each file's
+// name, path, size, hash and uploader — written on every index and every delta,
+// and read by the cross-group search of the time, which searched those records
+// instead of dialling anything.
+//
+// Search has dialled the nodes since 2026-08-28. The reader went with that
+// change and the writers stayed, so for weeks the browser kept building a
+// cleartext file listing that nothing consulted and no sign-out removed: the key
+// database is a different one. It is **L7** — code nothing calls does not sit
+// still, it accumulates.
+//
+// Showing a group's files while its node is unreachable was the only use left
+// for such a cache, and it is not wanted: a listing you cannot open is worse
+// than an honest absence.
+//
+// The store itself is left in the schema. Dropping it means a version bump, and
+// a version bump means an upgrade another tab can block — which would take
+// playlists down with it, since they share this database. Emptying it costs
+// nothing and leaves nothing behind.
+async function purgeGroupIndexCache() {
try {
const db = await openDB();
const tx = db.transaction(IDB_STORE, 'readwrite');
@@ -378,7 +362,7 @@ async function hubFetch(path, { method = 'GET', body, token, _retried } = {}) {
export {
HUB, navigate, session,
openDB, IDB_PLAYLISTS,
- cacheGroupIndex, getCachedGroupIndex, getAllCachedIndexes, clearAllCachedIndexes,
+ purgeGroupIndexCache,
_storeBundleKey, _loadBundleKey, _storeRecoveryKey, _loadRecoveryKey, _clearKeyDB,
loadAuth, saveAuth, setAuth, setAuthChangeListener,
tokenLifeLeft, refreshAccessToken, ensureFreshToken, logoutOnHub, hubFetch,
diff --git a/packages/meshbay-hub/src/meshbay_hub/static/search-page.js b/packages/meshbay-hub/src/meshbay_hub/static/search-page.js
index a096414..0a96237 100644
--- a/packages/meshbay-hub/src/meshbay_hub/static/search-page.js
+++ b/packages/meshbay-hub/src/meshbay_hub/static/search-page.js
@@ -5,7 +5,7 @@ import { t } from './i18n.js';
import { Icon } from './icon.js';
import { canPreview, downloadEntry } from './file-utils.js';
import {
- HUB, session, cacheGroupIndex, hubFetch, ensureFreshToken, _loadBundleKey,
+ HUB, session, hubFetch, ensureFreshToken, _loadBundleKey,
} from './hub-client.js';
import { FilesPanel, FilePreview } from './files-app.js';
import { VideoApp, groupVideoEntries } from './video-app.js';
@@ -295,7 +295,6 @@ async function fetchAllIndexes(groups, token, username, userId, onProgress, onRe
groupName: g.name,
groupOwner: g.owner_username,
});
- cacheGroupIndex(g.id, g.name, g.owner_username, result.entries, result.roots);
// Drawn now, not when this group's neighbours are done. Its index is
// already in hand; holding it back until a group that is not answering
// has finished not answering is ten seconds of blank page for work that
diff --git a/packages/meshbay-hub/tests/harness/search_fanout_harness.mjs b/packages/meshbay-hub/tests/harness/search_fanout_harness.mjs
index 74b4bb3..c05e23b 100644
--- a/packages/meshbay-hub/tests/harness/search_fanout_harness.mjs
+++ b/packages/meshbay-hub/tests/harness/search_fanout_harness.mjs
@@ -78,7 +78,6 @@ const shown = []; // [{ at, groups }] — one entry per render the reader gets
const session = { bundleKey: 'k' };
const _loadBundleKey = async () => 'k';
-const cacheGroupIndex = () => {};
// One group's index, answered on the clock rather than over a network.
const fetchGroupIndex = (groupId) => new Promise((resolve, reject) => {
@@ -106,7 +105,7 @@ const lift = (signature) => {
};
const make = new Function(
- 'session', '_loadBundleKey', 'cacheGroupIndex', 'fetchGroupIndex', 'localStorage',
+ 'session', '_loadBundleKey', 'fetchGroupIndex', 'localStorage',
`const MAX_IN_FLIGHT = ${ceiling[1]};
const DOWN_KEY = 'harness';
${lift('function lastKnownDown(')}
@@ -116,7 +115,7 @@ const make = new Function(
return fetchAllIndexes;`,
);
const fetchAllIndexes = make(
- session, _loadBundleKey, cacheGroupIndex, fetchGroupIndex, globalThis.localStorage);
+ session, _loadBundleKey, fetchGroupIndex, globalThis.localStorage);
// ── The scenario ─────────────────────────────────────────────────────────────
diff --git a/packages/meshbay-hub/tests/test_no_index_cache.py b/packages/meshbay-hub/tests/test_no_index_cache.py
new file mode 100644
index 0000000..c39bf00
--- /dev/null
+++ b/packages/meshbay-hub/tests/test_no_index_cache.py
@@ -0,0 +1,98 @@
+"""
+A group's index is never written to browser storage.
+
+`group_indexes` was an IndexedDB store holding a decrypted copy of every group's
+index — each file's name, path, size, hash and uploader — written on every index
+and every delta. The cross-group search of the time read it instead of dialling
+anything, which is what it was for.
+
+Search has dialled the nodes since 2026-08-28. That change removed the reader and
+kept the writers, so the browser went on building a cleartext file listing that
+nothing consulted, that no sign-out removed (the key database is a different
+one), and that grew with every group ever opened. **L7**, at rest.
+
+Showing a group's files while its node is unreachable is the only thing such a
+cache buys, and it is not wanted: a listing you cannot open is worse than an
+honest absence. So there is nothing left to read it with, and these tests keep it
+that way — a writer reintroduced without a reader would be invisible again, and
+the second time it would be invisible for the same reason as the first.
+"""
+
+import re
+from pathlib import Path
+
+STATIC = Path(__file__).resolve().parents[1] / "src" / "meshbay_hub" / "static"
+HUB_CLIENT = STATIC / "hub-client.js"
+APP = STATIC / "app.js"
+
+# The store name, read from the source rather than written down here: renaming it
+# must not quietly take these tests out of the picture.
+STORE = re.search(r"const IDB_STORE = '([^']+)';",
+ HUB_CLIENT.read_text(encoding="utf-8")).group(1)
+
+
+def _functions(src: str) -> dict[str, str]:
+ """Every top-level function in a module, by name."""
+ out = {}
+ starts = [(m.start(), m.group(1)) for m in
+ re.finditer(r"^(?:async )?function (\w+)\(", src, re.M)]
+ for i, (at, name) in enumerate(starts):
+ end = starts[i + 1][0] if i + 1 < len(starts) else len(src)
+ out[name] = src[at:end]
+ return out
+
+
+def test_only_the_purge_touches_the_old_store():
+ """
+ Creating it and emptying it, and nothing else.
+
+ `openDB` still creates the store because dropping it needs a version bump,
+ and a version bump is an upgrade another tab can block — which would take
+ playlists down with it, since they share this database. An empty store costs
+ nothing; the point is that nothing writes to it.
+ """
+ fns = _functions(HUB_CLIENT.read_text(encoding="utf-8"))
+ touching = sorted(n for n, body in fns.items() if "IDB_STORE" in body)
+ assert touching == ["openDB", "purgeGroupIndexCache"], (
+ f"{touching} touch the {STORE!r} store; only creating and emptying it "
+ "are allowed, and a write to it is a file listing kept on disk that "
+ "nothing will ever read")
+
+
+def test_nothing_writes_a_group_index_to_the_store():
+ """Stated on the operation rather than on the callers, so a new one is caught."""
+ fns = _functions(HUB_CLIENT.read_text(encoding="utf-8"))
+ purge = fns["purgeGroupIndexCache"]
+ assert ".clear()" in purge
+ for write in (".put(", ".add(", ".putAll("):
+ assert write not in purge, f"the purge does a {write} — it must only clear"
+
+
+def test_no_module_carries_a_cache_writer_any_more():
+ """
+ The functions are gone, so the way this comes back is a new one. Any export
+ of hub-client.js whose name is about caching an index is refused here rather
+ than discovered months later with a store full of filenames.
+ """
+ src = HUB_CLIENT.read_text(encoding="utf-8")
+ for name in re.findall(r"^(?:async )?function (\w+)\(", src, re.M):
+ assert not re.search(r"cache.*index|index.*cache", name, re.I) \
+ or name == "purgeGroupIndexCache", (
+ f"{name} looks like an index cache again — the store it would write "
+ "to has no reader, and adding one was decided against")
+
+
+def test_the_purge_is_actually_called():
+ """
+ The defect being cleaned up was a function nobody called. A purge nobody
+ calls is the same defect wearing the opposite hat: the data stays on every
+ machine that already has it, and nothing says so.
+ """
+ app = APP.read_text(encoding="utf-8")
+ assert "purgeGroupIndexCache" in app, "app.js no longer imports the purge"
+ call = re.search(r"purgeGroupIndexCache\(\)", app)
+ assert call, "the purge is imported and never called"
+ mount = app.index("const mount = () => {")
+ assert "purgeOnce()" in app[mount:mount + 400], (
+ "the purge is no longer run at start-up, so a browser that still holds "
+ "the old store keeps it")
diff --git a/packages/meshbay-hub/tests/test_search_unlisted.py b/packages/meshbay-hub/tests/test_search_unlisted.py
index 67eb3e6..94f3aee 100644
--- a/packages/meshbay-hub/tests/test_search_unlisted.py
+++ b/packages/meshbay-hub/tests/test_search_unlisted.py
@@ -48,10 +48,14 @@ def test_an_unlisted_group_is_neither_indexed_nor_cached_nor_unreachable():
body = _function(SEARCH_PAGE.read_text(encoding="utf-8"), "fetchAllIndexes")
branch = body[body.index("result.unlisted"):]
branch = branch[:branch.index("} else if (result)")]
- for forbidden in ("results.set", "cacheGroupIndex", "unreachable.push"):
+ # `cacheGroupIndex` used to be on this list. The store it wrote to is gone
+ # (hub-client.js `purgeGroupIndexCache`), so the way an unlisted group's
+ # index could now be kept is by being written anywhere at all — which is
+ # what test_no_group_index_is_written_to_storage guards, for every group.
+ for forbidden in ("results.set", "unreachable.push"):
assert forbidden not in branch, (
- f"an unlisted group reaches `{forbidden}` — it would be shown, "
- "cached, or reported as down")
+ f"an unlisted group reaches `{forbidden}` — it would be shown "
+ "or reported as down")
def test_every_search_view_is_built_from_the_indexed_groups_only():