diff options
| author | Christophe Besson <cbesson@gmail.com> | 2026-09-02 15:23:30 +0200 |
|---|---|---|
| committer | Christophe Besson <cbesson@gmail.com> | 2026-09-02 15:23:30 +0200 |
| commit | 313b72f15e8788ba3abcd3e44b5f7785fbc779fe (patch) | |
| tree | 1c8e9b024d78384da950730b37b8d41c2789e010 /packages/meshbay-hub/src/meshbay_hub/static/source-merge.js | |
| parent | 15d3eec914cf5e474e66f615c9fbaf602eebe575 (diff) | |
| download | meshbay-313b72f15e8788ba3abcd3e44b5f7785fbc779fe.tar.gz | |
fix(hub): one entry per file in the Search view's Videos grid
A library shared by two groups arrived in the cross-group Search view as
two entries per file: every film was two poster cards, every episode was
listed twice in the season list under the synopsis. Inside one group
this cannot happen — GroupIndex is keyed by blake3 — so the duplication
was the Search page's own, from concatenating N independently keyed
indexes.
source-merge.js folds entries on the content hash and resolves one
source per *unit* (a film, a whole show), so a season does not scatter
across two nodes. A group hosted by the reader's own node wins; failing
that the pick is a hash of the unit key and the reader's id, stable
across renders and reloads — a source that changed mid-stream would tear
down the connection under a film that is playing — and spread across
readers and units.
The units come from video-app.js's own groupVideoEntries rather than a
second copy of its keys here. Only the Videos view is wired up so far;
Music, Photos, failover and the "N sources" badge are phases 5-8 of
docs/refactoring-search.md.
Every test was checked against the fix removed. That is how the first
version of "a unit's files share its source" turned out to prove
nothing: with every episode in every group, per-file and per-unit
picking give the same answer, so it passed against a per-file
implementation. It now uses a unit whose files have unequal sources.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AbwJDbNTkiRUh7HTWEoyss
Diffstat (limited to 'packages/meshbay-hub/src/meshbay_hub/static/source-merge.js')
| -rw-r--r-- | packages/meshbay-hub/src/meshbay_hub/static/source-merge.js | 160 |
1 files changed, 160 insertions, 0 deletions
diff --git a/packages/meshbay-hub/src/meshbay_hub/static/source-merge.js b/packages/meshbay-hub/src/meshbay_hub/static/source-merge.js new file mode 100644 index 0000000..ccc4cfd --- /dev/null +++ b/packages/meshbay-hub/src/meshbay_hub/static/source-merge.js @@ -0,0 +1,160 @@ +/** + * One file, one entry — merging the same content announced by several groups. + * + * Only the cross-group Search view needs this. Inside a single group the case + * cannot arise: `GroupIndex` is keyed by blake3, so the same bytes at two + * paths are already one entry. The Search page is what creates duplicates, by + * concatenating N independently keyed indexes into one list — a directory + * shared by two groups then shows every film twice, every episode twice, every + * track twice. See docs/refactoring-search.md. + * + * Two rules decide everything here: + * + * - **Identity is the content hash.** `IndexEntry.id` is blake3 of the file, + * so two entries sharing an id are the same file whatever group announced + * it and whatever path it sits at. + * - **A source is chosen per unit, not per file.** A unit is a film, a whole + * show, a whole album — whatever the reader thinks of as one thing. Picking + * per file would scatter a season's episodes across two nodes, opening two + * connections and two metadata lookups for one show. + * + * No imports, and there must be none: the whole module is executed standalone + * by `test_search_source_merge.py`, which is the only evidence the merge has. + */ + +// FNV-1a, 32 bits. Spreads the source choice deterministically across readers +// — nothing here is security-bearing, and a cryptographic hash would buy +// nothing a multiplication does not. +function stableHash(s) { + let h = 0x811c9dc5; + for (let i = 0; i < s.length; i++) { + h ^= s.charCodeAt(i); + h = Math.imul(h, 0x01000193); + } + return h >>> 0; +} + +function _byGroupId(a, b) { + return String(a.groupId).localeCompare(String(b.groupId)); +} + +function _source(e) { + return { groupId: e.groupId, groupName: e.groupName, groupOwner: e.groupOwner }; +} + +/** + * Which of `sources` serves this unit. + * + * A group hosted by the reader's own node wins outright — for someone browsing + * their own libraries that is the whole of the reported case, and it makes the + * answer predictable where it matters most. Failing that the pick is a hash of + * the unit key and a per-reader salt: stable for one reader across renders and + * reloads (a source that changed mid-stream would tear down the connection + * under a film that is playing), different from one reader to the next, which + * is what spreading the load across sources was asking for. + * + * `isDown` marks a source whose connection has failed. Every source being down + * still returns one: a tile that fails to load is better than a film that + * vanished from the grid. + */ +function pickSource(sources, unitKey, opts) { + if (!sources || !sources.length) return null; + const { salt = '', isLocal, isDown } = opts || {}; + const sorted = [...sources].sort(_byGroupId); + const up = isDown ? sorted.filter((s) => !isDown(s.groupId)) : sorted; + const live = up.length ? up : sorted; + const local = isLocal ? live.filter((s) => isLocal(s.groupId)) : []; + const pool = local.length ? local : live; + return pool[stableHash(`${unitKey}\u0000${salt}`) % pool.length]; +} + +/** + * Merge the entries of each unit by content hash and resolve one source each. + * + * `units` is `[{ key, entries }]`, and the caller builds it by running the + * application's *own* grouping function over the un-merged list — never a + * second copy of that logic, which would keep agreeing right up until one of + * them changed. Two lists carrying the same key are one unit, which is what + * lets a caller emit a naive `{key: 'movie:<id>', entries: [e]}` per copy of a + * film and still get one card. + * + * Every field of a merged entry comes from the chosen source's own entry, and + * nothing is back-filled from another source. A `thumb_hash` or a + * `display_title` only one node computed is only fetchable over *that* node's + * connection, so borrowing it would produce a poster request the chosen + * transport cannot answer — merging two records field by field is the obvious + * thing to write here and it is wrong. + * + * The result carries `_sources`: every group that has the file, sorted by + * group id. That is what the badge counts, and it is the only trace left that + * more than one group was involved. + */ +function mergeUnitEntries(units, opts) { + const byKey = new Map(); + for (const u of units || []) { + if (!byKey.has(u.key)) byKey.set(u.key, []); + byKey.get(u.key).push(...(u.entries || [])); + } + + const merged = []; + for (const [key, entries] of byKey) { + // id -> one entry per group announcing it, first seen wins within a group + const byId = new Map(); + for (const e of entries) { + if (!byId.has(e.id)) byId.set(e.id, []); + const copies = byId.get(e.id); + if (!copies.some((c) => c.groupId === e.groupId)) copies.push(e); + } + + const unitSources = []; + const seen = new Set(); + for (const copies of byId.values()) { + for (const e of copies) { + if (seen.has(e.groupId)) continue; + seen.add(e.groupId); + unitSources.push(_source(e)); + } + } + const unitSource = pickSource(unitSources, key, opts); + + for (const copies of byId.values()) { + const sources = copies.map(_source).sort(_byGroupId); + let chosen = unitSource + && copies.find((e) => e.groupId === unitSource.groupId); + if (!chosen) { + // A file the unit's source does not hold — an episode only one of the + // groups has. Resolved over its own sources by the same rule and the + // same unit key, so every such file in the unit lands on the same + // fallback instead of scattering one per file. + const alt = pickSource(sources, key, opts); + chosen = (alt && copies.find((e) => e.groupId === alt.groupId)) || copies[0]; + } + merged.push({ ...chosen, _sources: sources }); + } + } + return merged; +} + +/** + * What the group badge should say for one entry. + * + * Returns `{ count, name, groupId }` rather than a rendered string: this module + * is executed standalone under node by its test, so it holds no reference to + * `i18n.js`. A caller renders `name` (a link to `groupId`) when `count` is 1, + * and a plural of `count` otherwise — which group was picked is deliberately + * never shown. + * + * An entry with no `_sources` at all is the single-group Group page, where + * there is one source by construction and no badge is drawn. + */ +function sourceLabel(entry) { + const sources = (entry && entry._sources) || []; + if (sources.length > 1) return { count: sources.length, name: '', groupId: null }; + return { + count: 1, + name: (entry && entry.groupName) || '', + groupId: (entry && entry.groupId) || null, + }; +} + +export { pickSource, mergeUnitEntries, sourceLabel, stableHash }; |