diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/apps.md | 101 | ||||
| -rw-r--r-- | docs/meshbay-draft-v6.md | 47 | ||||
| -rw-r--r-- | docs/refactor-groups.md | 906 |
3 files changed, 1031 insertions, 23 deletions
diff --git a/docs/apps.md b/docs/apps.md index ef8cc1a..7819dc9 100644 --- a/docs/apps.md +++ b/docs/apps.md @@ -130,9 +130,10 @@ any app with similar per-group local state. ## 3. Enable/disable: the mechanism -Same shape as `member_upload` (`meshbay-draft-v6.md` §2.1b) — an +Same shape as a root's `writable` flag (`refactor-groups.md` §1.1) — an operator-signed setting, stored on the node, enforced by absence rather than -by the client's honesty. +by the client's honesty. It used to be described against `member_upload`, +which was the group-wide upload switch; that was removed in the same refactor. **Node side** (`meshbay_node/roster.py`): ```python @@ -145,11 +146,22 @@ async def set_enabled_apps(group_id, apps, set_by="") -> list[str]: ... from exactly one place: `webrtc_server.py`'s `_admin_exec_apps_enabled`, after `_verify_admin_sig` — nothing is applied before the signature checks out. +**An app's directories are the same shape one level down** (2026-09-06): +`ops.set_app_directories(state, group_id, app_key, paths)`, stored under +`<app_key>_directories`, reached by one MNP message (`app_directories`) and one +loopback route. Adding an app adds no function, no message type and no route — +which is what "plugin architecture" has to mean to be worth the phrase. + `_do_apps_enabled` in `webrtc_server.py` validates before it ever issues a challenge: - `apps` non-empty — the operator can never lock a group down to nothing. -- every entry in `WebRTCPeerSession.ALLOWED_APPS` (`{"chat", "files"}` today) - — **this is the line a new app's node-side registration touches.** +- every entry in `WebRTCPeerSession.ALLOWED_APPS` + (`{"chat", "files", "video", "music", "photo"}` today) — **this is the line + a new app's node-side registration touches.** +- `files` is added to the list if it is absent, at both writers + (`_do_apps_enabled` and `ops.set_enabled_apps`, both at the front so the two + agree). It is not a toggle: MNP permits root exploration regardless of what + this list says, so hiding the tab only ever misled. The whole set is signed in one message (`apps_enabled`, `OP_APPS_ENABLED` in `meshbay_common.adminop`) rather than one op per app — ticking several boxes @@ -158,11 +170,42 @@ sorted, comma-joined app list (`"chat,files"`), built the same way on both sides so the operator's browser and the node arrive at identical bytes to sign/verify. -`enabled_apps` rides in `handshake_ack` and `node_status`, next to -`member_upload`. Changing it broadcasts `apps_enabled_ack` to everyone already -connected — `transport.js`'s `onAppsEnabled` — so a disabled tab disappears -without waiting for a reconnection, the same as `member_upload`'s live -broadcast. +`enabled_apps` rides in `handshake_ack` and `node_status`, next to the roots +table. Changing it broadcasts `apps_enabled_ack` to everyone already connected +— `transport.js`'s `onAppsEnabled` — so a disabled tab disappears without +waiting for a reconnection. The root ops (`root_update_ack`, `root_eject_ack`, +`root_plug_ack`) broadcast the same way, through `onRootsChanged`. + +### 3b. An app's settings + +Each app that has settings exports a component from +`static/<app>-app-settings.js` and names it in its `apps.js` entry. The Settings +page renders one collapsible section per registry entry, with the app's own +on/off switch in the header — the toggle *is* the enablement control, rather +than a checkbox list somewhere else that could disagree with it. + +Every pane takes the same props, and nothing else: `roots`, `dirs`, `settings`, +`saveDirectories` (bound to this app), `transport`, `signFn`. The split is the +point — **what every app has, the page does generically; what one app alone +has, the pane does itself.** Pointing an app at folders goes through +`saveDirectories`; a TMDB credential or a link-preview switch is the pane's own +business, made with the transport it is handed. An app that only needs +directories therefore touches neither `group-settings.js` nor `group-page.js`, +and `test_app_settings_plugin.py` fails if either of them starts naming apps +again. + +Two constraints that are not obvious: + +- **A pane must not import `group-settings.js`.** That is a cycle + (`group-settings` → `apps` → pane → `group-settings`), and ES modules answer + it with a temporal-dead-zone `ReferenceError` at first render — the component + does not appear, with nothing in the console to say why. The shared widgets + (`CollapsibleSection`, `ToggleSwitch`, `useSaver`) live in `settings-ui.js` + for this reason. +- **A new module must be added to `_ASSETS`** in `meshbay_hub/api/webapp.py`. + A file reached through the registry is not imported by name anywhere, so + nothing else would notice it changing, and a browser would go on serving the + cached copy. `test_asset_versioning` enforces it. **Client side:** `apps.js`'s `visibleApps(enabledKeys)` filters the registry; `group-page.js` calls it with `enabledApps` state (from the ack, `null` until @@ -181,8 +224,17 @@ registry, so a newly-registered app gets a checkbox for free. `icon.js` — do not re-implement `formatSize`, the download pipeline, or `Icon`. 2. **Register it** in `apps.js`'s `APPS` array: `{ key, icon, labelKey, - Component }`. `key` is the wire identifier — it must match what you add to - the node's allow-list next. + Component, Settings? }`. `key` is the wire identifier — it must match what + you add to the node's allow-list next, and it is also the row an app's + directories are stored under (`<key>_directories`). One identifier per app, + everywhere; `test_app_settings_plugin.py` checks the registry against + `ALLOWED_APPS`. +2b. **`<name>-app-settings.js`**, if the app has anything to configure, + exporting a component that takes `{ roots, dirs, settings, + saveDirectories, transport, signFn }` and nothing else (§3b). Folders go + through `saveDirectories`; anything only this app has, it does itself with + the transport. **Do not import `group-settings.js`** — that is a cycle, and + it fails as a component that silently does not render. 3. **Node-side allow-list**: add the key to `ALLOWED_APPS` in `webrtc_server.py`. Without this the node refuses `apps_enabled` for any set naming it (`"Unknown app(s): ..."`), so an operator can never turn it @@ -190,7 +242,7 @@ registry, so a newly-registered app gets a checkbox for free. 4. **i18n**: at minimum, a `group.tab_<name>` key (the tab's tooltip/label, reused as the Settings checkbox label) in all ten `static/locales/*.js` files. `test_locales.py` holds them to the same key set. -5. **`webapp.py`'s `_ASSETS`** tuple: add the new file. This is the +5. **`webapp.py`'s `_ASSETS`** tuple: add both new files. This is the cache-busting hash's input list — a file imported by the page but missing here can change without the served URL changing, which is the exact bug class `test_asset_versioning.py` exists for. Forgetting this step used to be @@ -210,6 +262,11 @@ registry, so a newly-registered app gets a checkbox for free. No protocol change, no hub change, no `daemon.py` change — steps 3 and 6 are the only node-side touches, and both are allow-lists, not new wire messages. +Directories in particular need nothing server-side at all: `app_directories` is +one generic op keyed by the app's name (§3), and an app storing its folders +under a key nobody wrote code for is the case +`test_app_directories.py::test_an_app_nobody_wrote_code_for_stores_its_directories` +pins. ## 5. What does not exist yet @@ -234,9 +291,17 @@ the only node-side touches, and both are allow-lists, not new wire messages. machinery again; unlike Videos/Music it needs several root folders per group rather than one, has a single album-grid view with no third-party matching step, and reads EXIF locally on the node instead. -- **The offline/loopback settings path.** `member_upload` can be toggled two - ways: over a live MNP connection, or (Electron only) via the node's local - HTTP API when MNP isn't connected (`platform.node.call('PUT', .../member- - upload')`, `group-settings.js`). `apps_enabled` only has the MNP path today. - Adding the loopback twin is a `meshbay_node.ui` endpoint plus a - `group-settings.js` branch, mirroring the existing `member_upload` one. +- **The offline/loopback settings path.** A root's flags can be changed two + ways: over a live MNP connection (any browser, anywhere), or — Electron + only, and only when MNP is not connected — via the node's local HTTP API + (`platform.node.call('PATCH', '/api/groups/<id>/roots/<name>')`, + `SharedDirectoriesTable` in `group-settings.js`). `apps_enabled` only has + the MNP path today. Adding the loopback twin is a `meshbay_node.ui` endpoint + plus a branch in the table's `run()` helper, mirroring the root ops. + + **MNP is the path that must exist, not the fallback.** The operator of a + node is not necessarily sitting at it. The first version of the shared + directories table read its roots exclusively from the loopback API, which + resolves to "not available" in a browser — so the whole section rendered for + nobody on the web, while the controls it replaced had worked there. Any + operator-facing setting added here needs the MNP route first. diff --git a/docs/meshbay-draft-v6.md b/docs/meshbay-draft-v6.md index 180050c..28aea0c 100644 --- a/docs/meshbay-draft-v6.md +++ b/docs/meshbay-draft-v6.md @@ -57,7 +57,7 @@ | 6 | Portability | exFAT/NTFS and Windows are the **common** case. Case folding and Unicode normalization become correctness requirements, not compatibility notes | E8 / decision 12 | | 7 | Accounts | Native registration is **hybrid**: passphrase-derived `auth_key` (the recovery path) plus a device Ed25519 key for day-to-day authentication | E3 / decision 4 | | 8 | Authorship | Chat senders are **cryptographically authenticated to each other**; an upload has a **provable owner** who may delete it, as the operator may. v5's node-asserted attribution is replaced | operator decision, §2.4b | -| 9 | Node authority | The operator may **close uploading to everyone but themselves**, per group. Signed MNP op, stored on the node, enforced by the node — the hidden button is a courtesy, the refusal is the control | §2.1b | +| 9 | Node authority | The operator decides **which directories accept uploads**, per root. Signed MNP op, stored on the node, enforced by the node — the hidden button is a courtesy, the refusal is the control. **Superseded 2026-09-06** by `docs/refactor-groups.md` §1.1: the group-wide `member_upload` switch this section described is replaced by RO/RW per root, and the "everyone but the operator" carve-out is gone | §2.1b | | 10 | Client | A group's UI is a **set of pluggable applications** (Chat, Files today), not one monolithic page. Which are shown is a per-group, operator-signed setting on the same pattern as change 9 | §2.7 | | 11 | Hub role | The hub gains a **runtime instance-policy store** (`hub_settings`). First policy: an admin switches **public groups off** hub-wide, enforced server-side on every hub-mediated path. `suspend` vs `revoke` on a group are now written down as the distinct things they are | §2.8 | | 12 | Group registry | A group name is **unique per owner account**, not globally; the group's identity is still its UUID. Listed everywhere as `name@owner` | §2.9 | @@ -74,10 +74,22 @@ v5 confines uploads to `shared_root/uploads/` with a filename allowlist, no overwrite, chunk ordering and a size cap. All four protections stand. Two amendments: -- There is no single `shared_root`. **The operator designates one root as the upload - destination**; the quarantine lives inside it. If that root is unavailable the upload - fails with a stated reason and never falls back to another; if none is designated, - uploads are refused rather than guessed. +- There is no single `shared_root`. **Each root is read-only or read-write**, and an + upload goes to the folder the sender is looking at, inside a writable root. If that + root is unavailable the upload fails with a stated reason and never falls back to + another; if the group has no writable root, uploads are refused rather than guessed. + (Amended 2026-09-06 — the original text designated *one* root as the upload + destination, and the client named none. See `docs/refactor-groups.md` §1.1.) +- **There is no `uploads/` quarantine directory any more** (2026-09-06). It was the + last of v5's, the per-user layer having gone on 2026-08-14, and it went for the same + reason: a folder appearing beside the operator's library because somebody sent a + file is the node deciding how their disk is arranged. **What made the quarantine + worth having was never the subdirectory** — it is the filename allowlist, the size + cap, the chunk ordering and the no-overwrite rule, and all four are unchanged. + The client now names the destination folder, which is safe for one reason and only + one: it is resolved through `RootSet.resolve()`, which refuses `..`, absolute + segments and anything escaping its root, symlinks included. A member answers "which + of this group's folders", never "which path on the operator's disk". - **The no-overwrite rule is unchanged and still holds on exFAT/NTFS.** An earlier draft claimed a string comparison let `README.TXT` land on `readme.txt` there. It does not: the check is `Path.exists()`, and `stat()` is itself case-insensitive on those @@ -91,6 +103,31 @@ device that asked, so the node keeps no thumbnail store. ### 2.1b §5.2 Uploads — the operator may close them +> **Superseded 2026-09-06.** `member_upload` is gone; the mechanism is `writable` on +> each root. What the three load-bearing properties below say is *unchanged* — read +> "the root's `writable` flag" for "`member_upload`" and every word of them still +> holds, which is why they are kept rather than deleted. What did change: +> +> - **It is per root, not per group.** A group can publish one library read-only and +> accept uploads into another, which the single switch could not express. +> - **There is no carve-out for the operator.** Read-only means read-only for +> everyone, because a published library that quietly accepts writes from whoever +> holds admin authority is not one. The paragraph below justifying the setting by +> "the only way to get a curated library was to designate no upload root at all, +> which refuses the operator too" is therefore the reasoning that was reversed: that +> *is* the model now, and refusing the operator is the point rather than the defect. +> - **The client names the destination root.** With several writable roots the node +> cannot choose without guessing, and a guess sends a member's file to a disk the +> operator did not intend. It names a root, never a path; everything below the root +> is still decided by the node. +> - The signed op is `OP_ROOT_UPDATE` (plus `OP_ROOT_EJECT` / `OP_ROOT_PLUG`) rather +> than `OP_MEMBER_UPLOAD`, and the flags live in `node.toml` — they are +> configuration — while the *ejected* runtime state lives in `roster.db`. +> `member_upload` survives on the handshake ack alone, computed as "any root is +> writable", for MNP 1.0 clients that read no other field. +> +> See `docs/refactor-groups.md` §1.1 and §1.5b. + New. A group where every member may add files is the default and stays the default; some groups want a library the operator curates, and until now the only way to get one was to designate no upload root at all, which refuses the operator too. diff --git a/docs/refactor-groups.md b/docs/refactor-groups.md new file mode 100644 index 0000000..147afda --- /dev/null +++ b/docs/refactor-groups.md @@ -0,0 +1,906 @@ +# Groups Refactor — Per-Root Permissions & App Plugin Architecture + +> Status: **Complete** (2026-09-07). All three phases built, reviewed and +> tested against a running node. +> +> This is the most significant refactoring of the project. It changes how roots +> are permissioned, how group applications are configured, and how the Settings +> and Create Group pages are structured. +> +> §7b, §7c and §7d record what each phase's review found and where the plan +> below was wrong. Several entries are rules rather than one-off fixes; §7d +> also lists what a person still has to test by hand. + +--- + +## 0. Summary of changes + +| Area | Before | After | +|---|---|---| +| Root permissions | One root marked `upload=True`; binary `member_upload` toggle per group | Each root is **RO** (default) or **RW**; multiple RW roots allowed; fully RO group is valid | +| Upload policy | Separate section in Settings; `member_upload` signed op | **Removed.** RO/RW on the root is the mechanism. Files shows Upload only on RW roots. Chat disables attachments when its configured directory is not on a RW root | +| Root metadata | `name, path, kind, upload, direct` | `name, path, kind, writable, removable, direct` | +| Removable flag | Not tracked | Per-root boolean, set by operator. Enables the **eject/plug** button for safe device removal | +| Safe eject | Auto-detected only (`path.is_dir()`) | Operator-initiated eject button in Settings AND Files root view. `ejected` state distinct from `available`. Indexer freezes entries, no data loss | +| Files app | Can be disabled | **Always enabled**, transparently. Not shown in the app toggle list | +| App selection at group creation | Checkbox list of all apps | **Removed.** Files is enabled automatically; other apps are configured later in Settings | +| Settings layout | Monolithic: apps checkboxes, TMDB, MusicBrainz, directories, uploads — all in `group-settings.js` | **Structured:** Shared directories (top, expanded) → per-app sections (each with toggle + icon + title, collapsed, settings hidden until enabled) → Scan/Danger/Devices/Members | +| App settings code | All inlined in `group-settings.js` (1338 lines) | Each app has `ui/<APP_NAME>-app-settings.js`; loaded by discovery (presence of the file) | +| App enablement UI | One "Applications" section with checkboxes | Each app is a collapsible section with its own toggle in the title. The toggle is the enablement control | +| Folder picker | Flat `<select>` with depth-indented names | **Folder tree popup**: modal, root icons, `[+]` expand, sub-directory selection | +| Server-side app ops | Per-app functions in `ops.py` (`set_video_root`, `set_audio_root`, `set_photo_roots`, ...) | Generic `set_app_directory()` / `set_app_directories()` + app-specific wrappers where needed | +| CLI | `group add --dir --upload-dir`; `member upload` concept | `group add --dir [--writable]`; `root add/remove/set` with `--writable`/`--read-only`/`--removable` | +| MNP | 1.0 | 1.1 (additive: new fields on roots, new generic app-directory messages). 1.0 peers still work | + +--- + +## 1. Design decisions + +### 1.1 RO/RW replaces upload + member_upload + +The current model has two orthogonal mechanisms: (a) one root is the upload target, +(b) `member_upload` toggles whether non-operators can upload there. The new model +collapses both into one property per root: **writable**. + +- `writable = false` (default): the root is read-only for everyone, including the + operator via the UI. Content is placed there out-of-band (filesystem, rsync, USB). +- `writable = true`: any group member may upload to this root (into the quarantine + subdirectory, same protections as today — allowlist, no overwrite, size cap). + +Multiple roots can be writable. Zero can be writable (fully read-only group). The +operator controls which roots are RW by toggling a switch in the shared directories +table. + +**What this removes:** +- The `member_upload` toggle and its `OP_MEMBER_UPLOAD` signed op +- The `member_upload` / `member_upload_ack` MNP message types (deprecated, still + parsed for backward compat) +- The Uploads section in Settings +- The concept of "the upload root" (singular) + +**What this preserves:** +- The quarantine directory, filename allowlist, no-overwrite check, size cap +- The `_do_file_upload` handler in `webrtc_server.py` — now checks `writable` on + the target root instead of checking `upload` + `member_upload` +- The operator's ability to create a read-only group (set all roots RO) + +### 1.2 Files is always enabled + +`files` is removed from the toggleable app list. It is always present in +`enabled_apps` and cannot be disabled. The current ability to hide it was misleading: +MNP still permits root exploration regardless. The tab bar always shows Files. + +`apps.js` keeps `files` in `APPS` but marks it `alwaysEnabled: true`. The Settings +page skips it when rendering app toggle sections. + +### 1.3 Per-app settings files + +Each app that has configurable settings exports a settings component from +`ui/<APP_NAME>-app-settings.js`. The file is optional — an app with no settings +(like Files today) has no settings file and gets only a toggle. + +The `APPS` registry in `apps.js` gains an optional `Settings` field per entry, +imported from the corresponding settings file. The group-settings page iterates +`APPS`, skips `files`, and renders a collapsible section for each, with: + +- The app's monochrome icon + localized title in the section header +- A toggle switch in the header (disabled by default) +- The app's `Settings` component below, **hidden until the toggle is on** +- A Save button per app section (some saves trigger caching — TMDB, MusicBrainz) + +**Discovery mechanism:** In the browser context, "file discovery" is registration in +`apps.js`. Adding a new app means: write `<APP>-app.js` + `<APP>-app-settings.js`, +add one entry to `APPS` in `apps.js`, add the key to `ALLOWED_APPS` on the node. +Server-side enforcement via `ALLOWED_APPS` prevents client-side hacks from enabling +an unrecognized app. + +### 1.4 Folder tree widget + +A reusable modal popup (`FolderTreePicker`) that: + +- Appears centered on screen, semi-transparent backdrop +- Lists root directories at the top level, with the same folder icons as Files +- Each root can be expanded via `[+]` / collapsed via `[-]`, explorer-tree style +- Sub-directories load from the existing `nodeDirs` data (already available from the + file index, no new endpoint needed) +- Supports **single-select** mode (Chat) and **multi-select** mode (Videos, Music, + Photos) +- Shows the root's RO/RW badge next to each root name +- For apps that require RW (Chat): RO roots and their children are greyed out / + unselectable, with a tooltip explaining why +- OK / Cancel buttons at the bottom +- Returns the selected path(s) relative to the root (e.g., `Movies/Action`) + +The widget replaces the current flat `<select>` dropdowns in all app settings. It is +also usable in other UI contexts (the video player's folder navigation already does +something similar ad-hoc). + +### 1.5 Shared directories table + +A reusable component (`SharedDirectoriesTable`) used in both the group Settings page +and the Create Group wizard (developed once, shared). Features: + +- Borderless table, one row per root +- Columns: **Name** (with folder icon), **Path** (truncated with tooltip on hover), + **RW toggle** (switch, default off), **Removable toggle** (checkbox), + **Eject/Plug button** (visible only when removable is checked — see §1.5b), + **Delete button** (trash icon, with confirmation) +- Ejected roots show a distinct visual state: greyed-out row, eject icon replaced by + a plug icon +- The first root in the Create Group wizard defaults to RW +- A concise explanatory sentence at the top: "At least one directory is required. + Read-write directories accept uploads from group members." +- Add button: opens the native folder picker (Electron) or a path input (web, admin + only) +- Cannot delete the last root (refused with explanation) +- Each change is a signed operator op (`ROOT_ADD`, `ROOT_REMOVE`, or new + `ROOT_UPDATE` for toggling writable/removable on an existing root) + +### 1.5b Safe eject for removable devices + +**Problem.** An operator stores data on a USB drive. Unplugging it without warning +triggers the watchdog — file deletions propagate as though the operator erased an +entire library. The existing `refresh_availability()` auto-detects this and freezes +entries (good), but there is no way to eject cleanly before unplugging, and no way +to re-plug without a full rescan. + +**The `ejected` state.** A root has two independent runtime states: + +- `ejected` (bool, default `false`): operator-controlled, persisted in `roster.db`. + Set by clicking the eject button; cleared by clicking plug. +- `available` (bool, runtime): computed as `not ejected and is_live()`. This is + what clients and the indexer see. + +The distinction matters: when the operator clicks "eject" but hasn't physically +unplugged yet, `is_live()` returns `true` but `available` is `false` because +`ejected` is `true`. Without this, `refresh_availability()` would immediately +flip it back to available. + +**Eject flow:** + +1. Operator clicks the eject button (⏏) on a removable root +2. Confirmation dialog: "Eject *Movies*? Files from this directory will be + temporarily hidden to all members. You can safely unplug the device." +3. On confirm: `PUT /api/groups/{gid}/roots/{name}/eject` → `ops.eject_root()` +4. `ops.eject_root()`: sets `ejected = true` in `roster.db`, marks root + `available = false`, stops the watchdog observer for that root +5. The indexer **freezes** all entries from that root (existing behavior — no + deletions, no index updates, cached data preserved) +6. `index_sync` update propagates to connected peers: the root's `available` is + now `false` +7. All apps filter out entries from unavailable roots (Files already does this + partially — needs to be complete across Videos, Music, Photos) +8. The operator can now safely unplug the device + +**Plug flow:** + +1. Operator plugs the device back in and clicks the plug button (🔌) +2. `PUT /api/groups/{gid}/roots/{name}/plug` → `ops.plug_root()` +3. `ops.plug_root()`: checks `is_live()` first — if the path is not accessible, + refuses with "Directory not found. Is the device connected?" +4. On success: sets `ejected = false`, marks root `available = true`, restarts + the watchdog observer +5. The indexer **rescans the root** — its frozen entries are dropped and the + directory is read again. (The plan said "a reconciliation, not a full + rescan"; it is a rescan, deliberately. It is the same path a root coming + back from `refresh_availability` already took, and a device people carry + around can come back arbitrarily different — the hash cache means unchanged + files are not re-read, which is where the cost would have been.) +6. `index_sync` update propagates — entries reappear in all apps + +**The flag is persisted, and restored at startup.** `ejected` lives in +`roster.db` (`root_ejected:<folded name>`), not in `node.toml`: it is runtime +state, and an operator's hand-written config must not be rewritten because a USB +drive was unplugged. It has to survive a restart — a restart is exactly what an +operator does after noticing a drive fell off, and a flag that only lived in +memory would let the scan that follows read the empty mount point as an erased +library. `daemon._build_roots()` merges the two sources; it is the only place +that builds a `RootSet` for a group. + +**Auto-detection safety net.** If a `removable` root's path suddenly disappears +(operator unplugged without clicking eject): + +- `refresh_availability()` detects `is_live() = false` +- Because `removable = true`, it sets `ejected = true` automatically (as if the + operator had clicked eject), and reports it through the indexer's + `on_root_ejected` callback so the daemon writes it to `roster.db` — an + auto-eject that only existed in memory would be undone by the next restart +- Entries freeze, no deletions propagate +- The root stays in "ejected" state until the operator explicitly plugs it back + +For non-removable roots, the existing behavior is unchanged: `available` flips +based on `is_live()`, entries freeze when unavailable, rescan when available again. + +**Eject button in Files app.** In addition to the Settings table, an eject button +appears in the Files app root-level view, next to each removable root's name. This +provides quick access without navigating to Settings. Same confirmation dialog, +same API call. Ejected roots show as greyed-out with a plug icon to re-enable. + +**What is NOT deleted on eject:** + +- Index entries (frozen, not removed) +- TMDB / MusicBrainz cached metadata +- Video thumbnails in `media_cache` +- Chat message history referencing files on that root +- App directory configurations pointing to that root (but flagged as temporarily + invalid — the app shows a warning, not an error) + +**MNP message:** `ROOT_EJECT` / `ROOT_EJECT_ACK` and `ROOT_PLUG` / `ROOT_PLUG_ACK` +— signed operator ops, same pattern as `ROOT_UPDATE`. Broadcast to all connected +peers so they see the availability change immediately without waiting for the next +`index_sync`. + +### 1.6 Server-side normalization (ops.py) + +Two generic functions replace the per-app specific ones: + +```python +def set_app_directory(state, group_id, app_key, path, *, require_writable=False): + """Set a single directory for an app. Validates path is within a named root. + If require_writable, refuses paths under RO roots.""" + +def set_app_directories(state, group_id, app_key, paths, *, require_writable=False): + """Set multiple directories for an app. Same validation.""" +``` + +Existing functions (`set_video_root`, `set_audio_root`, `set_photo_roots`) become +thin wrappers calling the generic versions, preserving the current MNP message types +and roster keys for backward compat. New apps use the generic functions directly. + +### 1.7 Chat settings additions + +- **Directory picker** (single, RW-only): selects the directory for chat file + attachments. If no RW root exists, the picker shows an explanation and the + attachment button is disabled in the chat UI. If the selected directory's root is + later set to RO, the setting is flagged as invalid and attachments are disabled + until corrected. +- **Link preview toggle** (new, server-side): the operator can disable link previews + for the group. Stored in `roster.db` as `chat_link_preview` (default: enabled). + The node's `linkpreview.py` checks this setting before unfurling. The toggle is a + `ToggleSwitch` in the Chat settings section. + +### 1.8 Videos/Music/Photos settings changes + +**Videos:** +- Directory picker changes from single to **multi-directory** (via folder tree widget) +- TMDB settings section moved here from the monolithic settings +- TMDB API key field: no longer says "optional" or mentions the default key. + Instead, a prompt to sign up on TMDB with a direct link to generate a key +- Per-app Save button triggers TMDB cache sweep + +**Music:** +- Directory picker changes from single to **multi-directory** +- MusicBrainz settings section moved here +- Per-app Save button triggers MusicBrainz cache sweep + +**Photos:** +- Multi-directory picker (already multi, just moves to the folder tree widget) +- No third-party service settings + +### 1.9 Handshake ack changes + +The `handshake_ack` payload gains per-root metadata: + +```python +# Current +"roots": [{"name": "Movies", "path": "/mnt/movies", ...}] +"member_upload": True + +# New +"roots": [{"name": "Movies", "path": "/mnt/movies", "writable": False, "removable": True, ...}] +# member_upload removed (deprecated, still parsed by old clients) +``` + +For backward compatibility with MNP 1.0 peers: +- A 1.0 client that does not see `writable` on roots falls back to the old model + (root with `upload=True` is writable, `member_upload` from the ack controls access) +- A 1.1 node continues to send `member_upload` as a computed value: `True` if any + root is writable, `False` otherwise — so old clients behave sensibly +- `member_upload` is no longer writable via MNP ops; the node computes it from roots + +### 1.10 CLI changes + +As built. The group is a `--group` option rather than a positional, matching +every other verb in this CLI, and the negative flags are spelled `--no-writable` +/ `--no-removable` rather than `--read-only`, so each pair reads as one setting. + +``` +# Group creation (first root defaults to RW) +meshbay-node group add <name> --dir <path> # first root, RW +meshbay-node group add <name> --dir <path> --no-writable # first root, RO + +# Root management (--group is optional with one group configured) +meshbay-node root list [--group <name>] +meshbay-node root add <path> [--name <name>] [--writable] [--removable] +meshbay-node root remove <name> [--yes] +meshbay-node root set <name> --writable | --no-writable +meshbay-node root set <name> --removable | --no-removable +meshbay-node root eject <name> # safe eject +meshbay-node root plug <name> # re-plug + +# Deprecated (accepted with a warning) +--upload-dir → "use --writable on the target root instead" +member upload → removed; use 'root set --no-writable' / '--writable' +``` + +### 1.11 HelloWorld proof-of-concept + +A minimal app that validates the plugin architecture end to end: + +- `static/helloworld-app.js`: renders a greeting and lists files in its configured + directory +- `static/helloworld-app-settings.js`: single-directory picker (via folder tree + widget), no other settings +- Entry in `apps.js` with `key: "helloworld"`, icon, label, Component, Settings +- `ALLOWED_APPS` extended on the node +- No dedicated `helloworld.py` — uses the generic `set_app_directory()` function, + which is the whole point + +The HelloWorld app is **not shipped in production**. It lives in the tree as a +reference implementation and can be excluded from the build. Its value is proving +that the plugin mechanism works: zero changes to `group-settings.js`, +`group-page.js`, `webrtc_server.py` or `ops.py` to add it. + +### 1.12 Migration (existing nodes) + +A script in `QE/migration/` (not versioned) handles Fedora and Ubuntu nodes: + +**node.toml:** +- `upload = true` → `writable = true` +- `upload = false` (or absent) → `writable = false` +- Add `removable = false` to all roots that lack it +- Remove `upload_dir` from `[[groups]]` blocks (if present) + +**roster.db:** +- If `member_upload = "off"` for a group: set all that group's roots to + `writable = false` (the intent was "no uploads") +- Rename `video_root` → `video_directories` (wrap single value in a list) +- Rename `audio_root` → `audio_directories` (same) +- `photo_roots` → `photo_directories` (rename only) +- Remove `member_upload` rows +- Add `chat_link_preview = "true"` default for groups with chat enabled +- Add `chat_directory` for groups that had an upload root (default: the upload + root's path) + +**Protocol version:** +- MNP version file bumped to 1.1 + +**The script is idempotent** — running it twice is safe. + +--- + +## 2. Phase 1 — Root RO/RW model + Shared Directories UI + +**Goal:** Change the data model from `upload` to `writable`/`removable`, build the +shared directories table, restructure the top of Settings and the Create Group +wizard. Remove the Uploads section. Files always enabled. + +**Testable after this phase:** Create a group with RO/RW roots, toggle RW in +Settings, add/remove roots in the new table, see the Upload button appear/disappear +in Files based on the current root's writable flag, eject a removable root and verify +files disappear from all apps without data loss, plug it back and verify files +reappear, CLI works with new syntax including `root eject/plug`. + +### 2.1 Backend changes + +| File | Change | +|---|---| +| `config.py` | `RootSpec`: add `writable: bool = False`, `removable: bool = False`. Remove `upload` field. `__post_init__` migration: `upload=True` → `writable=True`. Parse new fields from node.toml | +| `roots.py` | `Root` dataclass: add `writable`, `removable`, `ejected`. Remove `upload`. `available` becomes a computed property: `not self.ejected and self.is_live()`. `_settle_upload_root()` removed. `RootSet.build()`: validate at least one root exists (no RW minimum). `refresh_availability()`: when a `removable` root's path disappears, auto-set `ejected=True` (safety net). Collision checks unchanged | +| `ops.py` | `add_root()`: accept `writable`, `removable` params. `remove_root()`: refuse removing last root (unchanged). `attach_group()`: first root defaults to `writable=True`. Remove `set_member_upload()`. New: `update_root()` for toggling writable/removable on an existing root (signed op `OP_ROOT_UPDATE`). New: `eject_root()` — sets `ejected=True`, stops watchdog for that root. New: `plug_root()` — checks `is_live()`, sets `ejected=False`, triggers reconciliation | +| `roster.py` | No schema change (generic key/value). Remove `member_upload` handling from `_apply_group_settings()` | +| `ui/app.py` | `POST /api/groups/{gid}/roots`: accept `writable`, `removable`. New: `PATCH /api/groups/{gid}/roots/{name}` → `ops.update_root()`. New: `PUT /api/groups/{gid}/roots/{name}/eject` → `ops.eject_root()`. New: `PUT /api/groups/{gid}/roots/{name}/plug` → `ops.plug_root()`. Remove `PUT /api/groups/{gid}/member-upload` | +| `daemon.py` (CLI) | New `root` subcommand: `add`, `remove`, `set`, `list`. `group add --dir` defaults to `writable=True`. Deprecate `--upload-dir` with warning. Remove `member upload` command | +| `webrtc_server.py` | Handshake ack: add `writable`/`removable`/`ejected` per root. Compute `member_upload` for backward compat. `_do_file_upload`: check `root.writable` instead of `root.upload` + `member_upload_allowed`. Handle `ROOT_UPDATE`, `ROOT_EJECT`, `ROOT_PLUG` MNP messages. Broadcast root availability changes to all connected peers. `ALLOWED_APPS`: add `"files"` to always-enabled set | +| `protocol.py` | New message types: `ROOT_UPDATE` / `ROOT_UPDATE_ACK`, `ROOT_EJECT` / `ROOT_EJECT_ACK`, `ROOT_PLUG` / `ROOT_PLUG_ACK`. `ROOT_ADD` gains `writable`, `removable` fields | +| `indexer.py` | `eject_root()`: stop the watchdog observer for that root, do NOT touch entries. `plug_root()`: restart observer, trigger reconciliation pass. `refresh_availability()`: auto-eject removable roots whose path disappears (set `ejected=True` instead of just flipping `available`) | +| `handshake.py` | MNP version → 1.1 (minor, additive) | + +### 2.2 Frontend changes + +| File | Change | +|---|---| +| `group-settings.js` | New `SharedDirectoriesTable` component (reusable). Includes eject/plug button per removable root. Move to top of settings (after Invite/Pair). Remove the Uploads toggle section. Remove the old Directories section (root management part — app root pickers stay for now). Calls loopback API for add/remove/update/eject/plug root | +| `create-group-page.js` | Replace app checkboxes with nothing (Files auto-enabled). Replace directory section with `SharedDirectoriesTable` (same component). First root defaults `writable=true`. Step 2: remove `set enabled_apps` call (Files is automatic) | +| `files-app.js` | Upload button visibility: check `currentRoot.writable && currentRoot.available`. Upload target: the root currently being browsed. Hide upload affordances on RO roots. Eject/plug button next to each removable root name in root-level view. Ejected roots greyed out with plug icon. Entries from unavailable roots filtered out of all views | +| `chat-app.js` | Attachment button: disabled if no RW root exists or if the chat directory's root is RO or unavailable. Tooltip explaining why | +| `group-page.js` | `nodeRoots` state: include `writable`/`removable`/`ejected` from handshake ack. Handle `ROOT_EJECT_ACK`/`ROOT_PLUG_ACK` broadcasts to update root state live. Remove `memberUpload` state. Remove `onUploadPolicy` callback | +| `video-app.js` | Filter entries: exclude files from unavailable roots | +| `music-app.js` | Filter entries: exclude files from unavailable roots | +| `photos-app.js` | Filter entries: exclude files from unavailable roots | +| `apps.js` | Add `alwaysEnabled: true` to `files` entry | +| `transport.js` | Parse new root fields from handshake ack | + +### 2.3 Localization + +New keys in `locales/*.js`: +- `sharedDirectories`, `sharedDirectoriesHint` ("At least one directory is + required...") +- `readOnly`, `readWrite`, `removableDevice` +- `uploadNotAvailableRO` (tooltip: "This directory is read-only") +- `ejectRoot`, `ejectRootConfirm` ("Eject *{name}*? Files from this directory + will be temporarily hidden...") +- `plugRoot`, `plugRootFailed` ("Directory not found. Is the device connected?") +- `rootEjected` (status label shown on ejected roots) +- Deprecation: `uploadToggle*` keys can be removed + +--- + +## 3. Phase 2 — App Settings Plugin Architecture + Folder Tree Widget + +**Goal:** Split app settings into per-app files, build the folder tree widget, +restructure Settings with per-app collapsible sections. Normalize server-side +ops. + +**Testable after this phase:** Each app has its own settings section with toggle. +Folder tree popup works for directory selection. TMDB/MusicBrainz settings are in +their app sections. Chat has link preview toggle. Videos/Music use multi-directory. + +### 3.1 Folder tree widget + +| File | Change | +|---|---| +| `static/folder-tree.js` (new) | `FolderTreePicker` component. Props: `roots` (with writable/removable), `dirs` (flat list from index), `mode` ("single"/"multi"), `requireWritable` (bool), `selected` (current selection), `onSelect` callback. Renders a modal with tree-view of directories. Builds tree structure from flat `nodeDirs` paths | + +**Behavior:** +- Modal overlay with semi-transparent backdrop, centered panel +- Root level: each root with folder icon, name, RO/RW badge, removable badge +- `[+]` / `[-]` toggle to expand/collapse children +- Directories sorted alphabetically at each level +- Single mode: clicking a directory selects it (highlight), deselects previous +- Multi mode: clicking a directory toggles its selection (checkbox visual) +- `requireWritable=true`: RO roots and all their children are greyed out and + unclickable, with a brief explanation at the top of the modal +- Selected path shown at the bottom of the modal +- OK (disabled if nothing valid selected) and Cancel buttons +- Escape key closes + +### 3.2 Per-app settings files + +| File | Content | +|---|---| +| `static/chat-app-settings.js` (new) | `ChatSettings` component. Single-directory picker (folder tree, `requireWritable=true`). Link preview toggle (`ToggleSwitch`). Save button. Validates: selected directory must be on a RW root | +| `static/video-app-settings.js` (new) | `VideoSettings` component. Multi-directory picker (folder tree). TMDB section: toggle + API key + language (moved from `group-settings.js`). TMDB key prompt: "Sign up on TMDB to generate your API key" with link, no "optional" wording. Save triggers TMDB sweep | +| `static/music-app-settings.js` (new) | `MusicSettings` component. Multi-directory picker. MusicBrainz section: toggle (moved from `group-settings.js`). Save triggers MusicBrainz sweep | +| `static/photos-app-settings.js` (new) | `PhotoSettings` component. Multi-directory picker. No third-party settings. Save button | +| `static/apps.js` | Each `APPS` entry gains `Settings` field imported from the corresponding settings file. `files` has no `Settings` (no per-app config). `files` has `alwaysEnabled: true` (from Phase 1) | + +### 3.3 Settings page restructure + +| File | Change | +|---|---| +| `group-settings.js` | Remove all inlined app settings (TMDB section, MusicBrainz section, app root pickers). Remove the Applications checkbox section. New rendering loop: for each app in `APPS` where `!app.alwaysEnabled && app.Settings`, render a `CollapsibleSection` with: icon + title in header, toggle switch in header, `app.Settings` component inside (visible only when enabled). Section collapsed by default. App toggle triggers signed `OP_APPS_ENABLED` op | + +**Final settings layout:** + +``` +1. Invite form (if admin & invite-only) +2. Pair operator (if nodeAdmin & not paired) +── App configuration ── +3. Shared directories (expanded, SharedDirectoriesTable) +4. 💬 Chat [toggle] (collapsed) + └─ Directory picker, Link preview toggle, Save +5. 🎬 Videos [toggle] (collapsed) + └─ Directory picker (multi), TMDB settings, Save +6. 🎵 Music [toggle] (collapsed) + └─ Directory picker (multi), MusicBrainz settings, Save +7. 📷 Photos [toggle] (collapsed) + └─ Directory picker (multi), Save +8. [Future apps discovered from APPS registry] +── Node tuning ── +9. Scan tuning (reconcile, debounce) +── Danger zone ── +10. Leave / Delete group +── Identity ── +11. My devices +12. Members table +``` + +### 3.4 Server-side normalization + +| File | Change | +|---|---| +| `ops.py` | New generic: `set_app_directory(state, group_id, app_key, path, require_writable=False)` and `set_app_directories(state, group_id, app_key, paths, require_writable=False)`. Both validate path(s) within named roots and check writable if required. Existing `set_video_root` → wrapper calling `set_app_directories("video", ...)`. Same for audio, photo. New: `set_chat_directory(state, group_id, path)` → `set_app_directory("chat", path, require_writable=True)`. New: `set_chat_link_preview(state, group_id, enabled)` | +| `webrtc_server.py` | Handle new MNP messages: `CHAT_DIRECTORY` / `ACK`, `CHAT_LINK_PREVIEW` / `ACK`. Refactor `VIDEO_ROOT` handler to use generic. Handshake ack: add `chat_directory`, `chat_link_preview` | +| `protocol.py` | New message types: `CHAT_DIRECTORY`, `CHAT_LINK_PREVIEW` (and acks) | +| `ui/app.py` | New endpoints: `PUT /api/groups/{gid}/chat-directory`, `PUT /api/groups/{gid}/chat-link-preview`. Generic: `PUT /api/groups/{gid}/app-directories/{app_key}` | +| `linkpreview.py` | Check `chat_link_preview` setting before unfurling | + +### 3.5 Handshake ack additions + +```python +"chat_directory": "/Shared/uploads", +"chat_link_preview": True, +"video_directories": ["/Movies", "/Series"], # replaces video_root (single) +"audio_directories": ["/Music"], # replaces audio_root (single) +"photo_directories": ["/Photos", "/Camera"], # replaces photo_roots +``` + +Old field names (`video_root`, `audio_root`, `photo_roots`) still sent for backward +compat with MNP 1.0 clients. New clients read the `*_directories` form. + +--- + +## 4. Phase 3 — HelloWorld, CLI polish & Migration + +**Goal:** Prove the plugin architecture with a HelloWorld app, finalize CLI changes, +write and test the migration script. + +### 4.1 HelloWorld app + +| File | Content | +|---|---| +| `static/helloworld-app.js` (new) | Minimal component: renders "Hello, World!" heading + lists files from its configured directory. Uses `entries` from `commonProps`, filtered by the configured path | +| `static/helloworld-app-settings.js` (new) | `HelloWorldSettings` component: single-directory picker via `FolderTreePicker`, Save button. Uses `set_app_directory("helloworld", path)` on the generic endpoint | +| `apps.js` | New entry: `{ key: "helloworld", icon: "👋", labelKey: "helloWorld", Component: HelloWorldApp, Settings: HelloWorldSettings }` | +| `webrtc_server.py` | Add `"helloworld"` to `ALLOWED_APPS` | + +**Validation:** enabling HelloWorld in Settings, picking a directory, saving, and +seeing the file list in the HelloWorld tab — with **zero changes** to +`group-settings.js`, `group-page.js`, or `ops.py`. If this works, any future app +can be added the same way. + +### 4.2 CLI final polish + +- `root` subcommand fully tested: `add`, `remove`, `set`, `list` +- `group add --upload-dir` prints deprecation warning and maps to `--dir <path> + --writable` +- `member upload` command removed (prints migration guidance) +- Help text updated to reflect RO/RW model +- `test_cli_dispatch.py` updated for all new verbs + +### 4.3 Migration script + +`QE/migration/migrate_groups_v2.sh` (or `.py`) — not versioned, for Fedora/Ubuntu +nodes only. + +**node.toml transformations:** +``` +upload = true → writable = true +upload = false → writable = false +(no upload field) → writable = false +(add removable = false to every root that lacks it) +(remove upload_dir lines from [[groups]] blocks) +``` + +**roster.db transformations:** +```sql +-- Convert member_upload=off to all-RO roots (handled via ops on restart) +-- Rename app directory keys +UPDATE group_settings SET key = 'video_directories' WHERE key = 'video_root'; +UPDATE group_settings SET key = 'audio_directories' WHERE key = 'audio_root'; +UPDATE group_settings SET key = 'photo_directories' WHERE key = 'photo_roots'; +-- Wrap single values in JSON arrays for video/audio +-- Add chat defaults +INSERT INTO group_settings (group_id, key, value) + SELECT group_id, 'chat_link_preview', 'true' + FROM group_settings WHERE key = 'enabled_apps' AND value LIKE '%chat%'; +-- Remove member_upload rows +DELETE FROM group_settings WHERE key = 'member_upload'; +``` + +**Idempotency:** every transformation is guarded (`IF NOT EXISTS`, check before +rename, etc.). Safe to run twice. + +**Rollback:** the script backs up `node.toml` and `roster.db` before any change. + +### 4.4 Windows notes + +No migration script needed for Windows (manual setup). Functional non-regression +testing only: +- Drive letter roots (`D:\Movies`, `E:\Music`) work with RO/RW +- Removable flag on USB drives +- Folder tree widget handles backslash paths +- Create Group wizard with Windows paths + +--- + +## 5. Risk assessment + +| Risk | Mitigation | +|---|---| +| Upload regression | Phase 1 is self-contained: test uploads on RW roots, verify refused on RO roots, before touching app settings | +| Eject data loss | Eject freezes entries (existing `freeze-not-empty` path). Ejected roots are never rescanned. Auto-eject safety net for surprise unplugs on removable roots. Explicit reconciliation (not full rescan) on plug | +| Indexer race on eject | Watchdog observer is stopped synchronously before `ejected=true` is set. No window where the watchdog sees a missing path and processes deletions | +| MNP backward compat | Computed `member_upload` in ack for 1.0 clients. New fields additive. Old field names kept alongside new ones | +| Monolithic `group-settings.js` diff | Phase 2 extracts code into new files; the old code is deleted, not refactored. Clear before/after | +| Folder tree perf with large indexes | `nodeDirs` is already computed. Tree construction is O(n) on directory count, not file count. Lazy child rendering on expand | +| Windows path handling | Existing NFC normalization and path handling unchanged. New fields (`writable`, `removable`) are path-independent booleans. Drive letters work as root paths | +| Migration data loss | Script backs up before changes. Idempotent. Tested on a staging node before production | + +--- + +## 6. Files touched (by phase) + +### Phase 1 +``` +packages/meshbay-node/src/meshbay_node/config.py +packages/meshbay-node/src/meshbay_node/roots.py +packages/meshbay-node/src/meshbay_node/ops.py +packages/meshbay-node/src/meshbay_node/roster.py +packages/meshbay-node/src/meshbay_node/ui/app.py +packages/meshbay-node/src/meshbay_node/daemon.py +packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +packages/meshbay-common/src/meshbay_common/protocol.py +packages/meshbay-common/src/meshbay_common/handshake.py +packages/meshbay-hub/src/meshbay_hub/static/group-settings.js +packages/meshbay-hub/src/meshbay_hub/static/create-group-page.js +packages/meshbay-hub/src/meshbay_hub/static/group-page.js +packages/meshbay-hub/src/meshbay_hub/static/files-app.js +packages/meshbay-hub/src/meshbay_hub/static/chat-app.js +packages/meshbay-hub/src/meshbay_hub/static/apps.js +packages/meshbay-hub/src/meshbay_hub/static/transport.js +packages/meshbay-node/src/meshbay_node/indexer/indexer.py +packages/meshbay-hub/src/meshbay_hub/static/video-app.js +packages/meshbay-hub/src/meshbay_hub/static/music-app.js +packages/meshbay-hub/src/meshbay_hub/static/photos-app.js +packages/meshbay-hub/src/meshbay_hub/static/locales/*.js +``` + +### Phase 2 +``` +packages/meshbay-hub/src/meshbay_hub/static/folder-tree.js (new) +packages/meshbay-hub/src/meshbay_hub/static/chat-app-settings.js (new) +packages/meshbay-hub/src/meshbay_hub/static/video-app-settings.js (new) +packages/meshbay-hub/src/meshbay_hub/static/music-app-settings.js (new) +packages/meshbay-hub/src/meshbay_hub/static/photos-app-settings.js (new) +packages/meshbay-hub/src/meshbay_hub/static/group-settings.js (extract) +packages/meshbay-hub/src/meshbay_hub/static/apps.js +packages/meshbay-node/src/meshbay_node/ops.py +packages/meshbay-node/src/meshbay_node/ui/app.py +packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +packages/meshbay-node/src/meshbay_node/linkpreview.py +packages/meshbay-common/src/meshbay_common/protocol.py +packages/meshbay-hub/src/meshbay_hub/static/locales/*.js +``` + +### Phase 3 +``` +packages/meshbay-hub/src/meshbay_hub/static/helloworld-app.js (new) +packages/meshbay-hub/src/meshbay_hub/static/helloworld-app-settings.js (new) +packages/meshbay-hub/src/meshbay_hub/static/apps.js +packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py +packages/meshbay-node/src/meshbay_node/daemon.py +tests/test_cli_dispatch.py +QE/migration/migrate_groups_v2.py (new, not versioned) +``` + +--- + +## 7. Estimated effort + +| Phase | Scope | Approx. size | +|---|---|---| +| Phase 1 | Data model + shared dirs UI + eject/plug + CLI + remove uploads | ~1800 lines changed/added | +| Phase 2 | Folder tree + 4 app-settings files + settings restructure + server normalization | ~1800 lines changed/added | +| Phase 3 | HelloWorld + CLI polish + migration script | ~500 lines | + +Each phase is one focused Claude session. Test between phases. + +--- + +## 7b. Phase 1 review (2026-09-06) + +What the plan above got wrong, and what was actually built. The first three +entries are **rules for phases 2 and 3**, not one-off fixes: each describes a +shape the same code can take again. + +### The rules + +**An operator is not sitting at their node.** The shared directories table read +its roots exclusively from the loopback API (`platform.node.available`), which +resolves to "not available" in a browser. So the section rendered for nobody on +the web — while the Uploads controls it replaced *had* worked there — and the +`transport.updateRoot` / `ejectRoot` / `plugRoot` methods written next to it +were unreachable. §2.2 of the plan said "calls loopback API", and that was the +mistake: MNP is the path that must exist, and loopback is the fallback for a +local node with no live connection. Every operator-facing control phase 2 adds +(the folder tree, four app settings panes, the link-preview toggle) needs the +MNP route first. Pinned by `test_upload_controls_hidden.py`. + +**A reply that carries state nobody could have predicted has to be handed on.** +`transport.js` resolves an admin `*_ack` against the pending request and +returns, deliberately: every caller already updates local state from the value +it chose. The root acks are not like that — they carry the node's whole roots +table, including things only it knows (availability, the name it settled on, +the eject a failed plug left in place). Returning left the operator who clicked +Eject as the single client that never saw it happen, while every *other* peer +got the broadcast. Any phase-2 op returning computed state has the same shape. + +**A control that writes needs to name where.** The node was given a `root` +field on `file_upload` and no client ever sent it, so every upload went to +`writable_roots[0]` while the Files toolbar offered the button based on the root +being browsed. With two writable roots, uploading from one wrote into the other. +This is the failure `_settle_upload_root`'s deleted docstring existed to +prevent, reintroduced by removing it. Chat's attachments have the same problem +one level up and get an explicit `attachRoot` until §1.7 gives them a +configured directory. + +### The rest + +- **`ejected` was written to `roster.db` and never read back**, and the + auto-eject path did not persist at all. Both fixed; see §1.5b. +- **`PUT /api/groups/{gid}/member-upload` became a stub returning `200 + {"deprecated": true}`.** A route that answers OK and changes nothing is + indistinguishable from a working one to whoever calls it. Removed. +- **The wizard ignored the first root's RW switch** — `ops.attach_group` always + wrote `writable = true`. It takes the flag now. +- **`refresh_availability` was the only reader of a root's config.** The reload + path compared roots on `(name, path)`, so an operator editing `writable` in + `node.toml` and reloading saw nothing happen. The comparison includes the + flags. +- **The table had no Path column** (§1.5 asked for one). Two libraries whose + folders share a basename are indistinguishable without it, and the basename is + the identity — so it is the one thing that has to be visible. +- **Phase 1 shipped no tests.** 29 of the suite's failures were its own. The + gap that mattered was not the broken helpers but that eject, plug, per-root + upload refusal and the `node.toml` rewrite had no coverage at all: + `test_root_eject.py`, `test_root_writable_policy.py` and the new cases in + `test_ops.py` / `test_node_status.py` / `test_security_regressions.py` are + that. `test_member_upload_policy.py` is gone — it tested a removed feature. +- **`chat-app.js` was in §2.2's file list and was never touched.** +- **Eight of the ten locales were missing the new keys.** `test_locales.py` + holds them to `en.js`, so this was a failing test rather than a silent gap — + but it is worth noting that adding a key means adding it ten times. + +### Still open, deliberately + +- **The operator can no longer have a directory only they may write to.** RW is + open to every member; RO refuses everyone including the operator. This + reverses draft-v6's structural decision 9, which is annotated there. It is a + real capability removed, and if it turns out to be wanted the answer is a + third state on the root, not the old group-wide switch. +- `test_ops.py::test_a_backslash_path_written_into_node_toml_stays_parseable` + fails on any non-Windows machine and always has — it builds a + `PurePosixPath` from a Windows path. Unrelated to this refactor, left alone. + +--- + +## 7c. Phase 2 as built (2026-09-06) + +The plan held. Four things were done differently, and one of them is a rule. + +### The rule + +**A settings key added to the client must be added ten times.** `test_locales` +holds the nine other catalogues to `en.js`, so a missing key is a failing test +rather than a silent gap — but Phase 2 added 27 keys, and doing them one file +at a time is how the Phase 1 gap happened. Write the table, generate the +insert. + +**And a second one, which cost a bug in this phase:** `node --check foo.js` +does **not** reliably report a module syntax error. It accepted a file with +`${/* ... */''}` — htm template syntax, pasted into a plain object literal — +and reported success. Copying to `.mjs` first forces the module parser, which +reports it. `test_spa_syntax.py` now does that for every module; the suite had +no syntax check at all before, which is how the file was committed. + +### Done differently + +- **Almost no migration script.** The plan (§4.3) called for one to rename + `video_root` → `video_directories` in `roster.db`. Instead the roster falls + back to the old key when the new one is unset, and the first save through the + new path leaves it behind. A script that has to be run by hand on the machine + where it matters is a step that does not happen; a fallback is one that + cannot be skipped. `node.toml` needs nothing either — `upload = true` is read + as `writable`. + + **One transformation genuinely cannot be a fallback**, and an earlier draft + of this section wrongly said the script was unnecessary altogether. A group + whose operator had turned `member_upload` *off* has that switch ignored after + the upgrade, because nothing consults it any more — and its root still says + `upload = true`, so it accepts uploads from every member again. Nobody is + told. "Uploads are off for this group" and "this root is writable" are two + different sentences that happened to disagree, and only the operator knows + which they meant; there is nothing to infer. `QE/migration/check_upload_policy.py` + reads both files, reports the groups affected, and prints the `root set + --no-writable` line for each. Read-only, exits non-zero when something needs + a decision, so it can gate a deploy. +- **`music`, not `audio`.** The app's registry key was `music` while its + storage said `audio_root` and its ops said `set_audio_root`. One identifier + per app now — the registry key — with the correspondence in exactly one + table (`Roster.LEGACY_DIR_KEYS`). +- **One storage shape.** `set_app_directory` (single) writes a one-element + list, so there is no scalar form anywhere below the wire. `video_root` and + friends survive on the handshake ack only, *derived* from the list rather + than stored beside it — a second stored value drifts within one run, which + reads as "it works after a restart". +- **The panes call the transport themselves.** The plan had every pane report + through one `onSave`, which would have made the page a dispatcher naming + every app's settings keys — the thing the phase exists to remove. The line + is: what every app has (directories) the page does, generically; what one + app alone has (a TMDB key, a link-preview switch) the pane does with the + transport it is handed. An app that only wants directories touches neither + file, which is `test_app_settings_plugin.py`'s subject. + +### Worth knowing + +- `settings-ui.js` exists because `group-settings` → `apps` → a pane → + `group-settings` is an import cycle, and ES modules answer that with a + temporal-dead-zone `ReferenceError` at first render — the component simply + does not appear, which is the fault already recorded in CLAUDE.md about hook + ordering. The shared widgets live outside both. +- The folder picker asks the node for **nothing**. The tree is derived from + paths the client already holds, so it shows what the group's index contains + and no more — a folder the node never indexed does not exist as far as the + group is concerned. There is no folder-browsing protocol and this does not + add one. +- `_ASSETS` in `webapp.py` had to grow by six. Modules reached through the + registry rather than imported by name are exactly the ones nothing else would + notice changing, and a stale one is served from a browser cache with no + version bump. `test_asset_versioning` caught it. +- **What Phase 3 still owes:** the HelloWorld app (§4.1) — which is the actual + proof of the above, since every test here reads source rather than adding an + app and watching it work — plus the CLI polish and the Windows pass. + +--- + +## 7d. Phase 3 as built (2026-09-07) + +### HelloWorld earned its place + +It was written last and immediately found two things no amount of source +reading had: `group-settings.js` fell back to the *whole* registry when a group +had no `enabled_apps` yet — which would have enabled a hidden app for everyone +— and `group-page.js` wrote out `videoDirectories` / `musicDirectories` / +`photoDirectories` by hand, so a fifth app would have needed that file edited. +Both are fixed by making the code less app-specific, and the plugin claim is +now true rather than nearly true. + +That is the argument for keeping it: every other test of the architecture reads +source for the *absence* of app names, which proves nobody wrote a special case +— not that a new app works. Deleting HelloWorld would leave the claim resting +entirely on tests that read text. + +**It is hidden behind `?dev=1`**, not excluded from the build as §4.1 imagined. +There is no build step to exclude it from, and an unregistered app proves +nothing, since registration is exactly what is claimed to be sufficient. The +flag is the same opt-in shape as `transport.js`'s `?trace=1`. + +### The CLI + +**`--upload-dir` is gone, not deprecated.** It was documented as the old +spelling first, which was wrong and was caught in review. It wrote `upload_dir` +into a *brand-new* `[[groups]]` block, and `GroupConfig.__post_init__` reads +that key by forcing every other root read-only and appending that path as the +one writable one — so `group add --dir X --writable --upload-dir Y` silently +made X read-only. Two mechanisms deciding which directories accept uploads, one +of them invisible, in a group created after the model that replaced it. The +*read* path stays, because an existing node.toml must keep working; that is the +only legitimate use, and nothing writes the key any more. + +`member upload` reached the generic usage line for the other `member` verbs — +"usage: meshbay-node member upload <username>" — which advertises a removed +feature and sends the operator looking for a username it would then reject. It +names `root set --writable` now. `--upload-dir` still works, so an existing +script keeps working, but its help and the man page say it is the old spelling. + +### Windows + +`test_windows_root_shapes.py` covers what can be covered from here: drive +letters and UNC through `as_posix()` into TOML, a drive root having no basename +to derive a name from, and a case-insensitive collision — which on NTFS and +exFAT is one directory indexed as two roots. All pass. + +**What still needs a person on Windows**, and cannot be faked: + +- `ReadDirectoryChangesW` dropping events under load — the reason periodic + reconciliation is mandatory, and the reason eject exists at all +- `MAX_PATH` against a deep library, on download and on upload +- whether an eject actually lets the drive be removed, and a plug picks it back + up — the eject/plug pair is the least-exercised thing in all three phases +- the folder-tree picker against backslash paths in the UI +- the Create Group wizard with a drive-letter root + +### What the whole refactor still owes + +Nothing in the plan. Two things it did not think of: + +- **`_do_dir_delete` was never checked against RO/RW.** `_do_file_upload` and + `_do_dir_create` both gained the `writable` check; deletion is operator-only + and so is not the same hole, but the asymmetry is worth a look. +- **`index_delta` carries roots but not `dirs`.** A folder created by another + member does not reach a connected client's folder picker until a full + `index_sync`. Small, and the picker offers root names from the roots table + regardless, so nothing is unreachable — but it is the same class as the bug + §7d's roots fix closed. |