# Groups Refactor — Per-Root Permissions & App Plugin Architecture > Status: **Complete** (2026-09-07). All three phases built, reviewed and > tested against a running node. > > This is the most significant refactoring of the project. It changes how roots > are permissioned, how group applications are configured, and how the Settings > and Create Group pages are structured. > > §7b, §7c and §7d record what each phase's review found and where the plan > below was wrong. Several entries are rules rather than one-off fixes; §7d > also lists what a person still has to test by hand. --- ## 0. Summary of changes | Area | Before | After | |---|---|---| | Root permissions | One root marked `upload=True`; binary `member_upload` toggle per group | Each root is **RO** (default) or **RW**; multiple RW roots allowed; fully RO group is valid | | Upload policy | Separate section in Settings; `member_upload` signed op | **Removed.** RO/RW on the root is the mechanism. Files shows Upload only on RW roots. Chat disables attachments when its configured directory is not on a RW root | | Root metadata | `name, path, kind, upload, direct` | `name, path, kind, writable, removable, direct` | | Removable flag | Not tracked | Per-root boolean, set by operator. Enables the **eject/plug** button for safe device removal | | Safe eject | Auto-detected only (`path.is_dir()`) | Operator-initiated eject button in Settings AND Files root view. `ejected` state distinct from `available`. Indexer freezes entries, no data loss | | Files app | Can be disabled | **Always enabled**, transparently. Not shown in the app toggle list | | App selection at group creation | Checkbox list of all apps | **Removed.** Files is enabled automatically; other apps are configured later in Settings | | Settings layout | Monolithic: apps checkboxes, TMDB, MusicBrainz, directories, uploads — all in `group-settings.js` | **Structured:** Shared directories (top, expanded) → per-app sections (each with toggle + icon + title, collapsed, settings hidden until enabled) → Scan/Danger/Devices/Members | | App settings code | All inlined in `group-settings.js` (1338 lines) | Each app has `ui/-app-settings.js`; loaded by discovery (presence of the file) | | App enablement UI | One "Applications" section with checkboxes | Each app is a collapsible section with its own toggle in the title. The toggle is the enablement control | | Folder picker | Flat `` dropdowns in all app settings. It is also usable in other UI contexts (the video player's folder navigation already does something similar ad-hoc). ### 1.5 Shared directories table A reusable component (`SharedDirectoriesTable`) used in both the group Settings page and the Create Group wizard (developed once, shared). Features: - Borderless table, one row per root - Columns: **Name** (with folder icon), **Path** (truncated with tooltip on hover), **RW toggle** (switch, default off), **Removable toggle** (checkbox), **Eject/Plug button** (visible only when removable is checked — see §1.5b), **Delete button** (trash icon, with confirmation) - Ejected roots show a distinct visual state: greyed-out row, eject icon replaced by a plug icon - The first root in the Create Group wizard defaults to RW - A concise explanatory sentence at the top: "At least one directory is required. Read-write directories accept uploads from group members." - Add button: opens the native folder picker (Electron) or a path input (web, admin only) - Cannot delete the last root (refused with explanation) - Each change is a signed operator op (`ROOT_ADD`, `ROOT_REMOVE`, or new `ROOT_UPDATE` for toggling writable/removable on an existing root) ### 1.5b Safe eject for removable devices **Problem.** An operator stores data on a USB drive. Unplugging it without warning triggers the watchdog — file deletions propagate as though the operator erased an entire library. The existing `refresh_availability()` auto-detects this and freezes entries (good), but there is no way to eject cleanly before unplugging, and no way to re-plug without a full rescan. **The `ejected` state.** A root has two independent runtime states: - `ejected` (bool, default `false`): operator-controlled, persisted in `roster.db`. Set by clicking the eject button; cleared by clicking plug. - `available` (bool, runtime): computed as `not ejected and is_live()`. This is what clients and the indexer see. The distinction matters: when the operator clicks "eject" but hasn't physically unplugged yet, `is_live()` returns `true` but `available` is `false` because `ejected` is `true`. Without this, `refresh_availability()` would immediately flip it back to available. **Eject flow:** 1. Operator clicks the eject button (⏏) on a removable root 2. Confirmation dialog: "Eject *Movies*? Files from this directory will be temporarily hidden to all members. You can safely unplug the device." 3. On confirm: `PUT /api/groups/{gid}/roots/{name}/eject` → `ops.eject_root()` 4. `ops.eject_root()`: sets `ejected = true` in `roster.db`, marks root `available = false`, stops the watchdog observer for that root 5. The indexer **freezes** all entries from that root (existing behavior — no deletions, no index updates, cached data preserved) 6. `index_sync` update propagates to connected peers: the root's `available` is now `false` 7. All apps filter out entries from unavailable roots (Files already does this partially — needs to be complete across Videos, Music, Photos) 8. The operator can now safely unplug the device **Plug flow:** 1. Operator plugs the device back in and clicks the plug button (🔌) 2. `PUT /api/groups/{gid}/roots/{name}/plug` → `ops.plug_root()` 3. `ops.plug_root()`: checks `is_live()` first — if the path is not accessible, refuses with "Directory not found. Is the device connected?" 4. On success: sets `ejected = false`, marks root `available = true`, restarts the watchdog observer 5. The indexer **rescans the root** — its frozen entries are dropped and the directory is read again. (The plan said "a reconciliation, not a full rescan"; it is a rescan, deliberately. It is the same path a root coming back from `refresh_availability` already took, and a device people carry around can come back arbitrarily different — the hash cache means unchanged files are not re-read, which is where the cost would have been.) 6. `index_sync` update propagates — entries reappear in all apps **The flag is persisted, and restored at startup.** `ejected` lives in `roster.db` (`root_ejected:`), not in `node.toml`: it is runtime state, and an operator's hand-written config must not be rewritten because a USB drive was unplugged. It has to survive a restart — a restart is exactly what an operator does after noticing a drive fell off, and a flag that only lived in memory would let the scan that follows read the empty mount point as an erased library. `daemon._build_roots()` merges the two sources; it is the only place that builds a `RootSet` for a group. **Auto-detection safety net.** If a `removable` root's path suddenly disappears (operator unplugged without clicking eject): - `refresh_availability()` detects `is_live() = false` - Because `removable = true`, it sets `ejected = true` automatically (as if the operator had clicked eject), and reports it through the indexer's `on_root_ejected` callback so the daemon writes it to `roster.db` — an auto-eject that only existed in memory would be undone by the next restart - Entries freeze, no deletions propagate - The root stays in "ejected" state until the operator explicitly plugs it back For non-removable roots, the existing behavior is unchanged: `available` flips based on `is_live()`, entries freeze when unavailable, rescan when available again. **Eject button in Files app.** In addition to the Settings table, an eject button appears in the Files app root-level view, next to each removable root's name. This provides quick access without navigating to Settings. Same confirmation dialog, same API call. Ejected roots show as greyed-out with a plug icon to re-enable. **What is NOT deleted on eject:** - Index entries (frozen, not removed) - TMDB / MusicBrainz cached metadata - Video thumbnails in `media_cache` - Chat message history referencing files on that root - App directory configurations pointing to that root (but flagged as temporarily invalid — the app shows a warning, not an error) **MNP message:** `ROOT_EJECT` / `ROOT_EJECT_ACK` and `ROOT_PLUG` / `ROOT_PLUG_ACK` — signed operator ops, same pattern as `ROOT_UPDATE`. Broadcast to all connected peers so they see the availability change immediately without waiting for the next `index_sync`. ### 1.6 Server-side normalization (ops.py) Two generic functions replace the per-app specific ones: ```python def set_app_directory(state, group_id, app_key, path, *, require_writable=False): """Set a single directory for an app. Validates path is within a named root. If require_writable, refuses paths under RO roots.""" def set_app_directories(state, group_id, app_key, paths, *, require_writable=False): """Set multiple directories for an app. Same validation.""" ``` Existing functions (`set_video_root`, `set_audio_root`, `set_photo_roots`) become thin wrappers calling the generic versions, preserving the current MNP message types and roster keys for backward compat. New apps use the generic functions directly. ### 1.7 Chat settings additions - **Directory picker** (single, RW-only): selects the directory for chat file attachments. If no RW root exists, the picker shows an explanation and the attachment button is disabled in the chat UI. If the selected directory's root is later set to RO, the setting is flagged as invalid and attachments are disabled until corrected. - **Link preview toggle** (new, server-side): the operator can disable link previews for the group. Stored in `roster.db` as `chat_link_preview` (default: enabled). The node's `linkpreview.py` checks this setting before unfurling. The toggle is a `ToggleSwitch` in the Chat settings section. ### 1.8 Videos/Music/Photos settings changes **Videos:** - Directory picker changes from single to **multi-directory** (via folder tree widget) - TMDB settings section moved here from the monolithic settings - TMDB API key field: no longer says "optional" or mentions the default key. Instead, a prompt to sign up on TMDB with a direct link to generate a key - Per-app Save button triggers TMDB cache sweep **Music:** - Directory picker changes from single to **multi-directory** - MusicBrainz settings section moved here - Per-app Save button triggers MusicBrainz cache sweep **Photos:** - Multi-directory picker (already multi, just moves to the folder tree widget) - No third-party service settings ### 1.9 Handshake ack changes The `handshake_ack` payload gains per-root metadata: ```python # Current "roots": [{"name": "Movies", "path": "/mnt/movies", ...}] "member_upload": True # New "roots": [{"name": "Movies", "path": "/mnt/movies", "writable": False, "removable": True, ...}] # member_upload removed (deprecated, still parsed by old clients) ``` For backward compatibility with MNP 1.0 peers: - A 1.0 client that does not see `writable` on roots falls back to the old model (root with `upload=True` is writable, `member_upload` from the ack controls access) - A 1.1 node continues to send `member_upload` as a computed value: `True` if any root is writable, `False` otherwise — so old clients behave sensibly - `member_upload` is no longer writable via MNP ops; the node computes it from roots ### 1.10 CLI changes As built. The group is a `--group` option rather than a positional, matching every other verb in this CLI, and the negative flags are spelled `--no-writable` / `--no-removable` rather than `--read-only`, so each pair reads as one setting. ``` # Group creation (first root defaults to RW) meshbay-node group add --dir # first root, RW meshbay-node group add --dir --no-writable # first root, RO # Root management (--group is optional with one group configured) meshbay-node root list [--group ] meshbay-node root add [--name ] [--writable] [--removable] meshbay-node root remove [--yes] meshbay-node root set --writable | --no-writable meshbay-node root set --removable | --no-removable meshbay-node root eject # safe eject meshbay-node root plug # re-plug # Deprecated (accepted with a warning) --upload-dir → "use --writable on the target root instead" member upload → removed; use 'root set --no-writable' / '--writable' ``` ### 1.11 HelloWorld proof-of-concept A minimal app that validates the plugin architecture end to end: - `static/helloworld-app.js`: renders a greeting and lists files in its configured directory - `static/helloworld-app-settings.js`: single-directory picker (via folder tree widget), no other settings - Entry in `apps.js` with `key: "helloworld"`, icon, label, Component, Settings - `ALLOWED_APPS` extended on the node - No dedicated `helloworld.py` — uses the generic `set_app_directory()` function, which is the whole point The HelloWorld app is **not shipped in production**. It lives in the tree as a reference implementation and can be excluded from the build. Its value is proving that the plugin mechanism works: zero changes to `group-settings.js`, `group-page.js`, `webrtc_server.py` or `ops.py` to add it. ### 1.12 Migration (existing nodes) A script in `QE/migration/` (not versioned) handles Fedora and Ubuntu nodes: **node.toml:** - `upload = true` → `writable = true` - `upload = false` (or absent) → `writable = false` - Add `removable = false` to all roots that lack it - Remove `upload_dir` from `[[groups]]` blocks (if present) **roster.db:** - If `member_upload = "off"` for a group: set all that group's roots to `writable = false` (the intent was "no uploads") - Rename `video_root` → `video_directories` (wrap single value in a list) - Rename `audio_root` → `audio_directories` (same) - `photo_roots` → `photo_directories` (rename only) - Remove `member_upload` rows - Add `chat_link_preview = "true"` default for groups with chat enabled - Add `chat_directory` for groups that had an upload root (default: the upload root's path) **Protocol version:** - MNP version file bumped to 1.1 **The script is idempotent** — running it twice is safe. --- ## 2. Phase 1 — Root RO/RW model + Shared Directories UI **Goal:** Change the data model from `upload` to `writable`/`removable`, build the shared directories table, restructure the top of Settings and the Create Group wizard. Remove the Uploads section. Files always enabled. **Testable after this phase:** Create a group with RO/RW roots, toggle RW in Settings, add/remove roots in the new table, see the Upload button appear/disappear in Files based on the current root's writable flag, eject a removable root and verify files disappear from all apps without data loss, plug it back and verify files reappear, CLI works with new syntax including `root eject/plug`. ### 2.1 Backend changes | File | Change | |---|---| | `config.py` | `RootSpec`: add `writable: bool = False`, `removable: bool = False`. Remove `upload` field. `__post_init__` migration: `upload=True` → `writable=True`. Parse new fields from node.toml | | `roots.py` | `Root` dataclass: add `writable`, `removable`, `ejected`. Remove `upload`. `available` becomes a computed property: `not self.ejected and self.is_live()`. `_settle_upload_root()` removed. `RootSet.build()`: validate at least one root exists (no RW minimum). `refresh_availability()`: when a `removable` root's path disappears, auto-set `ejected=True` (safety net). Collision checks unchanged | | `ops.py` | `add_root()`: accept `writable`, `removable` params. `remove_root()`: refuse removing last root (unchanged). `attach_group()`: first root defaults to `writable=True`. Remove `set_member_upload()`. New: `update_root()` for toggling writable/removable on an existing root (signed op `OP_ROOT_UPDATE`). New: `eject_root()` — sets `ejected=True`, stops watchdog for that root. New: `plug_root()` — checks `is_live()`, sets `ejected=False`, triggers reconciliation | | `roster.py` | No schema change (generic key/value). Remove `member_upload` handling from `_apply_group_settings()` | | `ui/app.py` | `POST /api/groups/{gid}/roots`: accept `writable`, `removable`. New: `PATCH /api/groups/{gid}/roots/{name}` → `ops.update_root()`. New: `PUT /api/groups/{gid}/roots/{name}/eject` → `ops.eject_root()`. New: `PUT /api/groups/{gid}/roots/{name}/plug` → `ops.plug_root()`. Remove `PUT /api/groups/{gid}/member-upload` | | `daemon.py` (CLI) | New `root` subcommand: `add`, `remove`, `set`, `list`. `group add --dir` defaults to `writable=True`. Deprecate `--upload-dir` with warning. Remove `member upload` command | | `webrtc_server.py` | Handshake ack: add `writable`/`removable`/`ejected` per root. Compute `member_upload` for backward compat. `_do_file_upload`: check `root.writable` instead of `root.upload` + `member_upload_allowed`. Handle `ROOT_UPDATE`, `ROOT_EJECT`, `ROOT_PLUG` MNP messages. Broadcast root availability changes to all connected peers. `ALLOWED_APPS`: add `"files"` to always-enabled set | | `protocol.py` | New message types: `ROOT_UPDATE` / `ROOT_UPDATE_ACK`, `ROOT_EJECT` / `ROOT_EJECT_ACK`, `ROOT_PLUG` / `ROOT_PLUG_ACK`. `ROOT_ADD` gains `writable`, `removable` fields | | `indexer.py` | `eject_root()`: stop the watchdog observer for that root, do NOT touch entries. `plug_root()`: restart observer, trigger reconciliation pass. `refresh_availability()`: auto-eject removable roots whose path disappears (set `ejected=True` instead of just flipping `available`) | | `handshake.py` | MNP version → 1.1 (minor, additive) | ### 2.2 Frontend changes | File | Change | |---|---| | `group-settings.js` | New `SharedDirectoriesTable` component (reusable). Includes eject/plug button per removable root. Move to top of settings (after Invite/Pair). Remove the Uploads toggle section. Remove the old Directories section (root management part — app root pickers stay for now). Calls loopback API for add/remove/update/eject/plug root | | `create-group-page.js` | Replace app checkboxes with nothing (Files auto-enabled). Replace directory section with `SharedDirectoriesTable` (same component). First root defaults `writable=true`. Step 2: remove `set enabled_apps` call (Files is automatic) | | `files-app.js` | Upload button visibility: check `currentRoot.writable && currentRoot.available`. Upload target: the root currently being browsed. Hide upload affordances on RO roots. Eject/plug button next to each removable root name in root-level view. Ejected roots greyed out with plug icon. Entries from unavailable roots filtered out of all views | | `chat-app.js` | Attachment button: disabled if no RW root exists or if the chat directory's root is RO or unavailable. Tooltip explaining why | | `group-page.js` | `nodeRoots` state: include `writable`/`removable`/`ejected` from handshake ack. Handle `ROOT_EJECT_ACK`/`ROOT_PLUG_ACK` broadcasts to update root state live. Remove `memberUpload` state. Remove `onUploadPolicy` callback | | `video-app.js` | Filter entries: exclude files from unavailable roots | | `music-app.js` | Filter entries: exclude files from unavailable roots | | `photos-app.js` | Filter entries: exclude files from unavailable roots | | `apps.js` | Add `alwaysEnabled: true` to `files` entry | | `transport.js` | Parse new root fields from handshake ack | ### 2.3 Localization New keys in `locales/*.js`: - `sharedDirectories`, `sharedDirectoriesHint` ("At least one directory is required...") - `readOnly`, `readWrite`, `removableDevice` - `uploadNotAvailableRO` (tooltip: "This directory is read-only") - `ejectRoot`, `ejectRootConfirm` ("Eject *{name}*? Files from this directory will be temporarily hidden...") - `plugRoot`, `plugRootFailed` ("Directory not found. Is the device connected?") - `rootEjected` (status label shown on ejected roots) - Deprecation: `uploadToggle*` keys can be removed --- ## 3. Phase 2 — App Settings Plugin Architecture + Folder Tree Widget **Goal:** Split app settings into per-app files, build the folder tree widget, restructure Settings with per-app collapsible sections. Normalize server-side ops. **Testable after this phase:** Each app has its own settings section with toggle. Folder tree popup works for directory selection. TMDB/MusicBrainz settings are in their app sections. Chat has link preview toggle. Videos/Music use multi-directory. ### 3.1 Folder tree widget | File | Change | |---|---| | `static/folder-tree.js` (new) | `FolderTreePicker` component. Props: `roots` (with writable/removable), `dirs` (flat list from index), `mode` ("single"/"multi"), `requireWritable` (bool), `selected` (current selection), `onSelect` callback. Renders a modal with tree-view of directories. Builds tree structure from flat `nodeDirs` paths | **Behavior:** - Modal overlay with semi-transparent backdrop, centered panel - Root level: each root with folder icon, name, RO/RW badge, removable badge - `[+]` / `[-]` toggle to expand/collapse children - Directories sorted alphabetically at each level - Single mode: clicking a directory selects it (highlight), deselects previous - Multi mode: clicking a directory toggles its selection (checkbox visual) - `requireWritable=true`: RO roots and all their children are greyed out and unclickable, with a brief explanation at the top of the modal - Selected path shown at the bottom of the modal - OK (disabled if nothing valid selected) and Cancel buttons - Escape key closes ### 3.2 Per-app settings files | File | Content | |---|---| | `static/chat-app-settings.js` (new) | `ChatSettings` component. Single-directory picker (folder tree, `requireWritable=true`). Link preview toggle (`ToggleSwitch`). Save button. Validates: selected directory must be on a RW root | | `static/video-app-settings.js` (new) | `VideoSettings` component. Multi-directory picker (folder tree). TMDB section: toggle + API key + language (moved from `group-settings.js`). TMDB key prompt: "Sign up on TMDB to generate your API key" with link, no "optional" wording. Save triggers TMDB sweep | | `static/music-app-settings.js` (new) | `MusicSettings` component. Multi-directory picker. MusicBrainz section: toggle (moved from `group-settings.js`). Save triggers MusicBrainz sweep | | `static/photos-app-settings.js` (new) | `PhotoSettings` component. Multi-directory picker. No third-party settings. Save button | | `static/apps.js` | Each `APPS` entry gains `Settings` field imported from the corresponding settings file. `files` has no `Settings` (no per-app config). `files` has `alwaysEnabled: true` (from Phase 1) | ### 3.3 Settings page restructure | File | Change | |---|---| | `group-settings.js` | Remove all inlined app settings (TMDB section, MusicBrainz section, app root pickers). Remove the Applications checkbox section. New rendering loop: for each app in `APPS` where `!app.alwaysEnabled && app.Settings`, render a `CollapsibleSection` with: icon + title in header, toggle switch in header, `app.Settings` component inside (visible only when enabled). Section collapsed by default. App toggle triggers signed `OP_APPS_ENABLED` op | **Final settings layout:** ``` 1. Invite form (if admin & invite-only) 2. Pair operator (if nodeAdmin & not paired) ── App configuration ── 3. Shared directories (expanded, SharedDirectoriesTable) 4. 💬 Chat [toggle] (collapsed) └─ Directory picker, Link preview toggle, Save 5. 🎬 Videos [toggle] (collapsed) └─ Directory picker (multi), TMDB settings, Save 6. 🎵 Music [toggle] (collapsed) └─ Directory picker (multi), MusicBrainz settings, Save 7. 📷 Photos [toggle] (collapsed) └─ Directory picker (multi), Save 8. [Future apps discovered from APPS registry] ── Node tuning ── 9. Scan tuning (reconcile, debounce) ── Danger zone ── 10. Leave / Delete group ── Identity ── 11. My devices 12. Members table ``` ### 3.4 Server-side normalization | File | Change | |---|---| | `ops.py` | New generic: `set_app_directory(state, group_id, app_key, path, require_writable=False)` and `set_app_directories(state, group_id, app_key, paths, require_writable=False)`. Both validate path(s) within named roots and check writable if required. Existing `set_video_root` → wrapper calling `set_app_directories("video", ...)`. Same for audio, photo. New: `set_chat_directory(state, group_id, path)` → `set_app_directory("chat", path, require_writable=True)`. New: `set_chat_link_preview(state, group_id, enabled)` | | `webrtc_server.py` | Handle new MNP messages: `CHAT_DIRECTORY` / `ACK`, `CHAT_LINK_PREVIEW` / `ACK`. Refactor `VIDEO_ROOT` handler to use generic. Handshake ack: add `chat_directory`, `chat_link_preview` | | `protocol.py` | New message types: `CHAT_DIRECTORY`, `CHAT_LINK_PREVIEW` (and acks) | | `ui/app.py` | New endpoints: `PUT /api/groups/{gid}/chat-directory`, `PUT /api/groups/{gid}/chat-link-preview`. Generic: `PUT /api/groups/{gid}/app-directories/{app_key}` | | `linkpreview.py` | Check `chat_link_preview` setting before unfurling | ### 3.5 Handshake ack additions ```python "chat_directory": "/Shared/uploads", "chat_link_preview": True, "video_directories": ["/Movies", "/Series"], # replaces video_root (single) "audio_directories": ["/Music"], # replaces audio_root (single) "photo_directories": ["/Photos", "/Camera"], # replaces photo_roots ``` Old field names (`video_root`, `audio_root`, `photo_roots`) still sent for backward compat with MNP 1.0 clients. New clients read the `*_directories` form. --- ## 4. Phase 3 — HelloWorld, CLI polish & Migration **Goal:** Prove the plugin architecture with a HelloWorld app, finalize CLI changes, write and test the migration script. ### 4.1 HelloWorld app | File | Content | |---|---| | `static/helloworld-app.js` (new) | Minimal component: renders "Hello, World!" heading + lists files from its configured directory. Uses `entries` from `commonProps`, filtered by the configured path | | `static/helloworld-app-settings.js` (new) | `HelloWorldSettings` component: single-directory picker via `FolderTreePicker`, Save button. Uses `set_app_directory("helloworld", path)` on the generic endpoint | | `apps.js` | New entry: `{ key: "helloworld", icon: "👋", labelKey: "helloWorld", Component: HelloWorldApp, Settings: HelloWorldSettings }` | | `webrtc_server.py` | Add `"helloworld"` to `ALLOWED_APPS` | **Validation:** enabling HelloWorld in Settings, picking a directory, saving, and seeing the file list in the HelloWorld tab — with **zero changes** to `group-settings.js`, `group-page.js`, or `ops.py`. If this works, any future app can be added the same way. ### 4.2 CLI final polish - `root` subcommand fully tested: `add`, `remove`, `set`, `list` - `group add --upload-dir` prints deprecation warning and maps to `--dir --writable` - `member upload` command removed (prints migration guidance) - Help text updated to reflect RO/RW model - `test_cli_dispatch.py` updated for all new verbs ### 4.3 Migration script `QE/migration/migrate_groups_v2.sh` (or `.py`) — not versioned, for Fedora/Ubuntu nodes only. **node.toml transformations:** ``` upload = true → writable = true upload = false → writable = false (no upload field) → writable = false (add removable = false to every root that lacks it) (remove upload_dir lines from [[groups]] blocks) ``` **roster.db transformations:** ```sql -- Convert member_upload=off to all-RO roots (handled via ops on restart) -- Rename app directory keys UPDATE group_settings SET key = 'video_directories' WHERE key = 'video_root'; UPDATE group_settings SET key = 'audio_directories' WHERE key = 'audio_root'; UPDATE group_settings SET key = 'photo_directories' WHERE key = 'photo_roots'; -- Wrap single values in JSON arrays for video/audio -- Add chat defaults INSERT INTO group_settings (group_id, key, value) SELECT group_id, 'chat_link_preview', 'true' FROM group_settings WHERE key = 'enabled_apps' AND value LIKE '%chat%'; -- Remove member_upload rows DELETE FROM group_settings WHERE key = 'member_upload'; ``` **Idempotency:** every transformation is guarded (`IF NOT EXISTS`, check before rename, etc.). Safe to run twice. **Rollback:** the script backs up `node.toml` and `roster.db` before any change. ### 4.4 Windows notes No migration script needed for Windows (manual setup). Functional non-regression testing only: - Drive letter roots (`D:\Movies`, `E:\Music`) work with RO/RW - Removable flag on USB drives - Folder tree widget handles backslash paths - Create Group wizard with Windows paths --- ## 5. Risk assessment | Risk | Mitigation | |---|---| | Upload regression | Phase 1 is self-contained: test uploads on RW roots, verify refused on RO roots, before touching app settings | | Eject data loss | Eject freezes entries (existing `freeze-not-empty` path). Ejected roots are never rescanned. Auto-eject safety net for surprise unplugs on removable roots. Explicit reconciliation (not full rescan) on plug | | Indexer race on eject | Watchdog observer is stopped synchronously before `ejected=true` is set. No window where the watchdog sees a missing path and processes deletions | | MNP backward compat | Computed `member_upload` in ack for 1.0 clients. New fields additive. Old field names kept alongside new ones | | Monolithic `group-settings.js` diff | Phase 2 extracts code into new files; the old code is deleted, not refactored. Clear before/after | | Folder tree perf with large indexes | `nodeDirs` is already computed. Tree construction is O(n) on directory count, not file count. Lazy child rendering on expand | | Windows path handling | Existing NFC normalization and path handling unchanged. New fields (`writable`, `removable`) are path-independent booleans. Drive letters work as root paths | | Migration data loss | Script backs up before changes. Idempotent. Tested on a staging node before production | --- ## 6. Files touched (by phase) ### Phase 1 ``` packages/meshbay-node/src/meshbay_node/config.py packages/meshbay-node/src/meshbay_node/roots.py packages/meshbay-node/src/meshbay_node/ops.py packages/meshbay-node/src/meshbay_node/roster.py packages/meshbay-node/src/meshbay_node/ui/app.py packages/meshbay-node/src/meshbay_node/daemon.py packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py packages/meshbay-common/src/meshbay_common/protocol.py packages/meshbay-common/src/meshbay_common/handshake.py packages/meshbay-hub/src/meshbay_hub/static/group-settings.js packages/meshbay-hub/src/meshbay_hub/static/create-group-page.js packages/meshbay-hub/src/meshbay_hub/static/group-page.js packages/meshbay-hub/src/meshbay_hub/static/files-app.js packages/meshbay-hub/src/meshbay_hub/static/chat-app.js packages/meshbay-hub/src/meshbay_hub/static/apps.js packages/meshbay-hub/src/meshbay_hub/static/transport.js packages/meshbay-node/src/meshbay_node/indexer/indexer.py packages/meshbay-hub/src/meshbay_hub/static/video-app.js packages/meshbay-hub/src/meshbay_hub/static/music-app.js packages/meshbay-hub/src/meshbay_hub/static/photos-app.js packages/meshbay-hub/src/meshbay_hub/static/locales/*.js ``` ### Phase 2 ``` packages/meshbay-hub/src/meshbay_hub/static/folder-tree.js (new) packages/meshbay-hub/src/meshbay_hub/static/chat-app-settings.js (new) packages/meshbay-hub/src/meshbay_hub/static/video-app-settings.js (new) packages/meshbay-hub/src/meshbay_hub/static/music-app-settings.js (new) packages/meshbay-hub/src/meshbay_hub/static/photos-app-settings.js (new) packages/meshbay-hub/src/meshbay_hub/static/group-settings.js (extract) packages/meshbay-hub/src/meshbay_hub/static/apps.js packages/meshbay-node/src/meshbay_node/ops.py packages/meshbay-node/src/meshbay_node/ui/app.py packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py packages/meshbay-node/src/meshbay_node/linkpreview.py packages/meshbay-common/src/meshbay_common/protocol.py packages/meshbay-hub/src/meshbay_hub/static/locales/*.js ``` ### Phase 3 ``` packages/meshbay-hub/src/meshbay_hub/static/helloworld-app.js (new) packages/meshbay-hub/src/meshbay_hub/static/helloworld-app-settings.js (new) packages/meshbay-hub/src/meshbay_hub/static/apps.js packages/meshbay-node/src/meshbay_node/transport/webrtc_server.py packages/meshbay-node/src/meshbay_node/daemon.py tests/test_cli_dispatch.py QE/migration/migrate_groups_v2.py (new, not versioned) ``` --- ## 7. Estimated effort | Phase | Scope | Approx. size | |---|---|---| | Phase 1 | Data model + shared dirs UI + eject/plug + CLI + remove uploads | ~1800 lines changed/added | | Phase 2 | Folder tree + 4 app-settings files + settings restructure + server normalization | ~1800 lines changed/added | | Phase 3 | HelloWorld + CLI polish + migration script | ~500 lines | Each phase is one focused Claude session. Test between phases. --- ## 7b. Phase 1 review (2026-09-06) What the plan above got wrong, and what was actually built. The first three entries are **rules for phases 2 and 3**, not one-off fixes: each describes a shape the same code can take again. ### The rules **An operator is not sitting at their node.** The shared directories table read its roots exclusively from the loopback API (`platform.node.available`), which resolves to "not available" in a browser. So the section rendered for nobody on the web — while the Uploads controls it replaced *had* worked there — and the `transport.updateRoot` / `ejectRoot` / `plugRoot` methods written next to it were unreachable. §2.2 of the plan said "calls loopback API", and that was the mistake: MNP is the path that must exist, and loopback is the fallback for a local node with no live connection. Every operator-facing control phase 2 adds (the folder tree, four app settings panes, the link-preview toggle) needs the MNP route first. Pinned by `test_upload_controls_hidden.py`. **A reply that carries state nobody could have predicted has to be handed on.** `transport.js` resolves an admin `*_ack` against the pending request and returns, deliberately: every caller already updates local state from the value it chose. The root acks are not like that — they carry the node's whole roots table, including things only it knows (availability, the name it settled on, the eject a failed plug left in place). Returning left the operator who clicked Eject as the single client that never saw it happen, while every *other* peer got the broadcast. Any phase-2 op returning computed state has the same shape. **A control that writes needs to name where.** The node was given a `root` field on `file_upload` and no client ever sent it, so every upload went to `writable_roots[0]` while the Files toolbar offered the button based on the root being browsed. With two writable roots, uploading from one wrote into the other. This is the failure `_settle_upload_root`'s deleted docstring existed to prevent, reintroduced by removing it. Chat's attachments have the same problem one level up and get an explicit `attachRoot` until §1.7 gives them a configured directory. ### The rest - **`ejected` was written to `roster.db` and never read back**, and the auto-eject path did not persist at all. Both fixed; see §1.5b. - **`PUT /api/groups/{gid}/member-upload` became a stub returning `200 {"deprecated": true}`.** A route that answers OK and changes nothing is indistinguishable from a working one to whoever calls it. Removed. - **The wizard ignored the first root's RW switch** — `ops.attach_group` always wrote `writable = true`. It takes the flag now. - **`refresh_availability` was the only reader of a root's config.** The reload path compared roots on `(name, path)`, so an operator editing `writable` in `node.toml` and reloading saw nothing happen. The comparison includes the flags. - **The table had no Path column** (§1.5 asked for one). Two libraries whose folders share a basename are indistinguishable without it, and the basename is the identity — so it is the one thing that has to be visible. - **Phase 1 shipped no tests.** 29 of the suite's failures were its own. The gap that mattered was not the broken helpers but that eject, plug, per-root upload refusal and the `node.toml` rewrite had no coverage at all: `test_root_eject.py`, `test_root_writable_policy.py` and the new cases in `test_ops.py` / `test_node_status.py` / `test_security_regressions.py` are that. `test_member_upload_policy.py` is gone — it tested a removed feature. - **`chat-app.js` was in §2.2's file list and was never touched.** - **Eight of the ten locales were missing the new keys.** `test_locales.py` holds them to `en.js`, so this was a failing test rather than a silent gap — but it is worth noting that adding a key means adding it ten times. ### Still open, deliberately - **The operator can no longer have a directory only they may write to.** RW is open to every member; RO refuses everyone including the operator. This reverses draft-v6's structural decision 9, which is annotated there. It is a real capability removed, and if it turns out to be wanted the answer is a third state on the root, not the old group-wide switch. - `test_ops.py::test_a_backslash_path_written_into_node_toml_stays_parseable` fails on any non-Windows machine and always has — it builds a `PurePosixPath` from a Windows path. Unrelated to this refactor, left alone. --- ## 7c. Phase 2 as built (2026-09-06) The plan held. Four things were done differently, and one of them is a rule. ### The rule **A settings key added to the client must be added ten times.** `test_locales` holds the nine other catalogues to `en.js`, so a missing key is a failing test rather than a silent gap — but Phase 2 added 27 keys, and doing them one file at a time is how the Phase 1 gap happened. Write the table, generate the insert. **And a second one, which cost a bug in this phase:** `node --check foo.js` does **not** reliably report a module syntax error. It accepted a file with `${/* ... */''}` — htm template syntax, pasted into a plain object literal — and reported success. Copying to `.mjs` first forces the module parser, which reports it. `test_spa_syntax.py` now does that for every module; the suite had no syntax check at all before, which is how the file was committed. ### Done differently - **Almost no migration script.** The plan (§4.3) called for one to rename `video_root` → `video_directories` in `roster.db`. Instead the roster falls back to the old key when the new one is unset, and the first save through the new path leaves it behind. A script that has to be run by hand on the machine where it matters is a step that does not happen; a fallback is one that cannot be skipped. `node.toml` needs nothing either — `upload = true` is read as `writable`. **One transformation genuinely cannot be a fallback**, and an earlier draft of this section wrongly said the script was unnecessary altogether. A group whose operator had turned `member_upload` *off* has that switch ignored after the upgrade, because nothing consults it any more — and its root still says `upload = true`, so it accepts uploads from every member again. Nobody is told. "Uploads are off for this group" and "this root is writable" are two different sentences that happened to disagree, and only the operator knows which they meant; there is nothing to infer. `QE/migration/check_upload_policy.py` reads both files, reports the groups affected, and prints the `root set --no-writable` line for each. Read-only, exits non-zero when something needs a decision, so it can gate a deploy. - **`music`, not `audio`.** The app's registry key was `music` while its storage said `audio_root` and its ops said `set_audio_root`. One identifier per app now — the registry key — with the correspondence in exactly one table (`Roster.LEGACY_DIR_KEYS`). - **One storage shape.** `set_app_directory` (single) writes a one-element list, so there is no scalar form anywhere below the wire. `video_root` and friends survive on the handshake ack only, *derived* from the list rather than stored beside it — a second stored value drifts within one run, which reads as "it works after a restart". - **The panes call the transport themselves.** The plan had every pane report through one `onSave`, which would have made the page a dispatcher naming every app's settings keys — the thing the phase exists to remove. The line is: what every app has (directories) the page does, generically; what one app alone has (a TMDB key, a link-preview switch) the pane does with the transport it is handed. An app that only wants directories touches neither file, which is `test_app_settings_plugin.py`'s subject. ### Worth knowing - `settings-ui.js` exists because `group-settings` → `apps` → a pane → `group-settings` is an import cycle, and ES modules answer that with a temporal-dead-zone `ReferenceError` at first render — the component simply does not appear, which is the fault already recorded in CLAUDE.md about hook ordering. The shared widgets live outside both. - The folder picker asks the node for **nothing**. The tree is derived from paths the client already holds, so it shows what the group's index contains and no more — a folder the node never indexed does not exist as far as the group is concerned. There is no folder-browsing protocol and this does not add one. - `_ASSETS` in `webapp.py` had to grow by six. Modules reached through the registry rather than imported by name are exactly the ones nothing else would notice changing, and a stale one is served from a browser cache with no version bump. `test_asset_versioning` caught it. - **What Phase 3 still owes:** the HelloWorld app (§4.1) — which is the actual proof of the above, since every test here reads source rather than adding an app and watching it work — plus the CLI polish and the Windows pass. --- ## 7d. Phase 3 as built (2026-09-07) ### HelloWorld earned its place It was written last and immediately found two things no amount of source reading had: `group-settings.js` fell back to the *whole* registry when a group had no `enabled_apps` yet — which would have enabled a hidden app for everyone — and `group-page.js` wrote out `videoDirectories` / `musicDirectories` / `photoDirectories` by hand, so a fifth app would have needed that file edited. Both are fixed by making the code less app-specific, and the plugin claim is now true rather than nearly true. That is the argument for keeping it: every other test of the architecture reads source for the *absence* of app names, which proves nobody wrote a special case — not that a new app works. Deleting HelloWorld would leave the claim resting entirely on tests that read text. **It is hidden behind `?dev=1`**, not excluded from the build as §4.1 imagined. There is no build step to exclude it from, and an unregistered app proves nothing, since registration is exactly what is claimed to be sufficient. The flag is the same opt-in shape as `transport.js`'s `?trace=1`. ### The CLI `member upload` reached the generic usage line for the other `member` verbs — "usage: meshbay-node member upload " — which advertises a removed feature and sends the operator looking for a username it would then reject. It names `root set --writable` now. `--upload-dir` still works, so an existing script keeps working, but its help and the man page say it is the old spelling. ### Windows `test_windows_root_shapes.py` covers what can be covered from here: drive letters and UNC through `as_posix()` into TOML, a drive root having no basename to derive a name from, and a case-insensitive collision — which on NTFS and exFAT is one directory indexed as two roots. All pass. **What still needs a person on Windows**, and cannot be faked: - `ReadDirectoryChangesW` dropping events under load — the reason periodic reconciliation is mandatory, and the reason eject exists at all - `MAX_PATH` against a deep library, on download and on upload - whether an eject actually lets the drive be removed, and a plug picks it back up — the eject/plug pair is the least-exercised thing in all three phases - the folder-tree picker against backslash paths in the UI - the Create Group wizard with a drive-letter root ### What the whole refactor still owes Nothing in the plan. Two things it did not think of: - **`_do_dir_delete` was never checked against RO/RW.** `_do_file_upload` and `_do_dir_create` both gained the `writable` check; deletion is operator-only and so is not the same hole, but the asymmetry is worth a look. - **`index_delta` carries roots but not `dirs`.** A folder created by another member does not reach a connected client's folder picker until a full `index_sync`. Small, and the picker offers root names from the roots table regardless, so nothing is unreachable — but it is the same class as the bug §7d's roots fix closed.