diff options
Diffstat (limited to 'docs/refactor-node-ui.md')
| -rw-r--r-- | docs/refactor-node-ui.md | 403 |
1 files changed, 403 insertions, 0 deletions
diff --git a/docs/refactor-node-ui.md b/docs/refactor-node-ui.md new file mode 100644 index 0000000..7bab5d2 --- /dev/null +++ b/docs/refactor-node-ui.md @@ -0,0 +1,403 @@ +# Refactor: fold the node's local admin surface into the desktop Node page + +> Status: **complete** (2026-09-01) — all six phases landed; see §11 for what +> each did. This stays as the decision record. +> Scope: the node's *own* administration surface. Group-level settings +> (member-upload, apps, scan-settings) are not touched — they live in the group +> UI and stay there. +> +> **Net result:** `ui/app.py` is a JSON-only, ruff-clean loopback control API +> (1074 → 453 lines); the server-rendered dashboard, the `/audit` HTML page, +> the `meshbay-node ui` verb and three never-wired endpoints are gone; the +> desktop client's Node page (`static/node-page.js`) gained an Overview tab, +> node-wide Roster, Peers, a paginated Audit log with CSV export, index-cache +> Maintenance, and an Unlink action, organised into six tabs; the pointless +> `18000/tcp` firewall profiles were deleted. One capability is intentionally +> gone: browser-based admin on a headless server — the CLI covers every +> operation there. + +--- + +## 1. The distinction that matters + +"The local admin UI" is three things bundled in `packages/meshbay-node/src/meshbay_node/ui/`: + +| Component | Consumers today | Redundant with `node-page.js`? | +|---|---|---| +| **Loopback JSON API** `/api/*` (~35 routes), `127.0.0.1:{ui_port}`, per-run token | the **entire CLI** (`_daemon_api` — every verb is an HTTP call), `node-page.js` (via `platform.node.call`), `platform.node.detect()`, the create-group wizard's index-progress poll | **No.** This is the node's control plane. | +| **Server-rendered HTML** — `GET /` (dashboard), `GET /audit` (`_DASHBOARD` / `_AUDIT_HTML` templates + `_render_*` helpers) | a human with a browser (`meshbay-node ui` prints a tokened URL; `ssh -L` from a remote host) | **Yes, entirely.** | +| uvicorn server + middleware (token gate, CSP headers) | — | Stays — the API needs it. | + +**This refactor removes the HTML layer only.** Removing the module would break +the CLI, node detection, and the Node page itself. + +--- + +## 2. Decision + +1. Enrich `packages/meshbay-hub/src/meshbay_hub/static/node-page.js` so it covers + everything the HTML dashboard showed (peers, audit, index-cache maintenance, + node-wide roster, node identity/QUIC/hub display). +2. Add a **Unlink node** action (endpoint already exists, nothing surfaces it). +3. Fix the audit view to show real usernames. +4. Organise `node-page.js` into tabs. +5. Delete `GET /` and `GET /audit` from `ui/app.py`, plus the templates, the + `render` helpers, the `meshbay-node ui` CLI verb, and every doc/packaging + reference to a browser admin UI. +6. Keep the whole `/api/*` surface and the uvicorn server. + +**Accepted cost:** on a headless server (the meshbay.org model) the only admin +path becomes the CLI. The CLI already covers every operation, so this is +acceptable — but it is a real capability reduction and is recorded here as a +conscious choice, not an oversight. + +--- + +## 3. Inventory — nothing is lost + +### 3a. HTML dashboard content → destination + +| Dashboard element | Backing endpoint | Destination | +|---|---|---| +| Node status badge, `needs` hints | `GET /api/status` | Node page header (already partly there) | +| Stats: groups / files / peers / user | `GET /api/status` | Overview tab | +| **Connected Peers** table | `GET /api/peers` | **Peers tab (new wiring)** | +| Roster (node-wide) | `GET /api/roster?group_id=` | **Roster tab — extend the current per-group view to node-wide** | +| Groups list | `GET /api/groups` | Groups tab (already there) | +| Node Configuration: Hub URL, QUIC port, Node ID | `GET /api/status` (`endpoint_hint`, `quic_port`, `hub_url`) | Overview tab (new fields) | +| Node Configuration: 5 `[node]` settings | `GET /api/node-settings` | Settings tab (already there) | +| **Maintenance**: index-cache count + Prune | `GET /api/index-cache`, `POST /api/index-cache/prune` | **Settings/Maintenance tab (new wiring)** | +| `/audit` page | `GET /api/audit` | **Audit tab (new wiring)** | + +### 3b. `/api/*` endpoints — consumer and disposition + +| Endpoint | Consumed by | Disposition | +|---|---|---| +| `GET /api/status` | Electron `node:detect`, CLI `status`, node-page | keep | +| `DELETE /api/unlink` | *nothing* | keep + **surface in node-page** | +| `GET /api/groups` | CLI, node-page | keep | +| `POST /api/groups/attach` · `POST /api/groups/detach` | CLI, node-page (detach) | keep | +| `DELETE /api/groups/{g}/files/{f}` | CLI `file rm`, group Files UI | keep | +| `GET /api/groups/{g}/files` | CLI `file list`, HTML dashboard | keep (drop dashboard caller) | +| `GET /api/denylist` · `POST /api/denylist/clear` | CLI, node-page | keep | +| `GET /api/index-cache` · `POST /api/index-cache/prune` | **HTML dashboard only** | keep + **wire into node-page** | +| `POST /api/groups/{g}/video/rematch` | CLI, group Videos UI | keep | +| `GET /api/peers` | **HTML dashboard only** | keep + **wire into node-page** | +| `GET /api/audit` | **HTML `/audit` only** | keep + **wire into node-page** + username fix (§5) | +| `GET /api/config` | *nothing* | **triage — likely dead (§6b)** | +| `POST /api/operator/pair` | CLI, node-page | keep | +| `GET /api/roster` | CLI, node-page | keep (node-page uses node-wide mode too) | +| `POST /api/groups/{g}/invites` · `GET /api/resolve` | CLI | keep | +| `POST /api/members/{u}/revoke` | CLI | keep + optionally surface in node-page | +| `POST /api/members/{u}/unpin` | CLI, node-page | keep | +| `POST /api/groups/{g}/gek` | CLI `gek init/rotate`, node-page (rotate) | keep | +| `POST /api/groups/{g}/roots` · `DELETE .../roots/{n}` | node-page | keep | +| `GET /api/groups/{g}/index-status` | create-group wizard | keep | +| `PUT /api/groups/{g}/member-upload` · `.../apps` · `.../scan-settings` | group Settings UI | keep (out of scope) | +| `POST /api/reload` | CLI, node-page | keep | +| `GET /api/node-settings` · `PUT /api/node-settings` | CLI `stun`, node-page | keep | +| `GET /api/chat/history` · `WS /ws/chat` | *nothing* | **triage — likely dead (§6b)** | +| `GET /` · `GET /audit` | browser only | **DELETE** | + +### 3c. CLI verbs + +`init reset status ui gek-init gek operator member group file video denylist stun +reload restart-daemon calibrate-argon2`. + +Only **`ui`** goes away. `status` output drops its `admin UI meshbay-node ui` +line. Nothing else changes — the CLI keeps talking to the same `/api/*`. + +--- + +## 4. Gap fills in `node-page.js` + +New sections/tabs, each a thin `nodeCall`: + +- **Peers** — `GET /api/peers` → table (user, IP, group, state). Poll ~5 s while visible. +- **Audit** — `GET /api/audit` with the event/user/limit filters the HTML page had. + Render every field as **text** (htm/preact escapes by default — never + `dangerouslySetInnerHTML` on `detail`/`username`; `detail` carries + member-chosen filenames, H2). +- **Maintenance** — `GET /api/index-cache` (count) + `POST /api/index-cache/prune`. +- **Roster (node-wide)** — call `GET /api/roster` with no `group_id`. Show every + identity and its devices; keep the existing `unpin`, add `revoke_member` per + member. (`revoke_device` for a single device is a later item — the endpoint + does not exist yet.) +- **Overview** — Node ID (`endpoint_hint`), QUIC port, hub URL, version, from + `GET /api/status`. + +Tabs: **Overview · Groups · Roster · Peers · Audit · Settings**. Pure UI +refactor of the current 749-line single scroll; no protocol change. Split into +per-tab files if it grows (same move as `docs/apps.md` did for the group UI) — +optional. + +--- + +## 5. Audit usernames + +**Symptom:** rows show `8f7e206e` (that is `user_id[:8]`, the fallback in the +HTML template's `e.username || e.user_id.slice(0,8)`). + +**Cause:** the `username` column exists and `_audit()` already passes +`username=self._username`, but it is blank for (a) legacy rows written before the +column was populated and (b) events where the handshake payload carried no +username (pre-handshake `auth_failed`, some token shapes). + +**Fix — resolve at read time, no migration:** in the audit endpoint (or a new +`ops.get_audit`), build `{user_id: username}` from the roster +(`roster.list_members()` + identities already expose `username` node-wide, same +source `_render_roster` uses) and fill blanks: +`row.username or roster_name.get(row.user_id) or row.user_id[:8]`. + +**Security:** this is **not** the stored-XSS concern. That concern is rendering +attacker-influenced strings (`username` from the hub, `detail` filenames) as +**markup**. Showing the real name is fine *as text*. The rule for the Preact +Audit tab: interpolate, never inject HTML. + +--- + +## 6. Cleanup + +### 6a. Dead code / config once the HTML is gone + +- `ui/app.py`: `@app.get("/")`, `@app.get("/audit")`, `_DASHBOARD_*`/`_AUDIT_HTML` + templates, `_render_dashboard`, `_render_node_settings`, `_render_roster` (HTML + variant), `_render_audit_page`, and any `escape`-only helpers left unused. +- `daemon.py`: the `ui` CLI verb + its `argparse` choice + help text + `quiet` + list entry; the `_ui_url`/`ui` handler block; `log.info("Admin UI ready — open + it with: meshbay-node ui")`; the `admin UI meshbay-node ui` line in + `_print_status`. +- `packaging/firewall/ufw/meshbay` and + `packaging/firewall/firewalld/meshbay-node.xml`: drop the `18000/tcp` service. + The UI binds `127.0.0.1` only (`daemon.py` `host="127.0.0.1"`), so these rules + never did anything useful and become a footgun if the bind address ever + changes. Remove regardless of the rest of this refactor. +- Docs/man: `docs/QUICKSTART.md:110`, `QE/deploy/README.md:99`, + `man/meshbay-node.1` (the `meshbay-node ui` entry), `packaging/deb/.../control` + and `packaging/rpm/meshbay-node.spec` ("a local admin UI on localhost:18000"). + +**Keep:** the uvicorn server, the token middleware, the CSP middleware (harmless +and good hygiene on JSON responses), the `ui-token` file, `ui_port` in +`node.toml`. Renaming `ui_*` → `localapi_*` is churn for no user value — deferred, +noted only so the naming mismatch is a known thing. + +### 6b. Orphan endpoints — ✅ removed (phase 6) + +`GET /api/config`, `GET /api/chat/history`, `WS /ws/chat` (plus +`broadcast_chat_to_ui` / `app.broadcast_chat` / `_chat_subscribers`) were a +never-wired local operator chat view plus a superseded config dump. Confirmed +dead against git history and removed. See phase 6 above. + +--- + +## 7. Security review + +The loopback API's model is unchanged and adequate: bind `127.0.0.1`; per-run +token (`os.urandom(18)`, `ui-token` mode 0600, required as `?t=` or +`X-MeshBay-Token`); CSP `default-src 'none'; connect-src 'self'`. Threats +modelled: DNS-rebinding from the operator's browser, and any local process. The +token closes both. + +- **Folding functions into `node-page.js` adds no surface** — every call goes + through the same token-gated API; the Electron main process holds the token, + the renderer never sees it (same model as `hub:fetch`). +- **Do not shell out to the CLI from the Node page.** The CLI is itself an HTTP + client of this API; spawning it would be a strictly worse `nodeCall` (process + spawn, arg-injection, PATH assumptions, unstructured errors). If a function is + missing from the API, add the endpoint. `main.js` shelling out to + `systemctl --user` is a legitimate exception — OS service control has no API + equivalent; node *operations* stay on the API. +- **Audit tab:** render as text (§5). +- **Unlink copy:** unlink clears only the hub's stored linking key + (`DELETE /v1/users/me/node_key` → `user.pk_node_ed25519 = None`). It does **not** + touch the roster, GEKs, pins, or on-disk group data — the node keeps serving + anyone who reaches it directly over MNP. The UI must say "the hub stops + brokering connections to this node", not imply a wipe. `reset` stays the + destroy-everything path. After unlink, `POST /v1/nodes/auth` fails and the hub + WebSocket drops — expected; show it as "offline from hub", not an error. + +--- + +## 8. Node-key linking — confirmed automatic (native) + +`create-group-page.js` `linkNodeKey(pk)` is called from `detectNode()` after +`platform.node.detect()` returns the node's `pk_node_ed25519`; it +`PUT /v1/users/me/node_key`. So in the desktop wizard the key links +automatically on node detection. The manual paste box in `profile-page.js` is the +fallback for the browser-only case (no bridge to read the node's key) and for +re-linking. No change needed; the **unlink** counterpart is the only missing half +(§2.2, §7). + +--- + +## 9. Draft correction + +`docs/meshbay-draft-v6.md` §2.11 and §2.12 describe the Node page as connecting +**over MNP** (`node_status_ack`, `node_settings_set`). The code went +loopback-HTTP instead (`node-page.js` header comment says so explicitly), which is +better for a native client — no signaling round-trip, no GEK/session, works +before any group is joined. The MNP handlers +(`webrtc_server.py:_do_node_status`, `_do_node_settings_set`) are dormant. + +Update §2.11/§2.12 to state: the Node page uses the node's loopback API; the MNP +`node_*` message types are reserved for a possible future browser-side (no +Electron) path and are not the mechanism today. + +--- + +## 10. Tests + +- `test_cli_dispatch.py` — drop the `["ui", ...]` row; assert `ui` is no longer a + parser choice. +- New `ops`/endpoint test for audit username backfill from the roster. +- Node-page: extend whatever source-reading test pins its structure (per the SPA + testing posture) to cover the new tabs' `nodeCall` paths. +- `test_packaging_units.py` / packaging tests — assert the firewall service files + no longer open 18000. +- Grep guard (or manual): no `meshbay-node ui` string left in `man/`, `docs/`, + `packaging/`. + +--- + +## 11. Phases + +1. **Draft + firewall** — ✅ done 2026-09-01. Corrected draft-v6 §2.11/§2.12 + (loopback API, not MNP) with a 2026-09-01 amendment line. Deleted + `packaging/firewall/firewalld/meshbay-node.xml`; stripped the `[MeshBay Node]` + stanza from `packaging/firewall/ufw/meshbay` (the `[MeshBay Cast]` profile + stays). Follow-on edits so the build still passes: + `packaging/build/build-node.sh` (drop the firewalld copy), + `packaging/rpm/meshbay-node.spec` `%files` (drop the `.xml` line), + `docs/PACKAGING-GUIDE.md` (drop the "Node admin UI / TCP 18000" section). No + behaviour change. The deb/spec "local admin UI on localhost:18000" description + lines are left for phase 5. +2. **node-page.js gap fills** — ✅ done 2026-09-01. Added five sections to the + existing single-scroll page (tabs come in phase 4): **Overview** (version, + node id, QUIC port, hub — from `GET /api/status`, fetched in + `fetchStatus`/`refresh`), **Node roster** (`GET /api/roster` with no + `group_id`, node-wide, with `unpin`), **Connected peers** (`GET /api/peers`), + **Audit log** (`GET /api/audit` with event + limit filters, every field + rendered as text), **Maintenance** (`GET /api/index-cache` + + `POST /api/index-cache/prune`). Backend: `GET /api/audit` now backfills a + blank `username` from `roster.list_identities()` at read time (no migration). + ~31 `node.*` i18n keys added to all ten locale catalogues (English + placeholders, matching the existing node-page convention). `test_locales.py` + green; `node --check` green; node ops/audit/status/roster tests green. + Per-member `revoke` in the node-wide view is deferred — revoke is + group-scoped and the node-wide list has no group context. +3. **Unlink** — ✅ done 2026-09-01. Added a danger-styled "Unlink from hub" + section at the bottom of `node-page.js` (`unlinkNode` → `DELETE /api/unlink`, + confirm + consequence copy per §7, hidden while `status === + 'waiting_for_node_key'`). Backend endpoint was already present and unchanged. + 5 `node.unlink_*` i18n keys added to all ten locales; `.node-group-danger` + added to `style.css`. `node --check` + `test_locales.py` green. Note: the + endpoint 503s when the node has no hub session — the catch surfaces the + bridge message, which is acceptable for that edge case. +4. **Tabs** — ✅ done 2026-09-01. `node-page.js` now has a `tab` state and an + `.admin-tabs` bar (reusing the existing admin-page tab styles) with six tabs: + **Overview** (node info + Maintenance), **Groups** (per-group cards), + **Roster** (node-wide roster), **Peers**, **Audit**, **Settings** (denylist + + node settings + STUN + ICE + Unlink). Header, service panel, action message + and the operator-pair banner stay above the bar, visible on every tab. Each + section is gated `${tab === '<x>' && …}`; the groups IIFE is + `${tab === 'groups' && (() => {…})()}`. 6 `node.tab_*` i18n keys added to all + ten locales. Verified with a per-tab render probe (htm parses and the + component executes cleanly for all six tabs) plus `node --check` and + `test_locales.py`. +5. **Remove HTML** — ✅ done 2026-09-01. + - `ui/app.py`: deleted `GET /`, `GET /audit`, and everything below the + `# ── HTML UI ──` marker (`_render_page` + inline dashboard template, + `_render_roster`, `_render_node_settings`, `_render_audit_page`, + `_AUDIT_HTML`, `_fmt_size`). ~500 lines gone. Dropped now-unused imports + (`base64`, `time`, `html.escape`, `HTMLResponse`). Module docstring rewritten + — it is a JSON control API, not a web UI. The uvicorn server, the token + middleware and the CSP middleware stay. + - `daemon.py`: removed the `ui` argparse choice + its help text + `quiet` + entry + the `if args.command == "ui":` handler. Startup log line is now + `Control API on 127.0.0.1:<port>`; `_print_status` drops the `admin UI` + line; the "waiting_for_node_key" hint now points at `meshbay-node status` + and the desktop client. `_daemon_api` docstring reworded. (Net effect on + ruff: one F541 fewer; the file's pre-existing lint debt is untouched.) + - Tests: `test_cli_dispatch.py` drops the `["ui"]` case and asserts `ui` is + no longer a parser choice. The two `_render_page` H2 tests + (`test_admin_ui_escapes_filenames`, `_roster_usernames`) are replaced by + `test_node_control_api_serves_no_html`, which pins the routes at 404 and + the render helpers as absent. Full node suite: 749 passed, same 2 + pre-existing failures (`test_packaging_units` bad split logic; + `test_webrtc_transport` aiortc flake). + - Docs/packaging: `docs/QUICKSTART.md`, `man/meshbay-node.1` (the `ui` verb + entry deleted, "admin UI" → "control API" throughout), + `packaging/deb/.../control`, `packaging/rpm/meshbay-node.spec`, + `devel-phases-next.md` (14.1b struck through), `docs/invite-pairing-v1.md` + (`_render_roster` pointer → `node-page.js`), `CLAUDE.md` key-modules row, + `config.py` example-config comment. `QE/deploy/README.md` also touched but + it is gitignored. +6. **Orphan-endpoint triage** — ✅ done 2026-09-01. All three confirmed dead and + removed from `ui/app.py`: + - `GET /api/config` — no consumer (the CLI, SPA and Electron never call it); + its payload is a strict subset of `/api/status` + `/api/groups` + + `/api/node-settings`. `test_security_regressions.py` used it only as a + sample path for the token-gate check → switched to `/api/groups`. + - `GET /api/chat/history` — reads `state["chat_store"]`, a **global** store + that the daemon deliberately never sets (per-group stores since finding + H1), so it always returned `{"messages": []}`; the message shape was stale + too. The real chat-history path is the MNP `chat_history` handler in + `webrtc_server.py` (unchanged, `test_transport_contracts.py` still green). + - `WS /ws/chat` + `_chat_subscribers` + `broadcast_chat_to_ui` + + `app.broadcast_chat` — added in the Phase-7 "Node v2" commit and never + wired: `git grep broadcast_chat` finds only the definition and the + assignment, nothing invokes it, so the socket's subscribers received + nothing. + - Fallout: dropped now-unused imports (`json`, `WebSocket`, + `WebSocketDisconnect`) and ran `ruff --fix` for the file's remaining + pre-existing import debt (`Path`, `DEFAULT_CONFIG_PATH`, `generate_gek`, + `wrap_gek_aes`, `ROLE_MEMBER`, `ROLE_OPERATOR`, the function-local + `Roster`). **`ui/app.py` now passes `ruff check` with no errors** (1074 → + 453 lines across phases 5–6). + - Tightened the `_security_headers` CSP to `default-src 'none'; frame-ancestors + 'none'; base-uri 'none'` — the JSON-only API has no HTML, inline script or + stylesheet to allow, so the old `'unsafe-inline'` / `connect-src 'self'` + grants were dead weight. No test asserted the string. Middleware docstrings + refreshed ("admin UI" → "control API"). + +4b. **Node page UI polish** — ✅ done 2026-09-01 (operator feedback on phase 4): + - Overview/Maintenance, Roster, Peers and Audit tabs **auto-load on open** — + the per-tab "Load" buttons are gone (`useEffect` on `[tab, status]`, audit + also on filter/page). Per-group roster and denylist keep their buttons. + - **Names, not ids**: `ui/app.py` gained `_display_names(state)` → + `(user_id→username from roster, group_id→name from node.toml)`. `/api/peers` + resolves both (`username`, new `group_name`); `/api/audit` adds + `group_name` and keeps the username backfill. The SPA renders the name and + falls back to a shortened id. + - **Audit pagination**: `get_entries` gained `offset`; `/api/audit` takes + `offset`, clamps `limit` to 1000, and fetches `limit+1` to report + `has_more` without a count. The tab has Previous/Next + "Page N", newest + first, page size 50/100/200/500. `test_audit.py::test_pagination_newest_first`. + - **Audit overflow**: table wrapped in `.node-table-scroll` (`overflow-x:auto`); + `.node-table-audit` keeps time/event/user/ip/group on one line (IP never + clipped) and wraps only the detail column. + - **CSV export**: `Export CSV` button → `exportAuditCsv` walks the whole log + for the current filter in 1000-row pages (de-duped by id so an event + written mid-export cannot duplicate a row; 1M-row hard stop) and + `saveCsv()` writes every matching entry — native Save As on the desktop + (`platform.nativeSave`), blob download link otherwise. RFC-4180 quoting. + - Settings tab section order: node settings → STUN → ICE → **denylist** → + **Unlink** (denylist sits directly above the danger zone). + - 6 new i18n keys (`node.loading`, `node.audit_export/_exporting/_prev/_next/_page`), + 4 dead `*_load` keys removed, across all ten locales. Per-tab render probe + + `node --check` + `test_locales.py` + node test subset (193) green. + +Phases 1–4 lose nothing and can land independently. Phase 5 is the one that +removes a capability (browser admin on a headless box) and should be its own +reviewed commit. + +--- + +## 12. Out of scope + +- Group-level settings UI (member-upload, apps, scan-settings) — stays in the + group UI. +- Renaming `ui_port` / `ui-token` / `create_ui_app`. +- `revoke_device` (single-device revocation) — needs a new endpoint; track + separately. +- Any MNP browser-side node admin path. |