diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/MESHBAY_DESIGN.md | 318 | ||||
| -rw-r--r-- | docs/MESHBAY_HTTP_API.md | 8 | ||||
| -rw-r--r-- | docs/MESHBAY_NODE_PROTOCOL.md | 7 | ||||
| -rw-r--r-- | docs/PACKAGING-GUIDE.md | 30 | ||||
| -rw-r--r-- | docs/USERGUIDE.md | 161 | ||||
| -rw-r--r-- | docs/WINDOWS-PORT.md | 6 | ||||
| -rw-r--r-- | docs/windows-build.md | 10 |
7 files changed, 518 insertions, 22 deletions
diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 18b23cf..8d14e37 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -16,7 +16,7 @@ > them — it names the invariant that holds today, not the incident that produced > it. §13 is the register of those labels. > -> Wire versions at the time of writing: **MNP 6.0** (oldest peer accepted 4.0), +> Wire versions at the time of writing: **MNP 6.1** (oldest peer accepted 4.0), > **MHP 0.1**, packages **0.19.0**. The normative source for the wire format is > `MESHBAY_NODE_PROTOCOL.md`; this document states the design the protocol > serves, not its byte layout. @@ -1693,6 +1693,17 @@ Five protections, and they are the substance: disk one capped file at a time; - the target root must be **writable and available**, enforced by the node. +**A full disk is a stated refusal, `disk_full`.** At chunk 0 the node compares the +announced size (`total_chunks` × the chunk's length, an upper bound within one +chunk) with the free space of the destination's filesystem, and refuses when the +upload would leave less than `DISK_RESERVE_BYTES` (1 GiB) — a disk filled to its +last byte breaks the node's own databases and the operator's system too. Two +uploads can pass that check together, so a write that fails with `ENOSPC` or +`EDQUOT` is refused the same way, and its `.part` and state are dropped: they +cannot be finished until the operator makes room, and keeping them holds the +space that ran out. The client carries the code on the error and translates it, +so a caller can stop rather than retry. + **There is no quarantine subdirectory.** A folder appearing beside the operator's library because somebody sent a file is the node deciding how their disk is arranged. What made a quarantine worth having was never the subdirectory — it is @@ -1838,7 +1849,10 @@ chat opens at the newest page. A forwards pager is not what a chat opens with. `meshbay-node chat prune <days>` deletes **messages only, never an epoch key**. An epoch with no messages is harmless; an epoch key deleted while messages still need -it is an unreadable archive. +it is an unreadable archive. The operator's purge (the signed `chat_purge`, from +the Chat settings) is the same rule with no age: every message goes, the epoch +keys and the attachments stay. The replay index goes with the rows, which costs +nothing, since a sealed message is taken only from the device that signed it. **A message is bounded in size and in rate, like every other member-supplied write.** Sending one costs the operator a row that nothing expires, every other @@ -1987,8 +2001,9 @@ by accident (§2.4). **Stores:** accounts (username, encrypted email, status, role), the group registry and membership, IP logs (one year, legal retention), node registrations, refresh -tokens, notifications, the moderation blocklist, instance policy, and per-account -device keys for hub login. +tokens, notifications, the moderation blocklist, instance policy, per-account +device keys for hub login, and the phones that asked to be notified — a poll +secret's hash and, for a phone with a push distributor, its endpoint (§11.3). **Does not store:** file content, file names, private-group indexes, message content, private keys, group keys, keypair bundles, user identity keys, node IPs @@ -3121,6 +3136,16 @@ An album browser over the image files in the group's configured folders. It is An **album is a directory**, exactly as a season is a folder in Videos. Thumbnails are node-side, orientation-corrected, and delivered through the same chunk path. +**A video in a photo folder belongs to its album**: a phone's clips, backed up +beside its photos (§9.12), are where they were on the phone. The node probes +and thumbnails them with the video enricher (duration and a frame, nothing +looked up anywhere: matching is the Videos app's, asked by its own pages). +The album card counts photos and videos apart, a clip's tile carries a play +mark and its length, and the lightbox shows its thumbnail with a play button +that hands it to the group's video player, so a clip of a gigabyte is streamed +like a film and never downloaded whole into the lightbox. A slideshow is of +photos: it passes a clip by. The television follows photos only. + **EXIF is read locally on the node** and narrowed on purpose: a capture time and a camera, and **never GPS**, anywhere, in any field a client receives. The claim this supports is precise, and the one it must not make matters more: @@ -3308,6 +3333,234 @@ are two albums. --- +### 9.12 Photo backup (Android) + +The Android application sends the photos taken on the phone to **one folder of +a group the member owns and is the only member of**, once a day. It is a client feature over the +upload path that exists: a backed-up photo is a `file_upload` into a writable +root under a slot the node granted, with every rule of §6.4, and its owner is +recorded as for any upload. **No message, no node state and no hub state was +added for it**; the node cannot tell a backed-up photo from one sent by hand, +and does not need to. + +**Split.** The phone (`photos/` in the Android package, reached as +`platform.photoSync`) lists the photos through MediaStore, keeps the ledger and +hands the bytes over. The page (`photo-sync.js`) decides when a run is due and +does the sending, because the transport and the group key are there. Bytes +cross by an opaque token the page fetches from its own origin, +`/photosync/<token>`, served by the shell's request interception: valid for the +run that issued it and for the one photo it names. The page never receives a +`content://` URI. A browser and the desktop have no camera roll and no such +object — absent, not refusing (§11.3). + +**Where it lives.** Its controls are on a page of their own, **Android Sync**, +under a **Phone** heading of the side menu (`android-sync-page.js`), shown only +where `platform.phoneSync` exists. Settings holds what the account prefers on +every client; this page holds what one phone sends: photos, contacts (§9.13), +and the kinds to come (messages, calendar, files). + +**The personal profile only.** Every backup is of the phone's personal +profile, never of a work profile: a copy of the application installed inside +a work profile is offered none of them (`Profiles.kt`; the bridge objects are +absent, so the page shows no Android Sync). In the personal profile each +source reads that profile alone, since MediaStore and the contacts, SMS and +calendar providers answer for the profile they are asked from; none of the +cross-profile `ENTERPRISE_*` URIs is used. An account added to the personal +profile is part of it, whatever its use. + +**One destination for every kind.** The top of the page chooses it once: a +group and one of its writable folders (`Destination.kt`, `sync-destination.js`). +Each kind goes into its own folder under it, `<folder>/<account>-photos`, +`<folder>/<account>-contacts`, made if missing. Only a group **the account owns +and is the only member of, with nobody invited**, is offered, because what a +group's folder holds every member can read, and none of this is the group's. +The hub's member list is asked again **at every run, before anything is +read**: a group that has gained a member, an invitation or another owner stops +every backup (`not_private`, a lasting refusal said once) rather than being +read from then on. Moving the destination is a fresh start for every kind: +each is told, forgets what it sent (its ledger belongs to the old place) and +runs at once; what was sent stays where it is. The page says so before it +happens, and says too when the group's Photos tab does not show +`<folder>/<account>-photos`, since photos there are kept but shown nowhere. + +**When.** A run is due 24 hours after the last run that *finished*; an +interrupted one is retried at the next chance, at most every fifteen minutes. +Due-ness is checked at start, on return to the foreground, when the network +turns unmetered, and by one timer at the due time — nothing polls. A scheduled +run needs an **unmetered** network (`NET_CAPABILITY_NOT_METERED`, not "Wi-Fi": +a phone on another phone's hotspot is on Wi-Fi and spending that phone's data). +"Back up now" runs whenever; on a metered network it first states the count and +size and asks — the one confirmation in this feature that is not about where +the photos go. Turning metered mid-run stops after the file in flight; the +node's partial upload resumes it (§8.5). + +**One photo at a time, under a slot.** Each upload asks for a transfer slot +like a member sending by hand and gives it back, so a family's daily backups +never occupy the node's upload pool ahead of a person. + +**Where.** Under `<folder>/<account>-photos` of the destination, each photo +goes under `YYYY/YYYY-MM` from when it was taken, because an album is a +directory (§9.9) and one folder of twenty thousand is a slow album. Albums on +the phone are chosen too; the camera alone is the default, because screenshots +and saved images are where photos nobody meant to share live. **By default the +photos already on the phone are sent, newest first**, then each new one; "from +now on" is an option. A confirmation is drawn **when the backup starts or its +starting point changes, never per run and never for a change of albums alone**: +it names the folder and the count and size about to go. + +**Additive by construction.** The backup never deletes, renames or replaces +anything on the node: `photo-sync.js` reaches no such operation +(`test_photo_sync.py` reads it for them). **The ledger is the memory of what was +sent, never a mirror of the phone**: per (account, group, folder), each photo's +MediaStore id, modification date, size, the SHA-256 of the bytes sent and the +name the node's ack gave. A run sends what the ledger lacks and never compares +the other way, so a photo deleted on the phone stays in the group, and one +deleted on the node is not sent again. An empty ledger (a reinstall) is +reconciled against the folder by name and size before anything is sent. + +**Edits.** A photo edited in place keeps its id and changes its modification +date; when the size also changed, or the bytes no longer hash to what was sent, +the edit is sent **beside the original**, as `<stem>-edited-<YYYYMMDD-HHMMSS>.<ext>` +from the edit's date. A touch that left the bytes alone sends nothing. "Save as +copy" makes a new photo and needs no rule. + +**Videos.** Off by default, ticked in the same section: the videos of the +chosen albums (a MediaStore bucket holds both, under one id) go beside the +photos, in the same `YYYY/YYYY-MM` folders, under the same ledger, rules and +unmetered network. `READ_MEDIA_VIDEO` is asked for when they are ticked in. +The confirmation states the videos' count and size apart, because they are +most of it. A file is never whole in the page: the upload reads it a chunk at +a time and each chunk is a ranged request to the phone (`?range=a-b` in the +address, served by `ByteRange.kt`; not a `Range` header, which the WebView +does not hand to the application that answers, found on a Pixel 9), so a +video of gigabytes costs the WebView one chunk of memory; the SHA-256 the ledger keeps is then read from the phone once +the node has the file. The Photos tab shows them in their albums (§9.9). + +**A manifest, for a restore or a merge later.** The node keeps the files; only +the phone knew where each came from. Every file the node takes adds a line to +a log the phone keeps (`ManifestLog.kt`): its path on the node, its path and +album on the phone (`DCIM/Camera/PXL_….jpg`, `Camera`), when it was taken and +last modified, its size, MIME type and the SHA-256 of the bytes sent. A run +that sent anything ends by uploading the waiting lines as +`<kind folder>/meshbay-manifest/manifest-<date>.jsonl`; the phone forgets them +only once the node has that file, so a run cut short sends them with the next +one. Restoring puts each photo back into its album and each file back under +its own name and date; merging onto a phone that already has some of them +compares hashes without downloading anything. Photos, videos and files have +one; contacts, calendars and messages are each one file that describes +itself. What the manifest cannot bring back is a photo's location, removed +before it left the phone. + +**Location.** The application's Android manifest does not ask for +`ACCESS_MEDIA_LOCATION`, so a photo or video read through MediaStore has its +location **redacted by the platform** (Android 10+): a photo's EXIF, and in an MP4 or MOV the location +boxes MediaProvider finds (`IsoInterface`). A camera roll going to a group +does not say where its owner lives, and nobody had to do anything for it. On +Android 8 and 9 nothing redacts it. + +**Refusals.** A refusal that will hold tomorrow — `disk_full` (§6.4), a folder +no longer writable, gone or on a drive that is not plugged, the member removed +from the group, the photo permission withdrawn — stops the run, is said once in +a notification and on the Android Sync page, and is retried a day later rather +than at every opening. Anything else (the node offline, the network gone) is an interruption +and is retried. + +**Screen off.** The sending lives in the page, so a run holds a `dataSync` +foreground service (`BackupService`) and the WebView reported visible, like a +cast (§11.3). Android 15 limits that type to six hours a day; the service stops +when told and the run carries on at the next opening. + +### 9.13 Contacts, calendar, messages, files and WhatsApp backup (Android) + +The Android application sends a copy of the phone's **contacts**, its +**calendars** and its **text messages** to a folder of a group, on the photo backup's terms (§9.12): a client +feature over the upload path, additive, once a day, the page doing the sending +and the phone handing one file over by token (`/phonesync/<token>`, +`phonesync/` in the Android package, `phone-sync.js` on the page). Each kind is +a `DocSource` that writes what is new since a marker; the marker moves only on +the node's acknowledgement, so a file cut short is written again whole. + +**Where.** `<folder>/<account>-contacts/` of the destination every kind shares +(§9.12), checked at every run the same way. Each copy is a new +file named for when it was taken (`contacts-2026-10-10-0900.vcf`), never a +replacement: restoring is importing the newest one, and an older one is there +if a phone sync went wrong. + +**Contacts.** The whole address book as the platform exports it, one vCard per +contact through `ContactsContract.Contacts.CONTENT_VCARD_URI`, the format any +phone imports. It is sent only when its SHA-256 differs from the last copy the +node took; a day with no change sends nothing. + +**Calendar.** `<folder>/<account>-calendar/calendar-2026-10-10-0900.ics`: one +iCalendar file (RFC 5545, `Ics.kt`) of every calendar the person can write to +(`CALENDAR_ACCESS_LEVEL` contributor or above); the ones they only subscribe +to, public holidays or birthdays from contacts, are someone else's and come +back by subscribing again. Timed events keep their Olson zone (`TZID`), so a +weekly 9:00 stays 9:00 across summer time; recurrences keep their RRULE, +RDATE and EXDATE, Android's `P3600S` durations become RFC's `PT3600S`, and a +changed instance carries its series' UID and a RECURRENCE-ID. Reminders, +organizer and attendees come along; each event names its calendar in +`X-MESHBAY-CALENDAR`, which importers ignore. Like contacts it is sent when +its SHA-256, taken without the DTSTAMP lines, differs from the last copy. + +**Messages.** `<folder>/<account>-messages/YYYY/sms-2026-10-10-0900.xml`: the +SMS added since the last file the node took (the marker is the highest +provider `_id` sent; ids only grow, so a message restored onto the phone later +is sent too), the first file holding the whole history. The format is the +`<smses>` XML that SMS backup applications restore, so a copy goes back onto a +phone with tools that exist; `SmsXml.kt` writes it, keeping line breaks as +character references and leaving out the characters XML 1.0 cannot carry. +A contact name is filled in only when the person also allowed contacts. MMS +are not included. + +**Files.** `<folder>/<account>-drive/<chosen folder>/…`: the files of the +folders the person chose through the system's picker +(`ACTION_OPEN_DOCUMENT_TREE`, a persisted read grant per folder, no storage +permission, which Play keeps for a few kinds of application). It runs as +photos do, and is in fact the same runner (`createBackup` in `photo-sync.js`, +`drive-sync.js`): once a day, on an unmetered network, a file at a time under +a slot, read from the phone a range at a time (`/drivesync/<token>`), a ledger +by document id (`DriveLedger.kt`), additive. A file changed on the phone goes +beside the copy sent, named `-modified-<date>`; nothing is replaced. Nothing +is stored twice: `DCIM`, `Pictures` and `Movies` (or anything in them) and a +whole volume cannot be chosen (`DrivePlan.refusal`), and a file the photo +backup sends, wherever its album is, is left out by its path. Android 11 and +later do not let an application choose the whole of `Download`, only folders +in it; the section says so. Hidden files (`.x`) are skipped; a name the node +would refuse is mended (`DrivePlan.safeName`), never dropped. + +**WhatsApp.** `<folder>/<account>-whatsapp/`: WhatsApp's own folder, +`Android/media/com.whatsapp/WhatsApp`, chosen once through the same picker, +opened on it (`EXTRA_INITIAL_URI`; Android 17 lets it be chosen, checked on a +Pixel 9); any other folder is refused. The same class runs it as the files +backup (`DriveChannels` with `DriveKind.WHATSAPP`), the same runner sends it. +The chats are only ever in WhatsApp's own backups, encrypted with a key only +it (or the person's end-to-end backup password) opens: nothing here can read +them, a restore can use them. WhatsApp writes a full copy weekly +(`Databases/msgstore.db.crypt14`) and an increment a night +(`msgstore-increment-N.db.crypt14`), and renames last week's set with its +date when a new week starts; those dated copies are earlier versions this +backup already kept, so they are not sent again. `Backups/` (contacts, +settings, stickers) goes too. `Media/` only if the person ticks it, as it is +often gigabytes; what the photo backup sends is left out. Because files are +replaced in place, a restore must take one week's copy with that week's +increments: once anything in that set was sent, the manifest gets a +`restore-set` line naming, for each file of the set as it is now, the copy +on the node that goes with it. + +**Two builds.** Play's policy gives `READ_SMS` to the default SMS application +only, so the APK has a `store` dimension: `full`, distributed directly, and +`play`, which has neither the permission (`src/full/AndroidManifest.xml`) nor +the code that reads messages (`SmsSource.kt` is in `src/full`; `Flavor.kt` +answers null in `src/play`). The bridge offers `messageSync` only where the +shell has a message source (`MESSAGES` in its prelude), so the Play build shows +no Messages section rather than one that refuses. + +**Unlike photos**, these run on any network: a contacts or calendar file is small, and a +history of text messages is megabytes, not the gigabytes of a camera roll. +`READ_CONTACTS`, `READ_CALENDAR` and `READ_SMS` are asked for when the person +turns each on, never at start. + ## 10. Filesystem portability **exFAT and NTFS on Windows are the common case, not an edge case.** Most users are @@ -3442,11 +3695,48 @@ narrow bridge — and Android has all three: locks and the renderer's priority policy do not prevent it. A cast's pipeline lives in the page (WebRTC → decrypt → relay), so while one runs the shell keeps the WebView reported visible and holds a media-playback foreground - service; nowhere else, because a page never hidden is never throttled. + service. A photo backup does the same with a `dataSync` service of its own + (§9.12); nowhere else, because a page never hidden is never throttled. +- **Notifications reach a closed application with nothing to install, and + never through a vendor push service.** Android lets nothing hold a connection + for an application that is not running, so there are three ways to wake a + phone: a vendor push service (Google sees who is notified and when; not + F-Droid), a push distributor the owner installs, or the phone asking. **The + phone asks by default**, and uses a distributor when one is already there. + Turned on, the page registers the phone (`POST /v1/push/subscriptions`) and + gets a row id and a **poll secret**; a system job (`JobScheduler`, no library) + then fetches what is new with `POST /v1/push/poll` every fifteen minutes, + which Android stretches under Doze. The secret is the only credential the + background holds and it reads notification lines and nothing else — not a + session, so nothing renewing in the background can collide with the page's + rotating refresh token, and a sign-out, which deletes the row, ends it. **When + a UnifiedPush distributor is already installed** (ntfy, or an application + carrying one), the row also gets its endpoint and P-256 key and each + notification is sent there at once as one RFC 8291 record, so the push server + relays bytes it cannot read and learns only *when* — the metadata §7.1 already + concedes to the hub; the fetch then runs every four hours as a net under it. A + distributor that refuses, disappears or answers 404/410 is not an error: the + row loses its endpoint and the phone fetches again. Both paths carry the same + payload — the hub's own row (kind, title, group, a `#/` route, its date), + never a message, which the hub does not hold — and a line pushed then fetched + is drawn once. **Nothing reaches a phone that was not created**, and + `create_notification` creates nothing for a muted group *or for an account + that turned every notification off* — the second switch used to be read only + by the interface, which hid rows the hub went on writing, and a switch only a + renderer honours is no switch once a phone is told about every row. A push + endpoint is a URL a member chose and the hub fetches it, so a send resolves + it, refuses any non-public address and connects to the address it checked + (SNI and Host carry the name); no redirect is followed. The shell draws a + pushed message only if the connector decrypted it with the phone's key, and a + notification's link is a route in the page, applied as `location.hash`, never + loaded. No VAPID yet: a distributor that requires it refuses, and the phone + fetches instead. -Release builds are signed with the development key until the release key exists -(Stage D12); §2.3's sentence about who holds a signing key applies to whichever -store distributes them. +Release builds are signed with the release key, distributed as a direct APK; +the key stays outside the repository and a release build without it fails +rather than falling back to the development key. §2.3's sentence about who +holds a signing key applies to whichever store distributes them, if one ever +does. ### 11.4 Casting @@ -3804,6 +4094,7 @@ had already been asked. | **AV30** | **What one member's offers cost a node is bounded per account and per node, and the bound admits the heaviest ordinary account** (§7.2). Each offer makes the node allocate a peer connection. A budget of 120 per node refilled at two a second bounds a member there without touching their other nodes, and it is counted by account because a mobile carrier shares one IPv4 address among many subscribers. Pending offers are capped at 32 per account. Both refusals carry `Retry-After` and the client retries them, because a refused offer otherwise reads as a node that is down. An offer carries at most 64 ICE candidates (32 KiB), and its IP-log row — kept a year — is written only once it goes to a node, so naming nodes that do not exist costs the hub nothing. A node's `update_groups`, a database read each, is budgeted like `chat_notify` (ten a minute) and claims at most 1000 groups | | **AV32** | **A node hosts a group because its owner said so, not because its account belongs to it** (§7.2). Every member holds the key, so a member's node passes the handshake like the real host and could be the one a client keeps. A node may claim the groups its account owns and those whose owner approved it; any other claim is a pending request the owner sees | | **AV33** | **Nobody is made a member without saying yes** (§7.3). A membership makes the account's client list the group, name it in its tokens and dial its nodes, so an owner's addition is an invitation until the invitee accepts it. The MNP token names only the group it is minted for | +| **AV34** | **What a member's chat costs other members' phones is bounded twice, and what a phone's fetching costs the hub once** (§11.3). A pushed notification is one outbound request per subscription of the recipient, so the fan-out of one chat line is members × phones. It is bounded where it starts (`chat_notify`, ten a minute per node), a conversation reaches each phone at most once per 30 s (the phone shows one line per group, so the pushes in between would only replace it), and an account holds ten subscriptions at most. A subscription the push server reports gone (404/410) loses its endpoint rather than being retried for ever. A phone fetching instead costs one indexed read per poll, refused below a minute per row, and returns at most twenty lines | | **AV31** | **What waits for a signature is bounded** (§5.4). Any authenticated member can ask for an admin challenge, since the signature is checked afterwards, and a pending challenge kept its whole request until answered — measured, 200 requests of 1 MiB held 400 MiB for the life of one connection. At most eight pending per connection, 64 KiB each, expired ones dropped | ### 13.6 Chat design findings @@ -3996,10 +4287,11 @@ seeking, audio-language and subtitle selection, transfer leases with queueing, pause and resume, the group-application framework with Chat, Files, Videos, Music and Photos, cross-group search with source merging, per-account playlists, casting to a Chromecast with subtitles rebased onto the relay's clock, the +phone's photo backup (§9.12), the operator CLI and loopback control API, the desktop client through its identity and download stages, account recovery, the Windows port through packaging, and -the Android client — the shell, native keys, downloads and uploads, and casting -(§11.3). +the Android client — the shell, native keys, downloads and uploads, casting, +and notifications while it is closed, fetched or pushed (§11.3). The packages install: a machine has been taken from the built artefacts to a running hub and node on **Ubuntu 26.04 (`.deb`), Fedora 44 (`.rpm`) and @@ -4027,7 +4319,9 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | — | **Bitmap subtitles** (PGS, VOBSUB — about a fifth of the embedded streams). No WebVTT without OCR; they are not listed rather than listed and blank. Burn-in covers them and costs `-c:v copy`, which is what the eight-slot sizing assumes never happens | | — | Delegation (§3.4) | | — | Tier 3 roster attestation (§3.3) | -| — | **Android: phone behaviour and release** — the back button driving the page, recovery from a network handover, keeping a download alive with the screen off, lock-screen media controls, and a release key (§11.3) | +| — | **HEIC/HEIF in Photos.** The indexer does not classify `.heic`/`.heif` as images and no browser but Safari draws them, so a phone that shoots HEIC has its backed-up photos stored and listed in Files but absent from Photos. Needs `pillow-heif` in every package and a JPEG rendition for the viewer (§9.12) | +| — | **Photo backup with the application closed**, and **"remove what this phone sent"** (§9.12). A closed application has no page and so no transport; it runs at the next opening | +| — | **Android: phone behaviour and release** — the back button driving the page, recovery from a network handover, keeping a download alive with the screen off, lock-screen media controls, and an update channel (§11.3) | | — | **Federation between two hubs.** The protocol is written and switched off in the code (§7.6); what is not built is one run between two machines | ### 15.3 Open, and why each is where it is @@ -4038,6 +4332,8 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | **The desktop cast relay takes the first chunk for the whole header** | The node's first chunk can be the ftyp alone, the moov in the next; the receiver then gives up. Fixed in the Android relay (§11.4), not in `cast-relay.js` | | **A cast restart may pull far ahead** | Seen once on an emulator with a synthetic film: after the restart's reinit the node reported `duration=None`, and the page pulled most of the film at network speed. Not reproduced on a real film; the suspicion is a read-ahead budget computed without a duration | | **"Copy stream URL" after picking a receiver casts to that receiver** | The player keeps the last device chosen, so the copy-only path reconnects it instead of only starting the relay | +| **A folder renamed on the node's disk empties until the next reconciliation** | Seen live (2026-10-10, ntfs3): renaming a 2,441-file folder inside a root made the watcher drop every entry under the old path and announce none under the new one, so the group read as empty, and the loop was busy long enough for the hub's keepalive to time out. A restart (full scan) brought them back. A directory moved within a root should rescan its new subtree at once, not wait for the periodic scan | +| **A photo deleted from the group comes back after a reinstall** | The backup's memory of what it sent is the phone's ledger (§9.12); a reinstall or cleared data empties it, and the folder no longer holds the deleted photo, so it is sent again. Clean fix: the node keeps a tombstone per folder on `file_delete` (name, size, SHA-256 taken before deleting), the index carries them, and the backup records a match as sent. Not done: a protocol field and a node table for a rare case | | **The home page says "No groups yet" when the hub cannot be reached** | An unreachable hub reads as an account with no groups, rather than as an error | | **`meshbay-node init` says "Settings → Link Node"** | The control is on the Profile page, as QUICKSTART says | | **C4** for accounts with browser access | Closed against operators by the pepper; **open against an active hub**, which holds the pepper and can fetch a bundle with a token it mints — the adversary T3 already concedes for browsers (§3.7) | diff --git a/docs/MESHBAY_HTTP_API.md b/docs/MESHBAY_HTTP_API.md index 74b9d77..1bcb55d 100644 --- a/docs/MESHBAY_HTTP_API.md +++ b/docs/MESHBAY_HTTP_API.md @@ -180,6 +180,14 @@ hub also serves the web application at `/` and `/app/`, which are not listed. | DELETE | `/v1/notifications` | user or node | Throw them all away. | | POST | `/v1/notifications/read-all` | user or node | Dismiss every one — the same thing as `DELETE ""`, under the name an older client knows it by. | +### Push + +| Method | Path | Auth | What | +|---|---|---|---| +| POST | `/v1/push/subscriptions` | user | Register this phone, or update its row: with an endpoint and keys when it has a push distributor, without them when it will fetch instead. | +| POST | `/v1/push/poll` | none | What is new for this phone since `since`: the same payloads a push carries, oldest first, at most twenty. | +| DELETE | `/v1/push/subscriptions/{subscription_id}` | user | Stop telling one phone anything: turned off there, or signed out of. | + ## Node control API `http://127.0.0.1:<ui_port>`, port 18000 unless `ui_port` in `node.toml` says diff --git a/docs/MESHBAY_NODE_PROTOCOL.md b/docs/MESHBAY_NODE_PROTOCOL.md index 0045f78..1355aaa 100644 --- a/docs/MESHBAY_NODE_PROTOCOL.md +++ b/docs/MESHBAY_NODE_PROTOCOL.md @@ -1173,6 +1173,7 @@ broadcast, every connected peer in the group learns the change without reconnect | `chat_link_preview` | `on\|off` | operator | `chat_link_preview_ack{enabled}` | yes | | `search_listed` | `on\|off` | operator | `search_listed_ack{listed}` | yes | | `chat_epoch` | `group_id` | operator | `chat_epoch_ack{epoch}` | yes | +| `chat_purge` | `group_id`, always the connection's group | operator | `chat_purge_ack{removed}` | yes: every open chat panel empties | **Upload policy is not in this table**, and that is the design: whether a member may write is a property of each root (its `writable` flag), not a switch over the group. A single @@ -2126,6 +2127,7 @@ it back (§3.5). | `client_diag` | C→N | auth | the video player's own view of a stream, written to the node's log beside its own (a stream event at INFO, the periodic state at DEBUG); the node acts on none of it and sends no reply | | `member_revoke` / `_ack` | C→N / N→C | signed | stop serving the key to someone | | `chat_epoch` / `_ack` | C→N / N⇒C | signed | open a new chat epoch by hand | +| `chat_purge` / `_ack` | C→N / N⇒C | signed | delete the group's stored chat; epoch keys and attachments stay (6.1) | | `app_directories` / `_ack` | C→N / N⇒C | signed | one application's folders, keyed by app name | | `chat_directory` / `_ack` | C→N / N⇒C | signed | where chat attachments are written | | `chat_link_preview` / `_ack` | C→N / N⇒C | signed | whether the node unfurls posted links | @@ -2175,7 +2177,7 @@ message: ## 13. Versioning and compatibility -MNP versions independently of the package version. Current: **`6.0`**; oldest peer +MNP versions independently of the package version. Current: **`6.1`**; oldest peer accepted: **`4.0`**. 4.0 is the floor: a member presents a short-lived node-audience token bound to one node @@ -2198,6 +2200,9 @@ ten operator messages no client ever sent (§10.4, "The node's own controls"). T also refuses to sign them, so a node older than 6.0 cannot be driven into them by a script in its page either. +6.1 adds the signed `chat_purge` (§10.4), additively: a 6.0 node gives no +answer, as for any unknown type, and nothing else changes. + The two numbers are separate on purpose. `MNP_VERSION` says what this build speaks; `MNP_MIN_SUPPORTED` says what it will talk to, and moving the second is a decision about whether an older peer can still do anything useful: diff --git a/docs/PACKAGING-GUIDE.md b/docs/PACKAGING-GUIDE.md index fdc0a57..9f5d6cf 100644 --- a/docs/PACKAGING-GUIDE.md +++ b/docs/PACKAGING-GUIDE.md @@ -102,10 +102,13 @@ Two cases where the app does not do it on its own, because it would undo something: - the node was already set up for **another account or another hub** — the - **Start** button on the Node page asks before switching it; + Node page says so, and **Use this node for my account** asks before + switching it; - your account is already linked to a node on **another machine** — linking - this one would disconnect that one. The Node page shows this node's key; - **Profile → Link Node** moves the link here if that is what you want. + this one would disconnect that one. The Node page says so, and **Link this + node instead** asks before moving the link here. **Profile → Unlink** lets + go of the other machine's node from anywhere, which is the way out when that + machine is gone. The Node page's **Start / Stop / Restart** work in every mode, and its status is the node's own: **Running** when it answers, **Stopped** only once no @@ -127,6 +130,27 @@ firewall rules and the boot-time service task with one administrator confirmation (none if neither is there). None of this touches `%LOCALAPPDATA%\meshbay\` (the keystore). +### Microsoft Store version + +The same client and node, installed from the Microsoft Store. It never asks +for administrator rights, and Windows itself manages what the installer above +sets up with scripts: + +- **Firewall**: the rules for the node are part of the package. Windows adds + them at install, keeps them through updates and removes them with the app. +- **When the node runs**: **Only while MeshBay is open** (the default) or + **At sign-in**, chosen on the **Node** page. At sign-in is the + *MeshBay Node* entry of **Settings → Apps → Startup**; switching it off + there is the same as choosing **Only while MeshBay is open**, and the Node + page cannot switch it back on until it is on there again. +- **At boot, before anyone signs in, is not available**: it needs the + `.exe` installer above. +- **`meshbay-node` in a terminal** works as with the installer. + +Runtime data lives in the same `%LOCALAPPDATA%\meshbay\` and survives +uninstalling the app. Uninstall from **Settings → Apps**; it removes the +firewall rules and the sign-in entry with it. + ### Build from source See [`packaging/win/README.md`](../packaging/win/README.md). On a machine with diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index 2c37f56..bc16ba6 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -323,6 +323,9 @@ previews. off. - **History is kept on the node**, and stays readable to members. A member removed from the group cannot read anything written after their removal. +- The operator can **purge the chat** from the group's Settings, Chat + section: every message goes, for everyone, and cannot be brought back. + Attachments stay in their folder. - Sometimes a message reads *"Written before this device could read this conversation"* — that is a device added later, not an error. A message marked *"the signature does not match the sender"* is different and worth asking @@ -369,12 +372,135 @@ is a folder**. Thumbnails come from the node, already rotated correctly. A photo opens full size, with a slideshow button that moves on every five seconds and stops at the album's last photo. +Videos in an album's folder, such as a phone's clips, show among its photos +with a play mark and their length; open one and press play to watch it. A +slideshow skips them. + +A photo you uploaded can be deleted: right-click it, or use the bin in the bar +above an open photo (the way in on a phone). The node's operator can delete any +photo. Both ask first. + **Location data is never shown.** Photos shows when a picture was taken and what took it, and no coordinates anywhere. Worth knowing, though: the coordinates are still inside the photo itself, as your phone wrote them, so anyone who downloads the original file has them. Strip them before sharing if that matters to you. +### Backing up your phone + +On the Android application, **Phone → Android Sync** in the side menu backs up +your phone's photos, contacts, calendar, text messages and the files of +folders you choose to a group. + +Only your **personal profile** is backed up. If your phone has a work profile, +nothing in it is sent, and MeshBay installed inside the work profile offers no +backup at all. + +**Where backups go** is chosen once, at the top of the page, for every kind: + +- **Only a group you own and are the only member of** can receive them: your + photos and your address book are not the group's. If you have none, create a + group just for yourself first. If somebody joins it or is invited later, + every backup stops and tells you. +- You choose any folder of that group you may add to. Each kind of backup gets + a folder named after you inside it: `Backups/alice-photos`, + `Backups/alice-contacts`. The page tells you if the group's Photos tab does + not show the photos folder; add it in the group settings to see them there. +- Choosing another group or folder later starts every backup again in the new + place. What was already sent stays where it is. + +#### Photos + +- **Set up** asks for access to your photos, then for the albums to send. The + camera is ticked; screenshots and other albums are yours to add. +- **Videos too** adds the videos of the same albums, beside the photos. It is + off by default: videos are large, and the first backup can take a while. + They are sent on Wi-Fi like photos, and show in their albums in the Photos + tab. +- By default **the photos already on the phone are sent too**, newest first, + then each new one. Choose "Only photos taken from now on" to skip the ones + already there. Before the first send you are told how many photos and how + much that is. +- It runs **on Wi-Fi only** (more exactly, on a connection that is not charged + by the amount), once a day, while the application is open, with the screen + off too. If the application was not opened all day, it runs the next time it + is. **Back up now** runs at once; on mobile data it tells you how much it is + about to send and asks first. +- Photos go into folders by year, then year and month (`2026/2026-10`). **A + photo you delete from the phone stays in the group**, and one deleted from + the group is not sent again, unless the application is reinstalled. A photo + you edit on the phone is sent again beside the original, with `-edited-` and + the date in its name. +- **Where a photo or video was taken is removed** from the copy that is sent + (Android 10 and later). + +#### Contacts + +- **Turn on** asks for access to your contacts, then sends a copy once a day + when they have changed. +- Each copy is a `.vcf` file with the date in its name + (`contacts-2026-10-10-0900.vcf`). Any phone can import it. Older copies are + kept, nothing is ever replaced. +- It runs on any connection, since a copy is small. **Back up now** sends one + at once if your contacts changed since the last copy. + +#### Calendar + +- **Turn on** asks for access to your calendars, then sends a copy once a day + when they have changed. +- It holds every calendar you can edit: your own, local ones and those of the + accounts on the phone. Calendars you only subscribe to (public holidays, + birthdays) are left out. +- Each copy is an `.ics` file with the date in its name + (`calendar-2026-10-10-0900.ics`). Any calendar application can import it; + repeating events, their exceptions and reminders come along. + +#### Text messages + +- **Turn on** asks for access to your text messages. The first copy holds them + all; then, once a day, a new file holds only the ones that arrived or were + sent since. +- The files go by year (`alice-messages/2026/sms-2026-10-10-0900.xml`), in the + XML format SMS backup applications can restore onto a phone. Senders are + named when contacts backup is also allowed. Multimedia messages (MMS) are + not included. +- The version of the application from the Play Store has no text messages + backup: Play allows reading them only to the phone's messaging application. + +#### Files + +- **Add a folder** opens Android's folder picker: choose `Documents`, a folder + inside `Download`, or any folder of your own. Android does not let an + application take the whole of `Download`. +- `DCIM`, `Pictures` and `Movies` cannot be chosen: photos and videos are the + photo backup's, and a file it already sends is never sent twice. +- The files go into `alice-drive/<folder>/`, sub-folders kept, on Wi-Fi once a + day like photos. A file you change on the phone is sent again beside the + earlier copy, with `-modified-` and the date in its name; nothing is ever + replaced, and a file you delete on the phone stays in the group. + +Beside your photos and files, a `meshbay-manifest` folder holds a small file +per backup run that records where each one came from on the phone (its +album or folder, its original name and dates). It is what will let a restore +put everything back where it was. Leave it in place. + +#### WhatsApp + +- **Choose WhatsApp's folder** opens the picker on it + (`Android/media/com.whatsapp/WhatsApp`): press "Use this folder". +- What is copied is WhatsApp's own nightly backup of your chats, still + encrypted by WhatsApp: MeshBay cannot read it, but you can restore it. Turn + on **Media too** to also copy photos, videos, voice notes and documents + (often several gigabytes). +- To restore on a new phone, copy the `Databases` and `Backups` folders of the + latest copy into `Android/media/com.whatsapp/WhatsApp` before installing + WhatsApp, then restore with the same phone number (and your password or + 64-digit key if end-to-end encrypted backup is on). + +If a backup cannot go on (the node's disk is full, the folder no longer accepts +files, somebody else joined the group), it stops, says why once in a +notification and on the Android Sync page, and tries again the next day. + ### Search The magnifying glass in the sidebar searches **across every group you are in** @@ -443,10 +569,23 @@ designed and not built. ### Notifications and settings worth knowing +The bell lists what happened while you were away. An open MeshBay looks again +when you come back to it — to the window, or to the home page. + **Settings → Downloads** — save automatically to a folder, or ask every time. **Settings → Defaults** — which tab a group opens on, how many items per page. **Settings → Appearance** — theme and language (ten languages ship). **Group menu → Mute** — stop notifications for one group. +**Settings → Groups → Disable all notifications** — none are kept for you at +all, and none reach a phone. +**Settings → Groups → Notifications on this phone** (Android application) — +be told about new messages and invitations while MeshBay is closed. Nothing to +install: the phone checks about every fifteen minutes (Android may wait longer +when the phone is asleep). If you want them to arrive at once, install a +UnifiedPush *distributor* app such as **ntfy** (Play Store or F-Droid) — optional; +MeshBay then uses it by itself, encrypted so the distributor cannot read what it +carries. Muting a group, or disabling all notifications, stops what reaches the +phone too. A busy conversation is one line per group. --- @@ -835,6 +974,9 @@ anything missing. **"No nodes are currently online for this group."** The node is not running, is not reachable, or has not linked its key to the hub. On the node: `meshbay-node status`, then Step 4 and 5 of the quickstart. +In the desktop app, the **Node** page says when your account is linked to the +node of another machine (**Link this node instead**); **Profile → Unlink** +lets go of a node from anywhere, for a machine that is gone. **A member sees the group but no files.** They never redeemed an invitation code, or the group has no key @@ -1006,12 +1148,20 @@ Better to know now than to go looking for it: distribution. Until that ships, take them from the download page and from nowhere else. - **The Android application is not released yet.** It works — groups, chat, - films, downloads, casting — but is built and installed by hand and signed - with a development key, so there is no store page and no update channel. The - back button and the lock screen do not drive it yet. Music keeps playing + films, downloads, casting — and is signed with the release key, but is + installed by hand from an APK: there is no store page and no update channel. The + back button and the lock screen do not drive it yet. Notifications arrive within about + fifteen minutes, or at once with a UnifiedPush distributor app such as ntfy + — except one that demands a server key (VAPID), not supported yet. Music keeps playing with the screen off, from one track to the next, with a notification shown while it plays. A phone browser works too. - **A node cannot be hosted on Android**, and is not planned to be. +- **Photo backup only runs while the application is open** (the screen may be + off). There is no button yet to remove everything the phone sent to a group + in one go — delete those files from the group instead. +- **Photos in the HEIC format are backed up but do not show in Photos** yet: + they are in the folder (Files lists them) and can be downloaded. Most Android + phones take JPEG unless set otherwise. - **Hubs do not talk to each other yet.** Everyone in a group needs an account on the same hub. - **Casting reaches Chromecast devices** from the desktop and Android @@ -1022,8 +1172,9 @@ Better to know now than to go looking for it: text recognition. - **There is no overall storage quota.** Files are capped at 8 GB each — lower or raise it on the node, above — but somebody can still fill a disk one file - at a time. Worth a glance now and then if you have opened a folder to people - you do not know well. + at a time, up to the last gigabyte: the node always keeps 1 GB free, and + refuses uploads beyond that with "The node's disk is full". Worth a glance + now and then if you have opened a folder to people you do not know well. - **Searching the film database by hand has no per-member limit**, and it draws on the node's own quota. Only likely to matter on a large group. - **The record of who uploaded what belongs to the node.** It is kept and it is diff --git a/docs/WINDOWS-PORT.md b/docs/WINDOWS-PORT.md index 1894706..7e13aa0 100644 --- a/docs/WINDOWS-PORT.md +++ b/docs/WINDOWS-PORT.md @@ -723,5 +723,7 @@ uses a Scheduled Task (S4U logon), not `pywin32`/NSSM — see §5.3. packaging, no daemon lifecycle, no testing. - **Windows ARM** — not considered. Electron supports it; Python and ffmpeg availability would need checking. -- **Windows Store / MSIX** — not planned for v1. NSIS per-user installer - is the target. +- **Windows Store / MSIX** — built since (`npm run dist:win:msix`, + `packaging/win/electron-builder.msix.yml`, which says what the manifest + declares in place of the NSIS scripts). Starting at boot stays the NSIS + installer's. diff --git a/docs/windows-build.md b/docs/windows-build.md index 3f14664..dbd3828 100644 --- a/docs/windows-build.md +++ b/docs/windows-build.md @@ -25,6 +25,16 @@ git clone https://github.com/<owner>/meshbay.git cd meshbay ``` +Then put the shared TMDB token in `QE\default.env` (never versioned), one line, +saved without a BOM: + +``` +MESHBAY_TMDB_DEFAULT_TOKEN=eyJ... +``` + +The build copies it beside the node as is, and stops if it is missing. Linux +builds (`build-node.sh`) read the same file. + ## 2. Build ```powershell |