Merge origin/main into the chat encryption work

Both sides landed a breaking MNP change and both called it 2.0, which is right:
the sealed upload, the removal of `stream_seg` and mandatory chat encryption
share one flag day. They are recorded as one version in `__init__.py` rather
than as a race between two.

The resolutions that were decisions rather than mechanics:

* **`MNP_MIN_SUPPORTED` moves to "2.0".** The sealed upload alone was a
  *confined* break — a 1.x peer could still connect, browse, download, stream
  and chat, with only its uploads refused by `upload_not_sealed` — so the floor
  deliberately stayed at "1.0". Mandatory chat encryption ends that
  confinement: a 1.x peer can neither produce a sealed chat message nor read
  one, so it would connect, look fine, and be unable to say anything. Refusing
  it at the handshake is the honest form. The per-message `upload_not_sealed`
  path is untouched and still right if the floor is ever lowered.
* **`sendChat` throws on an `error` reply**, from origin, applied to the sealed
  send. It matters more after this change, not less: the node now refuses a
  stale epoch, a malformed envelope and a device claim that is not the
  connection's own, so there are three new ways for a message to be rejected
  and none of them may look like a message that was sent.
* **`req_id` supersedes the per-type routing** this branch added for
  `chat_keys_resp` and `device_hello_ack`. Both blocks are kept beside the
  existing `chat_hist_resp` one, for the same stated reason — a node too old to
  stamp — and their comments no longer claim to be the mechanism that closes
  the class. `req_id` is.
* **`chat_send_probe.py` is rebuilt on origin's structure**, not beside it: two
  scenarios, a stub that stamps `req_id`, `music_meta_req` as the older pending
  request. The encrypted path is layered on — a real Ed25519 device key
  generated in the page, and a `chat_keys_resp` sealed by the shipped Python,
  because a payload the page built itself would prove only that the page agrees
  with the page.
* **`test_reply_correlation.py` now sends a sealed message.** Its subject is
  which of the two messages leaving that handler carries the id; plaintext chat
  was only the fixture, and the node refuses one now.
* `groupbox` keeps both new purposes (`upload`, `chat_keys`); `protocol.py`
  keeps origin's removal of `STREAM_SEGMENT` and this branch's correction of
  the "Double Ratchet message" comment on `CHAT_MESSAGE`, which was wrong when
  it was written and is wrong differently now.

Full suite on the merged tree: 1993 passed, 11 failed — the same 11 that fail
on a pristine checkout (2 Windows service tests, 1 apps-enabled policy, 7
transcode tests that pass in isolation, and the WebRTC invite test that hangs
on its own).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZZxYjz8YeWRz13xDi8LJr

3 files changed, 203 insertions, 18 deletions

diff --git a/packages/meshbay-common/src/meshbay_common/__init__.py b/packages/meshbay-common/src/meshbay_common/__init__.py
index ca4c6eb..cb6ad15 100644
--- a/packages/meshbay-common/src/meshbay_common/__init__.py
+++ b/packages/meshbay-common/src/meshbay_common/__init__.py
@@ -90,19 +90,48 @@ __version__ = "0.11.0"
 # makes the *next* breaking change cost a refusal message instead of a second
 # flag day. `MNP_MIN_SUPPORTED` in `handshake.py` is the other half.
 #
-# **2.0 (2026-09-07): chat is encrypted, and there is no way to turn it off.**
-# A MAJOR bump because it is a real break: a 1.x peer cannot produce a sealed
-# chat message and cannot read one, so it is refused at the handshake with
-# `version_too_old` rather than connecting and then failing to speak. Expressing
-# the break in the version is what makes it a stated refusal instead of a
-# conversation that silently does not work — `MNP_MIN_SUPPORTED` moves with it.
+# **2.0 (2026-09-07): the write path is sealed, chat is encrypted, and the last
+# unencrypted content message is gone.** Three changes that landed together and
+# share one version, because they share one flag day.
 #
-# There is deliberately no per-group switch. Every node in existence is a test
-# node, so an opt-in flag would buy nothing and cost a compatibility path to
-# maintain; existing node data is migrated by `QE/migrate-chat-encryption.py`.
+#   - `file_upload` and `file_upload_ack` travel sealed under a GEK-derived
+#     subkey (`groupbox.PURPOSE_UPLOAD`). The filename, the destination folder
+#     and the bytes all ride inside the seal; `upload_id` — a fresh
+#     client-chosen correlation id — and `chunk_index` stay in clear because
+#     the node routes and orders on them. `filename` used to be the
+#     correlation key and cannot be any more, which is what forced `upload_id`.
+#   - `chat_msg` is sealed under a per-device subkey of the group's chat epoch
+#     key (`meshbay_common.chatbox`) and signed over the ciphertext with the
+#     device key the node pinned. There is deliberately **no per-group switch**:
+#     every node in existence is a test node, so an opt-in flag would buy
+#     nothing and leave a plaintext branch reachable. Existing node data is
+#     migrated by `QE/migration/migrate_chat_encryption.py`.
+#   - `stream_seg` is **removed**. It answered with an MPEG-TS segment as
+#     base64 with no encryption at all, on both transports, to any
+#     authenticated member — the one content-plane message that never went
+#     through a GEK-derived key. `stream_data` has done the job properly since
+#     Phase 12, and `fetchStreamSegment`, its only browser caller, was defined
+#     and never once invoked.
 #
-# The index at rest, `index_progress` (counters only, never a path — see
-# `groupbox.py` and daemon.py `_push_index_progress`), and file content on the
-# operator's disk are all deliberately unchanged.
+# **Breaking, on the wire every deployed client speaks**, and MAJOR by the same
+# rule 1.0 was.
+#
+# **`MNP_MIN_SUPPORTED` moves to "2.0" with it, and that is a change of plan
+# worth reading.** The sealed upload alone was a *confined* break: a 1.x peer
+# could still connect, browse, download, stream and chat, and only its uploads
+# were refused — so the floor stayed at "1.0" and the refusal was per message
+# (`upload_not_sealed`). Mandatory chat encryption ends that confinement. A 1.x
+# peer can neither produce a sealed chat message nor read one, so it would
+# connect, appear to work, and then be unable to say anything or read anything
+# anyone else said. Refusing it at the handshake with `version_too_old` and a
+# sentence saying so is the honest form: a stated refusal is a bug report, a
+# chat that quietly does not work is a support case. The per-message
+# `upload_not_sealed` path stays, unchanged — it is still the right answer if
+# the floor is ever lowered again.
+#
+# Still deliberately in clear, and none of it is content: the handshake itself,
+# `index_progress` (counters only — see daemon.py `_push_index_progress`), the
+# admin and configuration acks, and the media-metadata replies. The index at
+# rest and file content on the operator's disk are unchanged.
 MNP_VERSION = "2.0"
 MHP_VERSION = "0.1"
diff --git a/packages/meshbay-common/src/meshbay_common/groupbox.py b/packages/meshbay-common/src/meshbay_common/groupbox.py
index ff60bba..e020ed8 100644
--- a/packages/meshbay-common/src/meshbay_common/groupbox.py
+++ b/packages/meshbay-common/src/meshbay_common/groupbox.py
@@ -24,6 +24,14 @@ and group configuration. It buys nothing against a network observer (DTLS/TLS
 already covers that), nothing against the hub (it never sees channel traffic), and
 nothing against a member — who holds the GEK. That is the whole claim.
 
+**Both, for the upload (MNP 2.0).** `file_upload` carried the filename and the raw
+bytes in clear, and `file_upload_ack` carried the name it was stored under. The
+download path had been sealed end to end since the beginning — so the same file was
+ciphertext coming out of a node and plaintext going in, which is not a threat model,
+it is an oversight. The node holds the GEK for its own group, so unlike the index
+this direction seals *towards* the node: it opens the payload before it writes
+anything to disk, and refuses a chunk that does not open rather than guessing.
+
 Purpose separation is deliberate. `GroupIndex.serialize()` reuses
 `chunk_key_aes(gek, file_hash, chunk_index)` with a pseudo-file ("the index as chunk
 0 of a virtual index file"), which borrows a file's key space for something that is
@@ -41,6 +49,7 @@ from cryptography.hazmat.primitives.kdf.hkdf import HKDF
 
 PURPOSE_INDEX = "index"
 PURPOSE_ACK = "ack"
+PURPOSE_UPLOAD = "upload"
 # The chat epoch keys themselves, on their way to a member. The keys are what
 # the chat archive is encrypted under; this is only how they travel, which is
 # why rotating the group key costs a re-wrap and not a re-encryption.
@@ -52,9 +61,17 @@ PURPOSE_CHAT_KEYS = "chat_keys"
 _INFO = {
     PURPOSE_INDEX: b"meshbay:index:v1",
     PURPOSE_ACK: b"meshbay:ack:v1",
+    PURPOSE_UPLOAD: b"meshbay:upload:v1",
     PURPOSE_CHAT_KEYS: b"meshbay:chat_keys:v1",
 }
 
+# One subkey per purpose, and `seal` draws a fresh 96-bit nonce per message, so
+# the bound that matters is birthday collision under `PURPOSE_UPLOAD` — the only
+# purpose with real volume, one message per 48 KiB chunk. 2**32 chunks is 200 TB
+# uploaded under a single GEK before the collision probability reaches 2**-32,
+# and `gek_rotate` exists. Deriving the nonce from the payload instead would be
+# worse, not better: two chunks of identical bytes are ordinary in a file.
+
 NONCE_LEN = 12   # 96-bit, the WebCrypto AES-GCM standard
 
 
diff --git a/packages/meshbay-common/src/meshbay_common/protocol.py b/packages/meshbay-common/src/meshbay_common/protocol.py
index dfb5d56..cc5324a 100644
--- a/packages/meshbay-common/src/meshbay_common/protocol.py
+++ b/packages/meshbay-common/src/meshbay_common/protocol.py
@@ -6,8 +6,31 @@ MHP (Mesh Bay Hub Protocol) — v0.1
 
 All wire messages are length-prefixed msgpack (4-byte big-endian length header).
 Every message carries a "v" field for protocol version.
+
+**`req_id` — the correlation id (added 2026-09-07).** A request may carry one;
+the reply to it carries the same value back, and nothing else on the wire does.
+It is the caller's own key for its pending request, opaque to the node, and
+unique only within one connection.
+
+There was none for a long time, and its absence was not neutral. A reply named
+its own type and nothing else, so a caller with more than one request in flight
+had to work out which one a message answered from the message itself — and the
+replies that name nothing (a bare `ack`, and `{"type": "error"}`, which
+webrtc_server.py sends from 240 places while two of them say what they are
+about) could only be matched by arrival order. That is a guess, wrong whenever
+two replies reorder, and it does not fail quietly: one request is resolved with
+another's answer while the request that answer belonged to waits out its own
+timeout. Live symptom (2026-09-06): a chat send whose reply went astray left
+the composer disabled for thirty seconds, and the Chat tab read as frozen.
+
+Both halves are optional and degrade to what came before: a request without one
+is answered without one, and a client that gets no id back falls back to
+matching by type. Neither side may treat it as authentication or as a sequence
+number — it is a label chosen by the peer, and the only thing it decides is
+which local promise a reply belongs to.
 """
 
+import os
 from dataclasses import dataclass, field
 from typing import Any
 
@@ -15,6 +38,7 @@ from typing import Any
 # second copy here said "0.1" while every message on the wire carried "0.2".
 # Nothing imported it, which is the only reason it was harmless.
 from meshbay_common import MNP_VERSION, MHP_VERSION   # noqa: F401  (re-export)
+from meshbay_common.groupbox import PURPOSE_UPLOAD, seal, unseal
 from meshbay_common.webcrypto import (
     chunk_key_aes,
     decrypt_chunk_aes,
@@ -37,13 +61,21 @@ class MNP:
     INDEX_PROGRESS  = "index_progress"
     FILE_REQUEST    = "file_req"         # request chunk(s)
     FILE_CHUNK      = "file_chunk"       # encrypted chunk response
-    STREAM_SEGMENT  = "stream_seg"       # HLS/DASH segment
+    # STREAM_SEGMENT ("stream_seg") was removed in MNP 2.0. It served an
+    # MPEG-TS segment as base64 **with no encryption at all** — the one message
+    # on the content plane that never was under a GEK-derived key. It predates
+    # STREAM_DATA, which does the same job properly (`chunk_ciphertext`, keyed
+    # per segment), and its browser caller `fetchStreamSegment` was defined and
+    # never once invoked. A live handler on both transports, plaintext media,
+    # and no client: removed rather than repaired.
+    #
     # Not a Double Ratchet message, and never was — `first-review.md` C1
-    # rejected exactly that for groups. Plaintext until a group turns
-    # encryption on, then AES-256-GCM under a per-device subkey of the group's
-    # chat epoch key, signed with the sending device's pinned Ed25519 key
-    # (`chatbox.py`, docs/chat-sender-keys.md).
-    CHAT_MESSAGE    = "chat_msg"         # one chat message, plain or sealed
+    # rejected exactly that for groups. Since MNP 2.0 it is AES-256-GCM under a
+    # per-device subkey of the group's chat epoch key, signed over the
+    # ciphertext with the sending device's pinned Ed25519 key. There is no
+    # plaintext form on the wire (`chatbox.py`, docs/chat-sender-keys.md);
+    # `format` distinguishes a *stored* pre-2.0 row, which is still served.
+    CHAT_MESSAGE    = "chat_msg"         # one chat message, sealed and signed
     CHAT_ATTACHMENT = "chat_attach"      # attachment metadata
     CHAT_HISTORY    = "chat_hist"       # request message history (newest, or before a cursor)
     CHAT_HISTORY_RESPONSE = "chat_hist_resp"  # history response with messages
@@ -369,3 +401,110 @@ def file_chunk_plaintext(
         file_hash = bytes.fromhex(msg["file_id"])
     ckey = chunk_key_aes(gek, file_hash, msg["chunk_index"])
     return decrypt_chunk_aes(ckey, msg["nonce"], msg["ct"])
+
+
+# ── Uploads (MNP 2.0) ─────────────────────────────────────────────────────────
+#
+# The write path, sealed under the group key the way the read path always was.
+# One encoder for both directions, here rather than in the client, for the reason
+# `file_chunk` has one: two copies of a wire shape with a single consumer each is
+# how `index_sync` and `file_chunk` forked (finding C6), and nothing noticed
+# until someone went looking.
+#
+# What stays in clear, and why each has to:
+#   `type`, `v`     — routed and version-checked before anything can be decrypted
+#   `upload_id`     — the correlation key. It replaces `filename`, which used to
+#                     play that role and cannot any more: naming the file in clear
+#                     to match an ack against a request would give back exactly
+#                     what the seal is for. Client-chosen, opaque to the node,
+#                     unique within one connection; never an authorization input.
+#   `chunk_index`   — ordering, which the node enforces before it opens anything
+#   `total_chunks`  — how many to expect
+#
+# `group_id` is *not* on the message: the session already decided which group it
+# is on, and the node uses that as the AAD. A client naming its own group here
+# would be choosing which key its bytes are checked against.
+
+UPLOAD_ID_LEN = 16   # 128 bits of client-chosen correlation, hex on the wire
+
+
+def new_upload_id() -> str:
+    """A fresh correlation id for one upload."""
+    return os.urandom(UPLOAD_ID_LEN).hex()
+
+
+def file_upload_wire(
+    gek: bytes,
+    group_id: str,
+    *,
+    upload_id: str,
+    chunk_index: int,
+    total_chunks: int,
+    filename: str,
+    data: bytes,
+    dir: str = "",
+    root: str = "",
+) -> dict:
+    """
+    One sealed `file_upload` chunk.
+
+    `filename`, `dir` and `root` ride inside the seal with the bytes: sealing the
+    content and announcing the name beside it would be theatre. They are repeated
+    on every chunk rather than sent once — a hundred bytes against a 48 KiB chunk
+    — because a header that arrives once is state the node has to carry, and
+    upload state that can disagree with the chunk in hand is what `_free_name` and
+    the chunk-ordering rule exist to prevent.
+    """
+    payload = {"filename": filename, "data": data, "dir": dir, "root": root}
+    return {
+        "type": MNP.FILE_UPLOAD,
+        "v": MNP_VERSION,
+        "upload_id": upload_id,
+        "chunk_index": chunk_index,
+        "total_chunks": total_chunks,
+        **seal(gek, PURPOSE_UPLOAD, MNP.FILE_UPLOAD, group_id, payload),
+    }
+
+
+def file_upload_payload(gek: bytes, group_id: str, msg: dict) -> dict:
+    """
+    Open a `file_upload`. Raises on anything that does not open.
+
+    Never a partial result and never a default: a chunk that does not open is not
+    an empty file with an empty name, it is a peer we cannot talk to. `unseal`
+    says why at length.
+    """
+    return unseal(gek, PURPOSE_UPLOAD, MNP.FILE_UPLOAD, group_id, msg)
+
+
+def file_upload_ack_wire(
+    gek: bytes,
+    group_id: str,
+    *,
+    upload_id: str,
+    chunk_index: int,
+    filename: str,
+    stored_as: str,
+    dir: str = "",
+) -> dict:
+    """
+    The node's answer to one chunk, sealed the same way.
+
+    `stored_as` is the name the node settled on — it finds a free one rather than
+    replacing anything — and `dir` is where it landed. Both name the operator's
+    content, so both belong inside the seal; only `upload_id` and `chunk_index`
+    stay out, because the client matches on them.
+    """
+    payload = {"filename": filename, "stored_as": stored_as, "dir": dir}
+    return {
+        "type": MNP.FILE_UPLOAD_ACK,
+        "v": MNP_VERSION,
+        "upload_id": upload_id,
+        "chunk_index": chunk_index,
+        **seal(gek, PURPOSE_UPLOAD, MNP.FILE_UPLOAD_ACK, group_id, payload),
+    }
+
+
+def file_upload_ack_payload(gek: bytes, group_id: str, msg: dict) -> dict:
+    """Open a `file_upload_ack`. Raises on anything that does not open."""
+    return unseal(gek, PURPOSE_UPLOAD, MNP.FILE_UPLOAD_ACK, group_id, msg)