From 80ac882346b474bb7ea4e8c8afd4694bd13dd9dd Mon Sep 17 00:00:00 2001 From: Christophe Besson Date: Wed, 14 Oct 2026 10:17:49 +0200 Subject: feat(node): skip a file the disk stops answering for, then pause its root A Windows tester's scan stuck for good on one directory of a large disk: one read that never returned held the indexer's only thread, and with it every root behind that one. - A disk call is timed by its silence, not its length: each directory entry listed and each megabyte read counts as an answer. After 60 s of silence the call is abandoned on its daemon thread, the file is noted unread and the scan goes on. - Three silent files in a row pause the root (not persisted). Resume skips the files already known to give no answer; a restart, a replug or a change to the file retries them. - The walk is per directory with os.scandir: a subdirectory that cannot be listed costs itself, and a listing that stops answering pauses the root at once. A scan no longer stats anything from the event loop. - The log names the file after 10 s of silence, the indexing band shows it, and Group settings shows the paused root, where it stuck and the unread files with their reason, on the node's machine only. - generate_http_api.py writes LF on Windows too. Verified with real kernel-blocked opens (an unacknowledged oplock) in the installed frozen app. Co-Authored-By: Claude Opus 5.5 --- docs/MESHBAY_DESIGN.md | 50 +++++++++++++++++++++++++++++++++++++++++++++++ docs/USERGUIDE.md | 19 ++++++++++++++++++ docs/generate_http_api.py | 3 ++- 3 files changed, 71 insertions(+), 1 deletion(-) (limited to 'docs') diff --git a/docs/MESHBAY_DESIGN.md b/docs/MESHBAY_DESIGN.md index 0703993..1f6c96d 100644 --- a/docs/MESHBAY_DESIGN.md +++ b/docs/MESHBAY_DESIGN.md @@ -1783,6 +1783,53 @@ root is therefore not watched, not swept by reconcile, and not hashed by any wal there; the sweep indexes only what the index lacks. The request returns once the walk is queued, and the progress band shows it. +**A disk that stops answering costs a file, then a root, never the scan.** A read +on a failing drive, a share that dropped or a file a sync client has not fetched +does not fail: it does not return. The hash ran on the indexer's one thread with +nothing to bound it, so one such file stopped its group's scan where it stood: +the roots behind it unread, the enrichment that follows a scan never started, a +pause unable to take effect since it is looked at between two files, and nothing +in the log naming the file, because a file is named there only when it fails. + +- **What is timed is silence, not work.** Every directory entry listed and every + megabyte read counts as an answer (`_beat`). A listing of a million entries and + a file on a slow link take as long as they take; neither goes sixty seconds + (`STALL_SECS`) without one. A call waiting its turn behind another is not + timed for the wait. +- **The thread is left behind, because nothing can interrupt it.** It stays in + the kernel for as long as the kernel keeps it, and a new one takes the calls + that were waiting (`_DiskThread.abandon`). They are daemon threads: the + interpreter joins a pool's threads on its way out, and a node with one read + that never returned could not stop. +- **A root's walk asks the disk nothing from the event loop.** Each file's stat + is taken with the listing, on the indexer's thread, and handed down to the + hash. Asked again on the loop before each hash, as it was, it is the loop + that waits on a silent disk, and with it every connection of the node. The + one file a watcher event names is still asked for there. +- **The file is listed as unread and not asked for again until it changes.** + Reconcile meets it at every pass and would wait a minute on it each time. A + file the system refuses outright (a permission, a lock) is listed with what the + system said and is asked again, since asking costs nothing. A restart, or the + root going away and coming back, forgets the list. +- **Three in a row, with none read in between, and the root is paused.** A minute + a file is weeks on a music library, with a thread left behind each time. The + root is paused by the node as the operator would pause it (above): the rest of + the walk comes from the hash cache, the watcher and reconcile leave it alone, + what is indexed stays served, the next root is walked, and Resume starts it + again and gets past the files already known to give no answer. A listing that + stops answering pauses its root at once, having no file to give up on; a rescan + that gets no listing puts back the entries it had dropped. +- **This pause is not persisted.** The roster holds the operator's pauses. A + restart tries the root again, which is what brings back one whose drive or + share has returned, at the cost of three minutes when it has not. +- **The operator is told, on their own machine only.** Ten seconds of silence is + logged and shown in the progress band with the file; the group's shared + directories list the files given up on and why, and where a root went silent. + These are paths, so they are on the loopback API (`index-status`) and in + nothing a member receives: a member sees the root paused. +- **One directory that cannot be listed costs that directory.** The walk used to + skip a refusal in silence and to end the whole root's scan on any other error. + ### 6.4 Uploads Five protections, and they are the substance: @@ -4461,6 +4508,9 @@ process runs it — `systemctl --user` on Linux, Task Scheduler on Windows. | Item | Status | |---|---| +| **A disk that stops answering still stalls what is not the scan** | The indexer's own reads are bounded (§6.3). Serving a chunk, the availability check and the directory deletions go through the roots' I/O thread, which has no such limit, and the tag and photo readers leave a thread in the shared pool at each timeout | +| **A file that gave no answer is asked again at every start** | The list of unread files is memory. With three or more such files in a row the root is paused again three minutes into each start, until the operator resumes it or moves them | +| **Paths longer than 260 characters are not indexed on Windows** | With the system's default setting the walk cannot list them; it now says so once per directory instead of nothing. The reads already use the long form (`paths.long_path`), the walk does not | | **The desktop cast relay drops whole segments** | `cast-relay.js` drops a fragment once 8 MB wait for a receiver, and a fragment is a segment, 5 to 10 MB at a film's bitrate: the picture freezes for its length. Its backlog is bounded in fragments only. The Android relay spools a receiver's lead to disk and bounds its backlog in bytes (§11.4); the desktop has neither | | **The desktop cast relay takes the first chunk for the whole header** | The node's first chunk can be the ftyp alone, the moov in the next; the receiver then gives up. Fixed in the Android relay (§11.4), not in `cast-relay.js` | | **A cast restart may pull far ahead** | Seen once on an emulator with a synthetic film: after the restart's reinit the node reported `duration=None`, and the page pulled most of the film at network speed. Not reproduced on a real film; the suspicion is a read-ahead budget computed without a duration | diff --git a/docs/USERGUIDE.md b/docs/USERGUIDE.md index 9d1749f..bab2074 100644 --- a/docs/USERGUIDE.md +++ b/docs/USERGUIDE.md @@ -702,6 +702,19 @@ Nothing else in that directory is indexed until you click **Resume indexing**, not even a file a member uploads into it, and the pause survives a restart. Resuming picks up where it stopped: files already indexed are not read again. +**A disk that stops answering.** A failing drive, a network share that dropped +or a file a sync client has not downloaded can leave a read waiting for ever. +The node waits sixty seconds for an answer, then leaves that file and goes on. +After three such files in a row it takes the disk to be the problem and pauses +that directory by itself; the other directories go on being indexed, and what +was already indexed stays visible. On the node's own machine, the indexing band +names the file once the disk has been silent for ten seconds, and Group → +Settings shows **disk not answering** on the directory, where it stopped, and +the list **Files that could not be read** with the reason for each. Check the +disk, or move those files out, then click **Resume indexing**: the node gets +past the files it already knows give no answer, and tries them again when they +change or when it restarts. + One thing that surprises people: **two identical files in one group are one entry**, because a file is identified by its content. A scan that reports ten files and indexes nine has not lost anything. @@ -1037,6 +1050,12 @@ Choose a download folder in Settings → Downloads, which switches the browser t writing straight to disk, and retry. The desktop application does not have this class of problem. +**Indexing stops on one directory, or some of its files never appear.** +Group → Settings, on the node's own machine: a directory marked **disk not +answering** was paused by the node, and **Files that could not be read** lists +what it gave up on and why. See Indexing in section 6. The node's log names +the same files (`No answer from the disk`, `Cannot index`). + **Nothing appears in Videos, Music or Photos.** No folders have been chosen for that application yet. Group → Settings. The tab says so rather than just looking empty, but it is easy to miss. diff --git a/docs/generate_http_api.py b/docs/generate_http_api.py index d064519..cd99332 100644 --- a/docs/generate_http_api.py +++ b/docs/generate_http_api.py @@ -160,5 +160,6 @@ def render() -> str: if __name__ == "__main__": - OUT.write_text(render(), encoding="utf-8") + # Not `write_text`: on Windows it turns every line ending into CRLF. + OUT.write_bytes(render().encode("utf-8")) print(f"wrote {OUT}", file=sys.stderr) -- cgit v1.2.3