fix(watcher): writes through the brain root (lands #195 on current main); size cap; long-name case (#158, #198, #199) #210

Merged
mathias merged 7 commits from wt/watcher-write-safety into main 2026-10-20 18:19:17 +00:00
7 Commits
Author SHA1 Message Date
mathiasandClaude Sonnet 5.5 6717221989 fix(watcher): a held note is reported once; future-dated and both stale markers handled (review of #210, pass 2)
CI / Lint / Test / Vet (pull_request) Successful in 1m4s
CI / Mirror to GitHub (pull_request) Skipped
- a marker that blocks a note is warned about once, not silently forever
- a marker dated more than 10 minutes ahead (clock skew, a peer) is not
  believed and is treated as stale
- every stale marker is removed, not only the last; removal errors are logged
Each has a test that fails without it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:17:10 +02:00
mathiasandClaude Sonnet 5.5 1025d77795 fix(watcher): unique archive copies, read caps at the read, stale markers (review of #210)
CI / Lint / Test / Vet (pull_request) Successful in 1m5s
CI / Mirror to GitHub (pull_request) Skipped
From the independent review of #210:
- F1: processed/ and failed/ copies are created O_EXCL with -2,-3 suffixes via
  createUnique; same-named notes from different folders no longer replace one
  another's only surviving copy
- F2: extract.TextInMax / readAllMax cap the bytes actually read (a note can
  grow after the Lstat); extract.TextIn gets a 64 MiB default cap
- F3: a regular marker older than its note describes an earlier note of the
  same name; it no longer hides the new one (warn + remove); non-regular
  markers still count as done (no writes through links)
- F4: the planted-marker-link test is renamed to what it observes
Each has a test that fails without it; the read-cap test uses an endless reader
that errors at 8 MiB so a regression fails instead of exhausting memory.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:13:18 +02:00
mathiasandClaude Sonnet 5.5 d8c7defc78 fix(watcher): refuse oversize notes before reading them; pin the long-name marker case (#198, #199)
CI / Lint / Test / Vet (pull_request) Successful in 1m7s
CI / Mirror to GitHub (pull_request) Skipped
MaxNoteBytes (default 64 MiB) is checked on the Lstat size before extraction;
an oversize note stays in raw/, is reported once and not retried until it
changes. The #199 case (marker name over NAME_MAX) is covered by the
remember-on-any-failure retire path and now has a test that fails without it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:04:02 +02:00
mathiasandClaude Sonnet 5.5 2cc4f1b30d merge: land #195 (watcher/pipeline writes through the brain root, #158 part B) on top of inbox zero and the wing resolver
Reconciled with v0.25.0 semantics:
- markers are only the fallback when the original cannot be removed, and are
  now written through the brain root; the refused set (size+mtime keyed) replaces
  the path-only 'stuck' set
- #195's tests updated for inbox zero (archive written, original retired, no
  marker), and the planted-marker-link test keeps its point: nothing is ever
  written through the link, and the note is processed once
- the 'cannot retire' test now counts archive copies; it counted LLM calls,
  which pipeline.Run's source-hash check hides (mutation: removing the
  remember step left it green)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:02:15 +02:00
mathiasandClaude Opus 5.5 22d0653b61 test(watcher): a marker link planted during the run is refused and not retried (#158)
CI / Lint / Test / Vet (pull_request) Successful in 1m3s
CI / Mirror to GitHub (pull_request) Skipped
Review round 2 on PR #195: the marker writes had no test, so reverting
them to by-path writes, or the refusal to log-only, stayed green. The
link is planted from inside the fake model call, between the walk's
Lstat and the marker write, for both markers and both an outside and
an in-brain relative target; red on all four flips. The marker-entry
skip is now checked for .failed too.

Also pins cleanRel's acceptance of a backslash as a filename byte in
brainfs itself, and stops the buildAndWrite control claiming writes it
does not count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:47:24 +02:00
mathiasandClaude Opus 5.5 1a2a93d2f4 fix(watcher): refusals end in a terminal state; pin every converted writer (#158)
CI / Lint / Test / Vet (pull_request) Successful in 1m6s
CI / Mirror to GitHub (pull_request) Skipped
Review round 1 on PR #195:
- A refused marker or archive write left no marker, so the note was
  reprocessed every interval (archive copies piling up; an LLM call per
  round on the failure path). The marker check now uses Lstat (any entry
  at the marker path counts, so a dangling marker link skips the note
  instead of looping), and a note whose archive copy or marker write is
  refused is remembered (path, size, mtime) and not retried until it
  changes.
- cleanRel no longer refuses a backslash: it is a legal filename byte on
  Linux, not a traversal, and a raw note named with one was never
  archived or marked.
- Tests now reach every converted writer: the log.md append (real
  raw/failed so the log is reached, symlinked log.md), a dangling marker
  link, the refused-retry loop with a positive control, a backslash name,
  a buildAndWrite swap race, and the claude-session archive with in-brain
  and outside symlinked archive dirs. The stale lint allowlist entry for
  appendWatcherLog is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:40:19 +02:00
mathiasandClaude Opus 5.5 77c44f298c fix: pipeline, backfill, watcher and archive writes go through the brain root (#158 part B)
CI / Lint / Test / Vet (pull_request) Successful in 1m13s
CI / Mirror to GitHub (pull_request) Skipped
The remaining writers created or rewrote brain files by path:
- pipeline buildAndWrite: os.MkdirAll + os.WriteFile of every page.
- BackfillRefs: os.WriteFile of each concept page; a page swapped for a
  symlink after the read was written through (the race test overwrote an
  in-brain target before this change).
- watcher: the raw/processed and raw/failed archive copies (os.Create),
  the .processed/.failed markers and the log.md entry (os.OpenFile).
- cmd/server: the claude-session archive.

All now go through brainfs PutRegular/AppendRegular (new package-level
PutFile/AppendFile for one-off writers: the watcher's call-site lint bans
.Open calls in that file so raw/ reads stay on OpenRegular).

Stacked on #189 (needs PutRegular/AppendRegular).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:26:02 +02:00