Part of mathias/infra Track E umbrella. Captures the missing data source the volume gate exposed.
Goal
Scan ~/.claude/projects/*/<uuid>.jsonl on each host, parse turns, scrub secrets, classify, ingest into brain. Per-host instance: koala, flamingo, iguana.
Scope
New package ingestion/internal/claudewatcher/:
Watcher loop — claudewatcher.Watch(ctx, cfg) polls Claude Code projects dir on configurable interval (default 60s). New or modified .jsonl files trigger parse.
Parser — reads JSONL incrementally, tracks last-ingested offset per file in postgres (new table claude_session_cursors).
Scrubber — runs each turn's content through the secret-hygiene rules (regex set: bearer tokens, postgres URIs, API keys, SOPS-encrypted blocks, BEGIN PRIVATE KEY). Fail-closed: if any rule matches, drop the turn + log a warning (don't try to redact + ingest).
Classifier — uses existing brain_classify tool. Filter: ingest only entries where type ∈ {decision, lesson, correction, surprise}. Skip routine code / note / unknown to keep brain signal high.
Ingestor — emits to brain_ingest_raw with structured pages: {title, content, frontmatter: {wing: 'claude-sessions', hall: hall-by-type, session_id, project_root, host}}.
Wiring — cmd/server/main.go starts the watcher when CLAUDE_SESSIONS_DIR env var is set. Defaults to $HOME/.claude/projects. CLAUDE_INGEST_INTERVAL controls poll (default 60s).
koala: ingestion pod already runs here. Mount /root/.claude/projects (or wherever the act_runner / Claude Code session writes) as read-only into pod, set CLAUDE_SESSIONS_DIR to that mount.
flamingo: needs a small sidecar — either run a second ingestion-mode pod, or run claudewatcher as a launchd job that POSTs to brain HTTP API.
iguana: same shape as flamingo, launchd job.
Flamingo + iguana wiring is out of scope for this issue — land koala first as proof-of-life, file follow-ups for the other two hosts.
Acceptance
New skill_session, decision, correction entries appear in brain/wiki/claude-sessions/<hall>/*.md within 60s of a Claude session producing them on koala
Re-measured volume gate (wc -l brain/sessions/*.jsonl + classified-entries count) is ≥500/week within one month
Scrubber catches all 10 poisoned-fixture tests; zero credential strings reach the brain in spot-check of 100 ingested entries
task check green; trunk-based commit chain to main
Branch policy
If Track E.2 is being worked on simultaneously, use branch agent/track-e1-claudewatcher per CLAUDE.md TBD parallel-agent rule. Merge to main when complete + task check green.
Related
infra Track E umbrella
hyperguild#NN E.2 (parallel: brain_context tool)
Part of mathias/infra Track E umbrella. Captures the missing data source the volume gate exposed.
## Goal
Scan `~/.claude/projects/*/<uuid>.jsonl` on each host, parse turns, scrub secrets, classify, ingest into brain. Per-host instance: koala, flamingo, iguana.
## Scope
New package `ingestion/internal/claudewatcher/`:
1. **Watcher loop** — `claudewatcher.Watch(ctx, cfg)` polls Claude Code projects dir on configurable interval (default 60s). New or modified `.jsonl` files trigger parse.
2. **Parser** — reads JSONL incrementally, tracks last-ingested offset per file in postgres (new table `claude_session_cursors`).
3. **Scrubber** — runs each turn's content through the secret-hygiene rules (regex set: bearer tokens, postgres URIs, API keys, SOPS-encrypted blocks, `BEGIN PRIVATE KEY`). Fail-closed: if any rule matches, drop the turn + log a warning (don't try to redact + ingest).
4. **Classifier** — uses existing `brain_classify` tool. Filter: ingest only entries where `type` ∈ {`decision`, `lesson`, `correction`, `surprise`}. Skip routine `code` / `note` / `unknown` to keep brain signal high.
5. **Ingestor** — emits to `brain_ingest_raw` with structured pages: `{title, content, frontmatter: {wing: 'claude-sessions', hall: hall-by-type, session_id, project_root, host}}`.
6. **Wiring** — `cmd/server/main.go` starts the watcher when `CLAUDE_SESSIONS_DIR` env var is set. Defaults to `$HOME/.claude/projects`. `CLAUDE_INGEST_INTERVAL` controls poll (default 60s).
7. **Tests** — table-driven: parser handles partial JSONL, dedup cursor advances, scrubber catches 10 known-bad fixtures, classifier filter rules.
## Per-host deployment
- **koala**: ingestion pod already runs here. Mount `/root/.claude/projects` (or wherever the act_runner / Claude Code session writes) as read-only into pod, set `CLAUDE_SESSIONS_DIR` to that mount.
- **flamingo**: needs a small sidecar — either run a second ingestion-mode pod, or run `claudewatcher` as a launchd job that POSTs to brain HTTP API.
- **iguana**: same shape as flamingo, launchd job.
Flamingo + iguana wiring is **out of scope for this issue** — land koala first as proof-of-life, file follow-ups for the other two hosts.
## Acceptance
- New skill_session, decision, correction entries appear in `brain/wiki/claude-sessions/<hall>/*.md` within 60s of a Claude session producing them on koala
- Re-measured volume gate (`wc -l brain/sessions/*.jsonl` + classified-entries count) is ≥500/week within one month
- Scrubber catches all 10 poisoned-fixture tests; zero credential strings reach the brain in spot-check of 100 ingested entries
- `task check` green; trunk-based commit chain to main
## Branch policy
If Track E.2 is being worked on simultaneously, use branch `agent/track-e1-claudewatcher` per CLAUDE.md TBD parallel-agent rule. Merge to main when complete + task check green.
## Related
- infra Track E umbrella
- hyperguild#NN E.2 (parallel: brain_context tool)
Closed by claudewatcher rollout — pod ingestion-65c997f99f-2s9cp on koala running since 2026-05-26T05:15Z. Live numbers as of close: 16 wiki/claude-sessions notes, 45 batches, 140 scrubber drops across 9 rule types including 3 client-name catches. Tests green, image bad0581 deployed. Follow-up classifier-driven hall routing tracked in #29.
Closed by claudewatcher rollout — pod ingestion-65c997f99f-2s9cp on koala running since 2026-05-26T05:15Z. Live numbers as of close: 16 wiki/claude-sessions notes, 45 batches, 140 scrubber drops across 9 rule types including 3 client-name catches. Tests green, image bad0581 deployed. Follow-up classifier-driven hall routing tracked in #29.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Part of mathias/infra Track E umbrella. Captures the missing data source the volume gate exposed.
Goal
Scan
~/.claude/projects/*/<uuid>.jsonlon each host, parse turns, scrub secrets, classify, ingest into brain. Per-host instance: koala, flamingo, iguana.Scope
New package
ingestion/internal/claudewatcher/:claudewatcher.Watch(ctx, cfg)polls Claude Code projects dir on configurable interval (default 60s). New or modified.jsonlfiles trigger parse.claude_session_cursors).BEGIN PRIVATE KEY). Fail-closed: if any rule matches, drop the turn + log a warning (don't try to redact + ingest).brain_classifytool. Filter: ingest only entries wheretype∈ {decision,lesson,correction,surprise}. Skip routinecode/note/unknownto keep brain signal high.brain_ingest_rawwith structured pages:{title, content, frontmatter: {wing: 'claude-sessions', hall: hall-by-type, session_id, project_root, host}}.cmd/server/main.gostarts the watcher whenCLAUDE_SESSIONS_DIRenv var is set. Defaults to$HOME/.claude/projects.CLAUDE_INGEST_INTERVALcontrols poll (default 60s).Per-host deployment
/root/.claude/projects(or wherever the act_runner / Claude Code session writes) as read-only into pod, setCLAUDE_SESSIONS_DIRto that mount.claudewatcheras a launchd job that POSTs to brain HTTP API.Flamingo + iguana wiring is out of scope for this issue — land koala first as proof-of-life, file follow-ups for the other two hosts.
Acceptance
brain/wiki/claude-sessions/<hall>/*.mdwithin 60s of a Claude session producing them on koalawc -l brain/sessions/*.jsonl+ classified-entries count) is ≥500/week within one monthtask checkgreen; trunk-based commit chain to mainBranch policy
If Track E.2 is being worked on simultaneously, use branch
agent/track-e1-claudewatcherper CLAUDE.md TBD parallel-agent rule. Merge to main when complete + task check green.Related
Closed by claudewatcher rollout — pod ingestion-65c997f99f-2s9cp on koala running since 2026-05-26T05:15Z. Live numbers as of close: 16 wiki/claude-sessions notes, 45 batches, 140 scrubber drops across 9 rule types including 3 client-name catches. Tests green, image
bad0581deployed. Follow-up classifier-driven hall routing tracked in #29.