Commit Graph
317 Commits
Author SHA1 Message Date
mathiasandClaude Opus 4.8 38a2e91002 feat(capturehttp): 503 on audit-unavailable; wire degrading sink (#54)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
- Map capture.ErrAuditUnavailable → HTTP 503 (audit substrate down /
  confidential unauditable / floor).
- main: buildAuditSink selects the DegradingSink (loki central + durable
  file buffer under brainDir + optional ntfy) when BRAIN_LOKI_URL is set
  and starts the reconcile loop; else the plain slog sink. Notifier kept
  as a nil interface (not typed-nil) when unconfigured so the sink and
  reconcile skip it cleanly.

Env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN,
BRAIN_AUDIT_RECONCILE_INTERVAL (default 60s). Buffer at
<brain>/.audit-buffer/capture.jsonl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathiasandClaude Opus 4.8 77f5e06d6b feat(audit): DegradingSink + durable buffer + loki/ntfy + reconcile (#54)
The classification-aware I5 audit path (§4.4):

- DegradingSink.Reserve: central up → AuditCentral; central down +
  confidential → refuse (no buffer); central down + internal/public +
  buffer writable → AuditBuffered; central down + buffer unwritable →
  floor refuse. Record executes the reserved outcome and, when buffered,
  fires an ntfy alert.
- FileBuffer: durable JSONL buffer that survives process restart; Confirm
  rewrites the file without a record, so a buffered record is cleared ONLY
  after its central write is confirmed.
- LokiCentral: /ready probe + /loki/api/v1/push (full audit entry as the
  structured line). NtfyNotifier: degraded-state alerts; token only in the
  auth header, never logged (regression-tested).
- Reconcile + StartReconcile: replay buffered records to central on
  recovery, confirm-then-clear per record; a failed push keeps the record
  buffered (no loss). SlogSink updated to the two-phase shape (always
  central, never fails) — the default when no loki endpoint is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathiasandClaude Opus 4.8 202212e8d5 feat(capture): two-phase classification-aware AuditSink port (#54)
Splits the audit port into Reserve (before any write) + Record (after),
so "confidential + sink-down → refuse before any write" is literally true
even though the audit record — which lists what landed — can only be
written afterwards.

- AuditSink.Reserve(ctx, level) → AuditOutcome | error. The error path
  refuses the capture before writing: confidential + central sink down,
  or the all-tiers floor (nothing can record).
- AuditSink.Record(ctx, entry, outcome) persists per the reserved outcome.
- Service: I5 gate runs after the I1 gate and after the dry-run
  short-circuit (dry-run never probes the sink). AuditBuffered surfaces on
  the receipt. New ErrAuditUnavailable sentinel (→ HTTP 503).

The tier→behaviour decision lives in the sink impl (#54's DegradingSink),
not the service — the service just honours Reserve's verdict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathias b7938d4636 Merge pull request 'feat: POST /capture REST adapter + OAuth2 + I1 sovereignty gate (#53, capture 49d)' (#59) from feat/capture-rest-i1-gate into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 21:44:14 +00:00
mathiasandClaude Opus 4.8 a1997838b0 feat(capturehttp): POST /capture REST adapter + OAuth2 + origin resolver (#53)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
The HTTP door for the capture capability. Thin: authenticate → derive
trust-zone origin → decode → capture.Service → map receipt to status.

- Auth mirrors the chassis Bearer precedence (static token wins, then Dex
  JWT) but returns the resolved principal + auth path, which the chassis
  middleware hides — capture needs the principal to derive the origin.
  Depends on a small Validator interface (the chassis *JWTValidator
  satisfies it) so the JWT/origin path is testable without a live JWKS.
- OriginResolver maps principal → trust zone: static-token caller and
  allowlisted JWT subjects → sovereign; every other principal → us-nexus
  (fail safe, so the I1 gate refuses confidential by default). Principal
  and origin are server-set on the input, overwriting any body the caller
  sent.
- HTTP status: 200 all-ok / dry-run, 207 partial, 502 all-failed, 403 on
  the I1 refusal, 400 on fail-closed validation.
- Wired in main behind the same static+JWT credentials as /mcp, reusing
  the MCP server's graph-wired brain store (one implementation) and the
  classification tags (#50). Mounts only when a Gitea tracker is
  configured. Sovereign JWT principals via BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathiasandClaude Opus 4.8 77680c7445 feat(audit): minimal slog AuditSink for capture I5 (#53)
Emits the request-level audit record to structured logs (scraped by the
existing alloy/loki substrate) and surfaces security events at warn
level. Placeholder behind the AuditSink interface — the classification-
aware loki+buffer+reconcile sink (confidential fails closed, internal
degrades) lands in #54 and replaces this without touching callers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathiasandClaude Opus 4.8 d7a842f356 feat(capture): I1 sovereignty gate + server-derived origin (#53)
Adds the trust-zone Origin to CaptureContext and the I1 gate to the
use-case: a confidential effective classification through a us-nexus
origin is refused before ANY write (ErrSovereigntyRefused), and the
refusal is itself audited. A caller-asserted harness label that names a
different zone than the server-derived origin is logged as a security
event — context.Harness is descriptive-only, never a gate input.

The gate triggers only on an explicit ZoneUSNexus, so the unset default
(ZoneUnknown) can never make it fire on caller-controllable input; the
REST adapter always sets a concrete zone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathias aad90f2dfe Merge pull request 'feat: Gitea IssueTracker client + inject into brain server (#52, capture 49c)' (#58) from feat/capture-gitea-tracker into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 21:32:02 +00:00
mathias 07fca9ee73 Merge pull request 'feat: CaptureService use-case + BrainStore extraction (#51, capture 49b)' (#57) from feat/capture-service into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Has been cancelled
2026-06-22 21:31:47 +00:00
mathias f6bf9b5f57 Merge pull request 'feat: capture classification taxonomy + per-wing/repo tags (#50, capture 49a)' (#56) from feat/capture-classification into main
CI / Lint / Test / Vet (push) Has been cancelled
CI / Mirror to GitHub (push) Has been cancelled
2026-06-22 21:31:18 +00:00
mathiasandClaude Opus 4.8 6606b38a76 feat(gitea): IssueTracker client + inject into brain server (#52)
Implements the IssueTracker port as a real Gitea REST client (#49c) — the
new outbound dependency the brain server gains for capture.

- CreateIssue / CommentIssue / CloseIssue(+optional closing comment) over
  the Gitea API. Owner is the const "mathias", never caller-supplied, so
  a caller cannot redirect a write to another owner's repo.
- Token read once at construction (BRAIN_GITEA_TOKEN), held in the struct,
  travels only in the Authorization header — never logged or in argv.
  Error messages carry status + truncated body, never the token
  (regression-tested). gitea.New returns nil when URL or token is unset,
  so missing config = tracker disabled via one nil check.
- Injected into the MCP server behind the capture.IssueTracker interface
  via WithIssueTracker (constructor injection, swappable/testable); main
  wires it from BRAIN_GITEA_URL (default https://git.d-ma.be) +
  BRAIN_GITEA_TOKEN. Consumed by the capture use-case in #53.

Tests use httptest transports: create (owner+auth header asserted),
comment, close with/without comment, error path that proves the token
never leaks into an error string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:27:29 +02:00
mathiasandClaude Opus 4.8 4cfc98de56 refactor(capture): CloseIssue carries a closing comment (#52)
#52's IssueTracker spec is CloseIssue(repo, number, comment). Refine the
#51 port signature to match and have the service pass the ticket body as
the closing comment (empty ⇒ close only). Keeps the close-with-comment
flow first-class rather than forcing two separate ticket items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:27:29 +02:00
mathiasandClaude Opus 4.8 0ac165cca3 feat(brainstore): shared BrainStore impl; re-point MCP handlers (#51)
Extracts the #45 write/update/get logic + the wiki upkeep that must
accompany a write (wing _index rebuild, cross-wing auto-tunnel, graph
re-index) into a single concrete brainstore.Store implementing
capture.BrainStore. The MCP brain_write/brain_update/brain_get handlers
are re-pointed at it, so there is one implementation, not two — the DRY
payoff #51 is named for. capture and MCP now share the exact same brain
write path and read-after-write contract.

The Server gains a *brainstore.Store, constructed in NewServer and given
the graph store in WithGraph. Embedding refresh stays out-of-band
(mtime-driven vectorstore.Sync), unchanged. Existing MCP brain_update/
brain_get/brain_write tests pass unmodified — behaviour and the
{id, path, content_hash} response contract are preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:21:43 +02:00
mathiasandClaude Opus 4.8 43f92e3102 feat(capture): CaptureService use-case + ports + entities (#51)
The Clean-Architecture core of the capture capability (#49b). Pure
orchestration over ports — no HTTP, no live Gitea, no audit I/O — fully
unit-tested against fakes before any adapter exists.

- Ports: BrainStore (#45 write/update/get), IssueTracker, SummaryWriter,
  ClassificationPolicy (satisfied by #50's classification.Config),
  AuditSink. Entities: Insight, Ticket, Summary, CaptureContext,
  CaptureInput, CaptureReceipt.
- CaptureService.Capture: validate-before-write (fail-closed), resolve
  effective classification (stricter of declared vs target-derived;
  under-declaration logged as a security event), orchestrate insights
  (write/supersede) → tickets → summary best-effort, emit a request-level
  audit record of exactly what landed, return a partial-aware receipt.
- dry_run short-circuits after validation, writes nothing (not even audit).

Out of scope here, layered on later: the I1 origin sovereignty gate (#53,
needs the server-derived principal) and the classification-aware audit
degradation/refusal (#54). "Effective" is folded into the service as
classification.Stricter rather than a port method — the stricter-wins
rule is use-case policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:21:43 +02:00
mathiasandClaude Opus 4.8 98cfae595c feat(classification): data-sensitivity taxonomy + per-wing/repo tags (#50)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Implements the I1-prerequisite from #49/#50: the classification taxonomy
and the per-wing / per-repo tagging the capture server reads to derive a
target's sensitivity.

- Levels public < internal < confidential, ordered so "stricter wins"
  (spec §4.1 model C) is a plain max via Stricter().
- Tags read from an optional classification.yaml at the brain root
  (wings:/repos: maps). Absent file → defaults-only, not an error.
- Defaulting: client-* → confidential; hyperguild/homelab → internal;
  everything else → confidential. Fail-safe-to-strictest is the
  load-bearing property: a missing tag never silently downgrades.
- Config.Derive(Target) is the function the use-case calls; Wing/Repo
  are the per-kind helpers. ParseLevel rejects unknown tokens; a bad
  level in the config file is a hard load error.

Central classification.yaml (not _index.md frontmatter, not gitea repo
topics): classifying a repo needs no live Gitea client, so #50 has no
dependency on the tracker work (#52); it's auditable in one place; and
it avoids BuildWingIndex clobbering a wing's regenerated _index.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:42:05 +02:00
mathias 2a595b5a92 docs(capture): make Q4 audit-sink-down posture classification-aware (#49)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
Reconsidered Q4: instead of one global degrade-and-warn, the posture now
inherits from effective classification (Q1):
- confidential + audit-sink-down -> hard-refuse (no buffer; removes the
  buffer-integrity question for confidential data)
- internal/public + audit-sink-down -> degrade-and-warn + durable local
  buffer + ntfy + reconcile-on-recovery
- floor (all tiers): refuse if nothing can record the audit
Updated the I5 Gherkin scenarios + obligations row to match. Couples Q4
to the Q1 classification spine -> one coherent sensitivity model.
2026-06-22 20:20:40 +00:00
mathias b7a2cc5fdf docs(capture): resolve §4 open questions into binding decisions (#49)
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Q1 classification trust: model (C) — caller declares, server cross-checks
  target tag, stricter wins, mismatch logged. Needs a classification
  taxonomy + per-wing/repo tags (prerequisite, sub-task of #49).
Q2 harness origin: server-derived from authenticated principal;
  context.harness is descriptive-only, never a gate input.
Q3 central relay: ships in v1 (needed for claude.ai/Crush/Pi/LLM Council)
  + I2 security-baseline ledger entry is v1 work.
Q4 audit-sink-down: degrade-and-warn + durable local buffer + reconcile
  on recovery; refuse only if NOTHING can record the audit.

Updated the I1 sovereignty + I5 auditability Gherkin scenarios to match;
added classification-mismatch and origin-spoofing scenarios.
2026-06-22 20:17:07 +00:00
mathias db638cca11 docs(capture): add use-case + BDD spec for the capture capability (#49)
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 3s
Pre-build spec for the uniform cross-harness capture path. Grounds the
design in the homelab invariants (I1-I5, now canonical on infra main):
- I1 sovereignty gate: refuse confidential capture via us-nexus harness
- I2: distributed-library form (no high-degree observer node) is
  admissible; central relay needs a security-baseline ledger entry
- I5: capture is a privileged write path, must emit audit records
Gherkin scenarios cover the happy path, the sovereignty refusal,
supersession + staleness discipline (#45/#47), fail-closed validation,
best-effort partial-failure receipts, dry-run, and fidelity supersession.
Four open questions flagged for review (classification trust is the
highest-risk one).
2026-06-22 17:25:03 +00:00
mathias 38579598e0 feat(skills): add close-session workflow skill
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Disciplined end-of-session closeout for Claude.ai chats: harvest →
ground-truth gitea → confirm issue actions → commit session summary →
brain orientation note → safe-to-archive verdict.

Finalized against current infra (git.d-ma.be, koala:30401 LiteLLM,
Authentik) and the brain_update/brain_get verbs. Phase 5 uses
supersede-by-slug + brain_get read-after-write with the batch
no-semantic-query-after-supersede discipline. session_log/brain_tunnel
inlined (now callable from claude.ai). Summary frontmatter is the
reduced live-capture schema with fidelity:live-capture to distinguish
from batch-export summaries.
2026-06-22 15:57:02 +00:00
mathias d6fa92b176 Merge pull request 'chore(context): drop unsupported cursor + aider adapters' (#48) from chore/drop-cursor-aider-adapters into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 09:03:59 +00:00
mathias 9173f9058d chore(context): remove generated .aider.conventions.md
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Aider is not a supported harness; the generator no longer emits this.
Note: generator no longer writes .aider.conf.yml either.
2026-06-22 09:01:39 +00:00
mathias 3e84a41fed chore(context): remove generated .cursorrules
Cursor is not a supported harness; the generator no longer emits this.
2026-06-22 09:01:29 +00:00
mathias 0e0571c7da chore(context): stop policing cursor/aider adapters in task check
Drop the context:sync:cursor task and remove .cursorrules +
.aider.conventions.md from the drift-guard file list in `check`, now
that the generator no longer emits them.
2026-06-22 09:01:19 +00:00
mathias 7a27cf71a2 chore(context): drop cursor + aider adapters from generator
Cursor and Aider are not supported harnesses. Remove generate_cursor
and generate_aider, their no-arg calls, and their case arms. The active
adapters are claude (CLAUDE.md), agents (AGENTS.md), and system-prompt
(.context/system-prompt.txt).
2026-06-22 09:00:55 +00:00
mathias 63df6d3283 Merge pull request 'feat: brain_update supersede verb + brain_get / brain_write read-after-write handle (#45)' (#46) from feat/brain-update-supersede into main
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 08:59:28 +00:00
mathiasandClaude Opus 4.8 f04b03e07e chore(context): re-sync derived adapters after root rule-0 update
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
context-sync regenerated the adapters from the updated root AGENT.md
(rule 0 pre-task ritual + TDD constraint). The committed adapters had
drifted; this is the documented `task check` remedy, not a content
change in this repo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:16 +02:00
mathiasandClaude Opus 4.8 6c61f93146 feat(mcp): register brain_update + brain_get, extend brain_write handle
Wires the #45 verbs into the MCP surface (all three sites: tools()
descriptors, handleCall dispatch, package doc comment).

- brain_update: supersede-by-slug or full path; rebuilds wing _index and
  re-tunnels cross-wing matches against the new body (idempotent,
  best-effort), re-indexes the graph, returns {id, path, content_hash,
  superseded}.
- brain_get: fetch by id or path (both are the brain-relative handle);
  returns {id, path, content_hash, frontmatter, body}.
- brain_write: return contract extended from {path} to {id, path,
  content_hash} — path kept for backward compat — so the create path
  also yields a stable handle.

id == relPath; content_hash == sha256 of the file bytes. Tests cover the
supersede happy path, missing-target error + no-create, get by id/path,
write handle, and an end-to-end re-embed test that drives the real
vectorstore.Sync re-index after an update.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:10 +02:00
mathiasandClaude Opus 4.8 95a69fc2c1 feat(brain): add UpdateNote/ReadNote supersede primitives + frontmatter editor
Implements the api-layer half of #45. UpdateNote supersedes a note in
place (whole-note body replace, frontmatter re-stamp: updated_at,
supersedes=prior content hash, supersede_reason), preserving created_at,
wing, hall, and any custom fields. Never creates — a missing target is
an error so callers fall back to brain_write. ReadNote is the read-after-
write primitive (frontmatter + body + content_hash). ContentHash is the
sha256 handle that round-trips write/update → get.

Frontmatter is edited via a line-preserving ordered editor rather than a
yaml.v3 round-trip, which would reorder keys and strip comments — the
brain writes flat key:value frontmatter by hand.

Embeddings are not refreshed here: the rewritten file's mtime advances,
which the out-of-band vectorstore.Sync ticker uses to re-embed it — the
same mechanism brain_write relies on.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:00 +02:00
mathiasandClaude Opus 4.8 bb8bc0478c chore(cd): use git.d-ma.be for SSH alias + INFRA_REPO (post-rename consistency)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:19:42 +02:00
mathiasandClaude Opus 4.8 a961a3c064 fix(vectorstore): hard-split oversized heading-less chunks
A doc with no headings and no blank-line paragraphs (JSON-lines, e.g.
wiki/telos/decisions/human-intent-column.md) survived both chunk passes whole
and was sent to nomic-embed over its context window → 'input length exceeds
the context length' (400, the steady embed errors=1). Add a final hard-split
pass (line then UTF-8 rune boundaries) so no chunk exceeds maxBytes. TDD.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:19:42 +02:00
mathiasandClaude Opus 4.8 bec28f9014 fix(cd): point registry/patch/verify refs at git.d-ma.be after host rename
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
The gitea.d-ma.be→git.d-ma.be rename moved the infra deployment manifests, but
cd.yml still sed-patched 'gitea.d-ma.be/mathias/ingestion:' — which no longer
matches, so the patch produced no change and 'git commit' failed under set -e,
wedging all deploys (image stuck at e8dbcf6). Repoint the registry image refs,
the infra-patch sed patterns, and the rollout-verify EXPECTED values to
git.d-ma.be (same registry backend). SSH alias + INFRA_REPO left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:22:07 +02:00
mathiasandClaude Opus 4.8 b62ac57382 fix(brain_answer): reranker is a filter, not a gate — fall back to BM25 when it keeps nothing
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
The Qwen3-Reranker is a web-search cross-encoder. Against conversational /
personal-intent queries (e.g. 'what am I optimizing toward?') it scores every
candidate as 'no', so brain_answer collapsed to 'No relevant content found'
even though BM25 had retrieved on-topic notes (incl. the telos wing). Treat
the reranker as a filter: when it keeps zero results, fall back to the
BM25/vector ordering (capped to the no-reranker depth of 10). Refs brain#11.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:53:02 +02:00
mathiasandClaude Opus 4.8 aa918388b9 docs(brain): add two-column intent merge scaffold
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Drop-in unified findings doc for the brain-MCP intent study. Locks the
shared row schema + closed intent vocab both columns must conform to.
Agent column filled from agent-intent-column.jsonl (46 acts, 37%
mismatch); human column left as PENDING cells + <<SYNTH>> blocks so the
Claude.ai-history analysis merges in without re-deriving structure.

Pre-seeds the cross-consumer divergence questions: agent mismatch is
write-side-heavy (supersede + verify-landed); hypothesis is human
mismatch is read-side-heavy (semantic + answer) — interface may fail the
two consumers at opposite ends.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:45:24 +02:00
mathiasandClaude Opus 4.8 e8dbcf6eef docs(brain): add agent-consumer brain-MCP intent analysis
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Agent-consumer column of the two-part brain intent↔interface study.
Reconstructs the knowledge-act behind every brain MCP call in the
Claude Code agent transcripts on koala (the only corpus with brain
calls; brain/sessions and agentsquad eval logs carry none).

46 distinct knowledge-acts, 37% interface mismatch, 0% intent_unclear.
Headline gap: no update/supersede verb → agents blind re-write same
slug (5x); no read-after-write → lexical re-query of own note (4x);
lexical-only reads → semantic-as-keyword-stuffing chains (3x);
brain_answer hedged with parallel brain_query.

Canonical schema brain-intent-extraction.md absent on host; vocab
reconstructed, every row tagged schema_source=reconstructed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 19:44:59 +02:00
mathiasandClaude Opus 4.8 0eeb1df4a2 fix(claudewatcher): scrub bare 1Password service-account tokens (ops_)
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
A ~/.zshrc read surfaced OP_SERVICE_ACCOUNT_TOKEN into a transcript. The
_TOKEN= form was already caught by homelab-env-token, but a bare ops_<b64>
value was not. Add an op-service-account rule (ordered early). Tests cover
both env-assigned and bare forms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 00:28:47 +02:00
mathiasandClaude Opus 4.8 9febb1bba1 fix(claudewatcher): harden secret scrubber against \b evasion + add JWT
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
A shell mangle that glued a key to a preceding word ('yes'+'sk-...') had
no word boundary, so the leading \b in the openai-sk rule failed to match
and a LiteLLM master key leaked past the scrubber into ingest (2026-06-11).

- openai-sk: drop leading \b, match sk- shape anywhere ({32,} floor keeps
  short task-/disk- words clean).
- add jwt rule for bare header.payload.sig tokens (no Bearer prefix).
- regression tests: the exact yessk- evasion, standalone sk-, bare JWT,
  plus clean-content guards for task-/disk-.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 08:55:33 +02:00
mathias 5dc247b994 fix(ci): quote "on" key so Gitea parses workflow triggers
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
Bare on: parses as YAML boolean true (Norway problem); Gitea then ignores the triggers and silently skips jobs. Quoting forces the string key.
2026-06-03 08:38:46 +02:00
mathias 2125558196 docs: extend harness boundary decision to cover Crush as third harness
CI / Mirror to GitHub (push) Successful in 4s
CI / Lint / Test / Vet (push) Successful in 12s
2026-05-28 11:38:36 +00:00
mathias 2beaac2feb docs: add 2026-05-28 decisions — harness boundary + field benchmark definition
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
2026-05-28 10:31:41 +00:00
mathias 525811bc1a docs: add hypothesis statement and harness boundary clarification
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-05-28 10:30:39 +00:00
mathias bad0581623 merge: client-name scrubber rule (refs hyperguild#27)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-05-26 07:10:05 +02:00
mathias a94b860c2e feat(claudewatcher): client-name guard via RegisterRule + env
Pre-rollout guard. Source code stays clean — client identities come
from CLAUDE_INGEST_CLIENT_BLOCK env (sourced from a SOPS-encrypted k8s
secret in infra repo). Env value is a regex alternation; main wraps
it with `(?i)\b(...)\b` so word-boundary matching avoids false hits
inside longer identifiers (e.g. "Sebastian" doesn't trigger on "SEB").

DefaultRules (credential shapes) still take precedence so any leak
that's BOTH a client mention AND a credential shape logs as the
credential — strictly more dangerous, points triage at the right
thing. Tests cover precedence + case variations + word-boundary
respect + invalid-pattern rejection.

Refs: infra#73 Track E.1 pre-rollout grill (option B).

Bump-Type: minor
2026-05-26 07:10:05 +02:00
mathias f8cf27e5de merge: claudewatcher (closes hyperguild#27, refs infra#73)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-05-25 19:59:13 +02:00
mathias 49b188e9c9 feat(server): wire claudewatcher behind CLAUDE_SESSIONS_DIR
Opt-in by setting CLAUDE_SESSIONS_DIR to the ~/.claude/projects path.
When set, the server starts claudewatcher.Watch in a goroutine that
ticks every CLAUDE_INGEST_INTERVAL seconds (default 60). Requires
BRAIN_PG_DSN for the cursor table — fail-fast if missing.

Each Batch becomes one wiki note at:
  brain/wiki/claude-sessions/facts/session-<host>-<session_id>.md

with frontmatter type=source + domain=<project basename>. Per-turn
content capped at 2000 chars (full transcripts stay in
~/.claude/projects already); the brain entry is a digest, not a
mirror.

CLAUDE_INGEST_HOST overrides the os.Hostname()-derived host label,
useful when multiple ingestion pods consume the same DSN from
different machines.

Closes hyperguild#27.

Bump-Type: minor
2026-05-25 19:59:07 +02:00
mathias bc011cc1f0 feat(claudewatcher): ingest Claude Code session transcripts into brain
New package internal/claudewatcher. The volume gate (24 turns/week of
agentsquad logs vs 500/week gate) exposed that the real signal lives
in daily Claude Code usage at ~/.claude/projects/*/<uuid>.jsonl, not
in agentsquad output. This package captures that signal. See infra#73
Track E + hyperguild#27 for the full reframe.

Components:
- parser: tolerant JSONL parser over the observed Claude Code session
  schema (user / assistant / attachment / system + bookkeeping types).
  Skip-flag fast-paths queue-operation, last-prompt, permission-mode,
  ai-title, bridge-session, file-history-snapshot.
- scrubber: 11-rule fail-closed regex set for credential shapes
  (bearer, postgres URIs, PEM, ssh-key, ghp_/sk-/sk-ant-/AKIA, homelab
  env tokens, SOPS markers). Drop turn + log on match.
- cursor: postgres-backed claude_session_cursors table, keyed by
  (host, file_path) with byte_offset. Resumable across pod restarts.
- watcher: poll loop. Walks SessionsDir, processes each .jsonl from
  its cursor offset, runs scrubber, emits a Batch per file to a
  Sink interface, advances cursor on successful Ingest.

No classifier integration in this commit — every kept turn is emitted
in a per-session batch. The cmd/server wiring (next commit) routes
batches to brain/wiki/claude-sessions/facts/. Classifier-driven hall
routing (decisions / failures / hypotheses) is a follow-up.

19 unit tests across parser + scrubber + watcher. task check green.

Refs: infra#73, hyperguild#27
2026-05-25 19:58:58 +02:00
mathiasandClaude Opus 4.7 2726896079 feat(mcp): wire brain_context tool
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
Returns top-N relevant brain entries for a project context. Combines
BM25 hits on project name with 2-hop graph expansion via Track A's
graphstore (when BRAIN_GRAPH_ENABLED). Closes hyperguild#28.

Notes on implementation choices that deviate slightly from the spec:
- Excerpt length: 200 chars per spec (vs the 300 used by search.Result).
  truncateExcerpt clamps the already-stripped BM25 excerpt; graph-only
  neighbours load their excerpt from disk via a private readExcerpt
  helper (search.hydrate is unexported).
- Graph scoring: 0.6 / max(1, distance) per neighbour, so distance-1
  contributes 0.6 and distance-2 contributes 0.3. BM25 hits decay
  linearly from 3.0 (rank-0) to 1.0 (rank-2), giving BM25 hits a
  natural ceiling above pure-graph hits while still letting a doc
  surfaced via both edge types outrank a BM25-only one.
- Test placement: package mcp (internal) rather than mcp_test, because
  graphReader is unexported and WithGraph only accepts *PGStore; an
  internal test can install a dual-interface fake directly on s.graph
  without spinning up postgres.

Bump-Type: minor

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 18:53:14 +02:00
mathias 2b7bbe38c7 docs(eval): record M4 + M4b scorer runs — phase 2 gate cleared (infra#72)
CI / Lint / Test / Vet (push) Successful in 11s
CI / Mirror to GitHub (push) Successful in 4s
Tier-weighted retrieval against the qa-2026-05.md 20-question set:

| run                            | top-1 | top-3 |
|--------------------------------|-------|-------|
| baseline (pre-phase-1)         | 20%   | 65%   |
| post phase 1 (parser+content)  | 20%   | 70%   |
| post M4 (tier weighting)       | 30%   | 75%   |
| post M4b (entities → K tier)   | 35%   | 80%   |

Net Phase 2 lift: +15pt top-1, +15pt top-3 — comfortably above the
≥10pt close-gate set in infra#72.

Three remaining misses are content-keyword issues, not structure
issues (the questions don't share enough lexical surface with the
target entries to surface via BM25 alone). Vector search would
help here but the iguana embedder is off-mesh (see infra#64).
2026-05-25 18:51:29 +02:00
mathias 1b00cbc0ae fix(search,graph): M4b wiki/entities/ → tier=knowledge
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
Initial M4 mapping put wiki/entities/* in tier=note. Post-M4 eval
regressed qwen35-9b-fast from rank 2 → off top-5: knowledge entries
that cite the entity in passing now outscore the entity page itself
(1.5× weight vs 1.0×).

Entity anchor pages are durable facts about concrete things — they
map cleanly to the knowledge/facts/ slot in the post-M3 layout
target. Promote them now so the path inference matches.

Eval re-run after deploy is in infra#72.
2026-05-25 18:49:37 +02:00
mathias 4f78fecd06 feat(search): M4 tier-weighted BM25 re-rank (infra#72)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
The eval set under brain/eval/qa-2026-05.md showed BM25 top-1 at 20%
with 5 of the missing slugs being short focused knowledge entries
that lost to long aggregate docs on raw term-frequency. Tier weighting
addresses that without touching the BM25 algorithm itself.

How

- Result struct gains a Tier field, populated during the file walk
  via extractTier (frontmatter wins, path prefix as fallback —
  mirrors the graph.inferTierFromPath logic so the two callers stay
  in lockstep).
- After the existing sort (and optional hybridMerge), do a final
  stable re-sort by float64(Score) * tierWeight(Tier). Knowledge
  ×1.5, note ×1.0, inbox ×0.3, unknown ×1.0.
- hydrate() (vector-only hits) also fills Tier so re-ranking covers
  the hybrid path.

Test covers the load-bearing case: a long note-tier doc with raw=10
loses to a short knowledge-tier doc with raw=8 after weighting
(8×1.5=12 vs 10×1.0=10).

Measurement gate is in infra#72: re-run brain/eval/score.py against
the live brain after this image lands; close the issue when top-1
hit rate lifts by ≥10 absolute points.
2026-05-25 18:45:20 +02:00
mathias d5f112b600 feat(graph,graphstore): M2 parse tier+topic from frontmatter, persist via Upsert (infra#72)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
extract.go now reads `tier:` and `topic:` from YAML frontmatter, with
a path-based fallback when frontmatter is absent (the pre-M3 state on
every existing entry):

  knowledge/* → tier=knowledge
  notes/*     → tier=note
  wiki/**     → tier=note   (sources + concepts + entities are I-level)
  inbox/**, raw/**, sessions/**, clips/** → tier=inbox

Frontmatter wins when present — covers the M3-migrated case where an
entry's path may not match the tier the author chose for it.

UpsertEntity persists both columns. M1's schema already has them.

Backfill on next pod start populates tier for the whole corpus
without any file moves; M3 will follow up with the actual layout
migration and explicit frontmatter writes.
2026-05-25 12:35:38 +02:00