feat: I5 audit path + classification-aware degradation (#54, capture 49e) #60

Merged
mathias merged 3 commits from feat/capture-audit-degradation into main 2026-06-22 21:57:52 +00:00
Owner

Implements #54 (capture 49e) — the I5 audit path with Q4 classification-aware degradation. After this, direct-REST capture (CLI/Agentsquad) is fully I5-compliant. Branches from main (#53 merged).

Two-phase audit port (the load-bearing design)

Splitting the port into Reserve (before any write) + Record (after) is what makes "confidential + sink-down → refuse before any write" literally true, even though the audit record — which lists what landed — can only be written afterwards.

  • Reserve(ctx, level) (AuditOutcome, error)error ⇒ refuse the capture before writing.
  • Record(ctx, entry, outcome) — persist per the reserved outcome.

The service honours Reserve's verdict; the tier→behaviour decision lives in the sink.

audit.DegradingSink (§4.4)

Central Tier Reserve verdict
up any AuditCentral
down confidential refuse (no buffer)
down internal/public + buffer writable AuditBuffered
down buffer unwritable refuse (floor)

Record on AuditBuffered appends to the durable buffer + fires an ntfy alert.

Components

  • FileBuffer — durable JSONL, survives process restart; Confirm rewrites without a record, so a buffered record clears only after its central write is confirmed.
  • LokiCentral/ready probe + /loki/api/v1/push (full audit entry as the structured line).
  • NtfyNotifier — degraded-state alerts; token only in the auth header, never logged (TestNtfyDoesNotLeakTokenOnError).
  • Reconcile / StartReconcile — replay buffered → central on recovery, confirm-then-clear per record; a failed push keeps the record buffered (no loss).
  • SlogSink updated to the two-phase shape (always central, never fails) — the default when BRAIN_LOKI_URL is unset.

Wiring (main)

buildAuditSink: DegradingSink (loki + file buffer under <brain>/.audit-buffer/ + optional ntfy) + reconcile loop when BRAIN_LOKI_URL is set; else SlogSink. capture.ErrAuditUnavailableHTTP 503.
New env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN, BRAIN_AUDIT_RECONCILE_INTERVAL (default 60s).

Acceptance criteria

  • AuditSink writes request-level records to loki with the full field set
  • Confidential + sink-down → refused (hard, no buffer)
  • Internal/public + sink-down → proceeds, audit buffered locally, ntfy fired, receipt flagged (AuditBuffered)
  • Floor: nothing-can-record → refuse (all tiers)
  • Reconciliation replays buffered records on recovery; buffer cleared only after confirmed central write
  • Buffer durable across restart (TestFileBufferSurvivesRestart)
  • Tests: confidential-sink-down-refused, internal-sink-down-buffered, reconcile-on-recovery, floor-refuse, buffer-survives-restart (+ loki/ntfy httptest, push-fail-keeps-buffered, 503 mapping)
  • task check green; secret-handling respected

Deferred

  • k3s manifest (I3) → infra (#55). The buffer path is a brain-dir file; in k8s it needs a volume — note for the #55 manifest.

Not auto-merged — review + merge from claude.ai.

Closes #54

🤖 Generated with Claude Code

Implements #54 (capture 49e) — the I5 audit path with Q4 classification-aware degradation. After this, direct-REST capture (CLI/Agentsquad) is fully I5-compliant. Branches from `main` (#53 merged). ## Two-phase audit port (the load-bearing design) Splitting the port into **Reserve (before any write) + Record (after)** is what makes "confidential + sink-down → refuse *before any write*" literally true, even though the audit record — which lists what landed — can only be written afterwards. - `Reserve(ctx, level) (AuditOutcome, error)` — `error` ⇒ refuse the capture before writing. - `Record(ctx, entry, outcome)` — persist per the reserved outcome. The service honours Reserve's verdict; the tier→behaviour decision lives in the sink. ## `audit.DegradingSink` (§4.4) | Central | Tier | Reserve verdict | |---|---|---| | up | any | `AuditCentral` | | down | confidential | **refuse** (no buffer) | | down | internal/public + buffer writable | `AuditBuffered` | | down | buffer unwritable | **refuse (floor)** | Record on `AuditBuffered` appends to the durable buffer + fires an ntfy alert. ## Components - **`FileBuffer`** — durable JSONL, **survives process restart**; `Confirm` rewrites without a record, so a buffered record clears **only after its central write is confirmed**. - **`LokiCentral`** — `/ready` probe + `/loki/api/v1/push` (full audit entry as the structured line). - **`NtfyNotifier`** — degraded-state alerts; token only in the auth header, never logged (`TestNtfyDoesNotLeakTokenOnError`). - **`Reconcile` / `StartReconcile`** — replay buffered → central on recovery, confirm-then-clear per record; a failed push **keeps the record buffered (no loss)**. - `SlogSink` updated to the two-phase shape (always central, never fails) — the default when `BRAIN_LOKI_URL` is unset. ## Wiring (`main`) `buildAuditSink`: `DegradingSink` (loki + file buffer under `<brain>/.audit-buffer/` + optional ntfy) + reconcile loop when `BRAIN_LOKI_URL` is set; else `SlogSink`. `capture.ErrAuditUnavailable` → **HTTP 503**. New env: `BRAIN_LOKI_URL`, `BRAIN_NTFY_URL`, `BRAIN_NTFY_TOKEN`, `BRAIN_AUDIT_RECONCILE_INTERVAL` (default 60s). ## Acceptance criteria - [x] `AuditSink` writes request-level records to loki with the full field set - [x] Confidential + sink-down → refused (hard, no buffer) - [x] Internal/public + sink-down → proceeds, audit buffered locally, ntfy fired, receipt flagged (`AuditBuffered`) - [x] Floor: nothing-can-record → refuse (all tiers) - [x] Reconciliation replays buffered records on recovery; buffer cleared only after confirmed central write - [x] Buffer durable across restart (`TestFileBufferSurvivesRestart`) - [x] Tests: confidential-sink-down-refused, internal-sink-down-buffered, reconcile-on-recovery, floor-refuse, buffer-survives-restart (+ loki/ntfy httptest, push-fail-keeps-buffered, 503 mapping) - [x] `task check` green; secret-handling respected ## Deferred - **k3s manifest (I3)** → infra (#55). The buffer path is a brain-dir file; in k8s it needs a volume — note for the #55 manifest. Not auto-merged — review + merge from claude.ai. Closes #54 🤖 Generated with [Claude Code](https://claude.com/claude-code)
mathias added 3 commits 2026-06-22 21:54:44 +00:00
Splits the audit port into Reserve (before any write) + Record (after),
so "confidential + sink-down → refuse before any write" is literally true
even though the audit record — which lists what landed — can only be
written afterwards.

- AuditSink.Reserve(ctx, level) → AuditOutcome | error. The error path
  refuses the capture before writing: confidential + central sink down,
  or the all-tiers floor (nothing can record).
- AuditSink.Record(ctx, entry, outcome) persists per the reserved outcome.
- Service: I5 gate runs after the I1 gate and after the dry-run
  short-circuit (dry-run never probes the sink). AuditBuffered surfaces on
  the receipt. New ErrAuditUnavailable sentinel (→ HTTP 503).

The tier→behaviour decision lives in the sink impl (#54's DegradingSink),
not the service — the service just honours Reserve's verdict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The classification-aware I5 audit path (§4.4):

- DegradingSink.Reserve: central up → AuditCentral; central down +
  confidential → refuse (no buffer); central down + internal/public +
  buffer writable → AuditBuffered; central down + buffer unwritable →
  floor refuse. Record executes the reserved outcome and, when buffered,
  fires an ntfy alert.
- FileBuffer: durable JSONL buffer that survives process restart; Confirm
  rewrites the file without a record, so a buffered record is cleared ONLY
  after its central write is confirmed.
- LokiCentral: /ready probe + /loki/api/v1/push (full audit entry as the
  structured line). NtfyNotifier: degraded-state alerts; token only in the
  auth header, never logged (regression-tested).
- Reconcile + StartReconcile: replay buffered records to central on
  recovery, confirm-then-clear per record; a failed push keeps the record
  buffered (no loss). SlogSink updated to the two-phase shape (always
  central, never fails) — the default when no loki endpoint is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
feat(capturehttp): 503 on audit-unavailable; wire degrading sink (#54)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
38a2e91002
- Map capture.ErrAuditUnavailable → HTTP 503 (audit substrate down /
  confidential unauditable / floor).
- main: buildAuditSink selects the DegradingSink (loki central + durable
  file buffer under brainDir + optional ntfy) when BRAIN_LOKI_URL is set
  and starts the reconcile loop; else the plain slog sink. Notifier kept
  as a nil interface (not typed-nil) when unconfigured so the sink and
  reconcile skip it cleanly.

Env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN,
BRAIN_AUDIT_RECONCILE_INTERVAL (default 60s). Buffer at
<brain>/.audit-buffer/capture.jsonl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mathias merged commit c307b72bd5 into main 2026-06-22 21:57:52 +00:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#60