mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

capture #49e: I5 audit path + classification-aware degradation (loki + buffer + reconcile) #54

New Issue
Closed
opened 2026-06-22 20:27:44 +00:00 by mathias · 1 comment
mathias commented 2026-06-22 20:27:44 +00:00
Owner
Copy Link
Copy Source

Parent: #49 · Spec: specs/capture-bdd-spec.md §2 (I5), §4.4 · Depends on: #50 (classification), #53 (the adapter it instruments) · Independent of: #49f (relay)

Scope

The I5 audit path with Q4 classification-aware degradation. Needed by any capture deployment, including direct-REST/library-only — so it's split from the relay (#49f) and can land before it. After this, direct-REST capture (CLI/Agentsquad) is fully I5-compliant.

What to build (per §4.4, classification-aware)

  1. AuditSink impl → alloy/loki. Every capture emits a request-level audit record: actor/principal, server-derived harness origin, effective classification, items written (ids/numbers), timestamp, outcome.
  2. Classification-aware degradation:
    • Confidential + central sink unreachable → hard-refuse the capture (no buffer, no proceed). Confidential must be centrally auditable at write time.
    • Internal/public + central sink unreachable → degrade-and-warn: write the audit record to a durable local buffer, fire an ntfy alert, flag the receipt as locally-buffered.
    • Floor (all tiers): if neither central sink nor (for internal/public) the local buffer can be written → refuse. No tier writes wholly un-audited.
  3. Reconciliation: buffered (internal/public) records replay to the central sink on recovery; local buffer cleared only after confirmed central write.
  4. Buffer durability: the local buffer survives process restart (file/embedded-store, not in-memory).

Acceptance criteria

  • AuditSink writes request-level records to loki with the full field set
  • Confidential + sink-down → capture refused (hard, no buffer)
  • Internal/public + sink-down → proceeds, audit buffered locally, ntfy fired, receipt flagged
  • Floor: nothing-can-record → refuse (all tiers)
  • Reconciliation replays buffered records on recovery; buffer cleared only after confirmed central write
  • Buffer is durable across restart
  • Tests: confidential-sink-down-refused, internal-sink-down-buffered, reconcile-on-recovery, floor-refuse, buffer-survives-restart
  • task check green

Notes

Couples to #50's classification (the degradation tier is the effective classification) and instruments #53's adapter. The buffer/reconcile machinery is internal/public-tier only — confidential is a plain refuse, which is why the build is smaller than a global-buffer design. Gherkin scenarios this makes live: the entire I5 classification-aware audit block.

**Parent:** #49 · **Spec:** `specs/capture-bdd-spec.md` §2 (I5), §4.4 · **Depends on:** #50 (classification), #53 (the adapter it instruments) · **Independent of:** #49f (relay) ## Scope The I5 audit path with Q4 classification-aware degradation. Needed by **any** capture deployment, including direct-REST/library-only — so it's split from the relay (#49f) and can land before it. After this, direct-REST capture (CLI/Agentsquad) is fully I5-compliant. ## What to build (per §4.4, classification-aware) 1. **`AuditSink` impl → alloy/loki.** Every capture emits a request-level audit record: actor/principal, server-derived harness origin, effective classification, items written (ids/numbers), timestamp, outcome. 2. **Classification-aware degradation:** - **Confidential + central sink unreachable → hard-refuse** the capture (no buffer, no proceed). Confidential must be centrally auditable at write time. - **Internal/public + central sink unreachable → degrade-and-warn:** write the audit record to a **durable local buffer**, fire an **ntfy alert**, flag the receipt as locally-buffered. - **Floor (all tiers):** if neither central sink nor (for internal/public) the local buffer can be written → refuse. No tier writes wholly un-audited. 3. **Reconciliation:** buffered (internal/public) records replay to the central sink on recovery; local buffer cleared only after confirmed central write. 4. **Buffer durability:** the local buffer survives process restart (file/embedded-store, not in-memory). ## Acceptance criteria - [ ] `AuditSink` writes request-level records to loki with the full field set - [ ] Confidential + sink-down → capture refused (hard, no buffer) - [ ] Internal/public + sink-down → proceeds, audit buffered locally, ntfy fired, receipt flagged - [ ] Floor: nothing-can-record → refuse (all tiers) - [ ] Reconciliation replays buffered records on recovery; buffer cleared only after confirmed central write - [ ] Buffer is durable across restart - [ ] Tests: confidential-sink-down-refused, internal-sink-down-buffered, reconcile-on-recovery, floor-refuse, buffer-survives-restart - [ ] `task check` green ## Notes Couples to #50's classification (the degradation tier is the effective classification) and instruments #53's adapter. The buffer/reconcile machinery is **internal/public-tier only** — confidential is a plain refuse, which is why the build is smaller than a global-buffer design. Gherkin scenarios this makes live: the entire I5 classification-aware audit block.
mathias commented 2026-06-22 21:55:10 +00:00
Author
Owner
Copy Link
Copy Source

Implemented — PR #60 (open, base main)

PR: #60 · branches from main (#53 merged).

Two-phase port (the load-bearing design): Reserve(level) runs before any write and refuses (confidential+down, or floor) / grants central / grants buffered; Record(entry, outcome) runs after. Makes "refuse before any write" literally true even though the record lists what landed.

audit.DegradingSink (§4.4): central up → central; down+confidential → refuse (no buffer); down+internal/public+buffer-writable → buffer+ntfy+receipt flag; down+buffer-unwritable → floor refuse.

Components: FileBuffer (durable JSONL, survives restart, confirm-then-clear per record), LokiCentral (/ready + /loki/api/v1/push), NtfyNotifier (token never logged), Reconcile/StartReconcile (replay on recovery, failed push keeps record buffered). SlogSink → two-phase, default when no loki.

Wiring: buildAuditSink picks DegradingSink (loki+buffer+ntfy+reconcile) when BRAIN_LOKI_URL set, else slog. ErrAuditUnavailable → HTTP 503. Env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN, BRAIN_AUDIT_RECONCILE_INTERVAL.

All ACs met (incl. confidential-refuse, internal-buffer, floor-refuse, reconcile, survives-restart); task check green.

⚠️ For #55 (k3s manifest, I3): the durable buffer is a file at <brain>/.audit-buffer/capture.jsonl. In k8s it must be on a persistent volume, else "durable across restart" fails on pod reschedule. Flag for the relay manifest.

Brain: wiki/hyperguild/decisions/two-phase-reserve-record-audit-gate.

Milestone: #50–#54 complete capture for direct-REST harnesses, fully I1+I5-compliant. Only #55 remains (central relay + I2 security-baseline ledger — heaviest security weight; ledger must merge before relay ships).

## Implemented — PR #60 (open, base main) **PR:** https://git.d-ma.be/mathias/hyperguild/pulls/60 · branches from `main` (#53 merged). **Two-phase port** (the load-bearing design): `Reserve(level)` runs before any write and refuses (confidential+down, or floor) / grants central / grants buffered; `Record(entry, outcome)` runs after. Makes "refuse before any write" literally true even though the record lists what landed. **`audit.DegradingSink` (§4.4):** central up → central; down+confidential → refuse (no buffer); down+internal/public+buffer-writable → buffer+ntfy+receipt flag; down+buffer-unwritable → floor refuse. **Components:** `FileBuffer` (durable JSONL, survives restart, confirm-then-clear per record), `LokiCentral` (/ready + /loki/api/v1/push), `NtfyNotifier` (token never logged), `Reconcile`/`StartReconcile` (replay on recovery, failed push keeps record buffered). `SlogSink` → two-phase, default when no loki. **Wiring:** `buildAuditSink` picks DegradingSink (loki+buffer+ntfy+reconcile) when `BRAIN_LOKI_URL` set, else slog. `ErrAuditUnavailable` → HTTP 503. Env: `BRAIN_LOKI_URL`, `BRAIN_NTFY_URL`, `BRAIN_NTFY_TOKEN`, `BRAIN_AUDIT_RECONCILE_INTERVAL`. All ACs met (incl. confidential-refuse, internal-buffer, floor-refuse, reconcile, survives-restart); `task check` green. **⚠️ For #55 (k3s manifest, I3):** the durable buffer is a file at `<brain>/.audit-buffer/capture.jsonl`. In k8s it **must be on a persistent volume**, else "durable across restart" fails on pod reschedule. Flag for the relay manifest. Brain: `wiki/hyperguild/decisions/two-phase-reserve-record-audit-gate`. **Milestone:** #50–#54 complete capture for direct-REST harnesses, fully I1+I5-compliant. Only **#55** remains (central relay + I2 security-baseline ledger — heaviest security weight; ledger must merge before relay ships).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#54