mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat: uniform capture capability — REST+OAuth2 spine for session insight/ticket capture across all harnesses #49

New Issue
Closed
opened 2026-06-22 17:11:14 +00:00 by mathias · 8 comments
mathias commented 2026-06-22 17:11:14 +00:00
Owner
Copy Link
Copy Source

Intent

One uniform capability for capturing a session's valuable output — insights → brain, action items → Gitea tickets, optional summary → ai-sessions — callable identically from every environment Mathias works in: claude.ai (Chat/Cowork/Code/Design), Claude Code CLI (flamingo/koala/iguana), Crush, Pi, LLM Council (Open WebUI), Agentsquad executor/reviewer, or any future stack.

The goal is a single "command" with the same core logic everywhere and acceptable per-harness quirks only in how the args are assembled, never in what persistence does.

Architecture (decided 2026-06-22)

REST + OAuth2 is the uniform spine. SKILL is the trigger/harvest veneer. MCP is optional sugar. Rationale: REST over authed HTTP is reachable from literally every environment (superset of MCP coverage), stateless, robust, and reuses the existing Authentik/JWT auth. MCP coverage is uneven (great on claude.ai/Crush/Open WebUI, iffy headless) and flakier. So MCP is a convenience wrapper, never the only door.

Layering follows Clean Architecture / idiomatic Go / PragProg-DRY:

  • Use-case (the logic, written once): CaptureService.Capture(ctx, CaptureInput) (CaptureReceipt, error) in a new ingestion/internal/capture/ package. Depends only on ports. Owns: validation, the brain write/supersede + read-after-write discipline, ticket routing, summary write, partial-failure receipt assembly.
  • Ports (interfaces):
    • BrainStore — Write, Update, Get (the #45 verbs).
    • IssueTracker — CreateIssue, CloseIssue, CommentIssue.
    • SummaryWriter — WriteFile(repo, path, content).
  • Adapters (in): POST /capture (REST, primary) and later an MCP capture tool — both ~10-line forwarders to the same use-case.
  • Entities: Insight, Ticket, Summary, CaptureContext — plain structs.

Two ground-truth findings that scope this honestly

Read of ingestion/internal/api/handler.go + ingestion/internal/mcp/:

  1. The brain server has NO gitea client today. It does brain-local file ops only (WriteNote, pipeline, search) and knows nothing about Gitea. The IssueTracker port is a new outbound dependency the brain server must gain — not a wrap of existing code. Decide deliberately: brain server reaches out to Gitea directly (new HTTP client + token), OR capture lives in a small new service that composes brain + gitea. (Filing leans: add the client to the brain server, injected, behind the IssueTracker interface so it's swappable/testable.)
  2. brain_update/brain_get live in the MCP layer, not the REST/api layer. Only /write is exposed over REST today (WriteNote is a clean free function; update/get are MCP-only). The capture use-case needs update/get reachable from REST too. The right move is to extract the brain write/update/get logic into the shared BrainStore implementation that both the MCP handlers and the new REST/capture adapter call — i.e. a small refactor lifting #45's logic out of mcp/ into a layer both adapters share. This is the Clean-Arch payoff but it is real work, not a wrapper.

POST /capture contract

Auth: Bearer OAuth2 JWT (reuse existing middleware).

{
  "context": {
    "harness":     "claudeai-chat|claude-code|crush|pi|llm-council|agentsquad|...",
    "session_ref": "<chat uuid | transcript path | run id>",   // optional
    "fidelity":    "live-capture|transcript-parse|agent-runlog",
    "actor":       "mathias|<agent-id>"
  },
  "insights": [
    { "text": "...", "wing": "hyperguild", "hall": "decisions",
      "supersede_slug": "optional-prior-slug" }  // present ⇒ brain_update, absent ⇒ brain_write
  ],
  "tickets": [
    { "repo": "hyperguild", "action": "create|close|comment",
      "number": 47, "title": "...", "body": "..." }
  ],
  "summary": { "title": "...", "body": "...", "repos_touched": ["hyperguild"] },  // optional
  "dry_run": false
}

Behaviour (use-case, server-side, once):

  1. Validate the whole request before any write (fail-closed; bad wing/hall/repo → 400, nothing written).
  2. Insights → Update if supersede_slug else Write; capture {id, content_hash}. The #45 read-after-write + batch staleness discipline live HERE so no client ever has to know the rule (the DRY win).
  3. Tickets → create/close/comment via IssueTracker; owner always mathias.
  4. Summary (if present) → write to ai-sessions at summaries/<harness>/<YYYY-MM>/<date>-<slug>-<ref8>.md, stamping fidelity in frontmatter. Collision rule generalizes: richer fidelity supersedes thinner for the same session_ref (transcript-parse > live-capture).
  5. Receipt → structured, partial-aware (see below).

Failure semantics: best-effort + partial receipt (decided). If a ticket files but a brain write fails, do NOT roll back — return exactly what landed. The server owns partial-failure handling once; no client gets rollback logic.

{
  "insights": [ {"id":"...","path":"...","content_hash":"...","superseded":false,"ok":true} ],
  "tickets":  [ {"repo":"...","number":47,"action":"create","url":"...","ok":true} ],
  "summary":  {"path":"...","ok":true},
  "errors":   [ {"item":"insight[1]","error":"..."} ]   // per-item; partial success explicit
}

HTTP: 200 if all ok, 207-style body when partial (all-failed → 4xx/5xx as appropriate). dry_run: true validates + returns the would-be receipt, writes nothing (mirror existing /ingest dry-run).

Acceptance criteria (REST + use-case phase — this issue)

  • ingestion/internal/capture/ package with CaptureService + ports (BrainStore, IssueTracker, SummaryWriter) + entities
  • Brain write/update/get logic extracted into a shared BrainStore impl callable from BOTH the MCP handlers and capture (refactor #45's logic out of mcp/-only)
  • Gitea IssueTracker client added to the brain server, injected behind the interface, token via env (no secret in argv/logs — see AGENTS.md secret-handling)
  • POST /capture adapter: JWT-authed, decode → use-case → encode, thin
  • Validate-before-write (no partial write on a malformed request)
  • Best-effort execution + partial receipt with per-item ok/error
  • fidelity stamped into summary frontmatter; summaries/<harness>/... path
  • dry_run path writes nothing, returns would-be receipt
  • Tests: validation-rejects-bad-request, write+supersede insight, create/close/comment ticket, summary write, partial-failure receipt (one item fails others land), dry-run
  • task check green; routes registered; secret-handling respected

Follow-ups (separate issues, after this ships)

  • MCP capture tool — thin forwarder to CaptureService, for MCP-native clients that prefer it.
  • SKILL veneer(s) — the close-session SKILL (already at skills/close-session/SKILL.md) becomes the claude.ai trigger/harvest layer that POSTs /capture; thin equivalents (or direct programmed calls) for Crush/Pi.
  • OAuth2 token provisioning per environment — CLI client-credential, claude.ai OAuth session, headless service token. Bounded per-harness quirk.
  • Harvest adapters — transcript-parse (reuse ai-sessions extractors) vs chat-memory-reconstruct vs agent-runlog; each assembles /capture args at its own fidelity.

Related

  • #45 — brain_update/brain_get (the verbs this reuses; needs the extract refactor)
  • #47 — vector staleness (the discipline capture encodes server-side so clients don't carry it)
  • #43 / ADR-0009 (infra) — intent-named-verb interface direction; capture is the act-named verb
  • skills/close-session/SKILL.md — becomes the claude.ai veneer over this
  • ai-sessions extract_*_sessions.py — the harvest adapters for CLI/Crush/Pi
## Intent One **uniform capability** for capturing a session's valuable output — insights → brain, action items → Gitea tickets, optional summary → ai-sessions — callable identically from **every** environment Mathias works in: claude.ai (Chat/Cowork/Code/Design), Claude Code CLI (flamingo/koala/iguana), Crush, Pi, LLM Council (Open WebUI), Agentsquad executor/reviewer, or any future stack. The goal is a single "command" with the **same core logic everywhere** and acceptable per-harness quirks only in *how the args are assembled*, never in *what persistence does*. ## Architecture (decided 2026-06-22) **REST + OAuth2 is the uniform spine. SKILL is the trigger/harvest veneer. MCP is optional sugar.** Rationale: REST over authed HTTP is reachable from literally every environment (superset of MCP coverage), stateless, robust, and reuses the existing Authentik/JWT auth. MCP coverage is uneven (great on claude.ai/Crush/Open WebUI, iffy headless) and flakier. So MCP is a convenience wrapper, never the only door. Layering follows Clean Architecture / idiomatic Go / PragProg-DRY: - **Use-case (the logic, written once):** `CaptureService.Capture(ctx, CaptureInput) (CaptureReceipt, error)` in a new `ingestion/internal/capture/` package. Depends only on ports. Owns: validation, the brain write/supersede + read-after-write discipline, ticket routing, summary write, partial-failure receipt assembly. - **Ports (interfaces):** - `BrainStore` — `Write`, `Update`, `Get` (the #45 verbs). - `IssueTracker` — `CreateIssue`, `CloseIssue`, `CommentIssue`. - `SummaryWriter` — `WriteFile(repo, path, content)`. - **Adapters (in):** `POST /capture` (REST, primary) and later an MCP `capture` tool — both ~10-line forwarders to the same use-case. - **Entities:** `Insight`, `Ticket`, `Summary`, `CaptureContext` — plain structs. ## Two ground-truth findings that scope this honestly Read of `ingestion/internal/api/handler.go` + `ingestion/internal/mcp/`: 1. **The brain server has NO gitea client today.** It does brain-local file ops only (`WriteNote`, pipeline, search) and knows nothing about Gitea. The `IssueTracker` port is a **new outbound dependency** the brain server must gain — not a wrap of existing code. Decide deliberately: brain server reaches out to Gitea directly (new HTTP client + token), OR capture lives in a small new service that composes brain + gitea. (Filing leans: add the client to the brain server, injected, behind the `IssueTracker` interface so it's swappable/testable.) 2. **`brain_update`/`brain_get` live in the MCP layer, not the REST/api layer.** Only `/write` is exposed over REST today (`WriteNote` is a clean free function; update/get are MCP-only). The capture use-case needs update/get reachable from REST too. The right move is to **extract the brain write/update/get logic into the shared `BrainStore` implementation** that both the MCP handlers and the new REST/capture adapter call — i.e. a small refactor lifting #45's logic out of `mcp/` into a layer both adapters share. This is the Clean-Arch payoff but it is real work, not a wrapper. ## `POST /capture` contract Auth: Bearer OAuth2 JWT (reuse existing middleware). ```jsonc { "context": { "harness": "claudeai-chat|claude-code|crush|pi|llm-council|agentsquad|...", "session_ref": "<chat uuid | transcript path | run id>", // optional "fidelity": "live-capture|transcript-parse|agent-runlog", "actor": "mathias|<agent-id>" }, "insights": [ { "text": "...", "wing": "hyperguild", "hall": "decisions", "supersede_slug": "optional-prior-slug" } // present ⇒ brain_update, absent ⇒ brain_write ], "tickets": [ { "repo": "hyperguild", "action": "create|close|comment", "number": 47, "title": "...", "body": "..." } ], "summary": { "title": "...", "body": "...", "repos_touched": ["hyperguild"] }, // optional "dry_run": false } ``` **Behaviour (use-case, server-side, once):** 1. Validate the whole request before any write (fail-closed; bad wing/hall/repo → 400, nothing written). 2. Insights → `Update` if `supersede_slug` else `Write`; capture `{id, content_hash}`. **The #45 read-after-write + batch staleness discipline live HERE** so no client ever has to know the rule (the DRY win). 3. Tickets → create/close/comment via `IssueTracker`; owner always `mathias`. 4. Summary (if present) → write to `ai-sessions` at `summaries/<harness>/<YYYY-MM>/<date>-<slug>-<ref8>.md`, stamping `fidelity` in frontmatter. Collision rule generalizes: richer fidelity supersedes thinner for the same `session_ref` (transcript-parse > live-capture). 5. Receipt → structured, partial-aware (see below). **Failure semantics: best-effort + partial receipt** (decided). If a ticket files but a brain write fails, do NOT roll back — return exactly what landed. The server owns partial-failure handling once; no client gets rollback logic. ```jsonc { "insights": [ {"id":"...","path":"...","content_hash":"...","superseded":false,"ok":true} ], "tickets": [ {"repo":"...","number":47,"action":"create","url":"...","ok":true} ], "summary": {"path":"...","ok":true}, "errors": [ {"item":"insight[1]","error":"..."} ] // per-item; partial success explicit } ``` HTTP: 200 if all ok, 207-style body when partial (all-failed → 4xx/5xx as appropriate). `dry_run: true` validates + returns the would-be receipt, writes nothing (mirror existing `/ingest` dry-run). ## Acceptance criteria (REST + use-case phase — this issue) - [ ] `ingestion/internal/capture/` package with `CaptureService` + ports (`BrainStore`, `IssueTracker`, `SummaryWriter`) + entities - [ ] Brain write/update/get logic extracted into a shared `BrainStore` impl callable from BOTH the MCP handlers and capture (refactor #45's logic out of `mcp/`-only) - [ ] Gitea `IssueTracker` client added to the brain server, injected behind the interface, token via env (no secret in argv/logs — see AGENTS.md secret-handling) - [ ] `POST /capture` adapter: JWT-authed, decode → use-case → encode, thin - [ ] Validate-before-write (no partial write on a malformed request) - [ ] Best-effort execution + partial receipt with per-item `ok`/`error` - [ ] `fidelity` stamped into summary frontmatter; `summaries/<harness>/...` path - [ ] `dry_run` path writes nothing, returns would-be receipt - [ ] Tests: validation-rejects-bad-request, write+supersede insight, create/close/comment ticket, summary write, partial-failure receipt (one item fails others land), dry-run - [ ] `task check` green; routes registered; secret-handling respected ## Follow-ups (separate issues, after this ships) - **MCP `capture` tool** — thin forwarder to `CaptureService`, for MCP-native clients that prefer it. - **SKILL veneer(s)** — the close-session SKILL (already at `skills/close-session/SKILL.md`) becomes the claude.ai trigger/harvest layer that POSTs `/capture`; thin equivalents (or direct programmed calls) for Crush/Pi. - **OAuth2 token provisioning per environment** — CLI client-credential, claude.ai OAuth session, headless service token. Bounded per-harness quirk. - **Harvest adapters** — transcript-parse (reuse `ai-sessions` extractors) vs chat-memory-reconstruct vs agent-runlog; each assembles `/capture` args at its own fidelity. ## Related - #45 — `brain_update`/`brain_get` (the verbs this reuses; needs the extract refactor) - #47 — vector staleness (the discipline capture encodes server-side so clients don't carry it) - #43 / ADR-0009 (infra) — intent-named-verb interface direction; `capture` is the act-named verb - `skills/close-session/SKILL.md` — becomes the claude.ai veneer over this - ai-sessions `extract_*_sessions.py` — the harvest adapters for CLI/Crush/Pi
mathias commented 2026-06-22 17:25:22 +00:00
Author
Owner
Copy Link
Copy Source

Grounded against architecture + regulatory model — design must change before build

Consulted the brain and Gitea architecture record. Findings:

The canonical invariants now exist on main. The sovereign-AI architecture doc set (I1–I5) was sitting un-merged on a branch since 2026-06-05; canonized via infra PR #149. Current-state refresh tracked as infra #150.

This issue's "centralized capture service" framing is in tension with a decided invariant. Brain wiki/homelab/decisions/no-centralized-cross-harness-observer-2026-06-17 explicitly rejected a standing cross-harness observer ("high-degree node — exactly the concentration Zero Trust / sovereign-containment is suspicious of") and endorsed the retro-skill specifically as distributed, per-harness, no shared plumbing. A central /capture service with cross-harness brain+gitea write reach is closer to the rejected shape than the endorsed one. Per I2 (deliberate acceptance) it fails the admissibility test as written — it opens an undocumented acceptance.

Resolution (now in the spec): capture is a shared use-case/library invoked per-harness against the caller's own credentials (distributed consolidation, admissible without a ledger entry). A central authenticated relay is a fallback only for harnesses that can't run the library (Crush/Pi/headless), holds no standing visibility, retains nothing beyond the audit log — and if deployed, must be entered in infra/docs/security-baseline.md before it ships.

New acceptance gates this issue did not have:

  • I1 sovereignty gate — refuse capture of confidential-classified sessions through us-nexus (cloud) harnesses. Server-derived harness origin, not caller-asserted.
  • I5 auditability — every capture emits a request-level audit record (alloy/loki). A privileged write path that can't be audited must not run.
  • I3 — if deployed as a service, manifest under k3s/apps/** (no untracked runtime).

Spec committed: specs/capture-bdd-spec.md — use-case form + Gherkin scenarios (happy path, sovereignty refusal, supersession/staleness from #45/#47, fail-closed validation, partial-failure receipt, dry-run, fidelity supersession) + the invariant-obligations table.

Four open questions block the build (in the spec §4). The highest-risk: how is classification set and trusted? The whole I1 gate is only as strong as the classification it reads — a harness self-declaring "internal" to bypass it is an attack surface. Resolve before dispatch.

No BDD/use-case specs existed anywhere before this — this is the first. Worth noting as a process gap for privileged paths generally.

This issue should NOT be dispatched until §4 open questions are resolved and the distributed-vs-relay decision + (if relay) the ledger entry are settled.

## Grounded against architecture + regulatory model — design must change before build Consulted the brain and Gitea architecture record. Findings: **The canonical invariants now exist on main.** The sovereign-AI architecture doc set (I1–I5) was sitting un-merged on a branch since 2026-06-05; canonized via infra PR #149. Current-state refresh tracked as infra #150. **This issue's "centralized capture service" framing is in tension with a decided invariant.** Brain `wiki/homelab/decisions/no-centralized-cross-harness-observer-2026-06-17` explicitly rejected a standing cross-harness observer ("high-degree node — exactly the concentration Zero Trust / sovereign-containment is suspicious of") and endorsed the retro-skill specifically as *distributed, per-harness, no shared plumbing*. A central `/capture` service with cross-harness brain+gitea write reach is closer to the rejected shape than the endorsed one. Per I2 (deliberate acceptance) it fails the admissibility test as written — it opens an *undocumented* acceptance. **Resolution (now in the spec):** capture is a **shared use-case/library invoked per-harness against the caller's own credentials** (distributed consolidation, admissible without a ledger entry). A central authenticated relay is a *fallback only* for harnesses that can't run the library (Crush/Pi/headless), holds no standing visibility, retains nothing beyond the audit log — and **if deployed, must be entered in `infra/docs/security-baseline.md`** before it ships. **New acceptance gates this issue did not have:** - **I1 sovereignty gate** — refuse capture of confidential-classified sessions through us-nexus (cloud) harnesses. Server-derived harness origin, not caller-asserted. - **I5 auditability** — every capture emits a request-level audit record (alloy/loki). A privileged write path that can't be audited must not run. - **I3** — if deployed as a service, manifest under `k3s/apps/**` (no untracked runtime). **Spec committed:** `specs/capture-bdd-spec.md` — use-case form + Gherkin scenarios (happy path, sovereignty refusal, supersession/staleness from #45/#47, fail-closed validation, partial-failure receipt, dry-run, fidelity supersession) + the invariant-obligations table. **Four open questions block the build** (in the spec §4). The highest-risk: **how is `classification` set and trusted?** The whole I1 gate is only as strong as the classification it reads — a harness self-declaring "internal" to bypass it is an attack surface. Resolve before dispatch. **No BDD/use-case specs existed anywhere before this** — this is the first. Worth noting as a process gap for privileged paths generally. This issue should NOT be dispatched until §4 open questions are resolved and the distributed-vs-relay decision + (if relay) the ledger entry are settled.
mathias commented 2026-06-22 20:17:25 +00:00
Author
Owner
Copy Link
Copy Source

§4 open questions resolved — decisions binding, scope updated

Spec §4 resolved (commit b7a2cc5, specs/capture-bdd-spec.md):

  1. Classification trust → model (C): caller declares, server independently derives the target wing/repo's tag, stricter wins, mismatch logged as a security event. Caller can raise sensitivity, never lower below the target floor.
  2. Harness origin → server-derived from the authenticated principal. context.harness is descriptive-only, never a gate input (a control keyed on an attacker-suppliable value is not a control).
  3. Central relay → ships in v1 (claude.ai/Crush/Pi/LLM Council can't run the library in-process — those are the primary surfaces). The I2 security-baseline.md ledger entry is v1 work, not a follow-up.
  4. Audit-sink-down → degrade-and-warn + durable local buffer + reconcile-on-recovery. Proceed during a loki outage, buffer the audit locally, ntfy alert, replay on recovery. Floor: refuse only if nothing (central or local) can record the audit.

Scope changes these decisions force on #49

New v1 prerequisites (were not in the original AC):

  • Classification taxonomy (public / internal / confidential or similar) + a per-wing / per-repo classification tag the server can read. The I1 gate is not load-bearing until this exists. This is its own chunk of work — likely a sub-issue.
  • Server-side principal → harness-origin → trust-zone mapping (OAuth2 identity to sovereign/us-nexus). Load-bearing for I1.
  • I2 security-baseline ledger entry for the central relay, written before the relay ships.
  • Durable local audit buffer + loki reconciliation path (Q4) — not just "log to loki."
  • Relay is in scope for v1, not deferred.

Net: #49 is bigger than the original framing. Recommend splitting into sub-issues: (a) classification taxonomy + tagging, (b) the CaptureService use-case + ports + BrainStore extraction from #45's mcp-only logic, (c) REST/OAuth2 adapter + server-derived origin, (d) audit buffer + reconcile, (e) the I2 ledger entry + relay. (b) depends on (a) for the gate; (c)/(e) depend on (b).

Still not dispatch-ready — but now it's blocked on build sequencing/sub-issue breakdown rather than open design questions. The design is settled and BDD-specced.

## §4 open questions resolved — decisions binding, scope updated Spec §4 resolved (commit `b7a2cc5`, `specs/capture-bdd-spec.md`): 1. **Classification trust → model (C):** caller declares, server independently derives the target wing/repo's tag, **stricter wins**, mismatch logged as a security event. Caller can raise sensitivity, never lower below the target floor. 2. **Harness origin → server-derived** from the authenticated principal. `context.harness` is descriptive-only, **never a gate input** (a control keyed on an attacker-suppliable value is not a control). 3. **Central relay → ships in v1** (claude.ai/Crush/Pi/LLM Council can't run the library in-process — those are the primary surfaces). The I2 `security-baseline.md` ledger entry is **v1 work, not a follow-up**. 4. **Audit-sink-down → degrade-and-warn + durable local buffer + reconcile-on-recovery.** Proceed during a loki outage, buffer the audit locally, ntfy alert, replay on recovery. Floor: refuse only if *nothing* (central or local) can record the audit. ### Scope changes these decisions force on #49 **New v1 prerequisites (were not in the original AC):** - [ ] **Classification taxonomy** (`public` / `internal` / `confidential` or similar) + a **per-wing / per-repo classification tag** the server can read. The I1 gate is not load-bearing until this exists. This is its own chunk of work — likely a sub-issue. - [ ] **Server-side principal → harness-origin → trust-zone mapping** (OAuth2 identity to sovereign/us-nexus). Load-bearing for I1. - [ ] **I2 security-baseline ledger entry** for the central relay, written **before** the relay ships. - [ ] **Durable local audit buffer + loki reconciliation path** (Q4) — not just "log to loki." - [ ] Relay is **in scope for v1**, not deferred. **Net:** #49 is bigger than the original framing. Recommend splitting into sub-issues: (a) classification taxonomy + tagging, (b) the CaptureService use-case + ports + BrainStore extraction from #45's mcp-only logic, (c) REST/OAuth2 adapter + server-derived origin, (d) audit buffer + reconcile, (e) the I2 ledger entry + relay. (b) depends on (a) for the gate; (c)/(e) depend on (b). Still **not dispatch-ready** — but now it's blocked on *build sequencing/sub-issue breakdown* rather than open design questions. The design is settled and BDD-specced.
mathias commented 2026-06-22 20:20:53 +00:00
Author
Owner
Copy Link
Copy Source

Q4 revised — classification-aware audit degradation

Reconsidered Q4 (was: global degrade-and-warn + buffer). Now inherits from the Q1 classification spine:

  • Confidential + audit sink down → hard-refuse (no buffer — removes the "can a write tamper with its own pending audit record?" question for confidential data; simplicity of refuse is the assurance asset).
  • Internal/public + audit sink down → degrade-and-warn + durable local buffer + ntfy + reconcile-on-recovery.
  • Floor (all tiers): refuse if nothing can record the audit.

Spec commit 2a595b5. This simplifies the build relative to the prior Q4: the buffer + reconcile machinery only needs to handle internal/public tier, and the confidential path is a plain refuse. It also collapses Q4 into the same classification logic as I1, so there's one sensitivity model, not a separate availability policy.

Net effect on the AC delta from the previous comment: the "durable local audit buffer + reconciliation" item is now scoped to internal/public tier only — confidential needs no buffer path. Everything else from that comment stands.

Design is fully settled and BDD-specced. Remaining block is sub-issue breakdown for build sequencing, not design.

## Q4 revised — classification-aware audit degradation Reconsidered Q4 (was: global degrade-and-warn + buffer). Now inherits from the Q1 classification spine: - **Confidential + audit sink down → hard-refuse** (no buffer — removes the "can a write tamper with its own pending audit record?" question for confidential data; simplicity of refuse is the assurance asset). - **Internal/public + audit sink down → degrade-and-warn** + durable local buffer + ntfy + reconcile-on-recovery. - **Floor (all tiers):** refuse if nothing can record the audit. Spec commit `2a595b5`. This **simplifies** the build relative to the prior Q4: the buffer + reconcile machinery only needs to handle internal/public tier, and the confidential path is a plain refuse. It also collapses Q4 into the same classification logic as I1, so there's one sensitivity model, not a separate availability policy. Net effect on the AC delta from the previous comment: the "durable local audit buffer + reconciliation" item is now **scoped to internal/public tier only** — confidential needs no buffer path. Everything else from that comment stands. Design is fully settled and BDD-specced. Remaining block is sub-issue breakdown for build sequencing, not design.
mathias commented 2026-06-22 20:28:20 +00:00
Author
Owner
Copy Link
Copy Source

Sub-issue breakdown (2026-06-22) — #49 is now a tracking epic

Split into six sub-issues. Dependency graph (two roots converging):

#50 (49a) classification taxonomy + tags  ─┐
                                            ├─→ #53 (49d) REST adapter + I1 gate ─→ #54 (49e) audit path ─→ #55 (49f) relay + I2 ledger
#51 (49b) CaptureService + BrainStore ──────┤        ↑
              └─→ #52 (49c) Gitea tracker ──┘────────┘
# Sub Scope Depends on
#50 49a Classification taxonomy + per-wing/repo tags (I1 prerequisite) — (start now)
#51 49b CaptureService use-case + BrainStore extraction + ports/entities — (start now, fake ports)
#52 49c Gitea IssueTracker client (new outbound dep) #51
#53 49d POST /capture REST + OAuth2 + I1 gate (server-derived origin) #50, #51, #52
#54 49e I5 audit path + classification-aware degradation #50, #53
#55 49f Central relay + I2 ledger entry (non-library harnesses) #53, #54

Critical path: #50/#51 (parallel) → #52 → #53 → #54 → #55.

Two milestones worth naming:

  • After #54: capture works end-to-end and is I5-compliant for direct-REST harnesses (Claude Code CLI, Agentsquad). This is "capture works" for the contexts that can run against REST directly.
  • After #55: uniform across ALL harnesses incl. claude.ai/Crush/Pi/LLM Council. #55 carries the heaviest security weight (the I2 concentration acceptance) — its ledger entry must merge before relay code ships.

Start points: #50 and #51 have no dependencies and can begin in parallel. #51 is the biggest chunk (the BrainStore extraction + use-case). Either is a clean first dispatch to koala-Claude.

Design is fully settled (spec specs/capture-bdd-spec.md, §4 decisions binding). This epic now tracks build sequencing only.

## Sub-issue breakdown (2026-06-22) — #49 is now a tracking epic Split into six sub-issues. Dependency graph (two roots converging): ``` #50 (49a) classification taxonomy + tags ─┐ ├─→ #53 (49d) REST adapter + I1 gate ─→ #54 (49e) audit path ─→ #55 (49f) relay + I2 ledger #51 (49b) CaptureService + BrainStore ──────┤ ↑ └─→ #52 (49c) Gitea tracker ──┘────────┘ ``` | # | Sub | Scope | Depends on | |---|-----|-------|-----------| | **#50** | 49a | Classification taxonomy + per-wing/repo tags (I1 prerequisite) | — (start now) | | **#51** | 49b | `CaptureService` use-case + `BrainStore` extraction + ports/entities | — (start now, fake ports) | | **#52** | 49c | Gitea `IssueTracker` client (new outbound dep) | #51 | | **#53** | 49d | `POST /capture` REST + OAuth2 + I1 gate (server-derived origin) | #50, #51, #52 | | **#54** | 49e | I5 audit path + classification-aware degradation | #50, #53 | | **#55** | 49f | Central relay + I2 ledger entry (non-library harnesses) | #53, #54 | **Critical path:** #50/#51 (parallel) → #52 → #53 → #54 → #55. **Two milestones worth naming:** - **After #54:** capture works end-to-end and is I5-compliant for direct-REST harnesses (Claude Code CLI, Agentsquad). This is "capture works" for the contexts that can run against REST directly. - **After #55:** uniform across ALL harnesses incl. claude.ai/Crush/Pi/LLM Council. #55 carries the heaviest security weight (the I2 concentration acceptance) — its ledger entry must merge before relay code ships. **Start points:** #50 and #51 have no dependencies and can begin in parallel. #51 is the biggest chunk (the BrainStore extraction + use-case). Either is a clean first dispatch to koala-Claude. Design is fully settled (spec `specs/capture-bdd-spec.md`, §4 decisions binding). This epic now tracks build sequencing only.
mathias commented 2026-06-23 05:29:05 +00:00
Author
Owner
Copy Link
Copy Source

Epic complete — capture is uniform across all harnesses

All six sub-issues shipped and merged; tagged v0.11.0.

Sub Issue PR What
49a #50 #56 classification taxonomy + per-wing/repo tags (fail-safe to confidential)
49b #51 #57 CaptureService use-case + ports + BrainStore extraction (one impl with MCP)
49c #52 #58 Gitea IssueTracker (owner forced mathias; token never logged)
49d #53 #59 POST /capture REST + OAuth2 + I1 sovereignty gate (server-derived origin)
49e #54 #60 I5 audit path + classification-aware degradation (loki + durable buffer + reconcile)
49f #55 #61 + infra #151/#152 MCP capture relay tool + I2 ledger + I3 deploy

Architecture as decided: REST + OAuth2 is the uniform spine; the use-case logic is written once (CaptureService) and shared by the REST adapter and the MCP relay tool — distributed-library form for in-process harnesses (CLI/Agentsquad), the thin MCP relay for non-library ones (claude.ai/Crush/Pi/LLM Council) via the existing /mcp connector.

Invariants honoured: I1 (server-derived origin, confidential-through-us-nexus refused) · I2 (relay concentration accepted in infra/docs/security-baseline.md, merged before relay code) · I3 (env/secret via GitOps, Flux-reconciled) · I5 (request-level audit, classification-aware: confidential fails closed, internal/public degrades + reconciles).

Brain (reusable learnings from the build):

  • decisions/capture-classification-taxonomy
  • decisions/gate-on-server-derived-signals-fail-safe
  • decisions/two-phase-reserve-record-audit-gate
  • failures/mcp-bearer-middleware-discards-principal

Follow-ups not in this epic (per #49's own list): SKILL veneer over the close-session flow; per-harness token provisioning for Crush/Pi/LLM Council (claude.ai done via the existing connector); harvest adapters (transcript-parse vs chat-reconstruct). File as new issues when picked up.

Closing the epic.

## Epic complete — capture is uniform across all harnesses All six sub-issues shipped and merged; tagged **v0.11.0**. | Sub | Issue | PR | What | |---|---|---|---| | 49a | #50 | #56 | classification taxonomy + per-wing/repo tags (fail-safe to confidential) | | 49b | #51 | #57 | `CaptureService` use-case + ports + `BrainStore` extraction (one impl with MCP) | | 49c | #52 | #58 | Gitea `IssueTracker` (owner forced mathias; token never logged) | | 49d | #53 | #59 | `POST /capture` REST + OAuth2 + **I1 sovereignty gate** (server-derived origin) | | 49e | #54 | #60 | I5 audit path + **classification-aware degradation** (loki + durable buffer + reconcile) | | 49f | #55 | #61 + infra #151/#152 | MCP `capture` relay tool + **I2 ledger** + I3 deploy | **Architecture as decided:** REST + OAuth2 is the uniform spine; the use-case logic is written once (`CaptureService`) and shared by the REST adapter and the MCP relay tool — distributed-library form for in-process harnesses (CLI/Agentsquad), the thin MCP relay for non-library ones (claude.ai/Crush/Pi/LLM Council) via the existing `/mcp` connector. **Invariants honoured:** I1 (server-derived origin, confidential-through-us-nexus refused) · I2 (relay concentration accepted in `infra/docs/security-baseline.md`, merged before relay code) · I3 (env/secret via GitOps, Flux-reconciled) · I5 (request-level audit, classification-aware: confidential fails closed, internal/public degrades + reconciles). **Brain** (reusable learnings from the build): - `decisions/capture-classification-taxonomy` - `decisions/gate-on-server-derived-signals-fail-safe` - `decisions/two-phase-reserve-record-audit-gate` - `failures/mcp-bearer-middleware-discards-principal` **Follow-ups not in this epic** (per #49's own list): SKILL veneer over the close-session flow; per-harness token provisioning for Crush/Pi/LLM Council (claude.ai done via the existing connector); harvest adapters (transcript-parse vs chat-reconstruct). File as new issues when picked up. Closing the epic.
mathias commented 2026-06-23 05:41:40 +00:00
Author
Owner
Copy Link
Copy Source

As-built report committed

specs/capture-implementation-report.md (commit 06e21c0) — the durable record of what shipped, companion to specs/capture-bdd-spec.md:

  • Outcome + architecture (as-built diagram: both doors → one CaptureService → 5 ports)
  • Sub-issue → PR map (#50–#55 / #56–#61 + infra #151/#152)
  • I1–I5 compliance table
  • Operational env reference (BRAIN_GITEA_TOKEN, BRAIN_LOKI_URL, BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS, …)
  • Test coverage (67 test funcs → Gherkin mapping)
  • Deferred follow-ups + brain learning links

Onboarding + audit trail is now self-contained in the repo.

## As-built report committed `specs/capture-implementation-report.md` (commit `06e21c0`) — the durable record of what shipped, companion to `specs/capture-bdd-spec.md`: - Outcome + architecture (as-built diagram: both doors → one `CaptureService` → 5 ports) - Sub-issue → PR map (#50–#55 / #56–#61 + infra #151/#152) - I1–I5 compliance table - Operational env reference (`BRAIN_GITEA_TOKEN`, `BRAIN_LOKI_URL`, `BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS`, …) - Test coverage (67 test funcs → Gherkin mapping) - Deferred follow-ups + brain learning links Onboarding + audit trail is now self-contained in the repo.
mathias commented 2026-06-23 16:16:35 +00:00
Author
Owner
Copy Link
Copy Source

Post-ship fixes from the first claude.ai dogfood (#66 + #67) — both merged

The 2026-06-23 live dogfood surfaced two operability gaps; both fixed.

# Fix PR
#67 classification.yaml written at brain root — homelab wings/repos tagged internal (was: everything but hyperguild/homelab fail-safed to confidential → claude.ai capture refused) brain #12 ✅
#66 ai-sessions SummaryWriter wired (was: nil → "no summary writer configured") #68 ✅

#67: tagged internal — wings hyperguild/homelab; repos brain, ai-sessions, infra, hyperguild, homelab, tapir, agentsquad, jepa-fx-risk, swedsl. client-* and anything untagged stay confidential (fail-safe intact). Loaded at startup → picked up on the next pod roll.

#66: reused the existing BRAIN_GITEA_TOKEN (no new secret, no manifest change). A live token-scope probe on mathias/ai-sessions confirmed push:true (contents:write) and caught a real bug: gitea's contents API creates via POST, updates via PUT — WriteFile always PUT'd → 422 on new files. Fixed (POST create / PUT-with-sha update); the httptest mock had the same wrong assumption.

Validated under fire (worth noting): the partial-failure receipt did exactly its job — insights + ticket landed, summary failed honestly with a per-item error naming the component, no rollback. The best-effort design held.

Remaining: Flux rolls the pod (loads classification.yaml + summary writer) → re-dogfood from claude.ai with a summary block → expect all ok=true, effective_classification: internal. Both STOP points were resolved (map approved; token verified by probe, not assumed).

Brain learning: wiki/hyperguild/failures/mcp-bearer-middleware-discards-principal (from #55) + the dogfood finding note already filed.

## Post-ship fixes from the first claude.ai dogfood (#66 + #67) — both merged The 2026-06-23 live dogfood surfaced two operability gaps; both fixed. | # | Fix | PR | |---|-----|-----| | #67 | `classification.yaml` written at brain root — homelab wings/repos tagged `internal` (was: everything but hyperguild/homelab fail-safed to confidential → claude.ai capture refused) | brain #12 ✅ | | #66 | ai-sessions `SummaryWriter` wired (was: nil → `"no summary writer configured"`) | #68 ✅ | **#67:** tagged `internal` — wings `hyperguild`/`homelab`; repos `brain`, `ai-sessions`, `infra`, `hyperguild`, `homelab`, `tapir`, `agentsquad`, `jepa-fx-risk`, `swedsl`. `client-*` and anything untagged stay confidential (fail-safe intact). Loaded at startup → picked up on the next pod roll. **#66:** reused the existing `BRAIN_GITEA_TOKEN` (no new secret, no manifest change). A live token-scope probe on `mathias/ai-sessions` confirmed `push:true` (contents:write) **and** caught a real bug: gitea's contents API creates via **POST**, updates via **PUT** — `WriteFile` always PUT'd → 422 on new files. Fixed (POST create / PUT-with-sha update); the httptest mock had the same wrong assumption. **Validated under fire (worth noting):** the partial-failure receipt did exactly its job — insights + ticket landed, summary failed honestly with a per-item error naming the component, no rollback. The best-effort design held. **Remaining:** Flux rolls the pod (loads classification.yaml + summary writer) → re-dogfood from claude.ai with a `summary` block → expect all `ok=true`, `effective_classification: internal`. Both STOP points were resolved (map approved; token verified by probe, not assumed). Brain learning: `wiki/hyperguild/failures/mcp-bearer-middleware-discards-principal` (from #55) + the dogfood finding note already filed.
mathias commented 2026-06-24 20:49:59 +00:00
Author
Owner
Copy Link
Copy Source

Verified live (2026-06-24)

ingestion re-rolled (the 15:24Z pod predated the brain-sync of classification.yaml). Dry-run capture probe touching ai-sessions now returns effective_classification: internal (was confidential pre-roll). I1 gate no longer refuses claude.ai captures touching homelab repos; SummaryWriter wired + token contents:write confirmed. #66 + #67 done and confirmed in prod.

## Verified live (2026-06-24) ingestion re-rolled (the 15:24Z pod predated the `brain-sync` of classification.yaml). Dry-run capture probe touching `ai-sessions` now returns `effective_classification: internal` (was `confidential` pre-roll). I1 gate no longer refuses claude.ai captures touching homelab repos; SummaryWriter wired + token contents:write confirmed. #66 + #67 done and confirmed in prod.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#49