From 2beaac2febaf6344992449fc7faee53560a46d2e Mon Sep 17 00:00:00 2001 From: mathias Date: Thu, 28 May 2026 10:31:41 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20add=202026-05-28=20decisions=20?= =?UTF-8?q?=E2=80=94=20harness=20boundary=20+=20field=20benchmark=20defini?= =?UTF-8?q?tion?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- DECISIONS.md | 61 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/DECISIONS.md b/DECISIONS.md index 8f35e41..b360f25 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -4,6 +4,67 @@ Record *why* things are the way they are. Future-you will thank present-you. --- +## 2026-05-28 — hyperguild and agentsquad are separate harnesses, not competing implementations + +**Context:** After a grill-me architecture review, the question arose whether `hyperguild` +and `agentsquad` should be consolidated — they both involve agent orchestration and both +use LiteLLM routing. + +**Decision:** Keep separate. Different harnesses targeting different orchestration models. + +- **hyperguild** = Claude Code + MCP. Supervisor pattern. Skills as SKILL.md files invoked + inside Claude Code sessions. Routing pod routes skill calls to local models. Brain MCP + for knowledge accumulation. Target: disciplined solo coding sessions. +- **agentsquad** = OpenCode + LiteLLM. Flat peer-to-peer executor/reviewer agents. + Per-agent model routing via piguard virtual model names, risk-tiered approval middleware. + Target: multi-agent task execution pipelines. + +Skills (`mathias/skills`) are shared — both harnesses consume the same SKILL.md files. + +**Consequences:** No consolidation work. Both READMEs should state their harness explicitly. + +--- + +## 2026-05-28 — "field benchmark" for local models = pass-rate at scale (supersedes GOTTH eval suite) + +**Context:** The GOTTH eval suite (45 offline prompts across 5 categories) was replaced by +a "field benchmark" in May 2026, but the replacement was never defined concretely. + +**Decision:** The field benchmark is per-skill pass rate over real routing pod usage, +collected automatically by `internal/routing/passrate.go` and exposed at: + +``` +GET /pass-rate?skill=&window= +``` + +No separate eval suite. No synthetic prompts. The benchmark runs itself once the routing +pod receives real traffic. Target: 30-day rolling window per skill, reviewed monthly. + +**Bootstrap note:** With no session history, `passrate.go` returns `nil` and the router +defaults to the thinking model for every call. The fast-model path activates only after +real pass-rate data accumulates. Seed with real usage — do not pre-populate. + +**Consequences:** Zero maintenance overhead for the benchmark. The tradeoff is that results +are only meaningful after ~2 weeks of real usage, and skills that are rarely invoked will +have statistically thin pass-rate data. Revisit if a skill has fewer than 20 calls in 30 days. + +--- + +## 2026-05-28 — brain injection in skill handlers: review is done, others unverified + +**Context:** The April 2026 scope reset listed "brain_query injection into skill handlers" +as the top priority. As of 2026-05-28, `internal/skills/review/handlers.go` calls +`brain.Query(ctx, ...)` before dispatching to the LLM — confirmed in code review. +Status of debug, retrospective, and trainer handlers is unverified. + +**Decision:** Treat review as the reference implementation. Verify debug, retrospective, +trainer against the same pattern before shipping new skill work. + +**Consequences:** The April concern may be stale for review. A one-pass audit of the other +three skill handlers closes this fully. + +--- + ## 2026-04-08 — AGENTS.md as cross-tool standard, not CLAUDE.md **Context**: Multiple tools (Crush, Pi, Antigravity) read `AGENTS.md` natively. Claude Code reads `CLAUDE.md`. Building on `CLAUDE.md` as the primary format locks into one vendor.