Compare commits

...
Author SHA1 Message Date
mathiasandClaude Opus 4.8 00e5f62c8e docs(runbook): correct flywheel (nil→local) + instrumentation-fixed note
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
Two corrections after the #73 investigation: (1) the router routes cold
(nil pass-rate) calls to the LOCAL fast tier, not cloud — the earlier
"pay in on cloud" description was wrong (per policy.go). (2) Note that
pass-rate logging was broken until v0.11.1 (#73) and is now verified;
reset the window to 2026-06-30→07-14. Refs #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:16:03 +02:00
mathiasandClaude Opus 4.8 b9d03316fd fix(ingestion): migrate mcp-chassis import to git.d-ma.be path (#74)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Has been skipped
Unblocks CD: the build's go mod download failed because mcp-chassis was
imported via the old gitea.d-ma.be module path, which the renamed server no
longer serves a matching go-import meta tag for. Point at the renamed module
git.d-ma.be/mathias/mcp-chassis v0.2.0 (mcp-chassis 69d389d) + go mod tidy.
Fresh download now resolves; ingestion builds + tests green.

This restores the normal CI path so the routing pass-rate fix (da9bdc4,
#73) builds and deploys through gitea Actions. Refs #74, #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:39:41 +02:00
mathiasandClaude Opus 4.8 da9bdc4cbb fix(routing): repair pass-rate instrumentation (3 bugs) — #73, #35
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
The #35 data gate could never fill: a real review call routed cleanly to
qwen36 but /pass-rate stayed total:0 under every key. Root cause was three
independent defects in the session_log path, each alone fatal:

- A: a successful routed call logged final_status "skip", never "pass".
  /pass-rate computes pass/(pass+fail) and skips count as neither, so the
  >=0.90 gate was mathematically unreachable. Success now logs "pass".
- B: every record was written under skill "_routing", so /pass-rate?skill=
  review|debug (what #35 measures) always read zero. Now uses the real
  e.Skill; routing decisions stay groupable via session_id "_routing".
- C: the session_log POST to the bearer-gated ingestion /mcp carried no
  Authorization header → silent 401, swallowed by best-effort logging
  (the documented mcpclient-empty-token-silent-401 footgun). Logger now
  takes a token (BRAIN_MCP_TOKEN) and sets the bearer when non-empty.

Tests rewritten to assert correct behavior (they had encoded the bugs:
"skip" on success, "_routing" skill). New test covers the auth header and
the empty-token path.

Infra (BRAIN_MCP_TOKEN ExternalSecret + env on the routing deployment) and
redeploy follow separately. Refs #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 08:36:18 +02:00
mathiasandClaude Opus 4.8 dcb9ff4a56 docs(runbook): how to exercise review/debug traffic for the pass-rate gate
CI / Lint / Test / Vet (push) Successful in 20s
CI / Mirror to GitHub (push) Successful in 3s
The #35 data gate stays at zero because pass-rate only accrues from review/
debug calls *through the routing pod* — Crush, cloud chat, and the local
.skills all bypass it. Document the connect → route → flywheel steps, the
endpoints (koala:30310/mcp + routing-mcp.d-ma.be), the cold-start behavior
(nil pass-rate routes cloud until passes accrue past the 0.90 floor, then
local qwen36 activates), the 50-invocation / 2026-07-10 target, and the
Berget fallback. Refs #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:41:07 +02:00
mathias 0454527b83 Merge pull request 'feat: brain_pending + brain_promote — close the raw→wiki curation loop (#38)' (#70) from feat/brain-pending-promote into main
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
2026-06-26 14:49:27 +00:00
9 changed files with 178 additions and 24 deletions
+1 -1
View File
@@ -53,7 +53,7 @@ func main() {
router := &routing.Router{ router := &routing.Router{
Fetcher: routing.NewFetcher(cfg.BrainURL, "7d", time.Duration(cfg.PassRateTTLSeconds)*time.Second), Fetcher: routing.NewFetcher(cfg.BrainURL, "7d", time.Duration(cfg.PassRateTTLSeconds)*time.Second),
Logger: routing.NewLogger(cfg.BrainURL), Logger: routing.NewLogger(cfg.BrainURL, cfg.BrainMCPToken),
Policy: routing.Policy{Floor: cfg.RouteLocalFloor, Ceil: cfg.RouteLocalCeil}, Policy: routing.Policy{Floor: cfg.RouteLocalFloor, Ceil: cfg.RouteLocalCeil},
FastModel: cfg.FastModel, FastModel: cfg.FastModel,
ThinkingModel: cfg.ThinkingModel, ThinkingModel: cfg.ThinkingModel,
@@ -0,0 +1,104 @@
# Runbook: exercising review/debug traffic to fill the pass-rate dataset
**Why this exists:** the routing pod's local-vs-cloud decision is gated on a
pass-rate history that only accrues from real `review`/`debug` invocations
**through the pod**. Until the dataset has data, the fast (local) path never
activates and the core hypothesis (hyperguild #35) can't be validated. This
runbook is how you spin that flywheel.
## The one trap
Pass-rate accrues **only** when a skill tool is called via the routing pod's MCP
endpoint. These look like they should count but **do not**:
- **Crush** — talks to LiteLLM directly, bypasses the pod. No log.
- **claude.ai web / Claude Desktop without the connector** — no log.
- **Running the local `code-review` / `debug` skills** (`~/dev/.skills`) inline in
a Claude Code session — those are local skills, not the pod's MCP tools. No log.
Only a `tools/call` to the routing pod records a pass/fail.
## Endpoints
| Purpose | URL | Auth |
|---------|-----|------|
| Routing MCP (local, Tailscale) | `http://koala:30310/mcp` | Bearer `ROUTING_MCP_TOKEN` |
| Routing MCP (remote) | `https://routing-mcp.d-ma.be/mcp` | OAuth via `auth.d-ma.be` (audience `claude-ai`) |
| Pass-rate readout | `http://koala:30330/pass-rate?skill=<name>` | none (read-only) |
Tools advertised: **`review`**, **`debug`** (the two the #35 gate measures),
plus `session_log`, `retrospective`, `trainer`.
## Step 1 — connect the routing pod as an MCP server
**Local** (needs the bearer token; keep it out of argv via 1Password):
```bash
op run --env-file ~/.op-env -- \
claude mcp add routing --transport http http://koala:30310/mcp \
--header "Authorization: Bearer $ROUTING_MCP_TOKEN"
```
**Remote** (claude.ai / Claude Desktop): add a custom connector pointing at
`https://routing-mcp.d-ma.be/mcp`; it completes OAuth against `auth.d-ma.be`,
no static token.
Verify: a `tools/list` should return `review`, `debug`, `session_log`,
`retrospective`, `trainer`.
## Step 2 — route real work through it
In normal sessions, invoke the pod's tools instead of reviewing/debugging inline:
- *"Use the **routing** `review` tool on this diff."*
- *"**debug** this failure through the routing pod."*
Each call logs an outcome to ingestion → `/pass-rate` ticks up.
## Step 3 — how routing actually picks the model
Per `internal/routing/policy.go`:
1. pass-rate `nil` (cold) → **local** fast tier. The router defaults to local
from invocation #1, not to cloud — so the fast tier is exercised immediately.
2. pass-rate `>= 0.90` (floor) → **local**; `< 0.70` (ceil) → **cloud/thinking**;
in the `[0.70, 0.90)` band a request-hash bit samples 50/50.
3. On a local execution error the router falls open to the thinking model for
that one call (logged `thinking_fallback`).
So you are not "paying in on cloud" — cold calls already run on the (validated)
local fast tier **`koala/qwen36-35b-a3b`** (Qwen3.6-35B-A3B MTP, promoted
2026-06-29, infra `c66a195`, `HYPERGUILD_FAST_MODEL`). Accumulating passes just
keeps it there once real pass-rate is computed.
> **Instrumentation note (#73, fixed 2026-06-30):** until v0.11.1 the pod logged
> successes as `"skip"` (not `"pass"`), under `skill:"_routing"`, via an
> unauthenticated POST that silently 401'd — so `/pass-rate` stayed at zero no
> matter how much you used it. That's fixed and verified (a real review call now
> moves `/pass-rate?skill=review` 0→1). If you see traffic not registering,
> re-check #73's three failure modes first.
## Target & verification
- **50 logged invocations** across `review` + `debug` within the 14-day window.
The clock restarts **2026-06-30** (the day instrumentation was verified working;
the original 2026-06-26→07-10 window measured broken plumbing) → **kill-date
2026-07-14**, ~4 calls/day (1 already logged from the #73 smoke test).
- Check progress anytime:
```bash
curl -s "http://koala:30330/pass-rate?skill=review"
curl -s "http://koala:30330/pass-rate?skill=debug"
```
- If ~45/day isn't realistic alongside Crush, that is **not** a failure — per
#35 deliverable #1 it's the signal hyperguild isn't on the work critical path,
and the pre-decided **Berget fallback** (`gpt-oss-120b` / `qwen3-32b`) carries
the fast tier instead.
## Refs
- hyperguild #35 — the validation issue (data gate = deliverable #1)
- `docs/multi-model-routing.md` — routing policy
- brain: `wiki/homelab/hypotheses/qwen36-35b-a3b-fast-model-experiment-2026-05-28.md`
- infra `c66a195` — qwen36 promotion; `models.yml` / `llama-swap-configmap.yaml`
+1 -1
View File
@@ -13,7 +13,7 @@ import (
"strings" "strings"
"time" "time"
chassisauth "gitea.d-ma.be/mathias/mcp-chassis/auth" chassisauth "git.d-ma.be/mathias/mcp-chassis/auth"
"github.com/mathiasbq/hyperguild/ingestion/internal/api" "github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit" "github.com/mathiasbq/hyperguild/ingestion/internal/audit"
+8 -5
View File
@@ -2,19 +2,22 @@ module github.com/mathiasbq/hyperguild/ingestion
go 1.26.1 go 1.26.1
require github.com/stretchr/testify v1.11.1
require ( require (
github.com/lestrrat-go/jwx/v2 v2.1.6 github.com/kr/text v0.2.0 // indirect
github.com/stretchr/testify v1.11.1 github.com/lestrrat-go/jwx/v2 v2.1.6 // indirect
github.com/rogpeppe/go-internal v1.15.0 // indirect
) )
require ( require (
gitea.d-ma.be/mathias/mcp-chassis v0.1.0 // indirect git.d-ma.be/mathias/mcp-chassis v0.2.0
github.com/davecgh/go-spew v1.1.1 // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect
github.com/goccy/go-json v0.10.3 // indirect github.com/goccy/go-json v0.10.3 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect github.com/jackc/pgx/v5 v5.9.2
github.com/jackc/puddle/v2 v2.2.2 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/lestrrat-go/blackmagic v1.0.3 // indirect github.com/lestrrat-go/blackmagic v1.0.3 // indirect
github.com/lestrrat-go/httpcc v1.0.1 // indirect github.com/lestrrat-go/httpcc v1.0.1 // indirect
@@ -27,5 +30,5 @@ require (
golang.org/x/sync v0.17.0 // indirect golang.org/x/sync v0.17.0 // indirect
golang.org/x/sys v0.31.0 // indirect golang.org/x/sys v0.31.0 // indirect
golang.org/x/text v0.29.0 // indirect golang.org/x/text v0.29.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1
) )
+10 -3
View File
@@ -1,5 +1,6 @@
gitea.d-ma.be/mathias/mcp-chassis v0.1.0 h1:8RXO34+n7Vu8HnUMagars6fc4oemqRpMu7MVtjaj4qY= git.d-ma.be/mathias/mcp-chassis v0.2.0 h1:6fLmb7xqRa2nNVWsHaUbbfbArgDXJw/gDhb09clBIjo=
gitea.d-ma.be/mathias/mcp-chassis v0.1.0/go.mod h1:ajbLlwr2L7FAN3TBU39KucZkKJM02wTbKbDKDEW2YvE= git.d-ma.be/mathias/mcp-chassis v0.2.0/go.mod h1:Ks7EK2UnGAN0H3rJjKUxUagX8/ZBdtLrOlcUbv0RwH8=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -15,6 +16,10 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw=
github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lestrrat-go/blackmagic v1.0.3 h1:94HXkVLxkZO9vJI/w2u1T0DAoprShFd13xtnSINtDWs= github.com/lestrrat-go/blackmagic v1.0.3 h1:94HXkVLxkZO9vJI/w2u1T0DAoprShFd13xtnSINtDWs=
github.com/lestrrat-go/blackmagic v1.0.3/go.mod h1:6AWFyKNNj0zEXQYfTMPfZrAXUWUfTIZ5ECEUEJaijtw= github.com/lestrrat-go/blackmagic v1.0.3/go.mod h1:6AWFyKNNj0zEXQYfTMPfZrAXUWUfTIZ5ECEUEJaijtw=
github.com/lestrrat-go/httpcc v1.0.1 h1:ydWCStUeJLkpYyjLDHihupbn2tYmZ7m22BGkcvZZrIE= github.com/lestrrat-go/httpcc v1.0.1 h1:ydWCStUeJLkpYyjLDHihupbn2tYmZ7m22BGkcvZZrIE=
@@ -29,6 +34,8 @@ github.com/lestrrat-go/option v1.0.1 h1:oAzP2fvZGQKWkvHa1/SAcFolBEca1oN+mQ7eooNB
github.com/lestrrat-go/option v1.0.1/go.mod h1:5ZHFbivi4xwXxhxY9XHDe2FHo6/Z7WWmtT7T5nBBp3I= github.com/lestrrat-go/option v1.0.1/go.mod h1:5ZHFbivi4xwXxhxY9XHDe2FHo6/Z7WWmtT7T5nBBp3I=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys= github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys=
github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs= github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -46,9 +53,9 @@ golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk= golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4= golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+2
View File
@@ -14,6 +14,7 @@ type RoutingConfig struct {
LiteLLMBaseURL string // LITELLM_BASE_URL, default https://llm-api.d-ma.be LiteLLMBaseURL string // LITELLM_BASE_URL, default https://llm-api.d-ma.be
LiteLLMAPIKey string // LITELLM_API_KEY LiteLLMAPIKey string // LITELLM_API_KEY
BrainURL string // BRAIN_URL, default http://ingestion.supervisor:3300 BrainURL string // BRAIN_URL, default http://ingestion.supervisor:3300
BrainMCPToken string // BRAIN_MCP_TOKEN, bearer for the auth-gated ingestion /mcp (session_log)
FastModel string // HYPERGUILD_FAST_MODEL, default koala/qwen35-9b-fast FastModel string // HYPERGUILD_FAST_MODEL, default koala/qwen35-9b-fast
ThinkingModel string // HYPERGUILD_THINKING_MODEL, default iguana/gemma4-26b ThinkingModel string // HYPERGUILD_THINKING_MODEL, default iguana/gemma4-26b
// RouteLocalFloor and RouteLocalCeil intentionally invert the usual // RouteLocalFloor and RouteLocalCeil intentionally invert the usual
@@ -44,6 +45,7 @@ func LoadRouting() (RoutingConfig, error) {
LiteLLMBaseURL: envOr("LITELLM_BASE_URL", "https://llm-api.d-ma.be"), LiteLLMBaseURL: envOr("LITELLM_BASE_URL", "https://llm-api.d-ma.be"),
LiteLLMAPIKey: os.Getenv("LITELLM_API_KEY"), LiteLLMAPIKey: os.Getenv("LITELLM_API_KEY"),
BrainURL: envOr("BRAIN_URL", "http://ingestion.supervisor:3300"), BrainURL: envOr("BRAIN_URL", "http://ingestion.supervisor:3300"),
BrainMCPToken: os.Getenv("BRAIN_MCP_TOKEN"),
FastModel: envOr("HYPERGUILD_FAST_MODEL", "koala/qwen35-9b-fast"), FastModel: envOr("HYPERGUILD_FAST_MODEL", "koala/qwen35-9b-fast"),
ThinkingModel: envOr("HYPERGUILD_THINKING_MODEL", "iguana/gemma4-26b"), ThinkingModel: envOr("HYPERGUILD_THINKING_MODEL", "iguana/gemma4-26b"),
} }
+19 -6
View File
@@ -17,19 +17,24 @@ type LogEntry struct {
Message string // free-form, e.g. "model=qwen35, pass_rate=0.94" Message string // free-form, e.g. "model=qwen35, pass_rate=0.94"
ProjectRoot string ProjectRoot string
DurationMs int64 DurationMs int64
Failed bool // true → final_status: "fail"; false → "skip" Failed bool // true → final_status: "fail"; false → "pass"
} }
// Logger posts session_log entries to a brain MCP at BrainURL + /mcp. // Logger posts session_log entries to a brain MCP at BrainURL + /mcp.
type Logger struct { type Logger struct {
BrainURL string BrainURL string
Token string // bearer for the (auth-gated) ingestion /mcp; empty = no header
HTTP *http.Client HTTP *http.Client
} }
// NewLogger creates a Logger with a 2-second HTTP timeout. // NewLogger creates a Logger with a 2-second HTTP timeout. token authenticates
func NewLogger(brainURL string) *Logger { // to the bearer-gated ingestion /mcp; an empty token sends no Authorization
// header (and silently 401s against a gated server — see brain
// mcpclient-empty-token-silent-401-envfrom-missing-key).
func NewLogger(brainURL, token string) *Logger {
return &Logger{ return &Logger{
BrainURL: brainURL, BrainURL: brainURL,
Token: token,
HTTP: &http.Client{Timeout: 2 * time.Second}, HTTP: &http.Client{Timeout: 2 * time.Second},
} }
} }
@@ -37,7 +42,10 @@ func NewLogger(brainURL string) *Logger {
// LogDecision posts a session_log MCP call. Errors are returned but the caller // LogDecision posts a session_log MCP call. Errors are returned but the caller
// MUST NOT block real work on them — logging is best-effort. // MUST NOT block real work on them — logging is best-effort.
func (l *Logger) LogDecision(ctx context.Context, e LogEntry) error { func (l *Logger) LogDecision(ctx context.Context, e LogEntry) error {
status := "skip" // A completed routed call is a pass (liveness); only an execution error is a
// fail. There is no "skip" for routing — the prior default-to-"skip" meant a
// successful call never counted toward pass_rate, so the gate was unreachable.
status := "pass"
if e.Failed { if e.Failed {
status = "fail" status = "fail"
} }
@@ -48,8 +56,10 @@ func (l *Logger) LogDecision(ctx context.Context, e LogEntry) error {
"params": map[string]any{ "params": map[string]any{
"name": "session_log", "name": "session_log",
"arguments": map[string]any{ "arguments": map[string]any{
"session_id": e.SessionID, "session_id": e.SessionID,
"skill": "_routing", // The real skill, so /pass-rate?skill=review|debug sees these
// records; routing decisions stay groupable via session_id "_routing".
"skill": e.Skill,
"phase": "decide", "phase": "decide",
"final_status": status, "final_status": status,
"message": fmt.Sprintf("%s: %s — %s", e.Skill, e.Decision, e.Message), "message": fmt.Sprintf("%s: %s — %s", e.Skill, e.Decision, e.Message),
@@ -67,6 +77,9 @@ func (l *Logger) LogDecision(ctx context.Context, e LogEntry) error {
return fmt.Errorf("log: build request: %w", err) return fmt.Errorf("log: build request: %w", err)
} }
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
if l.Token != "" {
req.Header.Set("Authorization", "Bearer "+l.Token)
}
resp, err := l.HTTP.Do(req) resp, err := l.HTTP.Do(req)
if err != nil { if err != nil {
return fmt.Errorf("log: request: %w", err) return fmt.Errorf("log: request: %w", err)
+31 -6
View File
@@ -15,36 +15,44 @@ import (
func TestLoggerLogDecision(t *testing.T) { func TestLoggerLogDecision(t *testing.T) {
var captured map[string]any var captured map[string]any
var authHeader string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, http.MethodPost, r.Method) assert.Equal(t, http.MethodPost, r.Method)
assert.Equal(t, "/mcp", r.URL.Path) assert.Equal(t, "/mcp", r.URL.Path)
authHeader = r.Header.Get("Authorization")
body, _ := io.ReadAll(r.Body) body, _ := io.ReadAll(r.Body)
require.NoError(t, json.Unmarshal(body, &captured)) require.NoError(t, json.Unmarshal(body, &captured))
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{"content": []map[string]any{{"type": "text", "text": "ok"}}}}) _ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{"content": []map[string]any{{"type": "text", "text": "ok"}}}})
})) }))
defer srv.Close() defer srv.Close()
l := routing.NewLogger(srv.URL) l := routing.NewLogger(srv.URL, "test-token")
err := l.LogDecision(context.Background(), routing.LogEntry{ err := l.LogDecision(context.Background(), routing.LogEntry{
SessionID: "sess-1", SessionID: "sess-1",
Skill: "review", Skill: "review",
Decision: "local", Decision: "local",
Message: "model=qwen35, pass_rate=0.94", Message: "model=qwen36, pass_rate=0.94",
ProjectRoot: "/home/x/proj", ProjectRoot: "/home/x/proj",
DurationMs: 1234, DurationMs: 1234,
Failed: false, Failed: false,
}) })
require.NoError(t, err) require.NoError(t, err)
// Bug C fix: the POST authenticates to the bearer-gated ingestion /mcp.
assert.Equal(t, "Bearer test-token", authHeader)
params := captured["params"].(map[string]any) params := captured["params"].(map[string]any)
assert.Equal(t, "tools/call", captured["method"]) assert.Equal(t, "tools/call", captured["method"])
assert.Equal(t, "session_log", params["name"]) assert.Equal(t, "session_log", params["name"])
args := params["arguments"].(map[string]any) args := params["arguments"].(map[string]any)
assert.Equal(t, "_routing", args["skill"]) // Bug B fix: the record carries the real skill so /pass-rate?skill=review sees it.
assert.Equal(t, "review", args["skill"])
assert.Equal(t, "decide", args["phase"]) assert.Equal(t, "decide", args["phase"])
assert.Equal(t, "skip", args["final_status"]) // Bug A fix: a successful routed call logs "pass", not "skip".
assert.Equal(t, "pass", args["final_status"])
assert.Contains(t, args["message"].(string), "review: local") assert.Contains(t, args["message"].(string), "review: local")
// session grouping is preserved via session_id.
assert.Equal(t, "sess-1", args["session_id"]) assert.Equal(t, "sess-1", args["session_id"])
assert.Equal(t, "/home/x/proj", args["project_root"]) assert.Equal(t, "/home/x/proj", args["project_root"])
assert.Equal(t, float64(1234), args["duration_ms"]) assert.Equal(t, float64(1234), args["duration_ms"])
@@ -59,23 +67,40 @@ func TestLoggerLogFailure(t *testing.T) {
})) }))
defer srv.Close() defer srv.Close()
l := routing.NewLogger(srv.URL) l := routing.NewLogger(srv.URL, "test-token")
err := l.LogDecision(context.Background(), routing.LogEntry{ err := l.LogDecision(context.Background(), routing.LogEntry{
SessionID: "s", Skill: "debug", Decision: "local", Message: "litellm down", Failed: true, SessionID: "s", Skill: "debug", Decision: "local", Message: "litellm down", Failed: true,
}) })
require.NoError(t, err) require.NoError(t, err)
args := captured["params"].(map[string]any)["arguments"].(map[string]any) args := captured["params"].(map[string]any)["arguments"].(map[string]any)
assert.Equal(t, "debug", args["skill"])
assert.Equal(t, "fail", args["final_status"]) assert.Equal(t, "fail", args["final_status"])
} }
func TestLoggerOmitsAuthWhenTokenEmpty(t *testing.T) {
var authHeader string
hasAuth := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
authHeader = r.Header.Get("Authorization")
_, hasAuth = r.Header["Authorization"]
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{}})
}))
defer srv.Close()
l := routing.NewLogger(srv.URL, "")
require.NoError(t, l.LogDecision(context.Background(), routing.LogEntry{Skill: "review", SessionID: "_routing", Decision: "local"}))
assert.False(t, hasAuth, "no Authorization header should be set when token is empty")
assert.Equal(t, "", authHeader)
}
func TestLoggerSurfacesUpstreamError(t *testing.T) { func TestLoggerSurfacesUpstreamError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "down", http.StatusBadGateway) http.Error(w, "down", http.StatusBadGateway)
})) }))
defer srv.Close() defer srv.Close()
l := routing.NewLogger(srv.URL) l := routing.NewLogger(srv.URL, "test-token")
err := l.LogDecision(context.Background(), routing.LogEntry{Skill: "x", SessionID: "y", Decision: "local"}) err := l.LogDecision(context.Background(), routing.LogEntry{Skill: "x", SessionID: "y", Decision: "local"})
require.Error(t, err) require.Error(t, err)
} }
+2 -2
View File
@@ -50,7 +50,7 @@ func newRouter(t *testing.T, llm *fakeLLM, passRate float64) (*routing.Router, *
r := &routing.Router{ r := &routing.Router{
Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute), Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute),
Logger: routing.NewLogger(brain.URL), Logger: routing.NewLogger(brain.URL, ""),
Policy: routing.Policy{Floor: 0.9, Ceil: 0.7}, Policy: routing.Policy{Floor: 0.9, Ceil: 0.7},
FastModel: "koala/qwen35-9b-fast", FastModel: "koala/qwen35-9b-fast",
ThinkingModel: "iguana/gemma4-26b", ThinkingModel: "iguana/gemma4-26b",
@@ -117,7 +117,7 @@ func TestRouterDefaultsToFastWhenBrainUnreachable(t *testing.T) {
llm := &fakeLLM{resp: "ok"} llm := &fakeLLM{resp: "ok"}
r := &routing.Router{ r := &routing.Router{
Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute), Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute),
Logger: routing.NewLogger(brain.URL), Logger: routing.NewLogger(brain.URL, ""),
Policy: routing.Policy{Floor: 0.9, Ceil: 0.7}, Policy: routing.Policy{Floor: 0.9, Ceil: 0.7},
FastModel: "koala/qwen35-9b-fast", FastModel: "koala/qwen35-9b-fast",
ThinkingModel: "iguana/gemma4-26b", ThinkingModel: "iguana/gemma4-26b",