# Runbook: exercising review/debug traffic to fill the pass-rate dataset

**Why this exists:** the routing pod's local-vs-cloud decision is gated on a
pass-rate history that only accrues from real `review`/`debug` invocations
**through the pod**. Until the dataset has data, the fast (local) path never
activates and the core hypothesis (hyperguild #35) can't be validated. This
runbook is how you spin that flywheel.

## The one trap

Pass-rate accrues **only** when a skill tool is called via the routing pod's MCP
endpoint. These look like they should count but **do not**:

- **Crush** — talks to LiteLLM directly, bypasses the pod. No log.
- **claude.ai web / Claude Desktop without the connector** — no log.
- **Running the local `code-review` / `debug` skills** (`~/dev/.skills`) inline in
  a Claude Code session — those are local skills, not the pod's MCP tools. No log.

Only a `tools/call` to the routing pod records a pass/fail.

## Endpoints

| Purpose | URL | Auth |
|---------|-----|------|
| Routing MCP (local, Tailscale) | `http://koala:30310/mcp` | Bearer `ROUTING_MCP_TOKEN` |
| Routing MCP (remote) | `https://routing-mcp.d-ma.be/mcp` | OAuth via `auth.d-ma.be` (audience `claude-ai`) |
| Pass-rate readout | `http://koala:30330/pass-rate?skill=<name>` | none (read-only) |

Tools advertised: **`review`**, **`debug`** (the two the #35 gate measures),
plus `session_log`, `retrospective`, `trainer`.

## Step 1 — connect the routing pod as an MCP server

**Local** (needs the bearer token; keep it out of argv via 1Password):

```bash
op run --env-file ~/.op-env -- \
  claude mcp add routing --transport http http://koala:30310/mcp \
    --header "Authorization: Bearer $ROUTING_MCP_TOKEN"
```

**Remote** (claude.ai / Claude Desktop): add a custom connector pointing at
`https://routing-mcp.d-ma.be/mcp`; it completes OAuth against `auth.d-ma.be`,
no static token.

Verify: a `tools/list` should return `review`, `debug`, `session_log`,
`retrospective`, `trainer`.

## Step 2 — route real work through it

In normal sessions, invoke the pod's tools instead of reviewing/debugging inline:

- *"Use the **routing** `review` tool on this diff."*
- *"**debug** this failure through the routing pod."*

Each call logs an outcome to ingestion → `/pass-rate` ticks up.

## Step 3 — the flywheel (cold-start behavior is the gate)

The router's cold start **is** the data gate — same loop:

1. With pass-rate `nil` (cold), the policy routes to the **thinking / cloud**
   model — it won't trust the local fast model without history.
2. As `review` / `debug` accumulate **passes**, pass-rate climbs past the floor
   (`HYPERGUILD_ROUTE_LOCAL_FLOOR=0.90`) and the router starts sending those
   skills to the **local fast tier**.
3. So early calls "pay in" on cloud to build the record; then the local tier
   activates automatically. Exercising the traffic is what spins it up.

The fast tier is now **`koala/qwen36-35b-a3b`** (Qwen3.6-35B-A3B MTP), promoted
2026-06-29 after clearing the tool-call gate (infra `c66a195`,
`HYPERGUILD_FAST_MODEL`).

## Target & verification

- **50 logged invocations** across `review` + `debug` within the 14-day window
  (opened 2026-06-26, **kill-date 2026-07-10**) → ~4–5 calls/day.
- Check progress anytime:

  ```bash
  curl -s "http://koala:30330/pass-rate?skill=review"
  curl -s "http://koala:30330/pass-rate?skill=debug"
  ```

- If ~4–5/day isn't realistic alongside Crush, that is **not** a failure — per
  #35 deliverable #1 it's the signal hyperguild isn't on the work critical path,
  and the pre-decided **Berget fallback** (`gpt-oss-120b` / `qwen3-32b`) carries
  the fast tier instead.

## Refs

- hyperguild #35 — the validation issue (data gate = deliverable #1)
- `docs/multi-model-routing.md` — routing policy
- brain: `wiki/homelab/hypotheses/qwen36-35b-a3b-fast-model-experiment-2026-05-28.md`
- infra `c66a195` — qwen36 promotion; `models.yml` / `llama-swap-configmap.yaml`