docs(runbook): correct flywheel (nil→local) + instrumentation-fixed note
Two corrections after the #73 investigation: (1) the router routes cold (nil pass-rate) calls to the LOCAL fast tier, not cloud — the earlier "pay in on cloud" description was wrong (per policy.go). (2) Note that pass-rate logging was broken until v0.11.1 (#73) and is now verified; reset the window to 2026-06-30→07-14. Refs #73, #35. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -55,26 +55,35 @@ In normal sessions, invoke the pod's tools instead of reviewing/debugging inline
|
||||
|
||||
Each call logs an outcome to ingestion → `/pass-rate` ticks up.
|
||||
|
||||
## Step 3 — the flywheel (cold-start behavior is the gate)
|
||||
## Step 3 — how routing actually picks the model
|
||||
|
||||
The router's cold start **is** the data gate — same loop:
|
||||
Per `internal/routing/policy.go`:
|
||||
|
||||
1. With pass-rate `nil` (cold), the policy routes to the **thinking / cloud**
|
||||
model — it won't trust the local fast model without history.
|
||||
2. As `review` / `debug` accumulate **passes**, pass-rate climbs past the floor
|
||||
(`HYPERGUILD_ROUTE_LOCAL_FLOOR=0.90`) and the router starts sending those
|
||||
skills to the **local fast tier**.
|
||||
3. So early calls "pay in" on cloud to build the record; then the local tier
|
||||
activates automatically. Exercising the traffic is what spins it up.
|
||||
1. pass-rate `nil` (cold) → **local** fast tier. The router defaults to local
|
||||
from invocation #1, not to cloud — so the fast tier is exercised immediately.
|
||||
2. pass-rate `>= 0.90` (floor) → **local**; `< 0.70` (ceil) → **cloud/thinking**;
|
||||
in the `[0.70, 0.90)` band a request-hash bit samples 50/50.
|
||||
3. On a local execution error the router falls open to the thinking model for
|
||||
that one call (logged `thinking_fallback`).
|
||||
|
||||
The fast tier is now **`koala/qwen36-35b-a3b`** (Qwen3.6-35B-A3B MTP), promoted
|
||||
2026-06-29 after clearing the tool-call gate (infra `c66a195`,
|
||||
`HYPERGUILD_FAST_MODEL`).
|
||||
So you are not "paying in on cloud" — cold calls already run on the (validated)
|
||||
local fast tier **`koala/qwen36-35b-a3b`** (Qwen3.6-35B-A3B MTP, promoted
|
||||
2026-06-29, infra `c66a195`, `HYPERGUILD_FAST_MODEL`). Accumulating passes just
|
||||
keeps it there once real pass-rate is computed.
|
||||
|
||||
> **Instrumentation note (#73, fixed 2026-06-30):** until v0.11.1 the pod logged
|
||||
> successes as `"skip"` (not `"pass"`), under `skill:"_routing"`, via an
|
||||
> unauthenticated POST that silently 401'd — so `/pass-rate` stayed at zero no
|
||||
> matter how much you used it. That's fixed and verified (a real review call now
|
||||
> moves `/pass-rate?skill=review` 0→1). If you see traffic not registering,
|
||||
> re-check #73's three failure modes first.
|
||||
|
||||
## Target & verification
|
||||
|
||||
- **50 logged invocations** across `review` + `debug` within the 14-day window
|
||||
(opened 2026-06-26, **kill-date 2026-07-10**) → ~4–5 calls/day.
|
||||
- **50 logged invocations** across `review` + `debug` within the 14-day window.
|
||||
The clock restarts **2026-06-30** (the day instrumentation was verified working;
|
||||
the original 2026-06-26→07-10 window measured broken plumbing) → **kill-date
|
||||
2026-07-14**, ~4 calls/day (1 already logged from the #73 smoke test).
|
||||
- Check progress anytime:
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user