Issues #25 (skills platform overhaul) and #29 (skill-from-session generator) are both significant scopes. This issue captures the sequencing constraint surfaced in a Full Grill session (2026-05-29):
Brain injection is confirmed in all four skill handlers (#32 closed). The feedback loop is broken until this is done — investing in skill quality or skill generation before the brain is actually influencing outputs is premature.
At least one skill hits the 0.90 fast-model threshold in real routing traffic. The core hypothesis of the routing pod is that a fast local model can substitute for the thinking model on high-pass-rate skills. This has not been validated — no fast-model routing traffic has occurred yet (bootstrap note: routing defaults to thinking model until pass-rate data accumulates). If a dense 14B model on koala cannot reach 0.90 on any skill after 30 days of real usage, the routing architecture's value proposition is unproven and the scope of #25 should be reconsidered.
Gate criteria (must all be true before starting #25 or #29)
#32 closed (brain injection confirmed in all four handlers)
Routing pod has received ≥20 real skill invocations per active skill over a 30-day window
At least one skill has a GET /pass-rate?skill=<name>&window=30d score ≥ 0.90 using the fast local model (koala dense 14B or equivalent)
Canary check (run at day 30)
# From flamingo over Tailscale
curl "http://koala:30310/pass-rate?skill=review&window=30d"
curl "http://koala:30310/pass-rate?skill=debug&window=30d"
curl "http://koala:30310/pass-rate?skill=retrospective&window=30d"
curl "http://koala:30310/pass-rate?skill=trainer&window=30d"
Also run brain_answer on a known question to verify retrieval quality is non-trivial.
No skill at ≥0.90 → evaluate dense 14B model on koala; if no model reaches threshold, routing pod fast-path may not be viable — revisit #25 scope accordingly
Brain retrieval is weak → prioritise brain corpus quality before skill overhaul
#24 — lazy skill loading (may be prerequisite if call volume is low)
DECISIONS.md 2026-05-28 — field benchmark definition
## Context
Issues #25 (skills platform overhaul) and #29 (skill-from-session generator) are both significant scopes. This issue captures the sequencing constraint surfaced in a Full Grill session (2026-05-29):
**Neither #25 nor #29 should start until:**
1. **Brain injection is confirmed in all four skill handlers** (#32 closed). The feedback loop is broken until this is done — investing in skill quality or skill generation before the brain is actually influencing outputs is premature.
2. **At least one skill hits the 0.90 fast-model threshold** in real routing traffic. The core hypothesis of the routing pod is that a fast local model can substitute for the thinking model on high-pass-rate skills. This has not been validated — no fast-model routing traffic has occurred yet (bootstrap note: routing defaults to thinking model until pass-rate data accumulates). If a dense 14B model on koala cannot reach 0.90 on any skill after 30 days of real usage, the routing architecture's value proposition is unproven and the scope of #25 should be reconsidered.
## Gate criteria (must all be true before starting #25 or #29)
- [ ] #32 closed (brain injection confirmed in all four handlers)
- [ ] Routing pod has received ≥20 real skill invocations per active skill over a 30-day window
- [ ] At least one skill has a `GET /pass-rate?skill=<name>&window=30d` score ≥ 0.90 using the fast local model (koala dense 14B or equivalent)
## Canary check (run at day 30)
```bash
# From flamingo over Tailscale
curl "http://koala:30310/pass-rate?skill=review&window=30d"
curl "http://koala:30310/pass-rate?skill=debug&window=30d"
curl "http://koala:30310/pass-rate?skill=retrospective&window=30d"
curl "http://koala:30310/pass-rate?skill=trainer&window=30d"
```
Also run `brain_answer` on a known question to verify retrieval quality is non-trivial.
## If gate is not met at day 30
- Fewer than 20 calls/skill → skills are not being triggered enough; reconsider lazy-loading (#24) before #25
- No skill at ≥0.90 → evaluate dense 14B model on koala; if no model reaches threshold, routing pod fast-path may not be viable — revisit #25 scope accordingly
- Brain retrieval is weak → prioritise brain corpus quality before skill overhaul
## Related
- #25 — skills platform overhaul (blocked)
- #29 — skill-from-session generator (blocked)
- #32 — brain injection audit (must close first)
- #24 — lazy skill loading (may be prerequisite if call volume is low)
- DECISIONS.md 2026-05-28 — field benchmark definition
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Context
Issues #25 (skills platform overhaul) and #29 (skill-from-session generator) are both significant scopes. This issue captures the sequencing constraint surfaced in a Full Grill session (2026-05-29):
Neither #25 nor #29 should start until:
Brain injection is confirmed in all four skill handlers (#32 closed). The feedback loop is broken until this is done — investing in skill quality or skill generation before the brain is actually influencing outputs is premature.
At least one skill hits the 0.90 fast-model threshold in real routing traffic. The core hypothesis of the routing pod is that a fast local model can substitute for the thinking model on high-pass-rate skills. This has not been validated — no fast-model routing traffic has occurred yet (bootstrap note: routing defaults to thinking model until pass-rate data accumulates). If a dense 14B model on koala cannot reach 0.90 on any skill after 30 days of real usage, the routing architecture's value proposition is unproven and the scope of #25 should be reconsidered.
Gate criteria (must all be true before starting #25 or #29)
GET /pass-rate?skill=<name>&window=30dscore ≥ 0.90 using the fast local model (koala dense 14B or equivalent)Canary check (run at day 30)
Also run
brain_answeron a known question to verify retrieval quality is non-trivial.If gate is not met at day 30
Related