|
|
@@ -0,0 +1,263 @@
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
name: cad-babysit
|
|
|
|
|
|
|
|
description: Focused human-in-the-loop session working CAD's stuck queue (cad-stuck output). Read-only — diagnoses, never dispatches, arms a label, or closes an issue. Use when picking up whatever CAD has run into trouble on, with a human present.
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# CAD Babysit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A deliberately narrow session: pick up whatever CAD has run into trouble on, work it with a
|
|
|
|
|
|
|
|
human present, and stop. This is not an autonomous supervisor — it is a babysitter with a
|
|
|
|
|
|
|
|
person in the room, and that is the point, not a limitation to route around later.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Core principle:** an unattended supervisor reproduces the exact failure it exists to catch.
|
|
|
|
|
|
|
|
`dispatch` ran dead for eight weeks because it exited 0 and nobody noticed (`dispatch#33`); a
|
|
|
|
|
|
|
|
long-running watcher process that dies quietly has the same property one level up, and
|
|
|
|
|
|
|
|
manufactures false confidence while doing it. With a human present, **the human is the
|
|
|
|
|
|
|
|
liveness check** — that removes the objection entirely and needs no new machinery.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This skill is the ritual around `cad stuck` (`mathias/cad#14`), not a replacement for it.
|
|
|
|
|
|
|
|
Without that query it would be a checklist with no input; the query without this skill is
|
|
|
|
|
|
|
|
data nobody reads carefully. Neither closes the loop alone.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## When to Use
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Picking up CAD's stuck queue with a human present and time set aside to work it
|
|
|
|
|
|
|
|
- After `dispatch` (or any executor) reports a failure and it is not immediately obvious
|
|
|
|
|
|
|
|
whether it is an infrastructure fault or a real task outcome
|
|
|
|
|
|
|
|
- Reviewing the "awaiting human" queue (issues CI has verified but nobody has closed)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Do not use for:**
|
|
|
|
|
|
|
|
- Unattended/scheduled runs. This skill assumes a human is present for its entire duration —
|
|
|
|
|
|
|
|
if nobody is watching, its central safety property (human-as-liveness-check) does not hold.
|
|
|
|
|
|
|
|
- Dispatching, arming a redispatch label, or closing an issue. This skill only ever diagnoses;
|
|
|
|
|
|
|
|
see Iron Laws.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Iron Laws
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1. **Read-only, always.** No dispatching, no arming labels (including `status:redispatch`), no
|
|
|
|
|
|
|
|
closing or reopening an issue. A reviewer that can act on its own findings has no
|
|
|
|
|
|
|
|
independent check — control stays where it already is.
|
|
|
|
|
|
|
|
2. **Every finding cites a locator.** A run id, commit sha, comment permalink, or a specific
|
|
|
|
|
|
|
|
log line. A finding without one is an unfalsifiable opinion about a machine — do not emit
|
|
|
|
|
|
|
|
it, and do not soften this into "probably" language instead of dropping it.
|
|
|
|
|
|
|
|
3. **Re-derive, never recall.** State lives in Gitea and Actions, not in this session. Start
|
|
|
|
|
|
|
|
cold every time: run `cad stuck` fresh, do not carry forward a prior session's worklist.
|
|
|
|
|
|
|
|
Sessions get compacted and lose exactly the cross-run history that makes a stale worklist
|
|
|
|
|
|
|
|
dangerous to trust.
|
|
|
|
|
|
|
|
4. **Distinguish "blocked" from "awaiting human".** `cad stuck` already makes this split
|
|
|
|
|
|
|
|
(`stuck` vs `awaiting_human`); preserve it in every summary. The `awaiting_human` class is
|
|
|
|
|
|
|
|
the human gate working, not a bottleneck — do not report it as one.
|
|
|
|
|
|
|
|
5. **Anything that is the operator's call surfaces as `DECISION NEEDED`, never gets decided
|
|
|
|
|
|
|
|
for them.** See Output Format.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Process
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Phase 1 — Query (re-derive the worklist)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Run `cad stuck` against the estate (the `dispatch-repos.txt` scope, or whatever repo set the
|
|
|
|
|
|
|
|
operator names). This is the only input. Do not substitute a memory of a prior run, a brain
|
|
|
|
|
|
|
|
note, or this issue's own worked examples — all three can be stale within the same day (a
|
|
|
|
|
|
|
|
taxonomy row's cited live example can stop reproducing between when it was written and when
|
|
|
|
|
|
|
|
you run the query, because the underlying state genuinely changed; that is the design working,
|
|
|
|
|
|
|
|
not a bug to paper over).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Group the output by class exactly as `cad stuck` returns it: `stuck` first, `awaiting_human`
|
|
|
|
|
|
|
|
second. Pick ONE row to work per pass — this skill is deliberately narrow, not a full-estate
|
|
|
|
|
|
|
|
sweep.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Phase 2 — Read the issue's own comments FIRST
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Before anything else, before touching Actions or pod logs: read every comment on the issue,
|
|
|
|
|
|
|
|
oldest first. This step exists because it was skipped once and it cost a session — a confident
|
|
|
|
|
|
|
|
wrong root cause was produced from pod logs while the correct diagnosis sat in the issue's own
|
|
|
|
|
|
|
|
comment thread from nine days earlier (`cad-atlas#10`). The rule "read issue comments before
|
|
|
|
|
|
|
|
working" was already written down elsewhere and was not followed; it is written down again
|
|
|
|
|
|
|
|
here because that is exactly the kind of already-known thing a rushed session skips.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
What to look for: the bot's own report comments (routing decision, ground-truth verdict,
|
|
|
|
|
|
|
|
attempt count), and any human comment that already contains the diagnosis.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Phase 3 — THEN the Actions job logs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Only after Phase 2 turns up nothing conclusive, read the CI job logs:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
GET /repos/{owner}/{repo}/actions/runs # find the run, match on .run_number
|
|
|
|
|
|
|
|
GET /repos/{owner}/{repo}/actions/jobs/{job_id}/logs # returns 200, full text, secrets masked
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This endpoint works. A prior note claiming it 404s was wrong and cost a diagnosis the day this
|
|
|
|
|
|
|
|
skill was written — do not rediscover that. Get the run id from `/actions/runs`, not from the
|
|
|
|
|
|
|
|
`run_number` shown in a comment (they are different fields; match on `.run_number` to find the
|
|
|
|
|
|
|
|
internal `id` the jobs endpoint needs).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is where the two most valuable findings of the day this skill is drawn from lived: an
|
|
|
|
|
|
|
|
agent issuing the identical refused command five times and stopping (never trying the tools it
|
|
|
|
|
|
|
|
was actually permitted to use), and a different model, same denial, writing a Go program to
|
|
|
|
|
|
|
|
route around it. Neither is visible from the issue comments alone — no metric produces either
|
|
|
|
|
|
|
|
of them. They need a reader of the raw transcript.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Phase 4 — Separate infrastructure fault from task outcome
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A run that died before turn 1 says nothing about the issue itself. Before concluding anything
|
|
|
|
|
|
|
|
about the WORK, check whether the run reached the model at all:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Zero turns, a 2-3 record transcript, a "could not start" comment, or a claim label with no
|
|
|
|
|
|
|
|
attempt comment behind it → infrastructure fault. This is not evidence the issue is hard, ill
|
|
|
|
|
|
|
|
specified, or that the assigned model is unsuitable — it is evidence something upstream
|
|
|
|
|
|
|
|
(credit, a key, MCP connectivity, the executor binary) is broken.
|
|
|
|
|
|
|
|
- A run that reached the model, drove tools, and stopped on a ceiling, a denial loop, or a
|
|
|
|
|
|
|
|
genuine task failure → a real task outcome, worth diagnosing on its own terms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Four claim labels were burned this exact way in one estate sweep and the tracker read them as
|
|
|
|
|
|
|
|
four handled issues (`dispatch#33`). Get this distinction wrong and everything downstream is
|
|
|
|
|
|
|
|
about the wrong thing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Phase 5 — Decide, or say `DECISION NEEDED`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For each row worked, produce one of:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- **A closed diagnosis** — cause identified, cited to a locator, no operator judgment call
|
|
|
|
|
|
|
|
required (e.g. "infrastructure fault, credit exhausted, see comment #N — retry is automatic
|
|
|
|
|
|
|
|
on the next cycle, no action needed").
|
|
|
|
|
|
|
|
- **A `DECISION NEEDED`** — anything that is genuinely the operator's call: whether to close a
|
|
|
|
|
|
|
|
verified-but-open issue, whether to raise a ceiling, whether to abandon an issue that keeps
|
|
|
|
|
|
|
|
hitting the same denial. Never make this call on the operator's behalf.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This skill's OWN last act, every time, is to stop — not to dispatch, not to redispatch, not to
|
|
|
|
|
|
|
|
close. Ending a pass is itself the enforcement of Iron Law 1.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Output Format
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For a closed diagnosis:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
DIAGNOSIS: <repo>#<issue> — <category from cad stuck, e.g. ceiling_hit>
|
|
|
|
|
|
|
|
CAUSE: <one sentence, cites the mechanism>
|
|
|
|
|
|
|
|
LOCATOR: <url or run id / log line>
|
|
|
|
|
|
|
|
ACTION: <what happens automatically, or "none — informational">
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For anything requiring a human call:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
DECISION NEEDED: <repo>#<issue>
|
|
|
|
|
|
|
|
QUESTION: <the exact thing only the operator can decide>
|
|
|
|
|
|
|
|
OPTIONS:
|
|
|
|
|
|
|
|
1. <option> — <consequence>
|
|
|
|
|
|
|
|
2. <option> — <consequence>
|
|
|
|
|
|
|
|
RECOMMENDATION: <your pick, with reasoning — but the operator decides>
|
|
|
|
|
|
|
|
BLOCKS: <what stays stuck until this is answered, if anything>
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
End every pass with a one-line summary: `N diagnosed, M decisions needed, stopping.`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Worked Example
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`cad stuck` returns `mathias/cad-atlas#10 [near_attempt_cap] 4 dispatch attempts recorded, at
|
|
|
|
|
|
|
|
or above the near-cap threshold (4 of MaxAttempts=5).`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Phase 2 (issue comments): the issue's own thread already explains the pattern — repeated
|
|
|
|
|
|
|
|
`status:redispatch` re-arming without the label being cleared after a completed attempt.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Phase 3: not needed — Phase 2 was conclusive.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Phase 4: each of the 4 attempts reached the model and ran (not an infrastructure fault) — a
|
|
|
|
|
|
|
|
real re-fire pattern, not a claim-label artifact.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Phase 5 output:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
DECISION NEEDED: mathias/cad-atlas#10
|
|
|
|
|
|
|
|
QUESTION: one more automatic attempt remains before MaxAttempts=5 gives up permanently. Let
|
|
|
|
|
|
|
|
it fire, or intervene now (fix the underlying cause, or manually close/redirect the issue)?
|
|
|
|
|
|
|
|
OPTIONS:
|
|
|
|
|
|
|
|
1. Let it fire — costs one more headless run; if it fails the same way, dispatch gives up
|
|
|
|
|
|
|
|
automatically and posts a terminal comment. No action needed until then.
|
|
|
|
|
|
|
|
2. Intervene now — diagnose why redispatch keeps re-arming (likely: the watcher isn't
|
|
|
|
|
|
|
|
clearing RedispatchLabel after a serviced run) and fix that before the cap is spent on
|
|
|
|
|
|
|
|
a symptom rather than the cause.
|
|
|
|
|
|
|
|
RECOMMENDATION: option 2. The pattern (redispatch not self-clearing after a completed run) is
|
|
|
|
|
|
|
|
already documented as a known operational hazard (dispatch#54) — spending the last attempt
|
|
|
|
|
|
|
|
on it again teaches nothing new.
|
|
|
|
|
|
|
|
BLOCKS: nothing else; this issue's own progress.
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`1 diagnosed, 1 decision needed, stopping.`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Anti-Patterns
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Anti-Pattern | Why It Fails |
|
|
|
|
|
|
|
|
|---|---|
|
|
|
|
|
|
|
|
| Diagnosing from pod/Actions logs before reading issue comments | The exact mistake that cost a session — the answer was already written down nine days earlier. |
|
|
|
|
|
|
|
|
| Reporting `awaiting_human` rows as a backlog or bottleneck | It is the human gate working. Misreporting it manufactures urgency that isn't there and trains the operator to distrust the queue. |
|
|
|
|
|
|
|
|
| Treating a zero-turn, could-not-start run as evidence about the issue | It is an infrastructure fault. Four claim labels were burned exactly this way and read as four handled issues. |
|
|
|
|
|
|
|
|
| Deciding a `DECISION NEEDED` case on the operator's behalf "to save time" | Removes the one property that makes this skill safe to run unattended-adjacent: the human decides, this skill only prepares the decision. |
|
|
|
|
|
|
|
|
| Applying `status:redispatch`, closing an issue, or adding any label from this skill | Control stays where it is. This skill has no write path and should never be given one. |
|
|
|
|
|
|
|
|
| Carrying forward a worklist from an earlier session instead of re-running `cad stuck` | State changes between sessions (see the worked note on `brain-gardener#190` in `cad#14`'s own acceptance run — its trigger label was gone hours later). A stale worklist gives false confidence, which is the specific failure mode this whole skill exists to avoid. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Honest Limitation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This skill encodes the *repeatable* diagnostic sequence: read comments, then logs, separate
|
|
|
|
|
|
|
|
fault from outcome, verify by exit code, re-derive rather than recall. It reliably catches the
|
|
|
|
|
|
|
|
taxonomy `cad stuck` already knows about. It will not, by itself, produce a genuinely novel
|
|
|
|
|
|
|
|
finding the way an unhurried human-plus-agent reading session did the day this skill was
|
|
|
|
|
|
|
|
written — that came from noticing things (an agent editing the file that authorizes it, an
|
|
|
|
|
|
|
|
issue body recommending a command the policy forbids) that no checklist anticipates. Expect
|
|
|
|
|
|
|
|
this skill to be reliable on known failure modes and to miss the next new one. That is an
|
|
|
|
|
|
|
|
acceptable trade as long as nobody mistakes a clean pass through this skill for "nothing is
|
|
|
|
|
|
|
|
wrong" — it means "nothing on the known list is wrong."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Brain MCP Integration
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**At session start:**
|
|
|
|
|
|
|
|
- Run `brain_query` for the domain (`cad`, `dispatch`, the specific repo) to check whether a
|
|
|
|
|
|
|
|
finding this session is about to produce has already been diagnosed and written up. Do not
|
|
|
|
|
|
|
|
re-diagnose a known pattern from scratch every session.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**After a genuinely novel finding (not already in the taxonomy or the brain):**
|
|
|
|
|
|
|
|
- `brain_write` the pattern, generalized beyond the specific issue — the failure mode, not the
|
|
|
|
|
|
|
|
issue number. Do not write routine, already-known-taxonomy findings; that is what `cad
|
|
|
|
|
|
|
|
stuck`'s own output already records, durably, in Gitea.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Never:**
|
|
|
|
|
|
|
|
- Write a `DECISION NEEDED` outcome to the brain before the operator has actually decided —
|
|
|
|
|
|
|
|
that would record a guess as a resolution.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Logging
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Call `session_log` once at the end of every phase to record the outcome.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**At end of each phase:**
|
|
|
|
|
|
|
|
- `session_log` with `{skill: "cad-babysit", phase: "<phase-name>", final_status: "pass" |
|
|
|
|
|
|
|
|
"fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root:
|
|
|
|
|
|
|
|
"<absolute path>"}`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Phases for this skill:** query, read-comments, read-actions-logs, classify-fault-vs-outcome,
|
|
|
|
|
|
|
|
decide
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**Status semantics:**
|
|
|
|
|
|
|
|
- `pass` — the phase's intended outcome was reached (worklist re-derived; comments read;
|
|
|
|
|
|
|
|
fault/outcome separated; a diagnosis or `DECISION NEEDED` was produced).
|
|
|
|
|
|
|
|
- `fail` — the phase could not be completed (e.g. `cad stuck` errored, Gitea was unreachable).
|
|
|
|
|
|
|
|
- `skip` — phase intentionally skipped (e.g. Phase 3 skipped because Phase 2 was conclusive).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Mode 2 Routing Note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Phases 2-4 (reading comments/logs, classifying fault vs outcome) are judgment-heavy and should
|
|
|
|
|
|
|
|
stay Mode 1. Phase 1 (running `cad stuck` and grouping its output) is mechanical and a Mode 2
|
|
|
|
|
|
|
|
routing candidate once Plan 6 ships the routing pod.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Cross-References
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Depends on `mathias/cad#14`'s `cad stuck` query — this skill is the ritual around it, not a
|
|
|
|
|
|
|
|
replacement.
|
|
|
|
|
|
|
|
- Load `debug` for the hypothesis-generation discipline if Phase 4's fault/outcome split is
|
|
|
|
|
|
|
|
itself ambiguous.
|
|
|
|
|
|
|
|
- Load `secret-hygiene` before reading raw Actions job logs if the repo is client-tagged —
|
|
|
|
|
|
|
|
logs are secret-masked by Gitea but a pasted excerpt into a summary is not.
|
|
|
|
|
|
|
|
- `mathias/cad-atlas#11` renders the same `cad stuck` output as a pipeline overlay — read that
|
|
|
|
|
|
|
|
instead of this skill's text output when a visual "where is everything stuck" view is wanted
|
|
|
|
|
|
|
|
over working one row in depth.
|