Should SKILL.md define explicit escalation / failure conditions? #5

Closed
opened 2026-06-17 20:10:37 +00:00 by mathias · 1 comment
Owner

Question

Does the SKILL.md format need first-class fields for failure criteria and escalation / hand-back rules — i.e. "when should this capability stop and hand control back"?

Prompt

Allie K. Miller's "How I built my AI agent workforce" (2026-06-17) frames agent roles as job descriptions with four required fields: goals, success criteria, failure criteria, and escalation rules. Our SKILL.md files cover capability/purpose (≈ goals) and implicitly success, but do not define explicit stop/handback conditions. The routing pod decides which model; nothing defines when an agent role should escalate or abort.

Why it might matter

  • Maps onto the assessor-loop return path / conditional-approval shape (residual + re_attest_by): an escalation rule is the same structural object as a conditional approval — "proceed unless X, in which case hand back."
  • Currently these conditions are implicit per role across hyperguild / agentsquad / Crush. Implicit = unauditable.

Scope of this issue

  1. Audit: do current skills have implicit-only escalation/abort conditions, or none?
  2. Decide: should SKILL.md gain optional failure_criteria / escalation fields (Agent Skills standard permitting)?
  3. If yes: define the field semantics and how a harness is expected to honour them.

Refs

  • brain: wiki/homelab/decisions/no-centralized-cross-harness-observer-2026-06-17.md (sibling decision from same source read)
  • Related: hyperguild brain-injection verification (issue #32) for the hyperguild-side check.

Not blocking anything; sharpening question raised during an architecture read.

## Question Does the SKILL.md format need first-class fields for **failure criteria** and **escalation / hand-back rules** — i.e. "when should this capability stop and hand control back"? ## Prompt Allie K. Miller's "How I built my AI agent workforce" (2026-06-17) frames agent roles as job descriptions with four required fields: goals, success criteria, **failure criteria**, and **escalation rules**. Our SKILL.md files cover capability/purpose (≈ goals) and implicitly success, but do **not** define explicit stop/handback conditions. The routing pod decides *which model*; nothing defines *when an agent role should escalate or abort*. ## Why it might matter - Maps onto the assessor-loop return path / conditional-approval shape (`residual` + `re_attest_by`): an escalation rule is the same structural object as a conditional approval — "proceed unless X, in which case hand back." - Currently these conditions are implicit per role across hyperguild / agentsquad / Crush. Implicit = unauditable. ## Scope of this issue 1. Audit: do current skills have implicit-only escalation/abort conditions, or none? 2. Decide: should SKILL.md gain optional `failure_criteria` / `escalation` fields (Agent Skills standard permitting)? 3. If yes: define the field semantics and how a harness is expected to honour them. ## Refs - brain: `wiki/homelab/decisions/no-centralized-cross-harness-observer-2026-06-17.md` (sibling decision from same source read) - Related: hyperguild brain-injection verification (issue #32) for the hyperguild-side check. _Not blocking anything; sharpening question raised during an architecture read._
mathias added the risk/lowsize/stype/decision labels 2026-08-27 05:25:45 +00:00
Author
Owner

Decision

No new frontmatter fields. An ## Escalation section in the body, gated by task check for skills that can act. Decided by Mathias, 2026-08-29.

Audit (scope item 1)

Both, depending on the skill — and the split is informative.

Skills written here mostly do state their stop conditions, in prose, in the body:

  • debug — "Diagnose only — do not fix in this skill. The fix happens in a separate TDD cycle" (Iron Law 3).
  • cad-babysit — Iron Laws: never closes an issue, never dispatches, never writes a label; anything that is the operator's call surfaces as DECISION NEEDED with question/options/recommendation/blocks.
  • code-review — "This skill is read-only analysis."
  • secret-hygiene — an entire recovery-and-rotation path for the failure case.

Vendored superpowers-* skills mostly do not, and where they gesture at it they hand back to "your human partner", which is obra's convention rather than an operator this fleet has.

So the honest finding is not "conditions are implicit" but "the good skills already make them explicit, in the body, and nothing enforces that they do".

Why not the frontmatter fields (scope item 2)

Three reasons, in order of weight:

  1. No harness reads them. Claude Code consumes name and description; Crush, Pi and OpenCode read the file. A failure_criteria: key would be documentation that nothing honours, and the failure mode of that is worse than absence — it reads as a guarantee.
  2. Structured fields do not survive contact with the model. The stop condition has to be in the text the model actually reads at the point it is about to act. cad-babysit's Iron Laws work because they are prose in the middle of the procedure, not metadata at the top.
  3. This repo just spent a session learning that unenforced conventions drift — #15's index rows, #29's surfaces field. A field with no gate is the exact shape that drifts, and a gate on a field nothing consumes is ceremony.

The structural parallel to assessor-loop's residual + re_attest_by in the issue body is real, but it argues for the content being explicit, not for it being frontmatter. A conditional approval is prose in an attestation too.

How it gets enforced (scope item 3)

A skill that can act — write files, run a deploy, dispatch an agent, mutate a tracker, touch credentials — carries:

## Escalation

Stop and hand back when:
- <condition>
- <condition>

Hand back by: <the specific mechanism — DECISION NEEDED block, an issue comment, refusing the tool call>

task check fails when a skill in the acting set has no such section. Advisory and read-only skills are exempt and say so.

Implementation is filed as #37 rather than done here: the gate is small, but writing a real escalation rule for each acting skill is per-skill work under #19's bar, not a mechanical edit. This issue was the decision, and the decision is made.

## Decision **No new frontmatter fields. An `## Escalation` section in the body, gated by `task check` for skills that can act.** Decided by Mathias, 2026-08-29. ### Audit (scope item 1) Both, depending on the skill — and the split is informative. Skills written *here* mostly do state their stop conditions, in prose, in the body: - `debug` — "Diagnose only — do not fix in this skill. The fix happens in a separate TDD cycle" (Iron Law 3). - `cad-babysit` — Iron Laws: never closes an issue, never dispatches, never writes a label; anything that is the operator's call surfaces as `DECISION NEEDED` with question/options/recommendation/blocks. - `code-review` — "This skill is read-only analysis." - `secret-hygiene` — an entire recovery-and-rotation path for the failure case. Vendored `superpowers-*` skills mostly do not, and where they gesture at it they hand back to "your human partner", which is obra's convention rather than an operator this fleet has. So the honest finding is not "conditions are implicit" but "the good skills already make them explicit, in the body, and nothing enforces that they do". ### Why not the frontmatter fields (scope item 2) Three reasons, in order of weight: 1. **No harness reads them.** Claude Code consumes `name` and `description`; Crush, Pi and OpenCode read the file. A `failure_criteria:` key would be documentation that nothing honours, and the failure mode of that is worse than absence — it reads as a guarantee. 2. **Structured fields do not survive contact with the model.** The stop condition has to be in the text the model actually reads at the point it is about to act. `cad-babysit`'s Iron Laws work because they are prose in the middle of the procedure, not metadata at the top. 3. **This repo just spent a session learning that unenforced conventions drift** — #15's index rows, #29's surfaces field. A field with no gate is the exact shape that drifts, and a gate on a field nothing consumes is ceremony. The structural parallel to assessor-loop's `residual` + `re_attest_by` in the issue body is real, but it argues for the *content* being explicit, not for it being frontmatter. A conditional approval is prose in an attestation too. ### How it gets enforced (scope item 3) A skill that can **act** — write files, run a deploy, dispatch an agent, mutate a tracker, touch credentials — carries: ```markdown ## Escalation Stop and hand back when: - <condition> - <condition> Hand back by: <the specific mechanism — DECISION NEEDED block, an issue comment, refusing the tool call> ``` `task check` fails when a skill in the acting set has no such section. Advisory and read-only skills are exempt and say so. Implementation is filed as **#37** rather than done here: the gate is small, but writing a real escalation rule for each acting skill is per-skill work under #19's bar, not a mechanical edit. This issue was the decision, and the decision is made.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/skills#5