mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

skills(platform): major overhaul — succinct + brain-backed + lazy + reconsider MCP-vs-CLI #25

New Issue
Open
opened 2026-05-21 19:39:57 +00:00 by mathias · 2 comments
mathias commented 2026-05-21 19:39:57 +00:00
Owner
Copy Link
Copy Source

Background

The agent platform ships a growing set of skills (general engineering: TDD, SOLID, refactoring, problem-analysis, grill-me, code-review, cognitive-load, etc.) plus harness-specific skills (caveman, gsd-*, etc.). They are loaded eagerly by harness invocation, consume meaningful input tokens at session start (especially in scaffolded projects that inherit all general skills), and have no built-in feedback loop from brain.

Scaffolded projects (template-go-agent, template-go-web, and per-repo .skills softlinks per wire-skills) carry the same general skill set regardless of whether the project actually exercises it. Token spend is paid every session.

The MCP layer (brain MCP, gitea MCP, infra MCP, routing MCP) duplicates capability that, in some cases, would be lighter as direct CLI tools or thin REST clients inside a skill — particularly for read-heavy paths (brain_query, repo browse, kubectl get).

Goals

  1. Sharper skills. Each skill is specific, succinct, focused on one decision or workflow. No catch-all skills. Triggers and use-when criteria explicit.
  2. Brain-backed learning. Skills cite + draw from brain entries; a skill that fails or surprises an operator writes a brain note that future invocations consult. Make the trainer skill or an analogue the closed loop.
  3. Lazy loading. Skills should be discoverable from a small index, loaded on demand (per harness capability — Skill tool already supports this), not eagerly injected.
  4. Reconsider MCP boundary. For each MCP server, evaluate whether the agent-facing surface should remain MCP, or move to a CLI binary + skill wrapper, or a thin REST client embedded in a skill. Decision per server, not blanket.

Scope

  • General skills in scaffolded projects (template-go-agent, template-go-web, per-repo .skills)
  • Harness-level skills under ~/.claude/skills and the GSD skill cluster
  • SKILLS_INDEX.md schema and lazy-loading conventions
  • MCP servers maintained by Mathias: brain-mcp, gitea-mcp, infra-mcp, routing-mcp
  • Brain feedback loop wiring (trainer / session-retrospective integration)

Out of scope (this issue)

  • Replacing existing agent harnesses (Claude Code, Crush, Pi, Antigravity)
  • Brain corpus content refactor
  • Specific business-app skill content (cobalt-dingo, agentsquad)

Acceptance criteria

  • Audit: list every active skill across ~/.claude/skills, ~/dev/.skills, scaffolded .skills softlinks, and harness-specific skills. Score each on (a) used-this-month, (b) token cost when loaded, (c) overlap with another skill, (d) brain-citation count.
  • Skill quality bar codified: one decision/workflow per skill, explicit triggers, max-token-budget per skill, brain-citation requirement on rules of thumb.
  • Lazy-loading scheme: every harness that supports it loads skills via Skill tool (or equivalent) from an index; only the index loads at session start.
  • Brain feedback loop: defined trigger (skill failure, operator correction, surprise) writes a brain entry; future skill load surfaces relevant brain hits as context.
  • MCP-vs-CLI-vs-REST decision matrix: per server, a written verdict with rationale. Action items derived per server (keep / convert / split).
  • Migration plan: ordered set of issues / commits to move from current state to target state. Each step ships independently.

Risks

  • Over-narrowing skills creates a proliferation of micro-skills that themselves become hard to discover.
  • Lazy loading shifts cost from session start to skill load; net token spend may not improve without measurement.
  • Moving MCP capability to CLI/REST changes auth surface — each path needs the same Bearer model or it regresses security.

Forward references

  • Touches the mathias/hyperguild agent platform repo, mathias/template-go-agent, mathias/template-go-web, the four MCP server repos, and possibly ~/.claude/skills globally.
  • Related: docs/superpowers/specs/2026-05-homelab-architecture-review.md in mathias/infra flagged MCP federation cognitive load (620/1000); this issue is the agent-side counterpart.

First step

Audit existing skills. Output a scored table (one row per skill) into docs/skills-audit-2026-05.md here. Block on that before any deletion / rewrite / migration.

## Background The agent platform ships a growing set of skills (general engineering: TDD, SOLID, refactoring, problem-analysis, grill-me, code-review, cognitive-load, etc.) plus harness-specific skills (caveman, gsd-*, etc.). They are loaded eagerly by harness invocation, consume meaningful input tokens at session start (especially in scaffolded projects that inherit all general skills), and have no built-in feedback loop from brain. Scaffolded projects (`template-go-agent`, `template-go-web`, and per-repo `.skills` softlinks per `wire-skills`) carry the same general skill set regardless of whether the project actually exercises it. Token spend is paid every session. The MCP layer (brain MCP, gitea MCP, infra MCP, routing MCP) duplicates capability that, in some cases, would be lighter as direct CLI tools or thin REST clients inside a skill — particularly for read-heavy paths (`brain_query`, repo browse, `kubectl get`). ## Goals 1. **Sharper skills.** Each skill is specific, succinct, focused on one decision or workflow. No catch-all skills. Triggers and use-when criteria explicit. 2. **Brain-backed learning.** Skills cite + draw from brain entries; a skill that fails or surprises an operator writes a brain note that future invocations consult. Make the trainer skill or an analogue the closed loop. 3. **Lazy loading.** Skills should be discoverable from a small index, loaded on demand (per harness capability — Skill tool already supports this), not eagerly injected. 4. **Reconsider MCP boundary.** For each MCP server, evaluate whether the agent-facing surface should remain MCP, or move to a CLI binary + skill wrapper, or a thin REST client embedded in a skill. Decision per server, not blanket. ## Scope - General skills in scaffolded projects (`template-go-agent`, `template-go-web`, per-repo `.skills`) - Harness-level skills under `~/.claude/skills` and the GSD skill cluster - `SKILLS_INDEX.md` schema and lazy-loading conventions - MCP servers maintained by Mathias: brain-mcp, gitea-mcp, infra-mcp, routing-mcp - Brain feedback loop wiring (trainer / session-retrospective integration) ## Out of scope (this issue) - Replacing existing agent harnesses (Claude Code, Crush, Pi, Antigravity) - Brain corpus content refactor - Specific business-app skill content (cobalt-dingo, agentsquad) ## Acceptance criteria - [ ] Audit: list every active skill across `~/.claude/skills`, `~/dev/.skills`, scaffolded `.skills` softlinks, and harness-specific skills. Score each on (a) used-this-month, (b) token cost when loaded, (c) overlap with another skill, (d) brain-citation count. - [ ] Skill quality bar codified: one decision/workflow per skill, explicit triggers, max-token-budget per skill, brain-citation requirement on rules of thumb. - [ ] Lazy-loading scheme: every harness that supports it loads skills via `Skill` tool (or equivalent) from an index; only the index loads at session start. - [ ] Brain feedback loop: defined trigger (skill failure, operator correction, surprise) writes a brain entry; future skill load surfaces relevant brain hits as context. - [ ] MCP-vs-CLI-vs-REST decision matrix: per server, a written verdict with rationale. Action items derived per server (keep / convert / split). - [ ] Migration plan: ordered set of issues / commits to move from current state to target state. Each step ships independently. ## Risks - Over-narrowing skills creates a proliferation of micro-skills that themselves become hard to discover. - Lazy loading shifts cost from session start to skill load; net token spend may not improve without measurement. - Moving MCP capability to CLI/REST changes auth surface — each path needs the same Bearer model or it regresses security. ## Forward references - Touches the `mathias/hyperguild` agent platform repo, `mathias/template-go-agent`, `mathias/template-go-web`, the four MCP server repos, and possibly `~/.claude/skills` globally. - Related: `docs/superpowers/specs/2026-05-homelab-architecture-review.md` in `mathias/infra` flagged MCP federation cognitive load (620/1000); this issue is the agent-side counterpart. ## First step Audit existing skills. Output a scored table (one row per skill) into `docs/skills-audit-2026-05.md` here. Block on that before any deletion / rewrite / migration.
mathias commented 2026-06-26 14:52:54 +00:00
Author
Owner
Copy Link
Copy Source

BLOCKED on #35 (data gate) — sequencing guard

Per #34 + #35 deliverable 3: do not start the skills overhaul until /pass-rate has real data. Confirmed live 2026-06-26 — the endpoint works (ingestion:30330/pass-rate?skill=review&window=30d) but returns total:0 for both review and debug. The brain-feedback loop this overhaul depends on is meaningless until pass-rate data accrues, so building it now is building on sand. Unblocks when #35's 14-day data gate produces a populated dataset (and ideally one skill ≥0.90 on the fast model).

## BLOCKED on #35 (data gate) — sequencing guard Per #34 + #35 deliverable 3: do **not** start the skills overhaul until `/pass-rate` has real data. Confirmed live 2026-06-26 — the endpoint works (`ingestion:30330/pass-rate?skill=review&window=30d`) but returns `total:0` for both `review` and `debug`. The brain-feedback loop this overhaul depends on is meaningless until pass-rate data accrues, so building it now is building on sand. Unblocks when #35's 14-day data gate produces a populated dataset (and ideally one skill ≥0.90 on the fast model).
mathias commented 2026-08-19 18:19:48 +00:00
Author
Owner
Copy Link
Copy Source

Re-scope note per #75 (closed 2026-08-19), not closing this one — most of it still stands:

  • Its blocking gate (#34) is closed as obsolete — the routing-pod pass-rate validation it waited on was retired, not passed. This issue no longer has that gate; it can start whenever prioritized.
  • The "MCP servers maintained by Mathias" list in scope needs updating: routing-mcp no longer exists live (iceboxed with cmd/routing) — drop it from the MCP-vs-CLI decision matrix. Remaining scope: brain-mcp, gitea-mcp, infra-mcp (verify that one's still live), plus general skill-quality/lazy-loading/brain-feedback-loop work — all untouched by #75.
  • internal/skills/{review,debug,retrospective,trainer,project} — also iceboxed, not "eagerly-loaded skills consuming tokens" anymore since they don't run. If any of them get resurrected as skills on the new single-harness architecture, that's new scope here, not audit of dead code.

Everything else in this issue (skill audit, lazy-loading scheme, brain-citation requirement) is orthogonal to the routing decision and still real work. Cascade re-triage per hyperguild#75, 2026-08-19.

Re-scope note per #75 (closed 2026-08-19), not closing this one — most of it still stands: - **Its blocking gate (#34) is closed as obsolete** — the routing-pod pass-rate validation it waited on was retired, not passed. This issue no longer has that gate; it can start whenever prioritized. - **The "MCP servers maintained by Mathias" list in scope needs updating**: `routing-mcp` no longer exists live (iceboxed with `cmd/routing`) — drop it from the MCP-vs-CLI decision matrix. Remaining scope: brain-mcp, gitea-mcp, infra-mcp (verify that one's still live), plus general skill-quality/lazy-loading/brain-feedback-loop work — all untouched by #75. - **`internal/skills/{review,debug,retrospective,trainer,project}`** — also iceboxed, not "eagerly-loaded skills consuming tokens" anymore since they don't run. If any of them get resurrected as skills on the *new* single-harness architecture, that's new scope here, not audit of dead code. Everything else in this issue (skill audit, lazy-loading scheme, brain-citation requirement) is orthogonal to the routing decision and still real work. Cascade re-triage per hyperguild#75, 2026-08-19.
mathias added the risk/mediumsize/ltype/research labels 2026-08-27 05:25:41 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels risk/medium size/l type/research
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#25