hyperguild/README.md

# hyperguild

An MCP server that acts as a disciplined AI supervisor for Claude Code sessions.
Instead of letting Claude Code do whatever it wants, hyperguild enforces structured
workflows (TDD red/green/refactor), logs every session, and accumulates learnings
into a searchable brain.

**This repo is also where `brain-mcp` lives** — the MCP server at
`brain-mcp.d-ma.be` that `brain_query`, `brain_write`, `brain_read`, `brain_edit`
and `capture` are served from. It is not a separate deployable; see
[Where brain-mcp lives](#where-brain-mcp-lives).

## Hypothesis

> We believe routing skill tasks through local models, backed by brain context,
> produces measurably better outcomes than raw Claude Code alone —
> measurable by per-skill pass rate over rolling 30-day windows
> (available at `GET /pass-rate?skill=<name>&window=30d` on the brain pod).

This is the falsifiable claim the routing pod and pass-rate infrastructure exist to test.
If per-skill pass rates don't improve over baseline (all-cloud) after 30 days of real
usage, the fast-model routing path should be reconsidered.

## Harness

**hyperguild = Claude Code + MCP.** This is a supervisor for Claude Code sessions specifically.
For multi-agent orchestration (OpenCode + LiteLLM, executor/reviewer pipelines), see
[agentsquad](https://git.d-ma.be/mathias/agentsquad) — a separate harness for a different
orchestration model. Skills (mathias/skills) are shared between both.

## How it works

```
Your Claude Code session (in any project)
    │
    │  MCP over HTTP (Tailscale)
    ├──▶ routing     :3210 (NodePort 30310 on koala) — review, debug, retrospective, trainer
    └──▶ brain       :3300 (NodePort 30330 on koala) — brain_query, brain_write, brain_ingest, session_log
                       │
                       └─ also serves the legacy REST endpoints (/query, /write, /ingest, …)
    │
    ▼
brain/
├── sessions/       — JSONL log, one file per session_id
├── wiki/           — searchable knowledge (wing/hall layout)
│   ├── homelab/
│   ├── claude-sessions/
│   └── ...
└── knowledge/      — legacy flat notes (migration pending: hyperguild#22)
```

## Where brain-mcp lives

`brain-mcp` is served from **`ingestion/internal/mcp`** in this repo, at
**`brain-mcp.d-ma.be`**. It is a component *inside* the `ingestion` service, not
a standalone deployable — one pod in the `supervisor` namespace serves the MCP
endpoint, the REST API, and the claudewatcher together.

| You are looking for | It is here |
|---|---|
| The MCP tool surface (`brain_query`, `brain_write`, `brain_read`, `brain_edit`, `capture`, …) | `ingestion/internal/mcp` |
| The HTTP/REST API (`/query`, `/write`, `/ingest`) | `ingestion/internal/api` |
| Session ingestion from Claude Code transcripts | `ingestion/internal/claudewatcher` |
| The brain *content* (notes, wings, halls) | [`mathias/brain`](https://git.d-ma.be/mathias/brain) — a separate repo |
| Deployment manifests | `config/supervisor` |

### Why it is not its own repo

Asked and decided in [#91](https://git.d-ma.be/mathias/hyperguild/issues/91)
(part 2). `internal/mcp` imports 17 of ingestion's other internal packages and
has exactly one importer — it is the MCP facade over the service's own
internals, so "move it out" resolves to renaming `ingestion`, publishing 17
packages as a shared library, or maintaining a shim. The discoverability problem
that prompted the question is answered by naming it here instead.

Full reasoning, including the reversal trigger: brain
`wiki/hyperguild/decisions/brain-mcp-stays-in-ingestion-fix-discoverability-by-name`.

## Ingest revises, it does not overwrite

Re-ingesting a source **merges into the page already on disk** rather than
replacing it. Previously the write was unconditional, so a second ingest of the
same source discarded everything accumulated since — including hand-written
sections.

The merge is deliberately asymmetric in the existing page's favour:

| On the page | In the fresh extraction | Result |
|---|---|---|
| a section | the same section | **the on-disk version wins** |
| — | a new section | added |
| a bullet list (`Sources`, `Key Claims`, …) | more bullets | unioned, no duplicates |
| an append section (`Open Questions`, `Updates`, …) | more lines | only genuinely new lines appended |
| frontmatter | frontmatter | **the on-disk frontmatter wins** |

So an extraction can add to a note but never silently rewrite what a human put
there, which is what makes it safe to run on every ingest rather than behind a
review gate.

Re-ingesting unchanged content is **byte-identical** — no diff, so brain-sync
does not commit noise that hides real edits.

Two fields are **generator-owned** and do get refreshed on a revise:
`last_updated` and `source_hash`. Everything else on disk wins. The boundary is
an explicit allowlist, so a field nobody has thought about stays human-owned.

### Disagreement is reported, not resolved

When the fresh extraction produces a *different version* of a section that
already exists, the on-disk version still wins — but the run now says so:

```
wiki/sources/shape-up.md: source now disagrees on section "Summary" — on-disk version kept
```

That line lands in `log.md` with the rest of the run's warnings. Without it, "the
source is unchanged" and "the source changed its mind and we ignored it" look
identical from the outside.

This is deliberately **textual** disagreement, not semantic contradiction: cheap,
deterministic, no model on the write path. Deciding whether a disagreement is a
genuine contradiction stays with the Judge-gated `brain-enrich` pass, which is
approval-gated by design. Detection here, adjudication there.

## Re-ingesting an unchanged source is skipped

Ingest stores `source_hash` — sha256 of the ingested **body**, frontmatter
excluded — on the source page. A later ingest of the same source compares it
first and skips extraction entirely when it matches, reporting the skip rather
than doing it silently.

Body-only is deliberate: a re-crawl that stamps a new fetch date has not changed
the source, and treating that as drift would re-extract the corpus every time a
crawler ran. A missing or unreadable hash always means "extract" — a missed skip
costs one extraction, a wrong skip loses an update.

## Phase 1 tools (available now)

| Tool | What it does |
|------|-------------|
| `session_log` | Appends a structured entry to the session JSONL log |
| `retrospective` | Reads the session log, identifies novel learnings, writes to brain |
| `review` | Structured code review via local model, brain-context injected |
| `debug` | Hypothesis-driven debugging via local model |
| `brain_query` | Full-text search over brain/wiki/ |
| `brain_write` | Writes a note to brain (with wing/hall routing) |
| `brain_answer` | BM25 + LLM synthesis — Q&A over brain corpus |
| `tier` | Returns the current connectivity tier (1=cloud, 2=LAN, 3=offline) |

> **Note:** `tdd_red/green/refactor` and `spec` were retired in Plan 7 (2026-05-12).
> They are now SKILL.md files in [mathias/skills](https://git.d-ma.be/mathias/skills).

## Connect a project

Create `.mcp.json` in your project root:

```json
{
  "mcpServers": {
    "routing": {
      "type": "http",
      "url": "http://koala:30310/mcp"
    },
    "brain": {
      "type": "http",
      "url": "http://koala:30330/mcp"
    }
  }
}
```

Two MCP servers are exposed, both reachable over Tailscale:

- **`routing`** at `koala:30310` — skill workers (`review`, `debug`, `retrospective`, `trainer`).
  Routes each call to fast local model or thinking model based on per-skill pass rate.
- **`brain`** at `koala:30330` — knowledge access (`brain_query`, `brain_write`,
  `brain_ingest`, `brain_ingest_raw`, `brain_answer`, `brain_classify`) and `session_log`.

No local binary or stdio shim is required — Claude Code talks to both via HTTP.

Open Claude Code in your project — run `/mcp` to confirm both servers are listed.

## A typical session

```
1. Call review      → brain context injected + local model review → findings
2. Call session_log → log each phase result
3. Call retrospective → extracts learnings → brain
4. Future sessions: call brain_query / brain_answer to retrieve relevant context
```

## Tier detection

The routing pod probes connectivity at call time:

| Tier | Label | Condition |
|------|-------|-----------| 
| 1 | full-online | Can reach api.anthropic.com |
| 2 | lan-only | Can reach LiteLLM but not Anthropic |
| 3 | airplane | No external connectivity |

## Model routing

The routing pod selects models per skill call based on historical pass rate:

| Pass rate | Decision |
|-----------|----------|
| ≥ 0.90 (FLOOR) | Fast model (`HYPERGUILD_FAST_MODEL`) |
| ≤ 0.70 (CEIL) | Thinking model (`HYPERGUILD_THINKING_MODEL`) |
| between CEIL and FLOOR | Sample band — probabilistic routing |
| nil (no history yet) | Defaults to thinking model |

> **Bootstrap note:** With no session history, all calls route to the thinking model.
> The fast-model path activates only after real pass-rate data accumulates at `/pass-rate`.
> Seed with real usage — don't try to pre-populate.

## Key env vars

| Variable | Default | Purpose |
|----------|---------|---------| 
| `INGEST_BRAIN_DIR` | `../brain` | Brain directory for ingestion server |
| `INGEST_PORT` | `3300` | Ingestion server port |
| `INGEST_BASE_URL` | `http://localhost:3300` | Routing pod → brain |
| `LITELLM_BASE_URL` | — | LiteLLM proxy for Tier 2 model routing |
| `ROUTING_PORT` | `3210` | Routing pod's listen port |
| `ROUTING_MCP_TOKEN` | — | Optional bearer token; when empty, no auth enforced |
| `BRAIN_URL` | `http://ingestion.supervisor:3300` | Routing pod → brain (in-cluster) |
| `HYPERGUILD_FAST_MODEL` | `koala/qwen35-9b-fast` | Fast model for high-pass-rate skill calls |
| `HYPERGUILD_THINKING_MODEL` | `iguana/gemma4-26b` | Thinking model for low-pass-rate skill calls |
| `HYPERGUILD_ROUTE_LOCAL_FLOOR` | `0.90` | Fast model threshold |
| `HYPERGUILD_ROUTE_LOCAL_CEIL` | `0.70` | Thinking model threshold |
| `HYPERGUILD_PASS_RATE_TTL_SECONDS` | `60` | Per-skill pass-rate cache TTL |

> **Operator note:** LiteLLM at `LITELLM_BASE_URL` must register both `HYPERGUILD_FAST_MODEL`
> and `HYPERGUILD_THINKING_MODEL`. If a model is missing, the fail-open retry also fails and
> the only signal is `final_status: "fail"` on `_routing` entries in the brain.

## Open issues

See [issues](https://git.d-ma.be/mathias/hyperguild/issues) — key open items:

- **#25** — skills platform overhaul (audit first, then lazy loading + brain feedback loop)
- **#24** — reduce context burn from skill listing
- **#22** — migrate legacy brain notes to wing/hall layout (one-shot script, low risk)
- **#31** — connect routing-mcp to claude.ai as custom connector