Compare commits

..
2 Commits
Author SHA1 Message Date
mathiasandClaude Opus 5.5 21a53c4159 fix(ingestion): confine /ingest-path and brain_ingest paths to brain/raw/
CI / Lint / Test / Vet (pull_request) Successful in 35s
CI / Mirror to GitHub (pull_request) Skipped
Both read a caller-chosen server path with extract.Text, so any file the
pod could read (the ServiceAccount token, /etc/passwd) could be ingested
into the brain and read back with brain_query.

ResolveIngestPath resolves relative paths against brain/raw/ (the inbox
the watcher already owns), evaluates symlinks, and refuses anything
that lands outside. A directory ingest re-resolves every walked file,
so a symlinked file inside raw/ cannot lead out. Existing ingest-path
tests moved their fixtures under brain/raw/.

Refs #149

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 16:17:45 +02:00
mathiasandClaude Opus 5.5 99fedeaa98 fix(ingestion): authenticate the REST API as a whole, not route by route
Only /mcp was wrapped in BearerMiddleware; /query, /write, /ingest*,
/promote, /pending and the rest on the same mux had no auth, and
infra#517 found them on the internet (mitigated there by narrowing the
ingress). They are still open on the tailnet and LAN NodePort 30330.

mountREST puts the REST routes on their own mux, mounted at "/" behind
the same bearer middleware and credential as /mcp. Routes registered on
the outer mux stay explicitly public (/mcp has its own auth, /capture,
the HMAC webhook, OAuth metadata, /metrics); anything else, including a
route added later, is authenticated by default.

The hyperguild CLI (cmd/hyperguild and internal/brain) now sends
BRAIN_MCP_TOKEN, which it never did. brain-ui and brain-ingest-ntfy
already send theirs.

Refs #149

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 16:17:45 +02:00
12 changed files with 75 additions and 200 deletions
+1 -5
View File
@@ -64,11 +64,7 @@ the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`, The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300 `/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients, **with the same bearer token as `/mcp`** for shell scripts and non-MCP clients.
(`BRAIN_MCP_TOKEN`; hyperguild#149). It is not served on the public host
(infra#517): use the tailnet (`koala:8443`) or in-cluster
`ingestion.supervisor.svc:3300`. `/ingest-path` and `brain_ingest` `path`
accept only paths inside `brain/raw/`.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai `brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title, gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
+2 -6
View File
@@ -304,7 +304,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
|---------|-------|-------| |---------|-------|-------|
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool | | **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same | | **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` | | **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files | | **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
- **Scoping — the brain holds no confidential client content.** Nothing on the read path - **Scoping — the brain holds no confidential client content.** Nothing on the read path
@@ -493,11 +493,7 @@ the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`, The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300 `/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients, **with the same bearer token as `/mcp`** for shell scripts and non-MCP clients.
(`BRAIN_MCP_TOKEN`; hyperguild#149). It is not served on the public host
(infra#517): use the tailnet (`koala:8443`) or in-cluster
`ingestion.supervisor.svc:3300`. `/ingest-path` and `brain_ingest` `path`
accept only paths inside `brain/raw/`.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai `brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title, gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
+2 -6
View File
@@ -299,7 +299,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
|---------|-------|-------| |---------|-------|-------|
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool | | **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same | | **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` | | **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files | | **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
- **Scoping — the brain holds no confidential client content.** Nothing on the read path - **Scoping — the brain holds no confidential client content.** Nothing on the read path
@@ -488,11 +488,7 @@ the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`, The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300 `/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients, **with the same bearer token as `/mcp`** for shell scripts and non-MCP clients.
(`BRAIN_MCP_TOKEN`; hyperguild#149). It is not served on the public host
(infra#517): use the tailnet (`koala:8443`) or in-cluster
`ingestion.supervisor.svc:3300`. `/ingest-path` and `brain_ingest` `path`
accept only paths inside `brain/raw/`.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai `brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title, gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
+1 -5
View File
@@ -64,11 +64,7 @@ the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`, The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300 `/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients, **with the same bearer token as `/mcp`** for shell scripts and non-MCP clients.
(`BRAIN_MCP_TOKEN`; hyperguild#149). It is not served on the public host
(infra#517): use the tailnet (`koala:8443`) or in-cluster
`ingestion.supervisor.svc:3300`. `/ingest-path` and `brain_ingest` `path`
accept only paths inside `brain/raw/`.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai `brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title, gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
+9 -8
View File
@@ -64,7 +64,9 @@ func (c *brainClient) Query(ctx context.Context, topic string, limit int) (*Quer
return nil, fmt.Errorf("build request: %w", err) return nil, fmt.Errorf("build request: %w", err)
} }
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
resp, err := c.do(req) c.authorize(req)
resp, err := c.http.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("brain POST /query: %w", err) return nil, fmt.Errorf("brain POST /query: %w", err)
} }
@@ -106,7 +108,9 @@ func (c *brainClient) Write(ctx context.Context, wing, hall, filename string, co
return nil, fmt.Errorf("build request: %w", err) return nil, fmt.Errorf("build request: %w", err)
} }
req.Header.Set("Content-Type", "application/json") req.Header.Set("Content-Type", "application/json")
resp, err := c.do(req) c.authorize(req)
resp, err := c.http.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("brain POST /write: %w", err) return nil, fmt.Errorf("brain POST /write: %w", err)
} }
@@ -143,7 +147,7 @@ func (c *brainClient) PassRate(ctx context.Context, skill, window string) (*Pass
if err != nil { if err != nil {
return nil, fmt.Errorf("build request: %w", err) return nil, fmt.Errorf("build request: %w", err)
} }
resp, err := c.do(req) resp, err := c.http.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("brain GET /pass-rate: %w", err) return nil, fmt.Errorf("brain GET /pass-rate: %w", err)
} }
@@ -159,12 +163,9 @@ func (c *brainClient) PassRate(ctx context.Context, skill, window string) (*Pass
return &out, nil return &out, nil
} }
// do sends every brain request, adding the bearer token when one is // authorize adds the bearer token when one is configured.
// configured. All methods go through here so none can forget it (PassRate func (c *brainClient) authorize(req *http.Request) {
// did, PR #152 review).
func (c *brainClient) do(req *http.Request) (*http.Response, error) {
if c.token != "" { if c.token != "" {
req.Header.Set("Authorization", "Bearer "+c.token) req.Header.Set("Authorization", "Bearer "+c.token)
} }
return c.http.Do(req)
} }
+3 -10
View File
@@ -13,17 +13,13 @@ import (
// hyperguild#149: the ingestion REST API now requires a bearer token; the CLI // hyperguild#149: the ingestion REST API now requires a bearer token; the CLI
// sends BRAIN_MCP_TOKEN, or it gets 401 on every brain call. // sends BRAIN_MCP_TOKEN, or it gets 401 on every brain call.
func TestBrainClientSendsBrainMCPTokenOnQueryWriteAndPassRate(t *testing.T) { func TestBrainClientSendsBrainMCPToken(t *testing.T) {
var got []string var got []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
got = append(got, r.URL.Path+" "+r.Header.Get("Authorization")) got = append(got, r.URL.Path+" "+r.Header.Get("Authorization"))
switch r.URL.Path { if r.URL.Path == "/write" {
case "/write":
_, _ = w.Write([]byte(`{"path":"wiki/x/facts/y.md"}`)) _, _ = w.Write([]byte(`{"path":"wiki/x/facts/y.md"}`))
return return
case "/pass-rate":
_, _ = w.Write([]byte(`{}`))
return
} }
_, _ = w.Write([]byte(`{"results":[]}`)) _, _ = w.Write([]byte(`{"results":[]}`))
})) }))
@@ -36,10 +32,7 @@ func TestBrainClientSendsBrainMCPTokenOnQueryWriteAndPassRate(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
_, err = c.Write(context.Background(), "x", "facts", "y", strings.NewReader("# y")) _, err = c.Write(context.Background(), "x", "facts", "y", strings.NewReader("# y"))
require.NoError(t, err) require.NoError(t, err)
// PR #152 review: PassRate sent no token and would 401 in production. assert.Equal(t, []string{"/query Bearer cli-test-token", "/write Bearer cli-test-token"}, got)
_, _ = c.PassRate(context.Background(), "tdd", "7d")
assert.Equal(t, []string{"/query Bearer cli-test-token", "/write Bearer cli-test-token",
"/pass-rate Bearer cli-test-token"}, got)
} }
func TestBrainClientSendsNoAuthorizationWhenTokenUnset(t *testing.T) { func TestBrainClientSendsNoAuthorizationWhenTokenUnset(t *testing.T) {
@@ -24,7 +24,7 @@ Only a `tools/call` to the routing pod records a pass/fail.
|---------|-----|------| |---------|-----|------|
| Routing MCP (local, Tailscale) | `http://koala:30310/mcp` | Bearer `ROUTING_MCP_TOKEN` | | Routing MCP (local, Tailscale) | `http://koala:30310/mcp` | Bearer `ROUTING_MCP_TOKEN` |
| Routing MCP (remote) | `https://routing-mcp.d-ma.be/mcp` | OAuth via `auth.d-ma.be` (audience `claude-ai`) | | Routing MCP (remote) | `https://routing-mcp.d-ma.be/mcp` | OAuth via `auth.d-ma.be` (audience `claude-ai`) |
| Pass-rate readout | `http://koala:30330/pass-rate?skill=<name>` | Bearer `BRAIN_MCP_TOKEN` (since hyperguild#149) | | Pass-rate readout | `http://koala:30330/pass-rate?skill=<name>` | none (read-only) |
Tools advertised: **`review`**, **`debug`** (the two the #35 gate measures), Tools advertised: **`review`**, **`debug`** (the two the #35 gate measures),
plus `session_log`, `retrospective`, `trainer`. plus `session_log`, `retrospective`, `trainer`.
+15 -19
View File
@@ -503,10 +503,9 @@ func main() {
logger.Info("embed sync started", "interval_s", embedSyncInterval) logger.Info("embed sync started", "interval_s", embedSyncInterval)
} }
// Public routes only; buildMux mounts the authenticated REST API (#149). mux := http.NewServeMux()
public := map[string]http.Handler{}
if webhookHandler != nil { if webhookHandler != nil {
public["POST /webhooks/brain-sync"] = webhookHandler mux.Handle("POST /webhooks/brain-sync", webhookHandler)
} }
// Resource-metadata URL is only emitted on 401 when Dex OAuth is // Resource-metadata URL is only emitted on 401 when Dex OAuth is
// configured. Static-Bearer-only deployments leave this empty so // configured. Static-Bearer-only deployments leave this empty so
@@ -514,14 +513,21 @@ func main() {
var resourceMetadataURL string var resourceMetadataURL string
if dexURL := os.Getenv("DEX_ISSUER_URL"); dexURL != "" { if dexURL := os.Getenv("DEX_ISSUER_URL"); dexURL != "" {
resourceURL := os.Getenv("MCP_RESOURCE_URL") resourceURL := os.Getenv("MCP_RESOURCE_URL")
public["GET /.well-known/oauth-protected-resource"] = http.HandlerFunc( mux.HandleFunc("GET /.well-known/oauth-protected-resource",
chassisauth.ProtectedResourceHandler(resourceURL, dexURL)) chassisauth.ProtectedResourceHandler(resourceURL, dexURL))
if resourceURL != "" { if resourceURL != "" {
resourceMetadataURL = strings.TrimRight(resourceURL, "/") + "/.well-known/oauth-protected-resource" resourceMetadataURL = strings.TrimRight(resourceURL, "/") + "/.well-known/oauth-protected-resource"
} }
} }
public["/mcp"] = chassisauth.BearerMiddleware(mcpToken, jwtValidator, "brain", resourceMetadataURL, mcpSrv) mux.Handle("/mcp", chassisauth.BearerMiddleware(mcpToken, jwtValidator, "brain", resourceMetadataURL, mcpSrv))
// The REST API, authenticated as a whole (hyperguild#149). Same credential
// as /mcp: callers (brain-ui, brain-ingest-ntfy, the hyperguild CLI) send
// BRAIN_MCP_TOKEN. Registered after the public routes on purpose only for
// reading order; ServeMux precedence is by pattern specificity.
mountREST(mux, h, func(next http.Handler) http.Handler {
return chassisauth.BearerMiddleware(mcpToken, jwtValidator, "brain", resourceMetadataURL, next)
})
// POST /capture (#53/#54): the uniform capture REST door. Needs a ticket // POST /capture (#53/#54): the uniform capture REST door. Needs a ticket
// tracker to file action items, so it only mounts when Gitea is // tracker to file action items, so it only mounts when Gitea is
@@ -543,7 +549,7 @@ func main() {
sovereign := splitList(os.Getenv("BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS")) sovereign := splitList(os.Getenv("BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS"))
resolver := capturehttp.NewOriginResolver(sovereign) resolver := capturehttp.NewOriginResolver(sovereign)
captureH := capturehttp.New(captureSvc, jwtValidator, mcpToken, "local-cli", resolver) captureH := capturehttp.New(captureSvc, jwtValidator, mcpToken, "local-cli", resolver)
public["POST /capture"] = captureH mux.Handle("POST /capture", captureH)
// Same use-case behind the MCP `capture` tool (#55 relay) so MCP-native // Same use-case behind the MCP `capture` tool (#55 relay) so MCP-native
// harnesses (claude.ai, Crush, Pi, LLM Council) reach capture through // harnesses (claude.ai, Crush, Pi, LLM Council) reach capture through
// the existing /mcp OAuth connector. mcpSrv is already wrapped above; // the existing /mcp OAuth connector. mcpSrv is already wrapped above;
@@ -573,9 +579,9 @@ func main() {
logger.Error("OAUTH_CLIENT_ID/SECRET set but MCP_RESOURCE_URL is empty; cannot derive issuer") logger.Error("OAUTH_CLIENT_ID/SECRET set but MCP_RESOURCE_URL is empty; cannot derive issuer")
os.Exit(1) os.Exit(1)
} }
public["GET /.well-known/oauth-authorization-server"] = http.HandlerFunc( mux.HandleFunc("GET /.well-known/oauth-authorization-server",
oauth.MetadataHandler(issuer)) oauth.MetadataHandler(issuer))
public["POST /oauth/token"] = http.HandlerFunc(oauth.TokenHandler(oauth.TokenConfig{ mux.HandleFunc("POST /oauth/token", oauth.TokenHandler(oauth.TokenConfig{
ClientID: oauthID, ClientID: oauthID,
ClientSecret: oauthSecret, ClientSecret: oauthSecret,
AccessToken: mcpToken, AccessToken: mcpToken,
@@ -594,17 +600,7 @@ func main() {
// real request latency. /metrics itself is excluded from its own // real request latency. /metrics itself is excluded from its own
// observation by registering it on the outer mux (post-wrap). // observation by registering it on the outer mux (post-wrap).
reg := metrics.New() reg := metrics.New()
public["GET /metrics"] = http.HandlerFunc(reg.Handler()) mux.HandleFunc("GET /metrics", reg.Handler())
// The whole router, built in one place so the test of buildMux is a test
// of production wiring (PR #152 review). The REST API's credential is the
// same as /mcp's.
mux, err := buildMux(restAuth{token: mcpToken, validator: jwtValidator, realm: "brain",
resourceMetadataURL: resourceMetadataURL}, h, public)
if err != nil {
logger.Error("build router", "err", err)
os.Exit(1)
}
logger.Info("metrics endpoint registered", "path", "/metrics") logger.Info("metrics endpoint registered", "path", "/metrics")
addr := ":" + port addr := ":" + port
+6 -53
View File
@@ -1,65 +1,18 @@
package main package main
import ( import (
"errors"
"fmt"
"net/http" "net/http"
chassisauth "git.d-ma.be/mathias/mcp-chassis/auth"
"github.com/mathiasbq/hyperguild/ingestion/internal/api" "github.com/mathiasbq/hyperguild/ingestion/internal/api"
) )
// publicRoutes is the complete set of patterns allowed on the outer mux,
// outside the REST API's auth (hyperguild#149). Each authenticates itself or is
// meant to be open: /mcp (bearer/JWT), /capture (bearer/JWT), the webhook
// (HMAC), /oauth/token (client secret), OAuth metadata and /metrics (open).
// A REST pattern added here would beat the authenticated catch-all by
// specificity and ship open, which is why buildMux refuses anything else.
var publicRoutes = map[string]bool{
"/mcp": true,
"POST /capture": true,
"POST /webhooks/brain-sync": true,
"GET /.well-known/oauth-protected-resource": true,
"GET /.well-known/oauth-authorization-server": true,
"POST /oauth/token": true,
"GET /metrics": true,
}
// restAuth is the credential the REST API checks: the same as /mcp's.
type restAuth struct {
token string
validator *chassisauth.JWTValidator
realm string
resourceMetadataURL string
}
// buildMux is the server's whole router. The REST API is mounted at "/"
// behind bearer auth that buildMux constructs itself, so a caller cannot pass
// a no-op; public handlers are registered as given, and only for patterns on
// the publicRoutes allowlist. It returns an http.Handler, not the
// *ServeMux, so nothing can register a route on the result afterwards and
// bypass the allowlist.
func buildMux(auth restAuth, h *api.Handler, public map[string]http.Handler) (http.Handler, error) {
if auth.token == "" {
return nil, errors.New("buildMux: REST auth needs a token")
}
mux := http.NewServeMux()
for pattern, handler := range public {
if !publicRoutes[pattern] {
return nil, fmt.Errorf("buildMux: %q is not an allowed public route", pattern)
}
mux.Handle(pattern, handler)
}
mountREST(mux, h, func(next http.Handler) http.Handler {
return chassisauth.BearerMiddleware(auth.token, auth.validator, auth.realm, auth.resourceMetadataURL, next)
})
return mux, nil
}
// mountREST registers the REST API on its own mux and mounts it at "/" behind // mountREST registers the REST API on its own mux and mounts it at "/" behind
// auth: anything not registered as public lands here and is authenticated, // auth (hyperguild#149). Authentication is a property of the mux, not of each
// including routes added later. // route: anything not registered on the outer mux as explicitly public (/mcp,
// /capture, the HMAC webhook, OAuth metadata, /metrics) lands here and is
// authenticated, including routes added later. Before this, only /mcp was
// wrapped, and the REST routes beside it were open; infra#517 found them on
// the internet.
func mountREST(mux *http.ServeMux, h *api.Handler, auth func(http.Handler) http.Handler) { func mountREST(mux *http.ServeMux, h *api.Handler, auth func(http.Handler) http.Handler) {
rest := http.NewServeMux() rest := http.NewServeMux()
rest.HandleFunc("POST /query", h.Query) rest.HandleFunc("POST /query", h.Query)
+29 -61
View File
@@ -9,7 +9,8 @@ import (
"testing" "testing"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
chassisauth "git.d-ma.be/mathias/mcp-chassis/auth"
"github.com/mathiasbq/hyperguild/ingestion/internal/api" "github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline" "github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
@@ -17,27 +18,22 @@ import (
const routesTok = "routes-test-token" const routesTok = "routes-test-token"
func okHandler(body string) http.Handler { // hyperguild#149: only /mcp was authenticated; every REST route on the same
return http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte(body)) }) // mux was open, and infra#517 found them on the internet. The REST API is now
} // mounted at "/" behind the bearer middleware, so any route not registered
// as explicitly public is authenticated, including routes added later.
// testMux builds the router exactly as main does: buildMux with the full func testRouter(t *testing.T) http.Handler {
// public set main registers (stub handlers) and real bearer config.
// hyperguild#149: the REST API is authenticated as a whole; the PR #152
// review showed a test of a COPY of the router let main.go regress unseen.
func testMux(t *testing.T) http.Handler {
t.Helper() t.Helper()
h := api.NewHandler(t.TempDir(), slog.New(slog.NewTextHandler(os.Stderr, nil)), pipeline.Config{}) h := api.NewHandler(t.TempDir(), slog.New(slog.NewTextHandler(os.Stderr, nil)), pipeline.Config{})
public := map[string]http.Handler{} mux := http.NewServeMux()
for p := range publicRoutes { mux.HandleFunc("GET /metrics", func(w http.ResponseWriter, _ *http.Request) { _, _ = w.Write([]byte("ok")) })
public[p] = okHandler("public " + p) mountREST(mux, h, func(next http.Handler) http.Handler {
} return chassisauth.BearerMiddleware(routesTok, nil, "brain", "", next)
mux, err := buildMux(restAuth{token: routesTok, realm: "brain"}, h, public) })
require.NoError(t, err)
return mux return mux
} }
func do(t *testing.T, h http.Handler, method, path, authz string) (int, string) { func do(t *testing.T, h http.Handler, method, path, authz string) int {
t.Helper() t.Helper()
req := httptest.NewRequest(method, path, strings.NewReader(`{"query":"x"}`)) req := httptest.NewRequest(method, path, strings.NewReader(`{"query":"x"}`))
if authz != "" { if authz != "" {
@@ -45,62 +41,34 @@ func do(t *testing.T, h http.Handler, method, path, authz string) (int, string)
} }
rec := httptest.NewRecorder() rec := httptest.NewRecorder()
h.ServeHTTP(rec, req) h.ServeHTTP(rec, req)
return rec.Code, rec.Body.String() return rec.Code
} }
func TestBuildMuxRefusesRESTRoutesWithoutAValidToken(t *testing.T) { func TestRESTRoutesRefuseRequestsWithoutAToken(t *testing.T) {
r := testMux(t) r := testRouter(t)
for _, rt := range []struct{ method, path string }{ for _, rt := range []struct{ method, path string }{
{"POST", "/query"}, {"POST", "/write"}, {"POST", "/index"}, {"POST", "/ingest"}, {"POST", "/query"}, {"POST", "/write"}, {"POST", "/index"}, {"POST", "/ingest"},
{"POST", "/ingest-path"}, {"POST", "/ingest-raw"}, {"POST", "/backfill-refs"}, {"POST", "/ingest-path"}, {"POST", "/ingest-raw"}, {"POST", "/backfill-refs"},
{"GET", "/pending"}, {"POST", "/promote"}, {"POST", "/backfill-embeddings"}, {"GET", "/pass-rate"}, {"GET", "/pending"}, {"POST", "/promote"}, {"POST", "/backfill-embeddings"}, {"GET", "/pass-rate"},
} { } {
code, _ := do(t, r, rt.method, rt.path, "") assert.Equal(t, http.StatusUnauthorized, do(t, r, rt.method, rt.path, ""), "%s %s without a token", rt.method, rt.path)
assert.Equal(t, http.StatusUnauthorized, code, "%s %s without a token", rt.method, rt.path) assert.Equal(t, http.StatusUnauthorized, do(t, r, rt.method, rt.path, "Bearer wrong"), "%s %s with a wrong token", rt.method, rt.path)
code, _ = do(t, r, rt.method, rt.path, "Bearer wrong")
assert.Equal(t, http.StatusUnauthorized, code, "%s %s with a wrong token", rt.method, rt.path)
} }
} }
func TestBuildMuxServesRESTRoutesToAValidToken(t *testing.T) { func TestRESTRoutesServeAValidToken(t *testing.T) {
r := testMux(t) r := testRouter(t)
code, _ := do(t, r, "GET", "/pending", "Bearer "+routesTok) assert.NotEqual(t, http.StatusUnauthorized, do(t, r, "POST", "/query", "Bearer "+routesTok))
assert.Equal(t, http.StatusOK, code) assert.Equal(t, http.StatusOK, do(t, r, "GET", "/pending", "Bearer "+routesTok))
} }
func TestBuildMuxAuthenticatesAnUnregisteredPath(t *testing.T) { // Default-deny: a path nobody registered is still behind auth, so a route
code, _ := do(t, testMux(t), "POST", "/some-future-route", "") // added to the REST mux later cannot ship open by omission.
assert.Equal(t, http.StatusUnauthorized, code) func TestUnregisteredPathIsAuthenticatedByDefault(t *testing.T) {
assert.Equal(t, http.StatusUnauthorized, do(t, testRouter(t), "POST", "/some-future-route", ""))
} }
// buildMux registers public handlers as given, without adding auth: each // Explicitly public routes registered on the outer mux keep working.
// public route owns its own (bearer on /mcp, HMAC on the webhook, client func TestMetricsStaysOpenForPrometheus(t *testing.T) {
// secret on /oauth/token) or is meant to be open (/metrics, OAuth metadata). assert.Equal(t, http.StatusOK, do(t, testRouter(t), "GET", "/metrics", ""))
func TestBuildMuxDoesNotWrapPublicRoutes(t *testing.T) {
r := testMux(t)
for p := range publicRoutes {
method, path, ok := strings.Cut(p, " ")
if !ok {
method, path = "GET", p
}
code, body := do(t, r, method, path, "")
assert.Equal(t, http.StatusOK, code, p)
assert.Equal(t, "public "+p, body, p)
}
}
// The allowlist is the control: a REST pattern registered as public would win
// over the authenticated catch-all (more specific pattern), so it is refused.
func TestBuildMuxRefusesAPublicPatternOffTheAllowlist(t *testing.T) {
h := api.NewHandler(t.TempDir(), slog.New(slog.NewTextHandler(os.Stderr, nil)), pipeline.Config{})
for _, p := range []string{"POST /query", "GET /pending", "/", "POST /metrics"} {
_, err := buildMux(restAuth{token: routesTok, realm: "brain"}, h, map[string]http.Handler{p: okHandler("x")})
assert.Error(t, err, "public pattern %q must be refused", p)
}
}
func TestBuildMuxRefusesAnEmptyToken(t *testing.T) {
h := api.NewHandler(t.TempDir(), slog.New(slog.NewTextHandler(os.Stderr, nil)), pipeline.Config{})
_, err := buildMux(restAuth{token: "", realm: "brain"}, h, nil)
assert.Error(t, err)
} }
+6 -7
View File
@@ -2,14 +2,13 @@ package api
import ( import (
"errors" "errors"
"fmt"
"path/filepath" "path/filepath"
"strings" "strings"
) )
// ErrOutsideIngestRoot is the one refusal for every bad ingest path. It never // ErrOutsideIngestRoot means an ingest path does not resolve inside brain/raw/.
// carries the underlying error: "no such file" versus "permission denied" var ErrOutsideIngestRoot = errors.New("ingest path must be inside brain/raw/")
// would let a caller probe which server files exist (PR #152 review).
var ErrOutsideIngestRoot = errors.New("ingest path must be an existing file or directory inside brain/raw/")
// ResolveIngestPath confines a caller-supplied ingest path to brain/raw/, the // ResolveIngestPath confines a caller-supplied ingest path to brain/raw/, the
// inbox the watcher already owns (hyperguild#149). Relative paths resolve // inbox the watcher already owns (hyperguild#149). Relative paths resolve
@@ -17,18 +16,18 @@ var ErrOutsideIngestRoot = errors.New("ingest path must be an existing file or d
// a link inside raw/ that points elsewhere is refused. The path must exist. // a link inside raw/ that points elsewhere is refused. The path must exist.
func ResolveIngestPath(brainDir, p string) (string, error) { func ResolveIngestPath(brainDir, p string) (string, error) {
if strings.TrimSpace(p) == "" { if strings.TrimSpace(p) == "" {
return "", ErrOutsideIngestRoot return "", fmt.Errorf("%w: path is empty", ErrOutsideIngestRoot)
} }
root, err := filepath.EvalSymlinks(filepath.Join(brainDir, "raw")) root, err := filepath.EvalSymlinks(filepath.Join(brainDir, "raw"))
if err != nil { if err != nil {
return "", ErrOutsideIngestRoot return "", fmt.Errorf("%w: inbox unavailable: %v", ErrOutsideIngestRoot, err)
} }
if !filepath.IsAbs(p) { if !filepath.IsAbs(p) {
p = filepath.Join(root, p) p = filepath.Join(root, p)
} }
real, err := filepath.EvalSymlinks(filepath.Clean(p)) real, err := filepath.EvalSymlinks(filepath.Clean(p))
if err != nil { if err != nil {
return "", ErrOutsideIngestRoot return "", fmt.Errorf("%w: %v", ErrOutsideIngestRoot, err)
} }
rel, err := filepath.Rel(root, real) rel, err := filepath.Rel(root, real)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) { if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) {
@@ -67,22 +67,3 @@ func TestIngestPathDirectoryWalkSkipsSymlinksLeavingTheInbox(t *testing.T) {
assert.Contains(t, rec.Body.String(), "skipped") assert.Contains(t, rec.Body.String(), "skipped")
assert.NotContains(t, rec.Body.String(), "SENTINEL-149-OUTSIDE") assert.NotContains(t, rec.Body.String(), "SENTINEL-149-OUTSIDE")
} }
// PR #152 review: the raw EvalSymlinks error told an authenticated caller
// whether any server file existed ("no such file") or was unreadable
// ("permission denied"), and leaked the brain directory. Every refusal now
// reads the same.
func TestResolveIngestPathRefusalsAreIndistinguishable(t *testing.T) {
brain := t.TempDir()
require.NoError(t, os.MkdirAll(filepath.Join(brain, "raw"), 0o755))
var msgs []string
for _, p := range []string{"/etc/hostname", "/etc/definitely-not-here-149", "../x", "nope/missing.md", ""} {
_, err := api.ResolveIngestPath(brain, p)
require.Error(t, err, p)
msgs = append(msgs, err.Error())
assert.NotContains(t, err.Error(), brain, "refusal for %q leaks the brain dir", p)
}
for _, m := range msgs[1:] {
assert.Equal(t, msgs[0], m)
}
}