claudewatcher scrubber misses https://user:secret@host — the git credential-store shape passes every rule #142

New Issue

2026-09-30T11:24:16Z

mathias commented

2026-09-30 11:24:16 +00:00

The gap

ingestion/internal/claudewatcher/scrubber.go has a rule for Postgres URIs with embedded auth:

{Name: "postgres-uri-with-password", RE: regexp.MustCompile(`postgres(?:ql)?://[^:\s/]+:[^@\s/]+@`)},

There is no equivalent rule for the generic scheme://user:secret@host form, and that is exactly the shape a git credential store holds:

https://<user>:<token>@git.d-ma.be

Verified against all 13 DefaultRules using an invented value (https://mathias:0123…4567@git.d-ma.be, not a real credential): it matches none of them. Not authorization-header, not bearer-token, and not homelab-env-token — that last one requires a _TOKEN=/_PASSWORD= style prefix, which a bare URL does not have.

So a turn containing a git credential URL is not dropped. It is ingested.

Why it matters here specifically

~/.claude/projects is claudewatcher's input. A sweep of that tree today finds 12 occurrences of 2 distinct credential-shaped URLs for git.d-ma.be, across 4 transcript files in 3 different sessions — so this is not a single incident, it is a shape that has been passing the scrubber for a while.

The scrubber's own doc comment states the intent it is not currently meeting:

Fail-closed by design: we'd rather lose signal than ingest credentials into a public-readable brain.

It fails closed for the shapes it names. This shape it does not name, so it fails open.

Fix

Add a rule for credentials embedded in any URL userinfo, not just Postgres:

{Name: "url-userinfo-credential", RE: regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.\-]*://[^:\s/@]+:[^@\s/]{8,}@`)},

Notes on the shape:

generalises the existing Postgres rule rather than sitting beside it — consider whether postgres-uri-with-password should stay for the more specific log name, as authorization-header does before bearer-token
an {8,} floor on the secret half avoids matching http://user:@host and short non-secret userinfo
must NOT match a bare https://host:443/path — the [^:\s/@]+: + @ anchoring handles that, but it needs a test

A test with a table of shapes that must and must not scrub, including:

must scrub: https://u:<40hex>@git.d-ma.be, https://oauth2:<tok>@host, git+ssh://u:p@host
must NOT scrub: https://git.d-ma.be:443/mathias/infra.git, postgres://localhost:5432/db, https://host/path?a=b:c

Related, and the reason this was found

A test suite executing a script that configures git ran with the operator's real $HOME, so it read koala's live ~/.git-credentials and printed two live values into a session transcript (infra#466). Two credentials need rotation as a result — tracked separately, since that is a secret write.

The generalisable lesson is not "don't read the credential file". It is that the scrubber is the last line of defence for exactly this class of mistake, and this class walks straight through it. The mistake will recur; the scrubber is the control that is supposed to make it survivable.

Acceptance

a turn containing https://user:<secret>@host is dropped, with the rule named in the warn log
the must-not-scrub cases above still pass through
verified by table test in scrubber_test.go, both directions

Created via git-mcp on behalf of @mathias

## The gap `ingestion/internal/claudewatcher/scrubber.go` has a rule for Postgres URIs with embedded auth: ```go {Name: "postgres-uri-with-password", RE: regexp.MustCompile(`postgres(?:ql)?://[^:\s/]+:[^@\s/]+@`)}, ``` There is **no equivalent rule for the generic `scheme://user:secret@host` form**, and that is exactly the shape a git credential store holds: ``` https://<user>:<token>@git.d-ma.be ``` Verified against all 13 `DefaultRules` using an **invented** value (`https://mathias:0123…4567@git.d-ma.be`, not a real credential): it matches **none** of them. Not `authorization-header`, not `bearer-token`, and not `homelab-env-token` — that last one requires a `_TOKEN=`/`_PASSWORD=` style prefix, which a bare URL does not have. So a turn containing a git credential URL is **not dropped**. It is ingested. ## Why it matters here specifically `~/.claude/projects` is claudewatcher's input. A sweep of that tree today finds **12 occurrences of 2 distinct credential-shaped URLs** for `git.d-ma.be`, across **4 transcript files in 3 different sessions** — so this is not a single incident, it is a shape that has been passing the scrubber for a while. The scrubber's own doc comment states the intent it is not currently meeting: > Fail-closed by design: we'd rather lose signal than ingest credentials into a public-readable brain. It fails closed for the shapes it names. This shape it does not name, so it fails open. ## Fix Add a rule for credentials embedded in any URL userinfo, not just Postgres: ```go {Name: "url-userinfo-credential", RE: regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.\-]*://[^:\s/@]+:[^@\s/]{8,}@`)}, ``` Notes on the shape: - generalises the existing Postgres rule rather than sitting beside it — consider whether `postgres-uri-with-password` should stay for the more specific log name, as `authorization-header` does before `bearer-token` - an `{8,}` floor on the secret half avoids matching `http://user:@host` and short non-secret userinfo - must NOT match a bare `https://host:443/path` — the `[^:\s/@]+:` + `@` anchoring handles that, but it needs a test **A test with a table of shapes that must and must not scrub**, including: - must scrub: `https://u:<40hex>@git.d-ma.be`, `https://oauth2:<tok>@host`, `git+ssh://u:p@host` - must NOT scrub: `https://git.d-ma.be:443/mathias/infra.git`, `postgres://localhost:5432/db`, `https://host/path?a=b:c` ## Related, and the reason this was found A test suite executing a script that configures git ran with the operator's real `$HOME`, so it read koala's live `~/.git-credentials` and printed two live values into a session transcript (infra#466). Two credentials need rotation as a result — tracked separately, since that is a secret write. The generalisable lesson is not "don't read the credential file". It is that **the scrubber is the last line of defence for exactly this class of mistake, and this class walks straight through it.** The mistake will recur; the scrubber is the control that is supposed to make it survivable. ## Acceptance - a turn containing `https://user:<secret>@host` is dropped, with the rule named in the warn log - the must-not-scrub cases above still pass through - verified by table test in `scrubber_test.go`, both directions --- _Created via git-mcp on behalf of @mathias_

mathias referenced this issue from a commit

2026-09-30 14:08:29 +00:00

fix(claudewatcher): scrub credentials embedded in URL userinfo

mathias closed this issue

2026-09-30 14:08:29 +00:00

mathias commented

2026-09-30 14:08:40 +00:00

Fixed in 3c70b6f, tagged v0.18.4, pushed to main.

url-userinfo-credential added after postgres-uri-with-password, so a DSN still reports the more specific name (same convention as authorization-header before bearer-token):

{Name: "url-userinfo-credential", RE: regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.\-]*://[^:\s/@]+:[^@\s/]{8,}@`)},

Verified in both directions by TestScrub_URLUserinfoCredential, written red before the rule existed — 7 must-scrub cases failed, 7 must-not-scrub passed, then the rule turned the first 7 green without moving the second 7.

Must scrub (all secrets invented): git credential store with mathias: and oauth2: users, the in-context payload line, a clone URL in argv, http://, git+ssh://, redis://.

Must NOT scrub: https://git.d-ma.be:443/..., postgres://localhost:5432/brain, a colon in a query string, a plain issue URL, https://user:@host, the literal printf format string https://oauth2:%s@%s, and a bare 00:03:22 duration.

Confirmed which rule fires rather than assuming the suite going green meant the new one did it: the credential-store shape reports url-userinfo-credential, and postgres://user:s3cret@... still reports postgres-uri-with-password.

task check exit 0, go vet clean, gitleaks "no leaks found" over 1.92 MB.

Not closed by this: the two already-leaked credentials. Rotation is tracked on mathias/infra#466 and is Mathias's action, since the 12 occurrences already sitting in ~/.claude/projects predate this rule — the fix stops new ones being ingested, it does not retract old ones. Per his decision the transcripts stay intact as audit records and rotation is the containment.

Created via git-mcp on behalf of @mathias

Fixed in `3c70b6f`, tagged **v0.18.4**, pushed to main. `url-userinfo-credential` added after `postgres-uri-with-password`, so a DSN still reports the more specific name (same convention as `authorization-header` before `bearer-token`): ```go {Name: "url-userinfo-credential", RE: regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.\-]*://[^:\s/@]+:[^@\s/]{8,}@`)}, ``` **Verified in both directions** by `TestScrub_URLUserinfoCredential`, written red before the rule existed — 7 must-scrub cases failed, 7 must-not-scrub passed, then the rule turned the first 7 green without moving the second 7. Must scrub (all secrets invented): git credential store with `mathias:` and `oauth2:` users, the in-context payload line, a clone URL in argv, `http://`, `git+ssh://`, `redis://`. Must NOT scrub: `https://git.d-ma.be:443/...`, `postgres://localhost:5432/brain`, a colon in a query string, a plain issue URL, `https://user:@host`, the literal printf format string `https://oauth2:%s@%s`, and a bare `00:03:22` duration. Confirmed which rule fires rather than assuming the suite going green meant the new one did it: the credential-store shape reports `url-userinfo-credential`, and `postgres://user:s3cret@...` still reports `postgres-uri-with-password`. `task check` exit 0, `go vet` clean, gitleaks "no leaks found" over 1.92 MB. **Not closed by this:** the two already-leaked credentials. Rotation is tracked on mathias/infra#466 and is Mathias's action, since the 12 occurrences already sitting in `~/.claude/projects` predate this rule — the fix stops new ones being ingested, it does not retract old ones. Per his decision the transcripts stay intact as audit records and rotation is the containment. --- _Created via git-mcp on behalf of @mathias_

mathias referenced this issue

2026-09-30 14:26:00 +00:00

url-userinfo-credential: two residual shapes still pass — secret containing "/" and the scheme-less form #143

mathias referenced this issue

2026-09-30 14:27:01 +00:00

url-userinfo-credential: two residual shapes still pass — secret containing "/" and the scheme-less form #143

mathias referenced this issue from a commit

2026-09-30 14:47:48 +00:00

feat(monitoring): detect /opt drift against the repo, hourly

Sign in to join this conversation.