feat: email-triage-agent — multi-account Gmail digest + aging-based auto-archive via LLM classification #4

Open
opened 2026-06-17 10:43:29 +00:00 by mathias · 3 comments
Owner

Problem

5-6 Gmail accounts, ~100 emails/day, 30,000+ unread backlog. Manual triage hasn't happened and won't — the volume makes it impossible. Real human conversations and action-required items are buried under newsletter/retail/notification noise. No manual cleanup is planned or realistic.

Gate conditions (homelab#1, originally set for 2026-07-01) reviewed early:

  • Pain confirmed — no cleanup has happened, none planned
  • Multi-account missed action items: confirmed across accounts
  • brain-weekly-ingest gate dropped — email-triage-agent builds its own Gmail API OAuth pattern independently (brain-weekly-ingest repo is empty, not a dependency)

Design

Architecture

Go service, same pattern as tapir: Go + LiteLLM + ntfy + k3s CronJob + SOPS secrets. No dependency on brain-weekly-ingest or Gmail MCP connector (both are single-account only).

Two-dimensional classification

Every thread is classified on two axes simultaneously:

Logical filter (content-based, LLM):

  • HUMAN — a real person wrote this and expects a response
  • ACTION_REQUIRED — automated but requires your action (invoice, booking confirmation, password reset, bank alert)
  • NEWSLETTER — subscribed content, no action needed
  • NOISE — unsolicited, promotional, automated with no action needed

Aging filter (time-based, deterministic):

  • FRESH — last 7 days
  • RECENT — 7–30 days
  • STALE — 30–90 days
  • OLD — 90+ days

Auto-action matrix

FRESH RECENT STALE OLD
HUMAN Digest only Digest only Digest only Digest only
ACTION_REQUIRED Digest only Digest only Digest + archive if no reply Archive
NEWSLETTER Digest mention Archive Archive Archive
NOISE Archive Archive Archive Archive

Rules:

  • HUMAN threads are never auto-archived regardless of age — always surfaced in digest
  • NOISE is archived immediately regardless of age
  • NEWSLETTER: FRESH appears in digest as "subscribed content" section, then archived; older archived immediately
  • ACTION_REQUIRED: STALE archived only if you have no sent reply in the thread; OLD always archived

Daily digest format (ntfy → email-digest topic)

📬 Morning digest — 07:00 · 3 accounts · 14 threads need you

HUMAN (4)
• madeleine@... "weekend plans" — 2h ago [personal]
• client@... "Q3 proposal" — yesterday [work]
...

ACTION REQUIRED (3)
• Klarna "payment confirmation" — today
...

ARCHIVED (47 threads across 3 accounts)
  12 newsletters · 35 noise · 0 human

Multi-account OAuth

One OAuth token per Gmail account, stored in SOPS. Initial setup requires one browser-based consent per account (same pattern as tapir tapir auth). After that: fully unattended. Token refresh is automatic via golang.org/x/oauth2.

Scopes required per account:

  • gmail.readonly — read threads
  • gmail.modify — archive (add ARCHIVE label, remove INBOX)

Stack

  • Go service in new repo mathias/email-triage-agent
  • Template: template-go-agent (same as tapir)
  • LiteLLM routing: local model on iguana/koala for classification (low latency, no cloud cost for ~100 threads/day)
  • CronJob: daily 07:00 Stockholm time (0 7 * * * UTC+2 = 0 5 * * * UTC)
  • Secrets: SOPS, one key per account (gmail/account1-refresh-token, etc.)
  • Delivery: ntfy email-digest topic (homelab#3 Fix 2)
  • Namespace: email-triage in k3s

Phased rollout

Phase 1 — Read only (week 1): digest only, no archiving. Validate classification accuracy. Check digest every morning, note any misclassifications.
Phase 2 — Archive NOISE (week 2): enable auto-archive for NOISE class only. Lowest risk — these are definitionally unwanted.
Phase 3 — Full matrix (week 3+): enable full auto-action matrix including NEWSLETTER archiving and age-based ACTION_REQUIRED archiving.

Each phase requires a config flag change only — no code change.

Acceptance criteria

Phase 1 (MVP)

  • Repo mathias/email-triage-agent created from template-go-agent
  • Gmail API OAuth flow implemented (cmd/auth subcommand, one consent per account)
  • Multi-account token storage in SOPS verified
  • Thread fetch: last 24h per account via Gmail API
  • LLM classification: HUMAN / ACTION_REQUIRED / NEWSLETTER / NOISE per thread
  • Aging classification: FRESH / RECENT / STALE / OLD per thread
  • Digest formatted and delivered to ntfy email-digest topic at 07:00
  • Phase 1 flag: archiving disabled, digest only
  • k3s CronJob manifest in infra repo, Flux-managed
  • At least 2 Gmail accounts configured and tested end-to-end

Phase 2

  • Archive flag per class: archive_noise: true in config
  • NOISE threads archived via Gmail API (label ARCHIVE, remove INBOX)
  • Digest "archived" section shows count
  • No HUMAN or ACTION_REQUIRED threads archived in any test run

Phase 3

  • Full auto-action matrix implemented as config
  • Reply-detection for ACTION_REQUIRED STALE: check sent thread before archiving
  • Configurable per-account (some accounts may need different rules)

Risk

Phase 1: LOW — read only, no writes to Gmail
Phase 2: LOW-MEDIUM — archives only NOISE class; misclassification risk is low for clear noise
Phase 3: MEDIUM — broader archiving; reply-detection prevents false archive of awaited items

Dependencies

  • homelab#3 (ntfy Fix 2) — email-digest topic must exist before Phase 1 delivers
  • SOPS infrastructure — already live in infra repo
  • LiteLLM on koala:30401 — already live

Related

  • mathias/tapir — OAuth pattern reference (internal/auth/auth.go)
  • knowledge/ntfy-ios-app-basic-auth-only-no-bearer-token-field.md
  • homelab#3 — ntfy fix (dependency)
  • homelab#1 — original experiment brief (superseded by this issue)
## Problem 5-6 Gmail accounts, ~100 emails/day, 30,000+ unread backlog. Manual triage hasn't happened and won't — the volume makes it impossible. Real human conversations and action-required items are buried under newsletter/retail/notification noise. No manual cleanup is planned or realistic. Gate conditions (homelab#1, originally set for 2026-07-01) reviewed early: - ✅ Pain confirmed — no cleanup has happened, none planned - ✅ Multi-account missed action items: confirmed across accounts - ✅ brain-weekly-ingest gate dropped — email-triage-agent builds its own Gmail API OAuth pattern independently (brain-weekly-ingest repo is empty, not a dependency) ## Design ### Architecture Go service, same pattern as tapir: Go + LiteLLM + ntfy + k3s CronJob + SOPS secrets. No dependency on brain-weekly-ingest or Gmail MCP connector (both are single-account only). ### Two-dimensional classification Every thread is classified on two axes simultaneously: **Logical filter** (content-based, LLM): - `HUMAN` — a real person wrote this and expects a response - `ACTION_REQUIRED` — automated but requires your action (invoice, booking confirmation, password reset, bank alert) - `NEWSLETTER` — subscribed content, no action needed - `NOISE` — unsolicited, promotional, automated with no action needed **Aging filter** (time-based, deterministic): - `FRESH` — last 7 days - `RECENT` — 7–30 days - `STALE` — 30–90 days - `OLD` — 90+ days ### Auto-action matrix | | FRESH | RECENT | STALE | OLD | |---|---|---|---|---| | **HUMAN** | Digest only | Digest only | Digest only | Digest only | | **ACTION_REQUIRED** | Digest only | Digest only | Digest + archive if no reply | Archive | | **NEWSLETTER** | Digest mention | Archive | Archive | Archive | | **NOISE** | Archive | Archive | Archive | Archive | Rules: - HUMAN threads are never auto-archived regardless of age — always surfaced in digest - NOISE is archived immediately regardless of age - NEWSLETTER: FRESH appears in digest as "subscribed content" section, then archived; older archived immediately - ACTION_REQUIRED: STALE archived only if you have no sent reply in the thread; OLD always archived ### Daily digest format (ntfy → `email-digest` topic) ``` 📬 Morning digest — 07:00 · 3 accounts · 14 threads need you HUMAN (4) • madeleine@... "weekend plans" — 2h ago [personal] • client@... "Q3 proposal" — yesterday [work] ... ACTION REQUIRED (3) • Klarna "payment confirmation" — today ... ARCHIVED (47 threads across 3 accounts) 12 newsletters · 35 noise · 0 human ``` ### Multi-account OAuth One OAuth token per Gmail account, stored in SOPS. Initial setup requires one browser-based consent per account (same pattern as tapir `tapir auth`). After that: fully unattended. Token refresh is automatic via `golang.org/x/oauth2`. Scopes required per account: - `gmail.readonly` — read threads - `gmail.modify` — archive (add `ARCHIVE` label, remove `INBOX`) ### Stack - Go service in new repo `mathias/email-triage-agent` - Template: `template-go-agent` (same as tapir) - LiteLLM routing: local model on iguana/koala for classification (low latency, no cloud cost for ~100 threads/day) - CronJob: daily 07:00 Stockholm time (`0 7 * * *` UTC+2 = `0 5 * * *` UTC) - Secrets: SOPS, one key per account (`gmail/account1-refresh-token`, etc.) - Delivery: ntfy `email-digest` topic (homelab#3 Fix 2) - Namespace: `email-triage` in k3s ### Phased rollout **Phase 1 — Read only (week 1):** digest only, no archiving. Validate classification accuracy. Check digest every morning, note any misclassifications. **Phase 2 — Archive NOISE (week 2):** enable auto-archive for NOISE class only. Lowest risk — these are definitionally unwanted. **Phase 3 — Full matrix (week 3+):** enable full auto-action matrix including NEWSLETTER archiving and age-based ACTION_REQUIRED archiving. Each phase requires a config flag change only — no code change. ## Acceptance criteria ### Phase 1 (MVP) - [ ] Repo `mathias/email-triage-agent` created from `template-go-agent` - [ ] Gmail API OAuth flow implemented (`cmd/auth` subcommand, one consent per account) - [ ] Multi-account token storage in SOPS verified - [ ] Thread fetch: last 24h per account via Gmail API - [ ] LLM classification: HUMAN / ACTION_REQUIRED / NEWSLETTER / NOISE per thread - [ ] Aging classification: FRESH / RECENT / STALE / OLD per thread - [ ] Digest formatted and delivered to ntfy `email-digest` topic at 07:00 - [ ] Phase 1 flag: archiving disabled, digest only - [ ] k3s CronJob manifest in infra repo, Flux-managed - [ ] At least 2 Gmail accounts configured and tested end-to-end ### Phase 2 - [ ] Archive flag per class: `archive_noise: true` in config - [ ] NOISE threads archived via Gmail API (label `ARCHIVE`, remove `INBOX`) - [ ] Digest "archived" section shows count - [ ] No HUMAN or ACTION_REQUIRED threads archived in any test run ### Phase 3 - [ ] Full auto-action matrix implemented as config - [ ] Reply-detection for ACTION_REQUIRED STALE: check sent thread before archiving - [ ] Configurable per-account (some accounts may need different rules) ## Risk **Phase 1: LOW** — read only, no writes to Gmail **Phase 2: LOW-MEDIUM** — archives only NOISE class; misclassification risk is low for clear noise **Phase 3: MEDIUM** — broader archiving; reply-detection prevents false archive of awaited items ## Dependencies - homelab#3 (ntfy Fix 2) — `email-digest` topic must exist before Phase 1 delivers - SOPS infrastructure — already live in infra repo - LiteLLM on koala:30401 — already live ## Related - `mathias/tapir` — OAuth pattern reference (`internal/auth/auth.go`) - `knowledge/ntfy-ios-app-basic-auth-only-no-bearer-token-field.md` - homelab#3 — ntfy fix (dependency) - homelab#1 — original experiment brief (superseded by this issue)
Author
Owner

Phase 1 started — repo + risk-assessed, tested safety core

Created mathias/email-triage-agent and landed the safety-critical core first (TDD), before any Gmail/LLM code — because the dangerous failure mode is archiving a real person'"'"'s email.

Done:

  • Risk register (docs/risk-register.md) via the new regulatory-risk-assessment skill — R-DATA-01 (mis-archive a HUMAN), R-AGENT-01 (unattended archive), R-DATA-02 (PII leak), R-AUTH-01 (OAuth scope), R-API-01 (partial fetch).
  • Domain core (internal/domain), fully TDD'd:
    • ClassifyAge — deterministic FRESH/RECENT/STALE/OLD (boundary-tested).
    • Decide — the auto-action matrix, gated by rollout Phase (config flag, never code). Enforced invariants: HUMAN is never archived (any age/phase/reply), Phase 1 archives nothing, Phase 2 archives only NOISE, full Phase 3 matrix incl. reply-detection for stale action items.
  • All tests green, go vet clean.

Next (Phase 1 remaining): digest formatter (pure, TDD — addresses R-DATA-02) → Gmail OAuth (cmd/auth, needs one browser consent per account from you) → Gmail fetch (gmail.readonly) → LiteLLM classifier → ntfy email-digest delivery → k3s CronJob (Flux, 07:00 Stockholm). The OAuth consent is the one human-gated step.

The riskiest logic is locked down and reviewable now; everything remaining is I/O adapters following the tapir pattern.

## Phase 1 started — repo + risk-assessed, tested safety core Created **`mathias/email-triage-agent`** and landed the safety-critical core first (TDD), before any Gmail/LLM code — because the dangerous failure mode is archiving a real person'"'"'s email. **Done:** - **Risk register** (`docs/risk-register.md`) via the new `regulatory-risk-assessment` skill — R-DATA-01 (mis-archive a HUMAN), R-AGENT-01 (unattended archive), R-DATA-02 (PII leak), R-AUTH-01 (OAuth scope), R-API-01 (partial fetch). - **Domain core** (`internal/domain`), fully TDD'd: - `ClassifyAge` — deterministic FRESH/RECENT/STALE/OLD (boundary-tested). - `Decide` — the auto-action matrix, gated by rollout **Phase** (config flag, never code). Enforced invariants: **HUMAN is never archived (any age/phase/reply)**, **Phase 1 archives nothing**, **Phase 2 archives only NOISE**, full Phase 3 matrix incl. reply-detection for stale action items. - All tests green, `go vet` clean. **Next (Phase 1 remaining):** digest formatter (pure, TDD — addresses R-DATA-02) → Gmail OAuth (`cmd/auth`, **needs one browser consent per account from you**) → Gmail fetch (`gmail.readonly`) → LiteLLM classifier → ntfy `email-digest` delivery → k3s CronJob (Flux, 07:00 Stockholm). The OAuth consent is the one human-gated step. The riskiest logic is locked down and reviewable now; everything remaining is I/O adapters following the tapir pattern.
Author
Owner

Phase 1 pipeline COMPLETE + live-proven 🎉

The full agent is built (TDD throughout) and verified end-to-end against real systems — a real digest landed on the phone:

Done (repo mathias/email-triage-agent, 6 packages, all tests green)

  • Risk register (R-DATA-01…R-API-01) via the regulatory-risk-assessment skill
  • domain: triage matrix (HUMAN-never-archived, phase-gated) + thread model — TDD
  • adapters/imap: Gmail fetch via App Password (not OAuth) — live, 2 accounts (30 + 67 msgs/48h)
  • adapters/llm: LiteLLM classifier — live, koala/qwen36-35b-a3b classified 4/4 correctly
  • digest formatter + ntfy notifier — live delivery to phone
  • cmd/email-triage-agent: runnable multi-account binary + distroless Dockerfile

Auth decision (minimise manual work)

Dropped OAuth/Google-Console/Authentik-broker entirely → Google App Password + IMAP, secret in 1Password→ESO (same pattern as the ntfy fix). No consent screen, no 7-day token expiry, no verification. Per account: ~2 min (create App Password, drop in 1P). homelab#5 (Authentik Google broker) stays deferred and is now moot for this service.

Remaining = deploy only (no design decisions left)

  1. CI/CD (gitea Actions: test + buildah image + Flux patch) — needs REGISTRY_CREDS/INFRA_DEPLOY_KEY for the new repo
  2. k3s namespace + ESO secrets (both gmail accounts + LLM key + ntfy publish cred)
  3. ntfy ACL: an email-digest publish grant so the agent publishes with its own token (not the admin password used in the live test)
  4. CronJob 07:00 Stockholm (0 5 * * * UTC), Flux-managed

Phase 1 stays read-only (ETA_PHASE=1, archive nothing) for the validation week before Phase 2.

## Phase 1 pipeline COMPLETE + live-proven 🎉 The full agent is built (TDD throughout) and verified end-to-end against real systems — a real digest landed on the phone: ### Done (repo mathias/email-triage-agent, 6 packages, all tests green) - Risk register (R-DATA-01…R-API-01) via the regulatory-risk-assessment skill - domain: triage matrix (HUMAN-never-archived, phase-gated) + thread model — TDD - adapters/imap: Gmail fetch via **App Password** (not OAuth) — live, 2 accounts (30 + 67 msgs/48h) - adapters/llm: LiteLLM classifier — live, koala/qwen36-35b-a3b classified 4/4 correctly - digest formatter + ntfy notifier — live delivery to phone - cmd/email-triage-agent: runnable multi-account binary + distroless Dockerfile ### Auth decision (minimise manual work) Dropped OAuth/Google-Console/Authentik-broker entirely → **Google App Password + IMAP**, secret in 1Password→ESO (same pattern as the ntfy fix). No consent screen, no 7-day token expiry, no verification. Per account: ~2 min (create App Password, drop in 1P). homelab#5 (Authentik Google broker) stays deferred and is now moot for this service. ### Remaining = deploy only (no design decisions left) 1. CI/CD (gitea Actions: test + buildah image + Flux patch) — needs REGISTRY_CREDS/INFRA_DEPLOY_KEY for the new repo 2. k3s namespace + ESO secrets (both gmail accounts + LLM key + ntfy publish cred) 3. ntfy ACL: an email-digest publish grant so the agent publishes with its own token (not the admin password used in the live test) 4. CronJob 07:00 Stockholm (0 5 * * * UTC), Flux-managed Phase 1 stays **read-only** (ETA_PHASE=1, archive nothing) for the validation week before Phase 2.
Author
Owner

DEPLOYED — daily digest live (Phase 1, read-only)

email-triage-agent runs in k3s as a CronJob, verified end-to-end in-cluster:

  • CI/CD: gitea Actions (test/lint/vet → buildah → localhost:5000/email-triage-agent:latest).
  • Deploy: Flux-managed namespace + ESO secrets (gmail App Passwords + LLM key from 1P) + CronJob.
  • Schedule: 0 7 * * * Europe/Stockholm, ETA_PHASE=1 (digest only, archive nothing).
  • Delivery: anonymous publish to email-digest (mathias admin reads it).

Incident during deploy (fixed)

ESO had silently stopped resolving ntfy-mathias-ios/password (empty). On an ntfy restart the bootstrap ran change-pass with an empty password → tty prompt → set -e crash → ntfy initContainer CrashLoopBackOff → ~5 min ntfy outage. Fixed: hardened upsert_user to skip empty passwords (keep persisted auth.db value), and switched the agent to anonymous publish. ntfy recovered.

Follow-ups filed → homelab (see new issue)

  1. Root-cause why ESO/onepassword-connect resolves ntfy-mathias-ios/password empty (op reads it fine). 1P-based ntfy password rotation is broken until fixed (auth.db persists, so not urgent).
  2. Give the agent a dedicated ntfy publish token instead of anonymous email-digest.

Phase 1 runs read-only for the validation week before enabling Phase 2 (archive NOISE).

## DEPLOYED ✅ — daily digest live (Phase 1, read-only) email-triage-agent runs in k3s as a CronJob, verified end-to-end in-cluster: - **CI/CD:** gitea Actions (test/lint/vet → buildah → localhost:5000/email-triage-agent:latest). - **Deploy:** Flux-managed namespace + ESO secrets (gmail App Passwords + LLM key from 1P) + CronJob. - **Schedule:** 0 7 * * * Europe/Stockholm, ETA_PHASE=1 (digest only, archive nothing). - **Delivery:** anonymous publish to email-digest (mathias admin reads it). ### Incident during deploy (fixed) ESO had silently stopped resolving ntfy-mathias-ios/password (empty). On an ntfy restart the bootstrap ran change-pass with an empty password → tty prompt → set -e crash → ntfy initContainer CrashLoopBackOff → ~5 min ntfy outage. Fixed: hardened upsert_user to skip empty passwords (keep persisted auth.db value), and switched the agent to anonymous publish. ntfy recovered. ### Follow-ups filed → homelab (see new issue) 1. Root-cause why ESO/onepassword-connect resolves ntfy-mathias-ios/password empty (op reads it fine). 1P-based ntfy password rotation is broken until fixed (auth.db persists, so not urgent). 2. Give the agent a dedicated ntfy publish token instead of anonymous email-digest. Phase 1 runs read-only for the validation week before enabling Phase 2 (archive NOISE).
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/homelab#4