Compare commits

..
46 Commits
Author SHA1 Message Date
mathiasandClaude Sonnet 5.5 836a501576 feat(store): Store API for private uploaded transcripts (ADR-033)
Create/Start/Complete/Fail/Get/List plus the start-up sweep that fails
unfinished rows as interrupted (audio is not retained, so no resume). Every
call goes through withUser AND names user_id in its predicate, so isolation
does not rest on RLS alone and stays testable under the superuser test
connection; a non-owner gets ErrUploadNotFound, never a distinct forbidden.
List drops transcript text.

Mutation-checked: removing the user predicate from Get or from the UPDATEs
turns the cross-user tests red.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:38:25 +02:00
mathiasandClaude Sonnet 5.5 c29c0a7127 feat(domain): types for private uploaded transcripts (ADR-033)
Segment, Transcription, UploadStatus, UploadErrorKind and UploadedTranscript.
Error kinds are typed, never a provider's string. UploadedTranscript is a
distinct type from the shared public-caption Transcript on purpose.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:37:39 +02:00
mathiasandClaude Sonnet 5.5 4b557209c8 test(store): Store API for uploaded transcripts (red)
Lifecycle queued -> transcribing -> done|failed, per-row serving endpoint,
segments JSONB round trip, owner-only get/write (not-found, not forbidden),
list without transcript text, and the start-up sweep that fails unfinished
rows as `interrupted` for every user (audio is not retained, so no resume).

Red: CreateUpload and friends are undefined.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:37:30 +02:00
mathiasandClaude Sonnet 5.5 e36c4c1e0c feat(store): migration 017 -- private RLS-scoped uploaded_transcripts (ADR-033)
Separate table, not a user_id on the shared transcripts (ADR-021). ENABLE +
FORCE ROW LEVEL SECURITY with the migration-003 policy, user_id NOT NULL
ON DELETE CASCADE, typed error_kind, per-row serving endpoint, JSONB segments.
CHECK constraints: done => content, failed => error_kind.

Up and down are one commit: golang-migrate pairs them.

Mutation-checked: dropping FORCE, the policy, or the cascade each turn a test
red. FORCE is only observable structurally -- the app role is not the table
owner, so no behavioural test can see it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:36:50 +02:00
mathiasandClaude Sonnet 5.5 066ae6966a test(store): isolation tests for the private uploaded_transcripts table (red)
Drives RLS directly as the non-superuser app role: deny-all with no scope,
own-rows-only scoped reads, zero-row cross-user UPDATE/DELETE, rejected
cross-user INSERT, FORCE + NOT NULL user_id, cascade on DeleteUser. Also pins
the EXACT set of public tables with row security off, so a new table added
without RLS fails CI rather than passing it (ADR-033). The existing test only
named transcripts; #33's "the only non-RLS table" is false -- invitations,
user_identities and schema_migrations are off too.

Red: relation "uploaded_transcripts" does not exist. Set test passes today
and guards the migration that follows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:36:22 +02:00
mathiasandClaude Sonnet 5.5 38b3f1cf33 docs(adr): ADR-033 upload audio -> private Swedish transcript -> chat (#33)
Proposed, decider pending. Supersedes ADR-007's STT deferral for
user-supplied audio only; the YouTube audio-download rejection stands.
Settles #28 decision 1 as a separate RLS-scoped uploaded_transcripts table
and records the measured length ceiling, the ingress 413, and the 18k-char
chat truncation that #33 assumed did not exist. Two DECISION NEEDED items
(ffmpeg in the runtime image; berget fallback for private audio).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:35:04 +02:00
mathiasandClaude Sonnet 5.5 feb108def3 docs(research): measure the audio length ceiling before designing chunking (#33)
No transcription ceiling found up to 90 min: one 174 MB request through the
LiteLLM NodePort transcribed in 440 s. The first real failure is the public
gateway ingress rejecting bodies over ~8-10 MB with 413, not whisper or a
Metal OOM. Also records that srt arrives wrapped in a JSON envelope, that the
two backends differ in timestamp granularity, and that the existing chat path
truncates at 18k chars (~22 min of speech).

Synthetic Swedish TTS fixture only; no real transcript.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 16:33:34 +02:00
mathias 4bafd5f082 fix(web): views.templ imports the pre-rename gitea.d-ma.be host
CI / Build & Import (push) Successful in 11s
CI / Deploy via GitOps (push) Successful in 4s
CI / Lint / Test / Vet (push) Successful in 19s
internal/web/views.templ:7 still imports
  gitea.d-ma.be/mathias/tapir/internal/adapters/store
while go.mod declares
  git.d-ma.be/mathias/tapir

Left behind by the gitea rename (infra ADR-0004). The old hostname still
resolves in DNS but no longer serves — HTTPS to it fails outright — so the
import is genuinely broken, not merely outdated.

Why nothing caught it: the COMMITTED internal/web/views_templ.go is correct,
so builds and CI use a good generated file and stay green. The rot is only in
the source. It surfaces the moment anyone runs `templ generate` — which
regenerates the broken import over the good output, and then the build fails
with

  no required module provides package gitea.d-ma.be/mathias/tapir/internal/adapters/store

Found exactly that way while evaluating this repo for the estate-wide lint
config: regenerating templ as part of the check turned a green repo red, and
the cause was the source, not the check.

Source fixed; regenerating now reproduces the committed generated file
byte-for-byte, so views_templ.go is unchanged here. Verified: go build and
go vet pass.
2026-09-05 14:44:39 +02:00
mathias e3e78e042b style: apply gofumpt
CI / Build & Import (push) Successful in 10s
CI / Lint / Test / Vet (push) Successful in 21s
CI / Deploy via GitOps (push) Successful in 4s
Estate-wide consolidation pass. The canonical .golangci.yml
(infra ci-templates/golangci.yml, infra#309 pattern) enables the gofumpt
formatter; golangci-lint v2 default set enables NO formatter, which is why
this drift accumulated silently -- the same hole that let two files rot in
dma-cli until they were found by hand (dma-cli#12).

Mechanical and deterministic: `gofumpt -w .`, nothing hand-edited. Verified
per repo before commit -- gofumpt -l now reports 0, gofmt -l reports 0, and
go build + go vet both pass.

Clears this repo to adopt the canonical lint config in a later wave.
2026-09-05 12:39:31 +02:00
mathiasandClaude Opus 5 6f1e0be09b ci(deploy): refuse to patch a manifest with an empty IMAGE_TAG (infra#311)
CI / Lint / Test / Vet (push) Successful in 44s
CI / Build & Import (push) Successful in 9s
CI / Deploy via GitOps (push) Successful in 5s
On 2026-08-25 a deploy job ran with an empty IMAGE_TAG, wrote
`image: localhost:5000/gitea-mcp:` into the infra repo and pushed it.
That is invalid YAML, so Flux failed the ENTIRE apps Kustomization --
every app stopped reconciling for about an hour, including three
unrelated deploys queued behind it.

This job's existing check could not catch it. It greps for
`image: <repo>:${IMAGE_TAG}` after the sed, and with IMAGE_TAG empty that
pattern matches the corrupted line itself, so the verification passes
because of the bug rather than despite it.

Asserting the input up front is the fix: an empty tag now fails the job
before anything is written, let alone committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011PL5XcG68SvQqz3vGVbWbf
2026-08-25 17:25:24 +02:00
mathiasandClaude Opus 5 d45348ad2e ci: call the shared build-image script instead of an inline copy (infra#309)
CI / Lint / Test / Vet (push) Successful in 27s
CI / Build & Import (push) Successful in 11s
CI / Deploy via GitOps (push) Successful in 5s
The buildctl/skopeo step here was one of nineteen near-identical copies
across the estate. That duplication is why moving them off rootless
buildah (infra#241) took eleven commits and then had to be rediscovered
in both repo templates and the gitea-ci skill.

The implementation now lives in .gitea/build-image.sh, byte-identical to
mathias/infra's ci-templates/build-image.sh and held there by a drift
gate. What used to be textual differences between copies -- labels, tags,
build-args, build target -- are environment inputs.

Behaviour is unchanged: same registry, same tags, same labels, same
buildctl-against-host-buildkitd path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011PL5XcG68SvQqz3vGVbWbf
2026-08-25 11:29:55 +02:00
mathiasandClaude Opus 5 1c2ad2990f ci: build via buildctl instead of rootless buildah (infra#241)
CI / Build & Import (push) Successful in 8s
CI / Deploy via GitOps (push) Successful in 1s
CI / Lint / Test / Vet (push) Successful in 34s
Main has been CI-red since 2026-08-13. The build job dies extracting a base
image layer inside the gitea-runner pod:

  ApplyLayer ... potentially insufficient UIDs or GIDs available in user
  namespace (requested 0:42 for /etc/gshadow)

Root cause is the runner pod's securityContext, not this repo: it drops ALL
capabilities and sets allowPrivilegeEscalation:false, so newuidmap/newgidmap
can't gain CAP_SETUID/CAP_SETGID and rootless buildah falls back to a single
non-remapped uid/gid mapping. Adding those capabilities was tried live in the
infra repo and reverted as a confirmed no-op (no_new_privs blocks the
setuid-exec gain regardless of the container's capability set).

buildctl talks to koala's host buildkitd over /run/buildkit/buildkitd.sock,
which the runner pod already mounts, so the privileged layer work happens in
buildkitd on the host and the pod's hardened posture is untouched. Same
migration agentsquad (infra#233) and segment-embedder (infra#243) already made.

Drops the buildah smoke test with it: `buildah from ${REF}` would pull the
image back into the pod's buildah store and hit the very wall this migration
avoids, failing the job for a reason unrelated to the image. The deploy job's
`Verify rollout` is the stronger check — it requires the real pod to reach
Ready in-cluster, rather than the entrypoint printing its own name for 5s.
Tag builds lose their only image-level check (deploy runs on main pushes only);
accepted, since a tag is always cut from a main commit that already deployed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011PL5XcG68SvQqz3vGVbWbf
2026-08-17 12:07:51 +02:00
mathiasandClaude Opus 5 489b4bb2a1 chore(spike): recover the /bygge toolchain from tmpfs (#28)
CI / Lint / Test / Vet (push) Successful in 37s
CI / Build & Import (push) Failing after 5s
CI / Deploy via GitOps (push) Skipped
analyze_srt.py, build_page.py and transcode.yaml produced the /bygge
prototype. They were sitting in a session scratchpad on tmpfs, one reboot
from gone, while spike issues #29-#31 were written as if they had to be
built from scratch.

Committed as found, defects documented rather than fixed: max_tokens=6000
truncates anything past ~6 minutes of audio, the page title is hardcoded,
and the schema still carries fields no consumer renders.

The four-check validator is the part worth keeping — the coverage-gap and
Swedish-number-grounding checks catch errors that timestamp and citation
checks structurally cannot.

Real transcripts and analyses stay out: this repo is public and the
recordings are a named person discussing a client's project. Only the
synthetic fixture is committed, which exercises all four checks with no
model call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeAp5LTscnz2W7Ubt4eJ6m
2026-08-13 11:37:03 +02:00
mathias 21e7d6c74b fix(ci): smoke test hung 14min — buildah run doesn't die under plain timeout
CI / Lint / Test / Vet (push) Successful in 15s
CI / Build & Import (push) Successful in 20s
CI / Deploy via GitOps (push) Successful in 5s
Bare `/tapir` runs the long-running server, same as the old ctr-based smoke
test. ctr's --rm reliably force-killed it; a plain `timeout N buildah run`
does not — it only signals the wrapper, and the container process can
survive that and keep the log pipe open, hanging the whole job (observed
live: run 155, 14min before failure). Backgrounds the run and tears it down
with `buildah rm -f`, which forcibly kills regardless of wrapper state, and
captures output via a file instead of a blocking pipe.

Refs infra#132.
2026-07-26 06:17:18 +00:00
mathias 8814ba6673 ci(smoke): use buildah run instead of sudo k3s ctr for smoke test
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Failing after 13m42s
CI / Deploy via GitOps (push) Has been skipped
Same fix as cobalt-dingo — removes the sudo/host-containerd dependency so
this still works once the act_runner is containerized (infra#132).

Refs infra#132.
2026-07-26 05:57:37 +00:00
mathias 4b557a4325 ci(deploy): add Flux GitOps deploy job, mirroring cobalt-dingo's pattern
CI / Lint / Test / Vet (push) Successful in 27s
CI / Build & Import (push) Successful in 18s
CI / Deploy via GitOps (push) Successful in 5s
Fixes silent stale-deploy gap: CI built+pushed images but nothing bumped
k3s/apps/tapir/deployment.yaml, so merged features sat CI-green with zero
production effect (infra#111, infra#168). Flux native image-automation
can't scan localhost:5000 from inside k3s pods, so this patches the infra
repo directly via the existing INFRA_DEPLOY_KEY org secret (same key
cobalt-dingo and brain-gardener already use) on every push to main.

Refs infra#111.
2026-07-25 21:21:02 +00:00
mathiasandClaude Opus 4.8 72cb25111f test(store): address migrations by version, not step count (#8)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 12s
The up/down migration tests stepped a hard-coded number of Steps(-N)/Steps(+N)
down from HEAD and back. The counts assumed a specific latest migration, so
adding one shifted every count by one and unrelated tests (010/011/014) went
red with confusing off-by-one symptoms — a papercut on every new migration.

Drive the schema to an exact version with m.Migrate(version) via two helpers
(headVersion, migrateTo). Each test now steps to just below its target by
version, asserts the down effect, steps up to the target, asserts the up
effect, then restores to the captured HEAD. A migration added on top changes
HEAD but shifts no count, so no test needs editing.

Verified by adding a throwaway migration 017 on top: all four tests stayed
green with zero edits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QbdxXWxLefS5AwLN5eyze
2026-07-02 14:49:02 +02:00
mathiasandClaude Sonnet 4.6 38f222c931 chore: rename Go module path gitea.d-ma.be → git.d-ma.be
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 12s
Infra ADR-0004 renamed the Gitea host. Bulk replace across go.mod and
all .go import paths. Build and tests pass unchanged.

Closes #20

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dt6aHEDWRjkK14Voi6HnGh
2026-07-02 14:37:33 +02:00
mathias e042b6d26d chore: add .mcp.json (brain + gitea) — wire tapir for the single-harness workflow
CI / Lint / Test / Vet (push) Successful in 16s
CI / Build & Import (push) Successful in 12s
tapir had no MCP config at all — a hyperguild session here couldn't reach brain
or Gitea, breaking the report-back path before it could start. Matches the
shape now standard post-consolidation (hyperguild#75/#76, infra#178 template
refresh). Prerequisite for tapir#20 being run as the first real workflow test.
2026-07-02 07:00:53 +00:00
mathias c85a32e770 Merge pull request 'LANGUAGE.md: fix LiteLLM gateway never-say (stale endpoints, not piguard)' (#21) from language-md-litellm-fix into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s
2026-06-16 15:45:45 +00:00
mathias b246c0e688 fix(language): LiteLLM gateway never-say — stale endpoints, not piguard
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Build & Import (pull_request) Has been skipped
piguard is legitimately in the request path as the reverse proxy; only the
stale endpoint forms (piguard:4000, koala:4000) are wrong. Per brain#6 review
amendment #3 / tapir#18 follow-on.
2026-06-16 15:35:31 +00:00
mathias c7896cb3cc Merge pull request 'LANGUAGE.md pilot: hand-written vocabulary + caveman rubric (Phase 1)' (#19) from language-md-pilot into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s
2026-06-16 15:22:14 +00:00
mathiasandClaude Opus 4.8 b79fb892c8 docs(claude): point to LANGUAGE.md + caveman rubric
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Build & Import (pull_request) Has been skipped
Wires the Phase 1 vocabulary pilot into the canonical agent instructions
(tapir#18). CLAUDE.md is canonical here (no .context/ source, no generation
header).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 22:54:57 +02:00
mathias c315e3e003 feat(language): add hand-written vocabulary + caveman rubric pilot
Phase 1 of the ubiquitous-language system (tapir#18). Tapir-specific terms
verified against domain/ports code + README/VISION; homelab-core terms (brain
sink, LiteLLM gateway) derived from brain#5 glossary. Tooling-free pilot.
2026-06-12 20:52:23 +00:00
mathiasandClaude Opus 4.8 64e3368f5f feat(web): charm-reader visual refresh with light/dark theme toggle (ADR-032, #17)
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 10s
The UI read flat and boring. Reskin to one charm/TUI-inspired layout in two
palettes (CSS custom properties): a warm "reader" light theme (sketch B) and a
"cozy terminal" dark theme (sketch C).

- Palette chosen in cascade order: :root light default; an OS-preference dark
  block scoped to :root:not([data-theme]) so it applies only absent an explicit
  choice; and :root[data-theme="dark"|"light"] set by a header toggle that
  outranks the media query by specificity and persists in localStorage (guarded,
  degrades to OS default). A <head> init script applies the stored choice before
  paint, so no flash of the wrong palette.
- Charm touches via existing classes (no templ structure churn): monospace meta
  lines, accent uppercase section dividers with a trailing rule, pill buttons, a
  lifted/accent-edged expanded card.
- Error/danger shades become --err-* tokens so they follow the theme, replacing
  three per-block prefers-color-scheme dark overrides.
- Theme toggle wired into Layout and PublicLayout headers.

BDD: docs/use-cases/visual_theme.feature un-pended, mapped in scenarioCoverage.
TDD: internal/web/visual_theme_test.go (palettes, OS default, persisted toggle,
expanded-card embed). Verified light+dark on list/reader/welcome via web-shot.
Sketches kept as the design record. ui-spec.md as-built row added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 12:54:06 +02:00
mathias b01f8d5783 docs(bdd): visual_theme.feature scenarios (@pending until TDD, #17) 2026-06-12 12:29:31 +02:00
mathias 8ca767c144 docs(adr): ADR-032 visual refresh — light(B)+dark(C) charm-reader themes + sketches (issue #17) 2026-06-12 11:54:54 +02:00
mathiasandClaude Opus 4.8 f98b640531 feat(web): inline-expand summary + Q&A in the list (ADR-031, #16)
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 11s
Click a summarized card → its full summary (summaryBody) + chat dock (chatReveal)
expand in place via HTMX (GET /v/{id}/expand → expandedCard), collapse back via
GET /v/{id}/card → compact VideoCard. Same <li id>, outerHTML swap — the existing
list-fragment pattern. The card title carries href=/v/{id} as the no-JS fallback
(detail page stays for no-JS + deep links); only summarized cards expand. Reuses
summaryBody + chatReveal so the expanded card never drifts from the detail page.

BDD: inline_expand.feature un-pended + mapped. TDD: 6 handler/fragment tests
(expand/collapse fragments, chat dock, summarized-only, no-JS href, shared body).
Minimal CSS only — the TUI/charm restyle is #17.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 10:47:47 +02:00
mathias 607a8cbe8d docs(bdd): inline_expand.feature scenarios (@pending until TDD, #16) 2026-06-12 10:38:40 +02:00
mathias 54d60e53b9 docs(adr): ADR-031 inline-expand summary + Q&A in the list (issue #16) 2026-06-12 10:37:55 +02:00
mathias 9298e0c686 docs(homelab): TAPIR_METRICS_ADDR + key metric series (ADR-030, #15)
CI / Lint / Test / Vet (push) Successful in 10s
CI / Build & Import (push) Successful in 10s
2026-06-12 08:50:36 +02:00
mathiasandClaude Opus 4.8 cd461b95f8 feat(observability): instrument AI + HTTP paths, serve /metrics on a side port (ADR-030, #15)
CI / Build & Import (push) Successful in 11s
CI / Lint / Test / Vet (push) Successful in 10s
Wire the metrics package into the live paths and serve it:
- summarizer: per-endpoint latency by model/outcome(success|error|parse_error)/fallback + slog.
- youtube.FetchTranscript: latency by outcome (captions|none|rate_limited) + slog.
- chat: answer latency by model + slog.
- llm usage hook → token counts (prompt|completion) per model, wired in buildSummarizer/buildChat.
- oidc callback: login counter.
- cmdServe: wrap Router in metrics.HTTPMiddleware (request count + latency by bounded
  route pattern) and serve /metrics on TAPIR_METRICS_ADDR (default :9090), a SEPARATE
  port — never on the public app mux.

BDD: observability.feature scenarios un-pended + mapped. TDD: summarizer wiring tested
black-box via the /metrics scrape; metrics-not-on-public-mux asserted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 08:49:51 +02:00
mathiasandClaude Opus 4.8 9c2a04406b feat(llm): usage hook to surface token counts (ADR-030, #15)
WithUsageHook callback fires with model + prompt/completion tokens parsed from the
response usage block. Keeps the copied stdlib-only llm package decoupled from
metrics (ADR-004) — the caller wires it to internal/metrics. TDD covered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 08:36:02 +02:00
mathiasandClaude Opus 4.8 a5a8cf6f6d feat(metrics): Prometheus collectors + typed API + HTTP middleware (ADR-030, #15)
New internal/metrics package: AI metrics (summarize/caption/chat latency, LLM
tokens), session metrics (requests+latency by bounded route pattern, logins), and
a /metrics Handler. Adapters call a typed API; never touch prometheus types.
Dep: github.com/prometheus/client_golang (standard Go client; cluster runs
prometheus-operator). TDD: collectors + middleware route-pattern cardinality
covered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 08:32:41 +02:00
mathias 64d11af9ef docs(bdd): observability.feature scenarios (@pending until TDD, #15) 2026-06-12 08:28:46 +02:00
mathias b590d2708d docs(adr): ADR-030 observability — slog + Prometheus metrics (issue #15) 2026-06-12 08:27:56 +02:00
mathiasandClaude Opus 4.8 7314895ec4 fix(auth): stateless session cookie — stop logging users out on deploy (ADR-029)
CI / Lint / Test / Vet (push) Successful in 10s
CI / Build & Import (push) Successful in 11s
Pilot feedback: lots of re-logging-in on iPhone. Three causes: sessions lived in
an in-memory map (wiped on every pod restart/deploy), a 1h TTL (idle >1h forced
re-login on a check-back-tomorrow reader), and a session cookie with no Max-Age
(dropped on Safari close). Each re-login is the full IdP redirect dance.

Make sessions stateless: identity + absolute expiry live inside the existing
HMAC-signed cookie (no server table), TTL 1h → 30 days sliding, cookie now
persistent (Max-Age). Survives restarts (test: a cookie from one instance is
accepted by a fresh instance with the same secret), browser-close, and idle.
Trade: no server-side revocation — logout clears the cookie client-side; rotating
tapir-session-secret is the global logout lever. Accepted for the Stage-0 reader.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 23:07:13 +02:00
mathiasandClaude Opus 4.8 36dd182fb5 feat(report): Stage-0 usage gate counts from a baseline date (default 2026-06-11)
CI / Build & Import (push) Successful in 11s
CI / Lint / Test / Vet (push) Successful in 10s
The return-usage gate (ADR-016) counted distinct active weeks over ALL history,
so pre-launch noise — testing churn and the period the pilot sat blocked on zero
summaries — would inflate the signal. Add a baseline: ActiveWeeks(ctx, since)
filters login_events + summary_actions to seen_at/acted_at >= since. The report
command sets it to TAPIR_USAGE_GATE_START (YYYY-MM-DD, default 2026-06-11 — the
morning the pilot was unblocked) and prints the baseline. The gate now measures
whether users RETURN once it genuinely works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 22:52:15 +02:00
mathias 40808f2d4b docs(onboard): BDD scenarios + architecture for the ADR-028 burst
CI / Lint / Test / Vet (push) Successful in 10s
CI / Build & Import (push) Successful in 10s
- connect_account.feature: refine the burst scenario to "best recent" (likely-good
  selection, skips too-short/too-long) and add a scenario for the stronger model.
- scenario_coverage: remap to TestOnboardBurstVideoIDs + add the model scenario
  (the old NewestUnsummarizedVideoIDs test was removed).
- architecture.md: new "Connect-time onboarding burst" subsection — junk-avoiding
  selection over persisted duration, the burst-only stronger-model chain, and why
  has-captions/cached-first are not selection signals.
2026-06-11 19:54:29 +02:00
mathias 9232f49555 feat(onboard): burst picks likely-good videos, summarizes with stronger model (ADR-028)
Wire the onboarding burst to its quality-aware selection and a stronger model:
- main.go onboard uses OnboardBurstVideoIDs (junk-avoiding) instead of pure
  newest-first, bounded by MinVideoSeconds / OnboardMaxVideoSeconds.
- buildBurstProcessor builds a burst-only summarizer chain led by the onboard
  model (burstChainModels: onboard -> standard ADR-022 chain, deduped, NDA lever
  intact), over the same store/cache/sink. Collapses onto the shared Processor
  when the onboard model is empty/equal-to-primary or config is incomplete.
- summarizerEndpoint + newYouTubeSource extracted so standard and burst wiring
  share one definition.
- Remove now-superseded NewestUnsummarizedVideoIDs: OnboardBurstVideoIDs(.,0,0)
  is identical pure-newest behaviour and its test covers RLS + ordering.
2026-06-11 19:54:29 +02:00
mathias 582c1a2065 feat(store): OnboardBurstVideoIDs — junk-avoiding burst selection (ADR-028)
Newest-first but quality-aware: excludes a video when its duration is KNOWN and
outside [minSeconds, maxSeconds], dropping Shorts and multi-hour livestream VODs
that waste a scarce caption fetch on a poor first impression. NULL/unknown
duration is kept (degrade-open) but ranked after known-good rows. 0/0 bounds
disable the filter (pure newest-first, the reversibility lever). RLS-scoped.
NewestUnsummarizedVideoIDs is left in place for callers that want pure-newest.
2026-06-11 19:54:29 +02:00
mathias 9f0d8cf198 feat(discovery): persist video duration_s instead of discarding it (ADR-028)
ADR-023's filterLowValue already fetches each candidate's duration via the
cheap videos.list quota call to drop Shorts/live, then threw it away — the
videos.duration_s column (migration 001) was never written. Carry it onto the
kept domain.Video and have UpsertVideo persist it, COALESCE-preserving a known
value so an unknown (0) re-upsert never clobbers it (the channel_title backfill
stance, migration 014). This is the enabling change for length-aware burst
selection. No new migration — the column already exists.
2026-06-11 19:54:29 +02:00
mathias d21077303d feat(config): add OnboardSummarizerModel + OnboardMaxVideoSeconds (ADR-028)
Two knobs for the onboarding-burst quality work:
- TAPIR_ONBOARD_SUMMARIZER_MODEL (default iguana/gemma4-26b): the stronger model
  the burst leads its chain with; empty collapses the burst onto the shared
  processor (lookupOr, so explicit-empty disables).
- TAPIR_ONBOARD_MAX_VIDEO_SECONDS (default 14400/4h): upper duration bound for
  burst picks; 0 disables, negative clamps to 0.

Table-driven tests cover defaults, explicit, disable, and invalid input.
2026-06-11 19:54:29 +02:00
mathias 9b2ee2e765 docs: spec onboarding-wow-burst + ADR-028 (better picks, stronger model)
Phase-1 live-DB investigation found the connect-time burst (ADR-018) fires
but delivers a weak first impression: pure newest-first selection picks junk
(a livestream + regional news for pilot user Jonte), and all burst summaries
run on the weak koala/phi4-mini instead of the validated iguana/gemma4-26b.
Cached-first was investigated and rejected — ~3% cross-user overlap, 0
cached-and-unsummarized, newest-20 all uncached (newest-first and cached-first
are structurally incompatible).

ADR-028 records: persist duration_s at discovery (ADR-023 already fetches it,
just stops discarding), junk-avoiding burst selection (drop known too-short/
too-long), and a stronger burst-only summarizer chain. Not a throughput change.
2026-06-11 19:54:29 +02:00
mathias b8fbc5a805 docs: spec first-session wow — verify & improve onboarding summary burst (investigate-first)
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 10s
The concern is new-user first-contact: see some GOOD summaries fast or they
don't return. Reframed as curation/latency for ~3 videos, NOT a 429/throughput
problem (3 fetches is nowhere near the wall). Phase 1 (report-and-stop) verifies
whether the existing cap-3 onboarding burst even fires today, what it delivers,
and — critically — how much transcript-cache overlap exists between users (drives
the blend). Phase 2 levers: cached-transcript-first (instant, zero-fetch),
likely-good selection (has-captions/good-length, not just newest), and optionally
the stronger model for the burst's few summaries. Blend deferred to the
maintainer post-Phase-1. Explicitly NOT bulk-fetch, NOT credentials (ADR-010/026
dead end), NOT a client extension.
2026-06-11 15:36:09 +00:00
mathiasandClaude Opus 4.8 19ca4282a8 refactor(web): dock chat inline below the summary, integrated view (ADR-027)
CI / Lint / Test / Vet (push) Successful in 10s
CI / Build & Import (push) Successful in 10s
The chat was a separate page — opening it left the summary behind, the very
context you're asking about. Now the chat docks open IN PLACE below the summary:
"Dig deeper" reveals the chat section (HTMX, outerHTML over the closed dock) so
the summary stays on screen above it; no navigation. The no-JS fallback renders
the full summary AND the open chat on one page (the same integrated view), so
progressive enhancement holds.

Extracts a shared summaryBody templ so the detail page and the chat page render
one identical summary, not two divergent ones. GET /v/{id}/chat returns just the
open chat section as a fragment for the inline reveal, or the full summary+chat
page for a no-JS navigation; POST swaps the panel inline or re-renders the whole
page. Tests assert summary+chat coexist on the page and the reveal is a fragment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 14:02:39 +02:00
107 changed files with 5967 additions and 1239 deletions
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
# Build an OCI image with buildctl and push it to the local registry.
#
# CANONICAL COPY: mathias/infra ci-templates/build-image.sh (infra#309).
# Every repo carries a byte-identical copy at .gitea/build-image.sh, written by
# `task ci:sync` and held in place by the drift gate. DO NOT EDIT THE COPIES --
# edit here, re-sync, and the gate will tell you which repos still disagree.
#
# Why this exists: this step was hand-copied into a dozen repos, so migrating it
# off rootless buildah (infra#241) meant eleven near-identical commits, eleven
# verifications, and three separate rediscoveries of the same fix in the two
# repo templates and the gitea-ci skill. The differences between the copies were
# incidental -- a label here, a build-arg there -- never intentional. Those are
# inputs now, not forks of the implementation.
#
# Why buildctl and not buildah: the gitea-runner pod runs with
# `capabilities: drop [ALL]` and `allowPrivilegeEscalation: false`, which
# unconditionally blocks the setuid-root exec that rootless buildah's uid/gid
# remapping needs, and the pod has no OCI runtime at all. buildctl is a thin
# client -- it hands the build to the host's buildkitd over a socket, so the
# privileged layer work happens on the host and this pod's securityContext is
# not in the picture. buildah is absent from the runner image as of baked-v14
# and its absence is asserted at pod start (infra#308).
#
# Inputs, all via environment:
#
# IMAGE (required) image name, e.g. cobalt-dingo
# REGISTRY default localhost:5000
# SHA_TAG default `git rev-parse --short HEAD`
# VERSION_TAG optional extra tag, e.g. v1.2.3 on a tag build
# PUSH_LATEST default 1; set 0 to skip the :latest tag
# CONTEXT default . -- build context directory
# DOCKERFILE_DIR default $CONTEXT -- directory holding the Dockerfile
# DOCKERFILE_NAME optional, when the file is not named `Dockerfile`
# TARGET optional, multi-stage build target (--opt target=)
# BUILD_ARGS optional, space-separated K=V pairs
# BUILD_SECRETS optional, space-separated buildkit secret specs, each
# `id=NAME,src=PATH`. Pass the SPEC only -- the caller
# writes the secret to PATH itself. A secret VALUE must
# never be passed here: it would land in this script's
# argv and therefore in the CI log.
# EXTRA_LABELS optional, space-separated K=V pairs
# GIT_SHA optional, stamped as org.opencontainers.image.revision
# IMAGE_SOURCE optional, stamped as org.opencontainers.image.source
#
# Note on shell style below: every conditional is a full `if` block, never
# `[ -n "$X" ] && cmd`. Under `set -e` the && form exits the script whenever the
# test is false, which is a real bug this estate has already shipped once.
set -euo pipefail
if [ -z "${IMAGE:-}" ]; then
echo "build-image: IMAGE is required" >&2
exit 2
fi
REGISTRY="${REGISTRY:-localhost:5000}"
BUILDKIT_HOST="${BUILDKIT_HOST:-unix:///run/buildkit/buildkitd.sock}"
CONTEXT="${CONTEXT:-.}"
DOCKERFILE_DIR="${DOCKERFILE_DIR:-$CONTEXT}"
PUSH_LATEST="${PUSH_LATEST:-1}"
VERSION_TAG="${VERSION_TAG:-}"
if [ -z "${SHA_TAG:-}" ]; then
SHA_TAG="$(git rev-parse --short HEAD)"
fi
REF="${REGISTRY}/${IMAGE}:${SHA_TAG}"
TAR_FILE="$(mktemp)"
trap 'rm -f "$TAR_FILE"' EXIT
args=(
--frontend dockerfile.v0
--local "context=${CONTEXT}"
--local "dockerfile=${DOCKERFILE_DIR}"
)
if [ -n "${DOCKERFILE_NAME:-}" ]; then
args+=(--opt "filename=${DOCKERFILE_NAME}")
fi
if [ -n "${TARGET:-}" ]; then
args+=(--opt "target=${TARGET}")
fi
for spec in ${BUILD_SECRETS:-}; do
# Opaque id=NAME,src=PATH -- buildkit reads the file, nothing is echoed.
args+=(--secret "${spec}")
done
if [ -n "${GIT_SHA:-}" ]; then
args+=(--opt "label:org.opencontainers.image.revision=${GIT_SHA}")
fi
if [ -n "${IMAGE_SOURCE:-}" ]; then
args+=(--opt "label:org.opencontainers.image.source=${IMAGE_SOURCE}")
fi
for kv in ${BUILD_ARGS:-}; do
args+=(--opt "build-arg:${kv}")
done
for kv in ${EXTRA_LABELS:-}; do
args+=(--opt "label:${kv}")
done
echo "build-image: building ${REF}"
buildctl --addr "$BUILDKIT_HOST" build "${args[@]}" --output "type=oci,dest=${TAR_FILE}"
# buildctl has no registry-push equivalent to `buildah push`, so skopeo copies
# the OCI archive up -- once per tag, from the same archive, so every tag points
# at bit-identical content.
push_tag() {
echo "build-image: pushing ${1}"
skopeo copy --dest-tls-verify=false "oci-archive:${TAR_FILE}" "docker://${1}"
}
push_tag "$REF"
if [ "$PUSH_LATEST" = "1" ]; then
push_tag "${REGISTRY}/${IMAGE}:latest"
fi
if [ -n "$VERSION_TAG" ]; then
push_tag "${REGISTRY}/${IMAGE}:${VERSION_TAG}"
fi
echo "✓ build-image: ${REF} pushed"
+157 -27
View File
@@ -61,33 +61,163 @@ jobs:
echo "version-tag=${REF#refs/tags/}" >> "$GITHUB_OUTPUT"
fi
# infra#241: rootless buildah inside the gitea-runner pod can't remap the
# UID/GID range newuidmap/newgidmap need (the pod drops ALL capabilities
# and sets allowPrivilegeEscalation:false — both deliberate, it also holds
# CI secrets), so extracting a base-image layer that chowns outside the
# single available mapping fails with `ApplyLayer ... requested 0:42 for
# /etc/gshadow`. Main has been red on exactly this since 2026-08-13.
# buildctl instead talks to koala's host buildkitd over the socket the
# runner pod already mounts: the privileged extraction happens inside
# buildkitd on the host, never in this pod. Same migration agentsquad
# (infra#233) and segment-embedder (infra#243) already made — zero
# security tradeoff, unlike loosening the pod's capabilities.
#
# buildctl has no registry-push flag of its own: it writes an OCI archive
# and skopeo pushes that, which is also why every tag below is one copy of
# a single build rather than a rebuild.
- name: Build and push to local registry
run: |
REGISTRY="localhost:5000"
REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
buildah build \
--label "org.opencontainers.image.revision=${{ github.sha }}" \
--label "org.opencontainers.image.source=${{ github.repositoryUrl }}" \
-t ${REF} \
-t ${REGISTRY}/${{ env.IMAGE }}:latest \
.
buildah push --tls-verify=false ${REF}
buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
[[ -n "${{ steps.meta.outputs.version-tag }}" ]] && \
buildah push --tls-verify=false \
${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.version-tag }} || true
echo "Image pushed to ${REF}"
# infra#309: implementation lives in .gitea/build-image.sh, a
# byte-identical copy of mathias/infra's ci-templates/build-image.sh.
# Do not edit it here -- edit the canonical file, run `task ci:sync`,
# and infra's drift gate names any repo still carrying an old copy.
env:
SHA_TAG: ${{ steps.meta.outputs.sha-tag }}
VERSION_TAG: ${{ steps.meta.outputs.version-tag }}
GIT_SHA: ${{ github.sha }}
IMAGE_SOURCE: ${{ github.repositoryUrl }}
run: bash .gitea/build-image.sh
- name: Smoke test
run: |
REGISTRY="localhost:5000"
REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
CNAME="smoke-${{ steps.meta.outputs.sha-tag }}"
sudo k3s ctr images pull --plain-http ${REF}
OUTPUT=$(timeout 5 sudo k3s ctr run --rm ${REF} ${CNAME} /tapir 2>&1 || true)
sudo k3s ctr containers delete ${CNAME} 2>/dev/null || true
echo "$OUTPUT" | grep -q "tapir" \
&& echo "Smoke test passed" \
|| echo "Smoke test inconclusive: $OUTPUT"
# The buildah-based smoke test that used to sit here is gone with the
# buildctl migration above: `buildah from ${REF}` would pull the image
# back into the pod's buildah store and hit the very uid/gid wall the
# migration exists to avoid — it would fail the job for a reason that has
# nothing to do with the image being good. The `Verify rollout` step in
# the deploy job below is the stronger check anyway: it requires the real
# pod to reach Ready in-cluster, not just the entrypoint to print its own
# name for 5s. Tag builds lose their only image-level check (deploy is
# main-push only) — accepted; a tag is always built from a main commit
# that already deployed.
# ── 3. Mirror to GitHub — skipped for now (SSH key rotation pending) ─
# ── 3. Deploy via infra repo + Flux ─────────────────────────────────────────
# Flux native image-automation can't scan localhost:5000 from inside k3s pods
# (mathias/infra k3s/flux/flux-system/image-automation.yaml) — this job
# mirrors cobalt-dingo's proven pattern instead: patch the infra repo's
# manifest directly on every push to main, then annotate Flux for a fast
# reconcile. Fixes infra#111 (image built+pushed but manifest bump was
# manual, so merged features silently didn't deploy).
deploy:
name: Deploy via GitOps
needs: build
runs-on: self-hosted
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
environment: staging
steps:
- name: Update image tag in infra repo
env:
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
DEPLOY_KEY: ${{ secrets.INFRA_DEPLOY_KEY }}
run: |
set -euo pipefail
# infra#311: fail on an empty tag BEFORE patching anything. The
# grep verification further down CANNOT catch it -- with IMAGE_TAG
# empty its pattern matches the very corruption it exists to reject.
# The result is `image: <repo>:` in the manifest, which is invalid
# YAML, so Flux fails the ENTIRE apps Kustomization and every app
# stops reconciling, not just this one. That happened on 2026-08-25.
[ -n "${IMAGE_TAG:-}" ] || { echo "IMAGE_TAG is empty - refusing to patch"; exit 1; }
# INFRA_DEPLOY_KEY is a Gitea org secret (mathias org), already
# configured per docs/cd-pipeline.md in the infra repo — same key
# cobalt-dingo and brain-gardener use, no new secret needed.
mkdir -p ~/.ssh
echo "$DEPLOY_KEY" > ~/.ssh/id_infra
chmod 600 ~/.ssh/id_infra
ssh-keyscan -p 30022 10.0.1.20 >> ~/.ssh/known_hosts 2>/dev/null
export GIT_SSH_COMMAND="ssh -i ~/.ssh/id_infra -o IdentitiesOnly=yes"
rm -rf /tmp/infra
git clone -b main ssh://git@10.0.1.20:30022/mathias/infra.git /tmp/infra
cd /tmp/infra
DEPLOYMENT="k3s/apps/tapir/deployment.yaml"
# In-place update of the image tag. sed (not yq) so we don't
# depend on additional tooling on the runner — same as cobalt-dingo.
sed -i "s|image: localhost:5000/tapir:.*|image: localhost:5000/tapir:${IMAGE_TAG}|" "$DEPLOYMENT"
# Verify the patch took effect.
grep -q "localhost:5000/tapir:${IMAGE_TAG}" "$DEPLOYMENT" \
|| { echo "✗ image tag patch failed"; exit 1; }
if git diff --quiet "$DEPLOYMENT"; then
echo "ℹ image tag unchanged — skipping push"
else
git -c user.name="tapir CI" \
-c user.email="ci@tapir.local" \
commit -m "chore(deploy): tapir → ${IMAGE_TAG}" "$DEPLOYMENT"
git push origin main
echo "✓ pushed to infra repo"
fi
shred -u ~/.ssh/id_infra
- name: Trigger Flux reconcile (immediate)
run: |
# Without these annotations, Flux would still pick up the change
# within 30s (the apps Kustomization interval). The annotations
# cut latency to ~1s.
kubectl -n flux-system annotate gitrepository flux-system \
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
kubectl -n flux-system annotate kustomization apps \
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
- name: Wait for Flux to apply new image
env:
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
run: |
# Poll the Deployment spec until it reflects the new tag.
# Bound to 60s so a stuck Flux doesn't hang CI.
EXPECTED="localhost:5000/tapir:${IMAGE_TAG}"
for i in $(seq 1 60); do
CURRENT=$(kubectl get deploy tapir -n tapir \
-o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null || echo "")
if [ "$CURRENT" = "$EXPECTED" ]; then
echo "✓ Flux applied new image after ${i}s"
break
fi
sleep 1
done
# Final assertion (in case the loop exited without matching).
kubectl get deploy tapir -n tapir \
-o jsonpath='{.spec.template.spec.containers[0].image}' \
| grep -qx "$EXPECTED" \
|| { echo "✗ Flux did not apply new image within 60s"; exit 1; }
- name: Verify rollout
run: |
kubectl rollout status deployment/tapir \
--namespace tapir \
--timeout=120s \
|| {
echo "── pod status ──"
kubectl get pods -n tapir -o wide
echo "── events ──"
kubectl get events -n tapir --sort-by='.lastTimestamp' | tail -20
echo "── describe ──"
kubectl describe pods -n tapir -l app=tapir | tail -40
exit 1
}
- name: Confirm pod running new image
env:
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
run: |
kubectl get pods -n tapir \
-l app=tapir \
--field-selector=status.phase=Running \
-o jsonpath='{.items[*].spec.containers[0].image}' \
| grep -q "localhost:5000/tapir:${IMAGE_TAG}" \
&& echo "✓ pod running new image" \
|| { echo "✗ pod image mismatch"; exit 1; }
# ── 4. Mirror to GitHub — skipped for now (SSH key rotation pending) ─
+11
View File
@@ -11,3 +11,14 @@
.env.*
!.env.example
*.local
# Spike media (#28): real recordings, their transcripts and derived analyses are
# private third-party content and this repo is public. Only synthetic fixtures
# are committed — see scripts/spike-media/README.md.
/scripts/spike-media/*.mov
/scripts/spike-media/*.mp4
/scripts/spike-media/*.wav
/scripts/spike-media/*.srt
/scripts/spike-media/*.json
/scripts/spike-media/*.html
!/scripts/spike-media/fixtures/
+18
View File
@@ -0,0 +1,18 @@
{
"mcpServers": {
"brain": {
"type": "http",
"url": "https://brain-mcp.d-ma.be/mcp",
"headers": {
"Authorization": "Bearer ${BRAIN_MCP_TOKEN}"
}
},
"gitea": {
"type": "http",
"url": "https://git-mcp.d-ma.be/mcp",
"headers": {
"Authorization": "Bearer ${GITEA_MCP_TOKEN}"
}
}
}
}
+1
View File
@@ -12,6 +12,7 @@ docs it indexes.
3. `DECISIONS.md` — the ADRs. Decisions are settled here; do not re-litigate without a new ADR.
4. `docs/architecture/architecture.md`, `docs/data-model.md`, `docs/use-cases/*.feature`.
5. `docs/homelab-integration.md` — the concrete endpoints/conventions you'll need.
6. `LANGUAGE.md` — the project vocabulary. Apply the caveman rubric before destructive operations.
## How to work in this repo
+446 -1
View File
@@ -123,7 +123,9 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path.
## ADR-007 — Captions-first; audio-download + speech-to-text deferred
**Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API
`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand.
`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand
**for YouTube**. STT deferral superseded **for user-supplied audio only** by ADR-033; the
YouTube audio-download rejection is unchanged.
**Context.** YouTube's Data API does not expose transcripts. Options: official captions
(clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey,
@@ -1060,6 +1062,445 @@ no migration, no stored state to unwind. Spec: `docs/specs/chat-with-transcript.
---
## ADR-028 — Onboarding burst: pick likely-good videos, summarize them with a stronger model
**Status:** Accepted (2026-06-11). **Refines ADR-018** (the connect-time burst) and **ADR-020**
(recency-bounded auto-summarize). **Builds on ADR-022** (the endpoint chain), **ADR-023**
(the discovery-time `videos.list` enrichment), and **ADR-021** (the shared transcript cache).
Triggered by a Phase-1 investigation of the live pilot DB.
**Context.** A new user's first session decides whether they return (the Stage-0 gate, ADR-016).
The connect-time burst (ADR-018: summarize ≤`TAPIR_ONBOARD_SUMMARIZE_COUNT` newest videos so the
feed isn't empty) *fires* in production, but a live-DB investigation of the second pilot user
("Jonte") found it delivers a weak first impression for two reasons, and ruled out a third idea:
1. **Junk picks.** Selection was pure newest-first (`NewestUnsummarizedVideoIDs`,
`ORDER BY published_at DESC`) with **no quality signal**. Jonte's live burst-3 were a
stock-ticker **livestream** + two regional news clips — newest, not best. The cheap signals
that *could* gate this (duration, live status) are fetched by ADR-023's `videos.list`
enrichment at discovery and then **thrown away**: the `videos.duration_s` column (migration
001) was never written.
2. **Weakest model on the first impression.** All burst summaries ran on `koala/phi4-mini` — the
documented weak link (ADR-022 was born from its failures). The stronger, brain-validated
`iguana/gemma4-26b` was never used, even though the burst is only ~3 summaries.
3. **Cached-first instant summaries — REJECTED.** The idea: skip the fetch, summarize
already-cached transcripts (ADR-021) instantly. The pilot numbers kill it — only **11 videos**
overlap between the two users (~3% of each ~350–400-video library), **0** cached-and-
unsummarized, and a new user's newest-20 unsummarized are **20/20 NOT cached**. Newest-first
and cached-first are structurally incompatible: fresh uploads are exactly what nobody has
fetched. An empty lever at pilot scale.
**Decision.**
1. **Persist `duration_s` at discovery.** `filterLowValue` (ADR-023) already has each candidate's
duration in hand; carry it onto the kept `domain.Video` and have `UpsertVideo` write it,
COALESCE-preserving a known value (the channel-title backfill stance, migration 014). No new
migration — the column exists. The connect-triggered discovery pass runs *before* the burst,
so a fresh user's candidates are enriched in time.
2. **Junk-avoiding selection.** A new `OnboardBurstVideoIDs(userID, limit, minSeconds, maxSeconds)`
keeps the newest-first order but drops a video when its duration is *known* and outside
`[minSeconds, maxSeconds]` — `minSeconds` = `TAPIR_MIN_VIDEO_SECONDS` (60, the Shorts floor),
`maxSeconds` = new `TAPIR_ONBOARD_MAX_VIDEO_SECONDS` (default 14400 = 4h, to drop multi-hour
livestream VODs that pass the live filter once ended). A NULL duration is **unknown** — kept
(degrade-open) but ranked after known-good rows. **has-captions stays un-gateable pre-fetch**
(only knowable after a gate fetch or a ~0-probability cache hit); selection only *avoids
known-junk*, it does not *promise* captions.
3. **Stronger model for the burst only.** `TAPIR_ONBOARD_SUMMARIZER_MODEL` (default
`iguana/gemma4-26b`) leads a burst-specific summarizer chain (onboard model first, then the
standard ADR-022 chain as resilience, deduped), wrapped in a burst-specific processor over the
*same* store/cache/sink — a pure wiring choice; the engine and ports are unchanged (ADR-003).
Empty or equal-to-primary collapses the burst back onto the shared processor.
**Not a throughput change.** The caption rate gate (ADR-014) and the foreground priority lane
(ADR-026) are untouched — same pacing, same cap. This changes *which* ≤3 videos the burst spends
its fetches on and *which model* summarizes them, never how fast or how many. The engine's
existing read-stored-first (ADR-021) is unchanged and still yields a free instant summary on the
rare cache hit — we simply do not *select* for cache hits.
**Consequences.** Better odds of a strong first session: the burst avoids the obvious junk and
runs the better model on the one impression that decides return. The selection improvement is
forward-looking — existing rows have NULL `duration_s` until their next discovery pass backfills
it (lazy, like channel_title); a brand-new user benefits immediately because connect-discovery
runs first. `duration_s` becoming live also unblocks future length-aware features (feed sorting,
"long read" badges) for free.
**Reversibility.** Pure config + wiring + one column write + one query, no migration.
`TAPIR_ONBOARD_MAX_VIDEO_SECONDS=0` (and `TAPIR_MIN_VIDEO_SECONDS=0`) restores pure newest-first;
`TAPIR_ONBOARD_SUMMARIZER_MODEL=""` collapses the burst back to the shared processor.
Spec: `docs/specs/onboarding-wow-burst.md`.
---
## ADR-029 — Stateless session cookie (survives restarts, browser-close, idle)
**Status:** Accepted (2026-06-11). Triggered by pilot feedback: "lots of clicking to log in again
on iPhone." Supersedes the in-memory session store in the ADR-011 login.
**Context.** Three compounding causes made users re-login constantly:
1. **In-memory session store** (`sessionStore` map) — wiped on every pod restart, so each deploy
logged everyone out. During the active build period that was ~15 logouts.
2. **1-hour session TTL** — for a "check back tomorrow" reader, idle > 1h forced a re-login on
nearly every visit.
3. **No cookie Max-Age** — a session cookie (deleted on browser/app close); iPhone Safari closing
the tab dropped it.
Each re-login is the full Dex/Authentik redirect dance — many taps on mobile.
**Decision.** Make the session **stateless**: the identity (subject + email) and an absolute
expiry live INSIDE the existing HMAC-signed (HS256) cookie — no server-side table. Plus:
- **30-day sliding TTL** (was 1h), re-signed on each request so an active user never lapses.
- **Persistent cookie** (`Max-Age` set) so it survives browser/app close.
The cookie is HttpOnly + Secure + SameSite=Lax; the HMAC (keyed by the stable ESO
`tapir-session-secret`, which does NOT rotate per deploy) makes it tamper-proof. The payload is
identity, not secrets — the OIDC access/ID tokens are still discarded after callback.
**Consequences.** A deploy/restart no longer logs anyone out (proven by a test: a cookie issued by
one instance is accepted by a fresh instance with the same secret); works across replicas for
free. **Trade:** no server-side revocation — `logout` clears the cookie client-side, but a copied
cookie stays valid until expiry. Accepted for the Stage-0 reader pilot; revisit (server-side
revocation list, or shorter TTL + refresh) if it ever holds sensitive actions. Rotating
`tapir-session-secret` invalidates all sessions — the global logout lever.
**Not addressed here:** the tap-count of the IdP login page itself is Authentik's UX; with
re-login now rare (30-day idle or explicit logout), it matters far less.
---
## ADR-030 — Observability: slog timing + Prometheus metrics (AI-focused)
**Status:** Proposed (2026-06-11). Issue #15. **Draft for review — no code yet.**
**Context / requirements.** Nothing measures the activities that drive Tapir's performance and
UX, and the Stage-0 eval gate (ADR-016) needs a *performance* dimension to sit beside the
return-usage one. We need timing for: caption fetches (the scarce op), summarization (which model
won, how long, fallbacks), Q&A latency, LLM token spend, and basic session/usage (request rate,
latency by route, logins). Requirements:
- R1: structured `slog` timing at each AI call site (human-readable, already the logging stack).
- R2: Prometheus metrics for the same, scrapeable by the cluster's prometheus-operator.
- R3: **AI metrics are the priority** — summarize latency by `model`/`outcome`/`fallback`,
caption-fetch latency by `outcome`, chat latency by `model`, and LLM `tokens` by model+kind.
- R4: HTTP/session metrics via middleware — request count + latency by route, logins.
- R5: bounded label cardinality (no per-user, no raw-path labels).
- R6: `/metrics` must NOT be publicly exposed.
**Decision / architecture.**
1. **New package `internal/metrics`** owns all Prometheus collectors + a typed API
(`ObserveSummarize`, `ObserveCaptionFetch`, `ObserveChat`, `RecordTokens`, `IncLogin`,
`HTTPMiddleware`, `Handler`). Adapters call this API; they never import prometheus types.
2. **New dependency `github.com/prometheus/client_golang`.** Justification: it is *the* standard
Go Prometheus client and the cluster already runs prometheus-operator; hand-rolling exposition
is not worth it. (Needs the dep-justification note in the commit per repo rules.)
3. **The copied `llm` package stays stdlib-only (ADR-004).** It must not import `internal/metrics`.
Token usage is surfaced via an **optional callback** `llm.WithUsageHook(func(model string, prompt, completion int))`
set at wiring time (`buildSummarizer`/`buildChat`) to `metrics.RecordTokens`; `llm.Client` only
gains parsing of the response `usage` block. Our own adapters (`summarizer`, `youtube`, `chat`)
may import `internal/metrics` directly.
4. **HTTP middleware** reads `r.Pattern` AFTER routing (Go 1.22 sets it during ServeMux match), so
the `route` label is the bounded registered pattern (`GET /v/{videoId}`), satisfying R5;
unmatched → `other`.
5. **Dedicated metrics port** (`TAPIR_METRICS_ADDR`, default `:9090`) served by a second
`http.Server` in `cmdServe`; `/metrics` is never on the public app mux (R6). A **PodMonitor**
in `mathias/infra` scrapes it; the deployment exposes the port.
6. **slog** elapsed fields are emitted alongside each metric at the call sites (R1).
**Hook points (where the instrumentation lands).**
- `summarizer.Summarize` — per-endpoint timing + outcome (`success`/`parse_error`/`error`) + fallback flag.
- `youtube.FetchTranscript` — fetch timing + outcome from `domain.Transcript.Source`.
- `chat.Service` answer — timing by model.
- `llm.Client.Complete` — parse `usage`, fire the usage hook.
- `oidc.handleCallback` — `IncLogin`.
- `cmdServe` — wrap `Router()` in `metrics.HTTPMiddleware`; start the metrics server.
**Out of scope / later.** Persisting per-summary latency into Postgres for `tapir report`
(derive UX latency — publish/discovery → summary — from existing timestamps first; only persist
op-latency if the scrape proves insufficient). SPA view (#16) and visual refresh (#17).
**Reversibility.** Additive: a new package + middleware + a metrics port. Removing the PodMonitor
stops scraping; the app is unaffected. No schema change.
**Next steps (gated):** on approval of this ADR → BDD scenarios (`docs/use-cases/observability.feature`
+ scenario-coverage map) → TDD → implement → SemVer + docs + PodMonitor.
---
## ADR-031 — SPA-like reader: inline-expand summary + Q&A in the list (HTMX, no framework)
**Status:** Proposed (2026-06-12). Issue #16. **Draft for review — no code yet.**
**Context / requirements.** The reader is multi-page: a list of compact cards (`/`), then a
navigation to a separate detail page (`/v/{id}`) for the full summary + the docked chat (ADR-027).
It feels less fluid than a single integrated view. We want the full summary AND the per-video
Q&A to open **in place in the list**, no page hop. Requirements:
- R1: clicking a summarized card expands it in place to the full summary (summary/highlights/
takeaways) + the chat dock; a collapse returns it to the compact card.
- R2: **no SPA framework** — stay HTMX + Templ (ADR-003); reuse existing fragments, not a rewrite.
- R3: **progressive enhancement** — with JS off, the card link still navigates to `/v/{id}`
(the detail page stays as the no-JS + deep-link surface). Nothing becomes JS-only.
- R4: only **summarized** cards expand; pending/rate-limited/no-caption cards keep their current
footer behaviour (Summarize button, waiting/none states).
- R5: chat inside an expanded card works exactly as on the detail page (reuse `chatReveal`/
`chatSection` + the existing `/v/{id}/chat` endpoints, unchanged).
**Decision / architecture.**
1. **Reuse the existing fragments.** `summaryBody(r)` and `chatReveal(videoID)` already exist and
render the detail page; a new `expandedCard(r, chatEnabled)` composes the compact header + a
collapse control + `summaryBody` + `chatReveal`. `DetailPage` is refactored to also compose
`summaryBody` so the two never drift (DRY).
2. **Two fragment endpoints** (mirroring the existing list/status HTMX fragment pattern):
`GET /v/{videoId}/expand` → `expandedCard`; collapse reuses the existing compact `VideoCard`
via `GET /v/{videoId}/card`. Both are list-card `<li>` fragments with the SAME `id`
(`video-{id}`), swapped `outerHTML` — same mechanism as `processingCard`/`VideoCard` today.
3. **The compact card's title/"Read" affordance** becomes `hx-get=/v/{id}/expand`,
`hx-target=#video-{id}`, `hx-swap=outerHTML`, with `href=/v/{id}` as the no-JS fallback (R3).
The expanded card's collapse control is the inverse (`hx-get=/v/{id}/card`).
4. **Only when `r.Summarized`** does the expand affordance render (R4); the other states are
unchanged.
5. **v1 does NOT push the URL** (`hx-push-url`) — expand/collapse is ephemeral list UI state; the
detail page remains the deep-link/shareable URL. Deep-linking the open state via `hx-push-url`
is noted as a later option (needs list-state restore on back).
**Out of scope / later.** URL push / deep-linkable open state; the visual refresh (#17) — though
the expanded-card markup is where #17's TUI/charm styling will land, so they pair.
**Reversibility.** Additive: two fragment endpoints + one templ + an affordance swap on the
compact card. Removing the affordance reverts to plain list→detail navigation; the detail page is
untouched. No schema change.
**Next steps (gated):** on approval → BDD (`docs/use-cases/inline_expand.feature` + coverage
map) → TDD → implement → SemVer + docs.
---
## ADR-032 — Visual refresh: one charm-reader layout, light + dark themes
**Status:** Proposed (2026-06-12). Issue #17. **Draft for review — no code yet.** Follows the
sketch-first explore step (3 throwaway mockups in `docs/sketches/`, screenshotted for review).
**Context / decision.** The UI is flat. From the mockups, directions **B (light reader + charm)**
and **C (dark cozy terminal)** are the SAME layout — readable sans body, monospace meta, charm
palette accents, lipgloss-style bordered cards — in two palettes. Direction A (full-monospace TUI)
is dropped as too heavy to read long summaries. Decision: ship that one layout with **both a light
theme (B) and a dark theme (C)**, user-toggleable, defaulting to the OS preference.
**Requirements.**
- R1: one set of markup/structure; the two themes are pure palette (CSS variables), no duplicate templates.
- R2: a **theme toggle** persisted across visits; default to `prefers-color-scheme` when no choice stored.
- R3: charm language in both — mint/purple/pink accents, mono meta + section labels, lipgloss
bordered/gradient cards, the (fixed) ASCII tapir; readable sans body.
- R4: style the existing pieces — list, compact card, **expanded card incl. the already-present
video embed (ADR-031/summaryBody)**, detail page, chat dock, the queue note, the charm spinner.
- R5: WCAG-AA contrast for body text in BOTH themes; keep `prefers-reduced-motion` (already honoured).
- R6: stay HTMX+Templ; no CSS framework.
**Architecture.**
1. **Palette as CSS variables.** `:root` holds the light (B) tokens; `:root[data-theme="dark"]`
holds the dark (C) tokens; a `prefers-color-scheme: dark` media block sets the dark tokens when
no explicit `data-theme` is set. All component CSS references variables only (R1). The existing
`CharmMint/Purple/Pink/Cream/Dim` Go consts remain the source for the spinner's inline colours.
2. **Theme toggle** = a small inline script (a dozen lines, no framework) in `Layout`: on load,
apply stored theme (localStorage) or fall through to the media query; a header toggle button
flips `data-theme` on `<html>` and stores it. This is the one new bit of JS; everything else
stays server-rendered + HTMX. (Considered: cookie + server-render — rejected, a full round-trip
per toggle is clunky for a pure presentation flip.)
3. **Scope = the `styleTag` CSS in `view.go`** (the single style source) plus tiny class hooks in
the templ where needed; content/structure are unchanged, so existing view tests keep passing.
**Verification.** The deployed UI is behind auth (web-shot can't log in), so visual review is via
the mockups now + a styled full-set mockup screenshot before merge, then a live eyeball on device.
Automated tests stay structural/behavioural (theme tokens present, toggle persists, expanded card
embeds the video, no-JS still renders a readable default) — colours are not unit-tested.
**Reversibility.** A CSS theme swap + one small script + a few class hooks; revert `styleTag` to
roll back. No schema, no structural change.
**Next steps (gated):** on approval → BDD (`docs/use-cases/visual_theme.feature` + coverage map)
→ TDD → implement → SemVer + docs.
---
## ADR-033 — Upload audio, get a private Swedish transcript, discuss it (user-supplied STT)
**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #33 (audio-only subset of
#28). **Supersedes ADR-007's speech-to-text deferral for user-supplied audio only.** The
YouTube audio-download rejection in ADR-007 **stands untouched**. Builds on ADR-012 (isolation),
ADR-018 (single replica), ADR-021 (**the constraint**), ADR-022 (endpoint chain), ADR-025
(honest status), ADR-027 (chat). ADR-014 item 4 ("Whisper reconsideration is a future ADR,
gated on caption data") is a different question — whether to transcribe YouTube videos with no
captions — and this ADR does not answer it. Measurements: `docs/research/audio-length-ceiling-2026-10-01.md`.
**Context.** The maintainer has a concrete need: a Swedish Teams voice-only export, 30–90
minutes, to upload and then *discuss* (what was decided, what is open, who owes what). Every
piece exists — a Whisper endpoint on the gateway (infra#274), chat-with-transcript (ADR-027) —
and nothing connects them.
ADR-007 deferred STT for three reasons: yt-dlp fragility, ToS-grey scraping, and GPU contention
with the JEPA PoC. Upload-STT shares **none** of them: the user owns the file, nothing is
scraped, there is no caption rate gate, and the contention moved from koala to iguana. What it
does share is landing Whisper in the codebase, which ADR-007 deferred — so a new ADR is required,
and ADR-017's process note records the one time a guardrail-reversing change shipped without one.
**The constraint that shapes everything.** ADR-021 made `transcripts` a shared, **non-RLS**
table on the explicit grounds that it holds only public caption content and nothing
user-identifying. A recorded internal meeting is the opposite: private third-party content owned
by one user. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse
would leak across tenants **and pass CI green**.
**What was measured, not argued** (research doc above):
- **No transcription ceiling up to 90 minutes.** One 174 MB request through the LiteLLM NodePort
took 440 s (~12× realtime). The first failure is the public ingress `llm-api.d-ma.be`
returning `413` above roughly 8–10 MB (~5 min of WAV) — not whisper, and not a Metal OOM.
The 8:20 failure recorded in #33 is consistent with that cap.
- **`response_format=srt` survives LiteLLM, wrapped** in a JSON envelope (`{"text": "<srt>"}`).
- **The two backends differ.** `iguana/kb-whisper` returns whole-second segments;
`berget/kb-whisper` returns sub-second times and a different `language` token.
- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.**
#33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does
not retrieve; it stuffs the transcript into the prompt and cuts the tail.
**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on
iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal
OOM, deliberately not provoked.
**Decision.**
1. **Audio only; transcript is the only thing kept.** An authenticated user uploads an audio
file; Tapir transcribes it and stores the transcript. The audio itself is deleted after
transcription (temp file, removed in a `defer`, swept at start-up) — no custody problem to
solve, and nothing to hand to #31. Video is rejected with a message naming #28. YouTube
audio-download stays rejected. No public/anonymous upload (ADR-008).
2. **Private store: a separate RLS-scoped table, `uploaded_transcripts`** (resolves #28's
decision 1). `user_id NOT NULL REFERENCES users ON DELETE CASCADE`, `ENABLE` + `FORCE ROW LEVEL
SECURITY`, the same `tapir.current_user_id` policy as every other user-owned table. It carries
the transcript text and its timestamped segments (re-transcribing 90 minutes costs ~7
minutes; discarding the timestamps is irreversible), `status`, the `endpoint` that served
it, `audio_seconds`, and a typed `error_kind`. **Not** a `user_id` column on `transcripts`
with a partial policy, even though that would reuse more code: ADR-021's exception is
precisely that *that table holds no private row*, the isolation test asserts it has no
`user_id`, and a partial policy would turn "single exception" into "single exception, with a
private lane inside it" — the shape under which a future query drops the policy clause and
leaks. Two tables keep both boundaries independently assertable.
3. **The isolation test asserts both boundaries and the set.** (a) User A cannot read, update or
delete user B's uploaded transcript; no GUC set → deny-all. (b) `transcripts` is still
non-RLS with no `user_id`. (c) **The set of public-schema tables with row security off is
asserted exactly** — so a new table added without RLS fails CI instead of passing it. (c) is
new; the existing test only checks the named tables.
4. **Validate by probing, reject clearly.** Format and duration cannot be read from a filename
or MIME type — a Teams "voice-only" recording is typically an `.mp4` container with no video
stream, which is indistinguishable from video by extension. The upload is probed
(`ffprobe`): accept any container with an audio stream and **no** video stream; reject video
with "video is #28"; reject unreadable files. Limits, all configurable, all rejected
synchronously with a specific message rather than by timeout: **200 MB** upload and
**90 minutes** duration. 90 minutes is *the longest length measured*, not a ceiling that was
found; raising it is a measurement, not an edit.
5. **Transcode server-side, always, to 16 kHz mono PCM WAV** — even when the input already is,
rather than assuming. whisper-server's `--convert` is rejected (its `--tmp-dir` is created at
process start and rots under macOS periodic cleanup). The transcoder sits behind a port; the
tests use a fake. **This puts `ffmpeg` and `ffprobe` in the runtime image**, which is
distroless-static today. **Provisional: copy static ffmpeg/ffprobe binaries into the existing
distroless image** (no shell, no package manager retained). This is a heavy new dependency
and is **the maintainer's call** — see Open items.
6. **Transcription adapter via the gateway, never the public hostname.** A `Transcriber` port;
the implementation calls `POST {TAPIR_GATEWAY_URL}/audio/transcriptions` where the gateway
URL is the in-cluster NodePort (`koala:30401/v1`). Primary `iguana/kb-whisper`, fallback
`berget/kb-whisper` (immune to iguana contention, which can wedge permanently — infra#273).
- **Fallback triggers on any primary failure** (transport error, timeout, non-2xx). It never
branches on the body: `{"error":"failed to process audio"}` has had at least five distinct
causes.
- **`response_format=verbose_json`**, normalised to one segment shape: times clamped to the
audio duration (the final segment can end past it), one language token. Not `srt`: the
envelope adds a parse step and iguana's segment timing is the same either way.
- **No chunking in v1.** Nothing failed to 90 minutes (measured). Chunk-and-stitch is the
answer *if* the cap is raised or smaller requests become necessary — see triggers.
- **The serving endpoint is recorded per row**, and surfaced, because the two backends
differ in timestamp granularity: which one served a transcript is a quality fact.
- **Berget is a cloud endpoint.** On fallback the audio of a private meeting leaves the
homelab. #33 asks for this fallback; it is configurable (`TAPIR_TRANSCRIBE_FALLBACK_MODEL`,
empty disables), recorded per row, and called out in Open items rather than defaulted
silently.
- Swedish only (`language=sv`). Language detection and routing (#28 decision 2) are not
touched.
- `-H 'Expect:'`, multipart field `file` — encoded in the adapter and its tests, not rediscovered.
7. **Jobs run in-process, one at a time (ADR-018 stands).** Transcription is minutes (~7.5 min
for 90) but a single goroutine behind a one-slot semaphore: two simultaneous 90-minute jobs on
iguana were not measured, so concurrency is 1 until they are. No worker, no new deployable,
no second replica. Honest status per ADR-025: `queued → transcribing → done | failed`, polled
by the same `/status` pattern, every terminal state a rendered card, never a stuck spinner.
Audio is not retained, so a restart mid-job **cannot resume**: at start-up any
`queued`/`transcribing` row is marked `failed` with `error_kind = interrupted`, and the user
re-uploads. The single-replica assumption gains a second dependent (the semaphore is
process-local) and is recorded here.
8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers
look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism
as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model)
carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour
meeting to its first third, so uploaded transcripts take their own cap,
`TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at
measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a
guess until the chat slice measures what the offered models actually accept; it is not
evidence that they can.
9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py`
hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection;
per-transcript deletion UX (account deletion cascades; see Open items); retention of audio.
**On the sequencing question #28 asks of itself** — *is this the thing that best serves the
Stage-0 gate, or the most interesting thing to build?* Recorded answer: it proceeds **ahead of
#32** (the demand test) as the maintainer's explicit, knowing call (#28, comment of 2026-10-01),
on a concrete need rather than speculation. It is not a return-nudge (ADR-020/027): it adds
nothing that prompts a user to come back, so it does not contaminate the unprompted-return signal
ADR-016 measures. The honest risk is the one ADR-016 names: it is larger than anything ADR-020
deferred. The scope cut — chat first, no rich analysis, no video — is what keeps it bounded.
**Consequences.**
- **Positive.** The user's request is met end to end with existing parts. ADR-021's exception stays
singular and now provably so. The measured ceiling removes chunking from v1. No audio is stored.
- **Negative / watch.**
- Tapir's runtime image gains `ffmpeg`: a larger image and a new CVE surface to patch.
- iguana becomes load-bearing for a user-facing feature (#28 decision 3; infra#273 is still the
dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e.
private audio leaving the homelab, silently unless the per-row endpoint is surfaced.
- A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it.
- Chat over a long transcript is bounded by model context, not by anything in Tapir.
- Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`).
**Validation / resolution triggers** (each observable by someone other than the author):
- *Chunking*: build it when the duration cap is raised above 90 min, or when a measured
request at the current cap fails with whisper-side (not ingress) errors.
- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback,
swap the primary/fallback order and revisit infra#273.
- *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose
answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap.
- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one
replica, ADR-018 is revisited.
- *The decision itself*: if the maintainer has not used the feature unprompted within the
ADR-016 window, it was the interesting thing, not the useful one — remove it.
**Open items this ADR creates.**
- **DECISION NEEDED — ffmpeg in the runtime image.** *Options:* (A) static ffmpeg/ffprobe copied
into distroless **(recommended — the only option that satisfies "accept m4a/mp3/opus/mp4,
validate duration, reject video"; pure-Go decoders cover wav/mp3 only)**; (B) accept WAV
only, convert client-side (fails "I just want to upload it"); (C) a transcode sidecar
(second deployable, ADR-018's reasoning against). *Blocks:* the Dockerfile slice, not the
tests, which use the port.
- **DECISION NEEDED — berget fallback for private audio.** *Options:* (A) enabled by default as
#33 specifies, with the serving endpoint shown **(recommended, matches the issue)**; (B)
disabled by default, a failed iguana job fails honestly and the user retries. *Blocks:*
the production default only; the adapter ships either way.
- Verify Tapir's ingress `proxy-body-size` against the 200 MB cap (`infra`).
- Per-transcript deletion is not built; account deletion cascades. A private-content store
without a delete button is a gap worth a ticket, not a silent omission.
**Reversibility.** Additive: one table (down-migration drops it), one adapter, one upload
surface, one image change. Removing the upload route removes the feature; `transcripts`, `videos`
and ADR-021 are untouched.
---
## Rejected alternatives
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
@@ -1083,6 +1524,10 @@ maps to the ADR that settles it.
| Feedback-based Stage 0 gate (friends saying it's useful) | Politeness bias makes asked-for feedback the least reliable signal; return-usage is the real test | ADR-016 |
| Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 |
| k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 |
| Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 |
| Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 |
| whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 |
| Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 |
If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
not a silent reversal.
+26
View File
@@ -0,0 +1,26 @@
# Vocabulary (hand-written pilot 2026-06 — will be generated by langgen in Phase 2)
| Term | Means | Never say |
|---|---|---|
| transcript | Captions-first text of a video; the source for summarizing. Shared store keyed by `(provider, video_id)`. | "subtitles file", "the audio" |
| video | Per-user record of a seen video; the unit of work. Not globally deduped. | "the shared video" |
| subscription | A watched channel on a user's connection that Tapir polls for new videos. | "feed", "follow" |
| summary | A video's produced output: summary text + highlights + takeaways + AI provenance. | "transcript" |
| highlight | A notable point pulled from a video (`Summary.Highlights`). | "takeaway" |
| takeaway | An actionable conclusion from a video (`Summary.Takeaways`). | "highlight" |
| sink | A delivery destination for a summary (`store`, `brain`). New one = new adapter. | "the database" |
| AI router | Local-first chain: local Primary → local fallback → external worst-case. | "the API" |
| BYO-AI fallback | User's own external AI key, used only when local fails; opt-in. Without it, content is never sent externally. | "the default AI" |
| brain | Persistent homelab knowledge store; in Tapir, one optional HTTP sink (`brain_ingest`), not the filesystem package. | "the database", "the store" |
| LiteLLM gateway | Local AI gateway (`koala:30401/v1`); the Primary in the AI router. | "piguard:4000", "koala:4000", "the cloud" |
| connection | A connected video account a user authorizes via OAuth; subscriptions hang off it. | "login", "session" |
## Caveman rubric
Before any HIGH/CRITICAL operation, output one line:
`caveman: me <verb> <object>, not <excluded thing>`
Valid iff: (1) only vocabulary terms + plain verbs, (2) a stranger could identify
the exact operation, (3) names one thing explicitly NOT being done.
Examples:
- `caveman: me delete user connection, not the shared transcript`
- `caveman: me send transcript to BYO-AI fallback, not the LiteLLM gateway`
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"strings"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/config"
"git.d-ma.be/mathias/tapir/internal/config"
)
// TestChatModelsReuseTheChainLocalFirst: the switcher offers the ADR-022 chain in
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"log/slog"
"sync"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/runner"
)
// discoveryRunner runs one user's discovery pass.
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/runner"
)
// serialize must guarantee at most one discovery pass runs at a time, so a
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"fmt"
"os"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// Env var names for the read-only CLI. DSN and user id are never hardcoded — the
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
func TestFormatListColumnsAndOrdering(t *testing.T) {
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"strings"
"text/tabwriter"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// runList prints the user's stored summaries as a table, most recent first.
+51 -19
View File
@@ -23,14 +23,15 @@ import (
"sync"
"time"
"gitea.d-ma.be/mathias/tapir/internal/adapters/secrets"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/adapters/youtube"
"gitea.d-ma.be/mathias/tapir/internal/auth"
"gitea.d-ma.be/mathias/tapir/internal/config"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"gitea.d-ma.be/mathias/tapir/internal/web"
"gitea.d-ma.be/mathias/tapir/internal/web/oidc"
"git.d-ma.be/mathias/tapir/internal/adapters/secrets"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/youtube"
"git.d-ma.be/mathias/tapir/internal/auth"
"git.d-ma.be/mathias/tapir/internal/config"
"git.d-ma.be/mathias/tapir/internal/metrics"
"git.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/web/oidc"
)
func main() {
@@ -262,23 +263,36 @@ func cmdServe(ctx context.Context, log *slog.Logger) error {
// One lock shared by the scheduler and connect-triggered passes (#6) so
// they never fetch concurrently — the single-fetcher invariant (ADR-018).
runUser := serialize(&sync.Mutex{}, rawRunUser)
// Onboarding burst (Feature 1): after the connect-triggered discovery pass,
// summarize up to OnboardSummarizeCount of the user's NEWEST unsummarized
// videos so a fresh account gets real summaries in its first session. Hard
// cap; explicit, so it bypasses the recency window — but every fetch still
// goes through globalFetchGate via the Processor. No-op when disabled
// (count 0) or queue-only (no Processor).
// Onboarding burst (Feature 1, refined by ADR-028): after the connect-triggered
// discovery pass, summarize up to OnboardSummarizeCount of the user's newest
// LIKELY-GOOD unsummarized videos so a fresh account gets a strong first
// session. Selection avoids known-junk (Shorts/over-long/livestream VODs via
// the persisted duration); the burst leads its chain with the stronger onboard
// model. Hard cap; explicit, so it bypasses the recency window — but every
// fetch still goes through globalFetchGate. No-op when disabled (count 0) or
// queue-only (no processor).
//
// burstProcessor leads with the stronger model (ADR-028); it collapses onto the
// shared Processor when the onboard model is empty/equal-to-primary or the
// engine config is incomplete.
burstProcessor := app.Processor
if burstEngine, berr := buildBurstProcessor(cfg, st); berr != nil {
return berr
} else if burstEngine != nil {
burstProcessor = &engineProcessor{engine: burstEngine, store: st}
log.Info("onboarding burst uses a stronger model", "onboard_model", cfg.OnboardSummarizerModel)
}
onboard := func(ctx context.Context, userID string) {
if cfg.OnboardSummarizeCount <= 0 || app.Processor == nil {
if cfg.OnboardSummarizeCount <= 0 || burstProcessor == nil {
return
}
ids, err := st.NewestUnsummarizedVideoIDs(ctx, userID, cfg.OnboardSummarizeCount)
ids, err := st.OnboardBurstVideoIDs(ctx, userID, cfg.OnboardSummarizeCount, cfg.MinVideoSeconds, cfg.OnboardMaxVideoSeconds)
if err != nil {
log.Warn("onboarding: list newest unsummarized", "user", userID, "err", err)
log.Warn("onboarding: list burst candidates", "user", userID, "err", err)
return
}
for _, id := range ids {
if err := app.Processor.ProcessVideo(ctx, userID, id); err != nil {
if err := burstProcessor.ProcessVideo(ctx, userID, id); err != nil {
log.Warn("onboarding: summarize", "user", userID, "video", id, "err", err)
}
}
@@ -297,16 +311,34 @@ func cmdServe(ctx context.Context, log *slog.Logger) error {
srv := &http.Server{
Addr: cfg.HTTPAddr,
Handler: app.Router(),
Handler: metrics.HTTPMiddleware(app.Router()),
ReadHeaderTimeout: 10 * time.Second,
}
// Prometheus /metrics on a SEPARATE port (ADR-030) — never on the public app
// mux, so a scrape is in-cluster only. Empty TAPIR_METRICS_ADDR disables it.
var metricsSrv *http.Server
if cfg.MetricsAddr != "" {
mmux := http.NewServeMux()
mmux.Handle("GET /metrics", metrics.Handler())
metricsSrv = &http.Server{Addr: cfg.MetricsAddr, Handler: mmux, ReadHeaderTimeout: 10 * time.Second}
go func() {
log.Info("serving metrics", "addr", cfg.MetricsAddr)
if err := metricsSrv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Error("metrics server", "err", err)
}
}()
}
// Graceful shutdown on signal: stop accepting, drain in-flight requests.
go func() {
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = srv.Shutdown(shutdownCtx)
if metricsSrv != nil {
_ = metricsSrv.Shutdown(shutdownCtx)
}
}()
log.Info("serving web ui", "addr", cfg.HTTPAddr, "user", cfg.UserID)
+96 -28
View File
@@ -5,17 +5,18 @@ import (
"fmt"
"strings"
"gitea.d-ma.be/mathias/tapir/internal/adapters/chat"
"gitea.d-ma.be/mathias/tapir/internal/adapters/llm"
"gitea.d-ma.be/mathias/tapir/internal/adapters/secrets"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/adapters/summarizer"
"gitea.d-ma.be/mathias/tapir/internal/adapters/youtube"
"gitea.d-ma.be/mathias/tapir/internal/config"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"gitea.d-ma.be/mathias/tapir/internal/usecase"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/chat"
"git.d-ma.be/mathias/tapir/internal/adapters/llm"
"git.d-ma.be/mathias/tapir/internal/adapters/secrets"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/summarizer"
"git.d-ma.be/mathias/tapir/internal/adapters/youtube"
"git.d-ma.be/mathias/tapir/internal/config"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/metrics"
"git.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/usecase"
"git.d-ma.be/mathias/tapir/internal/web"
)
// videoFetcher adapts the YouTube adapter to web.VideoFetcher for the paste flow
@@ -52,13 +53,7 @@ func (f videoFetcher) FetchVideo(ctx context.Context, userID, videoID string) (d
// alias, not a second client config. Empty model entries are skipped, so a
// client deployment can set the cloud fallback empty to keep content local.
func buildSummarizer(cfg config.Config) *summarizer.Summarizer {
mk := func(model string) summarizer.Endpoint {
return summarizer.Endpoint{
Client: llm.New(cfg.GatewayURL, cfg.GatewayKey, model, cfg.SummarizerTimeout, llm.WithMaxTokens(cfg.SummaryMaxTokens)),
Provider: providerOf(model),
Model: model,
}
}
mk := summarizerEndpoint(cfg)
eps := []summarizer.Endpoint{mk(cfg.SummarizerModel)}
if cfg.FallbackModel != "" && cfg.FallbackModel != cfg.SummarizerModel {
eps = append(eps, mk(cfg.FallbackModel))
@@ -69,6 +64,55 @@ func buildSummarizer(cfg config.Config) *summarizer.Summarizer {
return summarizer.NewChain(eps, cfg.MaxTranscriptChars)
}
// summarizerEndpoint returns a constructor for a chain endpoint over the one
// LiteLLM gateway, varying only the model alias (the gateway fronts both
// llama-swap and berget). Shared by the standard and burst chains.
func summarizerEndpoint(cfg config.Config) func(model string) summarizer.Endpoint {
return func(model string) summarizer.Endpoint {
return summarizer.Endpoint{
Client: llm.New(cfg.GatewayURL, cfg.GatewayKey, model, cfg.SummarizerTimeout, llm.WithMaxTokens(cfg.SummaryMaxTokens), llm.WithUsageHook(metrics.RecordTokens)),
Provider: providerOf(model),
Model: model,
}
}
}
// burstChainModels is the ordered, deduped model list for the onboarding burst
// (ADR-028): the stronger onboard model leads, then the standard ADR-022 chain
// (primary → local fallback → cloud) follows as resilience. Empty entries are
// dropped and duplicates collapsed, so the NDA lever (empty cloud fallback) keeps
// the burst chain fully local exactly as the standard chain does.
func burstChainModels(cfg config.Config) []string {
var models []string
add := func(m string) {
if m == "" {
return
}
for _, e := range models {
if e == m {
return
}
}
models = append(models, m)
}
add(cfg.OnboardSummarizerModel)
add(cfg.SummarizerModel)
add(cfg.FallbackModel)
add(cfg.CloudFallbackModel)
return models
}
// buildBurstSummarizer builds the onboarding-burst summarizer chain (ADR-028):
// the onboard model first, then the standard chain as fallback, deduped.
func buildBurstSummarizer(cfg config.Config) *summarizer.Summarizer {
mk := summarizerEndpoint(cfg)
var eps []summarizer.Endpoint
for _, m := range burstChainModels(cfg) {
eps = append(eps, mk(m))
}
return summarizer.NewChain(eps, cfg.MaxTranscriptChars)
}
// chatModels is the ordered, local-first set of models offered in the chat
// switcher (ADR-027), reusing the ADR-022 chain: primary → local fallback →
// cloud. Empty entries are dropped and duplicates collapsed, so a client/NDA
@@ -107,7 +151,7 @@ func buildChat(cfg config.Config) *chat.Service {
return nil
}
newClient := func(model string) chat.Completer {
return llm.New(cfg.GatewayURL, cfg.GatewayKey, model, cfg.SummarizerTimeout, llm.WithMaxTokens(cfg.SummaryMaxTokens))
return llm.New(cfg.GatewayURL, cfg.GatewayKey, model, cfg.SummarizerTimeout, llm.WithMaxTokens(cfg.SummaryMaxTokens), llm.WithUsageHook(metrics.RecordTokens))
}
return chat.New(newClient, models, cfg.MaxTranscriptChars)
}
@@ -126,15 +170,7 @@ func buildProcessor(cfg config.Config, st *store.Store) (*usecase.Engine, error)
return nil, nil
}
secretStore := secrets.NewFileStore(cfg.SecretsFile)
src := youtube.New(youtube.Config{
ClientID: cfg.YTClientID,
ClientSecret: cfg.YTClientSecret,
TokenSecretRef: cfg.YTTokenRef,
PreferredLanguages: []string{"en"},
MinVideoSeconds: cfg.MinVideoSeconds,
}, secretStore)
src := newYouTubeSource(cfg, secrets.NewFileStore(cfg.SecretsFile))
sum := buildSummarizer(cfg)
// The store is both the summary sink and the shared transcript cache (ADR-021):
@@ -145,6 +181,38 @@ func buildProcessor(cfg config.Config, st *store.Store) (*usecase.Engine, error)
return eng, nil
}
// newYouTubeSource builds the captions-first VideoSource shared by the standard
// and burst processors — same per-process YouTube credentials and ADR-023 Shorts
// filter; only the summarizer chain differs between them.
func newYouTubeSource(cfg config.Config, secretStore ports.SecretStore) ports.VideoSource {
return youtube.New(youtube.Config{
ClientID: cfg.YTClientID,
ClientSecret: cfg.YTClientSecret,
TokenSecretRef: cfg.YTTokenRef,
PreferredLanguages: []string{"en"},
MinVideoSeconds: cfg.MinVideoSeconds,
}, secretStore)
}
// buildBurstProcessor wires a processor whose summarizer leads with the stronger
// onboard model (ADR-028), used only by the connect-time burst over the SAME
// store / transcript cache / sink — a wiring choice; the engine and ports are
// unchanged. Returns (nil, nil) — the collapse lever — when the onboard model is
// empty or equal to the primary (the burst then reuses the shared processor), or
// when the engine config is incomplete (queue-only, same as buildProcessor).
func buildBurstProcessor(cfg config.Config, st *store.Store) (*usecase.Engine, error) {
if cfg.OnboardSummarizerModel == "" || cfg.OnboardSummarizerModel == cfg.SummarizerModel {
return nil, nil
}
if cfg.GatewayURL == "" || cfg.YTClientID == "" || cfg.YTClientSecret == "" || cfg.SecretsFile == "" {
return nil, nil
}
src := newYouTubeSource(cfg, secrets.NewFileStore(cfg.SecretsFile))
eng := usecase.NewEngine(src, buildBurstSummarizer(cfg), st)
eng.Transcripts = st
return eng, nil
}
// engineProcessor adapts the engine (which works in terms of a domain.Video) to
// the web.Processor port (which works in terms of a stored video id): it loads the
// video row, runs the engine, and — on a produced summary — clears the manual
+63 -1
View File
@@ -3,7 +3,7 @@ package main
import (
"testing"
"gitea.d-ma.be/mathias/tapir/internal/config"
"git.d-ma.be/mathias/tapir/internal/config"
)
// TestBuildProcessorNilOnIncompleteConfig asserts the queue-only fallback: when a
@@ -45,3 +45,65 @@ func TestBuildProcessorNilOnIncompleteConfig(t *testing.T) {
})
}
}
// TestBurstChainModelsLeadsWithOnboardModel: the onboarding burst chain (ADR-028)
// leads with the stronger onboard model, then falls back through the standard
// ADR-022 chain (primary -> local fallback -> cloud), deduped.
func TestBurstChainModelsLeadsWithOnboardModel(t *testing.T) {
got := burstChainModels(config.Config{
OnboardSummarizerModel: "iguana/gemma4-26b",
SummarizerModel: "koala/phi4-mini",
FallbackModel: "iguana/gemma4-26b", // also the onboard model -> dedup
CloudFallbackModel: "berget/mistral-small",
})
want := []string{"iguana/gemma4-26b", "koala/phi4-mini", "berget/mistral-small"}
if len(got) != len(want) {
t.Fatalf("burstChainModels = %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("burstChainModels = %v, want %v", got, want)
}
}
}
// TestBurstChainModelsCloudAbsentWhenDisabled: the NDA lever holds for the burst
// too — empty cloud fallback keeps the burst chain fully local.
func TestBurstChainModelsCloudAbsentWhenDisabled(t *testing.T) {
got := burstChainModels(config.Config{
OnboardSummarizerModel: "iguana/gemma4-26b",
SummarizerModel: "koala/phi4-mini",
CloudFallbackModel: "",
})
for _, m := range got {
if m == "" || m == "berget/mistral-small" {
t.Fatalf("cloud model leaked into burst chain: %v", got)
}
}
}
// TestBuildBurstProcessorNilWhenCollapsed: an empty or primary-equal onboard model
// collapses the burst onto the shared processor (buildBurstProcessor returns nil).
func TestBuildBurstProcessorNilWhenCollapsed(t *testing.T) {
base := config.Config{
GatewayURL: "http://gw/v1",
YTClientID: "id",
YTClientSecret: "secret",
SecretsFile: "/tmp/secrets.json",
SummarizerModel: "koala/phi4-mini",
}
t.Run("empty onboard model", func(t *testing.T) {
base.OnboardSummarizerModel = ""
eng, err := buildBurstProcessor(base, nil)
if err != nil || eng != nil {
t.Fatalf("buildBurstProcessor = (%v, %v), want (nil, nil)", eng, err)
}
})
t.Run("onboard model equals primary", func(t *testing.T) {
base.OnboardSummarizerModel = "koala/phi4-mini"
eng, err := buildBurstProcessor(base, nil)
if err != nil || eng != nil {
t.Fatalf("buildBurstProcessor = (%v, %v), want (nil, nil)", eng, err)
}
})
}
+34 -4
View File
@@ -6,14 +6,36 @@ import (
"io"
"os"
"text/tabwriter"
"time"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// gateThreshold is the Stage-0 gate (VISION/ADR-016): usage in >= 2 distinct
// weeks. The gate passes when any user reaches it.
const gateThreshold = 2
// defaultGateStart is the date Stage-0 return-usage tracking begins: the morning
// the pilot was actually unblocked and summaries started flowing (2026-06-11).
// Activity before this — testing, the period the pilot was stuck on zero — is
// noise and must not count toward the gate. Override with TAPIR_USAGE_GATE_START
// (YYYY-MM-DD). The gate measures whether users RETURN once it genuinely works.
const defaultGateStart = "2026-06-11"
// gateStart resolves the baseline date from TAPIR_USAGE_GATE_START or the default,
// parsed as a UTC calendar day.
func gateStart() (time.Time, error) {
v := os.Getenv("TAPIR_USAGE_GATE_START")
if v == "" {
v = defaultGateStart
}
t, err := time.Parse("2006-01-02", v)
if err != nil {
return time.Time{}, fmt.Errorf("TAPIR_USAGE_GATE_START=%q: want YYYY-MM-DD: %w", v, err)
}
return t, nil
}
// runReport prints the Stage-0 usage gate: per-user distinct active weeks (reads
// UNION acts) and the pass/fail verdict. Read-only, cross-user — needs only
// TAPIR_DB_DSN (not TAPIR_USER_ID; the report enumerates all users itself).
@@ -28,16 +50,24 @@ func runReport(ctx context.Context, _ []string) error {
}
defer s.Close()
rows, err := s.ActiveWeeks(ctx)
since, err := gateStart()
if err != nil {
return err
}
return formatReport(os.Stdout, rows)
rows, err := s.ActiveWeeks(ctx, since)
if err != nil {
return err
}
return formatReport(os.Stdout, rows, since)
}
// formatReport renders the per-user week counts and the gate verdict. Pure: no DB,
// no env — so the layout and verdict logic are unit-testable without Postgres.
func formatReport(w io.Writer, rows []store.UserActiveWeeks) error {
func formatReport(w io.Writer, rows []store.UserActiveWeeks, since time.Time) error {
if _, err := fmt.Fprintf(w, "Counting usage since %s (Stage-0 gate baseline)\n\n", since.Format("2006-01-02")); err != nil {
return err
}
if len(rows) == 0 {
_, err := fmt.Fprintln(w, "no users yet")
return err
+8 -4
View File
@@ -3,12 +3,15 @@ package main
import (
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
var testSince = time.Date(2026, 6, 11, 0, 0, 0, 0, time.UTC)
func TestFormatReportColumnsAndGatePass(t *testing.T) {
rows := []store.UserActiveWeeks{
{UserID: "user-a", DisplayName: "Ada", ActiveWeeks: 3},
@@ -16,9 +19,10 @@ func TestFormatReportColumnsAndGatePass(t *testing.T) {
}
var b strings.Builder
require.NoError(t, formatReport(&b, rows))
require.NoError(t, formatReport(&b, rows, testSince))
out := b.String()
require.Contains(t, out, "since 2026-06-11", "report states the gate baseline date")
require.Contains(t, out, "USER")
require.Contains(t, out, "ACTIVE_WEEKS")
require.Contains(t, out, "Ada")
@@ -34,12 +38,12 @@ func TestFormatReportGateNotMet(t *testing.T) {
rows := []store.UserActiveWeeks{{UserID: "user-a", ActiveWeeks: 1}}
var b strings.Builder
require.NoError(t, formatReport(&b, rows))
require.NoError(t, formatReport(&b, rows, testSince))
require.Contains(t, b.String(), "NOT YET MET", "no user at >= 2 weeks fails the gate")
}
func TestFormatReportEmpty(t *testing.T) {
var b strings.Builder
require.NoError(t, formatReport(&b, nil))
require.NoError(t, formatReport(&b, nil, testSince))
require.Contains(t, b.String(), "no users yet")
}
+7 -7
View File
@@ -6,13 +6,13 @@ import (
"log/slog"
"time"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/adapters/youtube"
"gitea.d-ma.be/mathias/tapir/internal/config"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"gitea.d-ma.be/mathias/tapir/internal/usecase"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/youtube"
"git.d-ma.be/mathias/tapir/internal/config"
"git.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/usecase"
"git.d-ma.be/mathias/tapir/internal/web"
)
// buildUserRunner constructs a runner.Runner for one user, reusing the same
+2 -2
View File
@@ -11,8 +11,8 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/runner"
)
func quietLog() *slog.Logger {
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"os"
"strings"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// runShow prints the full summary for one video: text, highlights, takeaways,
+26
View File
@@ -250,6 +250,32 @@ After (newest-first): `[chanB-new, chanA-mid, chanA-old, chanB-null]`
The set of *processed* videos now also excludes auto-mode back-catalogue beyond the recency
window (those stay listed, summarised on demand); within the processed set, only order changes.
### Connect-time onboarding burst (ADR-018 → ADR-028)
On a successful YouTube connect, `ConnectHandler` enqueues a connect-triggered discovery pass;
the `discoveryTrigger` runs that pass and then fires the **onboarding burst** — a third entry path
that summarises up to `TAPIR_ONBOARD_SUMMARIZE_COUNT` (default 3, hard-capped) of the new user's
videos so the first session is not empty. The burst still flows through `globalFetchGate` (it is
not a throughput change); ADR-028 sharpened *which* videos and *which model*:
- **Selection** is `OnboardBurstVideoIDs`, not pure newest-first. It keeps newest-first order but
excludes a video whose **known** duration is outside `[TAPIR_MIN_VIDEO_SECONDS,
TAPIR_ONBOARD_MAX_VIDEO_SECONDS]` (drops Shorts and multi-hour livestream VODs). An unknown
(NULL) duration is degrade-open — kept, but ranked after known-good rows. The connect-triggered
discovery pass runs *before* the burst, and ADR-023's `videos.list` enrichment now **persists**
`duration_s` (instead of discarding it after the Shorts filter), so a fresh user's candidates
carry a duration in time for selection.
- **Model**: the burst runs through a dedicated summarizer chain led by
`TAPIR_ONBOARD_SUMMARIZER_MODEL` (default `iguana/gemma4-26b`, the stronger local model), with
the standard ADR-022 chain following as fallback. This is a wiring choice — a second
`engineProcessor` over the same store / transcript cache / sink; the engine and ports are
unchanged. Empty / equal-to-primary collapses it back onto the shared processor.
`has-captions` is deliberately **not** a selection signal — it is only knowable after a gate fetch
(or a ~0-probability cache hit at pilot scale), so the burst can avoid known-junk but cannot
promise captions. Cached-transcript-first selection was investigated and rejected (ADR-028:
~3% cross-user overlap).
---
## Sequence — core use case: new video summarized
+9
View File
@@ -224,6 +224,15 @@ knobs plus one load-bearing deployment constraint:
- `TAPIR_DISCOVERY_INTERVAL` — Go duration, e.g. `2h`. The cadence the serve process runs a
discovery pass for every registered user (run-once-on-startup, then every interval).
**Unset or `0` = disabled** (dev/tests never auto-fetch).
- `TAPIR_USAGE_GATE_START` — `YYYY-MM-DD`, default **`2026-06-11`** (the morning the pilot was
unblocked and summaries started flowing). `tapir report` counts return-usage (distinct active
weeks, ADR-016) only from this date, so pre-launch testing and the blocked period are excluded.
- `TAPIR_METRICS_ADDR` — listen address for the Prometheus `/metrics` endpoint (ADR-030).
**Default `:9090`** — a SEPARATE port from `TAPIR_HTTP_ADDR` so metrics are never on the public
app; scraped in-cluster only (PodMonitor). Empty disables the metrics server. Key series:
`tapir_summarize_duration_seconds{model,outcome,fallback}`, `tapir_caption_fetch_duration_seconds{outcome}`,
`tapir_chat_duration_seconds{model}`, `tapir_llm_tokens_total{model,kind}`,
`tapir_http_request_duration_seconds{method,route}`, `tapir_logins_total`.
- `TAPIR_FETCH_RATE` — Go duration, default `2s`. The **process-wide per-egress-IP caption-fetch
rate gate** (ADR-014 item 2). Every caption fetch — scheduler runners *and* the web "Summarize"
click-path — serialises through this one limiter so the pod cannot collectively trip 429s. `0`
@@ -0,0 +1,110 @@
# Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01)
Question (#33, "the main technical risk"): at what recording length does Swedish
transcription through the LiteLLM gateway first fail, and with what failure mode?
Measured **before** any chunking design, with iguana not holding a large ollama model,
so the result is the ceiling and not the Metal OOM of infra#273.
## Answer
**No transcription ceiling was found at or below 90 minutes.** A single request carrying
90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing
that actually fails is **not whisper**: it is the public gateway ingress, which rejects the
upload with `413` at roughly 5.5 minutes of WAV.
| Route | First failure | Failure mode |
|---|---|---|
| `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly |
| `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — |
| `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — |
The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a
30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the
size-dependent form described, a body-size cap explains it without one. (The OOM
mechanism in infra#273 is separately real; this run did not provoke it.)
The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the
ingress configuration, which lives in `infra`. "About 10 MB" is an inference.
## Method
- **Audio.** No known-good Swedish recording is available that is safe to use: the only real
one (`IMG_1233.wav`) is the private `/bygge` recording (#31 custody still open) and was
deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char
five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`).
Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe).
Nothing here is a real transcript; none of it is committed.
- **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`,
`response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only
through `op run` and never on argv.
- **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident:
nothing, or `nomic-embed-text` (0.37 GB). Never a large model.
## Results (`iguana/kb-whisper`, NodePort)
| Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars |
|---|---|---|---|---|---|---|
| 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — |
| 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — |
| 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 |
| 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 |
| 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 |
| 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 |
| 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 |
| 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 |
- Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is
a minutes-long job, not a request/response one (see ADR-033 on ADR-018).
- Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are
monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop
hallucination appeared across 90 minutes. This is a weak check — the input is a repeated
loop and synthetic voice — not evidence of accuracy.
- **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration`
330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`.
## Findings that change the design
1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the
gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the
gateway, never the host" and named the public URL; the gateway is right, that URL is not.
`TAPIR_GATEWAY_URL` already points at `koala:30401/v1`.
2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON
`{"text": "<the SRT>", "usage": …}`, not a bare SRT body — a client that parses the body
as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside
an envelope.
3. **The two backends return different timestamp granularity.** `iguana/kb-whisper`
segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real
sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`).
`verbose_json` and the default JSON were byte-identical on iguana. An adapter that
falls back between the two must normalise both, and "which endpoint served this" is
a quality fact, not just a log line.
4. **The existing chat path cannot answer about most of a long recording.** At
828 chars/min (this fixture; real speech is plausibly a little denser),
`TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute
transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping
the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of
a one-hour meeting get an answer from a transcript that does not contain it. #33 says
chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and
truncates**, and its ceiling is the model context. See ADR-033.
5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The
chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is
not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces
smaller requests — see ADR-033's resolution triggers.
## What this did not measure
- **Metal OOM under contention.** Deliberately avoided. A big resident ollama model
during a long transcription is still the known way to wedge iguana permanently.
- **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and
speaker overlap in a Teams export are untested. Speed is probably representative;
accuracy is not claimed.
- **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper
without a server-side transcode is not established here (ADR-033 treats it as needing one).
- **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were
not tried.
- **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not
size-capped at meeting length for 22 minutes; not proven at 90.
- **Time-of-day / other tenants.** One afternoon, one run per length.
Scripts were throwaway shell/Python in a session scratchpad and are not committed; the
method above reproduces them.
+63
View File
@@ -0,0 +1,63 @@
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>tapir — A · TUI panels</title>
<style>
:root{--bg:#0d0d12;--panel:#14141c;--line:#2a2a3a;--mint:#0EF9B6;--pink:#F740A0;--purple:#7653fc;--cream:#F5E9D6;--dim:#6b6b86;--text:#d9d9e6}
*{box-sizing:border-box}
body{margin:0;background:var(--bg);color:var(--text);font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,"Cascadia Code",monospace}
.wrap{max-width:760px;margin:0 auto;padding:18px 14px 60px}
header{display:flex;align-items:center;gap:10px;border-bottom:1px solid var(--line);padding-bottom:12px;margin-bottom:18px}
.brand{color:var(--mint);font-weight:700;letter-spacing:.5px}
.brand b{color:var(--cream)}
.tip{margin-left:auto;color:var(--dim);font-size:12px}
.note{color:var(--dim);font-size:12.5px;border-left:2px solid var(--purple);padding:6px 10px;margin:0 0 16px;background:#11111a}
/* lipgloss-style bordered panel */
.card{border:1px solid var(--line);border-radius:8px;background:var(--panel);padding:12px 14px;margin:0 0 12px;position:relative}
.card::before{content:"";position:absolute;left:0;top:10px;bottom:10px;width:3px;border-radius:3px;background:var(--purple)}
.card.ready::before{background:var(--mint)}
.title{color:var(--cream);font-size:15px;font-weight:600;margin:0 0 4px}
.meta{color:var(--dim);font-size:12px}
.chip{display:inline-block;border:1px solid var(--mint);color:var(--mint);border-radius:4px;padding:0 6px;font-size:11px;margin-left:6px}
.chip.q{border-color:var(--pink);color:var(--pink)}
.preview{color:var(--dim);margin:6px 0 0;font-size:13px}
/* expanded */
.exp{border-color:var(--purple)}
.exp .head{display:flex;justify-content:space-between;align-items:baseline}
.collapse{color:var(--mint);font-size:12px;text-decoration:none;border:1px solid var(--line);border-radius:4px;padding:1px 7px}
.sec h2{color:var(--mint);font-size:12px;text-transform:uppercase;letter-spacing:1px;margin:16px 0 6px;border-bottom:1px dashed var(--line);padding-bottom:3px}
.sec ul{margin:0;padding-left:18px}.sec li{margin:3px 0}
.body{color:var(--text)}
.dock{margin-top:16px;border-top:1px solid var(--line);padding-top:12px}
.ask{display:inline-block;background:linear-gradient(90deg,var(--purple),var(--pink));color:#fff;border:0;border-radius:6px;padding:7px 12px;font:inherit;font-size:13px;cursor:pointer}
pre.tapir{margin:0;color:var(--mint);font-size:10px;line-height:1.05}
</style></head><body><div class="wrap">
<header>
<pre class="tapir"> ▄█▓▓█▄ ∩
█▓( ◕ ◕ )▓█──┘</pre>
<span class="brand">◆ <b>tapir</b> · watch less, know more</span>
<span class="tip">261 in queue</span>
</header>
<p class="note">Tapir fetches captions slowly on purpose, to respect YouTube's limits — new summaries land gradually.</p>
<div class="card ready"><div class="title">How the Attention Economy Rewires Your Brain</div>
<div class="meta">youtube · 2026-06-11 · 18 min <span class="chip">ready</span></div>
<div class="preview">A tour of the incentive loops behind infinite feeds and three concrete ways to claw back focus…</div></div>
<article class="card exp ready">
<div class="head"><div class="title">Postgres 18 — What's Actually New</div><a class="collapse" href="#">collapse ↑</a></div>
<div class="meta">youtube · 2026-06-10 · 42 min · phi4-mini</div>
<div class="sec"><h2>Takeaways</h2><ul>
<li>Async I/O cuts cold-cache read latency materially on NVMe.</li>
<li>Skip-scan makes more multicolumn indexes usable without rewrites.</li>
<li>Upgrade path is smooth; test the new planner stats first.</li></ul></div>
<div class="sec"><h2>Highlights</h2><ul>
<li>Async I/O subsystem (effective_io_concurrency now matters more).</li>
<li>B-tree skip scan for leading-column gaps.</li>
<li>Better partition-wise joins.</li></ul></div>
<div class="sec"><h2>Summary</h2><p class="body">Postgres 18 is an incremental but meaningful release: the headline is the new asynchronous I/O path, with skip-scan and planner improvements close behind. For most homelabs the upgrade is low-risk and worth it for the read-latency wins.</p></div>
<div class="dock"><button class="ask">Dig deeper — ask about this video →</button></div>
</article>
<div class="card"><div class="title">RAG is dead, right?? — Conference Talk</div>
<div class="meta">youtube · 2026-06-09 · 31 min <span class="chip q">queued</span></div></div>
</div></body></html>
+62
View File
@@ -0,0 +1,62 @@
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>tapir — B · reader + charm accents</title>
<style>
:root{--bg:#faf7f2;--card:#fff;--ink:#1c1b22;--soft:#6a6878;--line:#e7e2d8;--mint:#0bbf8c;--purple:#6a4cf0;--pink:#e0379a}
*{box-sizing:border-box}
body{margin:0;background:var(--bg);color:var(--ink);font:16px/1.6 -apple-system,BlinkMacSystemFont,"Segoe UI",Inter,sans-serif}
.mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace}
.wrap{max-width:680px;margin:0 auto;padding:20px 16px 60px}
header{display:flex;align-items:center;gap:10px;margin-bottom:18px}
.brand{font-weight:800;font-size:18px;letter-spacing:-.3px}
.brand .dot{color:var(--mint)}
.tag{color:var(--soft);font-size:13px}
.tip{margin-left:auto;color:var(--soft);font-size:12px}
.note{color:var(--soft);font-size:13px;background:#fff;border:1px solid var(--line);border-left:3px solid var(--mint);border-radius:8px;padding:8px 12px;margin:0 0 18px}
.card{background:var(--card);border:1px solid var(--line);border-radius:12px;padding:16px 18px;margin:0 0 14px;box-shadow:0 1px 2px rgba(20,18,40,.04)}
.title{font-size:18px;font-weight:700;letter-spacing:-.2px;margin:0 0 4px;line-height:1.3}
.meta{color:var(--soft);font-size:13px}
.meta .mono{font-size:12.5px}
.chip{display:inline-block;background:rgba(11,191,140,.12);color:var(--mint);border-radius:999px;padding:1px 9px;font-size:12px;font-weight:600;margin-left:6px}
.chip.q{background:rgba(224,55,154,.12);color:var(--pink)}
.preview{color:var(--soft);margin:8px 0 0}
.exp{border-color:#d9d0ee;box-shadow:0 6px 24px rgba(106,76,240,.10)}
.head{display:flex;justify-content:space-between;align-items:baseline;gap:10px}
.collapse{color:var(--purple);font-size:13px;font-weight:600;text-decoration:none;white-space:nowrap}
.sec h2{font-size:12px;text-transform:uppercase;letter-spacing:1.2px;color:var(--purple);margin:18px 0 6px}
.sec ul{margin:0;padding-left:20px}.sec li{margin:5px 0}
.body{line-height:1.7}
.dock{margin-top:18px;border-top:1px solid var(--line);padding-top:14px}
.ask{display:inline-flex;align-items:center;gap:6px;background:var(--ink);color:#fff;border:0;border-radius:999px;padding:9px 16px;font:inherit;font-size:14px;font-weight:600;cursor:pointer}
pre.tapir{margin:0;color:var(--mint);font-size:10px;line-height:1.05}
</style></head><body><div class="wrap">
<header>
<pre class="tapir"> ▄█▓▓█▄ ∩
█▓( ◕ ◕ )▓█──┘</pre>
<span class="brand">tapir<span class="dot">.</span></span><span class="tag">watch less, know more</span>
<span class="tip mono">261 in queue</span>
</header>
<p class="note">Tapir fetches captions slowly on purpose, to respect YouTube's limits — new summaries land gradually.</p>
<div class="card"><div class="title">How the Attention Economy Rewires Your Brain</div>
<div class="meta mono">youtube · 2026-06-11 · 18 min <span class="chip">ready</span></div>
<div class="preview">A tour of the incentive loops behind infinite feeds and three concrete ways to claw back focus…</div></div>
<article class="card exp">
<div class="head"><div class="title">Postgres 18 — What's Actually New</div><a class="collapse" href="#">collapse ↑</a></div>
<div class="meta mono">youtube · 2026-06-10 · 42 min · phi4-mini</div>
<div class="sec"><h2>Takeaways</h2><ul>
<li>Async I/O cuts cold-cache read latency materially on NVMe.</li>
<li>Skip-scan makes more multicolumn indexes usable without rewrites.</li>
<li>Upgrade path is smooth; test the new planner stats first.</li></ul></div>
<div class="sec"><h2>Highlights</h2><ul>
<li>Async I/O subsystem (effective_io_concurrency now matters more).</li>
<li>B-tree skip scan for leading-column gaps.</li>
<li>Better partition-wise joins.</li></ul></div>
<div class="sec"><h2>Summary</h2><p class="body">Postgres 18 is an incremental but meaningful release: the headline is the new asynchronous I/O path, with skip-scan and planner improvements close behind. For most homelabs the upgrade is low-risk and worth it for the read-latency wins.</p></div>
<div class="dock"><button class="ask">✦ Ask about this video</button></div>
</article>
<div class="card"><div class="title">RAG is dead, right?? — Conference Talk</div>
<div class="meta mono">youtube · 2026-06-09 · 31 min <span class="chip q">queued</span></div></div>
</div></body></html>
+65
View File
@@ -0,0 +1,65 @@
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>tapir — C · cozy terminal</title>
<style>
:root{--bg:#16131d;--card:#1e1a28;--ink:#ece7f5;--soft:#9a92b4;--line:#322b44;--mint:#2ee6b6;--purple:#9d7bff;--pink:#ff6bdb;--cream:#f3ead8}
*{box-sizing:border-box}
body{margin:0;background:radial-gradient(1200px 600px at 70% -10%,#231b33,transparent),var(--bg);color:var(--ink);font:15px/1.6 -apple-system,BlinkMacSystemFont,"Segoe UI",Inter,sans-serif}
.mono{font-family:ui-monospace,SFMono-Regular,Menlo,monospace}
.wrap{max-width:700px;margin:0 auto;padding:20px 16px 60px}
header{display:flex;align-items:center;gap:12px;margin-bottom:18px}
.brand{font-weight:800;font-size:18px}.brand .c{color:var(--mint)}
.tag{color:var(--soft);font-size:13px}
.tip{margin-left:auto;color:var(--soft);font-size:12px}
.note{color:var(--soft);font-size:13px;background:#1b1726;border:1px solid var(--line);border-radius:10px;padding:9px 12px;margin:0 0 18px}
.note b{color:var(--mint);font-weight:600}
.card{position:relative;background:var(--card);border:1px solid var(--line);border-radius:12px;padding:14px 16px 14px 18px;margin:0 0 13px;overflow:hidden}
.card::before{content:"";position:absolute;left:0;top:0;bottom:0;width:4px;background:var(--purple)}
.card.ready::before{background:linear-gradient(var(--mint),var(--purple))}
.title{font-size:16px;font-weight:700;margin:0 0 4px;color:var(--cream)}
.meta{color:var(--soft);font-size:12.5px}
.chip{display:inline-block;border-radius:999px;padding:1px 9px;font-size:11px;font-weight:600;margin-left:6px;background:rgba(46,230,182,.14);color:var(--mint)}
.chip.q{background:rgba(157,123,255,.18);color:var(--purple)}
.preview{color:var(--soft);margin:7px 0 0;font-size:14px}
.exp{border-color:#473a63;box-shadow:0 10px 40px rgba(0,0,0,.35)}
.head{display:flex;justify-content:space-between;align-items:baseline;gap:10px}
.collapse{color:var(--mint);font-size:12px;text-decoration:none;border:1px solid var(--line);border-radius:6px;padding:2px 8px;white-space:nowrap}
.sec h2{font-size:11px;text-transform:uppercase;letter-spacing:1.3px;color:var(--mint);margin:18px 0 7px;display:flex;align-items:center;gap:8px}
.sec h2::after{content:"";flex:1;height:1px;background:var(--line)}
.sec ul{margin:0;padding-left:18px}.sec li{margin:5px 0}
.body{line-height:1.7;color:var(--ink)}
.dock{margin-top:18px;border-top:1px dashed var(--line);padding-top:14px;display:flex;align-items:center;gap:10px}
.ask{display:inline-flex;align-items:center;gap:7px;background:linear-gradient(90deg,var(--purple),var(--mint));color:#10101a;border:0;border-radius:999px;padding:9px 16px;font:inherit;font-weight:700;font-size:14px;cursor:pointer}
.dockhint{color:var(--soft);font-size:12px}
pre.tapir{margin:0;color:var(--mint);font-size:11px;line-height:1.05}
</style></head><body><div class="wrap">
<header>
<pre class="tapir"> ▄█▓▓█▄ ∩
█▓( ◕ ◕ )▓█──┘</pre>
<span class="brand">tapir<span class="c">_</span></span><span class="tag">watch less, know more</span>
<span class="tip mono">261 in queue</span>
</header>
<p class="note">Tapir fetches captions slowly on purpose, to respect YouTube's limits — <b>new summaries land gradually</b>.</p>
<div class="card ready"><div class="title">How the Attention Economy Rewires Your Brain</div>
<div class="meta mono">youtube · 2026-06-11 · 18 min <span class="chip">ready</span></div>
<div class="preview">A tour of the incentive loops behind infinite feeds and three concrete ways to claw back focus…</div></div>
<article class="card exp ready">
<div class="head"><div class="title">Postgres 18 — What's Actually New</div><a class="collapse" href="#">collapse ↑</a></div>
<div class="meta mono">youtube · 2026-06-10 · 42 min · phi4-mini</div>
<div class="sec"><h2>Takeaways</h2><ul>
<li>Async I/O cuts cold-cache read latency materially on NVMe.</li>
<li>Skip-scan makes more multicolumn indexes usable without rewrites.</li>
<li>Upgrade path is smooth; test the new planner stats first.</li></ul></div>
<div class="sec"><h2>Highlights</h2><ul>
<li>Async I/O subsystem (effective_io_concurrency now matters more).</li>
<li>B-tree skip scan for leading-column gaps.</li>
<li>Better partition-wise joins.</li></ul></div>
<div class="sec"><h2>Summary</h2><p class="body">Postgres 18 is an incremental but meaningful release: the headline is the new asynchronous I/O path, with skip-scan and planner improvements close behind. For most homelabs the upgrade is low-risk and worth it for the read-latency wins.</p></div>
<div class="dock"><button class="ask">◆ Ask about this video</button><span class="dockhint mono">answers only from this video's transcript</span></div>
</article>
<div class="card"><div class="title">RAG is dead, right?? — Conference Talk</div>
<div class="meta mono">youtube · 2026-06-09 · 31 min <span class="chip q">queued</span></div></div>
</div></body></html>
+128
View File
@@ -0,0 +1,128 @@
# Spec — Onboarding "wow" burst: better picks, stronger model
**Repo:** tapir · **Size:** medium · **Solo session** (not a swarm).
> **Status: built (v0.25.0, ADR-028).** This supersedes the original investigate-first brief
> (committed as the prior version of this file): Phase 1 was run against the live pilot DB and its
> findings are folded into "Why this exists" below; Phase 2 was built as described here. The one
> brief lever NOT built — the honest "the rest fill in over the coming days" framing copy — is
> listed under *Explicitly NOT in this slice*.
**Why this exists.** A new user's first session decides whether they return (the Stage-0 gate,
VISION.md). On connect, Tapir fires a capped burst (≤`TAPIR_ONBOARD_SUMMARIZE_COUNT`, default 3)
that summarizes the user's newest unsummarized videos so the feed isn't empty (the burst itself
works — wired in `cmd/tapir/discovery.go` → `cmd/tapir/main.go` `onboard`). A Phase-1
investigation of the live pilot DB found the burst *fires* but delivers a **weak first
impression** for two concrete reasons, and ruled out a third idea:
1. **Picks are junk.** Selection is pure newest-first (`videos.NewestUnsummarizedVideoIDs`,
`ORDER BY published_at DESC`) with **zero quality signal**. Pilot user "Jonte"'s live burst-3
were a stock-ticker **livestream** + two regional news clips — the newest, not the best.
2. **Weakest model on the first impression.** All of Jonte's summaries ran on
`koala/phi4-mini` (the documented weak link — ADR-022 was born from its failures). The
stronger, brain-validated `iguana/gemma4-26b` was never used for the burst.
3. **Cached-first is empty at pilot scale — REJECTED.** The idea (summarize already-cached
transcripts instantly, zero fetch) dies on the numbers: only **11 videos** overlap between the
two pilot users (~3% of each library), **0** cached-and-unsummarized, and a new user's
newest-20 unsummarized are **20/20 NOT cached** — newest-first and cached-first are
structurally incompatible (fresh uploads are exactly what nobody has fetched yet). Not built.
This is a **curation/latency problem for ~3 videos, NOT a throughput/429 problem** — fetching 3
captions is nowhere near the rate limit. Nothing here fetches harder or pressures the rate gate;
it picks the right few videos and runs a better model on them.
Read `CLAUDE.md`, `DECISIONS.md` (esp. ADR-014, ADR-018, ADR-020, ADR-021, ADR-022, ADR-023,
and the new **ADR-028**), and `VISION.md` (the Stage-0 gate) first. TBD — commit directly to
`main`, one logical change per commit, conventional commits, `task check` green before each
commit, `templ generate` if any view changes (none expected).
## Decisions already made (do not reopen)
- **Not a throughput change.** The caption rate gate (ADR-014) is untouched — same pacing, same
priority lane (ADR-026). This slice changes *which* ≤3 videos the burst spends its fetches on
and *which model* summarizes them, never how fast or how many.
- **Cached-first is dropped** (ADR-028, the 3% overlap). The engine's existing read-stored-first
(ADR-021, `resolveTranscript`) stays — it already gives a free instant summary on the rare
cache hit, transparently. We do not *select* for cache hits.
- **has-captions is not a pre-fetch signal.** It is only knowable after a gate fetch (or a cache
hit, ~0 for new videos). Selection can only *avoid known-junk* (Shorts/live/over-long) — it
cannot *guarantee* captions. The spec is honest about this: better odds, not a promise.
- **No credentialed caption fetch** (ADR-010/ADR-026 dead end). **No client extension.**
## 1. Persist `duration_s` at discovery (the enabling change)
The `videos.duration_s` column exists (migration 001) but is **never written** — ADR-023's
`filterLowValue` (`internal/adapters/youtube/youtube.go`) already fetches each candidate's
duration via the cheap quota `videos.list` call, uses it to drop Shorts/live, then **discards
it**. Stop discarding:
- Add `DurationSeconds int` to `domain.Video`.
- In `filterLowValue`, set `DurationSeconds` on each kept video from the `videos.list` `meta`.
- `UpsertVideo` writes `duration_s`, **COALESCE-preserving** a known value (never overwrite a
real duration with 0/unknown), mirroring the `channel_title` backfill stance (migration 014).
- No new migration — the column is already there.
Consequence: a fresh user's connect-triggered discovery pass runs **before** the onboard burst
(`Enqueue`: `run()` then `onboard()`), so duration is populated for the burst's candidates at
connect. Existing rows backfill on their next discovery pass; until then their `duration_s` is
NULL and treated as "unknown" (§2).
## 2. Junk-avoiding burst selection
New store method, RLS-scoped via `withUser`:
```
OnboardBurstVideoIDs(ctx, userID string, limit, minSeconds, maxSeconds int) ([]string, error)
```
- Same base as the old `NewestUnsummarizedVideoIDs`: the user's videos with no summary yet,
`ORDER BY published_at DESC NULLS LAST, seen_at DESC`, `LIMIT limit`.
- **Exclude known-junk**: a row is dropped only when `duration_s IS NOT NULL` **and**
(`duration_s < minSeconds` OR `duration_s > maxSeconds`). A NULL duration is **unknown** — kept
(degrade-open: never starve the burst because metadata is missing), but ordered *after* rows
with a known-good duration so a freshly-enriched good pick wins when both exist.
- `minSeconds` reuses `TAPIR_MIN_VIDEO_SECONDS` (default 60 — the Shorts floor, ADR-023).
`maxSeconds` is new: `TAPIR_ONBOARD_MAX_VIDEO_SECONDS` (default 14400 = 4h) — drops the
multi-hour livestream VODs that pass the live filter once ended.
- `minSeconds<=0` and `maxSeconds<=0` each disable that bound (so `0/0` == the old
newest-first behaviour, the reversibility lever).
- The burst switches to this method; `NewestUnsummarizedVideoIDs` is removed (fully superseded —
`OnboardBurstVideoIDs(., 0, 0)` is identical pure-newest behaviour).
## 3. Stronger model for the burst
The burst summarizes only ≤3 videos, so a slower, stronger model is affordable exactly here.
- New config `TAPIR_ONBOARD_SUMMARIZER_MODEL` (default `iguana/gemma4-26b` — the brain-validated
homelab general-purpose model, already the ADR-022 fallback).
- Build a **burst-specific summarizer chain** that puts the onboard model **first**, then the
standard chain (primary → local fallback → cloud) as resilience, deduped. Wrap it in a
burst-specific `engineProcessor` reusing the same store/transcript-cache/sink — a pure wiring
choice, engine and ports unchanged (Clean Architecture, ADR-003).
- The `onboard` closure uses the burst processor instead of `app.Processor`.
- **Collapse cleanly**: when `OnboardSummarizerModel` is empty or equals `SummarizerModel`, the
onboard path reuses `app.Processor` (no separate chain) — the reversibility lever.
- Local-first preserved: the onboard model is a local alias; the cloud endpoint stays last in the
chain, so a client/NDA deployment with `TAPIR_CLOUD_FALLBACK_MODEL=""` keeps burst content
local too.
## 4. Behaviour spec + docs
- Add scenarios to `docs/use-cases/connect_account.feature` (the connect → burst flow): burst
skips a too-long/live video in favour of a reasonable-length one; burst summarizes with the
stronger model first. Map them in `scenarioCoverage` so `TestScenarioCoverage` stays green.
- Update `docs/architecture/architecture.md` (the onboarding-burst section) to describe the
junk-avoiding selection + the burst model override.
- ADR-028 in `DECISIONS.md` records the rationale (incl. the rejected cached-first lever).
## Success criteria
- `task check` green (fmt, vet, lint, `go test -p 1 ./...`).
- A unit test proves `OnboardBurstVideoIDs` drops a known too-long / sub-min video and keeps a
good one, newest-first, RLS-scoped, unsummarized-only.
- A test proves discovery persists `duration_s` and does not clobber it on re-upsert.
- A test proves the burst chain leads with the onboard model (then the standard chain).
- Config defaults + bounds tested (`OnboardMaxVideoSeconds`, `OnboardSummarizerModel`).
- No change to the rate gate, fetch pacing, or burst cap. `0/0` + empty model == prior behaviour.
## Explicitly NOT in this slice
- Cached-first selection (rejected, ADR-028).
- Any caption-availability *guarantee* (impossible pre-fetch).
- **Honest "taster" framing copy** ("summaries of a few of your videos to get you started — the
rest fill in over the coming days"). A good lever from the original brief, but it's a UI/copy
change with no backend dependency; deferred to a UI pass, tracked as an issue.
- Backfilling `duration_s` for existing rows via a migration (it backfills lazily on discovery).
- Return-nudges / digests (ADR-020: poisons the unprompted-return signal).
- Raising fetch throughput, multi-IP, or Whisper (out of scope; the gate is deliberate).
+1
View File
@@ -179,4 +179,5 @@ distinguishable.
| **"Summarize now" foreground path** | Unified quiet nudge button on actionable non-summarized cards. Five explicit card states — (1) summarized: chip + no button; (2) no captions (`transcript_status = 'none'`): "No transcript available", no button; (3) queued: "Queued" chip, no button; (4) rate-limited: "Fetching soon…" + "Summarize now" → `POST /v/{id}/retry-now` (clears `rate_limited_at`, triggers engine); (5) pending: "Not summarized" + "Summarize now" → `POST /v/{id}/summarize` (queues + triggers engine). One verb, one style (`.btn-quiet`); backend difference invisible to user. Both handlers call `ProcessVideo` through `globalFetchGate`. Rate gate respected, not bypassed — this is onboarding prioritisation. | Fast onboarding value; honest dead-end for no-captions videos (no button that fails). | `internal/web/handlers.go` (`handleRetryNow`, `handleRequestSummarize`); `internal/web/views.templ` (`VideoCard`) |
| **Pipeline stats bar** | A one-line status bar above the video list: `N summarized · M fetching soon · K no captions`. Computed from the unfiltered row set; hidden when all videos are summarized. Gives the user a clear read on pipeline state without any interaction. | Replaces the "why is nothing happening?" confusion when most videos are pending or rate-limited. | `internal/web/view.go` (`PipelineStats`, `pipelineStats`) |
| **Unavailable channels (account page)** | The `/account` page shows a "Unavailable channels" section when any channels returned HTTP 404 on the last discovery pass. Lists channel name, an "unavailable" badge, and the first-seen date. Data sourced from the `channel_errors` table (migration 013). | Surfaces silent failures so users know why some subscribed channels produce no new videos. | migration 013; `internal/web/account.go`; `internal/adapters/youtube/youtube.go` (`domain.ErrChannelUnavailable`) |
| **Visual refresh — charm-reader theme + light/dark toggle (ADR-032)** | One layout in two palettes expressed as CSS custom properties: a warm "reader" light theme (sketch B) and a "cozy terminal" dark theme (sketch C). Palette is chosen in cascade order — `:root` light default, an OS-preference dark block scoped to `:root:not([data-theme])` so it only applies absent an explicit choice, and `:root[data-theme="dark"\|"light"]` set by a header toggle that outranks the media query and persists in `localStorage` (guarded; degrades to OS default). An init script in `<head>` applies the stored choice before paint (no flash). Charm touches: monospace meta lines, accent uppercase section dividers with a trailing rule, pill buttons, a lifted/accent-edged expanded card. Error/danger shades became `--err-*` tokens so they follow the theme without per-block dark overrides. Sketches kept as the design record under `docs/sketches/`. | The UI read "flat and boring"; the charm/TUI aesthetic makes it distinctive and gives a real light/dark choice rather than OS-only. | ADR-032; `docs/use-cases/visual_theme.feature`; `internal/web/visual_theme_test.go`; `internal/web/view.go` (`stylesheet`, `themeScript`), `internal/web/views.templ` (Layout/PublicLayout) |
| **Recency window + sparse-state honesty (ADR-020)** | Supersedes the copy/sort in the rows above. Auto-summarize is bounded to videos published within `TAPIR_AUTO_SUMMARIZE_WINDOW` (~7d); older un-summarized videos collapse behind a single "Show N older videos — summarize on demand" disclosure, and caption-less videos collapse to a one-line count (not N cards). List order is now `summarized-first, published_at DESC NULLS LAST`. Copy reframed for honest scarcity: pipeline bar reads "N ready · M in queue · K no captions" (no "fetching soon"); a gradual-fill note explains the rate limit; the nudge verb is "Summarize" (not "Summarize now"); the queued card says "summarizing shortly"; the empty-connected state drops the impossible `tapir run` instruction. Detail leads with Takeaways. Filters slimmed (no date pickers; hidden when empty); watched/skipped segmented; back link on detail; empty terms checkbox removed. | Make the sparse reality legible and honest instead of implying abundance/imminence; bound auto load so the back-catalogue doesn't re-drive the caption gate. Never fetch harder — scarcity is surfaced, not engineered around. | ADR-020; `2384c47`, `3df0459`, `40b703e`, `a1a5217`, `4a0a56e`, `9bf1c31`, `980638d`, `12fb031`, `f775441`, `51aa5d9` |
+6
View File
@@ -14,6 +14,12 @@ Feature: Chat with a video's stored transcript
Scenario: A summary view offers a deeper-dive into the video
When I view the summary
Then I see a "dig deeper" affordance that opens a chat about this video
And it opens the chat in place, below the summary, without leaving the page
Scenario: The summary and the chat are on one page
When I open the chat
Then the summary stays visible alongside the chat
And I can read the summary while I ask questions
Scenario: Ask a question answered from the stored transcript
When I ask a question in the chat
+9 -2
View File
@@ -24,12 +24,19 @@ Feature: Connect and manage video accounts
Then a discovery pass for my account is triggered right away
And I do not have to wait for the next scheduled pass to see my videos
Scenario: Connecting summarizes my newest videos right away
Scenario: Connecting summarizes my best recent videos right away
Given I have no connected video accounts
When I connect my YouTube account
Then up to the onboarding cap of my newest videos are summarized through the rate gate
Then up to the onboarding cap of my newest likely-good videos are summarized through the rate gate
And videos whose known duration is too short or too long are skipped
And the rest are left to the scheduled recency-bounded pass
Scenario: The onboarding burst summarizes with a stronger model
Given I have no connected video accounts
When I connect my YouTube account
Then the burst summarizes with the stronger onboarding model first
And the standard summarizer chain still follows as a fallback
Scenario: Tokens are never stored in the clear
When I connect any video account
Then no OAuth token value is stored in the database
+37
View File
@@ -0,0 +1,37 @@
Feature: Inline-expand summary + Q&A in the list (ADR-031, #16)
As a reader skimming my summaries
I want to open a summary and its Q&A in place in the list
So that I get the full read and follow-up without leaving the list (SPA-like, no page hop)
# HTMX inline-expand, no SPA framework (ADR-031). Each scenario maps to a Go test
# in scenario_coverage_test.go (the BDD name-coverage gate).
Scenario: A summarized card expands to the full summary in place
Given a summarized video in my list
When I expand its card
Then the full summary, highlights, and takeaways are returned as an in-place card fragment, not a full page
Scenario: An expanded card collapses back to the compact card
Given an expanded card
When I collapse it
Then the compact card fragment is returned in its place
Scenario: The expanded card offers the Q&A dock
Given chat is enabled
When a summarized card is expanded
Then the expanded card includes the deeper-dive chat affordance for that video
Scenario: Only a summarized card offers expand
Given a discovered but not-yet-summarized card
When the card is rendered
Then it shows its summarize/queue footer and no expand affordance
Scenario: With JS off the card still reaches the full summary
Given a summarized card
When it is rendered
Then its expand affordance carries an href to the detail page as a no-JS fallback
Scenario: The detail page and the expanded card show the same summary
Given a summarized video
When I view it on the detail page and as an expanded card
Then both render the same summary body (one shared fragment, no drift)
+46
View File
@@ -0,0 +1,46 @@
Feature: Observability — timing and metrics for performance and UX (ADR-030, #15)
As the maintainer running Tapir for pilot users
I want timing and Prometheus metrics for the activities that drive performance and UX
So that I can see latency, model behaviour, and usage — and feed the Stage-0 eval gate
# AI metrics are the priority (ADR-030 R3). Each scenario maps to a Go test in
# test/acceptance/scenario_coverage_test.go (the BDD name-coverage gate).
Scenario: Summarization latency is recorded per endpoint
Given the summarizer runs a transcript through its endpoint chain
When an endpoint returns a parseable summary
Then the summarize latency is recorded with the model, outcome "success", and whether it was a fallback
Scenario: A failing summarizer endpoint records its failure outcome
Given the summarizer runs a transcript through its endpoint chain
When an endpoint errors or returns unparseable output
Then the summarize latency is recorded with outcome "error" or "parse_error" before the chain advances
Scenario: Caption fetch latency is recorded by outcome
Given a caption fetch is attempted for a video
When it resolves to captions, no captions, or a rate limit
Then the caption-fetch latency is recorded labelled by that outcome
Scenario: LLM token usage is recorded from the completion
Given an LLM completion returns a usage block with prompt and completion tokens
When the client finishes the call
Then the prompt and completion tokens are recorded for that model
Scenario: Q&A answer latency is recorded
Given a user asks a question about a video
When the answer is produced from the stored transcript
Then the chat answer latency is recorded for the answering model
Scenario: HTTP requests are counted by route, method, and status
Given the metrics HTTP middleware wraps the app
When a request is served against a registered route
Then it is counted and timed under the bounded route pattern, not the raw path
Scenario: A successful login is counted
Given a user completes the OIDC callback and a session is established
Then the login counter is incremented
Scenario: The metrics endpoint is not on the public app port
Given the service is running
When the public app mux is inspected
Then it exposes no /metrics route — metrics are served on the dedicated metrics port only
+29
View File
@@ -0,0 +1,29 @@
Feature: Visual refresh — one charm-reader layout, light + dark themes (ADR-032, #17)
As a reader who likes a TUI/charm aesthetic
I want a fresh look with a light and a dark theme
So that the app feels distinctive and reads well in either mode
# Direction B (light) + C (dark) are one layout in two palettes (CSS variables),
# plus a persisted toggle and an OS-preference default. Colours are reviewed via
# the mockups in docs/sketches/, not unit-tested; these scenarios cover the
# testable structure. Each maps to a Go test in scenario_coverage_test.go.
Scenario: Light and dark themes share one layout via CSS variables
Given the app stylesheet
When it is rendered
Then it defines a light palette on the root and a dark palette under data-theme="dark", with no duplicate markup
Scenario: Without a stored choice the theme follows the OS preference
Given a visitor with no saved theme
When the page loads
Then a prefers-color-scheme dark block applies the dark palette automatically
Scenario: A persisted toggle switches light and dark
Given any page
When it is rendered
Then it includes a theme-toggle control and a small script that flips data-theme and persists the choice
Scenario: The expanded card embeds the video player
Given a summarized video with a valid provider id
When its card is expanded
Then the expanded card includes the embedded video player
+12 -2
View File
@@ -1,4 +1,4 @@
module gitea.d-ma.be/mathias/tapir
module git.d-ma.be/mathias/tapir
go 1.26.1
@@ -9,23 +9,33 @@ require (
github.com/go-jose/go-jose/v4 v4.1.4
github.com/golang-migrate/migrate/v4 v4.19.1
github.com/jackc/pgx/v5 v5.9.2
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/client_model v0.6.2
github.com/stretchr/testify v1.11.1
golang.org/x/crypto v0.45.0
golang.org/x/oauth2 v0.36.0
golang.org/x/time v0.15.0
)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.16.1 // indirect
github.com/rogpeppe/go-internal v1.15.0 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/text v0.31.0 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
+26 -6
View File
@@ -4,6 +4,10 @@ github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERo
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
@@ -37,8 +41,8 @@ github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-migrate/migrate/v4 v4.19.1 h1:OCyb44lFuQfYXYLx1SCxPZQGU7mcaZ7gH9yH4jSFbBA=
github.com/golang-migrate/migrate/v4 v4.19.1/go.mod h1:CTcgfjxhaUtsLipnLoQRWCrjYXycRz/g5+RWDuYgPrE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa h1:s+4MhCQ6YrzisK6hFJUX53drDT4UsSW3DEhKn0ifuHw=
github.com/jackc/pgerrcode v0.0.0-20220416144525-469b46aa5efa/go.mod h1:a/s9Lp5W7n/DD0VrVoyJ00FbP2ytTPDVOivvn2bMlds=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
@@ -49,10 +53,14 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw=
github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
@@ -61,6 +69,8 @@ github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
@@ -70,6 +80,14 @@ github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs=
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -91,8 +109,8 @@ go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mx
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
@@ -103,6 +121,8 @@ golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+8
View File
@@ -13,8 +13,12 @@ package chat
import (
"context"
"fmt"
"log/slog"
"strings"
"time"
"unicode/utf8"
"git.d-ma.be/mathias/tapir/internal/metrics"
)
// Completer is the minimal LLM chat surface the Service needs. *llm.Client
@@ -113,10 +117,14 @@ func (s *Service) Answer(ctx context.Context, req Request) (Reply, error) {
system := buildSystem(transcript, truncated)
user := buildUser(req.History, req.Question)
start := time.Now()
out, err := s.newClient(model).Complete(ctx, system, user)
if err != nil {
return Reply{}, fmt.Errorf("chat: %s: %w", model, err)
}
dur := time.Since(start)
metrics.ObserveChat(model, dur)
slog.Default().Info("chat answer", "model", model, "elapsed_ms", dur.Milliseconds())
answer := strings.TrimSpace(out)
if answer == "" {
return Reply{}, fmt.Errorf("chat: %s returned an empty answer", model)
+16
View File
@@ -32,6 +32,7 @@ type Client struct {
model string
maxTokens int
httpClient *http.Client
usageHook func(model string, prompt, completion int)
}
// Option configures a Client at construction. Variadic so the existing 4-arg
@@ -50,6 +51,14 @@ func WithMaxTokens(n int) Option {
}
}
// WithUsageHook registers a callback fired after a successful completion with the
// model and the prompt/completion token counts from the response usage block. It
// keeps this copied, stdlib-only package (ADR-004) decoupled from metrics: the
// caller wires it to internal/metrics, the client imports nothing. nil is ignored.
func WithUsageHook(fn func(model string, prompt, completion int)) Option {
return func(c *Client) { c.usageHook = fn }
}
// New constructs a Client.
func New(baseURL, apiKey, model string, timeout time.Duration, opts ...Option) *Client {
c := &Client{
@@ -81,6 +90,10 @@ type chatResponse struct {
Choices []struct {
Message message `json:"message"`
} `json:"choices"`
Usage struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
} `json:"usage"`
}
// Complete sends a system + user message and returns the assistant's reply.
@@ -152,5 +165,8 @@ func (c *Client) Complete(ctx context.Context, system, user string) (string, err
if len(cr.Choices) == 0 {
return "", fmt.Errorf("LLM returned no choices")
}
if c.usageHook != nil {
c.usageHook(c.model, cr.Usage.PromptTokens, cr.Usage.CompletionTokens)
}
return cr.Choices[0].Message.Content, nil
}
+24
View File
@@ -85,6 +85,30 @@ func TestClient_WithMaxTokens(t *testing.T) {
}
}
// TestClient_UsageHookRecordsTokens: the usage hook fires with the model and the
// prompt/completion token counts parsed from the response usage block.
func TestClient_UsageHookRecordsTokens(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{
"choices": []map[string]any{{"message": map[string]any{"content": "ok"}}},
"usage": map[string]any{"prompt_tokens": 123, "completion_tokens": 45},
})
}))
defer srv.Close()
var gotModel string
var gotPrompt, gotCompletion int
c := New(srv.URL, "", "test-model", 10*time.Second, WithUsageHook(func(model string, p, comp int) {
gotModel, gotPrompt, gotCompletion = model, p, comp
}))
if _, err := c.Complete(context.Background(), "sys", "user"); err != nil {
t.Fatalf("Complete: %v", err)
}
if gotModel != "test-model" || gotPrompt != 123 || gotCompletion != 45 {
t.Errorf("usage hook got (%q, %d, %d), want (test-model, 123, 45)", gotModel, gotPrompt, gotCompletion)
}
}
func TestClient_ReturnsErrorOnNon200(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "overloaded", http.StatusServiceUnavailable)
+1 -1
View File
@@ -18,7 +18,7 @@ import (
"path/filepath"
"sync"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// FileStore is a SecretStore backed by a single 0600 JSON file mapping opaque
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"path/filepath"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/adapters/secrets"
"git.d-ma.be/mathias/tapir/internal/adapters/secrets"
)
func TestPutThenGet(t *testing.T) {
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// seedUserRow inserts a bare users row (FK target for a connection) as the
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
const (
+48 -37
View File
@@ -3,6 +3,7 @@ package store_test
import (
"context"
"database/sql"
"errors"
"os"
"testing"
@@ -34,6 +35,30 @@ func fileMigrator(t *testing.T) *migrate.Migrate {
return m
}
// headVersion reports the current (HEAD) schema version so a test can restore
// to it after stepping down, without hard-coding what HEAD is. Adding a
// migration on top changes HEAD but no test that uses this needs editing.
func headVersion(t *testing.T, m *migrate.Migrate) uint {
t.Helper()
v, dirty, err := m.Version()
require.NoError(t, err)
require.False(t, dirty, "schema must not be dirty")
return v
}
// migrateTo drives the schema to an exact version *by version number*, not by
// step count. This is the whole point of the migrate-test design: a migration
// added above the target does not shift any count here, so unrelated tests stay
// green (see issue #8). ErrNoChange (already at that version) is not a failure.
func migrateTo(t *testing.T, m *migrate.Migrate, version uint) {
t.Helper()
err := m.Migrate(version)
if errors.Is(err, migrate.ErrNoChange) {
return
}
require.NoError(t, err)
}
// loginEventsExists reports whether the login_events relation is present.
func loginEventsExists(t *testing.T) bool {
t.Helper()
@@ -53,25 +78,15 @@ func TestMigration010LoginEventsUpDown(t *testing.T) {
require.True(t, loginEventsExists(t), "login_events must exist at latest migration")
m := fileMigrator(t)
// 011..016 sit above 010; step them down first so 010 is exercised in isolation.
require.NoError(t, m.Steps(-1), "down 016 drops channel_caption_state, login_events intact")
require.True(t, loginEventsExists(t), "016 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 015 reshapes transcripts, login_events intact")
require.True(t, loginEventsExists(t), "015 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 014 drops channel_title, login_events intact")
require.True(t, loginEventsExists(t), "014 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 013 drops channel_errors, login_events intact")
require.True(t, loginEventsExists(t), "013 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 012 is a no-op, login_events intact")
require.True(t, loginEventsExists(t), "012 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 011 must not touch login_events")
require.True(t, loginEventsExists(t), "011 down leaves login_events intact")
head := headVersion(t, m)
require.NoError(t, m.Steps(-1), "down 010 must drop login_events")
migrateTo(t, m, 9) // just below 010 — everything above steps down
require.False(t, loginEventsExists(t), "login_events must be gone after the down migration")
require.NoError(t, m.Steps(7), "up must recreate 010 then re-apply 011..016")
migrateTo(t, m, 10) // up 010
require.True(t, loginEventsExists(t), "login_events must be restored after the up migration")
migrateTo(t, m, head) // restore to HEAD for sibling tests
}
// autoSummarizeDefault reads the users.auto_summarize column default as text
@@ -93,21 +108,15 @@ func TestMigration011AutoSummarizeDefaultUpDown(t *testing.T) {
require.Equal(t, "true", autoSummarizeDefault(t), "011 sets the default to TRUE")
m := fileMigrator(t)
require.NoError(t, m.Steps(-1), "down 016 drops channel_caption_state")
require.NoError(t, m.Steps(-1), "down 015 reshapes transcripts")
require.NoError(t, m.Steps(-1), "down 014 drops channel_title")
require.NoError(t, m.Steps(-1), "down 013 drops channel_errors")
require.NoError(t, m.Steps(-1), "down 012 is a no-op")
require.NoError(t, m.Steps(-1), "down 011 reverts the column default")
head := headVersion(t, m)
migrateTo(t, m, 10) // just below 011 — reverts the column default
require.Equal(t, "false", autoSummarizeDefault(t), "default is FALSE after the down migration")
require.NoError(t, m.Steps(1), "up 011 re-applies the TRUE default")
migrateTo(t, m, 11) // up 011 re-applies the TRUE default
require.Equal(t, "true", autoSummarizeDefault(t))
require.NoError(t, m.Steps(1), "up 012 runs clean (no FORCE RLS on fresh schema)")
require.NoError(t, m.Steps(1), "up 013 creates channel_errors")
require.NoError(t, m.Steps(1), "up 014 recreates channel_title")
require.NoError(t, m.Steps(1), "up 015 reshapes transcripts to shared")
require.NoError(t, m.Steps(1), "up 016 recreates channel_caption_state (HEAD)")
migrateTo(t, m, head) // restore to HEAD for sibling tests
}
// channelTitleExists reports whether videos.channel_title is present.
@@ -127,17 +136,15 @@ func TestMigration014VideoChannelTitleUpDown(t *testing.T) {
require.True(t, channelTitleExists(t), "channel_title exists at latest migration")
m := fileMigrator(t)
require.NoError(t, m.Steps(-1), "down 016 drops channel_caption_state, channel_title intact")
require.True(t, channelTitleExists(t), "016 down leaves channel_title intact")
require.NoError(t, m.Steps(-1), "down 015 reshapes transcripts, channel_title intact")
require.True(t, channelTitleExists(t), "015 down leaves channel_title intact")
require.NoError(t, m.Steps(-1), "down 014 must drop channel_title")
head := headVersion(t, m)
migrateTo(t, m, 13) // just below 014 — drops channel_title
require.False(t, channelTitleExists(t), "channel_title must be gone after the down migration")
require.NoError(t, m.Steps(1), "up 014 must recreate channel_title")
migrateTo(t, m, 14) // up 014 recreates channel_title
require.True(t, channelTitleExists(t), "channel_title must be restored after the up migration")
require.NoError(t, m.Steps(1), "up 015 restores the shared transcripts shape")
require.NoError(t, m.Steps(1), "up 016 recreates channel_caption_state (HEAD)")
migrateTo(t, m, head) // restore to HEAD for sibling tests
}
// TestMigration012FixAutoSummarizeRLS proves 012 runs cleanly and flips any
@@ -148,7 +155,11 @@ func TestMigration012FixAutoSummarizeRLS(t *testing.T) {
// Round-trip: down 012, then up 012 — must be idempotent.
m := fileMigrator(t)
require.NoError(t, m.Steps(-1), "down 012 must not error")
require.NoError(t, m.Steps(1), "up 012 must re-apply cleanly")
head := headVersion(t, m)
migrateTo(t, m, 11) // down 012 must not error
migrateTo(t, m, 12) // up 012 must re-apply cleanly
require.Equal(t, "true", autoSummarizeDefault(t), "default still TRUE after 012 re-applied")
migrateTo(t, m, head) // restore to HEAD for sibling tests
}
@@ -0,0 +1,3 @@
-- Drops the private uploaded-transcript store and every transcript in it. There
-- is no revert of the data: the original audio was never kept (ADR-033).
DROP TABLE uploaded_transcripts;
@@ -0,0 +1,48 @@
-- Migration 017: uploaded_transcripts — PRIVATE transcripts of audio a user
-- uploaded themselves (ADR-033, tapir#33).
--
-- This is deliberately NOT the shared `transcripts` table (migration 015,
-- ADR-021). That table is non-RLS on the explicit grounds that it holds only
-- public caption content and nothing user-identifying. A recorded meeting is the
-- opposite: private third-party content owned by exactly one user. Reusing
-- `transcripts` — or adding a user_id to it with a partial policy — would put a
-- private row in the one table the isolation test asserts is shared, and would
-- pass CI green. A separate table keeps ADR-021's exception singular.
--
-- Row-level security follows migration 003 exactly: ENABLE + FORCE (the app
-- connects as the table owner, which bypasses RLS unless forced), and a policy
-- keyed off the per-request GUC tapir.current_user_id. With no GUC set the
-- predicate is NULL and the table is deny-all.
--
-- The original audio is never stored (ADR-033): this row is all that is kept.
-- segments holds the timestamped cues; re-transcribing 90 minutes costs ~7
-- minutes, so discarding the timestamps would be irreversible.
CREATE TABLE uploaded_transcripts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'queued'
CHECK (status IN ('queued', 'transcribing', 'done', 'failed')),
content TEXT,
segments JSONB,
endpoint TEXT, -- which backend served it (iguana | berget); a quality fact
audio_seconds DOUBLE PRECISION,
error_kind TEXT, -- typed, never a provider's error string (ADR-033)
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
finished_at TIMESTAMPTZ,
CONSTRAINT uploaded_transcripts_done_has_content
CHECK (status <> 'done' OR content IS NOT NULL),
CONSTRAINT uploaded_transcripts_failed_has_error_kind
CHECK (status <> 'failed' OR error_kind IS NOT NULL)
);
CREATE INDEX idx_uploaded_transcripts_user ON uploaded_transcripts (user_id, created_at DESC);
ALTER TABLE uploaded_transcripts ENABLE ROW LEVEL SECURITY;
ALTER TABLE uploaded_transcripts FORCE ROW LEVEL SECURITY;
CREATE POLICY uploaded_transcripts_isolation ON uploaded_transcripts
FOR ALL
USING (user_id = current_setting('tapir.current_user_id', true)::uuid);
COMMENT ON TABLE uploaded_transcripts IS
'PRIVATE per-user transcripts of user-supplied audio. RLS-scoped (ADR-012). Never merge into the shared transcripts table (ADR-021, ADR-033).';
+1 -1
View File
@@ -8,7 +8,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// seedVideo inserts a videos row whose id matches a summary's video_id, so the
+12 -6
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"sort"
"time"
"github.com/jackc/pgx/v5"
)
@@ -32,7 +33,12 @@ type UserActiveWeeks struct {
// Scope note: the enumeration covers users with a Dex identity (the web users the
// gate is about). A CLI-only user created by the store sink without an identity
// row would not appear — out of scope for this gate.
func (s *Store) ActiveWeeks(ctx context.Context) ([]UserActiveWeeks, error) {
// ActiveWeeks counts each user's distinct active weeks from `since` onward. A zero
// `since` means no lower bound (count all history). The Stage-0 gate baseline is
// set by the caller (the report command) to the date real usage tracking began,
// so pre-launch noise — testing, the period the pilot was blocked — does not count
// toward the return-usage signal (ADR-016).
func (s *Store) ActiveWeeks(ctx context.Context, since time.Time) ([]UserActiveWeeks, error) {
userIDs, err := s.identityUserIDs(ctx)
if err != nil {
return nil, err
@@ -40,7 +46,7 @@ func (s *Store) ActiveWeeks(ctx context.Context) ([]UserActiveWeeks, error) {
out := make([]UserActiveWeeks, 0, len(userIDs))
for _, uid := range userIDs {
row, err := s.activeWeeksFor(ctx, uid)
row, err := s.activeWeeksFor(ctx, uid, since)
if err != nil {
return nil, err
}
@@ -85,18 +91,18 @@ func (s *Store) identityUserIDs(ctx context.Context) ([]string, error) {
// activeWeeksFor counts one user's distinct active weeks (reads UNION acts) and
// reads their display name, RLS-scoped via withUser. The UNION dedups a week that
// has both a login and an action so it counts once.
func (s *Store) activeWeeksFor(ctx context.Context, userID string) (UserActiveWeeks, error) {
func (s *Store) activeWeeksFor(ctx context.Context, userID string, since time.Time) (UserActiveWeeks, error) {
res := UserActiveWeeks{UserID: userID}
if err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
if err := tx.QueryRow(ctx,
`WITH weeks AS (
SELECT date_trunc('week', seen_at) AS wk
FROM login_events WHERE user_id = $1
FROM login_events WHERE user_id = $1 AND seen_at >= $2
UNION
SELECT date_trunc('week', acted_at)
FROM summary_actions WHERE user_id = $1
FROM summary_actions WHERE user_id = $1 AND acted_at >= $2
)
SELECT count(DISTINCT wk) FROM weeks`, userID).Scan(&res.ActiveWeeks); err != nil {
SELECT count(DISTINCT wk) FROM weeks`, userID, since).Scan(&res.ActiveWeeks); err != nil {
return fmt.Errorf("store: count active weeks: %w", err)
}
if err := tx.QueryRow(ctx,
+29 -2
View File
@@ -3,6 +3,7 @@ package store_test
import (
"context"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
@@ -54,7 +55,7 @@ func TestActiveWeeksCountsDistinctWeeksAcrossReadsAndActs(t *testing.T) {
($1, 'vid-2', 'saved', '2026-01-19T18:00:00Z')`, userA)
require.NoError(t, err)
got, err := s.ActiveWeeks(ctx)
got, err := s.ActiveWeeks(ctx, time.Time{}) // zero since = no lower bound
require.NoError(t, err)
require.Len(t, got, 2, "both identity users must appear")
@@ -72,7 +73,33 @@ func TestActiveWeeksEmptyWhenNoUsers(t *testing.T) {
s := newStore(t)
resetDB(t, rawPool(t))
got, err := s.ActiveWeeks(ctx)
got, err := s.ActiveWeeks(ctx, time.Time{})
require.NoError(t, err)
require.Empty(t, got)
}
// TestActiveWeeksExcludesBeforeGateStart proves the baseline cutoff: activity
// before `since` does not count, so pre-launch noise (testing, the pilot's blocked
// period) is excluded from the Stage-0 return-usage gate (ADR-016).
func TestActiveWeeksExcludesBeforeGateStart(t *testing.T) {
ctx := context.Background()
s := newStore(t)
p := rawPool(t)
resetDB(t, p)
seedReportUser(t, p, userA, "subject-a", "Ada")
// One read well before the baseline, two reads in distinct weeks after it.
_, err := p.Exec(ctx,
`INSERT INTO login_events (user_id, seen_at) VALUES
($1, '2026-05-01T09:00:00Z'),
($1, '2026-06-12T09:00:00Z'),
($1, '2026-06-19T09:00:00Z')`, userA)
require.NoError(t, err)
since := time.Date(2026, 6, 11, 0, 0, 0, 0, time.UTC)
got, err := s.ActiveWeeks(ctx, since)
require.NoError(t, err)
require.Len(t, got, 1)
require.Equal(t, 2, got[0].ActiveWeeks, "only the two post-baseline weeks count; the May read is excluded")
}
+1 -1
View File
@@ -24,7 +24,7 @@ import (
_ "github.com/jackc/pgx/v5/stdlib" // register the "pgx" database/sql driver for migrate
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
//go:embed migrations/*.sql
+3 -3
View File
@@ -11,9 +11,9 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// Static check: Store satisfies the Sink port.
@@ -8,7 +8,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// seedBareVideo inserts a videos row with no summary, so the all-videos read and
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"github.com/jackc/pgx/v5"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// GetTranscript returns the shared, stored transcript for a video keyed by the
@@ -6,7 +6,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
func TestSetTranscriptStatus_RoundTrip(t *testing.T) {
+3 -3
View File
@@ -6,9 +6,9 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// Static check: Store satisfies the shared TranscriptStore port (ADR-021).
@@ -0,0 +1,199 @@
package store
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// ErrUploadNotFound is returned when an uploaded transcript does not exist OR is
// not the caller's. The two are deliberately indistinguishable: a non-owner learns
// nothing about whether someone else's upload exists (ADR-033).
var ErrUploadNotFound = errors.New("store: uploaded transcript not found")
// Uploaded transcripts are PRIVATE per-user content, stored in their own
// RLS-scoped table (migration 017) — never in the shared `transcripts` (ADR-021).
// Every method goes through withUser, so row-level security confines it to the
// caller's rows; each statement ALSO names user_id in its predicate, so the
// isolation does not rest on a single mechanism and stays testable under the
// superuser connection the unit tests use.
// CreateUpload records a new upload in the queued state and returns its id.
func (s *Store) CreateUpload(ctx context.Context, userID, title string) (string, error) {
var id string
if err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
return tx.QueryRow(ctx,
`INSERT INTO uploaded_transcripts (user_id, title) VALUES ($1, $2) RETURNING id`,
userID, title).Scan(&id)
}); err != nil {
return "", fmt.Errorf("store: create upload: %w", err)
}
return id, nil
}
// StartTranscribing moves an upload to transcribing.
func (s *Store) StartTranscribing(ctx context.Context, userID, id string) error {
return s.updateUpload(ctx, userID, id, "start transcribing",
`UPDATE uploaded_transcripts SET status = 'transcribing'
WHERE id = $1 AND user_id = $2`)
}
// CompleteUpload stores the finished transcription and moves the upload to done.
func (s *Store) CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error {
segs, err := json.Marshal(t.Segments)
if err != nil {
return fmt.Errorf("store: complete upload: marshal segments: %w", err)
}
return s.updateUpload(ctx, userID, id, "complete upload",
`UPDATE uploaded_transcripts
SET status = 'done', content = $3, segments = $4, endpoint = $5,
audio_seconds = $6, error_kind = NULL, finished_at = NOW()
WHERE id = $1 AND user_id = $2`,
t.Text, segs, t.Endpoint, t.AudioSeconds)
}
// FailUpload moves an upload to failed with a typed reason.
func (s *Store) FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error {
return s.updateUpload(ctx, userID, id, "fail upload",
`UPDATE uploaded_transcripts
SET status = 'failed', error_kind = $3, finished_at = NOW()
WHERE id = $1 AND user_id = $2`,
string(kind))
}
// updateUpload runs one UPDATE keyed on ($1 = id, $2 = userID) plus extra args,
// and maps "no row touched" to ErrUploadNotFound — which is what a write to another
// user's upload looks like under RLS (the row is invisible, not forbidden).
func (s *Store) updateUpload(ctx context.Context, userID, id, op, sql string, args ...any) error {
all := append([]any{id, userID}, args...)
return s.withUser(ctx, userID, func(tx pgx.Tx) error {
ct, err := tx.Exec(ctx, sql, all...)
if err != nil {
return fmt.Errorf("store: %s: %w", op, err)
}
if ct.RowsAffected() == 0 {
return ErrUploadNotFound
}
return nil
})
}
// GetUpload returns one of the caller's uploads, including the transcript text.
func (s *Store) GetUpload(ctx context.Context, userID, id string) (*domain.UploadedTranscript, error) {
var u domain.UploadedTranscript
err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
row := tx.QueryRow(ctx, uploadSelect+` WHERE id = $1 AND user_id = $2`, id, userID)
return scanUpload(row, &u, true)
})
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrUploadNotFound
}
if err != nil {
return nil, fmt.Errorf("store: get upload: %w", err)
}
return &u, nil
}
// ListUploads returns the caller's uploads, newest first, WITHOUT transcript text
// or segments: a list view must not drag every transcript along.
func (s *Store) ListUploads(ctx context.Context, userID string) ([]domain.UploadedTranscript, error) {
var out []domain.UploadedTranscript
err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
rows, err := tx.Query(ctx, uploadSelect+` WHERE user_id = $1 ORDER BY created_at DESC, id`, userID)
if err != nil {
return err
}
defer rows.Close()
for rows.Next() {
var u domain.UploadedTranscript
if err := scanUpload(rows, &u, false); err != nil {
return err
}
out = append(out, u)
}
return rows.Err()
})
if err != nil {
return nil, fmt.Errorf("store: list uploads: %w", err)
}
return out, nil
}
// FailInterruptedUploads is the start-up sweep. The audio of an upload is never
// retained (ADR-033), so a job that was queued or transcribing when the process
// stopped can never resume; leaving it would be a stuck spinner. Every such row,
// for every user, becomes failed/interrupted and the user re-uploads. It
// enumerates users from the un-RLS'd identity map and scopes each pass with
// withUser, exactly as scheduled discovery does (ADR-018). Returns rows failed.
func (s *Store) FailInterruptedUploads(ctx context.Context) (int, error) {
users, err := s.ListAllUsers(ctx)
if err != nil {
return 0, fmt.Errorf("store: fail interrupted uploads: %w", err)
}
total := 0
for _, u := range users {
err := s.withUser(ctx, u.UserID, func(tx pgx.Tx) error {
ct, err := tx.Exec(ctx,
`UPDATE uploaded_transcripts
SET status = 'failed', error_kind = $2, finished_at = NOW()
WHERE user_id = $1 AND status IN ('queued', 'transcribing')`,
u.UserID, string(domain.UploadErrInterrupted))
total += int(ct.RowsAffected())
return err
})
if err != nil {
return total, fmt.Errorf("store: fail interrupted uploads: user %s: %w", u.UserID, err)
}
}
return total, nil
}
const uploadSelect = `SELECT id, user_id, title, status, content, segments, endpoint,
audio_seconds, error_kind, created_at, finished_at FROM uploaded_transcripts`
// scanUpload scans one uploadSelect row. withText controls whether the transcript
// body and segments are kept (GetUpload) or dropped (ListUploads).
func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error {
var (
status, errKind, endpoint, content *string
segments []byte
seconds *float64
finished *time.Time
)
if err := row.Scan(&u.ID, &u.UserID, &u.Title, &status, &content, &segments, &endpoint,
&seconds, &errKind, &u.CreatedAt, &finished); err != nil {
return err
}
u.Status = domain.UploadStatus(deref(status))
u.Endpoint = deref(endpoint)
u.ErrorKind = domain.UploadErrorKind(deref(errKind))
if seconds != nil {
u.AudioSeconds = *seconds
}
if finished != nil {
u.FinishedAt = *finished
}
if !withText {
return nil
}
u.Content = deref(content)
if len(segments) > 0 {
if err := json.Unmarshal(segments, &u.Segments); err != nil {
return fmt.Errorf("unmarshal segments: %w", err)
}
}
return nil
}
func deref(s *string) string {
if s == nil {
return ""
}
return *s
}
@@ -0,0 +1,196 @@
package store_test
import (
"context"
"sort"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
)
// Isolation proof for ADR-033: an uploaded transcript is PRIVATE content owned by
// one user, so it lives in its own RLS-scoped table — never in the shared,
// non-RLS `transcripts` (ADR-021). These tests drive the database directly (raw
// SQL as the non-superuser `app` role) so they observe row-level security itself,
// not any application-layer WHERE clause. The superuser pool is used only to seed
// and to read pg_catalog; it bypasses RLS and so must never be the pool whose
// answers the assertions rest on.
// seedUploadedTranscript inserts one finished upload for userID as the superuser
// (RLS bypassed so both users' rows land regardless of any GUC) and returns its id.
// The text is synthetic — this repo is public, and a real transcript must never
// appear in it (#33).
func seedUploadedTranscript(t *testing.T, super *pgxpool.Pool, userID, text string) string {
t.Helper()
ctx := context.Background()
_, err := super.Exec(ctx, `INSERT INTO users (id) VALUES ($1) ON CONFLICT DO NOTHING`, userID)
require.NoError(t, err)
var id string
require.NoError(t, super.QueryRow(ctx,
`INSERT INTO uploaded_transcripts (user_id, title, status, content)
VALUES ($1, 'synthetic fixture', 'done', $2) RETURNING id`,
userID, text).Scan(&id))
return id
}
func TestUploadedTranscriptsRLS_ScopedReadsSeeOnlyOwnRows(t *testing.T) {
newStore(t)
super := rawPool(t)
resetDB(t, super)
seedUploadedTranscript(t, super, userA, "text owned by A")
seedUploadedTranscript(t, super, userB, "text owned by B")
app := appPool(t, super)
require.Equal(t, 0, scopedCount(t, app, "", "uploaded_transcripts"),
"no tapir.current_user_id must be deny-all: this proves RLS is live, not bypassed")
require.Equal(t, 1, scopedCount(t, app, userA, "uploaded_transcripts"),
"user A must see exactly its own upload")
require.Equal(t, 1, scopedCount(t, app, userB, "uploaded_transcripts"),
"user B must see exactly its own upload")
// Count alone cannot distinguish "A's row" from "B's row" — read the content.
var got string
ctx := context.Background()
tx, err := app.Begin(ctx)
require.NoError(t, err)
defer tx.Rollback(ctx) //nolint:errcheck
_, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA)
require.NoError(t, err)
require.NoError(t, tx.QueryRow(ctx, `SELECT content FROM uploaded_transcripts`).Scan(&got))
require.Equal(t, "text owned by A", got, "A's scoped read must return A's text, never B's")
}
func TestUploadedTranscriptsRLS_CrossUserWritesAffectZeroRows(t *testing.T) {
newStore(t)
super := rawPool(t)
resetDB(t, super)
seedUploadedTranscript(t, super, userA, "text owned by A")
bID := seedUploadedTranscript(t, super, userB, "text owned by B")
app := appPool(t, super)
require.Equal(t, int64(0), scopedRowsAffected(t, app, userA,
`UPDATE uploaded_transcripts SET content = 'hacked' WHERE id = $1`, bID),
"user A scoped UPDATE must touch zero of B's uploads")
require.Equal(t, int64(0), scopedRowsAffected(t, app, userA,
`DELETE FROM uploaded_transcripts WHERE id = $1`, bID),
"user A scoped DELETE must touch zero of B's uploads")
var content string
require.NoError(t, super.QueryRow(context.Background(),
`SELECT content FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&content))
require.Equal(t, "text owned by B", content, "B's upload must survive A's writes unchanged")
}
func TestUploadedTranscriptsRLS_InsertForAnotherUserIsRejected(t *testing.T) {
newStore(t)
super := rawPool(t)
resetDB(t, super)
seedUploadedTranscript(t, super, userA, "text owned by A")
_, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, userB)
require.NoError(t, err)
app := appPool(t, super)
ctx := context.Background()
tx, err := app.Begin(ctx)
require.NoError(t, err)
defer tx.Rollback(ctx) //nolint:errcheck
_, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA)
require.NoError(t, err)
// Scoped as A, plant a row that claims to belong to B. The policy's WITH CHECK
// (defaulting to USING under FOR ALL) must refuse it.
_, err = tx.Exec(ctx,
`INSERT INTO uploaded_transcripts (user_id, title, status, content)
VALUES ($1, 'planted', 'done', 'planted')`, userB)
require.Error(t, err, "a row scoped as A must not be insertable on behalf of B")
require.Contains(t, err.Error(), "row-level security")
}
func TestUploadedTranscripts_RowSecurityIsEnabledForcedAndKeyedByUser(t *testing.T) {
newStore(t)
super := rawPool(t)
ctx := context.Background()
var enabled, forced bool
require.NoError(t, super.QueryRow(ctx,
`SELECT relrowsecurity, relforcerowsecurity FROM pg_class
WHERE oid = 'public.uploaded_transcripts'::regclass`).Scan(&enabled, &forced))
require.True(t, enabled, "uploaded_transcripts must have row-level security enabled")
require.True(t, forced,
"FORCE is what binds the table-owner role tapir; without it the policy is dead in production")
var userIDNotNull bool
require.NoError(t, super.QueryRow(ctx,
`SELECT is_nullable = 'NO' FROM information_schema.columns
WHERE table_name = 'uploaded_transcripts' AND column_name = 'user_id'`).Scan(&userIDNotNull))
require.True(t, userIDNotNull, "an upload with no owner would match no policy and be unreachable")
}
func TestUploadedTranscripts_DeleteUserRemovesTheirUploads(t *testing.T) {
s := newStore(t)
super := rawPool(t)
resetDB(t, super)
seedUploadedTranscript(t, super, userA, "text owned by A")
bID := seedUploadedTranscript(t, super, userB, "text owned by B")
require.NoError(t, s.DeleteUser(context.Background(), userA))
var aRows, bRows int
ctx := context.Background()
require.NoError(t, super.QueryRow(ctx,
`SELECT count(*) FROM uploaded_transcripts WHERE user_id = $1`, userA).Scan(&aRows))
require.NoError(t, super.QueryRow(ctx,
`SELECT count(*) FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&bRows))
require.Equal(t, 0, aRows, "deleting a user must remove their private transcripts (ON DELETE CASCADE)")
require.Equal(t, 1, bRows, "deleting A must not touch B's transcripts")
}
// rowSecurityOffTables are the public-schema tables that deliberately have row
// security OFF. The list is exact on purpose: a NEW table added without RLS must
// fail this test and force a decision, instead of passing CI by omission — the
// failure mode ADR-033 names, where a naive reuse "leaks private content across
// tenants and passes CI green".
//
// - transcripts: shared public caption content, the single exception for CONTENT
// (ADR-021). It is the one that must never gain a private row.
// - user_identities: the Dex-subject -> user_id map, read at login before any
// user is known (migration 004). Holds no user content.
// - invitations: exists before the user does, so there is no user to scope by;
// the single-use token is the capability (migration 009).
// - schema_migrations: golang-migrate bookkeeping.
//
// #33 describes `transcripts` as "the only non-RLS table". That is true of content
// and false of tables; this test asserts the real set.
var rowSecurityOffTables = []string{
"invitations",
"schema_migrations",
"transcripts",
"user_identities",
}
func TestRowSecurityOffTablesAreExactlyTheKnownSet(t *testing.T) {
newStore(t)
super := rawPool(t)
rows, err := super.Query(context.Background(),
`SELECT c.relname FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p') AND NOT c.relrowsecurity
ORDER BY c.relname`)
require.NoError(t, err)
defer rows.Close()
var got []string
for rows.Next() {
var name string
require.NoError(t, rows.Scan(&name))
got = append(got, name)
}
require.NoError(t, rows.Err())
want := append([]string{}, rowSecurityOffTables...)
sort.Strings(want)
require.Equal(t, want, got,
"every public table must have row security ON unless it is in rowSecurityOffTables; "+
"adding a table here is a decision that needs an ADR")
}
@@ -0,0 +1,188 @@
package store_test
import (
"context"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// These exercise the Store API over uploaded_transcripts. The test Store connects
// as the superuser, which BYPASSES row-level security — so the cross-user cases
// here prove the query's own user_id predicate (defence in depth), while the
// database-enforced guarantee is proven separately, as the non-superuser app
// role, in uploaded_transcripts_isolation_test.go. Text is synthetic; this repo is
// public and no real transcript may appear in it (#33).
func seedUsers(t *testing.T, ids ...string) {
t.Helper()
super := rawPool(t)
resetDB(t, super)
for _, id := range ids {
_, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, id)
require.NoError(t, err)
}
}
func TestCreateUpload_StartsQueuedAndIsOwnedByTheCreator(t *testing.T) {
s := newStore(t)
seedUsers(t, userA)
ctx := context.Background()
id, err := s.CreateUpload(ctx, userA, "Teams export")
require.NoError(t, err)
got, err := s.GetUpload(ctx, userA, id)
require.NoError(t, err)
assert.Equal(t, id, got.ID)
assert.Equal(t, userA, got.UserID)
assert.Equal(t, "Teams export", got.Title)
assert.Equal(t, domain.UploadQueued, got.Status)
assert.Empty(t, got.Content)
assert.True(t, got.FinishedAt.IsZero(), "an unfinished upload has no finish time")
}
func TestUploadLifecycle_CompletedKeepsTextSegmentsEndpointAndDuration(t *testing.T) {
s := newStore(t)
seedUsers(t, userA)
ctx := context.Background()
id, err := s.CreateUpload(ctx, userA, "m")
require.NoError(t, err)
require.NoError(t, s.StartTranscribing(ctx, userA, id))
mid, err := s.GetUpload(ctx, userA, id)
require.NoError(t, err)
assert.Equal(t, domain.UploadTranscribing, mid.Status)
want := domain.Transcription{
Text: "God morgon allihopa. Vi går igenom läget.",
Segments: []domain.Segment{
{Start: 0.0, End: 1.5, Text: "God morgon allihopa."},
{Start: 1.5, End: 4.25, Text: "Vi går igenom läget."},
},
Endpoint: "iguana/kb-whisper",
AudioSeconds: 4.25,
}
require.NoError(t, s.CompleteUpload(ctx, userA, id, want))
got, err := s.GetUpload(ctx, userA, id)
require.NoError(t, err)
assert.Equal(t, domain.UploadDone, got.Status)
assert.Equal(t, want.Text, got.Content)
assert.Equal(t, want.Segments, got.Segments, "segment times must survive the JSONB round trip")
assert.Equal(t, "iguana/kb-whisper", got.Endpoint, "the serving endpoint is a quality fact and is recorded per row")
assert.InDelta(t, 4.25, got.AudioSeconds, 1e-9)
assert.False(t, got.FinishedAt.IsZero())
assert.Empty(t, got.ErrorKind)
}
func TestUploadLifecycle_FailedRecordsTypedKindAndNoContent(t *testing.T) {
s := newStore(t)
seedUsers(t, userA)
ctx := context.Background()
id, err := s.CreateUpload(ctx, userA, "m")
require.NoError(t, err)
require.NoError(t, s.StartTranscribing(ctx, userA, id))
require.NoError(t, s.FailUpload(ctx, userA, id, domain.UploadErrTranscriptionFailed))
got, err := s.GetUpload(ctx, userA, id)
require.NoError(t, err)
assert.Equal(t, domain.UploadFailed, got.Status)
assert.Equal(t, domain.UploadErrTranscriptionFailed, got.ErrorKind)
assert.Empty(t, got.Content)
assert.False(t, got.FinishedAt.IsZero())
}
func TestGetUpload_AnotherUsersIDIsNotFound(t *testing.T) {
s := newStore(t)
seedUsers(t, userA, userB)
ctx := context.Background()
id, err := s.CreateUpload(ctx, userA, "A's meeting")
require.NoError(t, err)
_, err = s.GetUpload(ctx, userB, id)
assert.ErrorIs(t, err, store.ErrUploadNotFound,
"a non-owner must get not-found, not forbidden: existence is not disclosed")
}
func TestUploadWrites_ByAnotherUserChangeNothing(t *testing.T) {
s := newStore(t)
seedUsers(t, userA, userB)
ctx := context.Background()
id, err := s.CreateUpload(ctx, userA, "A's meeting")
require.NoError(t, err)
assert.ErrorIs(t, s.StartTranscribing(ctx, userB, id), store.ErrUploadNotFound)
assert.ErrorIs(t, s.CompleteUpload(ctx, userB, id, domain.Transcription{Text: "planted"}), store.ErrUploadNotFound)
assert.ErrorIs(t, s.FailUpload(ctx, userB, id, domain.UploadErrTranscriptionFailed), store.ErrUploadNotFound)
got, err := s.GetUpload(ctx, userA, id)
require.NoError(t, err)
assert.Equal(t, domain.UploadQueued, got.Status, "B's writes must leave A's upload exactly as it was")
assert.Empty(t, got.Content)
}
func TestListUploads_ReturnsOnlyTheCallersNewestFirstWithoutTranscriptText(t *testing.T) {
s := newStore(t)
seedUsers(t, userA, userB)
ctx := context.Background()
first, err := s.CreateUpload(ctx, userA, "first")
require.NoError(t, err)
second, err := s.CreateUpload(ctx, userA, "second")
require.NoError(t, err)
_, err = s.CreateUpload(ctx, userB, "B's")
require.NoError(t, err)
require.NoError(t, s.StartTranscribing(ctx, userA, first))
require.NoError(t, s.CompleteUpload(ctx, userA, first, domain.Transcription{Text: "long text body", Endpoint: "e"}))
got, err := s.ListUploads(ctx, userA)
require.NoError(t, err)
require.Len(t, got, 2, "A must see only A's uploads")
assert.Equal(t, second, got[0].ID, "newest first")
assert.Equal(t, first, got[1].ID)
assert.Empty(t, got[1].Content, "a list row must not drag the whole transcript along")
assert.Equal(t, domain.UploadDone, got[1].Status)
}
func TestFailInterruptedUploads_FailsOnlyUnfinishedRowsForEveryUser(t *testing.T) {
s := newStore(t)
seedUsers(t, userA, userB)
super := rawPool(t)
ctx := context.Background()
for subject, uid := range map[string]string{"sub-a": userA, "sub-b": userB} {
_, err := super.Exec(ctx, `INSERT INTO user_identities (dex_subject, user_id) VALUES ($1, $2)`, subject, uid)
require.NoError(t, err)
}
queued, err := s.CreateUpload(ctx, userA, "queued")
require.NoError(t, err)
running, err := s.CreateUpload(ctx, userB, "running")
require.NoError(t, err)
require.NoError(t, s.StartTranscribing(ctx, userB, running))
done, err := s.CreateUpload(ctx, userA, "done")
require.NoError(t, err)
require.NoError(t, s.StartTranscribing(ctx, userA, done))
require.NoError(t, s.CompleteUpload(ctx, userA, done, domain.Transcription{Text: "kept", Endpoint: "e"}))
n, err := s.FailInterruptedUploads(ctx)
require.NoError(t, err)
assert.Equal(t, 2, n, "the queued and the transcribing row, across both users")
for _, c := range []struct{ user, id string }{{userA, queued}, {userB, running}} {
got, err := s.GetUpload(ctx, c.user, c.id)
require.NoError(t, err)
assert.Equal(t, domain.UploadFailed, got.Status)
assert.Equal(t, domain.UploadErrInterrupted, got.ErrorKind,
"audio is not retained, so a restart cannot resume: the honest state is failed/interrupted")
}
kept, err := s.GetUpload(ctx, userA, done)
require.NoError(t, err)
assert.Equal(t, domain.UploadDone, kept.Status, "a finished transcript must survive the sweep")
assert.Equal(t, "kept", kept.Content)
}
+36 -15
View File
@@ -7,7 +7,7 @@ import (
"github.com/jackc/pgx/v5"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// UpsertVideo persists a video's metadata and returns its durable store id (the
@@ -46,15 +46,16 @@ func (s *Store) UpsertVideo(ctx context.Context, v domain.Video) (string, error)
}
if err := tx.QueryRow(ctx,
`INSERT INTO videos (user_id, provider, provider_video_id, title, url, published_at, channel_title)
VALUES ($1, $2, $3, $4, $5, $6, $7)
`INSERT INTO videos (user_id, provider, provider_video_id, title, url, published_at, channel_title, duration_s)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (user_id, provider, provider_video_id) DO UPDATE SET
title = EXCLUDED.title,
url = EXCLUDED.url,
published_at = EXCLUDED.published_at,
channel_title = COALESCE(NULLIF(EXCLUDED.channel_title, ''), videos.channel_title)
channel_title = COALESCE(NULLIF(EXCLUDED.channel_title, ''), videos.channel_title),
duration_s = COALESCE(EXCLUDED.duration_s, videos.duration_s)
RETURNING id`,
v.UserID, provider, v.ProviderVideoID, v.Title, v.URL, nullTime(v.PublishedAt), v.ChannelTitle,
v.UserID, provider, v.ProviderVideoID, v.Title, v.URL, nullTime(v.PublishedAt), v.ChannelTitle, nullDuration(v.DurationSeconds),
).Scan(&id); err != nil {
return fmt.Errorf("store: upsert video: %w", err)
}
@@ -74,12 +75,27 @@ func nullTime(t time.Time) *time.Time {
return &t
}
// NewestUnsummarizedVideoIDs returns up to limit of the user's videos that have
// no summary yet, newest first (published_at DESC, NULLS LAST). It caps the
// connect-time onboarding burst (Feature 1) at a fixed count: the caller marks
// these for summarization through the shared rate gate. RLS-scoped via withUser,
// so it only ever sees the requesting user's rows. limit <= 0 returns nil.
func (s *Store) NewestUnsummarizedVideoIDs(ctx context.Context, userID string, limit int) ([]string, error) {
// nullDuration maps an unknown duration (0) to SQL NULL so the upsert's
// COALESCE(EXCLUDED.duration_s, videos.duration_s) preserves a previously-known
// value instead of clobbering it with 0 (ADR-028; the channel_title backfill
// stance, migration 014).
func nullDuration(seconds int) *int {
if seconds <= 0 {
return nil
}
return &seconds
}
// OnboardBurstVideoIDs returns up to limit of the user's unsummarized videos for
// the connect-time onboarding burst (ADR-028), newest-first but quality-aware: a
// video is excluded when its duration is KNOWN and outside [minSeconds, maxSeconds]
// — dropping Shorts (below min) and multi-hour livestream VODs (above max) that
// would waste a scarce caption fetch on a poor first impression. A NULL/unknown
// duration is kept (degrade-open) but ranked AFTER known-good rows, so a freshly
// enriched good pick wins when both exist. minSeconds<=0 / maxSeconds<=0 each
// disable that bound (0/0 == pure newest-first, the reversibility lever).
// RLS-scoped via withUser; limit <= 0 returns nil.
func (s *Store) OnboardBurstVideoIDs(ctx context.Context, userID string, limit, minSeconds, maxSeconds int) ([]string, error) {
if limit <= 0 {
return nil, nil
}
@@ -92,16 +108,21 @@ func (s *Store) NewestUnsummarizedVideoIDs(ctx context.Context, userID string, l
AND NOT EXISTS (
SELECT 1 FROM summaries su
WHERE su.user_id = v.user_id AND su.video_id = v.id)
ORDER BY v.published_at DESC NULLS LAST, v.seen_at DESC
LIMIT $2`, userID, limit)
AND NOT (
v.duration_s IS NOT NULL
AND ( ($3 > 0 AND v.duration_s < $3)
OR ($4 > 0 AND v.duration_s > $4) ))
ORDER BY (v.duration_s IS NOT NULL) DESC,
v.published_at DESC NULLS LAST, v.seen_at DESC
LIMIT $2`, userID, limit, minSeconds, maxSeconds)
if err != nil {
return fmt.Errorf("store: newest unsummarized: %w", err)
return fmt.Errorf("store: onboard burst videos: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return fmt.Errorf("store: scan newest unsummarized: %w", err)
return fmt.Errorf("store: scan onboard burst video: %w", err)
}
ids = append(ids, id)
}
+64 -15
View File
@@ -7,7 +7,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
func ytVideo(userID, provVideoID, title string) domain.Video {
@@ -52,6 +52,36 @@ func TestUpsertVideo_ReturnsStableID(t *testing.T) {
require.Equal(t, 1, count, "must not duplicate the row")
}
func TestUpsertVideo_PersistsAndPreservesDuration(t *testing.T) {
ctx := context.Background()
s := newStore(t)
resetDB(t, rawPool(t))
// First upsert carries a known duration (ADR-028: discovery enriches it).
v := ytVideo(userA, "dur0000001x", "with duration")
v.DurationSeconds = 750
id, err := s.UpsertVideo(ctx, v)
require.NoError(t, err)
p := rawPool(t)
readDuration := func() *int {
var d *int
require.NoError(t, p.QueryRow(ctx, `SELECT duration_s FROM videos WHERE id = $1`, id).Scan(&d))
return d
}
require.NotNil(t, readDuration())
require.Equal(t, 750, *readDuration(), "duration must persist")
// A later upsert that does NOT know the duration (0) must not clobber it —
// the channel_title backfill stance (migration 014): COALESCE-preserve.
v2 := ytVideo(userA, "dur0000001x", "title updated, duration unknown")
v2.DurationSeconds = 0
_, err = s.UpsertVideo(ctx, v2)
require.NoError(t, err)
require.NotNil(t, readDuration(), "a 0/unknown re-upsert must not erase a known duration")
require.Equal(t, 750, *readDuration())
}
func TestUpsertVideo_IDMatchesSummaryDedup(t *testing.T) {
ctx := context.Background()
s := newStore(t)
@@ -82,36 +112,55 @@ func TestUpsertVideo_PerUserIsolation(t *testing.T) {
require.NotEqual(t, idA, idB, "same provider video for two users must be two distinct rows")
}
func TestNewestUnsummarizedVideoIDs(t *testing.T) {
func TestOnboardBurstVideoIDs(t *testing.T) {
ctx := context.Background()
s := newStore(t)
resetDB(t, rawPool(t))
mk := func(user, pid string, day int) string {
mk := func(user, pid string, day, dur int) string {
v := ytVideo(user, pid, pid)
v.PublishedAt = time.Date(2026, 6, day, 12, 0, 0, 0, time.UTC)
v.DurationSeconds = dur // 0 == unknown (NULL)
id, err := s.UpsertVideo(ctx, v)
require.NoError(t, err)
return id
}
_ = mk(userA, "a1vid000001", 1)
id2 := mk(userA, "a2vid000002", 2)
id3 := mk(userA, "a3vid000003", 3)
id4 := mk(userA, "a4vid000004", 4)
mk(userB, "b1vid000009", 9) // userB's newest — must never leak via RLS
summarized := mk(userA, "summ0000001", 6, 600) // newest known-good, but already summarized
good1 := mk(userA, "good0000001", 5, 600) // 10m, newest UNsummarized known-good
tooLong := mk(userA, "toolong0001", 4, 20000) // > maxSeconds -> dropped
tooShort := mk(userA, "tooshort001", 3, 30) // < minSeconds -> dropped
unknown := mk(userA, "unknown0001", 2, 0) // NULL duration -> kept, ranked last
good2 := mk(userA, "good0000002", 1, 800) // known-good but oldest
mk(userB, "bvid0000009", 9, 600) // userB -> must not leak via RLS
// The newest (v4) is summarized, so it's excluded from "unsummarized".
require.NoError(t, s.Deliver(ctx, summary(userA, id4, "done")))
// The newest video is summarized, so it is excluded from the burst.
require.NoError(t, s.Deliver(ctx, summary(userA, summarized, "done")))
// Cap 2, newest-first unsummarized: v3 then v2 (v4 excluded; userB excluded).
got, err := s.NewestUnsummarizedVideoIDs(ctx, userA, 2)
const minSec, maxSec = 60, 14400
// Known-good ranked before unknown, each newest-first within its group; the
// too-long and too-short videos are excluded by their known duration.
got, err := s.OnboardBurstVideoIDs(ctx, userA, 5, minSec, maxSec)
require.NoError(t, err)
require.Equal(t, []string{id3, id2}, got)
require.Equal(t, []string{good1, good2, unknown}, got,
"known-good first (newest-first), then unknown-duration; junk excluded")
none, err := s.NewestUnsummarizedVideoIDs(ctx, userA, 0)
// Cap is honoured.
capped, err := s.OnboardBurstVideoIDs(ctx, userA, 2, minSec, maxSec)
require.NoError(t, err)
require.Empty(t, none, "limit 0 returns nothing")
require.Equal(t, []string{good1, good2}, capped)
// Bounds disabled (0/0) == pure newest-first, nothing excluded.
all, err := s.OnboardBurstVideoIDs(ctx, userA, 10, 0, 0)
require.NoError(t, err)
require.ElementsMatch(t, []string{good1, tooLong, tooShort, unknown, good2}, all,
"0/0 bounds disable the duration filter (prior newest-first behaviour)")
// limit <= 0 returns nothing.
none, err := s.OnboardBurstVideoIDs(ctx, userA, 0, minSec, maxSec)
require.NoError(t, err)
require.Empty(t, none)
}
func TestUpsertVideoPersistsChannelAndDistinctChannels(t *testing.T) {
+11 -2
View File
@@ -13,11 +13,13 @@ import (
"encoding/json"
"errors"
"fmt"
"log/slog"
"strings"
"time"
"unicode/utf8"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/metrics"
)
// Completer is the minimal LLM chat surface the Summarizer needs.
@@ -95,16 +97,23 @@ func (s *Summarizer) Summarize(ctx context.Context, v domain.Video, t domain.Tra
var errs []error
for i, ep := range s.endpoints {
fallback := i > 0
start := time.Now()
out, err := ep.Client.Complete(ctx, systemPrompt, user)
dur := time.Since(start)
if err != nil {
metrics.ObserveSummarize(ep.Model, "error", fallback, dur)
errs = append(errs, fmt.Errorf("%s/%s call: %w", ep.Provider, ep.Model, err))
continue
}
sum, perr := s.build(v, ep, i > 0, out)
sum, perr := s.build(v, ep, fallback, out)
if perr != nil {
metrics.ObserveSummarize(ep.Model, "parse_error", fallback, dur)
errs = append(errs, fmt.Errorf("%s/%s output: %w", ep.Provider, ep.Model, perr))
continue
}
metrics.ObserveSummarize(ep.Model, "success", fallback, dur)
slog.Default().Info("summarized", "model", ep.Model, "fallback", fallback, "elapsed_ms", dur.Milliseconds())
return sum, nil
}
return domain.Summary{}, fmt.Errorf("summarize: all %d endpoint(s) failed: %w", len(s.endpoints), errors.Join(errs...))
@@ -7,13 +7,36 @@ package summarizer
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/metrics"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// TestSummarizerRecordsMetric verifies the summarizer→metrics wiring (ADR-030)
// black-box: after a successful summarize, the public /metrics scrape shows a
// success observation for that endpoint's model.
func TestSummarizerRecordsMetric(t *testing.T) {
const model = "metrics-test-model"
s := New(Endpoint{Client: &fakeClient{reply: goodReply}, Provider: "local", Model: model}, nil)
if _, err := s.Summarize(context.Background(), testVideo(), testTranscript()); err != nil {
t.Fatalf("Summarize: %v", err)
}
rec := httptest.NewRecorder()
metrics.Handler().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/metrics", nil))
body := rec.Body.String()
if !strings.Contains(body, `tapir_summarize_duration_seconds`) ||
!strings.Contains(body, `model="`+model+`"`) ||
!strings.Contains(body, `outcome="success"`) {
t.Errorf("metrics scrape missing summarize success for %s", model)
}
}
// compile-time check: Summarizer satisfies the port.
var _ ports.Summarizer = (*Summarizer)(nil)
+32 -1
View File
@@ -7,10 +7,13 @@ import (
"encoding/xml"
"fmt"
"io"
"log/slog"
"net/http"
"strings"
"time"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/metrics"
)
// defaultPlayerBaseURL is the InnerTube / watch-page host. Overridable via
@@ -43,7 +46,35 @@ const maxCaptionBytes = 16 << 20 // 16 MiB
// fetch, or an unparseable body all yield SourceNone rather than an error. Only
// genuine transport (network) faults return an error. Audio download and
// speech-to-text remain absent (ADR-007).
// FetchTranscript times the caption fetch and records its latency by outcome
// (ADR-030) before returning. Transport errors are surfaced to the caller and not
// recorded as an outcome (logged upstream); the three resolved outcomes
// captions|none|rate_limited are the ones that consume the scarce fetch budget.
func (a *Adapter) FetchTranscript(ctx context.Context, v domain.Video) (domain.Transcript, error) {
start := time.Now()
tr, err := a.fetchTranscript(ctx, v)
if err == nil {
dur := time.Since(start)
outcome := captionOutcome(tr.Source)
metrics.ObserveCaptionFetch(outcome, dur)
slog.Default().Info("caption fetch", "video", v.ProviderVideoID, "outcome", outcome, "elapsed_ms", dur.Milliseconds())
}
return tr, err
}
// captionOutcome maps a transcript source to the metric outcome label.
func captionOutcome(s domain.TranscriptSource) string {
switch s {
case domain.SourceCaptions:
return "captions"
case domain.SourceRateLimited:
return "rate_limited"
default:
return "none"
}
}
func (a *Adapter) fetchTranscript(ctx context.Context, v domain.Video) (domain.Transcript, error) {
client := a.plainClient()
tracks, err := a.captionTracks(ctx, client, v.ProviderVideoID)
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"net/http"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
func TestVideoByID(t *testing.T) {
+6 -2
View File
@@ -27,8 +27,8 @@ import (
"golang.org/x/oauth2"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// defaultBaseURL is the YouTube Data API v3 root. Overridable via Config.BaseURL
@@ -302,6 +302,10 @@ func (a *Adapter) filterLowValue(ctx context.Context, client *http.Client, video
if m.seconds > 0 && m.seconds < a.cfg.MinVideoSeconds {
continue // Short / sub-threshold clip
}
// Carry the duration we already fetched onto the kept video so the store
// can persist it (ADR-028) — the burst's length-aware selection depends on
// it. Discarding it here was the gap the onboarding investigation found.
v.DurationSeconds = m.seconds
kept = append(kept, v)
}
return kept
+6 -1
View File
@@ -7,7 +7,7 @@ import (
"net/http/httptest"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// --- fakes ------------------------------------------------------------------
@@ -224,6 +224,11 @@ func TestNewVideosFiltersShortsAndLive(t *testing.T) {
if len(vids) != 1 || vids[0].ProviderVideoID != "long1" {
t.Fatalf("expected only long1 to survive the filter, got %+v", vids)
}
// The duration fetched for the filter is carried onto the kept video so the
// store can persist it (ADR-028) instead of discarding it.
if vids[0].DurationSeconds != 750 {
t.Fatalf("kept video DurationSeconds = %d, want 750 (PT12M30S)", vids[0].DurationSeconds)
}
}
// TestNewVideosNoFilterWhenDisabled: MinVideoSeconds=0 keeps the pre-ADR-023
+1 -1
View File
@@ -12,7 +12,7 @@ import (
"golang.org/x/oauth2"
"gitea.d-ma.be/mathias/tapir/internal/auth"
"git.d-ma.be/mathias/tapir/internal/auth"
)
// fakeWriter is a TokenWriter capturing the persisted (ref, value).
+34
View File
@@ -120,6 +120,21 @@ type Config struct {
// caption rate gate (ADR-014) — the cap bounds count, never the pacing. Default 3.
OnboardSummarizeCount int
// OnboardSummarizerModel is the summarizer alias the connect-time burst leads
// its chain with (ADR-028) — a stronger model is affordable on the ≤3 summaries
// that form a new user's first impression. It heads a burst-specific chain;
// the standard chain (ADR-022) follows as resilience. Empty (or equal to
// SummarizerModel) collapses the burst back onto the shared processor — the
// reversibility lever. Default iguana/gemma4-26b (the brain-validated model).
OnboardSummarizerModel string
// OnboardMaxVideoSeconds upper-bounds the duration of a video the onboarding
// burst will pick (ADR-028), so the burst does not spend a scarce caption fetch
// on a multi-hour livestream VOD that passed the live filter once it ended. Only
// a KNOWN duration outside [MinVideoSeconds, this] is dropped; a NULL/unknown
// duration is kept (degrade-open). 0 disables the upper bound. Default 14400 (4h).
OnboardMaxVideoSeconds int
// DiscoveryInterval, when > 0, makes `serve` run in-process scheduled discovery
// for ALL users on that cadence (ADR-018). Zero/unset = disabled, so dev and
// tests never auto-fetch. Single-replica assumption — see cmdServe.
@@ -128,6 +143,11 @@ type Config struct {
// HTTPAddr is the listen address for `tapir serve` (the Stage-0 web UI).
HTTPAddr string
// MetricsAddr is the listen address for the Prometheus /metrics endpoint
// (ADR-030). A SEPARATE port from HTTPAddr so /metrics is never exposed on the
// public app — only scraped in-cluster. Empty disables the metrics server.
MetricsAddr string
// PublicURL is the externally-reachable base URL of the deployed service,
// e.g. "https://tapir.d-ma.be". Used to build absolute links handed to humans
// (the `tapir invite` URL). No trailing slash is assumed — callers trim it.
@@ -163,12 +183,15 @@ const (
defaultYTConnectRedirectURL = "https://tapir.d-ma.be/oauth/youtube/callback"
defaultOAuthRedirectAddr = "localhost:8080"
defaultHTTPAddr = ":8080"
defaultMetricsAddr = ":9090"
defaultFetchBackoff = time.Hour
defaultFetchRate = 2 * time.Second
defaultPublicURL = "https://tapir.d-ma.be"
defaultAutoSummarizeWindow = 7 * 24 * time.Hour
defaultOnboardSummarizeCount = 3
maxOnboardSummarizeCount = 5
defaultOnboardSummarizerModel = "iguana/gemma4-26b"
defaultOnboardMaxVideoSeconds = 14400 // 4h
)
// Load reads the environment into a Config, applying defaults. It does not
@@ -181,6 +204,7 @@ func Load() (Config, error) {
GatewayURL: envOr("TAPIR_GATEWAY_URL", defaultGatewayURL),
GatewayKey: os.Getenv("TAPIR_GATEWAY_KEY"),
SummarizerModel: envOr("TAPIR_SUMMARIZER_MODEL", defaultSummarizerModel),
OnboardSummarizerModel: lookupOr("TAPIR_ONBOARD_SUMMARIZER_MODEL", defaultOnboardSummarizerModel),
FallbackModel: lookupOr("TAPIR_FALLBACK_MODEL", defaultFallbackModel),
CloudFallbackModel: lookupOr("TAPIR_CLOUD_FALLBACK_MODEL", defaultCloudFallbackModel),
DBDSN: os.Getenv("TAPIR_DB_DSN"),
@@ -191,6 +215,7 @@ func Load() (Config, error) {
SecretsFile: envOr("TAPIR_SECRETS_FILE", defaultSecretsFile()),
OAuthRedirectAddr: envOr("TAPIR_OAUTH_REDIRECT_ADDR", defaultOAuthRedirectAddr),
HTTPAddr: envOr("TAPIR_HTTP_ADDR", defaultHTTPAddr),
MetricsAddr: lookupOr("TAPIR_METRICS_ADDR", defaultMetricsAddr),
PublicURL: envOr("TAPIR_PUBLIC_URL", defaultPublicURL),
OIDCIssuer: os.Getenv("TAPIR_OIDC_ISSUER"),
DexClientID: os.Getenv("TAPIR_DEX_CLIENT_ID"),
@@ -286,6 +311,15 @@ func Load() (Config, error) {
}
c.OnboardSummarizeCount = onboard
onboardMax, err := intOr("TAPIR_ONBOARD_MAX_VIDEO_SECONDS", defaultOnboardMaxVideoSeconds)
if err != nil {
return Config{}, err
}
if onboardMax < 0 {
onboardMax = 0
}
c.OnboardMaxVideoSeconds = onboardMax
return c, nil
}
+59
View File
@@ -214,3 +214,62 @@ func TestValidateForAuth_PassesWhenComplete(t *testing.T) {
t.Errorf("ValidateForAuth: unexpected error %v", err)
}
}
func TestLoad_OnboardSummarizerModel(t *testing.T) {
cases := []struct {
name, env string
set bool
want string
}{
{"default", "", false, defaultOnboardSummarizerModel},
{"explicit", "koala/some-model", true, "koala/some-model"},
{"empty disables (collapses to shared processor)", "", true, ""},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
env := map[string]string{}
if c.set {
env["TAPIR_ONBOARD_SUMMARIZER_MODEL"] = c.env
}
setEnv(t, env)
cfg, err := Load()
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.OnboardSummarizerModel != c.want {
t.Fatalf("OnboardSummarizerModel = %q, want %q", cfg.OnboardSummarizerModel, c.want)
}
})
}
}
func TestLoad_OnboardMaxVideoSeconds(t *testing.T) {
cases := []struct {
name, env string
want int
}{
{"default", "", defaultOnboardMaxVideoSeconds},
{"explicit", "7200", 7200},
{"zero disables", "0", 0},
{"negative clamps to zero", "-9", 0},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
setEnv(t, map[string]string{"TAPIR_ONBOARD_MAX_VIDEO_SECONDS": c.env})
cfg, err := Load()
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.OnboardMaxVideoSeconds != c.want {
t.Fatalf("OnboardMaxVideoSeconds = %d, want %d", cfg.OnboardMaxVideoSeconds, c.want)
}
})
}
}
func TestLoad_OnboardMaxVideoSecondsInvalid(t *testing.T) {
setEnv(t, map[string]string{"TAPIR_ONBOARD_MAX_VIDEO_SECONDS": "long"})
if _, err := Load(); err == nil {
t.Fatal("Load: want error for non-numeric TAPIR_ONBOARD_MAX_VIDEO_SECONDS")
}
}
+64
View File
@@ -77,6 +77,11 @@ type Video struct {
URL string
PublishedAt time.Time
SeenAt time.Time
// DurationSeconds is the video length in seconds, when known (fetched by the
// ADR-023 videos.list enrichment at discovery). 0 means unknown — the store
// preserves a previously-known value rather than overwriting it with 0, and
// the onboarding burst (ADR-028) treats unknown as degrade-open (kept).
DurationSeconds int
}
// Transcript is the text of a video (or a record that none was available).
@@ -106,3 +111,62 @@ type Summary struct {
FallbackUsed bool
CreatedAt time.Time
}
// Segment is one timed span of a transcription. Times are seconds from the start
// of the audio.
type Segment struct {
Start float64 `json:"start"`
End float64 `json:"end"`
Text string `json:"text"`
}
// Transcription is the result of transcribing user-supplied audio (ADR-033):
// the full text, its timed segments, which backend served it, and how long the
// audio was. Endpoint is recorded because the backends differ in timestamp
// granularity — which one served a transcript is a quality fact.
type Transcription struct {
Text string
Segments []Segment
Endpoint string
AudioSeconds float64
}
// UploadStatus is the state of an uploaded-audio transcription job.
type UploadStatus string
const (
UploadQueued UploadStatus = "queued"
UploadTranscribing UploadStatus = "transcribing"
UploadDone UploadStatus = "done"
UploadFailed UploadStatus = "failed"
)
// UploadErrorKind is the typed reason an upload failed. It is deliberately NOT a
// provider's error string: the whisper server's "failed to process audio" has had
// at least five distinct causes, so nothing may branch on it (ADR-033).
type UploadErrorKind string
const (
// UploadErrInterrupted: the process restarted mid-job. The audio is not
// retained, so the job cannot resume; the user re-uploads.
UploadErrInterrupted UploadErrorKind = "interrupted"
// UploadErrTranscriptionFailed: every configured transcription endpoint failed.
UploadErrTranscriptionFailed UploadErrorKind = "transcription_failed"
)
// UploadedTranscript is a user's PRIVATE transcript of audio they uploaded. It is
// never stored in, or merged with, the shared public-caption Transcript (ADR-021).
// Content and Segments are empty until Status is UploadDone.
type UploadedTranscript struct {
ID string
UserID string
Title string
Status UploadStatus
Content string
Segments []Segment
Endpoint string
AudioSeconds float64
ErrorKind UploadErrorKind
CreatedAt time.Time
FinishedAt time.Time // zero until the job reaches done or failed
}
+139
View File
@@ -0,0 +1,139 @@
// Package metrics is Tapir's Prometheus instrumentation (ADR-030, issue #15). It
// owns the collectors and a small typed API the rest of the app calls — adapters
// never touch prometheus types directly. Two themes:
//
// - HTTP/session: request count + latency by route (the matched pattern, so
// cardinality stays bounded), and logins.
// - AI (the priority): summarization latency by model/outcome/fallback, caption
// fetch latency by outcome, chat latency by model, and LLM token usage.
//
// Handler() is served on a dedicated port (never the public app port) so a scrape
// is in-cluster only. slog timing lines are emitted at the call sites too.
package metrics
import (
"net/http"
"strconv"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
// latencyBuckets spans sub-second UI calls up to multi-minute model calls (a cold
// local model load is tens of seconds; the cloud fallback can be longer).
var latencyBuckets = []float64{0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10, 20, 30, 60, 120, 300}
var (
httpRequests = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "tapir_http_requests_total",
Help: "HTTP requests by method, matched route pattern, and status code.",
}, []string{"method", "route", "code"})
httpDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
Name: "tapir_http_request_duration_seconds",
Help: "HTTP request latency by method and matched route pattern.",
Buckets: []float64{0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2, 5},
}, []string{"method", "route"})
logins = promauto.NewCounter(prometheus.CounterOpts{
Name: "tapir_logins_total",
Help: "Successful OIDC logins (session established).",
})
summarizeDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
Name: "tapir_summarize_duration_seconds",
Help: "Per-endpoint summarization latency by model, outcome (success|parse_error|error), and whether it was a fallback.",
Buckets: latencyBuckets,
}, []string{"model", "outcome", "fallback"})
captionFetchDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
Name: "tapir_caption_fetch_duration_seconds",
Help: "Caption fetch latency by outcome (captions|none|rate_limited).",
Buckets: latencyBuckets,
}, []string{"outcome"})
chatDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
Name: "tapir_chat_duration_seconds",
Help: "Per-video Q&A answer latency by model.",
Buckets: latencyBuckets,
}, []string{"model"})
llmTokens = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "tapir_llm_tokens_total",
Help: "LLM tokens consumed by model and kind (prompt|completion).",
}, []string{"model", "kind"})
)
// Handler serves the Prometheus exposition format. Mount on the dedicated metrics
// port, never the public app mux.
func Handler() http.Handler { return promhttp.Handler() }
// IncLogin records a successful login.
func IncLogin() { logins.Inc() }
// ObserveSummarize records one summarization endpoint attempt.
func ObserveSummarize(model, outcome string, fallback bool, d time.Duration) {
summarizeDuration.WithLabelValues(model, outcome, strconv.FormatBool(fallback)).Observe(d.Seconds())
}
// ObserveCaptionFetch records one caption fetch by outcome.
func ObserveCaptionFetch(outcome string, d time.Duration) {
captionFetchDuration.WithLabelValues(outcome).Observe(d.Seconds())
}
// ObserveChat records one Q&A answer latency.
func ObserveChat(model string, d time.Duration) {
chatDuration.WithLabelValues(model).Observe(d.Seconds())
}
// RecordTokens records LLM token usage from a completion's usage block. Zero
// counts are skipped so a provider that omits usage adds nothing.
func RecordTokens(model string, prompt, completion int) {
if prompt > 0 {
llmTokens.WithLabelValues(model, "prompt").Add(float64(prompt))
}
if completion > 0 {
llmTokens.WithLabelValues(model, "completion").Add(float64(completion))
}
}
// HTTPMiddleware records request count + latency. It reads r.Pattern AFTER the
// inner handler routes (Go 1.22 sets it during ServeMux matching), so the label is
// the bounded registered pattern (e.g. "GET /v/{videoId}"), never the raw path
// with its high-cardinality ids. Unmatched requests bucket as "other".
func HTTPMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
sw := &statusWriter{ResponseWriter: w, code: http.StatusOK}
next.ServeHTTP(sw, r)
route := r.Pattern
if route == "" {
route = "other"
}
httpRequests.WithLabelValues(r.Method, route, strconv.Itoa(sw.code)).Inc()
httpDuration.WithLabelValues(r.Method, route).Observe(time.Since(start).Seconds())
})
}
// statusWriter captures the response status for the request-count label.
type statusWriter struct {
http.ResponseWriter
code int
wroteHeader bool
}
func (s *statusWriter) WriteHeader(code int) {
if !s.wroteHeader {
s.code = code
s.wroteHeader = true
}
s.ResponseWriter.WriteHeader(code)
}
func (s *statusWriter) Write(b []byte) (int, error) {
s.wroteHeader = true // an implicit 200
return s.ResponseWriter.Write(b)
}
+92
View File
@@ -0,0 +1,92 @@
package metrics
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/testutil"
dto "github.com/prometheus/client_model/go"
"github.com/stretchr/testify/require"
)
// histCount reads a histogram child's observation count (testutil.ToFloat64 only
// works on counters/gauges; a histogram's WithLabelValues child is an Observer).
func histCount(t *testing.T, o prometheus.Observer) uint64 {
t.Helper()
m, ok := o.(prometheus.Metric)
require.True(t, ok, "histogram child must be a prometheus.Metric")
var d dto.Metric
require.NoError(t, m.Write(&d))
return d.GetHistogram().GetSampleCount()
}
// TestObserveSummarizeRecordsModelOutcomeFallback: a success observation lands on
// the right model/outcome/fallback series.
func TestObserveSummarizeRecordsModelOutcomeFallback(t *testing.T) {
before := histCount(t, summarizeDuration.WithLabelValues("koala/phi4-mini", "success", "false"))
ObserveSummarize("koala/phi4-mini", "success", false, 1200*time.Millisecond)
after := histCount(t, summarizeDuration.WithLabelValues("koala/phi4-mini", "success", "false"))
require.Equal(t, before+1, after, "one success observation recorded for the model")
}
// TestObserveSummarizeRecordsFailureOutcomes: error and parse_error are distinct
// series so a fallback chain's failures are visible.
func TestObserveSummarizeRecordsFailureOutcomes(t *testing.T) {
e0 := histCount(t, summarizeDuration.WithLabelValues("m", "error", "false"))
p0 := histCount(t, summarizeDuration.WithLabelValues("m", "parse_error", "false"))
ObserveSummarize("m", "error", false, time.Second)
ObserveSummarize("m", "parse_error", false, time.Second)
require.Equal(t, e0+1, histCount(t, summarizeDuration.WithLabelValues("m", "error", "false")))
require.Equal(t, p0+1, histCount(t, summarizeDuration.WithLabelValues("m", "parse_error", "false")))
}
func TestObserveCaptionFetchByOutcome(t *testing.T) {
b := histCount(t, captionFetchDuration.WithLabelValues("captions"))
ObserveCaptionFetch("captions", 3*time.Second)
require.Equal(t, b+1, histCount(t, captionFetchDuration.WithLabelValues("captions")))
}
func TestChatAnswerLatencyRecorded(t *testing.T) {
b := histCount(t, chatDuration.WithLabelValues("iguana/gemma4-26b"))
ObserveChat("iguana/gemma4-26b", 2*time.Second)
require.Equal(t, b+1, histCount(t, chatDuration.WithLabelValues("iguana/gemma4-26b")))
}
// TestRecordTokens: prompt + completion land on their kind series; zero is skipped.
func TestRecordTokens(t *testing.T) {
p0 := testutil.ToFloat64(llmTokens.WithLabelValues("m", "prompt"))
c0 := testutil.ToFloat64(llmTokens.WithLabelValues("m", "completion"))
RecordTokens("m", 100, 40)
RecordTokens("m", 0, 0) // skipped, no panic
require.Equal(t, p0+100, testutil.ToFloat64(llmTokens.WithLabelValues("m", "prompt")))
require.Equal(t, c0+40, testutil.ToFloat64(llmTokens.WithLabelValues("m", "completion")))
}
func TestLoginCounted(t *testing.T) {
b := testutil.ToFloat64(logins)
IncLogin()
require.Equal(t, b+1, testutil.ToFloat64(logins))
}
// TestHTTPMiddlewareRecordsByRoutePattern: the request is counted under the bounded
// registered pattern (r.Pattern after routing), not the raw path with its ids.
func TestHTTPMiddlewareRecordsByRoutePattern(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("GET /v/{videoId}", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusTeapot)
})
h := HTTPMiddleware(mux)
before := testutil.ToFloat64(httpRequests.WithLabelValues("GET", "GET /v/{videoId}", "418"))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/v/abc-123", nil))
require.Equal(t, http.StatusTeapot, rec.Code)
after := testutil.ToFloat64(httpRequests.WithLabelValues("GET", "GET /v/{videoId}", "418"))
require.Equal(t, before+1, after, "counted under the pattern, not /v/abc-123")
require.Equal(t, float64(0), testutil.ToFloat64(httpRequests.WithLabelValues("GET", "/v/abc-123", "418")),
"raw path must never be a label value")
}
+1 -1
View File
@@ -7,7 +7,7 @@ package ports
import (
"context"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// VideoSource is a video platform Tapir watches (YouTube, Vimeo).
+3 -3
View File
@@ -19,9 +19,9 @@ import (
"slices"
"time"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"gitea.d-ma.be/mathias/tapir/internal/usecase"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/usecase"
)
// passCandidate is a video that passed all pre-filters (seen/manual/backoff)
+3 -3
View File
@@ -9,9 +9,9 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/runner"
"gitea.d-ma.be/mathias/tapir/internal/usecase"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/runner"
"git.d-ma.be/mathias/tapir/internal/usecase"
)
const testUser = "11111111-1111-1111-1111-111111111111"
+2 -2
View File
@@ -14,8 +14,8 @@ import (
"fmt"
"sync"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/ports"
)
// ErrNotImplemented marks scaffold methods awaiting implementation.
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"context"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// These tests pin the ADR-021 read-stored-first behaviour at the engine core:
+1 -1
View File
@@ -3,7 +3,7 @@ package web
import (
"net/http"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// handleAccount renders the account page: the user's display name, the
+2 -2
View File
@@ -9,8 +9,8 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/web"
)
// fakeSecrets is a SecretRemover that records the refs it was asked to delete, so
+18 -9
View File
@@ -6,8 +6,8 @@ import (
"net/http"
"strings"
"gitea.d-ma.be/mathias/tapir/internal/adapters/chat"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/chat"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// Chatter is the per-video chat backend (ADR-027). *chat.Service satisfies it;
@@ -64,13 +64,21 @@ func (a *App) handleChat(w http.ResponseWriter, r *http.Request) {
if !ok {
return
}
a.render(w, r, ChatPage(chatView{
view := chatView{
VideoID: row.VideoID,
Title: displayTitle(*row),
Available: hasText,
Models: a.Chat.Models(),
Selected: a.Chat.DefaultModel(row.AIModel),
}))
}
// HTMX (the in-place reveal from the summary) gets just the open chat section,
// swapped over the closed dock so the summary above it stays put. A no-JS
// navigation gets the full page: the whole summary plus the open chat.
if isHTMX(r) {
a.render(w, r, chatSection(view))
return
}
a.render(w, r, ChatPage(*row, view))
}
// handleChatMessage answers one question against the stored transcript (POST).
@@ -129,7 +137,7 @@ func (a *App) handleChatMessage(w http.ResponseWriter, r *http.Request) {
view.Truncated = reply.Truncated
}
}
a.renderChat(w, r, view)
a.renderChatTurn(w, r, *row, view)
}
// loadOwnedSummary fetches the summary for the path's video scoped to userID, or
@@ -166,14 +174,15 @@ func (a *App) readTranscript(w http.ResponseWriter, r *http.Request, row store.S
return t.Content, true, true
}
// renderChat returns the chat panel fragment for an HTMX request, or the full
// chat page otherwise (no-JS POST re-renders the whole page).
func (a *App) renderChat(w http.ResponseWriter, r *http.Request, v chatView) {
// renderChatTurn returns the chat panel fragment for an HTMX answer (swapped in
// place within the open dock), or the full chat page otherwise — the no-JS POST
// re-renders the whole summary + open chat with the new turn.
func (a *App) renderChatTurn(w http.ResponseWriter, r *http.Request, row store.SummaryRow, v chatView) {
if isHTMX(r) {
a.render(w, r, chatPanel(v))
return
}
a.render(w, r, ChatPage(v))
a.render(w, r, ChatPage(row, v))
}
// resolveModel keeps the posted model only when it is an offered option; anything
+39 -4
View File
@@ -12,10 +12,10 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/chat"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/chat"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/web"
)
// videoZ is a video id used by the isolation test for a DIFFERENT user's video.
@@ -137,6 +137,8 @@ func TestChatEntryAffordanceOnSummaryView(t *testing.T) {
html := body(t, do(t, withChat, httptest.NewRequest(http.MethodGet, "/v/"+videoX, nil)))
require.Contains(t, html, "Dig deeper", "the deeper-dive affordance is shown when chat is enabled")
require.Contains(t, html, "/v/"+videoX+"/chat", "it links to this video's chat")
require.Contains(t, html, `id="chat-section"`, "the dock lives on the detail page")
require.Contains(t, html, `hx-get="/v/`+videoX+`/chat"`, "it opens the chat in place (HTMX), not a navigation")
// With no chat backend wired the affordance is absent (routes unmounted).
noChat := newApp(t)
@@ -145,6 +147,39 @@ func TestChatEntryAffordanceOnSummaryView(t *testing.T) {
require.NotContains(t, html, "Dig deeper", "no affordance when chat is disabled")
}
// The summary and the chat live together (the integrated UX): the no-JS chat page
// renders the full summary alongside the chat, and the HTMX reveal returns just
// the open chat section as a fragment so it docks in below the summary already on
// screen — the summary is never navigated away from.
func TestChatIntegratedWithSummaryOnSamePage(t *testing.T) {
ctx := context.Background()
chatter := &fakeChatter{models: []string{"phi4-mini", "gemma4-26b"}}
app := newChatApp(t, chatter, nil)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "SUMMARY-BODY-MARKER"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
seedTranscript(t, app.Store.(*store.Store), videoX, "the transcript")
// No-JS full page: the summary payload and the chat are on one page.
html := body(t, getChat(t, app, videoX))
require.Contains(t, html, "SUMMARY-BODY-MARKER", "the summary text is shown on the chat page")
require.Contains(t, html, "takeaway one", "takeaways shown alongside the chat")
require.Contains(t, html, "highlight one", "highlights shown alongside the chat")
require.Contains(t, html, "Ask about this video", "the chat sits on the same page as the summary")
require.Contains(t, html, `name="question"`, "the ask form is present")
// HTMX reveal: the open chat section ONLY (a fragment) — no full-page chrome and
// no duplicated summary, so it swaps in below the summary already rendered.
req := httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/chat", nil)
req.Header.Set("HX-Request", "true")
frag := body(t, do(t, app, req))
require.NotContains(t, frag, "<html", "the reveal is a fragment, not a full page")
require.NotContains(t, frag, "SUMMARY-BODY-MARKER", "the reveal does not re-send the summary (it's already on screen)")
require.Contains(t, frag, `id="chat-section"`, "the fragment replaces the dock in place")
require.Contains(t, frag, `name="question"`, "the ask form is in the revealed section")
}
// THE KEY SAFETY ASSERTION (ADR-027): a chat answer is produced entirely from the
// stored transcript — the model receives the stored text, and neither the
// summarize→fetch path nor the YouTube fetch path is ever touched. The tripwire
+2 -2
View File
@@ -12,8 +12,8 @@ import (
"sync"
"time"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/auth"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/auth"
)
// Connections is the narrow write port the connect flow depends on (Clean
+3 -3
View File
@@ -11,9 +11,9 @@ import (
"github.com/stretchr/testify/require"
"golang.org/x/oauth2"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/auth"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/auth"
"git.d-ma.be/mathias/tapir/internal/web"
)
// fakeWriter is a TokenWriter capturing the persisted (ref, value).
+46 -2
View File
@@ -11,8 +11,8 @@ import (
"github.com/a-h/templ"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// Store is the read/write surface the web handlers depend on — a narrow port over
@@ -151,6 +151,8 @@ func (a *App) Router() http.Handler {
app := http.NewServeMux()
app.HandleFunc("GET /{$}", a.handleList)
app.HandleFunc("GET /v/{videoId}", a.handleDetail)
app.HandleFunc("GET /v/{videoId}/expand", a.handleExpand)
app.HandleFunc("GET /v/{videoId}/card", a.handleCard)
app.HandleFunc("POST /v/{videoId}/action", a.handleAction)
app.HandleFunc("POST /v/{videoId}/summarize", a.handleRequestSummarize)
app.HandleFunc("POST /v/{videoId}/retry-now", a.handleRetryNow)
@@ -283,6 +285,48 @@ func (a *App) handleDetail(w http.ResponseWriter, r *http.Request) {
a.render(w, r, DetailPage(*row, a.Chat != nil))
}
// handleExpand returns the inline-expanded card fragment — the full summary +
// chat dock swapped into the list card in place (ADR-031). Only summarized videos
// have a summary to expand; a non-summarized id is a 404 (the compact card never
// offers expand for it).
func (a *App) handleExpand(w http.ResponseWriter, r *http.Request) {
userID, ok := a.currentUserID(w, r)
if !ok {
return
}
videoID := r.PathValue("videoId")
row, err := a.Store.GetSummaryByVideo(r.Context(), userID, videoID)
if errors.Is(err, store.ErrNotFound) {
http.NotFound(w, r)
return
}
if err != nil {
a.serverError(w, r, "get summary", err)
return
}
a.render(w, r, expandedCard(*row, a.Chat != nil))
}
// handleCard returns the compact card fragment — the collapse target that returns
// an expanded card to its compact form in the list (ADR-031).
func (a *App) handleCard(w http.ResponseWriter, r *http.Request) {
userID, ok := a.currentUserID(w, r)
if !ok {
return
}
videoID := r.PathValue("videoId")
row, err := a.Store.GetVideoRow(r.Context(), userID, videoID)
if errors.Is(err, store.ErrNotFound) {
http.NotFound(w, r)
return
}
if err != nil {
a.serverError(w, r, "get video", err)
return
}
a.render(w, r, VideoCard(*row))
}
// handleAction toggles one action: re-clicking an active verb clears it, else it
// is set (the store enforces watched↔skipped exclusion atomically). It returns
// the refreshed button-group fragment for HTMX; without JS it redirects back to
+12 -3
View File
@@ -16,9 +16,9 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/web"
)
// dsn points at the in-process Postgres started in TestMain. Handler tests run
@@ -524,3 +524,12 @@ func TestListAutoModeBannerCopy(t *testing.T) {
require.Contains(t, html, "land gradually")
require.NotContains(t, html, "are not summarized automatically")
}
// TestMetricsNotOnPublicMux: the public app router exposes no /metrics route —
// Prometheus is served on the dedicated metrics port only (ADR-030, security R6).
func TestMetricsNotOnPublicMux(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
rec := do(t, app, httptest.NewRequest(http.MethodGet, "/metrics", nil))
require.Equal(t, http.StatusNotFound, rec.Code, "/metrics must not be on the public mux")
}
+107
View File
@@ -0,0 +1,107 @@
package web_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// TestExpandReturnsSummaryBodyFragment: GET /v/{id}/expand returns the full
// summary as an in-place card fragment (not a full page) — ADR-031.
func TestExpandReturnsSummaryBodyFragment(t *testing.T) {
ctx := context.Background()
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "the full summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/expand", nil)))
require.Contains(t, html, "the full summary text")
require.Contains(t, html, "Takeaways")
require.Contains(t, html, "highlight one")
require.Contains(t, html, "card-expanded", "rendered as the expanded card")
require.Contains(t, html, "/v/"+videoX+"/card", "carries a collapse affordance")
require.NotContains(t, html, "<html", "fragment, not a full page")
}
// TestCollapseReturnsCompactCard: GET /v/{id}/card returns the compact card with
// the expand affordance — the collapse target.
func TestCollapseReturnsCompactCard(t *testing.T) {
ctx := context.Background()
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/card", nil)))
require.Contains(t, html, `class="card"`, "compact card")
require.Contains(t, html, "/v/"+videoX+"/expand", "compact card offers expand")
require.NotContains(t, html, "card-expanded")
require.NotContains(t, html, "<html", "fragment, not a full page")
}
// TestExpandedCardOffersChatDock: with chat enabled, the expanded card includes
// the deeper-dive chat affordance.
func TestExpandedCardOffersChatDock(t *testing.T) {
ctx := context.Background()
app := newChatApp(t, &fakeChatter{models: []string{"m"}}, nil)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/expand", nil)))
require.Contains(t, html, "/v/"+videoX+"/chat", "expanded card wires the chat dock")
}
// TestCompactCardExpandOnlyWhenSummarized: a not-yet-summarized card shows its
// summarize footer and no expand affordance.
func TestCompactCardExpandOnlyWhenSummarized(t *testing.T) {
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
seedVideo(t, p, videoX, "Pending Title", "https://x", time.Time{}) // no summary
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/card", nil)))
require.Contains(t, html, "Not summarized")
require.NotContains(t, html, "/v/"+videoX+"/expand", "pending card offers no expand")
}
// TestCompactCardHasNoJSDetailFallback: the expand affordance carries an href to
// the detail page, so JS-off users still reach the full summary.
func TestCompactCardHasNoJSDetailFallback(t *testing.T) {
ctx := context.Background()
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/card", nil)))
require.Contains(t, html, `href="/v/`+videoX+`"`, "no-JS fallback to the detail page")
require.Contains(t, html, "/v/"+videoX+"/expand", "and the HTMX expand for JS users")
}
// TestDetailAndExpandShareSummaryBody: the detail page and the expanded card render
// the same summary body (one shared fragment, no drift).
func TestDetailAndExpandShareSummaryBody(t *testing.T) {
ctx := context.Background()
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "shared summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
detail := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX, nil)))
expand := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/expand", nil)))
for _, want := range []string{"shared summary text", "Takeaways", "highlight one"} {
require.Contains(t, detail, want)
require.Contains(t, expand, want)
}
}
+39 -45
View File
@@ -7,10 +7,11 @@
// Authentication is real (Dex OIDC) and is the only gate: any Dex-authenticated
// subject may sign in (ADR-012 dropped ADR-011's single-subject allowlist).
// Authorization/registration is layered on top in internal/web (an authenticated
// subject with no tapir user is routed to registration). Sessions are server-side
// (in-memory, fine for the single Stage-1 replica) addressed by an HMAC-signed
// (HS256) HttpOnly Secure SameSite=Lax cookie with a short TTL and sliding
// refresh. Tokens are never logged.
// subject with no tapir user is routed to registration). Sessions are STATELESS
// (ADR-029): the identity + expiry live inside an HMAC-signed (HS256) HttpOnly
// Secure SameSite=Lax persistent cookie with a long sliding TTL — no server-side
// table, so a deploy/restart never logs anyone out and the cookie also survives
// browser-close. Tokens are never logged; logout clears the cookie client-side.
//
// This is mcp-chassis's cousin but NOT the same code: mcp-chassis validates
// inbound Bearer JWTs for MCP APIs; this is a browser session login.
@@ -26,7 +27,8 @@ import (
"github.com/coreos/go-oidc/v3/oidc"
"golang.org/x/oauth2"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/metrics"
"git.d-ma.be/mathias/tapir/internal/web"
)
// Config is the OIDC + session configuration. cmd/tapir maps these from
@@ -47,7 +49,11 @@ type Config struct {
}
const (
defaultSessionTTL = time.Hour
// defaultSessionTTL is generous and sliding: Tapir is a "check back tomorrow"
// reader, so a short TTL meant a re-login (full IdP redirect dance) on almost
// every visit. 30 days, slid forward on each request, keeps a regular user
// logged in indefinitely while an abandoned session still lapses.
defaultSessionTTL = 30 * 24 * time.Hour
pendingTTL = 10 * time.Minute
sessionCookie = "tapir_session"
loginPath = "/auth/login"
@@ -59,7 +65,6 @@ type DexAuth struct {
oauth *oauth2.Config
verifier *oidc.IDTokenVerifier
sessions *sessionStore
pending *pendingStore
secret []byte
sessionTTL time.Duration
@@ -127,7 +132,6 @@ func New(ctx context.Context, cfg Config, opts ...Option) (*DexAuth, error) {
RedirectURL: cfg.RedirectURL,
Scopes: []string{oidc.ScopeOpenID, "profile", "email"},
},
sessions: newSessionStore(),
pending: newPendingStore(),
secret: []byte(cfg.SessionSecret),
sessionTTL: defaultSessionTTL,
@@ -159,31 +163,31 @@ func (d *DexAuth) Middleware(h http.Handler) http.Handler {
h.ServeHTTP(w, r)
return
}
sid, ok := d.sessionID(r)
c, err := r.Cookie(sessionCookie)
if err != nil {
d.redirectUnauthenticated(w, r)
return
}
user, _, ok := d.decodeSession(c.Value, d.now())
if !ok {
d.redirectUnauthenticated(w, r)
return
}
if _, ok := d.sessions.get(sid, d.now()); !ok {
d.redirectUnauthenticated(w, r)
return
}
d.sessions.refresh(sid, d.now().Add(d.sessionTTL)) // sliding refresh
// Sliding refresh: re-issue the cookie with a fresh expiry so an active
// user never lapses (the expiry lives in the cookie, so sliding = re-sign).
d.setSessionCookie(w, d.encodeSession(user, d.now().Add(d.sessionTTL)))
h.ServeHTTP(w, r)
})
}
// CurrentUser resolves the authenticated principal from the session cookie.
// CurrentUser resolves the authenticated principal from the stateless cookie.
func (d *DexAuth) CurrentUser(r *http.Request) (web.User, bool) {
sid, ok := d.sessionID(r)
if !ok {
c, err := r.Cookie(sessionCookie)
if err != nil {
return web.User{}, false
}
data, ok := d.sessions.get(sid, d.now())
if !ok {
return web.User{}, false
}
return data.user, true
user, _, ok := d.decodeSession(c.Value, d.now())
return user, ok
}
func (d *DexAuth) handleLogin(w http.ResponseWriter, r *http.Request) {
@@ -248,23 +252,16 @@ func (d *DexAuth) handleCallback(w http.ResponseWriter, r *http.Request) {
}
_ = idToken.Claims(&claims) // email is best-effort; subject is the identity
sid, err := randToken()
if err != nil {
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
d.sessions.put(sid, sessionData{
user: web.User{Subject: idToken.Subject, Email: claims.Email},
expiry: d.now().Add(d.sessionTTL),
})
d.setSessionCookie(w, sid)
user := web.User{Subject: idToken.Subject, Email: claims.Email}
d.setSessionCookie(w, d.encodeSession(user, d.now().Add(d.sessionTTL)))
metrics.IncLogin()
http.Redirect(w, r, "/", http.StatusFound)
}
func (d *DexAuth) handleLogout(w http.ResponseWriter, r *http.Request) {
if sid, ok := d.sessionID(r); ok {
d.sessions.delete(sid)
}
// Stateless sessions: clearing the cookie logs the browser out. There is no
// server-side record to delete (ADR-029); a copy of the cookie stays valid
// until its expiry — an accepted trade for the Stage-0 reader app.
d.clearSessionCookie(w)
// Land on the public landing page, not the login endpoint: a just-logged-out
// visitor should see /welcome, not be bounced straight back into a Dex login.
@@ -287,22 +284,19 @@ func (d *DexAuth) redirectToLogin(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, loginPath, http.StatusFound)
}
func (d *DexAuth) sessionID(r *http.Request) (string, bool) {
c, err := r.Cookie(sessionCookie)
if err != nil {
return "", false
}
return d.unsign(c.Value)
}
func (d *DexAuth) setSessionCookie(w http.ResponseWriter, sid string) {
// setSessionCookie writes the signed session value as a PERSISTENT cookie
// (Max-Age set), so it survives the browser/app being closed — a session cookie
// (no Max-Age) was dropped on iPhone Safari close, forcing re-login. value is the
// already-signed payload from encodeSession.
func (d *DexAuth) setSessionCookie(w http.ResponseWriter, value string) {
http.SetCookie(w, &http.Cookie{
Name: sessionCookie,
Value: d.sign(sid),
Value: value,
Path: "/",
HttpOnly: true,
Secure: !d.insecure,
SameSite: http.SameSiteLaxMode,
MaxAge: int(d.sessionTTL.Seconds()),
})
}
+35 -6
View File
@@ -14,8 +14,8 @@ import (
josev4 "github.com/go-jose/go-jose/v4"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/web"
"gitea.d-ma.be/mathias/tapir/internal/web/oidc"
"git.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/web/oidc"
)
const (
@@ -306,13 +306,42 @@ func TestLogoutClearsSession(t *testing.T) {
require.Equal(t, http.StatusFound, rec.Code)
require.Equal(t, "/welcome", rec.Header().Get("Location"), "logout lands on the public page")
cleared := sessionCookie(t, rec.Result())
require.Less(t, cleared.MaxAge, 0, "logout expires the cookie")
require.Less(t, cleared.MaxAge, 0, "logout expires the cookie so the browser drops it")
require.Empty(t, cleared.Value, "logout blanks the cookie value")
// The server-side session is gone: the original cookie no longer resolves.
// Sessions are stateless (ADR-029): logout clears the cookie client-side, so a
// request carrying the cleared (empty) cookie is unauthenticated. The original
// signed cookie remains technically valid until its expiry — the accepted
// trade for no server-side store; the browser no longer holds it.
check := httptest.NewRequest(http.MethodGet, "/", nil)
check.AddCookie(cookie)
check.AddCookie(cleared)
_, ok := auth.CurrentUser(check)
require.False(t, ok)
require.False(t, ok, "the cleared cookie does not authenticate")
}
// TestSessionSurvivesRestart is the core of ADR-029: a cookie issued by one
// process is accepted by a FRESH instance with the same session secret — so a
// deploy/pod-restart no longer logs users out (the old in-memory store did).
func TestSessionSurvivesRestart(t *testing.T) {
f := newFakeIssuer(t)
auth1 := newAuth(t, f)
cookie := authenticate(t, auth1, f)
auth2 := newAuth(t, f) // simulate a redeploy: new process, same SessionSecret
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(cookie)
user, ok := auth2.CurrentUser(req)
require.True(t, ok, "a session must survive a restart (stateless signed cookie)")
require.Equal(t, testSubject, user.Subject)
}
// TestSessionCookieIsPersistent: the cookie carries a positive Max-Age so it
// survives the browser/app being closed (a session cookie was dropped on iOS).
func TestSessionCookieIsPersistent(t *testing.T) {
f := newFakeIssuer(t)
auth := newAuth(t, f)
cookie := authenticate(t, auth, f)
require.Greater(t, cookie.MaxAge, 0, "session cookie must be persistent (Max-Age set)")
}
func TestExpiredSessionRejected(t *testing.T) {
+31 -42
View File
@@ -6,12 +6,13 @@ import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
"sync"
"time"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/web"
)
// randToken returns a URL-safe 256-bit random string for session IDs, OIDC
@@ -53,56 +54,44 @@ func (d *DexAuth) unsign(signed string) (string, bool) {
return value, true
}
// sessionData is the server-side session record.
type sessionData struct {
user web.User
expiry time.Time
// sessionClaims is the self-contained session payload carried INSIDE the signed
// cookie — there is no server-side session table. This is deliberate (ADR-029):
// an in-memory store was wiped on every pod restart, logging every user out on
// each deploy, and a stateless cookie also survives browser-close and works
// across replicas. It holds only the identity (subject + email, not secret) and
// an absolute expiry; the HMAC tag (sign/unsign) makes it tamper-proof.
type sessionClaims struct {
Sub string `json:"s"`
Email string `json:"e"`
Exp int64 `json:"x"` // unix seconds; absolute expiry
}
// sessionStore is an in-memory session table. Single replica at Stage 0, so an
// in-process map is sufficient; it is safe for concurrent use.
type sessionStore struct {
mu sync.Mutex
m map[string]sessionData
// encodeSession produces the signed cookie value for a user with the given expiry.
func (d *DexAuth) encodeSession(u web.User, exp time.Time) string {
b, _ := json.Marshal(sessionClaims{Sub: u.Subject, Email: u.Email, Exp: exp.Unix()})
return d.sign(base64.RawURLEncoding.EncodeToString(b))
}
func newSessionStore() *sessionStore { return &sessionStore{m: make(map[string]sessionData)} }
func (s *sessionStore) put(id string, d sessionData) {
s.mu.Lock()
defer s.mu.Unlock()
s.m[id] = d
}
// get returns the session if present and unexpired; expired entries are evicted.
func (s *sessionStore) get(id string, now time.Time) (sessionData, bool) {
s.mu.Lock()
defer s.mu.Unlock()
d, ok := s.m[id]
// decodeSession verifies the cookie's HMAC, parses the claims, and checks expiry.
// It returns the user and the absolute expiry on success.
func (d *DexAuth) decodeSession(cookieValue string, now time.Time) (web.User, time.Time, bool) {
payload, ok := d.unsign(cookieValue)
if !ok {
return sessionData{}, false
return web.User{}, time.Time{}, false
}
if !now.Before(d.expiry) {
delete(s.m, id)
return sessionData{}, false
raw, err := base64.RawURLEncoding.DecodeString(payload)
if err != nil {
return web.User{}, time.Time{}, false
}
return d, true
var c sessionClaims
if err := json.Unmarshal(raw, &c); err != nil {
return web.User{}, time.Time{}, false
}
// refresh slides an existing session's expiry forward; a no-op for unknown ids.
func (s *sessionStore) refresh(id string, expiry time.Time) {
s.mu.Lock()
defer s.mu.Unlock()
if d, ok := s.m[id]; ok {
d.expiry = expiry
s.m[id] = d
exp := time.Unix(c.Exp, 0)
if !now.Before(exp) {
return web.User{}, time.Time{}, false // expired
}
}
func (s *sessionStore) delete(id string) {
s.mu.Lock()
defer s.mu.Unlock()
delete(s.m, id)
return web.User{Subject: c.Sub, Email: c.Email}, exp, true
}
// pendingData holds the nonce bound to an in-flight authorization request.
+1 -1
View File
@@ -11,7 +11,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// fakeFetcher is a web.VideoFetcher returning a fixed video (or an error),
+2 -1
View File
@@ -3,9 +3,10 @@ package web
import (
"bytes"
"context"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"strings"
"testing"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
func TestParseYouTubeVideoID(t *testing.T) {
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"context"
"sync"
"gitea.d-ma.be/mathias/tapir/internal/domain"
"git.d-ma.be/mathias/tapir/internal/domain"
)
// Processor runs the core summarization use case for a single already-discovered
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/web"
)
// fakeProcessor records ProcessVideo calls. With block set it parks until the
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"strings"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
func renderVideoCard(t *testing.T, r store.SummaryRow) string {
+84 -34
View File
@@ -9,7 +9,7 @@ import (
"github.com/a-h/templ"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// youtubeIDRe matches a canonical 11-char YouTube video id (the provider's
@@ -190,6 +190,18 @@ func chatURL(videoID string) templ.SafeURL {
return templ.SafeURL("/v/" + videoID + "/chat")
}
// expandURL builds the inline-expand fragment path (GET) — the full summary + chat
// dock swapped into the list card in place (ADR-031).
func expandURL(videoID string) templ.SafeURL {
return templ.SafeURL("/v/" + videoID + "/expand")
}
// cardURL builds the compact-card fragment path (GET) — the collapse target that
// returns an expanded card to its compact form (ADR-031).
func cardURL(videoID string) templ.SafeURL {
return templ.SafeURL("/v/" + videoID + "/card")
}
// Charmbracelet-inspired palette for the summarizing animation (TapirSpinner) —
// a charm purple box, pink tapir, mint snout/eyes/progress. Kept as named consts
// so the inline span colours and the CSS track/fill share one source of truth.
@@ -566,33 +578,71 @@ func (f Filter) apply(rows []store.SummaryRow) []store.SummaryRow {
// not as template expressions — so the CSS is rendered as a raw node instead).
var styleTag = "<style>" + stylesheet + "</style>"
// The design system is a small set of CSS custom properties: one accent, a
// 4/8px-derived spacing scale, a single radius, and full light+dark palettes so
// color-scheme: light dark is actually honoured (review #1). Muted is #595959
// (~7:1 on white) / #9aa0a8 on dark to clear WCAG AA (review #4).
// themeScript powers the light/dark toggle (ADR-032). The init runs in <head>
// before paint: if the visitor has a stored choice it is applied as data-theme
// immediately, so there is no flash of the wrong palette; with no stored choice
// nothing is set and the CSS @media (prefers-color-scheme) default takes over.
// tapirToggleTheme flips to the opposite of the *effective* theme (reading
// matchMedia when no explicit choice is set yet) and persists it. localStorage
// access is guarded so a privacy-locked browser degrades to the OS default.
const themeScript = `
(function(){try{var t=localStorage.getItem('theme');if(t==='dark'||t==='light')document.documentElement.setAttribute('data-theme',t);}catch(e){}})();
function tapirToggleTheme(){var d=document.documentElement,c=d.getAttribute('data-theme');if(!c){c=window.matchMedia&&window.matchMedia('(prefers-color-scheme: dark)').matches?'dark':'light';}var n=c==='dark'?'light':'dark';d.setAttribute('data-theme',n);try{localStorage.setItem('theme',n);}catch(e){}}
`
// themeScriptTag is the init/toggle script, injected verbatim in <head>.
var themeScriptTag = "<script>" + themeScript + "</script>"
// themeToggleButton is the header control that calls tapirToggleTheme. Shared by
// the authenticated and public layouts so the toggle is on every page.
const themeToggleButton = `<button type="button" class="theme-toggle" onclick="tapirToggleTheme()" aria-label="Toggle light or dark theme" title="Toggle light/dark">◐</button>`
// The design system is one charm-reader layout in two palettes (ADR-032, #17):
// a warm "reader" light theme (sketch B) and a "cozy terminal" dark theme
// (sketch C), both expressed as CSS custom properties. The palette is selected
// three ways, in cascade order: the :root light default; an OS-preference dark
// block that applies only when the visitor has made no explicit choice
// (:root:not([data-theme])); and an explicit :root[data-theme="dark"|"light"]
// set by the persisted toggle, which outranks the media query by specificity.
// charmDarkVars is declared once and injected in both dark selectors so the two
// never drift. Muted clears WCAG AA on each background (#5b5968 on cream,
// #a59cc0 on the dark card). Error/danger shades are tokens so they follow the
// theme too, instead of needing per-block dark overrides.
const charmDarkVars = `
--bg:#16131d; --card:#1e1a28; --fg:#ece7f5; --muted:#a59cc0; --line:#322b44;
--mint:#2ee6b6; --purple:#9d7bff; --pink:#ff6bdb; --cream:#f3ead8;
--accent:#9d7bff; --accent-fg:#16131d; --accent-weak:#2a2340;
--badge-bg:#caa24a; --badge-fg:#1a1300;
--err-bg:#3a1714; --err-fg:#f3b5ae; --err-line:#a6362e;`
const stylesheet = `
:root {
color-scheme: light dark;
--bg:#fbfbfa; --card:#ffffff; --fg:#1a1a1a; --muted:#595959; --line:#e4e4e1;
--accent:#2b6cb0; --accent-fg:#ffffff; --accent-weak:#eaf1f8;
--bg:#faf7f2; --card:#ffffff; --fg:#1c1b22; --muted:#5b5968; --line:#e7e2d8;
--mint:#0bbf8c; --purple:#6a4cf0; --pink:#e0379a; --cream:#1c1b22;
--accent:#6a4cf0; --accent-fg:#ffffff; --accent-weak:#efebfd;
--badge-bg:#e7a13a; --badge-fg:#2a1d00;
--err-bg:#fce8e6; --err-fg:#8a1c10; --err-line:#d9534f;
--s1:.25rem; --s2:.5rem; --s3:1rem; --s4:1.5rem; --s5:2.5rem; --radius:.5rem;
--mono: ui-monospace, SFMono-Regular, Menlo, "Cascadia Code", monospace;
}
@media (prefers-color-scheme: dark) {
:root {
--bg:#15171b; --card:#1e2128; --fg:#e7e7e4; --muted:#9aa0a8; --line:#2d313a;
--accent:#76aae6; --accent-fg:#0c0f13; --accent-weak:#222b38;
--badge-bg:#caa24a; --badge-fg:#1a1300;
:root:not([data-theme]) {` + charmDarkVars + `
}
}
:root[data-theme="dark"] {` + charmDarkVars + `
}
* { box-sizing: border-box; }
body { font: 15px/1.6 system-ui, -apple-system, sans-serif; margin: 0; color: var(--fg); background: var(--bg); }
a { color: var(--accent); text-decoration: none; }
a:hover, a:focus-visible { text-decoration: underline; }
a:visited { color: var(--accent); }
header { padding: var(--s3) var(--s4); border-bottom: 1px solid var(--line); background: var(--card); display: flex; align-items: center; justify-content: space-between; gap: var(--s3); }
.brand { font-weight: 700; font-size: 1.05rem; color: var(--accent); }
.nav { display: flex; gap: var(--s3); font-size: .9rem; }
.brand { font-weight: 800; font-size: 1.05rem; letter-spacing: -.01em; color: var(--accent); }
.nav { display: flex; gap: var(--s3); align-items: center; font-size: .9rem; }
.theme-toggle { font: inherit; font-size: 1rem; line-height: 1; padding: .3rem .5rem; border: 1px solid var(--line); border-radius: 999px; background: var(--card); color: var(--muted); cursor: pointer; }
.theme-toggle:hover { border-color: var(--accent); color: var(--accent); }
.theme-toggle:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; }
main { max-width: 60rem; margin: 0 auto; padding: var(--s4) var(--s3); }
.muted { color: var(--muted); }
@@ -603,7 +653,7 @@ main { max-width: 60rem; margin: 0 auto; padding: var(--s4) var(--s3); }
.filters input:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; border-color: var(--accent); }
.filter-check { flex-direction: row !important; align-items: center; gap: var(--s2) !important; padding-bottom: .45rem; }
.filter-check input[type=checkbox] { width: 1rem; height: 1rem; min-width: 0; padding: 0; accent-color: var(--accent); cursor: pointer; }
.btn { font: inherit; font-weight: 600; padding: .45rem 1rem; border: 1px solid var(--accent); border-radius: var(--radius); background: var(--accent); color: var(--accent-fg); cursor: pointer; }
.btn { font: inherit; font-weight: 600; padding: .45rem 1.1rem; border: 1px solid var(--accent); border-radius: 999px; background: var(--accent); color: var(--accent-fg); cursor: pointer; }
/* anchors styled as buttons: the generic a{} / a:visited{} colour rules outrank
.btn on <a>, painting the label accent-on-accent (invisible). Restore the
button foreground for anchor buttons, visited included. */
@@ -615,14 +665,19 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
.cards { list-style: none; margin: 0; padding: 0; display: grid; gap: var(--s3); }
.card { background: var(--card); border: 1px solid var(--line); border-radius: var(--radius); padding: var(--s3) var(--s4); display: flex; flex-direction: column; gap: var(--s2); }
.card-title { font-size: 1.1rem; font-weight: 600; line-height: 1.3; }
.card-meta { color: var(--muted); font-size: .85rem; }
.card-meta { color: var(--muted); font-size: .8rem; font-family: var(--mono); }
.card-preview { color: var(--muted); font-size: .9rem; line-height: 1.5; display: -webkit-box; -webkit-line-clamp: 1; line-clamp: 1; -webkit-box-orient: vertical; overflow: hidden; }
.card-foot { display: flex; gap: var(--s2); align-items: center; flex-wrap: wrap; margin-top: var(--s1); }
/* Inline-expanded card (ADR-031/032): lifted, with a charm accent edge so the
open card reads as the focused one in the feed (sketch B/C). */
.card-expanded { border-color: var(--accent); border-left: 3px solid var(--mint); box-shadow: 0 6px 24px color-mix(in srgb, var(--accent) 14%, transparent); }
.card-expanded-head { display: flex; justify-content: space-between; align-items: baseline; gap: var(--s2); }
.card-collapse { font-size: .8rem; font-family: var(--mono); white-space: nowrap; }
.chip { display: inline-block; padding: .15rem .55rem; border-radius: 999px; background: var(--accent-weak); color: var(--accent); font-size: .72rem; font-weight: 600; }
/* passive "retrying later" chip: dim/grey (CharmDim), not the accent — it is a
status, not an action the user can take. */
.chip-retry { background: rgba(108, 108, 108, .16); color: #6c6c6c; }
.pipeline-bar { display: flex; gap: var(--s3); align-items: center; flex-wrap: wrap; margin-bottom: var(--s3); font-size: .8rem; color: var(--muted); }
.pipeline-bar { display: flex; gap: var(--s3); align-items: center; flex-wrap: wrap; margin-bottom: var(--s3); font-size: .78rem; font-family: var(--mono); color: var(--muted); }
.pipeline-bar span { display: flex; align-items: center; gap: var(--s1); }
.pipeline-bar span + span::before { content: "·"; margin-right: var(--s1); }
.pipeline-note { margin: calc(-1 * var(--s2)) 0 var(--s3); font-size: .8rem; line-height: 1.5; max-width: 40rem; }
@@ -708,10 +763,7 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
/* flash / notification banner */
.flash { padding: var(--s2) var(--s3); border-radius: var(--radius); margin-bottom: var(--s4); font-size: .92rem; border: 1px solid var(--line); }
.flash-success { background: var(--accent-weak); color: var(--accent); border-color: var(--accent); }
.flash-error { background: #fce8e6; color: #8a1c10; border-color: #d9534f; }
@media (prefers-color-scheme: dark) {
.flash-error { background: #3a1714; color: #f3b5ae; border-color: #a6362e; }
}
.flash-error { background: var(--err-bg); color: var(--err-fg); border-color: var(--err-line); }
/* htmx loading feedback */
.htmx-indicator { opacity: 0; transition: opacity .2s; color: var(--muted); font-size: .8rem; }
@@ -721,19 +773,22 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
.detail { max-width: 38rem; }
.detail .back { margin: 0 0 var(--s3); font-size: .85rem; }
.detail h1 { font-size: 1.7rem; line-height: 1.25; margin: 0 0 var(--s2); }
.detail .meta { color: var(--muted); font-size: .9rem; margin: 0 0 var(--s2); display: flex; gap: var(--s2); align-items: center; flex-wrap: wrap; }
.detail .meta { color: var(--muted); font-size: .82rem; font-family: var(--mono); margin: 0 0 var(--s2); display: flex; gap: var(--s2); align-items: center; flex-wrap: wrap; }
.detail .source { margin: 0 0 var(--s4); font-size: .9rem; }
.detail .embed { margin: 0 0 var(--s4); aspect-ratio: 16 / 9; border-radius: var(--radius); overflow: hidden; background: #000; border: 1px solid var(--line); }
.detail .embed iframe { display: block; width: 100%; height: 100%; border: 0; }
.detail section { margin-top: var(--s4); }
.detail section h2 { font-size: .78rem; text-transform: uppercase; letter-spacing: .05em; color: var(--muted); border-top: 1px solid var(--line); padding-top: var(--s3); margin: 0 0 var(--s2); }
.detail section h2 { font-size: .72rem; text-transform: uppercase; letter-spacing: .12em; color: var(--accent); display: flex; align-items: center; gap: var(--s2); margin: var(--s4) 0 var(--s2); }
.detail section h2::after { content: ""; flex: 1; height: 1px; background: var(--line); }
.detail .body { white-space: pre-wrap; line-height: 1.7; margin: 0; }
.detail ul { margin: 0; padding-left: 1.2rem; line-height: 1.6; }
.detail li { margin-bottom: var(--s1); }
/* deeper-dive chat (ADR-027) */
.dig-deeper { margin: var(--s3) 0 0; }
.chat .chat-scope { margin: 0 0 var(--s4); font-size: .9rem; }
/* deeper-dive chat (ADR-027) — docks in place below the summary */
.chat-dock { margin-top: var(--s5); border-top: 1px solid var(--line); padding-top: var(--s4); }
.chat-dock .chat-open { display: inline-block; }
.chat-heading { font-size: 1.1rem; margin: 0 0 var(--s2); }
.chat-scope { margin: 0 0 var(--s3); font-size: .9rem; }
.chat-panel { display: flex; flex-direction: column; gap: var(--s3); }
.chat-log { display: flex; flex-direction: column; gap: var(--s3); }
.chat-turn { border-radius: var(--radius); padding: var(--s2) var(--s3); }
@@ -742,8 +797,7 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
.chat-a { background: var(--card); border: 1px solid var(--line); }
.chat-a .body { white-space: pre-wrap; line-height: 1.6; }
.chat-note { margin: 0; font-size: .82rem; font-style: italic; }
.chat-error { margin: 0; color: #8a1c10; font-size: .9rem; }
@media (prefers-color-scheme: dark) { .chat-error { color: #f3b5ae; } }
.chat-error { margin: 0; color: var(--err-fg); font-size: .9rem; }
.chat-form { display: flex; flex-direction: column; gap: var(--s2); margin: var(--s2) 0 0; }
.chat-model { flex-direction: column; display: flex; gap: var(--s1); font-size: .78rem; text-transform: uppercase; letter-spacing: .04em; color: var(--muted); align-items: flex-start; }
.chat-model select { font: inherit; text-transform: none; letter-spacing: 0; padding: .4rem .55rem; border: 1px solid var(--line); border-radius: var(--radius); background: var(--card); color: var(--fg); }
@@ -793,17 +847,13 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
.danger-zone h2 { border-top-color: #d9534f; }
.confirm-delete > summary { display: inline-block; list-style: none; cursor: pointer; font: inherit; font-weight: 600; padding: .45rem 1rem; border: 1px solid #d9534f; border-radius: var(--radius); background: transparent; color: #c0392b; }
.confirm-delete > summary::-webkit-details-marker { display: none; }
.confirm-delete > summary:hover { background: #fce8e6; }
.confirm-delete > summary:hover { background: var(--err-bg); }
.confirm-delete[open] > summary { margin-bottom: var(--s3); }
.confirm-body { border: 1px solid #d9534f; border-radius: var(--radius); padding: var(--s3); background: #fce8e6; color: #8a1c10; }
.confirm-body { border: 1px solid #d9534f; border-radius: var(--radius); padding: var(--s3); background: var(--err-bg); color: var(--err-fg); }
.btn-danger { font: inherit; font-weight: 600; padding: .45rem 1rem; border: 1px solid #d9534f; border-radius: var(--radius); background: #d9534f; color: #fff; cursor: pointer; }
.btn-danger:hover { filter: brightness(1.05); }
.btn-danger:focus-visible { outline: 2px solid #d9534f; outline-offset: 1px; }
@media (prefers-color-scheme: dark) {
.confirm-body { background: #3a1714; color: #f3b5ae; }
.confirm-delete > summary { color: #f3b5ae; }
.confirm-delete > summary:hover { background: #3a1714; }
}
.confirm-delete > summary { color: var(--err-fg); }
/* public landing page (/welcome) — the Charm-box mascot hero plus the sign-in CTA */
.welcome { text-align: center; padding: var(--s5) var(--s3); display: flex; flex-direction: column; align-items: center; gap: var(--s4); }
+99 -29
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"strings"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"git.d-ma.be/mathias/tapir/internal/adapters/store"
)
// Layout is the shared HTML shell. HTMX drives the progressive interactions
@@ -17,13 +17,14 @@ templ Layout(title string) {
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>{ title }</title>
@templ.Raw(themeScriptTag)
<script src="/static/htmx.min.js" defer></script>
@templ.Raw(styleTag)
</head>
<body>
<header>
<a href="/" class="brand">Tapir</a>
<nav class="nav"><a href="/account">Account</a><a href="/auth/logout">Log out</a></nav>
<nav class="nav"><a href="/account">Account</a><a href="/auth/logout">Log out</a>@templ.Raw(themeToggleButton)</nav>
</header>
<main>
{ children... }
@@ -42,12 +43,14 @@ templ PublicLayout(title string) {
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>{ title }</title>
@templ.Raw(themeScriptTag)
<script src="/static/htmx.min.js" defer></script>
@templ.Raw(styleTag)
</head>
<body>
<header>
<a href="/" class="brand">Tapir</a>
<nav class="nav">@templ.Raw(themeToggleButton)</nav>
</header>
<main>
{ children... }
@@ -264,7 +267,16 @@ templ summaryList(b listBuckets, hasConnected bool, autoSummarize bool) {
templ VideoCard(r store.SummaryRow) {
<li class={ "card", templ.KV("card-pending", !r.Summarized) } id={ "video-" + r.VideoID }>
if r.Summarized {
<div class="card-title"><a href={ videoURL(r.VideoID) }>{ displayTitle(r) }</a></div>
// Expand the full summary + Q&A in place (ADR-031); href is the no-JS
// fallback to the detail page, so nothing becomes JS-only.
<div class="card-title">
<a
href={ videoURL(r.VideoID) }
hx-get={ string(expandURL(r.VideoID)) }
hx-target={ "#video-" + r.VideoID }
hx-swap="outerHTML"
>{ displayTitle(r) }</a>
</div>
} else {
<div class="card-title">{ displayTitle(r) }</div>
}
@@ -327,6 +339,32 @@ templ VideoCard(r store.SummaryRow) {
</li>
}
// expandedCard is a summarized list card opened IN PLACE (ADR-031): the full
// summary body + the deeper-dive chat dock, with a collapse control back to the
// compact card. It shares the <li id> with VideoCard so HTMX swaps it outerHTML,
// and reuses summaryBody + chatReveal so it never drifts from the detail page.
// Note: chatReveal uses a single #chat-section id, so this assumes one card open
// at a time; a per-video chat id is a follow-up if simultaneous expansion is wanted.
templ expandedCard(r store.SummaryRow, chatEnabled bool) {
<li class="card card-expanded" id={ "video-" + r.VideoID }>
<div class="card-expanded-head">
<span class="card-title">{ displayTitle(r) }</span>
<a
href={ videoURL(r.VideoID) }
hx-get={ string(cardURL(r.VideoID)) }
hx-target={ "#video-" + r.VideoID }
hx-swap="outerHTML"
class="card-collapse"
title="Collapse"
>collapse ↑</a>
</div>
@summaryBody(r)
if chatEnabled {
@chatReveal(r.VideoID)
}
</li>
}
// TapirSpinner is the summarizing animation: a Charmbracelet-style TUI panel —
// three richly coloured ASCII tapir frames (inline span colours, snout wiggling
// ∩→∪→~) cross-faded by CSS, plus a lipgloss-style progress bar whose mint fill
@@ -410,14 +448,11 @@ templ noCaptionsCard(r store.SummaryRow) {
</li>
}
// DetailPage is the full summary view: text, highlights, takeaways, metadata,
// and the action button group. chatEnabled adds the "dig deeper" affordance
// (ADR-027) — a quiet link into the per-video chat over the stored transcript.
templ DetailPage(r store.SummaryRow, chatEnabled bool) {
@Layout("Tapir — " + displayTitle(r)) {
<article class="detail">
<p class="back"><a href="/">← Summaries</a></p>
<h1>{ displayTitle(r) }</h1>
// summaryBody is the summary payload shared by the detail page and the no-JS
// chat page (so the chat page shows the same summary, not a separate view):
// metadata, embed, source, the action toggles, then the attention-saving order
// Takeaways → Highlights → Summary (UX review A8).
templ summaryBody(r store.SummaryRow) {
<p class="meta">
if detailMeta(r) != "" {
<span>{ detailMeta(r) }</span>
@@ -441,16 +476,7 @@ templ DetailPage(r store.SummaryRow, chatEnabled bool) {
if r.URL != "" {
<p class="source"><a href={ externalURL(r.URL) } rel="noopener noreferrer">watch on source ↗</a></p>
}
if chatEnabled {
// Where the "I want more" reaction goes (ADR-027). A quiet link, not a
// loud CTA — it deepens value for a reader already here, never nudges.
<p class="dig-deeper"><a class="btn-secondary" href={ chatURL(r.VideoID) }>Dig deeper — ask about this video →</a></p>
}
@ActionButtons(r.VideoID, actionSet(r.Actions))
// Lead with the attention-saving payload: Takeaways ("is this worth my
// time?") first, then Highlights, then the full Summary last (UX review
// A8). Takeaways/Highlights are conditional, so a video without them falls
// through to the Summary leading naturally.
if len(r.Takeaways) > 0 {
<section>
<h2>Takeaways</h2>
@@ -475,21 +501,65 @@ templ DetailPage(r store.SummaryRow, chatEnabled bool) {
<h2>Summary</h2>
<p class="body">{ r.Summary }</p>
</section>
}
// DetailPage is the full summary view: the summary payload, then (when chat is
// enabled) the deeper-dive dock (ADR-027) — a reveal that opens the chat IN PLACE
// below the summary, so the summary stays on screen as the context being asked
// about rather than being navigated away from.
templ DetailPage(r store.SummaryRow, chatEnabled bool) {
@Layout("Tapir — " + displayTitle(r)) {
<article class="detail">
<p class="back"><a href="/">← Summaries</a></p>
<h1>{ displayTitle(r) }</h1>
@summaryBody(r)
if chatEnabled {
@chatReveal(r.VideoID)
}
</article>
}
}
// ChatPage is the per-video deeper-dive chat (ADR-027): a question/answer surface
// over the video's ALREADY-STORED transcript. The back link returns to the
// summary it was entered from. All the interaction lives in chatPanel so the HTMX
// answer-swap and the no-JS full-page render share one component.
templ ChatPage(v chatView) {
@Layout("Tapir — Ask — " + v.Title) {
<article class="detail chat">
<p class="back"><a href={ videoURL(v.VideoID) }>← { v.Title }</a></p>
<h1>Ask about this video</h1>
// chatReveal is the CLOSED dock at the foot of the summary: a quiet affordance,
// not a loud CTA (it deepens value for a reader already here, never nudges). With
// JS it swaps itself for the open chat section in place (HTMX, summary stays
// above); without JS the same href navigates to the full chat page, which renders
// the summary alongside the chat. Either way the summary is never lost.
templ chatReveal(videoID string) {
<section id="chat-section" class="chat-dock">
<a
class="btn-secondary chat-open"
href={ chatURL(videoID) }
hx-get={ string(chatURL(videoID)) }
hx-target="#chat-section"
hx-swap="outerHTML"
>
Dig deeper — ask about this video →
</a>
</section>
}
// chatSection is the OPEN dock: heading + scope note + the chat panel, swapped in
// over the closed reveal (same #chat-section id, outerHTML). It is the HTMX reveal
// response AND the inline chat block on the no-JS chat page.
templ chatSection(v chatView) {
<section id="chat-section" class="chat-dock chat-dock-open">
<h2 class="chat-heading">Ask about this video</h2>
<p class="chat-scope muted">Answers come only from this video's stored transcript — Tapir never fetches anything new here.</p>
@chatPanel(v)
</section>
}
// ChatPage is the no-JS full-page render of the chat: the whole summary followed
// by the open chat dock, so a visitor without JS sees the same integrated view
// (summary beside the conversation) that JS users get inline via the reveal.
templ ChatPage(r store.SummaryRow, v chatView) {
@Layout("Tapir — " + displayTitle(r)) {
<article class="detail">
<p class="back"><a href="/">← Summaries</a></p>
<h1>{ displayTitle(r) }</h1>
@summaryBody(r)
@chatSection(v)
</article>
}
}
+1046 -755
View File
File diff suppressed because it is too large Load Diff
+67
View File
@@ -0,0 +1,67 @@
package web_test
import (
"context"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// page renders any GET path's full HTML (helper for theme/structure assertions).
func page(t *testing.T, app interface {
Router() http.Handler
}, path string,
) string {
t.Helper()
rec := httptest.NewRecorder()
app.Router().ServeHTTP(rec, httptest.NewRequest(http.MethodGet, path, nil))
return rec.Body.String()
}
// TestThemeHasLightAndDarkPalettes: one layout, two palettes via CSS variables —
// a root light palette and a data-theme="dark" dark palette (ADR-032).
func TestThemeHasLightAndDarkPalettes(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
html := page(t, app, "/welcome") // public, no DB needed
require.Contains(t, html, ":root", "light palette on the root")
require.Contains(t, html, `[data-theme="dark"]`, "explicit dark palette override for the toggle")
}
// TestThemeFollowsOSPreference: with no stored choice, the OS preference applies
// the dark palette automatically.
func TestThemeFollowsOSPreference(t *testing.T) {
app := newApp(t)
html := page(t, app, "/welcome")
require.Contains(t, html, "prefers-color-scheme: dark", "OS-preference dark default")
}
// TestThemeTogglePersists: every page carries a theme toggle control and a small
// script that flips data-theme and persists the choice.
func TestThemeTogglePersists(t *testing.T) {
app := newApp(t)
html := page(t, app, "/welcome")
require.Contains(t, html, "theme-toggle", "a theme toggle control")
require.Contains(t, html, "localStorage", "the choice is persisted")
require.Contains(t, html, "data-theme", "the toggle flips data-theme")
}
// TestExpandedCardEmbedsVideo: expanding a summarized card with a valid provider id
// includes the embedded video player (ADR-031/032).
func TestExpandedCardEmbedsVideo(t *testing.T) {
ctx := context.Background()
app := newApp(t)
p := rawPool(t)
resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "summary text"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
html := body(t, do(t, app, httptest.NewRequest(http.MethodGet, "/v/"+videoX+"/expand", nil)))
require.Contains(t, html, "<iframe", "expanded card embeds a player")
require.True(t, strings.Contains(html, "youtube.com/embed") || strings.Contains(html, "youtube-nocookie.com/embed"),
"the embed is the YouTube player")
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/web"
"git.d-ma.be/mathias/tapir/internal/web"
)
// fakeAuth is a configurable web.Auth for the landing-page tests: it reports a
+72
View File
@@ -0,0 +1,72 @@
# spike-media — the `/bygge` toolchain, recovered
Throwaway spike tooling for #28 (upload → transcript → takeaways). Committed
because it was found in `tmpfs` one reboot from gone, not because it is finished.
Nothing here is wired into tapir. No database, no HTTP handler, no ADR.
## What produced `/bygge`
The prototype at `tapir.d-ma.be/bygge` was **not** hand-built, which the epic and
the first round of spike issues both got wrong. It came from these three scripts:
```
IMG_1233.mov --transcode.yaml--> IMG_1233.web.mp4 (k3s Job, h264, +faststart)
IMG_1233.wav --whisper---------> IMG_1233.sv.srt (interactive — the one manual step)
IMG_1233.sv.srt --analyze_srt.py--> analys.json (berget/mistral-medium, temperature=0)
analys.json + video --build_page.py--> page HTML
```
The reference analysis was produced by **`berget/mistral-medium`** — a mid-tier
cloud model, not a frontier one. That matters when judging what a local model
has to clear: the bar is mistral-medium, and it should be re-run as the control
arm rather than excluded.
## The validator is the valuable part
`analyze_srt.py` grades its own output on four checks. Two of them catch classes
of error that timestamp- and citation-checking cannot:
| Check | Catches |
|---|---|
| citation exact / **partial** / absent | Paraphrase presented as a quote. Three tiers, because `alternativ` citations legitimately stitch non-contiguous passages — a strict substring test fails the reference output. |
| `TALET … FINNS INTE I CITATET` | A wrong number carried by a *verbatim* citation and a *real* timestamp. Converts Swedish number-words to digits first. It caught `45×230` where the transcript said `220`. |
| `TACKNINGSLUCKA` | Spans of the clip no section covers. **The recall term** — without it an empty analysis scores perfectly on every other check. |
| timestamp exists in the SRT | Fabricated seek targets. Near-solved: 74/75 on the reference, because the model copies the cue starts it was handed. |
## Known defects — read before running
- **`max_tokens: 6000` is too low.** The 4-minute reference analysis is ~4,700
output tokens, i.e. ~78% of budget. A 10-minute clip truncates for every model,
and truncation reads as a JSON-discipline failure if you are not counting
tokens. Record `finish_reason` and separate `truncated` from `malformed`.
- **`build_page.py` hardcodes the title** (`"Jonas – genomgång av tillbyggnad"`).
- **The schema carries fields nothing renders** — `projekt.sammanfattning`,
`projekt.skede`, `avsnitt.bildtid`, `avsnitt.bildmotiv`, `material[]`,
`oklart[]` — left over from an earlier still-frame/markdown output that
`build_page.py` replaced. They are paid for in output tokens on every run.
- **Speaker attribution is inferred, not diarized.** The output attributes
statements by name from an SRT with no speaker labels. All four checks above
are blind to a misattribution.
- **The web encode does not downscale.** The source is 720×1280; `-preset
veryfast -crf 24` on handheld motion is what makes it ~5 Mbps, not the
resolution. The lever is the preset.
## Fixtures
`fixtures/fake.sv.srt` + `fixtures/fake.analys.json` — a synthetic 7-cue Swedish
construction walkthrough with planted numbers. No real people, no real project.
It exercises the parser and all four validator checks **without a model call**,
which is what makes it usable in a test.
## Why the real artifacts are not here
**This repo is public.** The prototype's transcript and analysis are a verbatim
recording of a named person discussing a client's building project — private
third-party content, and exactly the class of data #28 calls out as the reason
uploaded transcripts cannot live in the shared `transcripts` table.
So `IMG_1233.sv.srt`, `payload.json` and the model-comparison runs stay out of
git here. That leaves `/bygge` without a versioned source of truth, which is a
real gap and a decision for #28: a private repo, an encrypted blob, or accepting
that the reference artifacts live outside version control.

Some files were not shown because too many files have changed in this diff Show More