The copied OpenAI-compatible client sent no max_tokens. Thinking models
(qwen3, deepseek-r1) spend their budget on the reasoning trace and return
EMPTY content when max_tokens is unset, which the summarizer treats as an
error. ADR-004 says change Tapir's copy rather than the hyperguild upstream,
so set a generous default (8192) leaving room for both reasoning and output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements ports.Sink over Postgres (pgx/v5 + pgxpool, DSN from env per
estate convention). This is the primary sink (ADR-003) and the source of
the engine's durable, cross-restart dedup — the in-engine processed map is
process-lifetime only.
- Migrations (golang-migrate, NNN_name.up/down.sql per estate convention,
applied from an embedded FS on New): users, videos, transcripts,
summaries, sink_deliveries. Every user-owned table carries user_id
(Stage-0 per-user isolation promise, data-model.md). summaries has
UNIQUE(user_id, video_id) — at most one summary per video; highlights /
takeaways are jsonb.
- Deliver upserts the summary idempotently on (user_id, video_id)
(ON CONFLICT DO UPDATE) inside one tx with its sink_delivery row. Re-
delivering the same summary updates in place, never duplicates or errors.
- Dedup reads (store methods, not a new port): HasSummary(ctx,userID,
videoID) and SeenVideoIDs(ctx,userID) — both user_id-scoped, so one
user never sees another's videos.
summaries.video_id is intentionally not FK-constrained to videos at Stage 0:
the sink receives only a Summary, so the dedup key stands alone; video-row
persistence is the engine/source's concern, deferred.
Tested against a real in-process Postgres via embedded-postgres (real SQL:
constraints, ON CONFLICT, jsonb, user_id scoping) — no docker, no live
cluster, no creds, fully offline.
Deps: golang-migrate/migrate/v4 and jackc/pgx/v5 (runtime),
fergusstrange/embedded-postgres + stretchr/testify (test-only). go mod tidy
raised the go directive to 1.25.0 (minimum required by the dep graph;
estate elsewhere already runs 1.26.1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements ports.VideoSource against the YouTube Data API v3:
ListSubscriptions (paginated), NewVideos (recent per channel), and
captions-first FetchTranscript — an absent caption track yields
domain.SourceNone (not an error) per ADR-007, with no audio download
or speech-to-text.
OAuth is written fresh on golang.org/x/oauth2 (ADR-006, distinct from
ingestion's inbound MCP auth); the Google token endpoint is inlined to
avoid the heavy x/oauth2/google dep. The per-connection refresh token is
resolved through the SecretStore port from an opaque TokenSecretRef and
is never stored on the adapter or logged.
Unit-tested against an httptest server + fake SecretStore (no live
googleapis egress): subscriptions list/pagination, new-video detection,
captions present -> Source set, captions absent -> SourceNone no error,
and secret-ref resolution failure surfacing as an error.
oauth2 pinned to v0.30.0 to keep the go directive at 1.23.x (koala
runner), not the v0.36 line that requires a newer toolchain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement ports.Summarizer in internal/adapters/summarizer. It routes through a
local Primary endpoint first and an optional BYO Fallback, owning the routing
itself (not delegating to llm.Router) so it can record AIProvider, AIModel, and
FallbackUsed on domain.Summary. Prompt asks for JSON {summary, highlights,
takeaways}; the parser tolerates thinking-model fences/reasoning and rejects an
empty summary.
The summarizer is the single egress point for content toward an AI model, so it
enforces the local-first guarantee from ai_routing.feature: with no BYO
configured (nil fallback) there is no external endpoint, so content reaches the
local stack and nowhere else. Tests assert all four scenarios via a fake client.
Model alias is config (TAPIR_SUMMARIZER_MODEL, host/name) — not hardcoded;
docs/homelab-integration.md notes it stays `confirm` and that thinking models
need an explicit max_tokens or they return empty content.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy hyperguild/ingestion/internal/llm into internal/adapters/llm and own it.
Tapir owes that repo nothing at the dependency level — no module dep added.
Router gives the local-Primary -> BYO-Fallback path needed for ai_routing.feature.
Copied tests rewritten from testify to stdlib testing to keep go.mod
dependency-free (repo has zero deps; acceptance tests are stdlib too).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conductor verified against the live koala k3s cluster before launching the
adapter workers: LiteLLM is in ns ai-stack, ClusterIP 10.43.159.89:4000,
off-cluster NodePort 30401. The doc's 31234 was actually llama-swap. Also
flag that sk-local-123 now 401s — use LITELLM_MASTER_KEY from the vault.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ProcessNewVideos walks a user's subscriptions and processes each newly
seen video. Two remaining .feature scenarios are now covered: a channel
the user is not subscribed to is never surfaced (so never processed), and
a video already processed in this engine's lifetime is not summarized
twice. Dedup is in-memory and per-user; durable cross-restart dedup stays
the store's concern (no new port), per docs/data-model.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve transcript -> summarize -> deliver, or skip when no usable
transcript. Sinks fail independently: a failing sink does not abort the
others and successful deliveries are kept; per-sink errors are returned
joined. Makes the two scaffolded acceptance scenarios GREEN.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
go-task 3.51.1 read the unquoted scalar 'echo "gofmt: files..."' as a
key:value mapping, failing every task with 'invalid keys in command' and
blocking the whole quality gate (not just the acceptance suite). Wrap the
command in an explicit double-quoted YAML scalar.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lists the egress the build assumes (Go module proxy, toolchain download,
raw.githubusercontent for golangci-lint, github for non-proxied modules) and the
runtime assumes (LiteLLM gateway, brain-mcp, YouTube/Vimeo APIs, per-user BYO-AI
hosts only on opt-in), so a locked-down koala act_runner or dev env knows what to
allow or which GOPROXY to set. Notes the claude.ai sandbox allowlist is separate
and unrelated.
Tells an agent to wire skills via `task skills` (gitignored symlinks, never
committed) and which skills matter for Tapir; documents the scaffolded-and-RED
state with the first build task spelled out; flags the unverified setup items
(Go version, brain-mcp URL, secret-ref naming, model alias) to resolve against
the live cluster.
Excludes build artifacts and the .claude/skills symlink (wired by the skills
installer, never committed — matches the mathias/skills convention) and local
env files.
check -> build -> mirror, self-hosted runner, buildah to localhost:5000, k3s
smoke test. Follows the gitea-ci skill template and its act_runner gotchas
(secrets inlined in run:, no heredocs). check will be RED until the engine is
implemented (acceptance suite). Deploy job omitted until k3s manifests exist in
infra. GH_DEPLOY_KEY secret must be set before mirror succeeds.
Defines `task check` (fmt-check + vet + lint + test) — the gate ADR-009 and CI
both invoke. Also `task skills` to wire the engineering skills library via the
canonical installer (symlinks, gitignored). lint no-ops locally when
golangci-lint is absent; CI installs it.
Minimal main that identifies the binary (gives the CI smoke test something to
grep). HTTP server, watcher, adapter wiring, and config come with the build.
Executable translation of docs/use-cases/summarize_new_video.feature: captioned
video is summarized and delivered to the store sink; no-transcript video is
skipped with reason "no transcript" and no delivery. Drives the engine through
fake adapters (no live YouTube/brain). Intentionally RED until the engine is
implemented — this is the swarm's target.
Engine wires the ports and exposes ProcessNewVideo, the core use case. Returns
ErrNotImplemented on purpose so the acceptance suite fails RED — implementing it
to make those tests pass is the first build task. Depends only on ports + domain
(dependencies point inward).
The hexagonal interfaces the engine depends on. Keeps the engine provider- and
sink-agnostic: YouTube/Vimeo implement VideoSource, the AI router implements
Summarizer, store/brain implement Sink, ESO implements SecretStore. This is what
makes standalone-vs-homelab a wiring choice (ADR-003).
Pure domain types matching docs/data-model.md: User, Subscription, Video,
Transcript, Summary, plus Provider and TranscriptSource enums. Stdlib-only,
no outward dependencies — the innermost layer. Video carries user_id per the
per-user-isolation decision (no global dedup).
Go module root for Tapir. Go 1.23 — confirm against the koala act_runner
toolchain; bump to match the estate (ingestion uses 1.26.1) if the runner has it.
Adds a consolidated table of approaches considered and deliberately not taken
(Python, Supabase, living in the monolith, shared-lib lift, filesystem brain
package, reusing inbound oauth, global dedup table, Whisper-in-core, SaaS-now,
swarm-delegating the spike), each mapped to the ADR that settles it. Prevents a
later session from re-proposing settled rejections as fresh ideas.
Operational context for independent agent sessions: orientation order, TBD +
conventional-commit workflow, the three "looks reusable but isn't" traps (llm is
copied not imported, brain sink is HTTP not filesystem, OAuth is fresh), the
settled decisions not to reopen, the Clean Architecture/BDD stance, and a
provenance trail back to the S5 spike and the llm source. Distills the operating
knowledge surfaced during the 2026-06-02 planning+grill session.
Points anyone (or any agent) landing cold at the vision, decisions, architecture,
data model, and BDD feature specs; states the Clean Architecture / TDD-BDD / TBD
approach and the homelab conventions reused. Notes the repo is pre-code and the
docs are the version-controlled design intent.
Gherkin spec for connecting YouTube/Vimeo accounts and configuring optional
per-provider BYO AI credentials: connections sync subscriptions, tokens/keys are
stored only as secret references (never in the clear), revocation stops watching
but preserves history. Encodes the secrets-by-reference and data-isolation
guardrails (ADR-002, ADR-006, data-model).
Gherkin spec for the routing guardrail: local produces the summary by default;
on local failure, fall back only to a user-configured BYO provider; with no BYO,
queue for retry and never send content to a third-party model. Encodes the
local-first/user-owned principle (VISION) as executable behavior.
Gherkin spec for the core use case: captioned video is summarized and delivered;
no-transcript video is recorded as skipped; unsubscribed channels are ignored;
already-summarized videos are not reprocessed. These scenarios seed the use-case
test suite (Clean Architecture core tested through fake adapters).
Per-user-isolated entities (no global cross-tenant video table per the S5/grill
correction), secrets stored by reference only (ESO/1Password, never the token),
brain delivery modelled as a sink_delivery row rather than brain-specific tables.
fallback_used recorded per summary as the Stage 0 quality signal. Future C dedup
and sharding explicitly out of scope.
Context and container diagrams, sequence diagrams for the core summarize-new-video
use case and the local-first/BYO AI routing fallback, and the Clean Architecture
layering. Ports & adapters keep the engine provider- and sink-agnostic, making
standalone-vs-homelab a wiring choice (ADR-003), not two codebases.
Records the decisions from the S5 spike and the Full Grill as append-only ADRs:
Go not Python; no Supabase; standalone-first with brain as one sink; copy the
llm package; brain sink via HTTP brain-mcp; fresh outbound OAuth; captions-first
with STT deferred; Future C deferred behind the Stage 0 gate; trunk-based dev.
The top-level guardrail for Tapir: the problem, the product, the principles
(local-first, standalone-first, attention-as-scarce-resource, data isolation),
who it's for (now / Future B / deferred Future C), and a staged, falsifiable
definition of success with Stage 0 ("useful to me") as the gate.
mathiasandClaude Opus 4.8