Compare commits

...
12 Commits
Author SHA1 Message Date
mathiasandClaude Opus 4.8 f66c1bcdcc feat(web): real channel filter — multi-select of the user's channels
CI / Build & Import (push) Successful in 10s
CI / Lint / Test / Vet (push) Successful in 11s
The free-text 'channel' filter was dead: it exact-matched SummaryRow.Channel,
which is just the provider ('youtube'), because videos never stored their source
channel. Now they do.

- migration 014: videos.channel_title (nullable; existing rows backfill on the
  next discovery pass, pasted videos immediately).
- discovery (NewVideos) + paste (VideoByID) populate channel_title; UpsertVideo
  persists it, preserving an existing title when an update arrives empty.
- store.DistinctChannels lists a user's channels (RLS-scoped); SummaryRow carries
  ChannelTitle via the shared projection.
- Filter: single Channel -> Channels []string, matching on ChannelTitle; the feed
  renders a multi-select of DistinctChannels (hidden until channels exist).
- migrate tests: 014 reversibility + fixed the relative-step counts in the 010/011
  up/down tests (014 shifted the topology).

TDD throughout: channel persist + distinct, adapter channel wiring, multi-channel
filter match, handler channel filter, migration up/down.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:02:54 +02:00
mathiasandClaude Opus 4.8 1e65c3b413 fix(web): show paste box to any connected user, not only on an empty feed
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 10s
hasConnected was computed only inside the buckets.empty() branch (it was added
for the empty-state copy), so a connected user WITH videos got hasConnected=false
and never saw the paste box (#2-regression of the v0.12.0 paste UI). Compute it
on every list render. Test: connected user with a non-empty feed sees the box.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 22:17:00 +02:00
mathiasandClaude Opus 4.8 87c978774f docs(bdd): scenarios for paste-a-URL + onboarding burst, mapped to tests
CI / Lint / Test / Vet (push) Successful in 10s
CI / Build & Import (push) Successful in 9s
Adds paste_url.feature (valid/invalid/not-found/dedup, +@pending no-captions)
and an onboarding-burst scenario on connect; all non-pending scenarios mapped in
the coverage gate to their existing Go tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 22:06:40 +02:00
mathiasandClaude Opus 4.8 70a9f1d4cd feat(web): in-feed paste box + onboarding-aware connect confirmation
5b: connected users get a 'Summarize any video' URL input on the feed; submit
posts to /paste (HTMX) and swaps the resulting card / inline error into the feed.
7: the connect flash now sets expectations for the async onboarding burst —
'finding your subscriptions, your newest videos will appear below as summarized'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 22:06:40 +02:00
mathiasandClaude Opus 4.8 1d5b2c6365 feat(serve): wire paste fetcher + connect-time onboarding burst
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 10s
5c: app.Fetcher = a per-user YouTube videoFetcher, so POST /paste mounts and
resolves arbitrary-video metadata (Feature 2 goes live).

6: the connect trigger now runs an onboarding burst after discovery — summarize
up to TAPIR_ONBOARD_SUMMARIZE_COUNT of the user's newest unsummarized videos via
the gated Processor (Feature 1). Hard cap; explicit so it bypasses recency; every
fetch still through globalFetchGate. No-op when count=0 or queue-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:58:39 +02:00
mathiasandClaude Opus 4.8 62acfee2ed feat(web): paste-a-URL handler — add an arbitrary video + summarize (Feature 2)
POST /paste: parse the video id, fetch metadata via the VideoFetcher port (Data
API, ungated), upsert a subscription-less row scoped to the user (idempotent =
dedup), and — unless already summarized — RequestSummarize + start immediate
processing through the SAME globalFetchGate as the Summarize button. Explicit
paste overrides the recency window; a captionless video degrades to the honest
'no transcript' terminal state via the engine (ADR-010). Invalid URL -> 400,
not-found -> 404, both add nothing. Route mounts only when a Fetcher is wired.

Moves the video-not-found sentinel to domain (shared by adapter + web, no
cross-adapter coupling). Tests: valid add+queue, invalid, not-found, dedup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:54:28 +02:00
mathiasandClaude Opus 4.8 2907801aca feat(youtube): VideoByID for arbitrary-video metadata (paste-a-URL)
videos.list (part=snippet) for a single id, including channels the user does
not follow. Data API call (1 quota unit), NOT the rate-limited caption path —
ungated metadata; only the later transcript fetch hits globalFetchGate. Returns
a subscription-less domain.Video scoped to the user, or ErrVideoNotFound for a
deleted/private/typo'd id. Foundation for Feature 2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:43:28 +02:00
mathiasandClaude Opus 4.8 59050c4db6 feat(store): NewestUnsummarizedVideoIDs for the onboarding cap
Returns up to limit of a user's newest videos (published_at DESC, NULLS LAST)
that have no summary yet. RLS-scoped via withUser — the test proves a second
user's newer video never leaks. Drives the connect-time onboarding burst
(Feature 1); the caller routes each through the shared rate gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:39:40 +02:00
mathiasandClaude Opus 4.8 c320ed88aa feat(config): add TAPIR_ONBOARD_SUMMARIZE_COUNT (default 3, hard cap 5)
Bounds the connect-time onboarding summary burst (Feature 1). Hard-capped at 5
and clamped (negative->0, >cap->cap) so onboarding can never bulk-fetch; 0
disables. The cap bounds COUNT only — every fetch still flows through the shared
caption rate gate (ADR-014).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:35:05 +02:00
mathiasandClaude Opus 4.8 bddd75d92e feat(web): parse YouTube video id from pasted URL forms
Pure parser for watch?v=, youtu.be/, shorts/, embed/, and bare ids; rejects
non-YouTube hosts and malformed input. Foundation for paste-a-URL summarize
(Feature 2). No fetch, no gate interaction — parsing only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:35:05 +02:00
mathiasandClaude Opus 4.8 e4c701c6f1 feat(discovery): trigger a discovery pass on YouTube connect
CI / Lint / Test / Vet (push) Successful in 11s
CI / Build & Import (push) Successful in 10s
A newly connected account showed no videos until the next 2h scheduled pass —
the gap that made onboarding look broken (a second user connected, saw nothing,
read as failure). The connect callback now fires an out-of-band discovery pass
for the connecting user, so videos appear promptly.

Concurrency: scheduled and connect-triggered passes share one lock (serialize),
preserving the single-fetcher invariant (ADR-018). A trigger interleaves between
the scheduler's per-user passes rather than fetching concurrently or waiting for
a whole pass. The trigger runs on the server ctx (survives the redirect) and is
non-blocking for the request goroutine.

Scope: connect-trigger only. The optional login-refresh / "Discover now" button
from #6 are intentionally not built — an unconditional login hook risks 429
storms (per the ticket's own recommendation); defer until wanted.

TDD: TestCallbackTriggersDiscovery, TestSerializeRunsOneAtATime,
TestDiscoveryTriggerEnqueueRunsUser; new BDD scenario mapped.

Refs #6

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 21:15:46 +02:00
mathias b527db9739 docs: bump last-tag reference to v0.9.0
CI / Lint / Test / Vet (push) Successful in 15s
CI / Build & Import (push) Successful in 11s
2026-06-09 21:03:44 +02:00
31 changed files with 1800 additions and 633 deletions
+1 -1
View File
@@ -85,7 +85,7 @@ Skills live in the canonical library `mathias/skills` and are wired into this re
## Current build state (start here for the first task) ## Current build state (start here for the first task)
The repo is **green and shipping** — last tag `v0.8.0`. `task check` passes (fmt, vet, lint, The repo is **green and shipping** — last tag `v0.9.0`. `task check` passes (fmt, vet, lint,
`go test -p 1 ./...`). Go is `1.26.1` (see `go.mod`). `go test -p 1 ./...`). Go is `1.26.1` (see `go.mod`).
- Clean Architecture core is implemented: `internal/domain` (entities), `internal/ports` - Clean Architecture core is implemented: `internal/domain` (entities), `internal/ports`
+50
View File
@@ -0,0 +1,50 @@
package main
import (
"context"
"log/slog"
"sync"
"gitea.d-ma.be/mathias/tapir/internal/runner"
)
// discoveryRunner runs one user's discovery pass.
type discoveryRunner func(ctx context.Context, userID string) (runner.Stats, error)
// serialize wraps run so calls never overlap: every discovery pass — scheduled
// or connect-triggered (#6) — acquires the same lock, preserving the
// one-fetcher-at-a-time invariant the scheduler relies on (ADR-018, the
// single-replica assumption). Locking is per-user, so a connect-triggered pass
// interleaves between the scheduler's users instead of waiting for a whole pass.
func serialize(mu *sync.Mutex, run discoveryRunner) discoveryRunner {
return func(ctx context.Context, userID string) (runner.Stats, error) {
mu.Lock()
defer mu.Unlock()
return run(ctx, userID)
}
}
// discoveryTrigger fires an out-of-band discovery pass for one user without
// blocking the caller (the connect HTTP handler). The pass runs on the server's
// long-lived ctx — not the request ctx — so it survives the post-connect
// redirect. run is the serialized runner, so a trigger never overlaps the
// scheduler. Satisfies web.DiscoveryTrigger.
type discoveryTrigger struct {
ctx context.Context
run discoveryRunner
// onboard, when set, runs after the discovery pass to summarize a capped number
// of the user's newest videos (Feature 1). Optional.
onboard func(ctx context.Context, userID string)
log *slog.Logger
}
func (t *discoveryTrigger) Enqueue(userID string) {
go func() {
if _, err := t.run(t.ctx, userID); err != nil {
t.log.Warn("discovery: connect-triggered pass had errors", "user", userID, "err", err)
}
if t.onboard != nil {
t.onboard(t.ctx, userID)
}
}()
}
+62
View File
@@ -0,0 +1,62 @@
package main
import (
"context"
"fmt"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/runner"
)
// serialize must guarantee at most one discovery pass runs at a time, so a
// connect-triggered pass never fetches concurrently with the scheduler.
func TestSerializeRunsOneAtATime(t *testing.T) {
var active, maxActive int32
run := func(_ context.Context, _ string) (runner.Stats, error) {
n := atomic.AddInt32(&active, 1)
for { // record the high-water mark of concurrent runs
m := atomic.LoadInt32(&maxActive)
if n <= m || atomic.CompareAndSwapInt32(&maxActive, m, n) {
break
}
}
time.Sleep(2 * time.Millisecond)
atomic.AddInt32(&active, -1)
return runner.Stats{}, nil
}
s := serialize(&sync.Mutex{}, run)
var wg sync.WaitGroup
for i := 0; i < 20; i++ {
wg.Add(1)
go func(i int) { defer wg.Done(); _, _ = s(context.Background(), fmt.Sprintf("u%d", i)) }(i)
}
wg.Wait()
require.Equal(t, int32(1), atomic.LoadInt32(&maxActive),
"serialize must run at most one pass at a time")
}
// Enqueue runs the user's pass out-of-band (non-blocking) on the trigger's ctx.
func TestDiscoveryTriggerEnqueueRunsUser(t *testing.T) {
done := make(chan string, 1)
run := func(_ context.Context, userID string) (runner.Stats, error) {
done <- userID
return runner.Stats{}, nil
}
tr := &discoveryTrigger{ctx: context.Background(), run: run, log: quietLog()}
tr.Enqueue("u1")
select {
case got := <-done:
require.Equal(t, "u1", got)
case <-time.After(2 * time.Second):
t.Fatal("Enqueue did not run the user's pass")
}
}
+37 -2
View File
@@ -20,6 +20,7 @@ import (
"net/http" "net/http"
"os" "os"
"os/signal" "os/signal"
"sync"
"time" "time"
"gitea.d-ma.be/mathias/tapir/internal/adapters/secrets" "gitea.d-ma.be/mathias/tapir/internal/adapters/secrets"
@@ -209,7 +210,10 @@ func cmdServe(ctx context.Context, log *slog.Logger) error {
ClientSecret: cfg.YTClientSecret, ClientSecret: cfg.YTClientSecret,
RedirectURL: cfg.YTConnectRedirectURL, RedirectURL: cfg.YTConnectRedirectURL,
}, secretStore, st, log) }, secretStore, st, log)
log.Info("web youtube connect enabled", "redirect", cfg.YTConnectRedirectURL) // Paste-a-URL (Feature 2): same YouTube credentials, per-user adapter built
// per request. Mounting the /paste route keys off app.Fetcher being set.
app.Fetcher = videoFetcher{cfg: cfg, secrets: secretStore}
log.Info("web youtube connect + paste enabled", "redirect", cfg.YTConnectRedirectURL)
} }
// Immediate summarization for the web "Summarize" button. When the engine can // Immediate summarization for the web "Summarize" button. When the engine can
@@ -238,13 +242,44 @@ func cmdServe(ctx context.Context, log *slog.Logger) error {
if cfg.DiscoveryInterval > 0 { if cfg.DiscoveryInterval > 0 {
log.Info("scheduled discovery enabled", "interval", cfg.DiscoveryInterval, "fetch_rate", cfg.FetchRate) log.Info("scheduled discovery enabled", "interval", cfg.DiscoveryInterval, "fetch_rate", cfg.FetchRate)
log.Warn("scheduled discovery assumes a SINGLE replica — running serve at >1 replica double-runs discovery (ADR-018)") log.Warn("scheduled discovery assumes a SINGLE replica — running serve at >1 replica double-runs discovery (ADR-018)")
runUser := func(ctx context.Context, userID string) (runner.Stats, error) { rawRunUser := func(ctx context.Context, userID string) (runner.Stats, error) {
r, err := buildUserRunner(cfg, st, secretStore, userID, log) r, err := buildUserRunner(cfg, st, secretStore, userID, log)
if err != nil { if err != nil {
return runner.Stats{}, err return runner.Stats{}, err
} }
return r.RunOnce(ctx) return r.RunOnce(ctx)
} }
// One lock shared by the scheduler and connect-triggered passes (#6) so
// they never fetch concurrently — the single-fetcher invariant (ADR-018).
runUser := serialize(&sync.Mutex{}, rawRunUser)
// Onboarding burst (Feature 1): after the connect-triggered discovery pass,
// summarize up to OnboardSummarizeCount of the user's NEWEST unsummarized
// videos so a fresh account gets real summaries in its first session. Hard
// cap; explicit, so it bypasses the recency window — but every fetch still
// goes through globalFetchGate via the Processor. No-op when disabled
// (count 0) or queue-only (no Processor).
onboard := func(ctx context.Context, userID string) {
if cfg.OnboardSummarizeCount <= 0 || app.Processor == nil {
return
}
ids, err := st.NewestUnsummarizedVideoIDs(ctx, userID, cfg.OnboardSummarizeCount)
if err != nil {
log.Warn("onboarding: list newest unsummarized", "user", userID, "err", err)
return
}
for _, id := range ids {
if err := app.Processor.ProcessVideo(ctx, userID, id); err != nil {
log.Warn("onboarding: summarize", "user", userID, "video", id, "err", err)
}
}
if len(ids) > 0 {
log.Info("onboarding burst complete", "user", userID, "summarized", len(ids), "cap", cfg.OnboardSummarizeCount)
}
}
if app.Connect != nil {
app.Connect.Discovery = &discoveryTrigger{ctx: ctx, run: runUser, onboard: onboard, log: log}
log.Info("connect-triggered discovery enabled", "onboard_cap", cfg.OnboardSummarizeCount)
}
go runScheduler(ctx, cfg.DiscoveryInterval, st, runUser, log) go runScheduler(ctx, cfg.DiscoveryInterval, st, runUser, log)
} else { } else {
log.Info("scheduled discovery disabled (TAPIR_DISCOVERY_INTERVAL unset or 0)") log.Info("scheduled discovery disabled (TAPIR_DISCOVERY_INTERVAL unset or 0)")
+20
View File
@@ -11,9 +11,29 @@ import (
"gitea.d-ma.be/mathias/tapir/internal/adapters/youtube" "gitea.d-ma.be/mathias/tapir/internal/adapters/youtube"
"gitea.d-ma.be/mathias/tapir/internal/config" "gitea.d-ma.be/mathias/tapir/internal/config"
"gitea.d-ma.be/mathias/tapir/internal/domain" "gitea.d-ma.be/mathias/tapir/internal/domain"
"gitea.d-ma.be/mathias/tapir/internal/ports"
"gitea.d-ma.be/mathias/tapir/internal/usecase" "gitea.d-ma.be/mathias/tapir/internal/usecase"
"gitea.d-ma.be/mathias/tapir/internal/web"
) )
// videoFetcher adapts the YouTube adapter to web.VideoFetcher for the paste flow
// (Feature 2). It builds a per-user adapter bound to that user's token ref and
// resolves a single video's metadata via the Data API — ungated; only the later
// transcript fetch goes through globalFetchGate.
type videoFetcher struct {
cfg config.Config
secrets ports.SecretStore
}
func (f videoFetcher) FetchVideo(ctx context.Context, userID, videoID string) (domain.Video, error) {
a := youtube.New(youtube.Config{
ClientID: f.cfg.YTClientID,
ClientSecret: f.cfg.YTClientSecret,
TokenSecretRef: web.YouTubeTokenRef(userID),
}, f.secrets)
return a.VideoByID(ctx, userID, videoID)
}
// buildProcessor wires the summarization engine — YouTube source (captions-first), // buildProcessor wires the summarization engine — YouTube source (captions-first),
// AI-router summarizer, store sink — shared by `tapir run` and the web // AI-router summarizer, store sink — shared by `tapir run` and the web
// "Summarize now" path so the wiring lives in one place. It returns (nil, nil) — // "Summarize now" path so the wiring lives in one place. It returns (nil, nil) —
+12
View File
@@ -18,6 +18,18 @@ Feature: Connect and manage video accounts
Then the connection is stored with status "active" Then the connection is stored with status "active"
And my subscriptions are synced And my subscriptions are synced
Scenario: Connecting an account discovers videos immediately
Given I have no connected video accounts
When I connect my YouTube account
Then a discovery pass for my account is triggered right away
And I do not have to wait for the next scheduled pass to see my videos
Scenario: Connecting summarizes my newest videos right away
Given I have no connected video accounts
When I connect my YouTube account
Then up to the onboarding cap of my newest videos are summarized through the rate gate
And the rest are left to the scheduled recency-bounded pass
Scenario: Tokens are never stored in the clear Scenario: Tokens are never stored in the clear
When I connect any video account When I connect any video account
Then no OAuth token value is stored in the database Then no OAuth token value is stored in the database
+30
View File
@@ -0,0 +1,30 @@
Feature: Paste a YouTube URL to summarize any video
As a user
I want to paste a YouTube link and get a summary
So that I can pull the specific video I want now, even from channels I don't follow
Scenario: Paste a valid YouTube URL
Given I am connected
When I paste a valid YouTube video URL
Then the video is added to my feed scoped to me
And it is queued for summarization through the shared rate gate
Scenario: Pasting an invalid link is rejected
When I paste something that is not a YouTube video URL
Then I get a clear error and nothing is added
Scenario: Pasting a video that cannot be found is honest
When I paste a URL whose video cannot be found
Then I am told it couldn't be found and nothing is added
Scenario: Pasting the same video twice does not duplicate it
Given I have pasted a video
When I paste the same video again
Then my feed still has exactly one entry for it
@pending
# Covered by the engine's ADR-010 no-transcript terminal state (degrade-never-error);
# there is no paste-specific test for it.
Scenario: A pasted video with no captions resolves honestly
When I paste a video that has no captions
Then it resolves to the "no transcript available" terminal state
+30 -2
View File
@@ -53,7 +53,9 @@ func TestMigration010LoginEventsUpDown(t *testing.T) {
require.True(t, loginEventsExists(t), "login_events must exist at latest migration") require.True(t, loginEventsExists(t), "login_events must exist at latest migration")
m := fileMigrator(t) m := fileMigrator(t)
// 011, 012, 013 sit above 010; step them down first so 010 is exercised in isolation. // 011, 012, 013, 014 sit above 010; step them down first so 010 is exercised in isolation.
require.NoError(t, m.Steps(-1), "down 014 drops channel_title, login_events intact")
require.True(t, loginEventsExists(t), "014 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 013 drops channel_errors, login_events intact") require.NoError(t, m.Steps(-1), "down 013 drops channel_errors, login_events intact")
require.True(t, loginEventsExists(t), "013 down leaves login_events intact") require.True(t, loginEventsExists(t), "013 down leaves login_events intact")
require.NoError(t, m.Steps(-1), "down 012 is a no-op, login_events intact") require.NoError(t, m.Steps(-1), "down 012 is a no-op, login_events intact")
@@ -64,7 +66,7 @@ func TestMigration010LoginEventsUpDown(t *testing.T) {
require.NoError(t, m.Steps(-1), "down 010 must drop login_events") require.NoError(t, m.Steps(-1), "down 010 must drop login_events")
require.False(t, loginEventsExists(t), "login_events must be gone after the down migration") require.False(t, loginEventsExists(t), "login_events must be gone after the down migration")
require.NoError(t, m.Steps(4), "up must recreate 010 then re-apply 011, 012, 013") require.NoError(t, m.Steps(5), "up must recreate 010 then re-apply 011, 012, 013, 014")
require.True(t, loginEventsExists(t), "login_events must be restored after the up migration") require.True(t, loginEventsExists(t), "login_events must be restored after the up migration")
} }
@@ -87,6 +89,7 @@ func TestMigration011AutoSummarizeDefaultUpDown(t *testing.T) {
require.Equal(t, "true", autoSummarizeDefault(t), "011 sets the default to TRUE") require.Equal(t, "true", autoSummarizeDefault(t), "011 sets the default to TRUE")
m := fileMigrator(t) m := fileMigrator(t)
require.NoError(t, m.Steps(-1), "down 014 drops channel_title")
require.NoError(t, m.Steps(-1), "down 013 drops channel_errors") require.NoError(t, m.Steps(-1), "down 013 drops channel_errors")
require.NoError(t, m.Steps(-1), "down 012 is a no-op") require.NoError(t, m.Steps(-1), "down 012 is a no-op")
require.NoError(t, m.Steps(-1), "down 011 reverts the column default") require.NoError(t, m.Steps(-1), "down 011 reverts the column default")
@@ -96,6 +99,31 @@ func TestMigration011AutoSummarizeDefaultUpDown(t *testing.T) {
require.Equal(t, "true", autoSummarizeDefault(t)) require.Equal(t, "true", autoSummarizeDefault(t))
require.NoError(t, m.Steps(1), "up 012 runs clean (no FORCE RLS on fresh schema)") require.NoError(t, m.Steps(1), "up 012 runs clean (no FORCE RLS on fresh schema)")
require.NoError(t, m.Steps(1), "up 013 creates channel_errors") require.NoError(t, m.Steps(1), "up 013 creates channel_errors")
require.NoError(t, m.Steps(1), "up 014 recreates channel_title")
}
// channelTitleExists reports whether videos.channel_title is present.
func channelTitleExists(t *testing.T) bool {
t.Helper()
var exists bool
require.NoError(t, rawPool(t).QueryRow(context.Background(),
`SELECT EXISTS (SELECT 1 FROM information_schema.columns
WHERE table_name = 'videos' AND column_name = 'channel_title')`).Scan(&exists))
return exists
}
// TestMigration014VideoChannelTitleUpDown proves 014 is reversible: down drops
// videos.channel_title, up recreates it.
func TestMigration014VideoChannelTitleUpDown(t *testing.T) {
newStore(t) // latest (014 applied)
require.True(t, channelTitleExists(t), "channel_title exists at latest migration")
m := fileMigrator(t)
require.NoError(t, m.Steps(-1), "down 014 must drop channel_title")
require.False(t, channelTitleExists(t), "channel_title must be gone after the down migration")
require.NoError(t, m.Steps(1), "up 014 must recreate channel_title")
require.True(t, channelTitleExists(t), "channel_title must be restored after the up migration")
} }
// TestMigration012FixAutoSummarizeRLS proves 012 runs cleanly and flips any // TestMigration012FixAutoSummarizeRLS proves 012 runs cleanly and flips any
@@ -0,0 +1 @@
ALTER TABLE videos DROP COLUMN channel_title;
@@ -0,0 +1,8 @@
-- Store the source channel's title per video so the list can offer a real
-- channel filter (multi-select of the user's channels) instead of the dead
-- free-text field that only ever matched the provider string. Nullable: existing
-- rows backfill on the next discovery pass (UpsertVideo writes it); pasted videos
-- get it immediately from videos.list. No FK to a channels table at Stage 0 — the
-- title is a denormalised display/filter value, consistent with the existing
-- subscription_id-stays-NULL stance (data-model.md).
ALTER TABLE videos ADD COLUMN channel_title TEXT;
+4 -1
View File
@@ -29,6 +29,7 @@ type SummaryRow struct {
ProviderVideoID string // videos.provider_video_id; empty when no videos row ProviderVideoID string // videos.provider_video_id; empty when no videos row
Title string // videos.title; empty when no videos row Title string // videos.title; empty when no videos row
Channel string // videos.provider for now; empty when no videos row Channel string // videos.provider for now; empty when no videos row
ChannelTitle string // videos.channel_title; the source channel, for display + filtering
URL string // videos.url; empty when no videos row URL string // videos.url; empty when no videos row
PublishedAt time.Time // videos.published_at; zero when absent PublishedAt time.Time // videos.published_at; zero when absent
Summary string Summary string
@@ -137,7 +138,8 @@ const selectVideo = `
COALESCE(s.created_at, v.seen_at), COALESCE(s.created_at, v.seen_at),
(s.id IS NOT NULL) AS summarized, (s.id IS NOT NULL) AS summarized,
v.summarize_requested, v.summarize_requested,
COALESCE(v.transcript_status, '') COALESCE(v.transcript_status, ''),
COALESCE(v.channel_title, '')
FROM videos v FROM videos v
LEFT JOIN summaries s ON s.video_id = v.id AND s.user_id = v.user_id` LEFT JOIN summaries s ON s.video_id = v.id AND s.user_id = v.user_id`
@@ -254,6 +256,7 @@ func scanVideoRow(rows pgx.Row) (SummaryRow, error) {
&row.Summarized, &row.Summarized,
&row.SummarizeRequested, &row.SummarizeRequested,
&row.TranscriptStatus, &row.TranscriptStatus,
&row.ChannelTitle,
); err != nil { ); err != nil {
return SummaryRow{}, fmt.Errorf("store: scan video: %w", err) return SummaryRow{}, fmt.Errorf("store: scan video: %w", err)
} }
+71 -4
View File
@@ -46,14 +46,15 @@ func (s *Store) UpsertVideo(ctx context.Context, v domain.Video) (string, error)
} }
if err := tx.QueryRow(ctx, if err := tx.QueryRow(ctx,
`INSERT INTO videos (user_id, provider, provider_video_id, title, url, published_at) `INSERT INTO videos (user_id, provider, provider_video_id, title, url, published_at, channel_title)
VALUES ($1, $2, $3, $4, $5, $6) VALUES ($1, $2, $3, $4, $5, $6, $7)
ON CONFLICT (user_id, provider, provider_video_id) DO UPDATE SET ON CONFLICT (user_id, provider, provider_video_id) DO UPDATE SET
title = EXCLUDED.title, title = EXCLUDED.title,
url = EXCLUDED.url, url = EXCLUDED.url,
published_at = EXCLUDED.published_at published_at = EXCLUDED.published_at,
channel_title = COALESCE(NULLIF(EXCLUDED.channel_title, ''), videos.channel_title)
RETURNING id`, RETURNING id`,
v.UserID, provider, v.ProviderVideoID, v.Title, v.URL, nullTime(v.PublishedAt), v.UserID, provider, v.ProviderVideoID, v.Title, v.URL, nullTime(v.PublishedAt), v.ChannelTitle,
).Scan(&id); err != nil { ).Scan(&id); err != nil {
return fmt.Errorf("store: upsert video: %w", err) return fmt.Errorf("store: upsert video: %w", err)
} }
@@ -72,3 +73,69 @@ func nullTime(t time.Time) *time.Time {
} }
return &t return &t
} }
// NewestUnsummarizedVideoIDs returns up to limit of the user's videos that have
// no summary yet, newest first (published_at DESC, NULLS LAST). It caps the
// connect-time onboarding burst (Feature 1) at a fixed count: the caller marks
// these for summarization through the shared rate gate. RLS-scoped via withUser,
// so it only ever sees the requesting user's rows. limit <= 0 returns nil.
func (s *Store) NewestUnsummarizedVideoIDs(ctx context.Context, userID string, limit int) ([]string, error) {
if limit <= 0 {
return nil, nil
}
var ids []string
if err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
rows, err := tx.Query(ctx,
`SELECT v.id
FROM videos v
WHERE v.user_id = $1
AND NOT EXISTS (
SELECT 1 FROM summaries su
WHERE su.user_id = v.user_id AND su.video_id = v.id)
ORDER BY v.published_at DESC NULLS LAST, v.seen_at DESC
LIMIT $2`, userID, limit)
if err != nil {
return fmt.Errorf("store: newest unsummarized: %w", err)
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return fmt.Errorf("store: scan newest unsummarized: %w", err)
}
ids = append(ids, id)
}
return rows.Err()
}); err != nil {
return nil, err
}
return ids, nil
}
// DistinctChannels returns the user's distinct, non-empty source channel titles
// (the channels they have videos from), alphabetically — the option list for the
// feed's channel filter. RLS-scoped via withUser.
func (s *Store) DistinctChannels(ctx context.Context, userID string) ([]string, error) {
var out []string
if err := s.withUser(ctx, userID, func(tx pgx.Tx) error {
rows, err := tx.Query(ctx,
`SELECT DISTINCT channel_title FROM videos
WHERE user_id = $1 AND channel_title IS NOT NULL AND channel_title <> ''
ORDER BY channel_title`, userID)
if err != nil {
return fmt.Errorf("store: distinct channels: %w", err)
}
defer rows.Close()
for rows.Next() {
var c string
if err := rows.Scan(&c); err != nil {
return fmt.Errorf("store: scan channel: %w", err)
}
out = append(out, c)
}
return rows.Err()
}); err != nil {
return nil, err
}
return out, nil
}
+58
View File
@@ -81,3 +81,61 @@ func TestUpsertVideo_PerUserIsolation(t *testing.T) {
require.NotEqual(t, idA, idB, "same provider video for two users must be two distinct rows") require.NotEqual(t, idA, idB, "same provider video for two users must be two distinct rows")
} }
func TestNewestUnsummarizedVideoIDs(t *testing.T) {
ctx := context.Background()
s := newStore(t)
resetDB(t, rawPool(t))
mk := func(user, pid string, day int) string {
v := ytVideo(user, pid, pid)
v.PublishedAt = time.Date(2026, 6, day, 12, 0, 0, 0, time.UTC)
id, err := s.UpsertVideo(ctx, v)
require.NoError(t, err)
return id
}
_ = mk(userA, "a1vid000001", 1)
id2 := mk(userA, "a2vid000002", 2)
id3 := mk(userA, "a3vid000003", 3)
id4 := mk(userA, "a4vid000004", 4)
mk(userB, "b1vid000009", 9) // userB's newest — must never leak via RLS
// The newest (v4) is summarized, so it's excluded from "unsummarized".
require.NoError(t, s.Deliver(ctx, summary(userA, id4, "done")))
// Cap 2, newest-first unsummarized: v3 then v2 (v4 excluded; userB excluded).
got, err := s.NewestUnsummarizedVideoIDs(ctx, userA, 2)
require.NoError(t, err)
require.Equal(t, []string{id3, id2}, got)
none, err := s.NewestUnsummarizedVideoIDs(ctx, userA, 0)
require.NoError(t, err)
require.Empty(t, none, "limit 0 returns nothing")
}
func TestUpsertVideoPersistsChannelAndDistinctChannels(t *testing.T) {
ctx := context.Background()
s := newStore(t)
resetDB(t, rawPool(t))
mk := func(pid, channel string) {
v := ytVideo(userA, pid, pid)
v.ChannelTitle = channel
_, err := s.UpsertVideo(ctx, v)
require.NoError(t, err)
}
mk("aa11111aaaa", "Acme Talks")
mk("bb22222bbbb", "Acme Talks") // same channel
mk("cc33333cccc", "Zeta Channel")
// userB's channel must not leak.
vb := ytVideo(userB, "dd44444dddd", "x")
vb.ChannelTitle = "Bravo Only"
_, err := s.UpsertVideo(ctx, vb)
require.NoError(t, err)
got, err := s.DistinctChannels(ctx, userA)
require.NoError(t, err)
require.Equal(t, []string{"Acme Talks", "Zeta Channel"}, got,
"distinct, alphabetical, user-scoped (no Bravo Only)")
}
@@ -0,0 +1,62 @@
package youtube
import (
"context"
"errors"
"net/http"
"testing"
"gitea.d-ma.be/mathias/tapir/internal/domain"
)
func TestVideoByID(t *testing.T) {
const id = "dQw4w9WgXcQ"
a, secrets := newTestAdapter(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/videos" {
t.Errorf("unexpected path %q (must use videos.list)", r.URL.Path)
}
if got := r.URL.Query().Get("id"); got != id {
t.Errorf("expected id=%s, got %q", id, got)
}
if got := r.URL.Query().Get("part"); got != "snippet" {
t.Errorf("expected part=snippet, got %q", got)
}
_, _ = w.Write([]byte(`{"items":[{"snippet":{"title":"Never Gonna Give You Up","channelTitle":"Rick Astley","publishedAt":"2026-05-20T09:00:00Z"}}]}`))
})
v, err := a.VideoByID(context.Background(), "u1", id)
if err != nil {
t.Fatalf("VideoByID: %v", err)
}
if v.UserID != "u1" {
t.Errorf("UserID = %q, want u1", v.UserID)
}
if v.ProviderVideoID != id || v.Title != "Never Gonna Give You Up" {
t.Errorf("unexpected video: %+v", v)
}
if v.ChannelTitle != "Rick Astley" {
t.Errorf("ChannelTitle = %q, want Rick Astley", v.ChannelTitle)
}
if v.Provider != domain.ProviderYouTube || v.URL != "https://www.youtube.com/watch?v="+id {
t.Errorf("video not wired correctly: %+v", v)
}
if v.PublishedAt.IsZero() {
t.Errorf("expected publishedAt parsed, got zero")
}
if v.SubscriptionID != "" {
t.Errorf("a pasted video must have no subscription, got %q", v.SubscriptionID)
}
if secrets.byRef == nil {
t.Errorf("token must be resolved by reference through the SecretStore")
}
}
func TestVideoByIDNotFound(t *testing.T) {
a, _ := newTestAdapter(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"items":[]}`))
})
_, err := a.VideoByID(context.Background(), "u1", "missingvid0")
if !errors.Is(err, domain.ErrVideoNotFound) {
t.Fatalf("VideoByID for missing id = %v, want domain.ErrVideoNotFound", err)
}
}
+43
View File
@@ -234,6 +234,7 @@ func (a *Adapter) NewVideos(ctx context.Context, sub domain.Subscription) ([]dom
Provider: domain.ProviderYouTube, Provider: domain.ProviderYouTube,
ProviderVideoID: vid, ProviderVideoID: vid,
Title: item.Snippet.Title, Title: item.Snippet.Title,
ChannelTitle: sub.ChannelTitle,
URL: "https://www.youtube.com/watch?v=" + vid, URL: "https://www.youtube.com/watch?v=" + vid,
PublishedAt: item.Snippet.PublishedAt, PublishedAt: item.Snippet.PublishedAt,
}) })
@@ -244,6 +245,38 @@ func (a *Adapter) NewVideos(ctx context.Context, sub domain.Subscription) ([]dom
return videos, nil return videos, nil
} }
// VideoByID fetches a single video's metadata (videos.list, snippet) for an
// arbitrary video id — including channels the user does not follow (paste-a-URL,
// Feature 2). This is a Data API call (1 quota unit), NOT the rate-limited
// caption path, so it is not gated: only the later transcript fetch goes through
// globalFetchGate. UserID is set on the result and SubscriptionID is left empty
// (a pasted video has no subscription parent). Returns ErrVideoNotFound when the
// id resolves to no video.
func (a *Adapter) VideoByID(ctx context.Context, userID, videoID string) (domain.Video, error) {
client, err := a.httpClient(ctx, a.cfg.TokenSecretRef)
if err != nil {
return domain.Video{}, err
}
q := url.Values{"part": {"snippet"}, "id": {videoID}}
var resp videoListResponse
if err := a.getJSON(ctx, client, "/videos", q, &resp); err != nil {
return domain.Video{}, fmt.Errorf("video by id %q: %w", videoID, err)
}
if len(resp.Items) == 0 {
return domain.Video{}, fmt.Errorf("video %q: %w", videoID, domain.ErrVideoNotFound)
}
it := resp.Items[0]
return domain.Video{
UserID: userID,
Provider: domain.ProviderYouTube,
ProviderVideoID: videoID,
Title: it.Snippet.Title,
ChannelTitle: it.Snippet.ChannelTitle,
URL: "https://www.youtube.com/watch?v=" + videoID,
PublishedAt: it.Snippet.PublishedAt,
}, nil
}
// uploadsPlaylistID derives a channel's uploads playlist id at zero API cost: // uploadsPlaylistID derives a channel's uploads playlist id at zero API cost:
// a standard channel id "UCxxxx" maps to uploads playlist "UUxxxx". Returns // a standard channel id "UCxxxx" maps to uploads playlist "UUxxxx". Returns
// ok=false for ids that don't follow this convention (caller falls back to // ok=false for ids that don't follow this convention (caller falls back to
@@ -342,6 +375,16 @@ type playlistItemListResponse struct {
} `json:"items"` } `json:"items"`
} }
type videoListResponse struct {
Items []struct {
Snippet struct {
Title string `json:"title"`
ChannelTitle string `json:"channelTitle"`
PublishedAt time.Time `json:"publishedAt"`
} `json:"snippet"`
} `json:"items"`
}
type channelListResponse struct { type channelListResponse struct {
Items []struct { Items []struct {
ContentDetails struct { ContentDetails struct {
+4 -1
View File
@@ -131,7 +131,7 @@ func TestNewVideos(t *testing.T) {
}`)) }`))
}) })
sub := domain.Subscription{ID: "s1", UserID: "u1", ChannelID: "UC_acme"} sub := domain.Subscription{ID: "s1", UserID: "u1", ChannelID: "UC_acme", ChannelTitle: "Acme Channel"}
vids, err := a.NewVideos(context.Background(), sub) vids, err := a.NewVideos(context.Background(), sub)
if err != nil { if err != nil {
t.Fatalf("NewVideos: %v", err) t.Fatalf("NewVideos: %v", err)
@@ -143,6 +143,9 @@ func TestNewVideos(t *testing.T) {
if v.ProviderVideoID != "vid1" || v.Title != "Designing for Attention" { if v.ProviderVideoID != "vid1" || v.Title != "Designing for Attention" {
t.Errorf("unexpected video: %+v", v) t.Errorf("unexpected video: %+v", v)
} }
if v.ChannelTitle != "Acme Channel" {
t.Errorf("ChannelTitle = %q, want Acme Channel", v.ChannelTitle)
}
if v.Provider != domain.ProviderYouTube || v.URL != "https://www.youtube.com/watch?v=vid1" { if v.Provider != domain.ProviderYouTube || v.URL != "https://www.youtube.com/watch?v=vid1" {
t.Errorf("video not wired correctly: %+v", v) t.Errorf("video not wired correctly: %+v", v)
} }
+34
View File
@@ -13,6 +13,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"sort" "sort"
"strconv"
"strings" "strings"
"time" "time"
) )
@@ -79,6 +80,13 @@ type Config struct {
// pre-recency behaviour). Default ~7 days. // pre-recency behaviour). Default ~7 days.
AutoSummarizeWindow time.Duration AutoSummarizeWindow time.Duration
// OnboardSummarizeCount caps how many of a freshly-connected user's newest
// videos are summarized immediately on connect (the onboarding "it works"
// burst). HARD-capped at maxOnboardSummarizeCount so onboarding can never
// bulk-fetch; 0 disables the burst. Every fetch still flows through the shared
// caption rate gate (ADR-014) — the cap bounds count, never the pacing. Default 3.
OnboardSummarizeCount int
// DiscoveryInterval, when > 0, makes `serve` run in-process scheduled discovery // DiscoveryInterval, when > 0, makes `serve` run in-process scheduled discovery
// for ALL users on that cadence (ADR-018). Zero/unset = disabled, so dev and // for ALL users on that cadence (ADR-018). Zero/unset = disabled, so dev and
// tests never auto-fetch. Single-replica assumption — see cmdServe. // tests never auto-fetch. Single-replica assumption — see cmdServe.
@@ -119,6 +127,8 @@ const (
defaultFetchRate = 2 * time.Second defaultFetchRate = 2 * time.Second
defaultPublicURL = "https://tapir.d-ma.be" defaultPublicURL = "https://tapir.d-ma.be"
defaultAutoSummarizeWindow = 7 * 24 * time.Hour defaultAutoSummarizeWindow = 7 * 24 * time.Hour
defaultOnboardSummarizeCount = 3
maxOnboardSummarizeCount = 5
) )
// Load reads the environment into a Config, applying defaults. It does not // Load reads the environment into a Config, applying defaults. It does not
@@ -183,6 +193,18 @@ func Load() (Config, error) {
} }
c.AutoSummarizeWindow = autoWindow c.AutoSummarizeWindow = autoWindow
onboard, err := intOr("TAPIR_ONBOARD_SUMMARIZE_COUNT", defaultOnboardSummarizeCount)
if err != nil {
return Config{}, err
}
if onboard < 0 {
onboard = 0
}
if onboard > maxOnboardSummarizeCount {
onboard = maxOnboardSummarizeCount
}
c.OnboardSummarizeCount = onboard
return c, nil return c, nil
} }
@@ -247,6 +269,18 @@ func envOr(key, fallback string) string {
return fallback return fallback
} }
func intOr(key string, fallback int) (int, error) {
v := os.Getenv(key)
if v == "" {
return fallback, nil
}
n, err := strconv.Atoi(v)
if err != nil {
return 0, fmt.Errorf("config: %s=%q: %w", key, v, err)
}
return n, nil
}
func durationOr(key string, fallback time.Duration) (time.Duration, error) { func durationOr(key string, fallback time.Duration) (time.Duration, error) {
v := os.Getenv(key) v := os.Getenv(key)
if v == "" { if v == "" {
+32
View File
@@ -53,6 +53,38 @@ func TestLoad_AppliesDefaults(t *testing.T) {
} }
} }
func TestLoad_OnboardSummarizeCount(t *testing.T) {
cases := []struct {
name, env string
want int
}{
{"default", "", defaultOnboardSummarizeCount},
{"explicit", "4", 4},
{"zero disables", "0", 0},
{"clamped to hard cap", "50", maxOnboardSummarizeCount},
{"negative clamps to zero", "-3", 0},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
setEnv(t, map[string]string{"TAPIR_ONBOARD_SUMMARIZE_COUNT": c.env})
cfg, err := Load()
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.OnboardSummarizeCount != c.want {
t.Fatalf("OnboardSummarizeCount = %d, want %d", cfg.OnboardSummarizeCount, c.want)
}
})
}
}
func TestLoad_OnboardSummarizeCountInvalid(t *testing.T) {
setEnv(t, map[string]string{"TAPIR_ONBOARD_SUMMARIZE_COUNT": "three"})
if _, err := Load(); err == nil {
t.Fatal("Load: want error for non-numeric TAPIR_ONBOARD_SUMMARIZE_COUNT")
}
}
func TestLoad_ParsesValues(t *testing.T) { func TestLoad_ParsesValues(t *testing.T) {
setEnv(t, map[string]string{ setEnv(t, map[string]string{
"TAPIR_USER_ID": "11111111-1111-1111-1111-111111111111", "TAPIR_USER_ID": "11111111-1111-1111-1111-111111111111",
+7
View File
@@ -3,10 +3,16 @@
package domain package domain
import ( import (
"errors"
"fmt" "fmt"
"time" "time"
) )
// ErrVideoNotFound is returned when a video id resolves to no video (deleted,
// private, or a typo'd paste). Defined in domain so adapters and the web layer
// share one sentinel without coupling to each other.
var ErrVideoNotFound = errors.New("video not found")
// ErrChannelUnavailable is returned by a VideoSource when a channel's upload // ErrChannelUnavailable is returned by a VideoSource when a channel's upload
// playlist returns HTTP 404 — the channel was deleted or made private. The runner // playlist returns HTTP 404 — the channel was deleted or made private. The runner
// stores these so the account page can surface them to the user. // stores these so the account page can surface them to the user.
@@ -67,6 +73,7 @@ type Video struct {
Provider Provider Provider Provider
ProviderVideoID string ProviderVideoID string
Title string Title string
ChannelTitle string
URL string URL string
PublishedAt time.Time PublishedAt time.Time
SeenAt time.Time SeenAt time.Time
+19
View File
@@ -23,6 +23,15 @@ type Connections interface {
UpsertConnection(ctx context.Context, userID string, c store.Connection) error UpsertConnection(ctx context.Context, userID string, c store.Connection) error
} }
// DiscoveryTrigger requests an out-of-band discovery pass for a user. The connect
// flow fires it the moment a YouTube account is linked so videos appear promptly
// instead of waiting for the next scheduled pass (#6). Enqueue must be
// non-blocking and safe to call from the request goroutine; the implementation
// owns serialization with the scheduler (one pass at a time). nil = no trigger.
type DiscoveryTrigger interface {
Enqueue(userID string)
}
// connectStateTTL bounds how long a generated CSRF state is valid between the // connectStateTTL bounds how long a generated CSRF state is valid between the
// connect redirect and the provider callback. // connect redirect and the provider callback.
const connectStateTTL = 10 * time.Minute const connectStateTTL = 10 * time.Minute
@@ -43,6 +52,10 @@ type ConnectHandler struct {
Conns Connections Conns Connections
Log *slog.Logger Log *slog.Logger
// Discovery, when set, is fired after a successful connect so the new
// connection's videos are discovered immediately (#6). Optional.
Discovery DiscoveryTrigger
states *connectStateStore states *connectStateStore
now func() time.Time now func() time.Time
} }
@@ -134,6 +147,12 @@ func (h *ConnectHandler) handleCallback(w http.ResponseWriter, r *http.Request)
return return
} }
// Discover this user's videos now rather than waiting for the next scheduled
// pass (#6). Non-blocking; the trigger serializes with the scheduler.
if h.Discovery != nil {
h.Discovery.Enqueue(userID)
}
setFlash(w, flashConnected) setFlash(w, flashConnected)
http.Redirect(w, r, "/", http.StatusSeeOther) http.Redirect(w, r, "/", http.StatusSeeOther)
} }
+48
View File
@@ -45,6 +45,24 @@ func (c *fakeConns) UpsertConnection(_ context.Context, userID string, conn stor
return nil return nil
} }
// fakeTrigger records Enqueue calls so a test can assert connect fired discovery.
type fakeTrigger struct {
mu sync.Mutex
users []string
}
func (f *fakeTrigger) Enqueue(userID string) {
f.mu.Lock()
defer f.mu.Unlock()
f.users = append(f.users, userID)
}
func (f *fakeTrigger) seen() []string {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.users...)
}
// tokenServer fakes Google's token endpoint, returning body for any POST. // tokenServer fakes Google's token endpoint, returning body for any POST.
func tokenServer(t *testing.T, body string) *httptest.Server { func tokenServer(t *testing.T, body string) *httptest.Server {
t.Helper() t.Helper()
@@ -126,6 +144,36 @@ func TestCallbackExchangesAndRecordsConnection(t *testing.T) {
require.Equal(t, wantRef, conns.conn.TokenRef) require.Equal(t, wantRef, conns.conn.TokenRef)
} }
func TestCallbackTriggersDiscovery(t *testing.T) {
srv := tokenServer(t,
`{"access_token":"at","refresh_token":"rt-secret","token_type":"Bearer","expires_in":3600}`)
app := newConnectApp(t, srv.URL, &fakeWriter{}, &fakeConns{})
trig := &fakeTrigger{}
app.Connect.Discovery = trig
state := connectState(t, app)
rec := do(t, app, httptest.NewRequest(http.MethodGet,
"/oauth/youtube/callback?state="+state+"&code=the-code", nil))
require.Equal(t, http.StatusSeeOther, rec.Code)
require.Equal(t, []string{userID}, trig.seen(),
"a successful connect must trigger discovery for the connecting user")
}
func TestCallbackNoDiscoveryOnFailedConnect(t *testing.T) {
srv := tokenServer(t,
`{"access_token":"at","refresh_token":"rt","token_type":"Bearer","expires_in":3600}`)
app := newConnectApp(t, srv.URL, &fakeWriter{}, &fakeConns{})
trig := &fakeTrigger{}
app.Connect.Discovery = trig
// No state → CSRF reject → nothing connected, so no discovery.
rec := do(t, app, httptest.NewRequest(http.MethodGet,
"/oauth/youtube/callback?code=the-code", nil))
require.Equal(t, http.StatusBadRequest, rec.Code)
require.Empty(t, trig.seen(), "a failed connect must not trigger discovery")
}
func TestCallbackRejectsMissingState(t *testing.T) { func TestCallbackRejectsMissingState(t *testing.T) {
srv := tokenServer(t, srv := tokenServer(t,
`{"access_token":"at","refresh_token":"rt","token_type":"Bearer","expires_in":3600}`) `{"access_token":"at","refresh_token":"rt","token_type":"Bearer","expires_in":3600}`)
+100 -10
View File
@@ -3,6 +3,8 @@ package web
import ( import (
"context" "context"
"errors" "errors"
"html/template"
"io"
"log/slog" "log/slog"
"net/http" "net/http"
"time" "time"
@@ -10,6 +12,7 @@ import (
"github.com/a-h/templ" "github.com/a-h/templ"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store" "gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"gitea.d-ma.be/mathias/tapir/internal/domain"
) )
// Store is the read/write surface the web handlers depend on — a narrow port over // Store is the read/write surface the web handlers depend on — a narrow port over
@@ -18,6 +21,9 @@ import (
// fake without a database. // fake without a database.
type Store interface { type Store interface {
ListVideos(ctx context.Context, userID string, limit int) ([]store.SummaryRow, error) ListVideos(ctx context.Context, userID string, limit int) ([]store.SummaryRow, error)
// DistinctChannels lists the user's source channels — the options for the
// feed's channel multi-select filter.
DistinctChannels(ctx context.Context, userID string) ([]string, error)
GetSummaryByVideo(ctx context.Context, userID, videoID string) (*store.SummaryRow, error) GetSummaryByVideo(ctx context.Context, userID, videoID string) (*store.SummaryRow, error)
GetVideoRow(ctx context.Context, userID, videoID string) (*store.SummaryRow, error) GetVideoRow(ctx context.Context, userID, videoID string) (*store.SummaryRow, error)
ActionsFor(ctx context.Context, userID string, videoIDs []string) (map[string][]string, error) ActionsFor(ctx context.Context, userID string, videoIDs []string) (map[string][]string, error)
@@ -30,6 +36,10 @@ type Store interface {
SetAutoSummarize(ctx context.Context, userID string, enabled bool) error SetAutoSummarize(ctx context.Context, userID string, enabled bool) error
RequestSummarize(ctx context.Context, userID, videoID string) error RequestSummarize(ctx context.Context, userID, videoID string) error
// UpsertVideo persists a pasted video (idempotent on user+provider+video id,
// so it also dedups) and returns its durable store id.
UpsertVideo(ctx context.Context, v domain.Video) (string, error)
// Account management (the /account page, disconnect, delete-account). // Account management (the /account page, disconnect, delete-account).
ConnectionsForUser(ctx context.Context, userID string) ([]store.Connection, error) ConnectionsForUser(ctx context.Context, userID string) ([]store.Connection, error)
DeleteConnection(ctx context.Context, userID, provider string) error DeleteConnection(ctx context.Context, userID, provider string) error
@@ -78,6 +88,9 @@ type App struct {
// background goroutine (the "Summarize" button kicks it off). Nil = queue-only: // background goroutine (the "Summarize" button kicks it off). Nil = queue-only:
// the button flips the DB flag and the next `tapir run` does the work. // the button flips the DB flag and the next `tapir run` does the work.
Processor Processor Processor Processor
// Fetcher, when non-nil, resolves an arbitrary YouTube video id to metadata for
// the paste-a-URL flow (Feature 2). Nil = the /paste route is not mounted.
Fetcher VideoFetcher
// Processing tracks in-flight immediate summarizations so the status endpoint // Processing tracks in-flight immediate summarizations so the status endpoint
// shows the animation until the summary lands. The zero value is ready to use. // shows the animation until the summary lands. The zero value is ready to use.
Processing ProcessingSet Processing ProcessingSet
@@ -132,6 +145,9 @@ func (a *App) Router() http.Handler {
app.HandleFunc("POST /v/{videoId}/action", a.handleAction) app.HandleFunc("POST /v/{videoId}/action", a.handleAction)
app.HandleFunc("POST /v/{videoId}/summarize", a.handleRequestSummarize) app.HandleFunc("POST /v/{videoId}/summarize", a.handleRequestSummarize)
app.HandleFunc("POST /v/{videoId}/retry-now", a.handleRetryNow) app.HandleFunc("POST /v/{videoId}/retry-now", a.handleRetryNow)
if a.Fetcher != nil {
app.HandleFunc("POST /paste", a.handlePaste)
}
app.HandleFunc("GET /v/{videoId}/status", a.handleStatus) app.HandleFunc("GET /v/{videoId}/status", a.handleStatus)
app.HandleFunc("GET /register", a.handleRegisterForm) app.HandleFunc("GET /register", a.handleRegisterForm)
app.HandleFunc("POST /register", a.handleRegister) app.HandleFunc("POST /register", a.handleRegister)
@@ -183,7 +199,7 @@ func (a *App) handleList(w http.ResponseWriter, r *http.Request) {
} }
q := r.URL.Query() q := r.URL.Query()
f := Filter{ f := Filter{
Channel: q.Get("channel"), Channels: nonEmptyStrings(q["channel"]),
From: q.Get("from"), From: q.Get("from"),
To: q.Get("to"), To: q.Get("to"),
OnlySummarized: q.Get("summarized") == "1", OnlySummarized: q.Get("summarized") == "1",
@@ -198,25 +214,30 @@ func (a *App) handleList(w http.ResponseWriter, r *http.Request) {
rows := f.apply(allRows) rows := f.apply(allRows)
buckets := bucketRows(rows, a.recencyCutoff()) buckets := bucketRows(rows, a.recencyCutoff())
// hasConnected drives the empty state: a fresh account with a connection but // Channel options for the multi-select filter (the user's source channels).
// no discovery pass yet has zero rows, and we want it to read "connected, channels, err := a.Store.DistinctChannels(r.Context(), userID)
// summaries land gradually" rather than "nothing here". Only needed when the if err != nil {
// list is empty. a.serverError(w, r, "distinct channels", err)
hasConnected := false return
if buckets.empty() { }
// hasConnected drives both the paste box (shown to ANY connected user, #2) and
// the empty-state copy (a fresh account with a connection but no discovery pass
// yet reads "connected, summaries land gradually" rather than "nothing here").
// Computed every render — not only when empty — so a user with videos still
// gets the paste box.
conns, err := a.Store.ConnectionsForUser(r.Context(), userID) conns, err := a.Store.ConnectionsForUser(r.Context(), userID)
if err != nil { if err != nil {
a.serverError(w, r, "connections for user", err) a.serverError(w, r, "connections for user", err)
return return
} }
hasConnected = len(conns) > 0 hasConnected := len(conns) > 0
}
if isHTMX(r) { if isHTMX(r) {
a.render(w, r, summaryList(buckets, hasConnected)) a.render(w, r, summaryList(buckets, hasConnected))
return return
} }
a.render(w, r, ListPage(buckets, f, stats, takeFlash(w, r), hasConnected)) a.render(w, r, ListPage(buckets, f, stats, takeFlash(w, r), hasConnected, channels))
} }
// handleDetail renders one summary in full (highlights, takeaways, action group). // handleDetail renders one summary in full (highlights, takeaways, action group).
@@ -324,6 +345,75 @@ func (a *App) handleRequestSummarize(w http.ResponseWriter, r *http.Request) {
a.render(w, r, VideoCard(*row)) a.render(w, r, VideoCard(*row))
} }
// handlePaste handles "paste a YouTube URL" (Feature 2). It parses the video id,
// fetches metadata (Data API — ungated), upserts a subscription-less video row
// scoped to the user (idempotent, so it also dedups), and — if the video isn't
// already summarized — requests a summary and kicks off immediate processing
// through the SAME rate gate as the Summarize button. An explicit paste is a
// manual request, so it summarizes regardless of the recency window. A video that
// turns out to have no captions resolves to the honest "no transcript" terminal
// state via the engine (ADR-010), not an error here.
func (a *App) handlePaste(w http.ResponseWriter, r *http.Request) {
userID, ok := a.currentUserID(w, r)
if !ok {
return
}
videoID, err := parseYouTubeVideoID(r.FormValue("url"))
if err != nil {
a.pasteFailure(w, http.StatusBadRequest, "That doesn't look like a YouTube video link.")
return
}
v, err := a.Fetcher.FetchVideo(r.Context(), userID, videoID)
if errors.Is(err, domain.ErrVideoNotFound) {
a.pasteFailure(w, http.StatusNotFound, "That video couldn't be found — it may be private or removed.")
return
}
if err != nil {
a.serverError(w, r, "paste fetch", err)
return
}
id, err := a.Store.UpsertVideo(r.Context(), v)
if err != nil {
a.serverError(w, r, "paste upsert", err)
return
}
row, err := a.Store.GetVideoRow(r.Context(), userID, id)
if err != nil {
a.serverError(w, r, "paste get video", err)
return
}
// Dedup: already in the feed with a summary — surface the existing entry,
// don't re-summarize.
if row.Summarized {
a.render(w, r, VideoCard(*row))
return
}
// New or unsummarized: queue + (if a Processor is wired) summarize now, through
// the shared gate. RequestSummarize makes it durable even if the process dies.
if err := a.Store.RequestSummarize(r.Context(), userID, id); err != nil {
a.serverError(w, r, "paste request summarize", err)
return
}
if a.Processor != nil {
a.startProcessing(userID, id)
a.render(w, r, processingCard(*row))
return
}
a.render(w, r, VideoCard(*row))
}
// pasteFailure renders a minimal inline error fragment for the paste form (HTMX
// swaps it in). No templ dependency so it renders even on a bad-input fast path.
func (a *App) pasteFailure(w http.ResponseWriter, status int, msg string) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(status)
_, _ = io.WriteString(w, `<p class="paste-error" role="alert">`+template.HTMLEscapeString(msg)+`</p>`)
}
// handleRetryNow handles the "Try now" button on rate-limited video cards. It // handleRetryNow handles the "Try now" button on rate-limited video cards. It
// clears the rate_limited_at backoff so the scheduler won't skip the video, then // clears the rate_limited_at backoff so the scheduler won't skip the video, then
// triggers an immediate ProcessVideo — same background path as handleRequestSummarize. // triggers an immediate ProcessVideo — same background path as handleRequestSummarize.
+5 -3
View File
@@ -207,13 +207,15 @@ func TestListChannelFilter(t *testing.T) {
resetDB(t, p) resetDB(t, p)
require.NoError(t, deliver(ctx, app, videoX, "body x")) require.NoError(t, deliver(ctx, app, videoX, "body x"))
seedVideo(t, p, videoX, "X Title", "https://x", time.Time{}) seedVideo(t, p, videoX, "X Title", "https://x", time.Time{})
_, err := p.Exec(ctx, `UPDATE videos SET channel_title = 'Acme Channel' WHERE id = $1`, videoX)
require.NoError(t, err)
// Channel is "youtube" for seeded rows; a non-matching filter hides them. // Selecting a different channel hides the row; selecting its channel shows it.
rec := do(t, app, httptest.NewRequest(http.MethodGet, "/?channel=vimeo", nil)) rec := do(t, app, httptest.NewRequest(http.MethodGet, "/?channel=Other+Channel", nil))
require.Equal(t, http.StatusOK, rec.Code) require.Equal(t, http.StatusOK, rec.Code)
require.NotContains(t, body(t, rec), "X Title") require.NotContains(t, body(t, rec), "X Title")
rec = do(t, app, httptest.NewRequest(http.MethodGet, "/?channel=youtube", nil)) rec = do(t, app, httptest.NewRequest(http.MethodGet, "/?channel=Acme+Channel", nil))
require.Contains(t, body(t, rec), "X Title") require.Contains(t, body(t, rec), "X Title")
} }
+62
View File
@@ -0,0 +1,62 @@
package web
import (
"fmt"
"net/url"
"regexp"
"strings"
)
// youtubeVideoID matches a canonical YouTube video id: exactly 11 URL-safe chars.
var youtubeVideoID = regexp.MustCompile(`^[A-Za-z0-9_-]{11}$`)
// parseYouTubeVideoID extracts the 11-character video id from a pasted YouTube
// URL (watch?v=, youtu.be/, shorts/, embed/) or a bare id. It rejects non-YouTube
// hosts and anything that doesn't yield a valid id, so the paste flow never tries
// to fetch a video that can't exist (Feature 2).
func parseYouTubeVideoID(raw string) (string, error) {
s := strings.TrimSpace(raw)
if s == "" {
return "", fmt.Errorf("empty input")
}
// Bare id (no URL) — accept directly.
if youtubeVideoID.MatchString(s) {
return s, nil
}
// Accept scheme-less URLs (youtube.com/watch?v=...) by giving url.Parse a host.
if !strings.Contains(s, "://") {
s = "https://" + s
}
u, err := url.Parse(s)
if err != nil {
return "", fmt.Errorf("not a URL: %w", err)
}
host := strings.ToLower(u.Hostname())
isYouTube := host == "youtu.be" || host == "youtube.com" || strings.HasSuffix(host, ".youtube.com")
if !isYouTube {
return "", fmt.Errorf("not a YouTube URL: %q", host)
}
var id string
switch {
case host == "youtu.be":
// youtu.be/<id>
id = strings.Trim(u.Path, "/")
case u.Path == "/watch":
id = u.Query().Get("v")
default:
// /shorts/<id>, /embed/<id>
parts := strings.Split(strings.Trim(u.Path, "/"), "/")
if len(parts) == 2 && (parts[0] == "shorts" || parts[0] == "embed") {
id = parts[1]
}
}
if !youtubeVideoID.MatchString(id) {
return "", fmt.Errorf("no YouTube video id in %q", raw)
}
return id, nil
}
+134
View File
@@ -0,0 +1,134 @@
package web_test
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"gitea.d-ma.be/mathias/tapir/internal/domain"
)
// fakeFetcher is a web.VideoFetcher returning a fixed video (or an error),
// scoped to whatever (userID, videoID) the handler asks for.
type fakeFetcher struct {
title string
err error
calls int
}
func (f *fakeFetcher) FetchVideo(_ context.Context, userID, videoID string) (domain.Video, error) {
f.calls++
if f.err != nil {
return domain.Video{}, f.err
}
return domain.Video{
UserID: userID,
Provider: domain.ProviderYouTube,
ProviderVideoID: videoID,
Title: f.title,
URL: "https://www.youtube.com/watch?v=" + videoID,
}, nil
}
func pasteReq(rawURL string) *http.Request {
req := httptest.NewRequest(http.MethodPost, "/paste",
strings.NewReader("url="+url.QueryEscape(rawURL)))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
return req
}
func TestPasteValidURLAddsAndRequests(t *testing.T) {
ctx := context.Background()
app := newApp(t)
resetDB(t, rawPool(t))
app.Fetcher = &fakeFetcher{title: "Pasted Talk"}
p := rawPool(t)
rec := do(t, app, pasteReq("https://youtu.be/dQw4w9WgXcQ"))
require.Equal(t, http.StatusOK, rec.Code)
var (
count, requested int
title string
)
require.NoError(t, p.QueryRow(ctx,
`SELECT count(*), coalesce(max(title),'') FROM videos
WHERE user_id=$1 AND provider_video_id='dQw4w9WgXcQ'`, userID).Scan(&count, &title))
require.Equal(t, 1, count, "pasted video added once, scoped to the user")
require.Equal(t, "Pasted Talk", title)
require.NoError(t, p.QueryRow(ctx,
`SELECT count(*) FROM videos
WHERE user_id=$1 AND provider_video_id='dQw4w9WgXcQ' AND summarize_requested`,
userID).Scan(&requested))
require.Equal(t, 1, requested, "pasted video is queued for summarization (through the gate)")
}
func TestPasteInvalidURLRejected(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
app.Fetcher = &fakeFetcher{title: "x"}
rec := do(t, app, pasteReq("definitely not a url"))
require.Equal(t, http.StatusBadRequest, rec.Code)
var count int
require.NoError(t, rawPool(t).QueryRow(context.Background(),
`SELECT count(*) FROM videos WHERE user_id=$1`, userID).Scan(&count))
require.Equal(t, 0, count, "invalid input adds nothing")
}
func TestPasteVideoNotFound(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
app.Fetcher = &fakeFetcher{err: domain.ErrVideoNotFound}
rec := do(t, app, pasteReq("https://youtu.be/dQw4w9WgXcQ"))
require.Equal(t, http.StatusNotFound, rec.Code)
var count int
require.NoError(t, rawPool(t).QueryRow(context.Background(),
`SELECT count(*) FROM videos WHERE user_id=$1`, userID).Scan(&count))
require.Equal(t, 0, count, "a not-found video adds nothing")
}
func TestPasteDedupNoDuplicate(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
app.Fetcher = &fakeFetcher{title: "Pasted Talk"}
require.Equal(t, http.StatusOK, do(t, app, pasteReq("https://youtu.be/dQw4w9WgXcQ")).Code)
require.Equal(t, http.StatusOK, do(t, app, pasteReq("https://www.youtube.com/watch?v=dQw4w9WgXcQ")).Code)
var count int
require.NoError(t, rawPool(t).QueryRow(context.Background(),
`SELECT count(*) FROM videos WHERE user_id=$1 AND provider_video_id='dQw4w9WgXcQ'`,
userID).Scan(&count))
require.Equal(t, 1, count, "pasting the same video twice must not duplicate the row")
}
func TestListShowsPasteFormForConnectedUserWithVideos(t *testing.T) {
app := newApp(t)
resetDB(t, rawPool(t))
app.Fetcher = &fakeFetcher{title: "x"}
p := rawPool(t)
// Connected user with a non-empty feed (the case the bug missed: hasConnected
// was only computed for an empty feed).
_, err := p.Exec(context.Background(),
`INSERT INTO video_connections (user_id, provider, token_ref, status)
VALUES ($1, 'youtube', 'youtube/x/refresh_token', 'active')`, userID)
require.NoError(t, err)
seedVideo(t, p, "11111111-1111-1111-1111-111111111111", "A talk", "https://youtu.be/aaaaaaaaaaa", time.Now())
rec := do(t, app, httptest.NewRequest(http.MethodGet, "/", nil))
require.Equal(t, http.StatusOK, rec.Code)
require.Contains(t, body(t, rec), `action="/paste"`,
"a connected user must see the paste box even when the feed has videos")
}
+97
View File
@@ -0,0 +1,97 @@
package web
import (
"bytes"
"context"
"gitea.d-ma.be/mathias/tapir/internal/adapters/store"
"strings"
"testing"
)
func TestParseYouTubeVideoID(t *testing.T) {
const id = "dQw4w9WgXcQ"
ok := []struct {
name, in string
}{
{"watch", "https://www.youtube.com/watch?v=" + id},
{"watch no www", "https://youtube.com/watch?v=" + id},
{"watch m", "https://m.youtube.com/watch?v=" + id},
{"watch extra params", "https://www.youtube.com/watch?v=" + id + "&t=42s&list=PLxyz"},
{"watch param after", "https://www.youtube.com/watch?list=PLxyz&v=" + id},
{"short link", "https://youtu.be/" + id},
{"short link param", "https://youtu.be/" + id + "?si=abcd&t=1"},
{"shorts", "https://www.youtube.com/shorts/" + id},
{"embed", "https://www.youtube.com/embed/" + id},
{"bare id", id},
{"http scheme", "http://youtube.com/watch?v=" + id},
{"no scheme", "youtube.com/watch?v=" + id},
{"trailing space", " https://youtu.be/" + id + " "},
}
for _, c := range ok {
t.Run(c.name, func(t *testing.T) {
got, err := parseYouTubeVideoID(c.in)
if err != nil {
t.Fatalf("parseYouTubeVideoID(%q) error: %v", c.in, err)
}
if got != id {
t.Fatalf("parseYouTubeVideoID(%q) = %q, want %q", c.in, got, id)
}
})
}
bad := []struct {
name, in string
}{
{"empty", ""},
{"blank", " "},
{"vimeo", "https://vimeo.com/123456789"},
{"other host", "https://example.com/watch?v=" + id},
{"watch no id", "https://www.youtube.com/watch?v="},
{"short id", "https://youtu.be/abc"},
{"long id", "https://youtu.be/" + id + "extra"},
{"bad chars", "https://youtu.be/dQw4w9Wg!cQ"},
{"not a url", "just some text"},
{"channel url", "https://www.youtube.com/@somechannel"},
}
for _, c := range bad {
t.Run("reject "+c.name, func(t *testing.T) {
if got, err := parseYouTubeVideoID(c.in); err == nil {
t.Fatalf("parseYouTubeVideoID(%q) = %q, want error", c.in, got)
}
})
}
}
func TestListPageShowsPasteFormOnlyWhenConnected(t *testing.T) {
render := func(connected bool) string {
var buf bytes.Buffer
if err := ListPage(listBuckets{}, Filter{}, PipelineStats{}, "", connected, nil).Render(context.Background(), &buf); err != nil {
t.Fatalf("render: %v", err)
}
return buf.String()
}
html := render(true)
if !strings.Contains(html, `name="url"`) || !strings.Contains(html, `action="/paste"`) {
t.Errorf("connected feed must show the paste form")
}
if strings.Contains(render(false), `name="url"`) {
t.Errorf("disconnected feed must not show the paste form")
}
}
func TestFilterMatchesMultipleChannels(t *testing.T) {
f := Filter{Channels: []string{"Acme", "Zeta"}}
row := func(ch string) store.SummaryRow { return store.SummaryRow{ChannelTitle: ch, Summarized: true} }
rows := []store.SummaryRow{row("Acme"), row("Beta"), row("Zeta")}
got := f.apply(rows)
if len(got) != 2 || got[0].ChannelTitle != "Acme" || got[1].ChannelTitle != "Zeta" {
t.Fatalf("multi-channel filter = %+v, want Acme+Zeta only", got)
}
// Empty selection = no channel constraint (all pass).
if n := len(Filter{}.apply(rows)); n != 3 {
t.Fatalf("no channel filter should pass all rows, got %d", n)
}
}
+10
View File
@@ -3,6 +3,8 @@ package web
import ( import (
"context" "context"
"sync" "sync"
"gitea.d-ma.be/mathias/tapir/internal/domain"
) )
// Processor runs the core summarization use case for a single already-discovered // Processor runs the core summarization use case for a single already-discovered
@@ -14,6 +16,14 @@ type Processor interface {
ProcessVideo(ctx context.Context, userID, videoID string) error ProcessVideo(ctx context.Context, userID, videoID string) error
} }
// VideoFetcher resolves an arbitrary YouTube video id to its metadata for the
// paste-a-URL flow (Feature 2). It is a Data API call, NOT the rate-limited
// caption path. Returns domain.ErrVideoNotFound for a deleted/private/typo'd id.
// cmd/tapir wires a per-user YouTube adapter; nil disables the paste route.
type VideoFetcher interface {
FetchVideo(ctx context.Context, userID, videoID string) (domain.Video, error)
}
// ProcessingSet tracks the (user, video) ids currently being summarized in-process // ProcessingSet tracks the (user, video) ids currently being summarized in-process
// so the status endpoint can show the animation until the summary lands. It is // so the status endpoint can show the animation until the summary lands. It is
// ephemeral (single-instance Stage-1): a restart drops it, and the DB holds the // ephemeral (single-instance Stage-1): a restart drops it, and the DB holds the
+27 -5
View File
@@ -2,6 +2,7 @@ package web
import ( import (
"regexp" "regexp"
"slices"
"strings" "strings"
"time" "time"
"unicode/utf8" "unicode/utf8"
@@ -333,7 +334,7 @@ type flashView struct {
// flashMessages maps each flash code to its banner. An unknown code renders no // flashMessages maps each flash code to its banner. An unknown code renders no
// banner (flashFor returns ok=false), so a forged cookie value is inert. // banner (flashFor returns ok=false), so a forged cookie value is inert.
var flashMessages = map[string]flashView{ var flashMessages = map[string]flashView{
flashConnected: {"success", "YouTube account connected."}, flashConnected: {"success", "YouTube account connected — finding your subscriptions. Your newest videos will appear below as they're summarized."},
flashConnectFailed: {"error", "Could not connect your YouTube account. Please try again."}, flashConnectFailed: {"error", "Could not connect your YouTube account. Please try again."},
flashDisconnected: {"success", "Account disconnected."}, flashDisconnected: {"success", "Account disconnected."},
flashDeleted: {"success", "Your account and all its data were deleted."}, flashDeleted: {"success", "Your account and all its data were deleted."},
@@ -469,7 +470,7 @@ func (b listBuckets) empty() bool {
// Dates are kept as the raw YYYY-MM-DD strings so the form re-renders the user's // Dates are kept as the raw YYYY-MM-DD strings so the form re-renders the user's
// input verbatim; parsing happens in matchFilter. // input verbatim; parsing happens in matchFilter.
type Filter struct { type Filter struct {
Channel string Channels []string // selected channel titles; empty = all channels
From string From string
To string To string
OnlySummarized bool // show only videos that have a summary OnlySummarized bool // show only videos that have a summary
@@ -480,7 +481,28 @@ type Filter struct {
// filter) the bar is hidden so the connect CTA stands alone (UX review C1); a // filter) the bar is hidden so the connect CTA stands alone (UX review C1); a
// filter that happens to match nothing still shows the bar so it can be cleared. // filter that happens to match nothing still shows the bar so it can be cleared.
func (f Filter) active() bool { func (f Filter) active() bool {
return f.Channel != "" || f.From != "" || f.To != "" || f.OnlySummarized return len(f.Channels) > 0 || f.From != "" || f.To != "" || f.OnlySummarized
}
// HasChannel reports whether a channel is currently selected (drives the
// multi-select's selected state in the view).
func (f Filter) HasChannel(c string) bool {
return slices.Contains(f.Channels, c)
}
// nonEmptyStrings drops blank entries. A channel multi-select submits real
// channel titles; this guards against a stray empty value reaching the filter.
func nonEmptyStrings(ss []string) []string {
out := ss[:0:0]
for _, s := range ss {
if strings.TrimSpace(s) != "" {
out = append(out, s)
}
}
if len(out) == 0 {
return nil
}
return out
} }
// matches reports whether a row satisfies the filter. Channel is an exact match; // matches reports whether a row satisfies the filter. Channel is an exact match;
@@ -491,7 +513,7 @@ func (f Filter) matches(r store.SummaryRow) bool {
if f.OnlySummarized && !r.Summarized { if f.OnlySummarized && !r.Summarized {
return false return false
} }
if f.Channel != "" && r.Channel != f.Channel { if len(f.Channels) > 0 && !slices.Contains(f.Channels, r.ChannelTitle) {
return false return false
} }
if from, ok := parseDate(f.From); ok { if from, ok := parseDate(f.From); ok {
@@ -521,7 +543,7 @@ func parseDate(s string) (time.Time, bool) {
// apply returns the subset of rows matching the filter, preserving order. // apply returns the subset of rows matching the filter, preserving order.
func (f Filter) apply(rows []store.SummaryRow) []store.SummaryRow { func (f Filter) apply(rows []store.SummaryRow) []store.SummaryRow {
if f.Channel == "" && f.From == "" && f.To == "" && !f.OnlySummarized { if len(f.Channels) == 0 && f.From == "" && f.To == "" && !f.OnlySummarized {
return rows return rows
} }
out := rows[:0:0] out := rows[:0:0]
+38 -4
View File
@@ -104,11 +104,14 @@ templ flashBanner(code string) {
// #summary-list region; a non-HTMX request renders the whole page. flash carries // #summary-list region; a non-HTMX request renders the whole page. flash carries
// a one-shot notification (e.g. "connected", "registered") surfaced on arrival // a one-shot notification (e.g. "connected", "registered") surfaced on arrival
// after a POST→redirect. // after a POST→redirect.
templ ListPage(b listBuckets, f Filter, stats PipelineStats, flash string, hasConnected bool) { templ ListPage(b listBuckets, f Filter, stats PipelineStats, flash string, hasConnected bool, channels []string) {
@Layout("Tapir — Summaries") { @Layout("Tapir — Summaries") {
@flashBanner(flash) @flashBanner(flash)
if hasConnected {
@pasteForm()
}
if !b.empty() || f.active() { if !b.empty() || f.active() {
@filterForm(f) @filterForm(f, channels)
} }
if stats.RateLimited > 0 || stats.Pending > 0 || stats.NoText > 0 { if stats.RateLimited > 0 || stats.Pending > 0 || stats.NoText > 0 {
@pipelineBar(stats) @pipelineBar(stats)
@@ -143,7 +146,29 @@ templ pipelineBar(s PipelineStats) {
</div> </div>
} }
templ filterForm(f Filter) { // pasteForm lets a connected user summarize any YouTube video by pasting its URL
// (Feature 2). The result (a video card, or an inline error) swaps into
// #paste-result; the next list refresh shows it inline. Summarization runs
// through the shared caption rate gate like every other fetch.
templ pasteForm() {
<form
class="paste"
method="post"
action="/paste"
hx-post="/paste"
hx-target="#paste-result"
hx-swap="innerHTML"
>
<label>
Summarize any video
<input type="url" name="url" placeholder="Paste a YouTube link…" required/>
</label>
<button type="submit">Add</button>
</form>
<div id="paste-result"></div>
}
templ filterForm(f Filter, channels []string) {
<form <form
class="filters" class="filters"
method="get" method="get"
@@ -153,7 +178,16 @@ templ filterForm(f Filter) {
hx-swap="innerHTML" hx-swap="innerHTML"
hx-indicator="#filter-indicator" hx-indicator="#filter-indicator"
> >
<label>Channel <input type="text" name="channel" value={ f.Channel } placeholder="any"/></label> if len(channels) > 0 {
<label>
Channels
<select name="channel" multiple size="4">
for _, c := range channels {
<option value={ c } selected?={ f.HasChannel(c) }>{ c }</option>
}
</select>
</label>
}
<label class="filter-check"> <label class="filter-check">
<input type="checkbox" name="summarized" value="1" if f.OnlySummarized { checked }/> <input type="checkbox" name="summarized" value="1" if f.OnlySummarized { checked }/>
Summarized only Summarized only
File diff suppressed because it is too large Load Diff
@@ -38,7 +38,15 @@ var scenarioCoverage = map[string]string{
// connect_account.feature // connect_account.feature
"Connect a YouTube account": "TestCallbackExchangesAndRecordsConnection", "Connect a YouTube account": "TestCallbackExchangesAndRecordsConnection",
"Connecting an account discovers videos immediately": "TestCallbackTriggersDiscovery",
"Connecting summarizes my newest videos right away": "TestNewestUnsummarizedVideoIDs",
"Tokens are never stored in the clear": "TestCallbackExchangesAndRecordsConnection", "Tokens are never stored in the clear": "TestCallbackExchangesAndRecordsConnection",
// paste_url.feature
"Paste a valid YouTube URL": "TestPasteValidURLAddsAndRequests",
"Pasting an invalid link is rejected": "TestPasteInvalidURLRejected",
"Pasting a video that cannot be found is honest": "TestPasteVideoNotFound",
"Pasting the same video twice does not duplicate it": "TestPasteDedupNoDuplicate",
"Revoking a connection stops watching but keeps history": "TestDisconnectRemovesTokenAndConnectionKeepsAccount", "Revoking a connection stops watching but keeps history": "TestDisconnectRemovesTokenAndConnectionKeepsAccount",
// summarize_mode.feature // summarize_mode.feature