Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a9be5f285b | ||
|
|
fe56e2fe01 | ||
|
|
beeb5bc31b | ||
|
|
09eb31d1fe |
+42
-4
@@ -827,10 +827,18 @@ The fix is resilience around it, not replacing it.
|
|||||||
**Decision.**
|
**Decision.**
|
||||||
1. **Ordered endpoint chain (`summarizer.NewChain`).** Endpoints are tried in order; the first to
|
1. **Ordered endpoint chain (`summarizer.NewChain`).** Endpoints are tried in order; the first to
|
||||||
return a *parseable* summary wins. Default chain:
|
return a *parseable* summary wins. Default chain:
|
||||||
`koala/phi4-mini` (primary, local) → `koala/phi4-14b` (fallback, local) →
|
`koala/phi4-mini` (primary, local) → `iguana/gemma4-26b` (fallback, local on a
|
||||||
`berget/mistral-small` (worst-case, external). All three are reached through the **one** LiteLLM
|
*different host*) → `berget/mistral-small` (worst-case, external). All three are reached through
|
||||||
gateway by alias — the gateway already fronts both llama-swap and berget — so a fallback is a
|
the **one** LiteLLM gateway by alias — the gateway already fronts both llama-swap and berget — so
|
||||||
different alias, not a second client config.
|
a fallback is a different alias, not a second client config.
|
||||||
|
|
||||||
|
**Update 2026-06-11:** the local fallback moved from `koala/phi4-14b` to `iguana/gemma4-26b`.
|
||||||
|
koala now carries other GPU loads, so keeping the fallback on koala competed with them; iguana
|
||||||
|
(M2 Ultra) has the headroom, and a different host is also a different egress IP for the rare
|
||||||
|
fallback fetch. `gemma4-26b` is the brain-validated homelab general-purpose model (agentsquad
|
||||||
|
H2/H3 executor) and returned valid summary JSON on the real prompt in a smoke test
|
||||||
|
(~37s incl. cold-load — fine for a path hit only when the fast primary fails). Pure config:
|
||||||
|
`TAPIR_FALLBACK_MODEL`.
|
||||||
2. **A parse failure advances the chain, same as a transport error.** "Reliably summarized" means
|
2. **A parse failure advances the chain, same as a transport error.** "Reliably summarized" means
|
||||||
*parseable summary returned*, not *HTTP 200*. This is the behaviour the old Primary→Fallback
|
*parseable summary returned*, not *HTTP 200*. This is the behaviour the old Primary→Fallback
|
||||||
shape missed.
|
shape missed.
|
||||||
@@ -958,6 +966,36 @@ schema change (reuses `transcript_status` from migration 007).
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## ADR-026 — Foreground caption fetches take priority; the credentials probe is dead
|
||||||
|
|
||||||
|
**Status:** Accepted (2026-06-10). **Pillar A of the manual-mode UX work** (Pillar B was
|
||||||
|
ADR-025). Builds on ADR-014 (the shared per-IP gate).
|
||||||
|
|
||||||
|
**Context.** Every caption fetch — the background sweep and the web click-path — shared one
|
||||||
|
process-wide rate gate equally. So a user waiting on a "Summarize" click competed with the
|
||||||
|
firehose for both pacing and the scarce pre-429 window; on a busy IP the click was slow or
|
||||||
|
429'd while the background churned.
|
||||||
|
|
||||||
|
**Decision.** A context-marked priority lane. The web path
|
||||||
|
(`engineProcessor.ProcessVideo`) wraps its context with `ForegroundContext`; the gate gives
|
||||||
|
foreground fetches a token immediately, while **background fetches yield** — they wait until no
|
||||||
|
foreground fetch is pending before taking a token. Threaded via a context value (not new
|
||||||
|
signatures) and a process-wide `foregroundPending` counter. Clicks are rare and bursty, so the
|
||||||
|
background barely loses throughput; the waiting human gets the next (and cleanest) slot.
|
||||||
|
|
||||||
|
**Credentials probe — rejected, not built.** The idea was to fetch captions with the user's
|
||||||
|
auth in manual mode to dodge 429s. It is a dead end, already settled by ADR-010 and the code:
|
||||||
|
the caption path is *deliberately anonymous* because the InnerTube/timedtext endpoints **reject
|
||||||
|
or break on authenticated requests** (`captions.go`: "no OAuth token — it can break the
|
||||||
|
timedtext endpoint"). The user's OAuth (a Data API credential) does not authenticate InnerTube
|
||||||
|
at all, and the official `captions.download` is owner-only (403 on third-party). So auth cannot
|
||||||
|
help here and can actively hurt. No probe needed — building one would only re-confirm the ADR.
|
||||||
|
|
||||||
|
**Reversibility.** Context-marker + a yield loop in the gate; removing the marker collapses to
|
||||||
|
the prior equal-share behaviour. No schema or API change.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Rejected alternatives
|
## Rejected alternatives
|
||||||
|
|
||||||
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
|
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
|
||||||
|
|||||||
@@ -113,6 +113,11 @@ type engineProcessor struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (p *engineProcessor) ProcessVideo(ctx context.Context, userID, videoID string) error {
|
func (p *engineProcessor) ProcessVideo(ctx context.Context, userID, videoID string) error {
|
||||||
|
// This is the user-initiated (foreground) path — a click on "Summarize",
|
||||||
|
// "Try now", or a pasted URL. Mark the context so the caption gate gives it
|
||||||
|
// priority over the background sweep (ADR-026, Pillar A).
|
||||||
|
ctx = youtube.ForegroundContext(ctx)
|
||||||
|
|
||||||
row, err := p.store.GetVideoRow(ctx, userID, videoID)
|
row, err := p.store.GetVideoRow(ctx, userID, videoID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("load video %q: %w", videoID, err)
|
return fmt.Errorf("load video %q: %w", videoID, err)
|
||||||
|
|||||||
+13
-4
@@ -143,8 +143,18 @@ func runScheduler(
|
|||||||
return // disabled
|
return // disabled
|
||||||
}
|
}
|
||||||
|
|
||||||
pass := 0
|
// Derive the rotation offset from wall-clock, NOT an in-memory counter. A
|
||||||
runDiscoveryPass(ctx, pass, lister, runUser, log)
|
// counter reset to 0 on every pod restart always hands the lead to the
|
||||||
|
// first-listed user — so frequent deploys re-starve whoever is last (exactly
|
||||||
|
// what happened to the first pilot user during a deploy-heavy session). A
|
||||||
|
// time-based offset advances with real time and is identical across restarts,
|
||||||
|
// so the lead rotates fairly regardless of how often the pod bounces.
|
||||||
|
runPass := func() {
|
||||||
|
pass := int(time.Now().Unix() / int64(interval/time.Second))
|
||||||
|
runDiscoveryPass(ctx, pass, lister, runUser, log)
|
||||||
|
}
|
||||||
|
|
||||||
|
runPass()
|
||||||
|
|
||||||
ticker := time.NewTicker(interval)
|
ticker := time.NewTicker(interval)
|
||||||
defer ticker.Stop()
|
defer ticker.Stop()
|
||||||
@@ -153,8 +163,7 @@ func runScheduler(
|
|||||||
case <-ctx.Done():
|
case <-ctx.Done():
|
||||||
return
|
return
|
||||||
case <-ticker.C:
|
case <-ticker.C:
|
||||||
pass++
|
runPass()
|
||||||
runDiscoveryPass(ctx, pass, lister, runUser, log)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -30,7 +30,9 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-06),
|
|||||||
- **Summarizer fallback chain (ADR-022).** The primary alias is the *first* of an ordered chain;
|
- **Summarizer fallback chain (ADR-022).** The primary alias is the *first* of an ordered chain;
|
||||||
on failure or unparseable output the summarizer advances to the next model. All reached through
|
on failure or unparseable output the summarizer advances to the next model. All reached through
|
||||||
the same gateway by alias.
|
the same gateway by alias.
|
||||||
- `TAPIR_FALLBACK_MODEL` — local fallback. **Default `koala/phi4-14b`.** Empty disables it.
|
- `TAPIR_FALLBACK_MODEL` — local fallback. **Default `iguana/gemma4-26b`** — on iguana, NOT
|
||||||
|
koala, so the fallback does not compete with koala's other GPU loads (and runs from a different
|
||||||
|
egress IP). Empty disables it.
|
||||||
- `TAPIR_CLOUD_FALLBACK_MODEL` — worst-case EXTERNAL fallback. **Default `berget/mistral-small`.**
|
- `TAPIR_CLOUD_FALLBACK_MODEL` — worst-case EXTERNAL fallback. **Default `berget/mistral-small`.**
|
||||||
**Set this empty (`""`) for any client/NDA deployment** so content never leaves the local
|
**Set this empty (`""`) for any client/NDA deployment** so content never leaves the local
|
||||||
stack — the chain then contains only local endpoints.
|
stack — the chain then contains only local endpoints.
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
|
"path/filepath"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
|
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
|
||||||
@@ -24,11 +25,22 @@ var _ ports.Sink = (*store.Store)(nil)
|
|||||||
var dsn string
|
var dsn string
|
||||||
|
|
||||||
func TestMain(m *testing.M) {
|
func TestMain(m *testing.M) {
|
||||||
const port = 54329
|
// Port + runtime/data dirs are per-process (PID-derived) so two concurrent
|
||||||
|
// `go test` invocations — e.g. a push-run and a tag-run firing together in CI —
|
||||||
|
// don't collide on a fixed port or a shared data dir (which silently failed
|
||||||
|
// both runs). CachePath is shared so the PG archive is downloaded once, not
|
||||||
|
// per process. Base 54000 keeps this package's range distinct from web's.
|
||||||
|
port := uint32(54000 + os.Getpid()%1000)
|
||||||
dsn = fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port)
|
dsn = fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port)
|
||||||
|
|
||||||
|
rt := filepath.Join(os.TempDir(), fmt.Sprintf("tapir-epg-store-%d", os.Getpid()))
|
||||||
pg := embeddedpostgres.NewDatabase(
|
pg := embeddedpostgres.NewDatabase(
|
||||||
embeddedpostgres.DefaultConfig().Port(port),
|
embeddedpostgres.DefaultConfig().
|
||||||
|
Port(port).
|
||||||
|
RuntimePath(rt).
|
||||||
|
DataPath(filepath.Join(rt, "data")).
|
||||||
|
BinariesPath(filepath.Join(rt, "bin")).
|
||||||
|
CachePath(filepath.Join(os.TempDir(), "tapir-epg-cache")),
|
||||||
)
|
)
|
||||||
if err := pg.Start(); err != nil {
|
if err := pg.Start(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "embedded-postgres start: %v\n", err)
|
fmt.Fprintf(os.Stderr, "embedded-postgres start: %v\n", err)
|
||||||
@@ -40,6 +52,7 @@ func TestMain(m *testing.M) {
|
|||||||
if err := pg.Stop(); err != nil {
|
if err := pg.Stop(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "embedded-postgres stop: %v\n", err)
|
fmt.Fprintf(os.Stderr, "embedded-postgres stop: %v\n", err)
|
||||||
}
|
}
|
||||||
|
_ = os.RemoveAll(rt)
|
||||||
os.Exit(code)
|
os.Exit(code)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package youtube
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"golang.org/x/time/rate"
|
"golang.org/x/time/rate"
|
||||||
@@ -29,9 +30,55 @@ func SetFetchRate(interval time.Duration) {
|
|||||||
globalFetchGate = rate.NewLimiter(rate.Every(interval), 1)
|
globalFetchGate = rate.NewLimiter(rate.Every(interval), 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// foregroundPending counts in-flight foreground (user-initiated) caption fetches.
|
||||||
|
// The background sweep yields the gate while this is non-zero so a human waiting
|
||||||
|
// on a click gets the next slot — and, on a near-throttled IP, the pre-429 window
|
||||||
|
// — instead of competing equally with the firehose (ADR-026, Pillar A). Clicks are
|
||||||
|
// rare and bursty, so background barely notices; the win to the click is large.
|
||||||
|
var foregroundPending atomic.Int64
|
||||||
|
|
||||||
|
// fgCtxKey marks a context as foreground (user-initiated). Unexported; set via
|
||||||
|
// ForegroundContext and read via isForeground so only this package owns the key.
|
||||||
|
type fgCtxKey struct{}
|
||||||
|
|
||||||
|
// ForegroundContext marks ctx as a user-initiated (foreground) fetch so the gate
|
||||||
|
// gives it priority. The web "Summarize"/paste/retry path wraps its context with
|
||||||
|
// this; the background scheduler leaves it unset.
|
||||||
|
func ForegroundContext(ctx context.Context) context.Context {
|
||||||
|
return context.WithValue(ctx, fgCtxKey{}, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func isForeground(ctx context.Context) bool {
|
||||||
|
v, _ := ctx.Value(fgCtxKey{}).(bool)
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// fgYieldPoll is how often a background waiter re-checks whether a foreground
|
||||||
|
// fetch is still pending. Short enough to feel immediate, long enough not to spin.
|
||||||
|
const fgYieldPoll = 200 * time.Millisecond
|
||||||
|
|
||||||
// WaitFetchGate blocks until the process-wide gate allows one timedtext fetch,
|
// WaitFetchGate blocks until the process-wide gate allows one timedtext fetch,
|
||||||
// respecting ctx cancellation. Called from httpDo before every live outbound
|
// respecting ctx cancellation. Called from httpDo before every live outbound
|
||||||
// caption fetch so the scheduler and the click-path share the same egress budget.
|
// caption fetch so the scheduler and the click-path share the same egress budget.
|
||||||
|
//
|
||||||
|
// Foreground (user-initiated) fetches take priority: they register as pending and
|
||||||
|
// acquire a token immediately. Background fetches first yield — they wait until no
|
||||||
|
// foreground fetch is pending — so a live click is never stuck behind the
|
||||||
|
// background sweep and gets the cleaner slot against the per-IP limit (ADR-026).
|
||||||
func WaitFetchGate(ctx context.Context) error {
|
func WaitFetchGate(ctx context.Context) error {
|
||||||
|
if isForeground(ctx) {
|
||||||
|
foregroundPending.Add(1)
|
||||||
|
defer foregroundPending.Add(-1)
|
||||||
|
return globalFetchGate.Wait(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Background: defer to any pending foreground fetch before taking a token.
|
||||||
|
for foregroundPending.Load() > 0 {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
case <-time.After(fgYieldPoll):
|
||||||
|
}
|
||||||
|
}
|
||||||
return globalFetchGate.Wait(ctx)
|
return globalFetchGate.Wait(ctx)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -82,3 +82,37 @@ func TestSetFetchRateZeroIsUnlimited(t *testing.T) {
|
|||||||
require.NoError(t, WaitFetchGate(context.Background()))
|
require.NoError(t, WaitFetchGate(context.Background()))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestForegroundContextMarker(t *testing.T) {
|
||||||
|
require.False(t, isForeground(context.Background()), "plain context is background")
|
||||||
|
require.True(t, isForeground(ForegroundContext(context.Background())), "marked context is foreground")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestWaitFetchGateForegroundProceedsImmediately: a foreground fetch acquires a
|
||||||
|
// token without yielding, even when background callers exist.
|
||||||
|
func TestWaitFetchGateForegroundProceedsImmediately(t *testing.T) {
|
||||||
|
SetFetchRate(0) // unlimited limiter — isolate the yield logic from pacing
|
||||||
|
foregroundPending.Store(0)
|
||||||
|
t.Cleanup(func() { foregroundPending.Store(0) })
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
|
||||||
|
defer cancel()
|
||||||
|
require.NoError(t, WaitFetchGate(ForegroundContext(ctx)), "foreground proceeds immediately")
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestWaitFetchGateBackgroundYieldsToForeground: while a foreground fetch is
|
||||||
|
// pending, a background fetch yields (does not take a token) until the foreground
|
||||||
|
// clears — proven by a background wait timing out against its own deadline, then
|
||||||
|
// succeeding once the foreground is done.
|
||||||
|
func TestWaitFetchGateBackgroundYieldsToForeground(t *testing.T) {
|
||||||
|
SetFetchRate(0)
|
||||||
|
foregroundPending.Store(1) // simulate a foreground fetch in flight
|
||||||
|
t.Cleanup(func() { foregroundPending.Store(0) })
|
||||||
|
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 250*time.Millisecond)
|
||||||
|
defer cancel()
|
||||||
|
require.Error(t, WaitFetchGate(ctx), "background yields (blocks) while foreground is pending")
|
||||||
|
|
||||||
|
foregroundPending.Store(0) // foreground done
|
||||||
|
require.NoError(t, WaitFetchGate(context.Background()), "background proceeds once foreground clears")
|
||||||
|
}
|
||||||
|
|||||||
@@ -33,7 +33,10 @@ type Config struct {
|
|||||||
SummarizerModel string
|
SummarizerModel string
|
||||||
// FallbackModel is the LOCAL fallback alias tried when the primary fails or
|
// FallbackModel is the LOCAL fallback alias tried when the primary fails or
|
||||||
// returns unparseable output (ADR-022). Kept local so content stays on the
|
// returns unparseable output (ADR-022). Kept local so content stays on the
|
||||||
// homelab stack. Empty disables it. Default a bigger-context local model.
|
// homelab stack. Default is an IGUANA model (not koala) so the fallback runs
|
||||||
|
// on a different host than the koala primary — koala carries other loads, and
|
||||||
|
// a different host also means a different egress IP for the (rare) fallback.
|
||||||
|
// Empty disables it.
|
||||||
FallbackModel string
|
FallbackModel string
|
||||||
// CloudFallbackModel is the worst-case EXTERNAL fallback alias, tried only
|
// CloudFallbackModel is the worst-case EXTERNAL fallback alias, tried only
|
||||||
// after every local endpoint has failed (ADR-022). For client deployments set
|
// after every local endpoint has failed (ADR-022). For client deployments set
|
||||||
@@ -148,7 +151,7 @@ func (c Config) DexConfigured() bool { return strings.TrimSpace(c.OIDCIssuer) !=
|
|||||||
const (
|
const (
|
||||||
defaultGatewayURL = "http://koala:30401/v1"
|
defaultGatewayURL = "http://koala:30401/v1"
|
||||||
defaultSummarizerModel = "koala/phi4-mini"
|
defaultSummarizerModel = "koala/phi4-mini"
|
||||||
defaultFallbackModel = "koala/phi4-14b"
|
defaultFallbackModel = "iguana/gemma4-26b"
|
||||||
defaultCloudFallbackModel = "berget/mistral-small"
|
defaultCloudFallbackModel = "berget/mistral-small"
|
||||||
defaultSummaryMaxTokens = 1500
|
defaultSummaryMaxTokens = 1500
|
||||||
defaultMaxTranscriptChars = 18000
|
defaultMaxTranscriptChars = 18000
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -26,10 +27,22 @@ import (
|
|||||||
var dsn string
|
var dsn string
|
||||||
|
|
||||||
func TestMain(m *testing.M) {
|
func TestMain(m *testing.M) {
|
||||||
const port = 54330 // distinct from the store package's embedded PG (54329)
|
// Per-process port + dirs so concurrent `go test` runs (e.g. a push-run and a
|
||||||
|
// tag-run in CI) never collide on a fixed port or shared data dir. Base 55000
|
||||||
|
// keeps web's range distinct from the store package (54000). Shared CachePath
|
||||||
|
// downloads the PG archive once.
|
||||||
|
port := uint32(55000 + os.Getpid()%1000)
|
||||||
dsn = fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port)
|
dsn = fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port)
|
||||||
|
|
||||||
pg := embeddedpostgres.NewDatabase(embeddedpostgres.DefaultConfig().Port(port))
|
rt := filepath.Join(os.TempDir(), fmt.Sprintf("tapir-epg-web-%d", os.Getpid()))
|
||||||
|
pg := embeddedpostgres.NewDatabase(
|
||||||
|
embeddedpostgres.DefaultConfig().
|
||||||
|
Port(port).
|
||||||
|
RuntimePath(rt).
|
||||||
|
DataPath(filepath.Join(rt, "data")).
|
||||||
|
BinariesPath(filepath.Join(rt, "bin")).
|
||||||
|
CachePath(filepath.Join(os.TempDir(), "tapir-epg-cache")),
|
||||||
|
)
|
||||||
if err := pg.Start(); err != nil {
|
if err := pg.Start(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "embedded-postgres start: %v\n", err)
|
fmt.Fprintf(os.Stderr, "embedded-postgres start: %v\n", err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
@@ -38,6 +51,7 @@ func TestMain(m *testing.M) {
|
|||||||
if err := pg.Stop(); err != nil {
|
if err := pg.Stop(); err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "embedded-postgres stop: %v\n", err)
|
fmt.Fprintf(os.Stderr, "embedded-postgres stop: %v\n", err)
|
||||||
}
|
}
|
||||||
|
_ = os.RemoveAll(rt)
|
||||||
os.Exit(code)
|
os.Exit(code)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user