feat(discovery): drop Shorts and livestreams before the caption fetch (ADR-023)
The scarce resource is the per-IP timedtext caption fetch (ADR-014); the pilot's candidate set was mostly Shorts/clips/livestreams, each burning a fetch (a "none" result is a completed fetch — it costs budget even when it yields nothing). NewVideos now enriches candidates with one cheap Data API videos.list call (contentDetails.duration + snippet.liveBroadcastContent — the quota API, a DIFFERENT limit from the timedtext 429) and drops, before returning: videos shorter than TAPIR_MIN_VIDEO_SECONDS (default 60) and any live/upcoming broadcast. Dropped videos are never persisted, so the list declutters too. Degrade-open: MinVideoSeconds=0 disables it (no quota call); a videos.list error returns candidates unfiltered so discovery never breaks on a metadata hiccup. The paste-a-URL path (VideoByID) is not filtered — an explicit request is honoured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -46,6 +46,12 @@ type Config struct {
|
||||
// MaxTranscriptChars bounds the transcript text sent to the model so a long
|
||||
// transcript does not overflow a small-context primary. 0 disables truncation.
|
||||
MaxTranscriptChars int
|
||||
|
||||
// MinVideoSeconds drops videos shorter than this from discovery (Shorts and
|
||||
// other sub-minute clips that are noise and waste the scarce caption-fetch
|
||||
// budget, ADR-014/ADR-023). Enforced via a cheap Data API videos.list lookup at
|
||||
// discovery, never the rate-limited caption path. 0 disables the filter.
|
||||
MinVideoSeconds int
|
||||
// SummarizerTimeout bounds a single completion call. Thinking models are
|
||||
// slow, so the default is generous.
|
||||
SummarizerTimeout time.Duration
|
||||
@@ -138,6 +144,7 @@ const (
|
||||
defaultCloudFallbackModel = "berget/mistral-small"
|
||||
defaultSummaryMaxTokens = 1500
|
||||
defaultMaxTranscriptChars = 18000
|
||||
defaultMinVideoSeconds = 60
|
||||
defaultSummarizerTimeout = 5 * time.Minute
|
||||
defaultYTTokenRef = "youtube/refresh_token"
|
||||
defaultYTConnectRedirectURL = "https://tapir.d-ma.be/oauth/youtube/callback"
|
||||
@@ -230,6 +237,15 @@ func Load() (Config, error) {
|
||||
}
|
||||
c.MaxTranscriptChars = maxChars
|
||||
|
||||
minVideo, err := intOr("TAPIR_MIN_VIDEO_SECONDS", defaultMinVideoSeconds)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if minVideo < 0 {
|
||||
minVideo = 0
|
||||
}
|
||||
c.MinVideoSeconds = minVideo
|
||||
|
||||
onboard, err := intOr("TAPIR_ONBOARD_SUMMARIZE_COUNT", defaultOnboardSummarizeCount)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
|
||||
Reference in New Issue
Block a user