# Whisper track — self-transcription feasibility for tapir (recon, 2026-06-02)

Investigated inline by the Conductor (sub-agents are network-sandboxed this session).
Context: ADR-007 deferred audio-download + STT for (a) yt-dlp fragility, (b) ToS, (c) GPU
contention with the JEPA PoC on koala. With ADR-010 (timedtext captions) now landed and
working, Whisper is a **fallback for videos with NO captions**, not the primary path.

## What exists on the infra (verified from koala)

- **koala host has NONE of the tooling**: no `ffmpeg`, no `yt-dlp`, no `whisper` /
  `whisper.cpp` / `faster-whisper` / `mlx_whisper`. Only `python3`. Audio extraction would
  need a container/pod that bundles yt-dlp + ffmpeg.
- **Host GPU is currently unusable**: `nvidia-smi` → "Driver/library version mismatch"
  (NVML 610.43). The RTX 5070 is reachable only via k3s pods with `runtimeClassName: nvidia`
  (matches the known koala-GPU brain note), not from the host.
- **A Whisper model is ALREADY on the gateway**: `berget/whisper-large-v3` is served via
  LiteLLM (`http://koala:30401/v1`). BUT `berget/` = berget.ai = **cloud (EU)**. Using it
  ships audio off-box — acceptable under the homelab's berget-as-trusted-fallback posture,
  but NOT "local-first" in the strict VISION sense.
- **iguana** has `mlx-whisper` (per infra docs) — a local Mac-GPU option, no contention with
  koala's JEPA PoC — but it is **not exposed on the gateway** and its availability is
  uncertain (prior notes had iguana offline at times). Needs-verification.

## The pipeline Whisper would require
1. **Audio extraction**: `yt-dlp -f bestaudio` + `ffmpeg` to wav/16k. yt-dlp is exactly the
   ToS-grey + breakage risk ADR-007 named. Volume matters: full-audio download for many
   candidates is far heavier (bandwidth + time) than fetching caption text.
2. **Transcription compute** — three options:
   - **A. berget/whisper-large-v3 (cloud, ready now)** — zero new infra, fast, but external;
     gate as opt-in per the local-first principle.
   - **B. Local GPU whisper pod on koala** (nvidia runtime) — true local, but: deploy work +
     GPU contention with JEPA (ADR-007's stated reason) + the driver-mismatch to sort.
   - **C. iguana mlx-whisper** — local, no koala-GPU contention, but needs exposing + uptime.

## Verdict
- **Whisper is viable but remains a deferred fallback, and it reverses a settled ADR** — so
  it needs a new ADR (extends/supersedes ADR-007's deferral) before building.
- **Lowest-effort path if/when pursued**: route no-caption videos' audio to the existing
  `berget/whisper-large-v3` on the gateway, behind an explicit opt-in flag (it's cloud).
  Local purity → iguana mlx-whisper or a koala nvidia whisper pod, both more setup.
- **Priority**: LOW right now. ADR-010 timedtext captions already cover captioned videos
  (the bulk of the maintainer's subs). Whisper only buys the no-caption tail — measure how
  big that tail actually is (the run's `skipped_no_text` count) before investing.

**Biggest risks:** (1) yt-dlp fragility + ToS (the original ADR-007 objection, unchanged);
(2) audio-download volume/latency vs. caption-text; (3) "local-first" is only truly honored
by options B/C, both of which carry real setup/contention cost.