ffprobe stops at 1000 streams by itself, but 900 streams in a 1.3 MB file cost about 570 MB RSS in ffprobe and 570 MB in ffmpeg, against tapir's 512 Mi container limit (infra#520): an authenticated user can OOM-kill the pod (tapir only; every job in flight is lost). Worst CPU time seen was 3 s, so this is a memory problem, not CPU.
A 120 MB WAV with escapable tag bytes made ffprobe emit 491 MB of JSON, which cmd.Output() buffers in Go.
The probe in Accept runs on the request context with no deadline of its own (a stub ffprobe that sleeps held Probe for the full 5 s); the server has only ReadHeaderTimeout, so a hung probe can pin a request for as long as Traefik allows (up to an hour). ToWAV is already bounded by the 60-minute job context.
Not a problem (checked): the alis/dref external-reference worry in #46 item 1: under the application's flags ffmpeg logs "Skipped opening external track" and writes a 78-byte WAV; only -enable_drefs 1 would read the sibling.
Fix, three commits, each red test first
-max_streams 32 on both ffprobe and ffmpeg, plus a narrower probe output (-show_entries format=duration:stream=codec_type:stream_disposition=attached_pic; the investigator checked ParseProbe accepts it, cover art included). Red: a live-ffprobe test (skipped when the binary is absent; the Go test generates the media itself: no binary fixtures) on a 900-stream file must be refused as unreadable and stay under 100 MB RSS (in the scratch run, -max_streams 32 failed after 0.00 s at 18 MB).
A stdout cap on the probe. Red: a fake ffprobe script emitting more than the cap must give ErrUnreadable.
A probe deadline (FFprobe.Timeout, default 45 s; TAPIR_FFPROBE_TIMEOUT optional). Red: TestFFprobeProbe_AHungProbeTimesOutOnItsOwnDeadlineAsUnreadableNotAsOurFault (stub exec sleep 30, Timeout: 200ms, returns within about 3 s with ErrUnreadable) and ..._ParentCancellationIsStillNotUnreadable. Stub gotcha: use exec sleep, a plain sleep in an sh stub holds the pipe for the 2 s WaitDelay.
Review: control code (probe restrictions and limits): adversarial pass verified by execution, in the same round as the concurrent-upload cap (#47 item 1). Acceptance: the tests red on main and green after; task check and golangci-lint run ./... exit 0; a recorded verdict here.
**P0 of the audio-upload follow-up sprint (milestone v0.31.1). Merges #46 item 2 and #47 item 2; found by the phase-1 investigations.**
**Reproduced (synthetic media, ffmpeg/ffprobe 7.0.2):**
- ffprobe stops at 1000 streams by itself, but **900 streams in a 1.3 MB file cost about 570 MB RSS in ffprobe and 570 MB in ffmpeg**, against tapir's **512 Mi** container limit (infra#520): an authenticated user can OOM-kill the pod (tapir only; every job in flight is lost). Worst CPU time seen was 3 s, so this is a memory problem, not CPU.
- A 120 MB WAV with escapable tag bytes made ffprobe emit 491 MB of JSON, which `cmd.Output()` buffers in Go.
- The probe in `Accept` runs on the request context with no deadline of its own (a stub ffprobe that sleeps held `Probe` for the full 5 s); the server has only `ReadHeaderTimeout`, so a hung probe can pin a request for as long as Traefik allows (up to an hour). `ToWAV` is already bounded by the 60-minute job context.
- Not a problem (checked): the `alis`/`dref` external-reference worry in #46 item 1: under the application's flags ffmpeg logs "Skipped opening external track" and writes a 78-byte WAV; only `-enable_drefs 1` would read the sibling.
**Fix, three commits, each red test first**
1. `-max_streams 32` on both ffprobe and ffmpeg, plus a narrower probe output (`-show_entries format=duration:stream=codec_type:stream_disposition=attached_pic`; the investigator checked `ParseProbe` accepts it, cover art included). Red: a live-ffprobe test (skipped when the binary is absent; the Go test generates the media itself: no binary fixtures) on a 900-stream file must be refused as unreadable and stay under 100 MB RSS (in the scratch run, `-max_streams 32` failed after 0.00 s at 18 MB).
2. A stdout cap on the probe. Red: a fake ffprobe script emitting more than the cap must give `ErrUnreadable`.
3. A probe deadline (`FFprobe.Timeout`, default 45 s; `TAPIR_FFPROBE_TIMEOUT` optional). Red: `TestFFprobeProbe_AHungProbeTimesOutOnItsOwnDeadlineAsUnreadableNotAsOurFault` (stub `exec sleep 30`, `Timeout: 200ms`, returns within about 3 s with `ErrUnreadable`) and `..._ParentCancellationIsStillNotUnreadable`. Stub gotcha: use `exec sleep`, a plain `sleep` in an `sh` stub holds the pipe for the 2 s WaitDelay.
**Review:** control code (probe restrictions and limits): adversarial pass verified by execution, in the same round as the concurrent-upload cap (#47 item 1). **Acceptance:** the tests red on main and green after; `task check` and `golangci-lint run ./...` exit 0; a recorded verdict here.
---
_Created via git-mcp on behalf of @mathias_
mathias
added this to the v0.31.1 - security and robustness milestone 2026-10-03 18:48:06 +00:00
Shipped in v0.31.1 (e037c6b, live). Measured with the static ffmpeg/ffprobe 7.0.2 on the 900-stream file: 564 MB (ffprobe) / 566 MB (ffmpeg) RSS without -max_streams, 18 MB with -max_streams 32 (probe exits 1, upload reported unreadable). Probe output narrowed with -show_entries and capped at 1 MiB; probe deadline 45 s (own timeout = file's fault, caller cancel is not); ffmpeg's output now a bounded head (review found a hostile mp3 produced ~83 MB of stderr). Reviewer verified wav/mp3+cover/m4a+cover/mp4 still parse and a 33-stream mkv is refused. Not verified: behaviour on the production pod with a hostile file (would need an authenticated upload of one; the measurements above are the same binaries and flags). Closing.
Shipped in v0.31.1 (e037c6b, live). Measured with the static ffmpeg/ffprobe 7.0.2 on the 900-stream file: 564 MB (ffprobe) / 566 MB (ffmpeg) RSS without `-max_streams`, 18 MB with `-max_streams 32` (probe exits 1, upload reported unreadable). Probe output narrowed with `-show_entries` and capped at 1 MiB; probe deadline 45 s (own timeout = file's fault, caller cancel is not); ffmpeg's output now a bounded head (review found a hostile mp3 produced ~83 MB of stderr). Reviewer verified wav/mp3+cover/m4a+cover/mp4 still parse and a 33-stream mkv is refused. Not verified: behaviour on the production pod with a hostile file (would need an authenticated upload of one; the measurements above are the same binaries and flags). Closing.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
P0 of the audio-upload follow-up sprint (milestone v0.31.1). Merges #46 item 2 and #47 item 2; found by the phase-1 investigations.
Reproduced (synthetic media, ffmpeg/ffprobe 7.0.2):
cmd.Output()buffers in Go.Acceptruns on the request context with no deadline of its own (a stub ffprobe that sleeps heldProbefor the full 5 s); the server has onlyReadHeaderTimeout, so a hung probe can pin a request for as long as Traefik allows (up to an hour).ToWAVis already bounded by the 60-minute job context.alis/drefexternal-reference worry in #46 item 1: under the application's flags ffmpeg logs "Skipped opening external track" and writes a 78-byte WAV; only-enable_drefs 1would read the sibling.Fix, three commits, each red test first
-max_streams 32on both ffprobe and ffmpeg, plus a narrower probe output (-show_entries format=duration:stream=codec_type:stream_disposition=attached_pic; the investigator checkedParseProbeaccepts it, cover art included). Red: a live-ffprobe test (skipped when the binary is absent; the Go test generates the media itself: no binary fixtures) on a 900-stream file must be refused as unreadable and stay under 100 MB RSS (in the scratch run,-max_streams 32failed after 0.00 s at 18 MB).ErrUnreadable.FFprobe.Timeout, default 45 s;TAPIR_FFPROBE_TIMEOUToptional). Red:TestFFprobeProbe_AHungProbeTimesOutOnItsOwnDeadlineAsUnreadableNotAsOurFault(stubexec sleep 30,Timeout: 200ms, returns within about 3 s withErrUnreadable) and..._ParentCancellationIsStillNotUnreadable. Stub gotcha: useexec sleep, a plainsleepin anshstub holds the pipe for the 2 s WaitDelay.Review: control code (probe restrictions and limits): adversarial pass verified by execution, in the same round as the concurrent-upload cap (#47 item 1). Acceptance: the tests red on main and green after;
task checkandgolangci-lint run ./...exit 0; a recorded verdict here.Created via git-mcp on behalf of @mathias
Shipped in v0.31.1 (
e037c6b, live). Measured with the static ffmpeg/ffprobe 7.0.2 on the 900-stream file: 564 MB (ffprobe) / 566 MB (ffmpeg) RSS without-max_streams, 18 MB with-max_streams 32(probe exits 1, upload reported unreadable). Probe output narrowed with-show_entriesand capped at 1 MiB; probe deadline 45 s (own timeout = file's fault, caller cancel is not); ffmpeg's output now a bounded head (review found a hostile mp3 produced ~83 MB of stderr). Reviewer verified wav/mp3+cover/m4a+cover/mp4 still parse and a 33-stream mkv is refused. Not verified: behaviour on the production pod with a hostile file (would need an authenticated upload of one; the measurements above are the same binaries and flags). Closing.Created via git-mcp on behalf of @mathias