Observed 2026-10-03 19:40:25Z (pod tapir-6cb44996bf-l7dhr, image e037c6b): the container exited with status 1 in the same second it started and Kubernetes restarted it twice before it stayed up (RESTARTS 2, 170 min before this was filed). Previous-container log:
level=ERROR msg="command failed" command=serve err="dex oidc: oidc: discover issuer: 503 Service Unavailable: no available server"
So serve treats a transient failure of the identity provider's discovery endpoint (the issuer was briefly unavailable behind its proxy) as fatal. It recovered by itself through the restart policy, which is why it cost nothing this time.
Why it is worth a ticket: the same thing happens on every deploy that coincides with the identity provider being down or restarting, and each restart is a pod start that also runs the start-up sweep (it marks rows interrupted). A retry with a short bounded back-off around discovery (say 5 tries over ~30 s, then exit non-zero as now) would keep the process up and make the failure visible in one log line instead of a restart count.
Source pointer: the OIDC provider is built at start-up in cmd/tapir (the "web auth: dex oidc" path); the error text is the go-oidc NewProvider discovery failure.
Not a regression of v0.31.1 or v0.32.0: the exit happened at start-up before any of this sprint's code ran. Found while checking the pod before merging PR #58 (sprint plan: docs/sprints/audio-upload-followups-2026-10.md). Milestone 11 is the natural home, risk/low.
**Observed 2026-10-03 19:40:25Z** (pod tapir-6cb44996bf-l7dhr, image e037c6b): the container exited with status 1 in the same second it started and Kubernetes restarted it twice before it stayed up (RESTARTS 2, 170 min before this was filed). Previous-container log:
`level=ERROR msg="command failed" command=serve err="dex oidc: oidc: discover issuer: 503 Service Unavailable: no available server"`
So `serve` treats a transient failure of the identity provider's discovery endpoint (the issuer was briefly unavailable behind its proxy) as fatal. It recovered by itself through the restart policy, which is why it cost nothing this time.
**Why it is worth a ticket:** the same thing happens on every deploy that coincides with the identity provider being down or restarting, and each restart is a pod start that also runs the start-up sweep (it marks rows interrupted). A retry with a short bounded back-off around discovery (say 5 tries over ~30 s, then exit non-zero as now) would keep the process up and make the failure visible in one log line instead of a restart count.
**Source pointer:** the OIDC provider is built at start-up in `cmd/tapir` (the "web auth: dex oidc" path); the error text is the go-oidc `NewProvider` discovery failure.
**Not a regression of v0.31.1 or v0.32.0**: the exit happened at start-up before any of this sprint's code ran. Found while checking the pod before merging PR #58 (sprint plan: docs/sprints/audio-upload-followups-2026-10.md). Milestone 11 is the natural home, risk/low.
---
_Created via git-mcp on behalf of @mathias_
mathias
added this to the v0.32.0 - audio upload follow-ups milestone 2026-10-03 22:34:30 +00:00
Fixed on main in 550764d (fix(oidc): retry issuer discovery with bounded back-off at start-up).
oidc.New now retries discovery 5 times with 2+4+8+16 s back-off (about 30 s), then fails as before. A cancelled ctx ends the back-off immediately. Tests in internal/web/oidc/discovery_retry_test.go, against an httptest issuer whose discovery returns 503 N times:
TestNewRetriesDiscoveryAfter503UntilIssuerAnswers: 2×503 then 200 → ok, 3 hits
TestNewReturnsDiscoveryErrorAfterAttemptsExhausted: error after exactly 3 hits
TestNewStopsDiscoveryBackoffWhenContextCancelled: returns context.DeadlineExceeded mid-back-off, 1 hit
Fixed on main in 550764d (`fix(oidc): retry issuer discovery with bounded back-off at start-up`).
`oidc.New` now retries discovery 5 times with 2+4+8+16 s back-off (about 30 s), then fails as before. A cancelled ctx ends the back-off immediately. Tests in `internal/web/oidc/discovery_retry_test.go`, against an httptest issuer whose discovery returns 503 N times:
- `TestNewRetriesDiscoveryAfter503UntilIssuerAnswers`: 2×503 then 200 → ok, 3 hits
- `TestNewReturnsDiscoveryErrorAfterAttemptsExhausted`: error after exactly 3 hits
- `TestNewStopsDiscoveryBackoffWhenContextCancelled`: returns `context.DeadlineExceeded` mid-back-off, 1 hit
Ships in the next patch tag.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Observed 2026-10-03 19:40:25Z (pod tapir-6cb44996bf-l7dhr, image
e037c6b): the container exited with status 1 in the same second it started and Kubernetes restarted it twice before it stayed up (RESTARTS 2, 170 min before this was filed). Previous-container log:level=ERROR msg="command failed" command=serve err="dex oidc: oidc: discover issuer: 503 Service Unavailable: no available server"So
servetreats a transient failure of the identity provider's discovery endpoint (the issuer was briefly unavailable behind its proxy) as fatal. It recovered by itself through the restart policy, which is why it cost nothing this time.Why it is worth a ticket: the same thing happens on every deploy that coincides with the identity provider being down or restarting, and each restart is a pod start that also runs the start-up sweep (it marks rows interrupted). A retry with a short bounded back-off around discovery (say 5 tries over ~30 s, then exit non-zero as now) would keep the process up and make the failure visible in one log line instead of a restart count.
Source pointer: the OIDC provider is built at start-up in
cmd/tapir(the "web auth: dex oidc" path); the error text is the go-oidcNewProviderdiscovery failure.Not a regression of v0.31.1 or v0.32.0: the exit happened at start-up before any of this sprint's code ran. Found while checking the pod before merging PR #58 (sprint plan: docs/sprints/audio-upload-followups-2026-10.md). Milestone 11 is the natural home, risk/low.
Created via git-mcp on behalf of @mathias
Fixed on main in
550764d(fix(oidc): retry issuer discovery with bounded back-off at start-up).oidc.Newnow retries discovery 5 times with 2+4+8+16 s back-off (about 30 s), then fails as before. A cancelled ctx ends the back-off immediately. Tests ininternal/web/oidc/discovery_retry_test.go, against an httptest issuer whose discovery returns 503 N times:TestNewRetriesDiscoveryAfter503UntilIssuerAnswers: 2×503 then 200 → ok, 3 hitsTestNewReturnsDiscoveryErrorAfterAttemptsExhausted: error after exactly 3 hitsTestNewStopsDiscoveryBackoffWhenContextCancelled: returnscontext.DeadlineExceededmid-back-off, 1 hitShips in the next patch tag.
Created via git-mcp on behalf of @mathias