fix(ingestion): tripwire alerts survive outages; drop only permanent rejections
CI / Lint / Test / Vet (pull_request) Successful in 41s
CI / Mirror to GitHub (pull_request) Skipped

PR #160 round 2: dropping a message after 5 failures could not tell a
poison message from an ntfy outage. A 15-minute outage dropped the only
incident with nothing reported, and drops were not counted (74 of 300
removals were neither delivered nor counted).

- audit.NtfyNotifier returns *RejectedError for non-2xx; Permanent() is
  true for 4xx other than 429.
- The tripwire queue drops a message only on a permanent rejection and
  counts it; every other failure is an outage and is retried, however
  long. Each flush tries every queued message, so an undeliverable one
  cannot block the rest.
- Overflow and drops go into one counter, sent as a summary retried
  every tick and never dropped. Every removal is delivered or counted.

Tests (red first): an outage of ~100 ticks still delivers the only
incident; delivered + counted equals the number of removals; a
permanently rejected message is dropped and counted. The short-outage
test is renamed to say it is short. The Watch doc no longer claims
'loses none' unconditionally.

Refs #157

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-03 18:30:46 +02:00
co-authored by Claude Opus 5.5
parent 01de163452
commit 4ba9d20b4e
4 changed files with 173 additions and 31 deletions
@@ -0,0 +1,27 @@
package audit
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// A non-2xx ntfy response is a *RejectedError, and only a client-side refusal
// other than rate limiting is permanent: callers may drop a message on a
// permanent rejection and must retry the rest (hyperguild#157, PR #160).
func TestNotifyClassifiesRejectionsAsPermanentOrNot(t *testing.T) {
for status, permanent := range map[int]bool{413: true, 400: true, 403: true, 429: false, 500: false, 503: false} {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(status) }))
err := NewNtfyNotifier(srv.URL, "").Notify(context.Background(), "m")
srv.Close()
var rej *RejectedError
require.True(t, errors.As(err, &rej), "status %d: %v", status, err)
assert.Equal(t, status, rej.Status)
assert.Equal(t, permanent, rej.Permanent(), "status %d", status)
}
}