docs(bdd): observability.feature scenarios (@pending until TDD, #15)

This commit is contained in:
2026-06-12 08:28:46 +02:00
parent b590d2708d
commit 64d11af9ef
+54
View File
@@ -0,0 +1,54 @@
Feature: Observability — timing and metrics for performance and UX (ADR-030, #15)
As the maintainer running Tapir for pilot users
I want timing and Prometheus metrics for the activities that drive performance and UX
So that I can see latency, model behaviour, and usage and feed the Stage-0 eval gate
# AI metrics are the priority (ADR-030 R3). Each scenario maps to a Go test in
# scenarioCoverage once implemented; tagged @pending until the TDD step lands it.
@pending # TestObserveSummarizeRecordsModelOutcomeFallback
Scenario: Summarization latency is recorded per endpoint
Given the summarizer runs a transcript through its endpoint chain
When an endpoint returns a parseable summary
Then the summarize latency is recorded with the model, outcome "success", and whether it was a fallback
@pending # TestObserveSummarizeRecordsFailureOutcomes
Scenario: A failing summarizer endpoint records its failure outcome
Given the summarizer runs a transcript through its endpoint chain
When an endpoint errors or returns unparseable output
Then the summarize latency is recorded with outcome "error" or "parse_error" before the chain advances
@pending # TestObserveCaptionFetchByOutcome
Scenario: Caption fetch latency is recorded by outcome
Given a caption fetch is attempted for a video
When it resolves to captions, no captions, or a rate limit
Then the caption-fetch latency is recorded labelled by that outcome
@pending # TestLLMUsageHookRecordsTokens
Scenario: LLM token usage is recorded from the completion
Given an LLM completion returns a usage block with prompt and completion tokens
When the client finishes the call
Then the prompt and completion tokens are recorded for that model
@pending # TestChatAnswerLatencyRecorded
Scenario: Q&A answer latency is recorded
Given a user asks a question about a video
When the answer is produced from the stored transcript
Then the chat answer latency is recorded for the answering model
@pending # TestHTTPMiddlewareRecordsByRoutePattern
Scenario: HTTP requests are counted by route, method, and status
Given the metrics HTTP middleware wraps the app
When a request is served against a registered route
Then it is counted and timed under the bounded route pattern, not the raw path
@pending # TestLoginCounted
Scenario: A successful login is counted
Given a user completes the OIDC callback and a session is established
Then the login counter is incremented
@pending # TestMetricsNotOnPublicMux
Scenario: The metrics endpoint is not on the public app port
Given the service is running
When the public app mux is inspected
Then it exposes no /metrics route metrics are served on the dedicated metrics port only