docs(use-cases): scenarios for two-engine reconciliation (ADR-034)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -106,3 +106,53 @@ Feature: Upload an audio recording, get a private Swedish transcript, discuss it
|
|||||||
Given my recording's transcript is longer than the caption default
|
Given my recording's transcript is longer than the caption default
|
||||||
When I ask a question about it
|
When I ask a question about it
|
||||||
Then the transcript is sent with the upload budget, not the 18,000-character caption default
|
Then the transcript is sent with the upload budget, not the 18,000-character caption default
|
||||||
|
|
||||||
|
# ADR-034, two engines and a local reconciler (off unless TAPIR_RECONCILE_BASE_MODEL is set)
|
||||||
|
|
||||||
|
Scenario: Two engines transcribe the audio and their disagreements are reconciled
|
||||||
|
Given two-engine transcription is enabled
|
||||||
|
When a recording is transcribed
|
||||||
|
Then both engines run on the same audio, the fast one first
|
||||||
|
And a local model chooses between their disagreements
|
||||||
|
And the endpoint names every engine and the model that took part
|
||||||
|
|
||||||
|
Scenario: Trivial disagreements never reach the model
|
||||||
|
When the engines differ only by a compound written as one word or two, or by a filler word
|
||||||
|
Then the base reading is kept without a model call
|
||||||
|
|
||||||
|
Scenario: Differences close together are one decision
|
||||||
|
When two differences are separated by a single shared word
|
||||||
|
Then they are asked about as one dispute
|
||||||
|
|
||||||
|
Scenario: An unsure or changed reading is flagged for me
|
||||||
|
When the model changes a reading or cannot decide
|
||||||
|
Then the page lists the spot with a minute mark, both readings and which was used
|
||||||
|
And an unsure spot keeps the base reading and is marked uncertain
|
||||||
|
|
||||||
|
Scenario: The model cannot be a cloud model
|
||||||
|
When the reconciliation model or base engine is not a koala/ or iguana/ alias
|
||||||
|
Then the service refuses to start
|
||||||
|
And a cloud alias that merely contains a local host name is refused too
|
||||||
|
|
||||||
|
Scenario: One engine failing does not fail the upload
|
||||||
|
When one of the two engines fails
|
||||||
|
Then the other engine's transcript is used
|
||||||
|
And the page says only one engine produced it and why
|
||||||
|
And the cloud fallback is not used
|
||||||
|
|
||||||
|
Scenario: Only when both local engines fail does the cloud fallback run
|
||||||
|
When both local engines fail and a fallback is configured
|
||||||
|
Then the fallback transcribes it and the row records that it did
|
||||||
|
|
||||||
|
Scenario: A model that is down gives an unreconciled transcript, not a failed job
|
||||||
|
When the reconciliation model fails three times in a row
|
||||||
|
Then the base transcript is kept unreconciled with the reason
|
||||||
|
And the page says it could not be reconciled
|
||||||
|
|
||||||
|
Scenario: A cancelled job never reaches the cloud
|
||||||
|
When the job is cancelled during either engine
|
||||||
|
Then neither the second engine nor the cloud fallback is called
|
||||||
|
|
||||||
|
Scenario: A single-engine transcript claims no second opinion
|
||||||
|
When only one engine was configured
|
||||||
|
Then the page does not mention reconciliation
|
||||||
|
|||||||
Reference in New Issue
Block a user