Found 2026-10-02 running the first blind check of ADR-034's two-engine reconcile on a real 63.6-minute recording (docs/research/reconcile-blind-check-2026-10-02.md, PR #39).
1. Kept-base spots are not stored, so ADR-034's second condition cannot be measured.Reconciliation.Contested holds only spots where the alternative reading was used or the model was unsure (131 of 282 adjudicated). The 151 spots where the model kept pianissimo are counted but not recorded. ADR-034 says to keep the feature only if the reconciled text is right at least 85% of the time and no worse than the better single engine; comparing with kb-whisper alone needs exactly the kept-base spots, and the audio is not retained, so they cannot be recomputed later. The first check could only compare against pianissimo.
Store every adjudicated spot with its verdict (base, alt, unsure), still bounded (MaxContested, currently 200, was not hit: 131). Keep the page list as is (changed and uncertain); the extra spots are for evaluation, not for display.
Insertion and deletion spots (39 of 131) cannot be placed in the base text from the stored fields alone. Store a few words of left context with each spot, which is also what a blind check and the page's "check against the audio" view need.
Privacy: the stored spots live in the RLS-scoped row like the transcript itself; nothing new leaves it. Evaluation exports must stay aggregate when committed (public repo).
2. Saved segments keep the base text. After reconcile, segments still holds pianissimo's wording and only content is merged: 102 of 104 sampled base readings were found verbatim in the segment text near their spot. Anything that reads segments (timed display, future jump-to-audio, retrieval per #36) shows unreconciled text. Decide whether segments should be rewritten with the merged words (timing stays the base's, per ADR-034) or documented as base-only.
Both are small changes to internal/reconcile and the store. Test: a reconcile with two kept-base and one changed spot stores all three; segments for a changed spot contain the reconciled word.
Found 2026-10-02 running the first blind check of ADR-034's two-engine reconcile on a real 63.6-minute recording (`docs/research/reconcile-blind-check-2026-10-02.md`, PR #39).
**1. Kept-base spots are not stored, so ADR-034's second condition cannot be measured.** `Reconciliation.Contested` holds only spots where the alternative reading was used or the model was unsure (131 of 282 adjudicated). The 151 spots where the model kept pianissimo are counted but not recorded. ADR-034 says to keep the feature only if the reconciled text is right at least 85% of the time **and no worse than the better single engine**; comparing with kb-whisper alone needs exactly the kept-base spots, and the audio is not retained, so they cannot be recomputed later. The first check could only compare against pianissimo.
- Store every adjudicated spot with its verdict (`base`, `alt`, `unsure`), still bounded (`MaxContested`, currently 200, was not hit: 131). Keep the page list as is (changed and uncertain); the extra spots are for evaluation, not for display.
- Insertion and deletion spots (39 of 131) cannot be placed in the base text from the stored fields alone. Store a few words of left context with each spot, which is also what a blind check and the page's "check against the audio" view need.
- Privacy: the stored spots live in the RLS-scoped row like the transcript itself; nothing new leaves it. Evaluation exports must stay aggregate when committed (public repo).
**2. Saved segments keep the base text.** After reconcile, `segments` still holds pianissimo's wording and only `content` is merged: 102 of 104 sampled base readings were found verbatim in the segment text near their spot. Anything that reads segments (timed display, future jump-to-audio, retrieval per #36) shows unreconciled text. Decide whether segments should be rewritten with the merged words (timing stays the base's, per ADR-034) or documented as base-only.
Both are small changes to `internal/reconcile` and the store. Test: a reconcile with two kept-base and one changed spot stores all three; segments for a changed spot contain the reconciled word.
---
_Created via git-mcp on behalf of @mathias_
Built on PR #39 (head 7029dd1), per ADR-035. Closes on merge.
Done: (1) every judged spot is stored with its verdict and the 8 words before it, capped at 2,000 (option MaxContested); the page still lists only changed and uncertain ones. The 39 insertion/deletion spots of the first run would now carry context, and kept spots are stored, so ADR-034's "no worse than the better single engine" can be measured on the next real run. (2) Saved segments carry the reconciled words with the base's timing; when segments cannot be aligned to the merged text they stay the base's.
Behaviour changes made on purpose and recorded in ADR-035: three tests that encoded the old behaviour (kept spots not stored, cap 200, segments untouched) were rewritten.
Not done: the stored-spot export for an evaluation is still ad hoc (I read the row with a throwaway program). A small tapir subcommand or script that prints an aggregate-only summary would keep that off the public repo.
Built on PR #39 (head `7029dd1`), per ADR-035. Closes on merge.
Done: (1) every judged spot is stored with its verdict and the 8 words before it, capped at 2,000 (option `MaxContested`); the page still lists only changed and uncertain ones. The 39 insertion/deletion spots of the first run would now carry context, and kept spots are stored, so ADR-034's "no worse than the better single engine" can be measured on the next real run. (2) Saved `segments` carry the reconciled words with the base's timing; when segments cannot be aligned to the merged text they stay the base's.
Behaviour changes made on purpose and recorded in ADR-035: three tests that encoded the old behaviour (kept spots not stored, cap 200, segments untouched) were rewritten.
Not done: the stored-spot export for an evaluation is still ad hoc (I read the row with a throwaway program). A small `tapir` subcommand or script that prints an aggregate-only summary would keep that off the public repo.
---
_Created via git-mcp on behalf of @mathias_
Merged in PR #39 (squash 4a20670), released as v0.31.0. Every judged spot is stored with its verdict and the 8 words before it (cap 2,000); the page lists only changed and uncertain spots; saved segments carry the merged words with the base's timing. ADR-034's second condition (no worse than the better single engine) can now be measured on the next real run. Closing.
Merged in PR #39 (squash `4a20670`), released as **v0.31.0**. Every judged spot is stored with its verdict and the 8 words before it (cap 2,000); the page lists only changed and uncertain spots; saved segments carry the merged words with the base's timing. ADR-034's second condition (no worse than the better single engine) can now be measured on the next real run. Closing.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Found 2026-10-02 running the first blind check of ADR-034's two-engine reconcile on a real 63.6-minute recording (
docs/research/reconcile-blind-check-2026-10-02.md, PR #39).1. Kept-base spots are not stored, so ADR-034's second condition cannot be measured.
Reconciliation.Contestedholds only spots where the alternative reading was used or the model was unsure (131 of 282 adjudicated). The 151 spots where the model kept pianissimo are counted but not recorded. ADR-034 says to keep the feature only if the reconciled text is right at least 85% of the time and no worse than the better single engine; comparing with kb-whisper alone needs exactly the kept-base spots, and the audio is not retained, so they cannot be recomputed later. The first check could only compare against pianissimo.base,alt,unsure), still bounded (MaxContested, currently 200, was not hit: 131). Keep the page list as is (changed and uncertain); the extra spots are for evaluation, not for display.2. Saved segments keep the base text. After reconcile,
segmentsstill holds pianissimo's wording and onlycontentis merged: 102 of 104 sampled base readings were found verbatim in the segment text near their spot. Anything that reads segments (timed display, future jump-to-audio, retrieval per #36) shows unreconciled text. Decide whether segments should be rewritten with the merged words (timing stays the base's, per ADR-034) or documented as base-only.Both are small changes to
internal/reconcileand the store. Test: a reconcile with two kept-base and one changed spot stores all three; segments for a changed spot contain the reconciled word.Created via git-mcp on behalf of @mathias
Built on PR #39 (head
7029dd1), per ADR-035. Closes on merge.Done: (1) every judged spot is stored with its verdict and the 8 words before it, capped at 2,000 (option
MaxContested); the page still lists only changed and uncertain ones. The 39 insertion/deletion spots of the first run would now carry context, and kept spots are stored, so ADR-034's "no worse than the better single engine" can be measured on the next real run. (2) Savedsegmentscarry the reconciled words with the base's timing; when segments cannot be aligned to the merged text they stay the base's.Behaviour changes made on purpose and recorded in ADR-035: three tests that encoded the old behaviour (kept spots not stored, cap 200, segments untouched) were rewritten.
Not done: the stored-spot export for an evaluation is still ad hoc (I read the row with a throwaway program). A small
tapirsubcommand or script that prints an aggregate-only summary would keep that off the public repo.Created via git-mcp on behalf of @mathias
Merged in PR #39 (squash
4a20670), released as v0.31.0. Every judged spot is stored with its verdict and the 8 words before it (cap 2,000); the page lists only changed and uncertain spots; saved segments carry the merged words with the base's timing. ADR-034's second condition (no worse than the better single engine) can now be measured on the next real run. Closing.Created via git-mcp on behalf of @mathias