mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

capture follow-up: harvest adapters (transcript-parse / chat-reconstruct / agent-runlog) #64

New Issue
Open
opened 2026-06-23 05:45:02 +00:00 by mathias · 1 comment
mathias commented 2026-06-23 05:45:02 +00:00
Owner
Copy Link
Copy Source

Parent epic: #49 (capture, complete, v0.11.0) · deferred follow-up

Scope

Build the per-harness harvest adapters that assemble capture arguments (insights, tickets, summary, context) from whatever a given environment exposes as "the session." capture itself is uniform and shipped; harvest is the deliberately-non-uniform front half (the "acceptable per-harness quirk" from #49 — only how args are assembled varies, never what persistence does).

The three fidelity tiers

Each harness has a different notion of "the session," yielding different fidelity:

  1. transcript-parse (richest) — Claude Code CLI, Crush, Pi: a .jsonl / run-log transcript on disk. Reuse the existing ai-sessions extractors (extract_claude_sessions.py, extract_crush_sessions.py, extract_pi_sessions.py) to parse, then map to capture args. Highest fidelity → its summary supersedes a thinner one for the same session_ref (server-enforced collision rule).
  2. chat-reconstruct (lossy) — claude.ai Chat/Cowork/Design: no transcript file; the model reconstructs from conversation memory. This is what the close-session SKILL veneer (#62) does. fidelity: live-capture.
  3. agent-runlog — Agentsquad executor/reviewer: structured run output → capture args. fidelity: agent-runlog.

What to do

  • A harvest adapter per tier that emits a valid capture request.
  • transcript-parse: wrap the ai-sessions extractors; map their output to insights/summary.
  • Define how insights (the durable, brain-worthy learnings) are distinguished from raw session content at harvest time — the SKIP-bias / "generalizable learning vs project status" filter (cf. brain-weekly-ingest design).
  • Ensure fidelity + session_ref are set so the supersession/collision rule works across re-captures of the same session at improving fidelity.

Acceptance

  • transcript-parse adapter (reuses ai-sessions extractors) → valid capture request
  • chat-reconstruct path (via #62 SKILL) → valid capture request
  • agent-runlog adapter → valid capture request
  • Insight-extraction filter (durable learning vs noise) defined + applied
  • fidelity/session_ref set so a richer re-capture supersedes a thinner one
  • At least one tier tested end-to-end into a live capture

Related

  • #49 (capture epic), #62 (SKILL veneer = the chat-reconstruct tier)
  • ai-sessions extract_*_sessions.py (the transcript-parse reuse path)
  • brain-weekly-ingest (the SKIP-bias insight-extraction precedent)
**Parent epic:** #49 (capture, complete, v0.11.0) · deferred follow-up ## Scope Build the per-harness **harvest** adapters that assemble `capture` arguments (`insights`, `tickets`, `summary`, `context`) from whatever a given environment exposes as "the session." `capture` itself is uniform and shipped; harvest is the deliberately-non-uniform front half (the "acceptable per-harness quirk" from #49 — only *how args are assembled* varies, never what persistence does). ## The three fidelity tiers Each harness has a different notion of "the session," yielding different `fidelity`: 1. **transcript-parse** (richest) — Claude Code CLI, Crush, Pi: a `.jsonl` / run-log transcript on disk. **Reuse the existing `ai-sessions` extractors** (`extract_claude_sessions.py`, `extract_crush_sessions.py`, `extract_pi_sessions.py`) to parse, then map to `capture` args. Highest fidelity → its summary supersedes a thinner one for the same `session_ref` (server-enforced collision rule). 2. **chat-reconstruct** (lossy) — claude.ai Chat/Cowork/Design: no transcript file; the model reconstructs from conversation memory. This is what the close-session SKILL veneer (#62) does. `fidelity: live-capture`. 3. **agent-runlog** — Agentsquad executor/reviewer: structured run output → capture args. `fidelity: agent-runlog`. ## What to do - A harvest adapter per tier that emits a valid `capture` request. - transcript-parse: wrap the `ai-sessions` extractors; map their output to `insights`/`summary`. - Define how `insights` (the durable, brain-worthy learnings) are distinguished from raw session content at harvest time — the SKIP-bias / "generalizable learning vs project status" filter (cf. brain-weekly-ingest design). - Ensure `fidelity` + `session_ref` are set so the supersession/collision rule works across re-captures of the same session at improving fidelity. ## Acceptance - [ ] transcript-parse adapter (reuses ai-sessions extractors) → valid capture request - [ ] chat-reconstruct path (via #62 SKILL) → valid capture request - [ ] agent-runlog adapter → valid capture request - [ ] Insight-extraction filter (durable learning vs noise) defined + applied - [ ] `fidelity`/`session_ref` set so a richer re-capture supersedes a thinner one - [ ] At least one tier tested end-to-end into a live `capture` ## Related - #49 (capture epic), #62 (SKILL veneer = the chat-reconstruct tier) - ai-sessions `extract_*_sessions.py` (the transcript-parse reuse path) - brain-weekly-ingest (the SKIP-bias insight-extraction precedent)
mathias added the risk/mediumsize/ltype/feat labels 2026-08-27 05:25:42 +00:00
mathias commented 2026-09-23 12:58:19 +00:00
Author
Owner
Copy Link
Copy Source

DECISION NEEDED — not implemented this sprint.

Question: What's the concrete insight-extraction filter algorithm/schema for harvest-time capture, distinguishing "durable, generalizable learning" from "raw session noise"?

Why this blocks starting: the issue's own acceptance criteria require the filter to be defined and applied before any adapter can emit valid capture insights arguments — without it, an adapter either never fires (a permissive placeholder that never filters) or invents an arbitrary threshold with no product spec behind it. The issue points to brain-weekly-ingest's SKIP-bias heuristic as precedent but doesn't specify whether that heuristic should be reused as-is, redesigned for harvest-time's different input shape, or deferred.

Additional scope note found during research: this issue also spans two repos not present in mathias/hyperguild — the actual capture pipeline lives in mathias/brain, and the transcript extractors it's meant to reuse (extract_claude_sessions.py etc.) live in mathias/ai-sessions. Verifying current extractor signatures/output shape against what a harvest adapter needs has to happen in those repos, not here — a second reason this wasn't attempted blind in this sprint.

Options:

  1. Port brain-weekly-ingest's SKIP-bias heuristic as-is (fastest, but it was designed for a different input shape — weekly digest text, not raw transcripts — verify the underlying "is this generalizable" heuristic actually transfers).
  2. Design a new filter scoped specifically to harvest-time (higher upfront cost, right-sized to the actual input).
  3. Defer the filter, ship the three tiered adapters (transcript-parse / chat-reconstruct / agent-runlog) with a permissive placeholder, and file the filter as its own follow-up once real harvested output exists to design against.

Blocks: starting any of the three adapters (transcript-parse, chat-reconstruct, agent-runlog) that must call this filter before emitting insights.

🤖 Generated with Claude Code

Created via git-mcp on behalf of @mathias

**DECISION NEEDED** — not implemented this sprint. **Question:** What's the concrete insight-extraction filter algorithm/schema for harvest-time capture, distinguishing "durable, generalizable learning" from "raw session noise"? **Why this blocks starting:** the issue's own acceptance criteria require the filter to be defined and applied before any adapter can emit valid `capture` `insights` arguments — without it, an adapter either never fires (a permissive placeholder that never filters) or invents an arbitrary threshold with no product spec behind it. The issue points to `brain-weekly-ingest`'s SKIP-bias heuristic as precedent but doesn't specify whether that heuristic should be reused as-is, redesigned for harvest-time's different input shape, or deferred. **Additional scope note found during research**: this issue also spans two repos not present in `mathias/hyperguild` — the actual `capture` pipeline lives in `mathias/brain`, and the transcript extractors it's meant to reuse (`extract_claude_sessions.py` etc.) live in `mathias/ai-sessions`. Verifying current extractor signatures/output shape against what a harvest adapter needs has to happen in those repos, not here — a second reason this wasn't attempted blind in this sprint. **Options:** 1. Port `brain-weekly-ingest`'s SKIP-bias heuristic as-is (fastest, but it was designed for a different input shape — weekly digest text, not raw transcripts — verify the underlying "is this generalizable" heuristic actually transfers). 2. Design a new filter scoped specifically to harvest-time (higher upfront cost, right-sized to the actual input). 3. Defer the filter, ship the three tiered adapters (transcript-parse / chat-reconstruct / agent-runlog) with a permissive placeholder, and file the filter as its own follow-up once real harvested output exists to design against. **Blocks:** starting any of the three adapters (transcript-parse, chat-reconstruct, agent-runlog) that must call this filter before emitting `insights`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- _Created via git-mcp on behalf of @mathias_
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels risk/medium size/l type/feat
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#64