Phase C roadmap — live trace viewer (remaining increments) #5

Open
opened 2026-07-19 23:04:57 +00:00 by mathias · 5 comments
Owner

Done (increment 1)

  • Live latest CI run on stage 06, read from the in-cluster Gitea Actions API (public read, no token), coloured by outcome. LatestRun/RunNode, 30s cache, authored fallback.

Remaining increments (each shippable on its own)

  1. Per-job status — instead of only the overall run, show each job (guard/check/build/deploy) green/red from the run's job detail. Turns the generated job list into a live status strip.
    Status (2026-07-20): already shipped (3ff922a, same day as this issue) — the checklist above was just never updated. Verified live via kubectl port-forward to the running pod: stage 06 shows 6 nodes (run summary + 5 per-job), e.g. ▶ run #59 · success.
  2. Recent-runs timeline — last N runs as a sparkline/strip (pass/fail history), not just the latest.
    Status: already shipped (995e428). Verified live: 12-entry timeline sparkline present in /api/atlas.json, rendered in the UI header.
  3. Live CD/deploy state (stage 07) — the actually-deployed image tag + rollout readiness, read from the k8s Deployment. Needs RBAC add (deployments get/list in the cad-atlas ns) — extend the existing read-only SA.
    Status: already shipped (3d1f769), RBAC already in mathias/infra k3s/apps/cad-atlas/rbac.yaml. Verified live: ◆ deployed · cad-atlas:b2761a4 · 1/1 ready.
  4. Flux events — reconciliation status of the apps Kustomization / the cad-atlas resources (stage 07). Read via the k8s API (fluxcd CRDs) — RBAC add.
    Status: already shipped (1f7c9ab), RBAC in place. Verified live: ⟳ Flux · reconciled · main@d654d1f.
  5. assessor-loop ledger + session_log — the core CAD audit trace. Blocked: no live CAD run data exists for arbitrary repos yet, and the ledger format/access isn't documented (brain confirmed). Revisit once the ledger is a queryable service with data (assessor-loop #6/#10). Still blocked — unchanged.
  6. Replay a real trace — the reel's slow-mo pulse driven by a real run's job timeline (start→end per stage), not a synthetic sweep.
    Shipped as a partial slice (2026-07-20, 6967d12): only stages 06 (CI) and 07 (CD) have a real timing source (per-job created_at/updated_at from the latest Gitea Actions run — Job.Seconds(), atlas.StageSeconds(), TDD'd). Stages 00–05/08 still have no live timing source at all (same root cause as #5's ledger blocker), so they keep the original synthetic constant-speed sweep — a fully "real end-to-end trace" isn't possible until those stages are instrumented too. The pixel-time-budget the 06/07 segments already had is re-split by real CI:CD duration ratio instead of raw pixel width; falls back to the exact prior behavior when no run data or a skipped job (0 duration) is present. Verified: ci_duration_s/cd_duration_s flow through /api/atlas.json against a real run (18s observed), full-page screenshot confirms no visual regression.

Architecture note

Each stage sources from its real system (cd.yml→CI jobs, k8s nodes→substrate, k8s ns→ns line, Gitea runs→live status), overlaid onto authored narrative with graceful fallback. New sources plug into liveOverlay() + a pure parser (TDD'd). The thesis holds: the audit chain is the viz data.

Source: this build session (Phase B complete, Phase C increment 1 live at atlas.d-ma.be).

Remaining before this issue can close

Only #5 (assessor-loop ledger) — genuinely blocked on an upstream service, not on cad-atlas work. This issue stays open until that ledger exists and is queryable, per this repo's honesty rule (docs/INCEPTION-OATH.md): a blocked clause is tracked, never claimed satisfied.

## Done (increment 1) - Live **latest CI run** on stage 06, read from the in-cluster Gitea Actions API (public read, no token), coloured by outcome. `LatestRun`/`RunNode`, 30s cache, authored fallback. ## Remaining increments (each shippable on its own) 1. [x] **Per-job status** — instead of only the overall run, show each job (guard/check/build/deploy) green/red from the run's job detail. Turns the generated job list into a live status strip. **Status (2026-07-20): already shipped** (`3ff922a`, same day as this issue) — the checklist above was just never updated. Verified live via `kubectl port-forward` to the running pod: stage 06 shows 6 nodes (run summary + 5 per-job), e.g. `▶ run #59 · success`. 2. [x] **Recent-runs timeline** — last N runs as a sparkline/strip (pass/fail history), not just the latest. **Status: already shipped** (`995e428`). Verified live: 12-entry timeline sparkline present in `/api/atlas.json`, rendered in the UI header. 3. [x] **Live CD/deploy state** (stage 07) — the actually-deployed image tag + rollout readiness, read from the k8s Deployment. Needs RBAC add (`deployments` get/list in the cad-atlas ns) — extend the existing read-only SA. **Status: already shipped** (`3d1f769`), RBAC already in `mathias/infra` `k3s/apps/cad-atlas/rbac.yaml`. Verified live: `◆ deployed · cad-atlas:b2761a4 · 1/1 ready`. 4. [x] **Flux events** — reconciliation status of the `apps` Kustomization / the cad-atlas resources (stage 07). Read via the k8s API (fluxcd CRDs) — RBAC add. **Status: already shipped** (`1f7c9ab`), RBAC in place. Verified live: `⟳ Flux · reconciled · main@d654d1f`. 5. [ ] **assessor-loop ledger + session_log** — the core CAD audit trace. **Blocked:** no live CAD run data exists for arbitrary repos yet, and the ledger format/access isn't documented (brain confirmed). Revisit once the ledger is a queryable service with data (assessor-loop #6/#10). **Still blocked — unchanged.** 6. [x] **Replay a real trace** — the reel's slow-mo pulse driven by a *real* run's job timeline (start→end per stage), not a synthetic sweep. **Shipped as a partial slice (2026-07-20, `6967d12`)**: only stages 06 (CI) and 07 (CD) have a real timing source (per-job `created_at`/`updated_at` from the latest Gitea Actions run — `Job.Seconds()`, `atlas.StageSeconds()`, TDD'd). Stages 00–05/08 still have no live timing source at all (same root cause as #5's ledger blocker), so they keep the original synthetic constant-speed sweep — a fully "real end-to-end trace" isn't possible until those stages are instrumented too. The pixel-time-budget the 06/07 segments already had is re-split by real CI:CD duration ratio instead of raw pixel width; falls back to the exact prior behavior when no run data or a skipped job (0 duration) is present. Verified: `ci_duration_s`/`cd_duration_s` flow through `/api/atlas.json` against a real run (18s observed), full-page screenshot confirms no visual regression. ## Architecture note Each stage sources from its real system (cd.yml→CI jobs, k8s nodes→substrate, k8s ns→ns line, Gitea runs→live status), overlaid onto authored narrative with graceful fallback. New sources plug into `liveOverlay()` + a pure parser (TDD'd). The thesis holds: the audit chain is the viz data. Source: this build session (Phase B complete, Phase C increment 1 live at atlas.d-ma.be). ## Remaining before this issue can close Only #5 (assessor-loop ledger) — genuinely blocked on an upstream service, not on cad-atlas work. This issue stays open until that ledger exists and is queryable, per this repo's honesty rule (`docs/INCEPTION-OATH.md`): a blocked clause is tracked, never claimed satisfied.
Author
Owner

Increment 2 done: per-job status (roadmap item 1). Stage 06 now shows the latest run's per-job outcomes (Detect/Lint/Build/Deploy), coloured, in pipeline order, from /actions/tasks per-job entries — replaces the static job-id list when live. LatestRunJobs/RunNodes test-first. Live at atlas.d-ma.be.

Also (separate ask): the UI version is now build-injected from git describe --tags (no more label drift). Tagged v0.2.0; header shows it.

Remaining: recent-runs timeline · live CD/deploy state (needs deployments RBAC) · Flux events · assessor-loop ledger (still blocked, no data).

**Increment 2 done: per-job status** (roadmap item 1). Stage 06 now shows the latest run's per-job outcomes (Detect/Lint/Build/Deploy), coloured, in pipeline order, from `/actions/tasks` per-job entries — replaces the static job-id list when live. `LatestRunJobs`/`RunNodes` test-first. Live at atlas.d-ma.be. Also (separate ask): the UI version is now build-injected from `git describe --tags` (no more label drift). Tagged **v0.2.0**; header shows it. Remaining: recent-runs timeline · live CD/deploy state (needs `deployments` RBAC) · Flux events · assessor-loop ledger (still blocked, no data).
Author
Owner

Increment 3 done: live CD/deploy state (roadmap item 3). Stage 07 shows the pod's own deployed image tag + N/M replicas ready, green only when fully rolled out, read from the k8s Deployment via a namespaced deployments-read Role. DeployState/DeployNode test-first. Live at atlas.d-ma.be (v0.2.1).

Remaining: recent-runs timeline · Flux events (reconciliation status) · assessor-loop ledger (still blocked, no data). The SemVer→UI auto-flow (git describe → ldflags → served) now covers all future releases.

**Increment 3 done: live CD/deploy state** (roadmap item 3). Stage 07 shows the pod's own deployed image tag + N/M replicas ready, green only when fully rolled out, read from the k8s Deployment via a namespaced deployments-read Role. `DeployState`/`DeployNode` test-first. Live at atlas.d-ma.be (v0.2.1). Remaining: recent-runs timeline · Flux events (reconciliation status) · assessor-loop ledger (still blocked, no data). The SemVer→UI auto-flow (git describe → ldflags → served) now covers all future releases.
Author
Owner

Increment 4 done: recent-runs timeline (roadmap item 2). A strip of the last 12 runs below the substrate ribbon — aggregate pass/fail/running per run, coloured, newest-first, run# on hover. RecentRuns test-first, reuses the existing tasks fetch (no new RBAC). Live at atlas.d-ma.be (v0.2.3).

Also fixed (dogfooding catch): skipped jobs (deploy is skipped on tag-push runs) were mislabelled running in the aggregate → now count as OK. Test-first.

Remaining: Flux events (reconciliation status, needs Flux CRD read RBAC) · assessor-loop ledger (still blocked, no data). After that, Phase C is essentially complete bar the ledger.

**Increment 4 done: recent-runs timeline** (roadmap item 2). A strip of the last 12 runs below the substrate ribbon — aggregate pass/fail/running per run, coloured, newest-first, run# on hover. `RecentRuns` test-first, reuses the existing tasks fetch (no new RBAC). Live at atlas.d-ma.be (**v0.2.3**). Also fixed (dogfooding catch): `skipped` jobs (deploy is skipped on tag-push runs) were mislabelled `running` in the aggregate → now count as OK. Test-first. Remaining: **Flux events** (reconciliation status, needs Flux CRD read RBAC) · assessor-loop ledger (still blocked, no data). After that, Phase C is essentially complete bar the ledger.
Author
Owner

Increment 5 done: live Flux reconcile status (roadmap item 4). Stage 07 shows the Flux apps Kustomization state — reconciled/failed + last-applied revision (main@shortsha) — alongside the live deploy node. FluxStatus/FluxNode test-first; read-only Role in flux-system, cross-ns bound to the cad-atlas SA. Live at atlas.d-ma.be (v0.2.4).

Phase C status: essentially complete

Live from real systems, all TDD, graceful fallback, least-privilege read-only RBAC per source:

  • Signals/narrative — authored (atlas.json)
  • CI stage — per-job live status (Gitea)
  • CD stage — live deploy state + Flux reconcile status (k8s)
  • Recent-runs timeline (Gitea)
  • Substrate — live nodes + namespaces (k8s)
  • Version — build-injected git tag

Only remaining: assessor-loop ledger (item 5) — still blocked, no live CAD data for arbitrary repos yet; revisit when the ledger is a queryable service (assessor-loop #6/#10). The overlay pattern + a pure parser is ready for it to plug in.

**Increment 5 done: live Flux reconcile status** (roadmap item 4). Stage 07 shows the Flux `apps` Kustomization state — reconciled/failed + last-applied revision (`main@shortsha`) — alongside the live deploy node. `FluxStatus`/`FluxNode` test-first; read-only Role in flux-system, cross-ns bound to the cad-atlas SA. Live at atlas.d-ma.be (**v0.2.4**). ## Phase C status: essentially complete Live from real systems, all TDD, graceful fallback, least-privilege read-only RBAC per source: - Signals/narrative — authored (atlas.json) - CI stage — per-job live status (Gitea) - CD stage — live deploy state + **Flux reconcile status** (k8s) - Recent-runs timeline (Gitea) - Substrate — live nodes + namespaces (k8s) - Version — build-injected git tag **Only remaining:** assessor-loop ledger (item 5) — still blocked, no live CAD data for arbitrary repos yet; revisit when the ledger is a queryable service (assessor-loop #6/#10). The overlay pattern + a pure parser is ready for it to plug in.
Author
Owner

Flagging a possible terminology collision found during a cross-repo review (FlowBrain vs. CAD stack, 2026-07-23), not a confident claim either way: this issue's blocker (#5, item 5) names "assessor-loop ledger + session_log" as the missing queryable source. Separately, mathias/cad has its own append-only attestation Ledger (domain/runevent.go) fed by a RunEvent wire contract that's golden-test-locked to stay in sync with producers — described in cad's own docs as what Atlas should render "from the event stream alone." Worth checking explicitly whether cad's Ledger is the same thing this issue is blocked on, a partial subset (execution/CI events but not risk/case-level assessor-loop data), or a genuinely separate artifact — if it's queryable now, part of this blocker might already be unblocked without assessor-loop's own ledger existing yet.

Flagging a possible terminology collision found during a cross-repo review (FlowBrain vs. CAD stack, 2026-07-23), not a confident claim either way: this issue's blocker (#5, item 5) names "assessor-loop ledger + session_log" as the missing queryable source. Separately, `mathias/cad` has its own append-only attestation `Ledger` (`domain/runevent.go`) fed by a `RunEvent` wire contract that's golden-test-locked to stay in sync with producers — described in `cad`'s own docs as what Atlas should render "from the event stream alone." Worth checking explicitly whether `cad`'s Ledger is the same thing this issue is blocked on, a partial subset (execution/CI events but not risk/case-level assessor-loop data), or a genuinely separate artifact — if it's queryable now, part of this blocker might already be unblocked without assessor-loop's own ledger existing yet.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/cad-atlas#5