vectorstore.Sync currently only embeds files it has never seen — once a path appears in brain_embeddings, edits to the underlying .md are invisible to the vector store. The code has a TODO marker for this:
// internal/vectorstore/sync.goif_,ok:=known[relSlash];ok{// Already embedded — TODO: compare mtime once Store exposes// updated_at so we re-embed on edit. For now, skip.returnnil}
Result: any note revised after its first sync silently drifts from its embedding.
Proposed change
Extend Store to expose KnownPathsWithMtime() map[string]time.Time — returns each path's updated_at from brain_embeddings. (Or KnownPaths returns a richer struct; small API choice.)
In Sync, compare each file's mtime against the store's updated_at. If newer → re-embed + upsert.
Cover with a unit test that mutates a file's mtime after first sync and asserts a second Upsert is invoked.
Acceptance criteria
Sync re-embeds files whose mtime > updated_at
Adds/deletes paths still behave as today
Unit test added (probably via stubStore with controllable known-mtime map)
No new round-trips when nothing changed (the index pass should be cheap)
Out of scope
Content-hash invalidation (mtime is enough for the brain's edit pattern — Syncthing preserves it).
## Context
`vectorstore.Sync` currently only embeds files it has never seen — once a path appears in `brain_embeddings`, edits to the underlying `.md` are invisible to the vector store. The code has a TODO marker for this:
```go
// internal/vectorstore/sync.go
if _, ok := known[relSlash]; ok {
// Already embedded — TODO: compare mtime once Store exposes
// updated_at so we re-embed on edit. For now, skip.
return nil
}
```
Result: any note revised after its first sync silently drifts from its embedding.
## Proposed change
1. Extend `Store` to expose `KnownPathsWithMtime() map[string]time.Time` — returns each path's `updated_at` from `brain_embeddings`. (Or `KnownPaths` returns a richer struct; small API choice.)
2. In `Sync`, compare each file's `mtime` against the store's `updated_at`. If newer → re-embed + upsert.
3. Cover with a unit test that mutates a file's mtime after first sync and asserts a second Upsert is invoked.
## Acceptance criteria
- [ ] `Sync` re-embeds files whose `mtime > updated_at`
- [ ] Adds/deletes paths still behave as today
- [ ] Unit test added (probably via stubStore with controllable known-mtime map)
- [ ] No new round-trips when nothing changed (the index pass should be cheap)
## Out of scope
Content-hash invalidation (mtime is enough for the brain's edit pattern — Syncthing preserves it).
Store interface evolved from KnownPaths() map[string]struct{} → KnownPathsWithTime() map[string]time.Time. PGStore: SELECT path, updated_at FROM brain_embeddings. Sync groups chunks by parent and tracks the earliest updated_at per parent — if a file's mtime is after that, at least one chunk is stale, so the file is re-embedded.
Re-embed path deletes every old chunk for the parent first, then re-chunks + re-embeds + re-upserts. Handles shrunk files cleanly (no orphan #NNNN rows at higher indexes).
Tests
TestSync_ReembedsFileWhenMtimeNewer// mtime > store updated_at → re-embed + old chunks deletedTestSync_SkipsFileWhenMtimeOlder// mtime < store updated_at → no-op
Existing 15 tests updated for the new stub signature (stubStore.known is now map[string]time.Time; zero values default to a far-future sentinel so "skip if already known" tests keep passing without per-test setup).
Backward compatibility
brain_embeddings rows pre-dating this change carry valid updated_at values — the column was always populated via DEFAULT now() + ON CONFLICT ... updated_at = now() on every upsert. No schema migration. Live pod will start re-embedding any file whose source has been edited since its chunks were originally written.
Acceptance
Sync re-embeds files whose mtime > updated_at
Adds/deletes paths still behave as before
Unit tests added (stubStore with controllable known-mtime map)
No new round-trips when nothing changed (single SELECT path, updated_at per cycle replaces single SELECT path)
Closes.
Shipped in commit `8157397`.
## Approach
Store interface evolved from `KnownPaths() map[string]struct{}` → `KnownPathsWithTime() map[string]time.Time`. PGStore: `SELECT path, updated_at FROM brain_embeddings`. Sync groups chunks by parent and tracks the **earliest** updated_at per parent — if a file's mtime is after that, at least one chunk is stale, so the file is re-embedded.
Re-embed path deletes every old chunk for the parent first, then re-chunks + re-embeds + re-upserts. Handles shrunk files cleanly (no orphan `#NNNN` rows at higher indexes).
## Tests
```go
TestSync_ReembedsFileWhenMtimeNewer // mtime > store updated_at → re-embed + old chunks deleted
TestSync_SkipsFileWhenMtimeOlder // mtime < store updated_at → no-op
```
Existing 15 tests updated for the new stub signature (`stubStore.known` is now `map[string]time.Time`; zero values default to a far-future sentinel so "skip if already known" tests keep passing without per-test setup).
## Backward compatibility
`brain_embeddings` rows pre-dating this change carry valid `updated_at` values — the column was always populated via `DEFAULT now()` + `ON CONFLICT ... updated_at = now()` on every upsert. No schema migration. Live pod will start re-embedding any file whose source has been edited since its chunks were originally written.
## Acceptance
- [x] Sync re-embeds files whose `mtime > updated_at`
- [x] Adds/deletes paths still behave as before
- [x] Unit tests added (stubStore with controllable known-mtime map)
- [x] No new round-trips when nothing changed (single `SELECT path, updated_at` per cycle replaces single `SELECT path`)
Closes.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Context
vectorstore.Synccurrently only embeds files it has never seen — once a path appears inbrain_embeddings, edits to the underlying.mdare invisible to the vector store. The code has a TODO marker for this:Result: any note revised after its first sync silently drifts from its embedding.
Proposed change
Storeto exposeKnownPathsWithMtime() map[string]time.Time— returns each path'supdated_atfrombrain_embeddings. (OrKnownPathsreturns a richer struct; small API choice.)Sync, compare each file'smtimeagainst the store'supdated_at. If newer → re-embed + upsert.Acceptance criteria
Syncre-embeds files whosemtime > updated_atOut of scope
Content-hash invalidation (mtime is enough for the brain's edit pattern — Syncthing preserves it).
Shipped in commit
8157397.Approach
Store interface evolved from
KnownPaths() map[string]struct{}→KnownPathsWithTime() map[string]time.Time. PGStore:SELECT path, updated_at FROM brain_embeddings. Sync groups chunks by parent and tracks the earliest updated_at per parent — if a file's mtime is after that, at least one chunk is stale, so the file is re-embedded.Re-embed path deletes every old chunk for the parent first, then re-chunks + re-embeds + re-upserts. Handles shrunk files cleanly (no orphan
#NNNNrows at higher indexes).Tests
Existing 15 tests updated for the new stub signature (
stubStore.knownis nowmap[string]time.Time; zero values default to a far-future sentinel so "skip if already known" tests keep passing without per-test setup).Backward compatibility
brain_embeddingsrows pre-dating this change carry validupdated_atvalues — the column was always populated viaDEFAULT now()+ON CONFLICT ... updated_at = now()on every upsert. No schema migration. Live pod will start re-embedding any file whose source has been edited since its chunks were originally written.Acceptance
mtime > updated_atSELECT path, updated_atper cycle replaces singleSELECT path)Closes.