Contents
- 01From Chatbot to Answer Engine
- 02Model the Personal Knowledge Graph
- 03Engineer Trust, Provenance, and Uncertainty
- 04Build a Self-Wiring Ingestion Flow
- 05Retrieve, Traverse, and Synthesize
- 06Build the Dream and Consolidation Cycle
- 07Secure and Operate a Multi-User Brain
- 08Evaluate Trust and Ship the Product
Build the Dream and Consolidation Cycle
Let the system improve in the background while preserving evidence, approvals, and rollback.
By the end
You can schedule bounded consolidation that proposes deduplication, links, summaries, stale claims, and knowledge gaps without silently changing beliefs.
Move from accumulation to consolidation
Ingestion accumulates sources, mentions, and candidate claims. Consolidation turns repeated evidence into stronger observations, links isolated nodes, identifies contradictions, refreshes summaries, and archives redundant derivations. Without it, the graph grows while understanding does not. With unsafe consolidation, the engine quietly edits the user's world model.
Separate the hot path from the sleep path
The hot path handles the current user request and should stay fast, bounded, and minimally mutating. The sleep path reviews recent sources and operations on a schedule. It can perform expensive duplicate search, community detection, stale checks, and lesson extraction, but should emit proposals and digests rather than surprise edits.
Consolidate bounded batches with checkpoints
Select a time window, project, entity neighborhood, or operation batch. Snapshot the base revision, compute proposals, validate them, and stop when change volume or uncertainty exceeds limits. Checkpoint progress so a provider failure does not produce an unknowable half-run. An unchanged second run should be mostly no-op.
Separate automatic maintenance from belief changes
Normalizing dates, repairing deterministic backlinks, and updating derived counters can be automatic. Merging identities, accepting inferred ownership, superseding decisions, changing current summaries, and deleting sources require review. Define policy by consequence and reversibility rather than by which model proposed the change.
Publish a useful morning-after digest
Each cycle should report what it reviewed, accepted deterministic repairs, proposed knowledge changes, new contradictions, stale areas, discovered gaps, cost, and rollback point. The operator needs a small decision surface, not a stream of internal reasoning. The digest is both interface and receipt.
Working reference
Commands and patterns
Preview a complete maintenance cycle
mkdir -p ../dream-receipts git status --short git rev-parse HEAD | tee ../dream-receipts/base-revision.txt gbrain dream --dir . --dry-run --json | tee ../dream-receipts/preview.json
Run this inside a dedicated Git-tracked brain checkout and stop if git status prints anything: the review needs a clean base. The explicit --dir . matters because GBrain does not infer the brain directory from the current Git checkout. The command records the exact base commit and the real CycleReport outside the checkout. Dry-run prevents content-changing Dream application and skips Sonnet synthesis, but it is not offline or free: on cache misses it still sends candidate transcript content to the configured significance judge (Haiku by default), incurs provider cost, and caches verdicts in dream_verdicts. Run it only when that data transfer and spend are permitted.
Preview one maintenance phase deliberately
gbrain dream --dir . --phase lint --dry-run --json | tee ../dream-receipts/lint-preview.json
A phase-specific dry run narrows diagnosis without applying lint fixes and saves the real CycleReport. Use gbrain dream --pull only when the approved cycle should also begin by pulling remote changes.
Review the changes that actually landed
gbrain dream --dir . --phase lint --json | tee ../dream-receipts/lint-run.json git status --short | tee ../dream-receipts/after-status.txt git diff --stat "$(cat ../dream-receipts/base-revision.txt)" | tee ../dream-receipts/diff-stat.txt git diff --word-diff "$(cat ../dream-receipts/base-revision.txt)" | tee ../dream-receipts/diff.patch
After explicit approval, this runs the write-capable lint phase and captures its actual CycleReport. Git then compares tracked working-tree changes with the exact base commit recorded before the run; status also exposes new untracked files that git diff does not include. Review those files directly before committing or accepting anything. Every receipt path and revision used here is created by the documented commands.
Install scheduled maintenance only after rehearsal
gbrain autopilot --install
Installs GBrain's scheduled maintenance workflow. Do this only after the dry runs and the reviewed write-capable rehearsal are clean, and after you have a rollback point and review policy.
When the happy path breaks
Failure modes
Symptom
The graph changes overnight and nobody knows why.
Likely cause
The dream cycle writes directly without a manifest or digest.
Recovery
Require a base revision, bounded change set, reasons, receipts, and approval for belief-changing mutations.
Symptom
The same pages are rewritten on every cycle.
Likely cause
Prompts and policies are unstable or lack material-change thresholds.
Recovery
Version the policy, compare semantic and text changes, and enforce second-run idempotency.
Symptom
The cycle grows more expensive as the corpus grows.
Likely cause
Every run scans and resynthesizes the entire graph.
Recovery
Process recent deltas and affected neighborhoods, cache deterministic work, and schedule global reviews separately.
Hands-on lab
Run a safe overnight simulation
Consolidate a bounded fixture containing duplication, contradiction, staleness, and a missing link.
- 01Create a dedicated Git-tracked fixture containing one safe deterministic repair, one duplicate, one contradiction, and one stale claim; commit that fixture as the base revision.
- 02Confirm git status is empty, then record HEAD and the dry-run CycleReport in a sibling receipt directory.
- 03Classify the dry-run findings in a review manifest: deterministic maintenance may be automated, while belief-changing proposals stay approval-required.
- 04After explicit approval, run the write-capable lint phase to apply only its deterministic repairs; do not imply that the stock command stages your separate belief-change proposals.
- 05Review git status and the working-tree diff against the recorded base revision, then verify citations, inbound links, and rollback.
- 06Rerun unchanged input and publish a compact digest.
Deliverable: A consolidation manifest, approval decision, applied patch, rollback proof, idempotent rerun, and operator digest.
Reusable artifact
Dream-cycle policy
A governance contract for background knowledge maintenance.
Schedule, trigger, scope, and maximum batch size Base revision and checkpoint cadence Automatic deterministic operations Approval-required operations Prohibited autonomous operations Confidence and material-change thresholds Cost and runtime budget Validation and idempotency checks Digest fields and approval destination Rollback and incident procedure
Before you move on
Receipt checklist
- Bounded input batch and base revision.
- Separated automatic and approval-required proposals.
- Reviewed claim-changing diff.
- Post-apply citation and graph checks.
- No-op rerun and operator digest.
Primary sources
