Contents
- 01From Chatbot to Answer Engine
- 02Model the Personal Knowledge Graph
- 03Engineer Trust, Provenance, and Uncertainty
- 04Build a Self-Wiring Ingestion Flow
- 05Retrieve, Traverse, and Synthesize
- 06Build the Dream and Consolidation Cycle
- 07Secure and Operate a Multi-User Brain
- 08Evaluate Trust and Ship the Product
Engineer Trust, Provenance, and Uncertainty
Turn evidence quality, conflict, freshness, and missing knowledge into visible system state.
By the end
You can build a claim-evidence ledger and derive confidence from observable evidence rather than model tone.
Keep the entire evidence chain
A citation to a wiki summary is useful but incomplete. The engine should trace from answer claim to accepted graph claim, derived page or extraction, source span, and original. Each transformation records who or what created it and when. This chain lets an operator audit a conclusion and repair the earliest broken step.
Evaluate evidence without inventing a universal score
Authority depends on the question. A signed contract may outrank a planning note for obligations; a direct customer quote may outrank an analyst summary for that customer's view. Represent source type, ownership, directness, completeness, recency, and conflicts. Use these factors during reranking and explanation rather than hiding them behind one unexplained number.
Derive confidence from conditions
Confidence should respond to evidence coverage, source quality, agreement, freshness, identity certainty, retrieval path, and scope. Define deterministic downgrade rules before asking a model for judgment. For example, missing dates create a freshness gap; one weak source creates a narrow-evidence warning; unresolved conflict caps confidence at low. Return the triggered rules beside the answer so a reader can inspect the rating, challenge an assumption, and identify the next piece of evidence that would change it.
Surface contradictions as a result
Do not force contradictory claims into one blended summary. Return the competing claims, their sources and dates, any active status, and the reason the system cannot reconcile them. A human can then decide, request more evidence, or preserve the disagreement. Conflict is information about the world and the corpus.
Treat freshness as question-specific
A decade-old mathematical definition may be fine; yesterday's pricing can already be stale. Assign review policies by claim type and domain, record both observed time and system ingestion time, and report the newest relevant evidence. Freshness means fit for this question, not merely recent.
Working reference
Commands and patterns
Inspect raw retrieval and ranking evidence
gbrain search 'What evidence supports the current plan?' --explain
search returns hybrid retrieval results; --explain exposes why they ranked, which is the right surface for auditing evidence coverage.
Request synthesis with gaps and citations
gbrain think 'What evidence supports the current plan?' --json
think synthesizes over retrieved evidence. Configure a supported model for real synthesis; without one, GBrain can degrade to gather-only behavior, and --save must not pretend synthesis succeeded.
When the happy path breaks
Failure modes
Symptom
A high confidence badge appears with one weak source.
Likely cause
Confidence was generated by the same model that wrote the answer.
Recovery
Calculate evidence conditions first, cap confidence with explicit rules, and show the reasons.
Symptom
The answer cites a summary that cites another summary.
Likely cause
The provenance chain stops before the original evidence.
Recovery
Track derivation IDs and require a resolvable original or a visible provenance gap.
Symptom
Two contradictory sources are blended into a false compromise.
Likely cause
Synthesis optimized for a single smooth answer.
Recovery
Represent conflicts, show both claims, and require resolution evidence before accepting one.
Hands-on lab
Create a claim-evidence ledger
Audit ten claims and turn trust from intuition into inspectable conditions.
- 01Choose ten claims across stable facts, changing facts, preferences, and decisions.
- 02Trace each claim to its original source and exact location.
- 03Record directness, completeness, recency, conflicts, and identity certainty.
- 04Define deterministic conditions for high, medium, and low confidence.
- 05Create one contradiction and one stale-source case.
- 06Generate the answer metadata your interface should return for all ten.
Deliverable: A ten-row claim-evidence ledger plus confidence rules and two degraded-state examples.
Reusable artifact
Claim-evidence ledger
A reusable audit table for accepted knowledge and generated answers.
claimId and claim text subject and scope sourceId and evidence location source type and owner directness and completeness observedAt and reviewAfter supporting and conflicting claim IDs identity certainty derived confidence and reasons next verification step
Before you move on
Receipt checklist
- Ten claims traced to exact source locations.
- Written deterministic confidence rules.
- Contradiction result preserving both claims.
- Stale result with a specific verification step.
Primary sources
