04 / 08Builder47 min lesson + lab

Build a Self-Wiring Ingestion Flow

Capture durable pages and turn explicit links into a typed graph while keeping model suggestions outside canonical truth until reviewed.

By the end

You can use GBrain capture, put_page, deterministic link extraction, schema validation, backfills, and an optional reviewed proposal layer without confusing them.

01

Use capture for sources and put_page for MCP writes

gbrain capture accepts direct text, a file, or stdin and returns the created slug or a JSON receipt. It is a CLI operation, not an MCP tool. Agents connected through MCP write durable content with put_page. Keep source capture distinct from later interpretation so a model, parser, or human edit never replaces original evidence silently.

02

Validate every durable page at the write boundary

A put_page write should match the active schema pack, preserve source references, and fail visibly on invalid structure. Use gbrain schema active, then gbrain schema validate <pack-name> to inspect the exact installed contract by name. If you build a model extractor on top, validate its output before calling put_page and permit unknown or no change instead of forcing a confident page.

03

Understand what stock self-wiring actually does

Stock GBrain scans explicit Markdown wikilinks and typed-link syntax when put_page writes a page. Those references become typed graph edges through deterministic pattern matching with zero LLM calls. An unknown target may become a stub, and older pages can be backfilled with gbrain extract links. This is inspectable self-wiring: the author writes the relationship and the system indexes it.

  • Write an explicit reference such as [[wiki/people/alex]] when the relationship is known.
  • Use the schema's typed-link fields when direction and relationship type matter.
  • Run extraction in dry-run mode before backfilling historical pages.
  • Inspect counts and neighborhoods with gbrain stats and gbrain graph-query.
04

Treat LLM-generated edges as an optional extension

GBrain's deterministic self-wiring does not infer every relationship in prose. If you add an LLM that suggests missing edges, label that as custom application code outside stock behavior. Each suggestion needs from, type, to, source span, confidence, and explanation; it remains a proposal until schema validation and human review. Never describe an inferred edge as one GBrain extracted deterministically.

05

Prove reruns and interruptions are safe

Use source IDs and extraction versions so the same source does not duplicate nodes on every run. Produce a change manifest before persistence and an operation journal during it. If a run stops halfway, recovery should resume or roll back predictably. Rerunning unchanged input should produce no material graph change.

Working reference

Commands and patterns

Capture text, files, and pipes with real receipts

gbrain capture 'A durable thought with [[wiki/projects/atlas]]' --json
gbrain capture --file ./notes/today.md
echo 'Captured from stdin' | gbrain capture --stdin

These are stock CLI capture paths. Use --quiet when a shell workflow needs only the created slug.

Preview and apply deterministic link backfill

gbrain extract links --source db --dry-run | head -5
gbrain extract links --source db
gbrain stats

First inspect historical link extraction, then apply it and verify graph counts. This follows explicit references; it does not ask an LLM to invent relationships.

When the happy path breaks

Failure modes

01

Symptom

Every mention becomes a new node.

Likely cause

Extraction creates entities before canonical lookup and usefulness checks.

Recovery

Resolve identity first, update canonical nodes, and allow mentions to remain source annotations.

02

Symptom

The graph contains confident unsupported edges.

Likely cause

The model returned relationships without evidence-span validation.

Recovery

Require source and span for each edge, constrain allowed types, and quarantine invalid proposals.

03

Symptom

An interrupted ingest leaves sources and graph out of sync.

Likely cause

Persistence has no manifest, journal, or transaction boundary.

Recovery

Stage the batch, journal writes, verify counts, and implement resume or rollback.

Hands-on lab

Ingest one source into a reviewed graph patch

Build an end-to-end flow that remains inspectable at every stage.

  1. 01Capture the source with gbrain capture --json and preserve its slug and receipt.
  2. 02Create or update a schema-valid page containing one explicit wikilink and one typed link through put_page or the supported page workflow.
  3. 03Run deterministic link extraction as a dry run and inspect the proposed source and target slugs.
  4. 04Apply the backfill, inspect stats, and query the target neighborhood.
  5. 05Optionally build one LLM edge suggestion, but keep it in a separate proposal file until reviewed.
  6. 06Rerun unchanged extraction and prove the graph counts do not inflate.

Deliverable: A source packet, validated graph proposal, review decision, applied patch, and no-op rerun receipt.

Reusable artifact

Graph ingest manifest

A complete review surface for one proposed graph mutation.

operationId, sourceId, extractionVersion, baseRevision
nodesToCreate: [{ id, type, name, evidence }]
nodesToUpdate: [{ id, fieldChanges, evidence }]
claimsToCreate: [{ claim, source, span, time }]
edgesToCreate: [{ from, type, to, evidence, confidence }]
identityProposals: [{ candidates, evidenceFor, evidenceAgainst }]
conflicts and validationWarnings
automatic, approvalRequired, and rejected groups
expected counts and rollback reference

Before you move on

Receipt checklist

  • Preserved source and span-aware extraction.
  • Schema-valid node, claim, and edge proposal.
  • Identity-resolution decisions with reasons.
  • Reviewed manifest and applied graph revision.
  • No-op second-run receipt.

Primary sources

Continue with the source

Home
Book
Blog
Calugaru Labs
GitHub
LinkedIn
Email
Theme