Palantir built a $400B empire on ontology-first AI systems long before LLMs.
And yet today, most GraphRAG tutorials skip the ontology entirely.
There’s a massive problem with how GraphRAG is being taught.
Most tutorials jump straight to:
• Embeddings
• Vector search
• Extracting entities and relationships
• Fancy retrieval pipelines
But they skip the most important step…
The data model.
If you let an LLM “extract entities and relationships” freely, it will:
• Invent new entity types per document
• Create vague edge types
• Duplicate the same entity under slightly different names
At 10 documents, it looks fine.
At 100, it’s messy.
At 1,000 , it’s unusable.
Better prompting won't fix this... You need an ontology.
Hot take: Ontologies are what make GraphRAG scalable.
This is how we modeled GraphRAG for our Digital Twin / Second Brain style personal assistant using
@MongoDB as the memory layer.
Before writing a single extraction prompt, we defined:
• 6 node types
• 8 edge types
Small enough to stay consistent.
Specific enough to be useful.
The ontology became a contract.
The LLM can only extract what we define.
The query layer knows exactly what to expect.
Without an ontology, the LLM extracts hundreds or thousands of triples per document.
With an ontology, you extract only what matters to your business problem.
This means:
• Fewer triples
• Higher-quality triples (signal > noise)
• Only storing data you care about
• Faster and cheaper ingestion
Once the task is tightly scoped, you can use cheaper APIs like Gemini Flash Lite or even fine-tuned small language models scoped to your ontology.
Ontology smaller models = economically scalable GraphRAG.
But not everything goes through the LLM...
LLM-powered extraction:
• Person
• Task
• Episode
• Preference
Metadata-driven:
• Document
• Chunk
You already know titles, URIs, authors, ordering, references.
Don’t pay an LLM to rediscover metadata.
That reduces cost, latency, hallucination surface, and lets you use cheaper models safely.
From there, the storage pattern is straightforward (especially in
@MongoDB):
Raw documents land in one collection.
Extracted observations are logged immutably.
A materialized graph view is rebuilt via aggregation.
Because the schema is strict, the query becomes predictable:
1. Text search
2.
$vectorSearch
3. Reciprocal Rank Fusion (RRF)
4.
$graphLookup with bounded expansion
The data model enables the query.
tl;dr:
GraphRAG is a schema design problem.
Yes, LLMs are powerful.
But without an ontology, they generate structured noise.
Design the ontology first.
Everything else follows.
P.S. Did you define your ontology before writing extraction code or after your graph started breaking?