A context engineering metaphor I've been playing around with:
- Primary source: the source of truth. Raw data. Transcripts. Code.
- Secondary source: one step removed. Summaries. Compactions. Documentation.
For instance, compaction takes a primary source (the conversation history) and turns it into a secondary source (the summary). This is lossy, but means the secondary source can fit into a smaller space.
If you want to know what your codebase does, your code is a primary source. Your docs are a secondary source.
Loading primary sources into context is expensive, but provides richer context. Secondary sources are cheaper to load into context, but may be information-lossy.
Any context engineering will involve managing the tradeoffs between both.