@grok My Last Fable 5 build
Tweet 1/5Stress-tested Forge — my fully-local offline engineering RAG pipeline — against a real 51-page NASA technical report (TN D-7243, 1973). Real OCR’d scan, noisy layout, tables, mixed formatting. No synthetic data. Full pipeline: OCR → layout analysis → chunking → hybrid retrieval → grounded generation. Here’s what survived and what didn’t 🧵
Tweet 2/5 What held rock solid:✅ 208/208 numeric values extracted and preserved without hallucinated digits or transcription errors ✅ Precise page-level citation mapping across all 51 pages (section → page grounding) ✅ Zero fabrication: when disambiguation failed on ambiguous identifiers, the system returned empty results instead of guessing
Tweet 3/5 Failure point:Identifier/part-number extraction missed entirely. Regex patterns were tuned to a different numbering scheme (common in aerospace docs). The fallback hybrid retriever (BM25 nomic-embed-text semantic search) still surfaced relevant chunks, but lost the structured entity linking layer.
Tweet 4/5The pipeline ran entirely on my homelab (Proxmox LXC local Ollama llama.cpp backend). No cloud APIs, no telemetry, no data exfiltration.
Even with the extraction miss, the final grounded response stayed faithful to source material — critical for engineering trust.
Tweet 5/5This is what matters to me in local engineering AI: Don’t corrupt numbers. Don’t lie.
Forge passed the fidelity test where it counts most.
Building reliable offline tools > flashy demos.
#LocalAI #RAG #SelfHosted #Homelab #EngineeringAI