I was able to get roughly 2× the indexing performance in Exograph, my Elixir code search engine, by switching the default backend to DuckDB.
Search got a lot faster on several important paths too.
Yes, I know PostgreSQL can be extended with things like ParadeDB for full-text search, Citus for distributed Postgres and columnar storage, and all kinds of other serious machinery. But in practice, those setups are more operational complexity than I want here. My DuckDB-based setup is much simpler to operate, and I have full control over it through sweet Elixir APIs.
So I’ve finally solved one of the two major blockers for the OpenPencil backend (yes, you heard that right — there will be a backend soon) and its companion platform.
I think I’ll be ready to tease a few more building blocks this week, followed by a public roadmap for the whole ecosystem.
Stay tuned.