Fascinating performance design direction: GenAI specializing your DB to your workload.
This is fascinating to me due to some key parallels:
* I already practice "eval-driven AI coding loops", and database performance is one of the happiest cases for this - clear conformance suites in correctness and clear goal to hill climb via optimization.
* Moving this idea closer to the use case and runtime, while still staying safely offline, are both cool
* In GFQL, our open-source GPU graph query language, we have started stacking specialization layers in the engine, where we take advantage of another phrasing of this: pay-as-you-go semantics. Simpler base layers can ignore complications of fancier language features so simpler queries can go faster, and fancier features get case-split so we optimize their different scenarios. Instead of one path straddling all cases, we have specialization layers and pockets. It used to be a LOT harder to identify the scenarios, refine the solutions, and build the safety & maintenance layers for this, while now we can easily adapt big conformance suites and run all sorts of test amplifiers whenever we add a new specialization.
Overall, this paper leans into using AI to handle more complexity in, for now, narrow domains. It realizes databases are the happy case of easy for AI to test & verify, and the space of many optimizations that are well-understood: these make it a lot easier for eval-driven AI coding loops to hill climb up the performance charts for arbitrary workloads.
Link: