What if we combined RLMs with DataFrames?
Recursive language models by
@a1zhang and
@lateinteraction are a promising new direction for LLMs...
... but initially only operate on strings. The REPL they use is capable of much more than that.
By combining RLMs, DSPy, and DataFrame support you can do things like achieve ~87% on DABench in 15 lines of code, largely by "getting out of the way" of the model.
The approach is fairly simple:
- Import the required libraries into the REPL sandbox for your type, in this case pandas and pyarrow for efficiency
- Define a serialization approach to get the data from host to sandbox
- Define the Output fields you want
- Let the model loose in the REPL; it will iteratively explore and analyze the data until it's ready to return the response