I've gotten a lot of really cool and thought provoking outreach since I started thinking outloud about some of the ways AI might change (and hopefully, improve) research.
One of the ideas floating around, which
@sreeramkannan and
@ben_chain brought up, was that LLMs could evaluate how well pre-analysis plans were followed in the ultimate paper.
It turns out someone has already been working on that!
Jamie Cummins sent me his fascinating tool called RegCheck which produces full reports (see below) on how closely the plan is followed. Super cool!
Since I extended my own research using AI, I've been thinking about how it's going to reshape research and universities.
We can now build new institutions where research is continuously updated, automatically verified, and carried out at immensely greater scale.
Picture a research institute where senior scholars direct dozens or even hundreds of AI agents on coordinated programs. Small teams providing questions and judgment while agents handle collection, analysis, and verification.
What would it take to build? The requirements are almost comically simple: (1) compute funding for researchers, and (2) a commitment to hire ambitious people and get out of their way.
This new institute can unlock totally new way to do research:
--Living research that automatically updates any time new data arrives, so our knowledge stays up to date
--Automatically verified research that we know replicates from the moment it's posted publicly
--Hyperscaled descriptive work that ingests enormous bodies of political data, like the entire history of changes to the US tax code or every bill introduced in every state legislature
--Prototypes for new governance tools that are built for communities and then tested alongside them
I think we're stepping into a crazy new era of how social science is done. I offer more thoughts on what's changing and how we might design an AI-first university of the future in my post, linked below.