Thoughts from today’s lunch w/ friends & colleagues. Anyone who suggests that knowledge work is safe because AIs are not yet good enough misses the fundamental point. They do not write like skilled human writers; they cannot conduct strong unguided literature reviews; they cannot produce slides exactly as desired on the first try; they often fail at advanced statistics; they are not intutitive, and so on. All of these will change quickly. Where do humans retain an edge in a few years? I think it rests on two areas: - Very complex and open-ended environments where AI lacks sufficient training data (such as robotics in residential settings), though this will improve over time. - Tasks that involve human frictions in society and require direct human interaction. This is a pessimistic view, and I cannot offer a more optimistic one at the moment. I also feel that human societies, especially political systems, are not at least prepared for extremely powerful AIs controlled by a small number of entities. And even if access is uniform, the efficiency gain will be highly heterogeneous. I'm both extremely thrilled and extremely worried.

This is where I need to say that needling economics is fun because it has a few arrogant, loud people & everyone else is a good sport. Social science is hard, econ has contributed **so much** & this is all a team sport. But also there is no defensible disciplinary supremacy here

I did ask some social scientists if they understood these models, and most said they would not have thought to do them. But they're simple: compare siblings, compare the same person over time. Maybe it's ignorance holding social scientists back from testing theories properly.

Enough is enough. Just because you can generate an academic paper in minutes doesn't mean you should. When your name is on something, you should check every reference and claim before submitting. If you can't be bothered to do that, you should be banned from submitting.

Hot take: Decoders (GPT-4 etc.) are a distraction for text classification. Encoders and embedding models are fundamentally better and more efficient at this. If decoders have an advantage it's only because of massive time/research investment disparities.

Realistically, there are a range of ways to “use an LLM for research” that would lead to different levels of error probabilities and everyone is going to choose a different point along that tradeoff curve. Heterogeneous risk profiles are ok, I think!

Claude whispering in my ear after a week of iterating on a codebook: "Just saying... if you picked something with readily available data we could have done three papers by now."