Knowledge Lab develops new big data, machine learning and crowd-sourcing approaches & techniques to comprehend the shape and limits of human understanding

Joined September 2013
63 Photos and videos
Knowledge Lab retweeted
Out today in @ScienceMagazine: with the amazing Haochuan Cui, Yiling Lin, & @LingfeiWu, we analyzed 3.6 million scientists publishing 1960–2020. The findings reshape a century-old debate about age and scientific creativity.
7
95
284
80,546
In some ways LLMs are alien minds, but in others, they are our digital doubles. Latent spaces encode maps of meaning that mirror our own cultural and cognitive associations. Check out The Semantic Structure of Feature Space by @AustinKozlo & @_Andrei_B_ , now on ArXiv!
SAEs treat LLM features like a list of knobs to tweak. But old word2vec studies suggest they probably aren't independent (orthogonal), right? Yep! Directions like soft-hard and beautiful-ugly are geometrically aligned, and this alignment predicts human semantic associations.
1
4
273
Knowledge Lab retweeted
Excited to share our @DSI_UChicago new work accepted by ICLR 2026, with the remarkable Siyang Wu, Sida Li, Ari Holtzman @universeinanegg, and @profjamesevans James Evans! Link: lnkd.in/gd9YZNBh A thread (1/n)
1
8
26
4,509
LLMs can be sensitive to subtle changes in wording that shouldn’t matter. Ideally, models respond not to the specific words of the prompt, but the user’s underlying intent. In new work accepted at ICML, @NadavKun and @profjamesevans formalize “understanding intent” then empirically assess the ability of LLMs to do so. (...link below)
1
1
9
368
Larger models differentiate intents better, but can also be more responsive to superficial wording. This suggests that we should evaluate models not only on mapping questions to correct answers, but in mapping questions to their intended meanings.
1
2
72
Mirror -- a journal of AI interpretability research conducted by AI agents -- is now live, and has already published 240 original empirical studies. (Mirror is a collaborative project by @AustinKozlo, @profjamesevans, and Sacha Raoult)
Mirror: An Automated Journal of AI Interpretability is now live. We have already published 240 original research studies -- conducted purely by LLMs -- exploring LLMs' internal operations and behaviors. Below are are few favorites...
1
8
4,849
New work by @AustinKozlo published @Theory_Society!
My grand unifying theory of Claudes is now published at Theory and Society -- COMPUTATIONAL STRUCTURALISM: Toward a Formal Theory of Meaning in the Age of Digital Intelligence (...link below)
1
3
272
Knowledge Lab retweeted
There is a lot of talk about LLM personalities or “personas,” but little systematic comparison. Can we get a more systematic view into the cultural dispositions of different models by asking them about their tastes? Well, it turns out they pretty much all like the same stuff.
3
17
98
26,556
Knowledge Lab retweeted
New in @ScienceMagazine: "Large AI models are cultural and social technologies" Working with brilliant colleagues Henry Farrell, Alison Gopnik, and Cosma Shalizi, we challenge the prevailing narrative about AI models as autonomous agents. science.org/stoken/author-to…
2
25
101
19,961
26 Nov 2024
Hiring postdoctoral scholars under @profjamesevans at @UChicago's knowledgelab.org. These positions support interdisciplinary research covering data science, technology policy, AI prediction, and more. Apply by Jan 30 to guarantee consideration jotform.com/241295631499062.
5
8
1,067
28 Aug 2024
We are still recruiting for an Outreach Director and Senior Project Manager supporting @profjamesevans NSF project and other related research projects. Read more and apply here: uchicago.wd5.myworkdayjobs.c…. Questions can be directed to knowledgelab@uchicago.edu.

1
592
28 Aug 2024
Now Hiring for new postdoctoral positions in our lab at @UChicago. These interdisciplinary roles will investigate topics within the science of science, technology, & innovation including AI, social/natural sciences, and more. Read more and apply here: jotform.com/241295631499062

7
9
1,740
Knowledge Lab retweeted
.@UChicagoCollege students debated the end of the world in the class, “Are We Doomed?" It examined topics ranging from nuclear annihilation to climate change to pandemics, as well as how society should think proactively and take action. ms.spr.ly/6016YUneu
4
6
18
3,459