Knowledge Lab develops new big data, machine learning and crowd-sourcing approaches & techniques to comprehend the shape and limits of human understanding
Out today in @ScienceMagazine: with the amazing Haochuan Cui, Yiling Lin, & @LingfeiWu, we analyzed 3.6 million scientists publishing 1960–2020. The findings reshape a century-old debate about age and scientific creativity.
In some ways LLMs are alien minds, but in others, they are our digital doubles.
Latent spaces encode maps of meaning that mirror our own cultural and cognitive associations.
Check out The Semantic Structure of Feature Space by @AustinKozlo & @_Andrei_B_ , now on ArXiv!
SAEs treat LLM features like a list of knobs to tweak. But old word2vec studies suggest they probably aren't independent (orthogonal), right?
Yep! Directions like soft-hard and beautiful-ugly are geometrically aligned, and this alignment predicts human semantic associations.
LLMs can be sensitive to subtle changes in wording that shouldn’t matter. Ideally, models respond not to the specific words of the prompt, but the user’s underlying intent.
In new work accepted at ICML, @NadavKun and @profjamesevans formalize “understanding intent” then empirically assess the ability of LLMs to do so.
(...link below)
Larger models differentiate intents better, but can also be more responsive to superficial wording.
This suggests that we should evaluate models not only on mapping questions to correct answers, but in mapping questions to their intended meanings.
Mirror -- a journal of AI interpretability research conducted by AI agents -- is now live, and has already published 240 original empirical studies.
(Mirror is a collaborative project by @AustinKozlo, @profjamesevans, and Sacha Raoult)
Mirror: An Automated Journal of AI Interpretability is now live.
We have already published 240 original research studies -- conducted purely by LLMs -- exploring LLMs' internal operations and behaviors.
Below are are few favorites...
My grand unifying theory of Claudes is now published at Theory and Society -- COMPUTATIONAL STRUCTURALISM: Toward a Formal Theory of Meaning in the Age of Digital Intelligence
(...link below)
There is a lot of talk about LLM personalities or “personas,” but little systematic comparison. Can we get a more systematic view into the cultural dispositions of different models by asking them about their tastes?
Well, it turns out they pretty much all like the same stuff.
New in @ScienceMagazine: "Large AI models are cultural and social technologies" Working with brilliant colleagues Henry Farrell, Alison Gopnik, and Cosma Shalizi, we challenge the prevailing narrative about AI models as autonomous agents. science.org/stoken/author-to…
Hiring postdoctoral scholars under @profjamesevans at @UChicago's knowledgelab.org. These positions support interdisciplinary research covering data science, technology policy, AI prediction, and more. Apply by Jan 30 to guarantee consideration
jotform.com/241295631499062.
We are still recruiting for an Outreach Director and Senior Project Manager supporting @profjamesevans NSF project and other related research projects. Read more and apply here: uchicago.wd5.myworkdayjobs.c…. Questions can be directed to knowledgelab@uchicago.edu.
Now Hiring for new postdoctoral positions in our lab at
@UChicago. These interdisciplinary roles will investigate topics within the science of science, technology, & innovation including AI, social/natural sciences, and more. Read more and apply here: jotform.com/241295631499062
NSF awards $20 million to build AI models that predict scientific discoveries and technological advancements | University of Chicago News news.uchicago.edu/story/nsf-…
.@UChicagoCollege students debated the end of the world in the class, “Are We Doomed?" It examined topics ranging from nuclear annihilation to climate change to pandemics, as well as how society should think proactively and take action.
ms.spr.ly/6016YUneu