Introducing Vega: See who you are, from the people who matter.

THE WAY OF CODE, a project by @rickrubin in collaboration with Anthropic:

Why did only humans invent graphical systems like writing? 🧠✍️ In our new paper at @cogsci_soc, we explore how agents learn to communicate using a model of pictographic signification similar to human proto-writing. 🧵👇

Excited to share Penzai, a JAX research toolkit from @GoogleDeepMind for building, editing, and visualizing neural networks! Penzai makes it easy to see model internals and lets you inject custom logic anywhere. Check it out on GitHub: github.com/google-deepmind/p…

a nyt connections game with "orca", "llama", "alpaca" and "vicuna" would such a dogwhistle

Compositional generalization is a major challenge for neural networks. In a #NeurIPS2023 spotlight paper with @tserre and @Brown_NLP, we ask whether neural networks learn the types of representations that are a prerequisite for compositionality! (1/14)

Introducing Gemini 1.0, our most capable and general AI model yet. Built natively to be multimodal, it’s the first step in our Gemini-era of models. Gemini is optimized in three sizes - Ultra, Pro, and Nano Gemini Ultra’s performance exceeds current state-of-the-art results on 30 of the 32 widely-used academic benchmarks. With a score of 90.0%, Gemini Ultra is the first model to outperform human experts on MMLU. blog.google/technology/ai/go…

Domain experts often have intuitions about the algorithms that transformers may use to solve tasks, but do models actually use them? In new work with @tserre and @Brown_NLP, we introduce circuit probing, a method for uncovering circuits that compute intermediate variables. (1/15)

Now that you’ve no doubt solved your Sunday crossword puzzle, looking to read about crosswords and linguistics? In The Atlantic theatlantic.com/science/arch…, Scott AnderBois, @NickATomlin, and I talk about what linguistics can tell us about crosswords and vice versa. Thread.

LLMs such as ChatGPT and BLOOMZ claim that they are multilingual, but does this mean they can generate code-mixed data? Follow this 🧵 to find out. (1/N) Paper: arxiv.org/abs/2303.13592

The new scaling laws

Last year, we criticized LMs for performing “too well” with pathological prompts, and many papers have now shown similar results with corrupted ICL or CoT. In our new work, we find that *humans* also perform surprisingly well with irrelevant prompts! (But not misleading ones.) ⅕

Why does ChatGPT work so well? Is it “just scaling up GPT-3” under the hood? In this 🧵, let’s discuss the “Instruct” paradigm, its deep technical insights, and a big implication: “prompt engineering” as we know it may likely disappear soon:👇

My new blog post takes a look at the state of multilingual AI. 🌍 How multilingual are current models in NLP, vision, and speech? 🏛 What are the recent contributions in this area? ⛰ What challenges remain and how we can we address them? ruder.io/state-of-multilingu…

very excited to share our paper on reconstructing language from non-invasive brain recordings! we introduce a decoder that takes in fMRI recordings and generates continuous language descriptions of perceived speech, imagined speech, and possibly much more biorxiv.org/content/10.1101/…

OK, debates about the necessity or "priors" (or lack thereof) in learning systems are pointless. Here are some basic facts that all ML theorists and most ML practitioners understand, but a number of folks-with-an-agenda don't seem to grasp. Thread. 1/

New research-y project: Blueprints for Intelligence, a visual history of artificial neural networks from 1943 to 2020 philippschmitt.com/blueprint…

🔥Our work has now been accepted to NeurIPS 2022 !! `Toward a realistic model of speech processing in the brain with self-supervised learning’: arxiv.org/abs/2206.01685 Let’s meet in New Orleans on Tue 29 Nov 2:30pm PST (Hall J #524). A recap of the 3 main results below 👇

Lots of folks are talking about *emergence* in Deep Learning as if it's a new thing, that happens only in large language models at scale. It's not! It has been happening for decades and in very small networks. 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵