💫 After four PhD years on all things multimodal, pre- and post-training, I’m super excited for a new research chapter @GoogleDeepMind 🇨🇭! Biggest thanks to @zeynepakata and @OriolVinyalsML for all the guidance, support, and incredibly eventful and defining research years ♥️!

Released last week, and already more than 4M downloads on HuggingFace alone 😊 This makes Gemma 4 12B the most popular encoderfree VLM by a large margin. In addition to being the first-ever general purpose LLM with encoderfree audio input!

We just dropped Gemma 4 Quantization-Aware Training (QAT) checkpoints on Hugging Face! All Gemma 4 model sizes and their drafters are now optimized with QAT to cut memory requirements and maximize on-device performance!

This video is at normal speed. Gemma 4 12B MLX version running locally at 50 tokens/sec. Thank you Google DeepMind team. This model feels really solid for a lot of small local tasks and everyday AI workflows.

I am SO excited to be sharing that I am joining @BerntBornich and @1x_tech to lead the new 1X World Model Lab aimed at building the next frontier of embodied AI! The core guiding principle of the lab is: scale up along every damn axis!! 🚀 Robotics data is NOT a second-class citizen - it is too important of a problem to be left to fine tuning! Your model needs to see your most important tokens from step 0 We need to think about robotics through the first principles of AI: how do we best utilize the vast amounts of web-scale media and how do we create a data-flywheel to collect millions of hours of rich robot interactions. There is no other moat in AI outside of data and @1x_tech has done an INCREDIBLE job scaling manufacturing, production and hardware to build humanoid robots that can create a unique data-flywheel in unstructured environments. Scaling data collection for highly dexterous on-policy robot data will be the only way for creating a moat in AI. @JackMonas and team have made great progress in building World Models, and now the goal is to supercharge this effort by starting a hyper-focused scale and data-pilled lab. Before scaling compute / data / models, we are currently RAPIDLY scaling our team and hiring across the 4 core pillars of AI: model data, data infra, ML infra and evals. Looking for folks that are excited about the 0->1 problem and share the same principles as us. There’s a single application for everyone in the lab - if you’re a good at engineering and ML, we will find a place for you in the team ❤️ AGI won’t be solved by fine-tuning… Let’s build the next frontier of AI together 🚀 My DMs are always open!!

Gemma 4 12B was a large team effort over more than a year. The model’s encoder-free tech was developed by @ASusanoPinto @AndreasPSteiner @confusezius @kmisiunas & myself with many contributions from @ashkamath20 @LawrenceSt72142 @OlivierBachem @armandjoulin & the whole Gemma Team

For the past years my research focus was on unifying models and training paradigms across modalities. Today I'm excited that we're releasing our latest model aligned with this theme: Gemma 4 12B, a dense encoder-free model which processes raw text, image, and audio inputs! 1/

Gemma 4 Encoder-Free is here!! 🥳📷🔊 Super excited by this Gemma 4 12B model with great performance on vision and audio, without modality specific encoders. Closing in on the 26B while being 2x smaller in memory! Available now on Hugging Face, Kaggle, llama.cpp, and others!

Meet Gemma 4 12B! A unified, encoder-free multimodal model designed to bring high-performance intelligence directly to your laptop, and released under an Apache 2.0 license. Bridging the gap between edge efficiency and advanced reasoning. Here is what’s new with Gemma 4 12B: 👇

Gemma 4 12B in action: Object detection, function calling, voice command, segmentation, language switch, translation - all of this and much more without vision/audio encoders! (Inputs and outputs are real, but FC2 data shown as code, and generation speedified)

Really proud of everyone in the team that worked hard for this release: Gemma 4 is here! blog.google/innovation-and-a…

Meet Gemma 4! Purpose-built for advanced reasoning and agentic workflows on the hardware you own, and released under an Apache 2.0 license. We listened to invaluable community feedback in developing these models. Here is what makes Gemma 4 our most capable open models yet: 👇

I'm in Nashville for CVPR and wow, the Music City name is not exaggerated. If you're around, we'll be presenting our work on temporal model merging with @vishaal_urao, @confusezius, and @AmyPrb on Saturday 5-7 pm in ExHall D (poster #445). Come say hi!