Research Scientist @ Google. Past: PostDoc at UoE. PhD in NLP/MT @edinburghnlp. All opinions are my own.

Joined March 2018
33 Photos and videos
Pinned Tweet
9 Jul 2025
T5Gemma!
The Gemma family is growing today. First up: T5Gemma ✨, the new generation of encoder-decoder models ↓ developers.googleblog.com/en…
1
1
7
1,058
Biao Zhang retweeted
Introducing T5Gemma 2, the next generation of encoder-decoder models 🚀 Built on top of Gemma 3, we were able to build compact models at sizes of 270m-270m, 1B-1B, and 4B-4B sizes. While most models today are decoder-only, T5Gemma 2 is the first (I'm aware of) multimodal, long-context, and heavily multilingual (140 languages) encoder-decoder model out there. We hope this model enables the model research community as well as the community of devs ready to explore with new architectures. Blog: blog.google/technology/devel… Models: huggingface.co/collections/g… Paper: arxiv.org/abs/2512.14856
10
95
1,673
130,521
Biao Zhang retweeted
We just release 2 new open-weight Gemma models. FunctionGemma and T5Gemma optimized for on-device agentic actions and multimodal applications. FunctionGemma 🤖 270M parameter built for on-device tool use. 📂 32K token context window. 📱 85% accuracy on mobile system call identification. 🧠 Trained on 6 trillion tokens. T5Gemma 2 🖼️ Multimodal encoder-decoder architecture handling both text and image inputs. 🌐 128K context window across over 140 languages. 📏 Available in three sizes: 270M, 1B, and 4B parameters. 👁️ Normalizes images to 896x896 resolution, encoded into 256 tokens each.
13
49
542
31,395
Biao Zhang retweeted
Meet T5Gemma 2, the next evolution of Google's encoder-decoder family! 🚀 Building on Gemma 3, these models bring major upgrades to efficiency and capability: 🖼️ Multimodal: Understands images text out of the box. 📚 128K Context: Handles massive datasets with long-context support. 🌍 140 Languages: Massive multilingual training. ⚡ Efficient Architecture: New tied embeddings & merged attention for faster inference and smaller footprints (270M, 1B, and 4B sizes). Check out the pre-trained checkpoints on Kaggle and Hugging Face now! 🛠️✨
1
2
4
392
18 Dec 2025
Today, we are thrilled to release T5Gemma 2, the next generation of T5Gemma with multilingual, multi-modal, and long-context capabilties. Read the announcement👉 blog.google/technology/devel…
Introducing T5Gemma 2, the next generation of encoder-decoder models 🚀 Built on top of Gemma 3, we were able to build compact models at sizes of 270m-270m, 1B-1B, and 4B-4B sizes. While most models today are decoder-only, T5Gemma 2 is the first (I'm aware of) multimodal, long-context, and heavily multilingual (140 languages) encoder-decoder model out there. We hope this model enables the model research community as well as the community of devs ready to explore with new architectures. Blog: blog.google/technology/devel… Models: huggingface.co/collections/g… Paper: arxiv.org/abs/2512.14856
1
1
6
290
18 Dec 2025
That foundation inspired T5Gemma, our first recipe for adapting modern strong decoder-only models into encoder-decoder models🔗 arxiv.org/abs/2504.06225 Now we extend it to the multimodal and long-context regime with T5Gemma 2!
1
1
159
18 Dec 2025
Can't wait to see what you build!
1
72
Biao Zhang retweeted
Introducing T5Gemma 2, the next generation of encoder-decoder models, built on the powerful capabilities of Gemma 3. Key innovations and upgraded capabilities include: Multimodality Extended long context Support of 140 languages out of the box Architectural improvements for efficiency And more blog.google/technology/devel…
39
227
1,827
280,296
Biao Zhang retweeted
Introducing T5Gemma 2, the next generation of encoder-decoder models 🚀 Built on top of Gemma 3, we were able to build compact models at sizes of 270m-270m, 1B-1B, and 4B-4B sizes. While most models today are decoder-only, T5Gemma 2 is the first (I'm aware of) multimodal, long-context, and heavily multilingual (140 languages) encoder-decoder model out there. We hope this model enables the model research community as well as the community of devs ready to explore with new architectures. Blog: blog.google/technology/devel… Models: huggingface.co/collections/g… Paper: arxiv.org/abs/2512.14856
75
169
1,471
247,795
Biao Zhang retweeted
Introducing EmbeddingGemma, our newest open model that can run completely on-device. It's the top model under 500M parameters on the MTEB benchmark and comparable to models nearly 2x its size – enabling state-of-the-art embeddings for search, retrieval more.
199
500
7,388
535,179
Biao Zhang retweeted
Introducing EmbeddingGemma, our new open embedding model for on-device AI applications. - Highest ranking open model under 500M on the MTEB benchmark. - Runs on less than 200MB of RAM with quantization. - Dynamic output dimensions from 768 down to 128. - Input context length of 2048 tokens. - Trained on over 100 languages. - Based on Gemma 3 270M. Start building today with @GoogleDeepMind Embedding Gemma and @huggingface Sentence Transformers, @ollama, Llama.cpp, MLX, @lmstudio, @weaviate_io, @googlecloud Vertex AI, @AMD, @baseten, @Cloudflare, @nvidia, and more.
25
58
454
38,675
Biao Zhang retweeted
Introducing EmbeddingGemma🎉 🔥With only 308M params, this is the top open model under 500M 🌏Trained on 100 languages 🪆Flexible embeddings (768 to 128 dims) with Matryoshka 🤗Works with your favorite open tools 🤏Runs with as little as 200MB developers.googleblog.com/en…
27
152
1,160
83,734
Biao Zhang retweeted
Introducing EmbeddingGemma: our new open, state-of-the-art embedding model designed for on-device AI 📱
35
112
970
153,826
4 Sep 2025
From encoder-decoder to world-class embeddings! 🚀 Super excited to introduce EmbeddingGemma, our new open embedding model. It leverages the strong encoder from its sibling, T5Gemma, to achieve new SOTA performance for models under 500M! Dive in 👇
EmbeddingGemma is our new best-in-class open embedding model designed for on-device AI. 📱 At just 308M parameters, it delivers state-of-the-art performance while being small and efficient enough to run anywhere - even without an internet connection.
9
798
Biao Zhang retweeted
Our Google Translate team is bringing a strong presence to #ACL2025 in Vienna this week! 🇦🇹 My group is excited to present several of our latest papers. 👇 Don't miss them!
1
5
53
3,256
Biao Zhang retweeted
Voxtral uses online DPO!
22 Jul 2025
In our continued commitment to open-science, we are releasing the Voxtral Technical Report: arxiv.org/abs/2507.13264 The report covers details on pre-training, post-training, alignment and evaluations. We also present analysis on selecting the optimal model architecture, which pre-training format to use, and the benefits of DPO.
3
7
30
3,386
Biao Zhang retweeted
Introducing T5Gemma: the next generation of encoder-decoder/T5 models! 🔧Decoder models adapted to be encoder-decoder 🔥32 models with different combinations 🤗Available in Hugging Face and Kaggle developers.googleblog.com/en…
21
132
773
85,145