Excited to share some new work: we show how to efficiently train small vision-language models to use a zoom tool with GRPO. We identify key points for doing small-scale RL for tool use in VLMs generally, and share what works (and what doesn’t) under significant resource constraints. arXiv link below ⬇️

We've open-sourced a MCP that allows big models to use huggingface computer vision models as tools. This allows Claude to act as a "visual agent", using other task specific models to help it solve problems. Below, is an example of Claude using an open vocab object detector to zoom in on small details to solve a hard problem that it could not solve natively. Additionally, we've written a blog post discussing why outsourcing vision capabilities from large models is something you should consider. MCP Repo: github.com/groundlight/mcp-v… Blog post: groundlight.ai/blog/vision-a…

We just released an open-source framework that makes it easy to build visual reasoning agents (with GRPO). github.com/groundlight/r1_vl…

It’s my pleasure making some minor contribution to this great project! This attention rollout video is my favorite “visualization of the year” so far. Try out our demo if you are interested and stay tuned for our future research!

Just dropped ⚡ - our latest youtube tutorial, running Groundlight's computer vision on a Raspberry Pi! Watch it here: youtube.com/watch?v=YpNKHjuZ… Tim Huff, our Robotics Engineer, walks you step-by-step how to create an application to get notified if someone steals your parking spot. Unlike traditional computer vision: ☑ No need for a dataset ☑ Works on day one ☑ Adapts to your real environment

#CVPR2024 down the street gets our science team out of the office! 10,000 computer vision researchers in Seattle figuring out what's next in the field from multimodal foundation models to the most creative applications.

🚀 Exciting News from Groundlight AI! 🚀 Introducing, the Groundlight Hub, your easy access to computer vision. youtu.be/u3xPnOnvTcE

Our Robotics Engineer, Tim Huff, gives a demo of Groundlight AI's Visual Inspections 🔍 and Anomaly Detection solution, compatible with @Universal_Robot 🤖 cobots:

Importance of Demand Generation ✍️ 🌟 The number of GPUs we own and the robustness of our L1 integration are crucial, but they're not the sole determinants of our long-term success. It's all about creating demand for our product. 🥅 Our ultimate goal is to stimulate demand for our GPU nodes through diverse integrations and utilities. We're committed to providing solutions that meet the needs of our users and drive adoption. 📈 In just the last 3 days, we've seen a significant uptick, accumulating nearly 400 additional rental hours for our $GPUs. This demonstrates the growing interest and demand for our platform. 🥇 Our primary focus remains clear: to establish #NodeAI as the premier infrastructure provider for high-performance AI. Demand generation is key, and we're dedicated to making it happen. #AI #DEPIN #GPU

Super excited to share this paper! Temporal chaos in pretraining corpora greatly impacts language model’s capabilities in answering time-sensitive questions. We will publish our codes for dataset generation and temporal alignment later. Stay tuned!

This is a joint work co-led with my awesome mentees @BowenROIM and @zanbrum. Please reach out to them if you have job or collaboration opportunities for them! Finally, always many thanks to our fantastic advisors @HannaHajishirzi and @nlpnoah ❤️❤️❤️

What if we combine PPO with Monte-Carlo Tree Search – the secret sauce for AlphaGo to reach superhuman performance? Spoiler: MAGIC!! Our inference-time decoding method, PPO-MCTS, achieves impressive results across many text generation tasks. 📜 arxiv.org/abs/2309.15028 🧵(1/n)

How robust are the instructions in your instruction-tuned model? In our most recent work (w/ @ChantalShaib and @byron_c_wallace), we show that there is a considerable dip in performance on in-domain tasks when you slightly vary the instruction. arxiv.org/abs/2306.11270

Perisic and Icardi winning league titles and still in the UCL while Conte & Inter........