Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: go.meta.me/591040 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: go.meta.me/305985 These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

Meet SAM 3, a unified model that enables detection, segmentation, and tracking of objects across images and videos. SAM 3 introduces some of our most highly requested features like text and exemplar prompts to segment all objects of a target category. Learnings from SAM 3 will help power new features in Instagram Edits and Vibes, bringing advanced segmentation capabilities directly to creators. 🔗 Learn more: go.meta.me/591040

I am humbled to be re-featured as Women in Computer Vision for the BEST of CVPR section of the Computer Vision News July Magazine. It was great chatting with Ralph Anzarouth. I hope my unconventional career path can encourage more female researchers. rsipvision.com/ComputerVisio…

Knowledge vs Large Models? Welcome to our #CVPR23 tutorial "Knowledge-Driven Vision-Language Encoding" with @Xudong_Lin_AI @jayleicn @mohitban47 @cvondrick @Shih-Fu Chang @elgreco_winter Jun 19: 9:00-12:30 Loc: East 8 Website:blender.cs.illinois.edu/tuto… Zoom:cvpr2023.thecvf.com/virtual/…

Can pretrained vision models generalize to audio-visual data without any audio pretraining? In our #CVPR2023 paper "Vision Transformers are Parameter-Efficient Audio-Visual Learners," we show that they can! yanbo.ml/project_page/LAVISH… w/ @yblin98 @yilin_sung @jayleicn @mohitban47

What makes modern Video-Language (VidL) perform well? Check out our #CVPR2023 paper "VindLU: A Recipe for Effective Video-and-Language Pretraining" where we demystify the most critical factors in the VidL model design. klauscc.github.io/vindlu.htm… @fncheng2333 @jayleicn @mohitban47

What is the value of knowledge in the era of large-scale pretraining? Welcome to our #AAAI23 tutorial "Knowledge-Driven Vision-Language Pretraining" with @Xudong_Lin_AI @jayleicn @mohitban47 @Shih-Fu Chang @elgreco_winter Feb 8: 2-6pm Loc: Room 201 Zoom: underline.io/events/389/sess…

Self-attention for VL tasks (esp. video text) is too expensive! Check out our #WACV2023 paper “Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention” arxiv.org/abs/2211.11701 github.com/zinengtang/Percei… @ZinengTang* @jmin__cho* @jayleicn @mohitban47 🧵

Do you still get Out-of-Memory error even when you've saved >95% params w. adapter/prompt-tuning? Try Ladder Side-Tuning (LST 🪜) for both Parameter & Memory Efficient Transfer Learning (in NLP VL tasks)! arxiv.org/abs/2206.06522 github.com/ylsung/Ladder-Sid… @jmin__cho @mohitban47 🧵

Can GPT-3 understand videos? Glad to share our new work VidIL on prompting LLMs to understand videos using image descriptors (frame caption visual token). We show strong few-shot video-to-text generation ability WITHOUT the need to train on ANY videos: arxiv.org/abs/2205.10747

TVLT: Textless Vision-Language Transformer abs: arxiv.org/abs/2209.14156 github: github.com/zinengtang/TVLT

🥳🥳 Check out our #ECCV2022 oral paper. We propose ECLIPSE 🌒 that integrates audio🔊🎵 into popular CLIP to have 2.92x faster and 2.34x memory-efficient for long-range video retrieval. arxiv.org/abs/2204.02874 yanbo.ml/project_page/eclips… w. @jayleicn @mohitban47 @gberta227 🧵👇

Interested in Vision Language Pre-training (VLP) but do not know where to start? Hard to track the rapid progress in VLP? Come and join us at our CVPR2022 VLP tutorial on 19th Jun (9am-5pm CDT) in person in New Orleans or virtually. vlp-tutorial.github.io #CVPR2022