How do we generate videos on the scale of minutes, without drifting or forgetting about the historical context? We introduce Mixture of Contexts. Every minute-long video below is the direct output of our model in a single pass, with no post-processing, stitching, or editing. 1/4

Glad to share Seaweed-7B, a cost-effective foundation model for video generation. Our tech report highlights the key designs that significantly improve compute efficiency and performance given limited resources, achieving comparable quality against other industry-level models. To unleash the power of the foundation model, Seaweed-7B further enables a wide range of downstream applications including image-to-video generation, human video generation, subject-consistent video generation, video-audio joint generation, long video generation and storytelling, real-time generation, super-resolution generation, camera controlled generation. Check out our webpage and report for more details: Webpage: seaweed.video/ Paper: seaweed.video/seaweed.pdf It's a wonderful journey of the last year. Thanks to all teammates for their contributions, sincerely.

Synthetic Video Enhances Physical Fidelity in Video Synthesis A turtle swimming in a green background. video matting illustration

We propose Long Context Tuning (LCT) for scene-level video generation to bridge the gap between current single-shot generation and real-world narrative video productions. Homepage: guoyww.github.io/projects/lo… Report: arxiv.org/abs/2503.10589

Want the deep dive? • arXiv: arxiv.org/abs/2501.06173 • Project Page: videoauteur.github.io See how VideoAuteur CookGen are shaping long narrative video generation. Big shout out to my co-authors and advisors: @fncheng2333 @liangkegui @YuilleAlan @roadjiang

Seaweed APT Diffusion Adversarial Post-Training for One-Step Video Generation Existing diffusion and autoregressive generative models require repeated neural network evaluations. It is extremely slow for the high-resolution video generation task, as a few-second video can take many minutes to generate. Our work is the first to demonstrate the generation of an entire video using a single neural function evaluation (1NFE) by using our proposed adversarial post-training technique. Our model generates 2 seconds of 1280x720 24fps videos in real-time. We showcase some of the results below:

Interesting comparison between our VideoPoet and other competitive models. The comparison is incredibly helpful and reinforces my belief that VideoPoet excels in generating larger motions. We know the exact reasons for this and are working on improving single frame quality.

Excited to be at #NeurIPS2023 this week! Can't wait to reconnect with colleagues and make new connections. If you're up for a coffee chat, feel free to reach out. Find me at our spotlight/posters. arxiv.org/abs/2306.17842 Tue 12 5:15 p.m. arxiv.org/abs/2306.00983 Wed 13 10:45 a.m

We introduce W.A.L.T, a diffusion model for photorealistic video generation. Our model is a transformer trained on image and video generation in a shared latent space. 🧵👇

📢 Call for Papers! International Journal of Computer Vision (IJCV) invites submissions for its special issue on "Generative Models for Content Creation and Manipulation." 🗓️ Manuscript Submission Deadline: February 28, 2024 🔗 Check it out here: springer.com/journal/11263/u…