Check out our work on 3D scene relighting! We rendered purely synthetic data and used it to fine-tune a video model for relighting. Even though it was only trained on synthetic data, it generalizes to real captured scenes—shadows, reflections, and all!

[1/6] We introduce GR3EN, a generative approach for relighting 3D environments. It gives you full control over every light source in a 3D scene — enabling adding new light sources, changing the color of existing ones or turning them off entirely. Shadows, reflections, and interreflections all update realistically. Accepted to SIGGRAPH 2026 🔥 Project_page: gr3en-relight.github.io/ From: @GoogleDeepMind @GoogleResearch @UvA_Amsterdam

🎥 What if 3D capture could gracefully handle moving scenes and varying illumination? 🎯Come see how video models generate exactly the data you need at our poster, SimVS! 📍CVPR, June 14th (afternoon), Poster #60.

⚡️ Introducing Bolt3D ⚡️ Bolt3D generates interactive 3D scenes in less than 7 seconds on a single GPU from one or more images. It features a latent diffusion model that *directly* generates 3D Gaussians of seen and unseen regions, without any test time optimization. 🧵👇 (1/9)

🚀 Introducing SimVS: our new method that simplifies 3D capture! 🎯 3D reconstruction assumes consistency—no dynamics or lighting changes—but reality constantly breaks this assumption. ✨ SimVS takes a set of inconsistent images and makes them consistent with a chosen frame.

We’ll be presenting NeRF-Casting at SIGGRAPH Asia next week! NeRF-Casting enables photorealistic rendering of scenes with highly reflective surfaces—something that was previously impossible with models like Zip-NeRF and 3DGS. (1/6)

(1/N) Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering Website: benattal.github.io/flash-cac… tl;dr our #ECCV2024 (oral ✨) paper presents a new system for inverse rendering that is more physically accurate, and therefore less biased, than existing approaches.

I'm going to present our work at the oral session tomorrow (Wednesday), 9am at #CVPR2024. Come check it out and hang out at the poster session (ours is number 399) immediately after!

IllumiNeRF lets you relight objects in 3D. Instead of the classical inverse rendering approach — disentangling the object geometry, materials, and lighting — we use a relighting diffusion model to relight each input image and distill the relit samples into 3D by optimizing a latent NeRF. This project was led by the talented @xmzhao_ who is completing his PhD and is currently on the job market!

Wondering how to easily relight an object? Inverse rendering, maybe the first thing that comes to mind, is brittle and expensive due to differentiable Monte Carlo rendering. Check out IllumiNeRF for simple, effective 3D relighting without it! illuminerf.github.io (1/n)

IllumiNeRF enables relighting without expensive inverse rendering. We use a diffusion model trained to relight a single image, and turn its samples into a consistent 3D relit NeRF. With @xmzhao_ (currently on the job market!) @_pratul_ @KeunhongP @rmbrualla @philipphenzler

IllumiNeRF 3D Relighting without Inverse Rendering Existing methods for relightable view synthesis -- using a set of images of an object under unknown lighting to recover a 3D representation that can be rendered from novel viewpoints under a target illumination

Check out CAT3D! Image(s)-to-3D in 1 minute! cat3d.github.io Given any number of real or generated images, CAT3D uses a multi-view diffusion prior to create consistent novel views. These views are used to reconstruct a 3D scene using NeRF/3DGS.

The cat’s out of the bag! Image(s) to 3D in one minute with a multi-view diffusion model. cat3d.github.io

Videos are cool and all...but everything's more fun when it's interactive. Check out our new project, ✨CAT3D✨, that turns anything (text, image, & more) into interactive 3D scenes! Don't miss the demo!! cat3d.github.io/

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors @holynski_, @poolio and an amazing team!

Scientists, led by a team at Caltech, used AI and telescope data to create the first 3D video of mysterious bright flares around the supermassive black hole at the center of our galaxy. caltech.edu/about/news/ai-an…

We just finished a joint code release for CamP (camp-nerf.github.io/) and Zip-NeRF (jonbarron.info/zipnerf/). As far as I know, this code is SOTA in terms of image quality (but not speed) among all the radiance field techniques out there. Have fun! github.com/jonbarron/camp_zi…

Introducing Eclipse, a method for recovering lighting and materials even from diffuse objects! The key idea is that standard "NeRF-like" data has all we need: a photographer moving around a scene to capture it causes "accidental" lighting variations. dorverbin.github.io/eclipse/ (1/3)

NeRFs are cool, but they're HARD to edit. Turn 'em into a mesh, and the geometry and UV maps are a dumpster fire -- making simple texture editing mission impossible for your 3D artist. Well, not anymore! Google AI's latest paper Nuvo, employs neural fields for UV mapping, letting you edit cleanly parameterized chunks of the model. Benefit? Now you can use 2D editing tools like Photoshop or Firefly in-painting to edit the texture of a NeRF model :) It doesn't even matter if it's a real life scan or a generated one. This technique does a great job of dealing with the chaotic (but good looking) geometry you get out of NeRFs.