Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think discuss: huggingface.co/papers/2409.11… Recent work showed that large diffusion models can be reused as highly precise monocular depth estimators by casting depth estimation as an image-conditional image

Recent work showed that large diffusion models can be reused as highly precise monocular depth estimators by casting depth estimation as an image-conditional image generation task. This paper shows that the perceived inefficiency was caused by a flaw in the inference pipeline

Check out our work on fine-tuning of image-conditional diffusion models for depth and normal estimation. Widely used diffusion models can be improved with single-step inference and task-specific fine-tuning, allowing us to gain better accuracy while being 200x faster!⚡ 🧵(1/6)

Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think Gonzalo Martin Garcia Karim Abou Zeid Christian Schmidt Daan de Geus Alexander Hermans Bastian Leibe tl;dr: there is a bug in Marigold noise DDIM scheduler, if you fix ot, you can do 1 step depth estimation. arxiv.org/abs/2409.11355

Make things simpler and get sota, what's there not to like?

This is the most exciting paper I've read in a while. Alternate title could have been: "One weird trick to increase your depth map inference 200x." arxiv: arxiv.org/abs/2409.11355 github: github.com/VisualComputin… Let's go through the details 🧵 1/9

Time to give credit to this paper -- it gets it right! Kudos to the authors. The fix is a proper way to speed up the original Marigold. If you’re not aiming for an end-to-end network from Stable Diffusion, just add one flag to the DDIM scheduler for instant depth predictions in