Exciting work from Google on one of my pet ideas: learning local learning rules!

High dimensional, multi-modal probabilistic scene prediction has fascinated me for a long time and I'm excited to share our first attempt to model this problem. A critical component for autonomous driving and model-based reinforcement learning.

Super impressive use of CPPNs! Each scene takes about 1M params - pretty compact encoding. I also suspect that the optimisation can be extended to tune the camera intrinsics and positions as well!

It takes a lot of clicking to label images with manual tools, especially for semantic and instance segmentation. This is how we make it easy and fast 👇 ✨ Try it yourself! segments.ai/join

Moving away from negative pairs in self-supervised representation learning: our new SotA method, Bootstrap Your Own Latent (BYOL), narrows the gap between self-supervised & supervised methods simply by predicting previous versions of itself. See here: bit.ly/30MCugQ

NTK analysis on regression with low-dim. input (e.g., position to color) shows that standard MLP fails to learn high-frequency content. Fourier features (PE in NeRF) can learn this because it enables us to use a stationary kernel with tunable bandwidth. people.eecs.berkeley.edu/~bmild/fourfea…

New paper on diffusion probabilistic models with @ajayjain318 & Pieter Abbeel: hojonathanho.github.io/diffusion/ Likelihood-based generative model with SOTA FID=3.17 on unconditional CIFAR10, and ProgressiveGAN-like quality on 256x256 LSUN & CelebA-HQ (sometimes generates dataset watermarks!)

How can you speed up your labeling workflow? ⚡️ One way is to leverage ML models from the start: label a small set of images, train a draft model, predict more labels, and correct them. This is super easy to set up with Segments.ai: segments.ai/blog/speed-up-…

We have updated our survey on multi-task learning for dense prediction tasks. The paper features an extensive literature review, intuitive comparisons, thorough experiments, etc. arxiv.org/abs/2004.13379 Code can be found here 🥳: github.com/SimonVandenhen…

I made an overview of public datasets for panoptic segmentation, so others can make pretty predictions like this too ✨ segments.ai/blog/panoptic-…

Shipped my first ML feature for Segments.ai today🚀 Autosegment makes labeling images easier by automatically creating a segmentation mask within a bounding box. Made possible by Hugging Face behind the scenes🤗 Thx Niels Rogge for the useful notebooks + implementations

In our new guide, @nielsrogge and I show how you can use Hugging Face and @segmentsai to fine-tune SegFormer on your own data. We used it to build a semantic segmentation model for a pizza delivery robot🍕🤖 📰 Blog: huggingface.co/blog/fine-tune… 📒Notebook: colab.research.google.com/drive/1BImTyBj…

Labeling point clouds for ML is very tedious & time consuming. Luckily, you can use model predictions to speed up the annotation. Check out my new guide for Segments.ai where I explain how to do so! ➡️ segments.ai/blog/fast-poin…

Niels Rogge and I wrote a blog post on zero-shot image segmentation for Hugging Face. The blogpost shows how you can use CLIPSeg with 🤗Transformers and upload the predictions to Segments.ai 📰 Blog: huggingface.co/blog/clipseg-z… 📒 Notebook: colab.research.google.com/github/hugging…

🎄🤖 Right before Christmas, OpenAI came out with its Point-E model. We had some fun with it during the holidays and integrated it in a Segments.ai workflow. Read with me, or give it a run for its money yourself!

I don't share a lot of web dev work on here, but check out these cool image viewers I made with Three.js! The first viewer follows your pointer in the 3D scene. The second is locked to one image, and allows you to inspect the image + 3D overlay.

The Segment Anything Model (SAM) from @MetaAI is absolutely wonderful. We've now added it as an edit mode to Segments.ai! segments.ai/blog/faster-la…

Zero-shot panoptic segmentation with SAM is here! This week, I developed a pipeline using Segment Anything, Grounding DINO, and CLIPSeg. Learn how in the thread 👇 Gradio demo: huggingface.co/spaces/segment… Project page: github.com/segments-ai/pa… Colab: colab.research.google.com/github/segment…

Today is #ICRA2023's workshop on Scalable Autonomous Driving. I'm looking forward to speaking alongside an all-star cast of Yuning Chai, Yin Zhou, Bert De Brabandere, Adrien Gaidon, Fisher Yu, Alexander Amini , Anca Dragan, Rowan McAllister, @kashyap7x, and Marco Pavone!

My first Three.js tutorial is live! It explains how to simulate a real camera in Three.js in order to overlay a 3D scene on a camera image. I cover both the basic pinhole model and how you can simulate lens distortion with a post-processing shader. segments.ai/blog/simulatin…