This is happening today! Will be presenting at 1h30pm CEST, poster #317 Feel free to drop by to say hi

Some thoughts on how to think about "world models" in language models and beyond: lingo.csail.mit.edu/blog/world_mod…

We're looking for a postdoc to work with us in FAIR Montreal AI at Meta. Interested in building generative visual models of the world and leveraging them to train dowsntream ML models? Apply: metacareers.com/jobs/376087892… cc:Melissa Hall Reyhane Askari Jakob Verbeek Michal Drozdzal

I can't believe that no one has done that yet!

Stick-Breaking Attention: Out-of-box length extrapolation, thanks to removing the position embedding; Better performance than Softmax+RoPE on almost every task; Similar efficient implementation like Flash Attention. Do we still need Softmax+RoPE for Language Models?

As I'm heading out of Vancouver, I'm also wrapping up my postdoc at FAIR. It was a fun week with my friends/collabs and I made many great connections. But, hey! I'm still looking for full-time positions. Reach out if you work on multimodal generation/understanding #NeurIPS2024

Happy to share our latest work on #diffusion models without data: building theoretical bridges between existing methods, analysing their continuous-time asymptotics, and showing some cool practical implications. arxiv.org/abs/2501.06148 #MachineLearning 1/9

The code and model weights for this paper are finally open! Despite being a little late for releasing them, I hope you will find them useful! Code: github.com/facebookresear… Models: - (ViT-G): huggingface.co/lavoies/llip-v… - (ViT-B): huggingface.co/lavoies/llip-v…

We're releasing a cool paper! DLCs are image tokens that enable better diffusion modelling. For now, we show this is the right representation. But in the future, this can allow LLMs to "speak in images"🤯to enable visual reasoning and more powerful text-image generalization. ⬇️