🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Today, we are introducing SmolVLA: a 450M open-source vision-language action model. Best-in-class performance and inference speed! And the best part? We trained it using all the open-source LeRobot datasets in the Hugging Face hub! But how? 🫳🏀

Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2

Very happy to have played a (very small) part in the release of this very large fully open dataset. We finally have an answer to the question: "how good a model can we get with fully permissible data"? Turns out, not bad at all

Finally had a bit of time to jot down some thoughts on this solid, open data engineering work from Essential AI. This work brings Essential-Web, a 24T-token pre-training corpus, to the open-source community. I've always appreciated open-source research, as it can significantly