🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

The Diffusion Duality "The arg max operation transforms Gaussian diffusion into Uniform-state diffusion" Adapts consistency distillation to diffusion language models, unlocking few-step generation by accelerating sampling by two orders of magnitude. Introduces a curriculum

the extra pages when you request a buffer from CUDACachingAllocator that is two pages smaller than the largest free cached block and you didn’t set PYTORCH_CUDA_ALLOC_CONF=“expandable_segments:True”

Check out our recent paper on the "duality" between discrete and Gaussian diffusion. We show how you can exploit that relationship to massively speed up discrete diffusion by two orders of magnitude.

Our computer vision textbook is now available for free online here: visionbook.mit.edu We are working on adding some interactive components like search and (beta) integration with LLMs. Hope this is useful and feel free to submit Github issues to help us improve the text!

The Diffusion Duality Author's Explanation: x.com/ssahoo_/status… Overview: Duo narrows the performance gap of uniform-state discrete diffusion models for text generation by leveraging their inherent connection to underlying Gaussian diffusion. This method introduces a

Interesting blog fabianfalck.com/posts/spectral…

The Diffusion Duality unlock few-step generation in discrete diffusion language models via the underlying Gaussian diffusion

The Python Steering Council has voted to remove the "experimental" label from the free-threaded ("nogil") builds for Python 3.14. Big step towards making them the default in a future version of CPython!

Yuchen Jin interestingly, the same was said about compilers in the 1950s. compilers were considered "cheating"

sheesh if anyone wants some VC funded, unsustainable B200s, I found some for $1.50 an hour

elie I'm not 100% sure about that. As an example I was just browsing through the DCLM-baseline datamix (which is ~SOTA) and it is *terrible*. Compared to what I could in principle imagine. Major concessions are made in data quality to gather enough data quantity.

Don't trust your lying eyes! The DCLM paper has a great section on human-model quality assessment alignment that shows that humans are pretty bad at assessing data quality arxiv.org/pdf/2406.11794…. I'm often surprised at what makes it into our good vs. rejected token piles

Checkout out "The Diffusion Duality" on HF papers! huggingface.co/papers/2506.10… Also see the author's collection: huggingface.co/collections/s-…

Thrilled to collaborate on the launch of 📚 CommonPile v0.1 📚 ! Introducing the largest openly-licensed LLM pretraining corpus (8 TB), led by Nikhil Kandpal Brian Lester Colin Raffel. 📜: arxiv.org/pdf/2506.05209 📚🤖 Data & models: huggingface.co/common-pile 1/

Laude Institute is a non-profit that gives the right resources to the right researchers at the right time. We help more researchers go from idea to impact. laude.org

TIL localhost can have subdomains

The Bitter Lesson is coming for tokenization too: lucalp.dev/bitter-lesson-…

As I'm diving into Sequence/Context parallelism in the last few days I wanted to share this write up in 2 parts that nicely compares the few approaches out there and some of their combinations with papers: p1: insujang.github.io/2024-01-11/ten… p2: insujang.github.io/2024-09-20/int…

We have finally released the 📝paper for 🥂FineWeb2, our large multilingual pre-training dataset. Along with general (and exhaustive) multilingual work, we introduce a concept that can also improve English performance: deduplication-based upsampling, which we call rehydration.