🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

stay with me now

Excited to see further studies into early fusion vs late fusion models, in particular a great analysis into multimodal MoE’s aligned with our findings in MoMa on designing parameter specialization in multimodal LLMs. A few key things that helped us on top of the results presented

So excited to see the next batch of models out of Malaysia

turn all dumb matter into computronium

very cool. we found similar results in diffusion model training where EMA on model weights & const LR is more common. section 5.3 arxiv.org/pdf/2405.13218

i think we should stop arguing about what year AGI will arrive and start arguing about what year the first self-replicating spaceship will take off

just do it

4th of July came early to the comma shop! Get a comma 3X for just $799 - its lowest price ever.

thank you chatgpt, very cool

Laker and I are presenting this work in an hour at ICML poster E-2103. It’s on a theoretical framework and language (modula) for optimizers that are fast (like Shampoo) and scalable (like muP). You can think of modula as Muon extended to general layer types and network topologies