🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

We are hiring our first product engineers. Come join an incredible team advancing Human AI interaction! Please check our jobs page for all the open positions jobs.ashbyhq.com/essentialai

🗞️ We just launched our new landing page and dropped a fresh blog post on how LLMs learn to reflect and revise their thinking: In order to advance reasoning, it's vital to measure and understand its constituents, such as reflection. More to come - essential.ai

Please check out our thorough study on the advantages of Muon. Second-order optimization is a promising path to more efficient LLM pretraining.

Cool paper from Essential AI Ashish Vaswani huggingface.co/papers/2505.02…

Super excited to share how we achieved significant performance gains in Muon optimizer for Pre-training LLMs at Essential AI! This was a great team effort — special thanks to Philip Monk , whose core ideas were instrumental in driving this improvement, and to Ashish Vaswani

Terrific work speeding up Muon led by Philip Monk, in collaboration with Yash Vanjani , Ishaan , and Karl Stratos . We’re exited about the benefits we’re seeing with Muon.

💪

I am trying out this Thought-Boi Thing. Give it a read. The Hidden Cost of Augmentation: Every Tool You Use Changes You. open.substack.com/pub/spacemanid…

Excited to share what I worked on during my time at Meta. - We introduce a Triton-accelerated Transformer with *2-simplicial attention*—a tri-linear generalization of dot-product attention - We show how to adapt RoPE to tri-linear forms - We show 2-simplicial attention scales

We just released the best 3B model, 100% open-source, open dataset, architecture details, exact data mixtures and full training recipe including pre-training, mid-training, post-training, and synthetic data generation for everyone to train their own. Let's go open-source AI!