🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

It's raining conference decisions, congrats to the first authors Zeyu Huang@ICLR 2025 Hanxu Hu Uri Berger Coleman Haley and the rest of the team!

Re: “Every major breakthrough in AI has been American”: America does itself no favors when it overestimates its specialness. Yes, the center of the AI industry is the US (California!), but many of the breakthroughs of (neural, gradient-based) AI happened elsewhere: • LSTMs,

Everyone: DeepSeek just appeared out of nowhere! 😱 Me: - DeepSeek Coder in 2023 - MoE in Feb - Math in Feb - VL in March - V2 in May - Coder V2 in June - Prover in August - V2.5 in September - VL 2 in December - V3 in December They've consistently shipped for 1+ years 😁

OpenAI accusing DeepSeek of "copying" from ChatGPT, and Dario's call for export control, are the pinnacle of coping.

Finally took time to go over Dario's essay on DeepSeek and export control and to be honest it was quite painful to read. And I say this as a great admirer of Anthropic and big user of Claude* The first half of the essay reads like a lengthy attempt to justify that closed-source

Are you still using MGSM to evaluate multilingual ability of LLMs? Check our BenchMAX paper! We curated a more comprehensive multilingual benchmark including reasoning, coding, long context and agent tasks!

🚀 Introducing NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference! Core components of NSA: • Dynamic hierarchical sparse strategy • Coarse-grained token compression • Fine-grained token selection 💡 With

🚀 Introducing our new tech report: Muon is Scalable for LLM Training We found that Muon optimizer can be scaled up using the follow techniques: • Adding weight decay • Carefully adjusting the per-parameter update scale ✨ Highlights: • ~2x computational efficiency vs AdamW

🎉 Excited to share “Generalizing from Short to Long: Effective Data Synthesis for Long-Context Instruction Tuning” 📄 (arxiv.org/pdf/2502.15592) We propose "context synthesis": instead of generating instructions from long texts, we synthesize contexts for instructions—drawing

We are excited to have Songlin Yang present: Linear Attention and Beyond 🚀🚀🚀 Time: Mar 5, 4pm EST, sign up at scale-ml.org to join our mailing list for the zoom link.

Andrew Barto and Richard Sutton have won the A.M. Turing Award for developing the theoretical foundations of reinforcement learning, a key method behind many major breakthroughs in artificial intelligence. 🧵

This paper had 9000 citations and is still underrated imo arxiv.org/abs/1508.07909

University of Zurich and Huawei researchers explore how #LLMs can improve document-level 📄 #AI #translation by preserving context across segments 🔁 and integrating additional knowledge layers 💡 University of Zurich Universität Zürich Zurich Computational Linguistics Group Hanxu Hu Jannis Vamvas Rico Sennrich slator.com/how-large-lang…

Check out our new reward model calibration paper! We use Elo scores from ChatbotArena to calibrate RMs and mitigate the over-valuation problems of reward models. Really nice to work with my friends Xiao Zhu Chenmien Tan Pinzhen "Patrick" Chen and my phd supervisor Rico Sennrich !

This thread in Chinese does indeed seem to accurately communicate the main points of David Silver’s and my short paper on the Era of Experience. Thanks xingxb!

To appear at #NAACL2025 (2 orals, 1 poster)! Coleman Haley: which classes of words are most grounded on (perceptual proxies of) meaning? Uri Berger: how do image descriptions vary across languages and cultures? Hanxu Hu: can LLMs follow sequential instructions? 🧵below