🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

📢 Today, we release #MMLUProX, which upgrades MMLU-Pro to 29 languages across 14 disciplines—11,829 reasoning-heavy Qs per language (≈342 k total). The toughest multilingual stress test for today’s LLMs! 🌐🧠 Heartfelt thanks to everyone who contributed.🤝

We are proud to invite applications for a new Google DeepMind Academic Fellowship, hosted by the Fleming Initiative at Imperial College London (Imperial College London), that supports groundbreaking research at the intersection of AMR and AI ⤵️ imperial.ac.uk/news/264673/fl…

🚨 New Paper Alert 🚨 We found that Supervised Fine-tuning on ONE problem can achieve similar performance gain as RL on ONE problem with 20x less compute! Paper: arxiv.org/abs/2506.03295 Recently, people have shown that RL can work even with ONE example. This indicates that the

After over 2 years of work, I'm glad to be finally wrapping up NaijaVoices with our accepted paper at #Interspeech 25! We created 1800+ hours of multilingual speech data in Hausa, Igbo, & Yoruba across 100 themes with a community we built entirely from scratch (see paper)!

🚨 New for May 2025: Highly irresponsible reviewers/ACs may become ineligible to commit papers to EMNLP/ARR next cycle. ❗️Reviewers must follow guidelines & deadlines. ❗️Chairs must be notified for emergency. ❗️Do not share with third parties such as commercial LLM services.

If you want to learn more about how culturally inclusive current vision-language models are and what the outstanding research questions in this area are, do stop by our VLMs4All - CVPR 2025 Workshop workshop on June 12th -- sites.google.com/corp/view/vlms…

Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2

Do LLMs hallucinate randomly? Not quite. Our #ACL2025 (Main) paper shows that hallucinations under irrelevant contexts follow a systematic failure mode — revealing how LLMs generalize using abstract classes + context cues, albeit unreliably. 📎 Paper: arxiv.org/abs/2505.22630 1/n

Today we’re announcing Sarvam-Translate, an open-weights model that translates text across 22 Indian languages, with support for long-form text and the ability to handle diverse formats, contexts, and styles. Sarvam-Translate stands out for its ability to handle the complexities

We know speech LID systems flunk on accented speech. But why? And what to do about it?🤔Our work arxiv.org/abs/2506.00628 (Interspeech '25) finds that *accent-language confusion* is an important culprit, ties it to the length of feature that a model relies on, and proposes a fix.

Really excited about our recent large collaboration work on NLP for Social Good. The work stems from our discussions at the NLP for Positive Impact Workshop (EMNLP 2024) Workshop at #EMNLP2024 EMNLP 2025. Thanks to all our awesome collaborators, workshop attendees and all supporters!

Where does one language model outperform the other? We examine this from first principles, performing unsupervised discovery of "abilities" that one model has and the other does not. Results show interesting differences between model classes, sizes and pre-/post-training.

Truly excellent video by Machine Learning Street Talk about how a handful of providers have systematically overfit to lmarena.ai. 26 mins of video showcase how easy it has been to distort the rankings. As scientists, we must do better. As a community, I hope we can demand better.

Our CTO #DataScience Speaker Series welcomes McGill University's David Adelani (David Ifeoluwa Adelani 🇳🇬) to our Engineering office in NYC to talk with our #AI research engineers about scaling multilingual evaluation of #LLMs to many languages bloom.bg/3ZGFIAf #GenAI #NLProc

🚀 Help shape African language tech! Take a quick 10-15 min survey by DSFSI's to build fair, community-driven machine translation & small language models for African languages. Your voice matters! 🌍🗣️ 👉 forms.gle/NrDg7kn3jqs3dt… cc Masakhane #AfricanLanguages

🚩 💬 We're running the NADI 2025 shared task, focused on Multidialectal Arabic Speech Processing. Welcoming y'all! #NADI2025 #ArabicSpeech #ASR #ArabicNLP

How can we make language models more flexible to adapt to new languages after pretraining? 🌏 🧠 Our latest work investigates whether a tokenizer trained on more languages than the pretraining target can improve language plasticity without compromising pretraining performance.

🚨 New Paper 🧵 How effectively do reasoning models reevaluate their thought? We find that: - Models excel at identifying unhelpful thoughts but struggle to recover from them - Smaller models can be more robust - Self-reevaluation ability is far from true meta-cognitive awareness

The NaijaVoices Dataset (accepted to Interspeech 2025) arXiv link: arxiv.org/abs/2505.20564 video overview: supabase.manatee.work/storage/v1/obj…