🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

✨New Report✨ Our data ecosystem audit across text, speech, and video (✏️,📢,📽️) finds: 📈 Rising reliance on web, synthetic, and YouTube data. 🛑 80%+ datasets carry hidden restrictions. 🌍 Relative representation in languages and creators has not improved for 10+ yrs.

🎉 Thrilled to share that our paper "Surprising effectiveness of pretraining ternary language models at scale" earned a spotlight at #ICLR2024! We dive into Ternary Language Models (TriLMs), systematically studying their training feasibility and scaling laws against FloatLMs.

Worried about memory bottlenecks and slow inference? Need small but mighty LLMs on edge devices? Train ternary LLMs - you'll get the best trade-off between performance vs memory (+ significant inference speed-up), as compared to full-precison models and post-training

How do MoE transformers, like DeepSeek, behave under distribution shifts? Do their routers collapse? Can they still match full re-training performance? Excited to present “Continual Pre-training of MoEs: How robust is your router?”!🧵arxiv.org/abs/2503.05029 1/N

🚀 Really excited to launch #AgentX competition hosted by UC Berkeley RDI UC Berkeley alongside our LLM Agents MOOC series (a global community of 22k+ learners & growing fast). Whether you're building the next disruptive AI startup or pushing the research frontier, AgentX is your

1/  Most RL methods assumes a turn-based setup-- agent acts, environment responds. But in the real world, the environment doesn’t wait. In real-time RL, slow inference means missed actions or delayed ones. This leads to two key challenges:  • Inaction Regret  • Delay Regret

I’m really excited about Diffusion Steering Lens, an intuitive and elegant new “logit lens” technique for decoding the attention and MLP blocks of vision transformers! Vision is much more expressive than language, so some new mech interp rules apply:

📢 Come say hi at our SFM poster at #ICLR2025, Poster Session 5 – #572! We’re presenting a method for Inverse Reinforcement Learning via Successor Feature Matching — a non-adversarial approach that works without action labels. Excited to share and chat!

🚀Excited to present a simple and scalable RL framework for multi-turn code generation with one-step recoverability and learned verifiers. Come say hi at our poster at VerifAI and SSI-FM workshops today, and Reasoning and Planning workshop tomorrow.

Our Spectra paper on efficient quantized LLMs was presented at ICLR (spotlight and poster) by the amazing Ayush Kaushal and Tejas Vaidhya (Mila - Institut québécois d'IA / Nolano.ai ) - great job, and so much more to build next, for any data modalities (work in progress)! Welcome to the world of

Context is Key🗝️ is accepted at ICML 2025! 📈 Let's catch up if you'll be at ICML 🛬 See the poster and tweet thread below for a preview of CiK 👇 x.com/arjunashok37/s… And stay tuned for new results ;)

totally! I love vibe-coding, the efficiency is unreal

Quite impressive!

Grateful for the IVADO Exploratory Grant with @IrinaRish & Tommaso Tosato on how #LLMs express personality traits & socio-emotional responses—toward safer #AI in Health & Education ivado.ca/en/2025/04/09/…

IVADO Irina Rish Tommaso Tosato You can already check our recent works on this topic: - LLMs and Personalities: Inconsistencies Across Scales openreview.net/forum?id=vBg3O… - Lost in Translation: The Algorithmic Gap Between LMs and the Brain arxiv.org/abs/2407.04680

MatFormers are very powerful alternatives to transformers. Similar to a regular transformer, but after training, you can split up the model to any size you like and get very strong performance that scales just like a regular transformer. So train once, get models of all sizes!

Huh. Looks like Plato was right. A new paper shows all language models converge on the same "universal geometry" of meaning. Researchers can translate between ANY model's embeddings without seeing the original text. Implications for philosophy and vector databases alike.

🥳Nice! Our project “Towards a Quantum #NeuroIA” just got seed funding from @ai_UNIQUE! After a year in stealth w/Annemarie Wolff, our benchmarks show #quantum speedups for brain data simulation & analysis using Qiskit + IBM QS1 —> Next: #OpenSource tools & intl. collab 🇯🇵🔄🇨🇦

MuLoCo: Muon x DiLoCo = ❤️ arxiv.org/abs/2505.23725 from Benjamin Thérien, Xiaolong Huang, Irina Rish, Eugene Belilovsky * Using Muon as inner optimizer * Add quantization of the outer gradient to 2 bits (!) * Add error feedback

🤨 chatgpt.com/share/683ce589…