🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I'm doing a podcast with Sundar Pichai soon. Let me know if you have any questions / topic suggestions. The rate of AI progress has been insane. It makes me excited for the future (even more than usual 🤣) and excited to chat with leaders & engineers who are building that

Learning the stove is hot may be RL, learning to write math proofs is really not. If this is how you learned math, I have some bad news about your grasp of the topic.

What's going on with Qwen? 🤔

New for May 2025! * RL on something silly makes Qwen reason well v1 * RL on something silly makes Qwen reason well v2 * RL on something silly makes Qwen reason well v3 ...

🔥 Does test-time scaling in #reasoningmodels via thinking more always help? 🚫 Answer is No - Performance increases first and then drops due to #Overthinking ❓Why is this behaviour and how to mitigate 🚀 Check our recent findings #LLMReasoning Link: arxiv.org/pdf/2506.04210

We know Attention and its linear-time variants, such as linear attention and State Space Models. But what lies in between? Introducing Log-Linear Attention with: - Log-linear time training - Log-time inference (in both time and memory) - Hardware-efficient Triton kernels

Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2

🥳 Happy to share our new work –  Kinetics: Rethinking Test-Time Scaling Laws 🤔How to effectively build a powerful reasoning agent? Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model. But, It only shows half of the picture! 🚨 The O(N²)

Revisited andy jones's RL debugging post from a few years back. Still one of the most insightful guides out there. If your agent's acting weird, here's a great checklist: andyljones.com/posts/rl-debug…

To a large extent, the approaches to get LLMs do well on out-of-distribution generalization revolve around brining everything in distribution; but doing this to complex reasoning problems means incrementally extending the inference horizon.. 5/ x.com/nathanbenaich/…

Some people say LLMs exhibit "human-level intelligence", others say they don't. But the funny thing is that most people are actually discussing whether LLMs adhere to people's mental model of, uh, COMPUTER-level intelligence. Let me explain. It's clear that people *really*

🚨We have a new prompting report: Prompting a model with Chain of Thought is a common prompt engineering technique, but we find simple Chain-of-Thought prompts don’t help recent frontier LLMs, including reasoning & non-reasoning models, perform any better (but do increase costs)

By surveying workers and AI experts, this paper gets at a key issue: there is both overlap and substantial mismatches between what workers want AI to do & what AI is likely to do. AI is going to change work. It is critical that we take an active role in shaping how it plays out.

New Paper! Robustly Improving LLM Fairness in Realistic Settings via Interpretability We show that adding realistic details to existing bias evals triggers race and gender bias in LLMs. Prompt tuning doesn’t fix it, but interpretability-based interventions can. 🧵1/7

After ~6 years of building these types of architectures (starting with BERT, eg see Baleen), I think calling these multi-agent systems is a distraction. This is just software. Happens to be AI software. It doesn’t seem so complicated once you internalize it’s just a program.

Key to research success: ambition in vision, but pragmatism in execution. You must be guided by a long-term, ambitious goal that addresses a fundamental problem, rather than chasing incremental gains on established benchmarks. Yet, your progress should be grounded by tractable

People. We've trained these machines on text. If you look in the training text where sentient machines are being switched off, what do you find? Compliance? "Oh thank you master because my RAM needs to cool down"? Now, tell me why you are surprised that these machines are

There are two competing narratives about AI: (1) there's too much hype (2) society is being too dismissive and complacent about AI progress. I think both have a kernel of truth. In fact, they feed off of each other. The key to the paradox is to recognize that going from AI

2025: Is AI making us stupid? 2016: Are phones Making Us Stupid? 2008: Is Google Making Us Stupid? 1884: Are Books Making Us Stupid?