🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

TLDR: You can get far with: vanilla Transformer (2017). Scrape a massive (though weakly-labeled) dataset, use simple supervised learning. Multi-task. Eval in zero-shot regime. More perf expected from further model+data scaling. Eval is hard. Some parts (decoding) feel hacky.

10/ We are releasing our model (huggingface.co/neeva/query2qu…) and golden set used for eval (huggingface.co/datasets/neeva…) on Hugging Face. Take a look at our latest blog post for more information 👁️ ⤵️ neeva.com/blog/state-of-…

This particular example that generates a 2min long video based on a changing story is really cool Congrats to all the authors!

🪩The State of AI 2022 is live!🪩 In its 5th year, the #stateofai report condenses what you *need* to know in AI research, industry, safety, and politics. This open-access report is our contribution to the AI ecosystem. Here's my director's cut 🧵: stateof.ai

We are excited to announce Essential AI, founded by Ashish Vaswani and niki parmar essential.ai

Honored to have been awarded the 2024 NEC C&C Prize together with my wonderful friends and Transformer coauthors Ashish Vaswani Noam Shazeer niki parmar Illia (root.near) (🇺🇦, ⋈) Command A(idan) Jakob Uszkoreit Llion Jones Learn more: nec.com/en/press/20241…

Today is as good a day as any to share that I joined Anthropic last Dec :) Claude 3.7 is a remarkable model at complex tasks, especially coding, and I'm thrilled to have contributed to its development. From winning Pokémon badges to vibes coding, Claude's got you covered!

I love ClaudeCode!! ❤️ Talking to it and watching it go is oddly satisfying.

(1/11) Evolutionary biology offers powerful lens into Transformers learning dynamics! Two learning modes in Transformers (in-weights & in-context) mirror adaptive strategies in evolution. Crucially, environmental predictability shapes both systems similarly.

Claude Opus 4 and Sonnet 4 are the best coding models, setting new records across the board. 🚀 We are pushing the limits (80.2% on SWE-Bench!!), advancing the frontier while keeping up the momentum. The benchmarks may soon become saturated but the capabilities will not!

Excited to share what I worked on during my time at Meta. - We introduce a Triton-accelerated Transformer with *2-simplicial attention*—a tri-linear generalization of dot-product attention - We show how to adapt RoPE to tri-linear forms - We show 2-simplicial attention scales