🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Vijay Karunamurthy isn't just in AI; he's shaping it. 🤖 As Field CTO at Scale AI, he's instrumental in advancing AI across industries. This former Google and YouTube engineer (think personalized human interfaces!) co-founded AVOS Systems and now leads the charge in data labeling,

New SEAL Leaderboard: EnigmaEval, a benchmark assessing models’ puzzle-solving capabilities. o1 leads. scl.ai/enigma-eval EnigmaEval is part of a new class of extremely challenging benchmarks that expose current models' limitations. The benchmark comprises 1184 puzzles

Happy 20th birthday, YouTube. How you've grown...!

Great talking with Summer Yue and Dan Hendrycks about Humanity’s Last Exam, and pushing the frontiers of model evaluation, reasoning and calibration.

Great talking to Alex Kantrowitz on the Big Technology substack, on the challenge ahead for building human-level AI for the real economy. bigtechnology.com/p/scale-ai-cto…

🚨 Gemini 2.5 Pro Exp dropped and it's now #1 across SEAL leaderboards: 🥇 Humanity’s Last Exam 🥇 VISTA (multimodal) 🥇 (tie) Tool Use 🥇 (tie) MultiChallenge (multi-turn) 🥉 (tie) Enigma (puzzles) Congrats to Demis Hassabis Sundar Pichai & team! 🔗 scale.com/leaderboard

We're happy to have Scale AI be a gold sponsor of the CMU agent workshop! The workshop is coming up soon 4/10-11, so please sign up if you're interested in agents and can come to the workshop: cmu-agent-workshop.github.io

🎉 Excited to share more about the work we’ve been doing at Scale AI to help AI labs: ✅ evaluate model performance ✅ analyze weaknesses ✅ drive targeted improvements Thank you to Will Knight WIRED for covering!

Neat to see a very ‘timely’ idea (optionality and time scales in RL) - published 25 years ago.

Excited to be back on campus for the UN AI for Good law track with DLA Piper!

If a model lies when pressured—it’s not ready for AGI. The new MASK leaderboard is live. Built on the private split of our open-source honesty benchmark (w/ Center for AI Safety), it tests whether models lie under pressure—even when they know better. 📊 Leaderboard:

“The five-year deal will include creating an AI personalized learning platform and AI teacher assistant.. and work with governments in Asia and Europe could account for a significant piece of sales.” Overview of our work empowering global citizens: cnbc.com/2025/04/16/sca…

We're releasing a multimodal benchmark for troubleshooting complex virology protocols. Expert virologists score only 22.1%, even with internet access and answering questions in their subdomains of expertise. Frontier LLMs score up to 43.8%.

Navigating the agent landscape as an enterprise is tricky as it grows more complex. Enterprise agents need more than reasoning—they need precision, feedback loops, and to live inside well-designed products

"Remember what Illinois has already taught you, take those calculated risks, question your assumptions, and pivot without fear when reality doesn't match the plan." - From Steve Chen's 2025 commencement speech ▶️ bit.ly/43Wdjrt

Incredible getting a tour of the Simons Institute for the Theory of Computing at Berkeley this morning - new candidate for a transmon qubit (in a Cal enclosure!)

“How do we create the right virtual environments for agents to act in… so we know when to intervene [and] escalatory thresholds.” Great deep-dive with Scale AI ‘s Christina Knight on Lawfare Daily! podcasts.apple.com/us/podcast/the…

Fdr and Exec Director Jamil N. Jaffer spoke on a panel at the World Economic Forum Technology Retreat 2025 titled “AI and the Privatization of Sovereignty.” Mod: Cathy Li of Centre for AI Excellence, WEF Co-Panellists: Eileen Donahoe of Global Digital Policy Incubator; Vijay Karunamurthy of Scale AI;

Thank you Alexandr Wang

New Scale AI research in collaboration with Anthropic introduces SHADE-Arena, a benchmark to test for AI sabotage. SHADE-Arena evaluates an AI agent's ability to complete a task while secretly pursuing a harmful objective, all while being watched by an AI monitor. 🧵