🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

When scalable oversight techniques like debate show empirical success, what additional evidence will we need to ensure the resulting models are aligned? We work through the details determining what we need to assume about deployment context, training data, training dynamics, and

Want to build an aligned ASI? Our new paper explains how to do that, using debate. Tl;dr: Debate + exploration guarantees + no obfuscated arguments + good human input = outer alignment Outer alignment + online training = inner alignment* * sufficient for low-stakes contexts

We wrote out a very speculative safety case sketch for low-stakes alignment, based on safe-but-intractable computations using humans, scalable oversight, and learning theory + exploration guarantees. It does not work yet; the goal is to find and clarify alignment subproblems. 🧵

More work from my team on alignment safety cases!

New work from my colleagues! We want AIs to do open-ended research tasks with no single right answer. How do we make sure AIs don't use that freedom to subtly mislead or cause harm? The proposal: Check that the answers are random along relevant dimensions. V cool work!

Come work with me!! I'm hiring a research manager for AI Security Institute's Alignment Team. You'll manage exceptional researchers tackling one of humanity’s biggest challenges. Our mission: ensure we have ways to make superhuman AI safe before it poses critical risks. 1/4

More safety case sketches!

Do you know cognitive scientists / folks who run behavioural experiments with human participants? Refer them to join my team!