🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

While GenAI is fun, I think its economic value is grossly over estimated, because it’s unreliable, risky and expensive to make and serve. It’s fine for creative tasks, but not (yet) autonomous agents

Will be at ICLR 2024 in Vienna 🇦🇹🇦🇹. Come say Hi!

Did you know that the optimizer Sharpness Aware Minimization (SAM) is very robust to heavy label noise, with gains tens of percent above SGD? In our new work, we deep dive into how SAM achieves these gains. As it turns out, it’s not at all about sharpness at convergence!

Driving endless stretches of hwy1 to run thru redwoods, waterfalls, up mtns, and swim in the pacific - complete freedom and the feeling that the future has infinite possibilities that is yours to take. Few things embody the American spirit, the American ethos better. Happy 4th!

Back to Vienna for ICML, Mon evening to Fri evening. DM me if you're around and want to chat. And stop by our poster! arxiv.org/abs/2406.03682 (Tues poster session 2; also at high dim learning workshop on Fri)

Want to watermark the outputs of an LLM but only have 3p API access to it? We propose a "distortion-free" LLM watermarking scheme that only requires a way to sample responses for the prompt. Joint work with John Wieting and others Google DeepMind (arxiv.org/abs/2410.02099).

Doing good science is 90% finding a science buddy to constantly talk to about the project.

Check out our new paper1 We propose a watermarking scheme for LLMs that works with black-box access, is distortion-free, has provable guarentees, and can be chained with secret keys. Able to outperform white-box methods in comprehensive experiments. arxiv.org/abs/2410.02099.

IIRC, This project began as mostly one researcher’s (Alex) interest at Google Brain. Nearly a decade later, it is a full fledged company that has made technology to transport scents digitally. A role model of patient research and dogged pursuit of a mission!

Maksym is one of the best researchers I've had the opportunity to work with!

I'll be at #NeurIPS2024 Wed-Sat. Msg me if you want to chat!

🚨 New Paper Alert: Open Problem in Machine Unlearning for AI Safety 🚨 Can AI truly "forget"? While unlearning promises data removal, controlling emergent capabilities is a inherent challenge. Here's why it matters: 👇 Paper: arxiv.org/pdf/2501.04952 1/8

Really fun work with Richard Song! TLDR; we show that sequence modeling of numeric strings can be a powerful paradigm for tasks like regression and density estimation.

"Mountains are not stadiums where I satisfy my ambition to achieve, they are the cathedrals where I practice my religion....I go to them as humans go to worship. From their lofty summits I view my past, dream of the future and, with an unusual acuity, am allowed to experience the

After 6+ months in the making and burning over a year of GPU compute time, we're super excited to finally release the "Ultra-Scale Playbook" Check it out here: hf.co/spaces/nanotro… A free, open-source, book to learn everything about 5D parallelism, ZeRO, fast CUDA kernels,

Excited to be at the APS March Meeting this year! American Physical Society I'll be giving a talk in the Tues afternoon session MAR-J58, Physics of Learning & Adaptation I.

Exactly why we spent last year on LLM regressors -> reward models to simulate expensive world feedback, not just humans. "To discover new ideas that go far beyond existing human knowledge, it is instead necessary to use grounded rewards: signals that arise from the environment

Richard Feynman playing bongo drums with Ralph Leighton on the song "Orange juice".

Just arrived in Singapore 🇸🇬 for ICLR! Excited to meet old and new friends alike. Let me know if you're around!