🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Excited to be at #NeurIPS2023 and looking forward to meeting new and old friends! Interested in concept control for text-to-image models? Find our poster tomorrow (Tue) at 10:45 AM arxiv.org/abs/2302.03693 w/ Lin Gui Jeffrey Negrea and Victor Veitch 🔸

Transforming and Combining Rewards for Aligning Large Language Models paper page: huggingface.co/papers/2402.00… A common approach for aligning language models to human preferences is to first learn a reward model from preference data, and then use this reward model to update the

LLM best-of-n sampling works great in practice---but why? Turns out: it's the best possible policy for maximizing win rate over the base model! Then: we use this to get a truly sweet alignment scheme: easy tweaks, huge gains w Lin Gui Cristina Garbacea arxiv.org/abs/2406.00832

Fundamentally, high-level concepts group into categorical variables---mammal, reptile, fish, bird---with a semantic hierarchy---poodle is a dog is a mammal is an animal. How do LLMs internally represent this structure? arxiv.org/abs/2406.01506

Are LLMs just doing next token predictions? It is believed that if an LLM can accurately predict the next tokens in a Wikipedia entry, it essentially "learns" the information. But do pre-trained LLMs actually need to understand context sentences to solve this task? The answer is

Excited to present my paper at #ICML2024 on transforming and combining reward models for RLHF! Join me on Wed, July 24, 11:30 a.m. - 1 p.m. CEST at Hall C 4-9 #2710.

🧵 RATE: Score Reward Models with Imperfect Rewrites of Rewrites 1/ How do you measure whether a reward model incentivizes helpfulness without accidentally measuring length, complexity, etc? Rewrites of rewrites give good counterfactuals, without needing to list all confounders!

Semantics in language is naturally hierarchical, but attempts to interpret LLMs often ignore this. Turns out: baking semantic hierarchy into sparse autoencoders can give big jumps in interpretability and efficiency. Thread + bonus musings on the value of SAEs: