🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Been wild to hear from Yi how things in the wild are, definitely worth a read!

idk who needs to hear this but span corruption != bidirectional attention, you could have one, both, or neither. addendum: ul2 could be implemented completely with a casual decoder!!!!

Exciting News from Chatbot Arena! Google DeepMind's new Gemini 1.5 Pro (Experimental 0801) has been tested in Arena for the past week, gathering over 12K community votes. For the first time, Google Gemini has claimed the #1 spot, surpassing GPT-4o/Claude-3.5 with an impressive

bratpropagation

New paper📢 LLM folks have been supervised finetuning their models with data from large and expensive models (e.g., Gemini Pro). However, we achieve better perf. by finetuning on the samples from the smaller and weaker LLMs (e.g., Flash)! w/Mehran Kazemi Arian Hosseini Rishabh Agarwal vinh q. tran

just like many lessons in LLMs past, take the compute-match comparisons seriously and you will prosper check out Hritik's excellent internship work!

Welcome back Yi!! 🚀

Have you wondered why next-token prediction can be such a powerful training objective? Come visit our poster to talk about language and fractals and how to predict downstream performance in LLMs better. Poster #3105, Fri 13 Dec 4:30-7:30pm x.com/ibomohsin/stat… See you there!

they don't realize it yet but legacy brain and legacy deepmind are secretly the yin and yang of AI research -- forced together in GDM they are differing, chasing, yet necessary to push Gemini beyond the frontier

Prediction: People say pretraining will end, and I think everyone will be surprised how many multipliers we can squeeze from existing data through all kinds of algorithms.

pretraining is a mindset

Take BIG-Bench Hard but make it EVEN HARDER!! Check out this cool new benchmark that really shows how much further our models still have to go on general reasoning!

the first of many exciting days 🥇

even deeper BTS: semantic ids in DSI were thought of and implemented almost completely from the hospital since I was in poor health back then (kidney failure, all better now!) -- sometimes doing interesting research is a great distraction from other more difficult things going on

maybe there's some cool stuff going on in 2.5 Flash-Lite

Congrats to everyone on this remarkable milestone! Incredible to see the outrageous power of general models and training methods.