🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

🔥 Great to see LoRA+ getting so much attention! Pro tip: Since LoRA+ gains are (mostly) "orthogonal" to other methods, combining it with different LoRA variants could give even better results 🚀 Also: LoRA+ is now part of HuggingFace (PEFT). #AI #LLMs #LoRA #finetuning

Due to a high demand for registrations, NeurIPS will be moving towards a randomized lottery system, effective immediately. Authors of accepted conference and workshop papers are still guaranteed registration, but this may change as we release spots to the lottery, so we urge

Saying 'LLMs are just next token predictors' is like saying 'A polynomial function is just a sum of monomials (x^k)'. Scale is key - with the right scale, a polynomial function can approximate incredibly complex behaviors. 🧮🤖

Said 'wassup' to Gemini 2.0 Flash Thinking and I think I gave it social anxiety... it wrote a whole research paper on how to respond casually 😭

People compare AI to past historic breakthroughs 🔄 (industrial revolution, internet, etc), but there's a crucial difference: In previous advancements, humans remained the most intelligent beings. This time, we're creating something that could surpass us 🤖. It's a singularity!⚡️

It turns out o1 style reasoning is just base model + RL (GPPO)! 🧐 github.com/deepseek-ai/De…

100%! For instance, if you have good understanding of concentration of random variables, you should be able to infer (without a lot of engineering) how to scale init and learning rate with width (µP, Mean-Field, etc), or with depth (Stable ResNet, Depth-µP, etc.)

Great idea!

I had a similar experience as well. I have a feeling that these systems will probably do most of what current PhD students can do. Strong PhD students will benefit from this by effectively using and directing these systems. Interesting times.

Academia

Scale is the only permanent feature of state-of-the-art AI models. All other characteristics are subject to change and innovation.

something is off with Claude Sonnet 4. It hallucinates more often than Sonnet 3.5/3.7

It seems that most gains from RL comes from the pretrained model itself. The format reward stuff (GRPO etc) just extracts those capabilities. It helps to have good reward signal, but it's not the main ingredient.

the conference is getting crazy over it today we're unveiling our 1st robot Hugging Face 🤝 Pollen Robotics a low-cost $250 open-source robot designed as an open-source platform for fun human computer interactions powered by HF Spaces-models-community > discord.com/invite/jsvMRQx…

This is the most obvious 1B$+ startup idea right now

The current debate on reasoning in LLMs: Group A: "We see it, we feel it, therefore it exists" Group B: "We don't see it, we don't feel it, therefore it doesn't exist" Group C (<0.01%): "What is reasoning?"