My experience on Pro vs non-Pro modes

This incredible work reveals mitigating a hidden bias could contribute to improving long context performance, potentially opening up the path to a formulation that can direct attention equally to tokens deeper in the sequence. 👏 Francesco Pappone

Excellent eval, and open weight models holding ground 🙌

This is probably nothing 👀

Come chat to Aarush Sah about OpenBench! 🔥 Ask him about: eval saturation, harness influence, LLM-as-a-judge, up-and-coming evals, which evals are useful, open source vs proprietary models on various evals, and much much more!

And it is really fast, including chunking, merging, RTFx factor of 200x+ (20 seconds for 1 hour and 8 minutes of audio)

I need an optimizer as cheap as Adam, as good as Muon, and as stable as shampoo. Come on optimizer people!

Begun, the bundling wars have.

Core takeaway from open-sci-ref 0.01: data quality > scale. A 1.7B model trained on Nemotron-CC-HQ for 1T tokens matches SmolLM2-1.7B trained on ~11T. Further confirms the value of curating high quality (open) datasets.

RL finds a way, intelligence is finding the path of least resistance. We just need to make sure the path of least resistance is a useful one.