🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

The 2024 Brain Prize goes to pioneers of computational and theoretical neuroscience: Larry Abbott, Haim Sompolinsky, and Terry Sejnowksi. It's fabulous to see the field being recognized in a big way, and I can't think of a more deserving group of laureates for it.

🇯🇵JaColBERTv2 is out 🎉🎉 It's an extension of JaColBERTv1 using distillation - with Omar Khattab's MSMarco scores applied to MMARCO-🇯🇵 JaColBERTv2 tops セコン's JQaRa, doing 5.6% better than the best single-vector model & outperforming models w/ 5x the param count!

Together with Hugging Face + Answer.AI +, in true #FOSS spirit, many amazing volunteers, we just released bitsandbytes==0.43.0, with these cool new additions: - 🛫FSDP+QLoRA alpha release - ⊞🌈 Windows support via pip install - 📄 huge docs update Find out more about FSDP:

Thanks to the amazing work of Jeremy Howard, the Answer.AI team (special thanks to Jonathan Whitaker & Benjamin Warner for walking through the changes with me) for getting FSDP + QLoRA working. We've managed to integrate their findings into Axolotl and now have additional

New MoE alert! 🔔 Qwen1.5-MoE-A2.7B just got released. Qwen-MoE is a 14B parameter Mixture of Experts Models matching the performance of 7B models, but with the speed of 2B models. 🏎️⚡️ TL;DR 🧮 14.3B parameters with 2.7B activated during generation 👨‍🏫  60 experts with

Vik is an extraordinary person. He won the challenging Kaggle essay scoring competition, founded the super successful company Dataquest, created top tier NLP tools like Surya & Marker, & more. Now he's joining Answer.AI. Read his journey to deep learning mastery here:

I love these tips from Jonathan Whitaker, just posted to Answer.AI, on attacking "high-surface-area problems"; i.e problems where "when something doesn’t work it can be hard to find out where the issues may be, let alone what we need to do to fix them." answer.ai/posts/2024-04-…

Announcing FastHTML. A new way to create modern interactive web apps. Scales down to a 6-line python file; scales up to complex production apps. Auth, DBs, caching, styling, etc built-in & replaceable and extensible. 1-click deploy to Railway, Vercel, Hugging Face, & more.

Announcing Cold Compress 1.0 with Answer.AI A hackable toolkit for using and creating KV cache compression methods. Built on top of Horace He and Team’s GPT-Fast for torch.compilable, light-weight performance. Develop novel methods in as little as 1 line of new code.

ModernBERT, BERT revisited in the age of LLMs and Generative AI! LightOn and Answer.AI modernized BERT! Improved architecture with 8192 context length, flash attention, and trained on 2T tokens. ModernBERT outperforms version BERT and RoBERTa versions! 👀 TL;DR; 2️⃣ Comes

"We're gonna need a bigger graph"

For those of you looking to get started with finetuning Answer.AI's new ModernBERT models with Hugging Face Transformers ... I got you covered github.com/AnswerDotAI/Mo…

Not too shabby...