Multimodal RAG: Just use ColPali/DSE then pass your screenshots to the LLM This is the dream, but how well do LLMs read text contained in images? We wanted to know, so we tried a simple thing: do results change on evals when using screenshots rather than text as input? Yes.

You can just continue pre-train things ✨ Happy to announce the release of BioClinical ModernBERT, a ModernBERT model whose pre-training has been continued on medical data The result: SOTA performance on various medical tasks with long context support and ModernBERT efficiency

I'll be covering Reason-ModernColBERT in tonight's presentation, so please come if you are interested! maven.com/p/1973fe/going… (And please be gentle, this is the first time I will be speaking live in front of this many people 😭)

With LightOn we are thrilled to release pylate-rs 🚀⭐️ An efficient inference engine for late-interaction models written in Rust and based on Candle ⚡️ pylate-rs is the best Python library / Rust crate / NPM package to spawn late-interaction models in milliseconds.

Magical Raphaël back at it again You can now compute ColBERT embeddings in Rust at light speed for any PyLate models (thus any ColBERT models) The best part? You can use it with WebAssembly to create awesome demos/visualizations in the browser!

🚀 Insane day yesterday for the Knowledge squad LightOn! Raphaël Sourty shipped PyLate-rs and Antoine Chaffin delivered a beautiful lecture on late interaction models supremacy, LFG ❤️

Should we just focus our pre-training efforts on decoders? To answer this, we trained Ettin, various identically trained encoders and decoders, ranging from 17M to 1B parameters on 2T tokens of open data (beating Llama 3.2 and ModernBERT in the process)!

We are at #ACL2025 with Oskar Hallström Do not hesitate to come discuss with us if you are interested in IR, encoders, late interaction or VLMs! I am attaching a picture of us because I figured people do not know our faces due to our profile pictures 🥲

📍 ACL 2025: Encoders-only coffee chat anyone? Antoine Chaffin & Oskar Hallström are in Vienna to present ModernBERT Paper at ACL 2025 📅 Don’t miss the Poster Session today 11am. ➡️ Poster 115 ☕ Or feel free to catch them in the #ACL2025NLP aisles! 👉 To know more

Today is the national encoder day at #ACL2025 Make sure to come to the board 115 to claim your encoder party membership card!

Happy to release the 1.3.0 version of PyLate at LightOn with my handsome co-maintainer Antoine Chaffin 😗 Fast-Plaid is now the default backend for PyLate retrieval. It's faster and as accurate as the original Stanford PLAID on both CPU and GPU

4/10 Efficiency Single H100 GPU (80 GB): • 5.71 pages / s ≈ 493 000 pages / day • 6.49× faster than dots.ocr • 2.67× faster than PaddleOCR-VL-0.9B • 1.73× faster than DeepSeekOCR • < $0.01 per 1 000 pages A compact model that’s both high-quality and cost-efficient.

LightOn joins the OCR mania We release a 1B model achieving SOTA results while being much faster than all the recent releases It is also an end-to-end trainable solution for easy adaptation to your specific domains We also share interesting insights (and soon the dataset!)

1B VLM dedicated to OCR. State of the art, cooked at LightOn Compatible with HF and VLLM. I have been amazed by the quality of the output on scientific papers, huge congrats staghado Baptiste Aubertin and the whole R&D team 🐐

SotA OCR model that is FRENCH and can process up to 493k pages per day on a H100!!

The recipe for a fast, performant OCR model: 1. tell Said that OCR is solved 2. let him rage about the state of OCR 3. get a few smart people in a GMeet with him 4. tell them there are GPUs available 5. wait a bit 6. enjoy🦉 Soon deployed in your favorite Enterprise environments

Last few days have been insane in the OCR land with releases from DeepSeek, PaddlePaddle and others. Now we at LightOn are entering the game with our latest release, pushing the state of the art even further. Kudos staghado Baptiste Aubertin Adrien Cavaillès 🥳

Shoutout to our Grand Retrieval Master and Model Whisperer Amélie Chatelain. I had so much FOMO for this talk that I decided to go to London myself to see it. See you there!!

Had an amazing time giving a talk on Retrieval in the Age of Agents at Weights & Biases's #FullyConnected2025! Feeling very grateful to have had this opportunity as well as fascinating discussions with the other attendees ❤️.

This is now, booth 190, come get your encoder and late interaction party subscription! Edit: it’s at CIKM in Seoul in case you missed the context from my previous tweets 🥹