🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Photon: A New SOTA for Decentralized LLM pre-training at MLSys Conference 2025. Poster today; talk on Thursday. Lorenzo Sani converting one person at a time to the merits of federated everywhere for everything. Paper: arxiv.org/abs/2411.02908

We're hiring at Cambridge ML Systems Lab for engineering support to scale our decentralized AI research. Find me at MLSys Conference this week, or reach out online. Help build the next version of Photon, and an opportunity to also collaborate with Flower. jobs.cam.ac.uk/job/51202/

Next Friday, May 23rd, our ML Theory group is excited to host Frederik Kunstner for a session on "Heavy-tailed imbalance and why Adam outperforms gradient descent on language models." Special shoutout to Anier Velasco Sotomayor, Andrej Jovanović, and Thang Chu for organizing this event 🎉

We are super excited to host Frederik next Friday 🪇 Cannot wait to dig deep into why Adam is such a robust optimiser, and why it is the de facto standard for many applications. Do tune in! 📺

Don't forget to join us today!

Join me to hear about decentralised training, why it works and what opportunities it can unlock 🚀. Many thanks to harsha for the invitation!

Our ML Efficiency Group is excited to welcome Andrej Jovanović on Tuesday, June 17th for an insightful presentation of "Communication-efficient training for foundation models through federated learning"

Super excited to host Ambroise Odonnat next week, particularly to hear about the connection between LLMs and their n-gram predecessors!

The future of open-source AI is decentralized — and it’s arriving in San Francisco this September.

This week, we are delighted to host Max Ryabinin (Max Ryabinin) of Together AI who will be speaking at the Cambridge ML Systems Seminar Series. His talk is at 3pm June 18th, LT1 Cambridge Computer Science

Be sure to join our ML Efficiency Group tomorrow, June 17th as they host Andrej Jovanović for a session on "Communication-efficient training for foundation models through federated learning" Learn more: cohere.com/events/Cohere-…

Introducing RL2: Ray-Less Reinforcement Learning for LLMs 🚀 Want to run RL experiments but tired of complicated abstractions? We've got you covered with <1K lines PPO/REINFORCE implementation: 🎯 Ray Less = Launch RL exps with torchrun just like SFT ⚡ Long-context

Here is the recording with the slides for those interested! 🎤 youtu.be/UONvP1TL0-g?fe… 📊drive.google.com/file/d/14ZIopS… 📑arxiv.org/pdf/2410.02724 Cohere Labs Cohere Labs

Join our ML Theory group next week as they welcome Tony S.F. on July 3rd for a presentation on "Training neural networks at any scale" Thanks to Andrej Jovanović Anier Velasco Sotomayor and Thang Chu for organizing this session 👏 Learn more: cohere.com/events/Cohere-…

An absolute privilege to be involved with all the fantastic individuals at the Cohere Labs community! This is your sign to join :D

Don't forget to join us tomorrow, July 3rd as we host Tony S.F. for a session on "Training neural networks at any scale" Learn more: cohere.com/events/Cohere-…

Do you believe in local LLMs?

The Hugging Face folks deserve far more credit for being a pillar of open-source and still managing to push out SOTA results across the board, along with a full write-up of the entire model’s lifecycle.