🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

outperforming deepseek in 6/10 categories... while being x6 smaller

I added cohere command A to this chart, I had to extend the axis a bit though….

UPDATE: my numbers were off, external benchmarking actually shows we are faster and better. GPQA-diamond: 53% miliseconds per token: 5.36 artificialanalysis.ai/providers/cohe…

I really enjoyed my Machine Learning Street Talk chat with Tim at #NeurIPS2024 about some of the research we've been doing on reasoning, robustness and human feedback. If you have an hour to spare and are interested in some semi-coherent thoughts revolving around AI robustness, it may be worth

I'm excited to the tech report for our @Cohere Cohere For AI Command A and Command R7B models. We highlight our novel approach to model training including the use of self-refinement algorithms and model merging techniques at scale. Command A is an efficient, agent-optimised

Merging 🍇 + polishing 🧽 = ⌘🧑🏼‍🍳

I'm really proud to have led the model merging work that went into cohere Command A and R7B, all made possible by an amazing group of collaborators. Check out the report for loads of details on how we trained a GPT-4o level model that fits on 2xH100!

Following the open-weight release of Command A and Command R7B models, we're excited to have collaborated with @Cohere colleagues on a tech report highlighting our novel approach to model training, including self-refinement algorithms and model merging techniques at scale.

We’re redefining what’s possible with AI. With the release of our latest model, Command A, optimized for real-world agentic and multilingual tasks, we’re demonstrating our commitment to bringing enterprises AI that goes beyond the ordinary, and offers security & efficiency.

the complete cooking guide with all the ingredients, seasonings and garnishes for this soup of a model is here! 🍲🧂🌶️🔥 couldn’t be more proud to have continued my exploration of model merging and translated it to our A-class flagship model, Command A, with the best team! ✨

The most interesting part of their post training is just how much they use model merging both in SFT and RL. Their process is: - Train an instruct model - Train 6 SFT in 6 domains(Code, Safety, RAG, Math, Multilingual, and General Long-Context) - Merge - Use this merge to train

The next section on Merging is the most interesting imo. As a summary of what we discussed earlier, they used expert merging: merge the SFT then merge the preference tuned models

They find that linear merging is pretty interpretable ie upweight an expert leads to better performance in that domain. However, the corresponding drop in performance is unpredictable Interestingly, they add cross-domain data for each expert as a form of regularization.

Some thoughts: First, the paper is pretty well-written and easy to follow. They have so many benchmarks and results. I think its similar in spirit to the llama3 paper but they are more complementary imo as this paper focuses heavily on post training while Llama3 didn't do a good

Now feels like a good time to plug cohere Command A: - model evaled on lmarena.ai is same as hosted on Hugging Face - claimed performance is reproducible - not trained on the test set - uses the cohere hybrid attention architecture for long context - fits on 2xH100 not 8x

When we came up with RAG five years ago, we weren't creating a workaround for small context windows—we were designing a principled approach to augment models with external knowledge. The core challenges RAG addresses remain unsolved with just larger context windows: • Accessing

You like Code, you like LLMs, you are looking for a leadership position? We are searching for somebody who can support our amazing team and bring code agents for enterprises to new heights! jobs.ashbyhq.com/cohere/758e544…

How exactly was the initial Chatbot Arena version of Llama 4 Maverick different from the public HuggingFace version?🕵️ I used our Feedback Forensics app to quantitatively analyse how exactly these two models differ. An overview…👇🧵

TIL cohere's best LLM (Command A) is higher than Anthropic's best LLM on the Arena

How does sparse attention reshape LLM scaling? 🔍 We’re excited to share this work by former @Cohere intern Piotr Nawrot, “The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs.”