Great to see a collaboration between Andrew Ng, Databricks and DSPy ! 🌟 DSPy is a powerful and thoughtful framework. The way it treats an LLM system as a broad search space and optimizes the entire system is very impressive. It is part of the production workflow in

Highly recommend this DeepLearning.AI course by Chen Qian as a way to learn about DSPy & how you can leverage it to build & optimize agents! 🤖⭐ Chen did a phenomenal job crafting these lessons with helpful hands-on tutorials. Let us know what you think! 🧩

👀

Agent Bricks is officially launched! 🤖🧱 It's been incredibly fun working on these products with the rest of Databricks Mosaic Research & the Databricks engineering team. Excited to see what folks are able to build with them!

Congratulations to Omar Khattab and the DSPy team! DSPy 3.0!! 🚀 Continuing to push the frontier for building AI software!🔥🔥🔥

I’ll be at #ICML this week! ✈️🇨🇦 Excited to chat with folks about DSPy, automatic prompt optimization methods, compound AI systems, and research roles at Databricks (we’re hiring!). If these topics interest you, feel free to reach out or find me at the Databricks booth!

Come to our Databricks booth at #ICML if you want to chat about the path to building real AI systems reliably, sustainably, and at scale 🚀

The #SIGIR2025 Best Paper just awarded to the WARP engine for fast late interaction! Congrats to Luca Scheerer🎉 WARP was his ETH Zurich MS thesis, completed while visiting us at @StanfordNLP. Incidentally, it's the fifth Paper Award for a ColBERT paper since 2020!* Luca did an

🚨 Can your LLM really do math—or is it cramming the test set? 📢 Meet Putnam-AXIOM, a advanced mathematics contamination-resilient benchmark that finally hurts FMs. 1. openreview.net/forum?id=kqj2C… 2. icml.cc/virtual/2025/p… #ICML2025 East Exhibition Hall A-B, #E-2502 🧵1/14

Really cool work! And great seeing our new optimizer SIMBA out in the wild 🦁

There is a rich set of research questions in design and optimization of agentic workflows with a ton of room for theoretical & algorithmic work! A great starting point to get exposed to them is the MIPRO paper (Krista Opsahl-Ong Omar Khattab et al.) and the DSPy framework.

Paper: arxiv.org/abs/2507.19457 GEPA will be open-sourced soon as a new DSPy optimizer. Stay tuned! Incredibly grateful to the wonderful team Shangyin Tan Dilara Soylu Noah Ziems Rishi Khare Krista Opsahl-Ong Arnav Singhvi Herumb Shandilya Michael Ryan @ ACL 2025 🇦🇹 Meng Jiang Christopher Potts

RLVR isn't just for math and coding! At Databricks, it's impacting products and users across domains. One example: SQL Q&A. We hit the top of the BIRD single-model single-generation leaderboard with our standard TAO+RLVR recipe - the one rolling out in our Agent Bricks product.

Prof. Yann LeCun has now christened GEPA settling any debate on it's pronunciation! x.com/ylecun/status/…

Since joining Databricks, our research team has been hard at work on Agent Bricks, a new product that helps enterprises develop state-of-the-art domain-specific agents. We are now releasing a research blog about Agent Learning from Human Feedback (ALHF) databricks.com/blog/agent-lea…

Really excited about ALHF, new work from our research team that lets users give natural language feedback to agents and optimizes them for it. It sort of upends the traditional supervision paradigm where you get a scalar reward, and it makes AI more customizable for non-experts.

Ever wonder what it'd look like if an LLM Judge and a Reward Model had a baby? So did we, which is why we created PGRM -- the Prompt-Guided Reward Model. TLDR: You get the instructability of an LLM judge + the calibration of an RM in a single speedy package (1/n)

Not that I have a favorite recent project, but... 🧵 LLM judges are the popular way to evaluate generative models. But they have drawbacks. They're: * Generative, so slow and expensive. * Nondeterministic. * Uncalibrated. They don't know how uncertain they are. Meet PGRM!

Automated prompt optimization (GEPA) can push open-source models beyond frontier performance on enterprise tasks — at a fraction of the cost! 🔑 Key results from our research Databricks Mosaic Research: 1⃣ gpt-oss-120b + GEPA beats Claude Opus 4.1 on Information Extraction (+2.2 points) —

Prompt optimization is becoming a powerful technique for improving AI that can even beat SFT! Here are some of our research results with GEPA at Databricks, in difficult Agent Bricks info extraction tasks. We can match the best models at 90x lower cost, or improve them by ~6%.