SWE-agent 1.0 is so much more flexible than before. It has never been easier to set it up with various tool bundles or multiple LMs. And you can combine them all in a multi-attempt scheme!

Watch Princeton's SWE-agent Kilian Lieret reveal research on autonomous coding agents at Daytona AI Builders GitHub HQ! From benchmarks to production-ready frameworks, see why AI agents are 3x better than last year 🚀 Link in reply ⬇️

We just updated the SWE-bench Multimodal leaderboard. Congrats to Globant, Zencoder, and the Agentless team from UIUC for their strong results.

The creators of LiveCodeBench just released a new, private, SWE-bench like benchmark in Java, C++, Python, JavaScript, TypeScript. SWE-agent is at the top! All system use Claude 3.7.

Join carlos and me today at GenAI Collective NYC as we break down SWE-Bench, SWE-agent, and the future of AI-driven software engineering. What works? What’s next? What does this mean for developers? Let's discuss! 📍 Today 1–4pm, Brooklyn Navy Yard

Had a great time talking about building agents, SWE-agent, SWE-bench, and more

Introducing SWE-bench Multilingual: a new eval in the SWE-bench family to test LLM coding abilities in *9* programming languages, fully integrated with SB so it can plug into existing workflows. Claude 3.7 gets 43% on SB Multilingual vs 63% on SB Verified, a 20 pt drop!🧵

40% with just 1 try per task: SWE-agent-LM-32B is the new #1 open source model on SWE-bench Verified. We built it by synthesizing a ton of agentic training data from 100+ Python repos. Today we’re open-sourcing the toolkit that made it happen: SWE-smith.

Can GPT, Claude, and Gemini play video games like Zelda, Civ, and Doom II? 𝗩𝗶𝗱𝗲𝗼𝗚𝗮𝗺𝗲𝗕𝗲𝗻𝗰𝗵 evaluates VLMs on Game Boy & MS-DOS games given only raw screen input, just like how a human would play. The best model (Gemini) completes just 0.48% of the benchmark! 🧵👇