🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

I’m excited to release a paper (and model weights!) for InCoder: a generative code model that can infill as well as do left-to-right generation. Project page: sites.google.com/view/incoder-c… Demo: huggingface.co/spaces/faceboo… Paper: github.com/dpfried/incode… Thread (1/n):

Late post but let’s do this! Happy to share our #EMNLP2022 work on translating natural language to executable code with execution-aware minimum Bayes risk decoding 📝Paper: arxiv.org/pdf/2204.11454… 📇Code: github.com/facebookresear… 📦Data (codex output):dl.fbaipublicfiles.com/mbr-exec/mbr-e… (1/n)

🧑‍💻Code review is an important practice in software development. We take this idea into code generation and propose Coder-Reviewer reranking. Our method reranks via the product of a coder model p(code|instruction) and a reviewer model p(instruction|code). arxiv.org/abs/2211.16490

Execution results are strong indicators of program correctness. But how can we improve LLMs for code generation with execution? In our new paper, we propose LEVER, a simple method that learns to verify and rerank LLM-generated programs with their execution results. 🧵👇 (1/n)

📢Introducing CRUXEval, a benchmark to measure Python code execution! 🏠Homepage: crux-eval.github.io 📜Paper: crux-eval.github.io/paper/cruxeval… 🏆Leaderboard: crux-eval.github.io/leaderboard.ht… 🔎Sample Explorer: crux-eval.github.io/demo.html 📊HF Dataset: huggingface.co/datasets/cruxe…

📢📢Excited to introduce our new work LiveCodeBench! 📈 Live evaluations to ensure fairness and reliability 🔍 Holistic evaluations using 4 code-related scenarios 💡Insights from comparing 20+ code models 🚨🚨We use problem release dates to detect and prevent contamination

LLMs for code should do much better if they can iterate on tests -- but they don't. Our new work (RLEF) addresses this with execution feedback at RL *training time* to use execution feedback at *inference time*. arxiv.org/abs/2410.02089 is just out! 1/6

We're launching SWE-bench Multimodal to eval agents' ability to solve visual GitHub issues. - 617 *brand new* tasks from 17 JavaScript repos - Each task has an image! Existing agents struggle here! We present SWE-agent Multimodal to remedy some issues Led w/ carlos 🧵

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution arxiv.org/abs/2502.18449 by Yuxiang Wei Sida Wang and the whole team! Get started with your favorite model here github.com/facebookresear…

Learning language through communication in blocksworld, awesome work by Christopher Manning &c arxiv.org/pdf/1606.02447…