🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

TODAY it is time for the Algorithmic Fairness Workshop #AFME2024 at #NeurIPS2024! 📍West Meeting 111-112! Excited for our 5 contributed spotlight talks today! Benjamin Laufer Danny To Eun Kim (@teknology.bsky.social) Alex Tamkin Prakhar Ganesh Natalie Mackraz

In our second contributed talk, Danny To Eun Kim (@teknology.bsky.social) presents “Towards Fair RAG: The Impact of Fair Ranking in Retrieval-Augmented Generation.” #AFME2024 #NeurIPS2024

🚨🚨 Preprint Alert 🚨🚨 🚀🚀 As AI become agents 🤖, how can we reliably delegate tasks to them, if they cannot communicate their limitations😭 or ask for help or test-time compute 🧑‍🚒 when needed? We present our new pre-print **Self-Regulation and Requesting Interventions**

How to construct repo-level coding environments in a scalable way? Checkout RepoST: an automated framework to construct repo-level environments using Sandbox Testing (repost-code-gen.github.io) Models trained with RepoST data can generalize well to other datasets (e.g., RepoEval)

#NLProc New paper on "evaluation-time scaling", a new dimension to leverage test-time compute! We replicate the test-time scaling behaviors observed in generators (e.g., o1, r1, s1) with evaluators by enforcing to generate additional reasoning tokens. arxiv.org/abs/2503.19877

If you're interested in OpenAI including shopping results, you might also be interested in Danny To Eun Kim (@teknology.bsky.social)'s paper relating retrieval diversity/fairness and generation by downstream RAG models. This has implications for individuals selling products online. arxiv.org/abs/2409.11598

OpenAI x.com/841io/status/1…

Ever trusted a metric that works great on average, only for it to fail in your specific use case? In our #NAACL2025 paper (w/ Fernando Diaz), we show why global evaluations are not enough and why context matters more than you think. 📄 aclanthology.org/2025.findings-… #NLP #Evaluation (🧵1/9)

Please share the word and register to participate! #TREC2025ToT

🖋️ Curious how writing differs across (research) cultures? 🚩 Tired of “cultural” evals that don't consult people? We engaged with researchers to identify & measure ✨cultural norms✨in scientific writing, and show that❗LLMs flatten them❗ 📜 arxiv.org/abs/2506.00784 1/11