I'm not at #NeurIPS2024, but go talk with Oliver and Nate who will present our work today at 4:30pm PST in the East Exhibit Hall A-C [Poster #2605] arxiv.org/abs/2402.14897

📢New Paper Alert!🚀 Human alignment balances social expectations, economic incentives, and legal frameworks. What if LLM alignment worked the same way?🤔 Our latest work explores how social, economic, and contractual alignment can address incomplete contracts in LLM alignment🧵

🚨🚨 New preprint 🚨🚨 Ever wonder whether CoTs correspond to the internal reasoning process of the model? We propose a novel parametric faithfulness approach, which erases information contained in CoT steps from parameters to assess CoT faithfulness. arxiv.org/abs/2502.14829

SO excited to see this one released! Several works, incl. our TMLR'24 paper, are doubtful about measuring faithfulness purely behaviorally. Martin Tutek formulated how to measure faithfulness by actually connecting verbalized CoT reasoning to weights. See more insights in his thread👇

I'll talk about measuring faithfulness of verbalized reasoning today at Repl4NLP at *9:45* #NAACL2025

Hey #NAACL2025 friends! You are all invited to join us at the RepL4NLP workshop with an amazing lineup of speakers & panelists Ana Marasović Najoung Kim 🫠 Akari Asai (starting TODAY 9:30am Ballroom A, floor 2) and posters (Hall 3, floor 1)!

Slides available here: docs.google.com/presentation/d…

📢Announcing 𝐭𝐡𝐞 𝐟𝐢𝐫𝐬𝐭 𝐰𝐨𝐫𝐤𝐬𝐡𝐨𝐩 𝐨𝐧 𝐭𝐡𝐞 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐋𝐋𝐌 𝐄𝐱𝐩𝐥𝐚𝐢𝐧𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐭𝐨 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐚𝐧𝐝 𝐏𝐥𝐚𝐧𝐧𝐢𝐧𝐠 at Conference on Language Modeling! We welcome perspectives from LLM, XAI, and HCI! CFP (Due June 23): …reasoning-planning-workshop.github.io

𝐖𝐡𝐚𝐭 𝐇𝐚𝐬 𝐁𝐞𝐞𝐧 𝐋𝐨𝐬𝐭 𝐖𝐢𝐭𝐡 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧? I'm happy to announce that the preprint release of my first project is online! Developed with the amazing support of Abhilasha Ravichander and Ana Marasović (Full link below 👇)

Having built hard reasoning-over-text benchmarks the "old-fashioned" way (with crowdworkers), we had to ask: what if we used LLMs instead? Answer: we'd get easier benchmarks. More in the thread by amazing Alex Gill 👇

1/ 🚨NEW PAPER: "BriefMe: A Legal NLP Benchmark for Assisting with Legal Briefs", accepted to ACL Findings 2025! We introduce the first benchmark specifically designed to help LLMs assist lawyers in writing legal briefs 🧑‍⚖️

Really excited about this! As backstory, Jesse Woo started this project when I taught a ML Datasets class at Columbia. Then we joined up with Ana Marasović and Fateme Hashemi Chaleshtori and really kicked it into high gear. Would not have happened without the full team!

A new contribution to our line of work building useful AI assistance, this time in legal space. Grateful to co-authors for teaching me about how people do their work in this domain. See more in Fateme's thread 👇

Check out the paper here: arxiv.org/abs/2507.06329 And the website here: mclemcrew.github.io/mixassist-webs… S/O to Ana Marasović for all the leadership, support, and expertise they put into this work! We are so happy with how it turned out and hope it helps the community of music producers!

My first audio AI paper, thanks to MClem who introduced me to a whole new world of music producing! Getting to work with students who bring their unique passion into PhD work is one of the best perks of a professor's job. Check more in the thread 👇🏻 Soon at #COLM2025

Now in hall X4, board 407! #ACL2025 #ACL2025NLP

Very pleased that FUR was accepted to EMNLP 2025 Main🎉 In case you can’t wait so long to hear about it in person, it will also be presented as an oral at INTERPLAY Workshop Conference on Language Modeling🥳 FUR is a parametric test assessing whether CoTs faithfully verbalize latent reasoning.

Check Martin Tutek's thread!