One paper, lead by kstechly (w/ Matthew M), evaluated the claims over a suite of graph coloring problems. The setup allows for GPT4 guessing a valid coloring in stand alone and self-critiquing modes. There is an external sound verifier outside the self-critiquing loop. 2/

📢 Check out these posters on LLM Self-Critiquing (in)abilities in reasoning and planning tasks, being presented at the #NeurIPS2023 "Foundation Models for Decision Making" workshop today (12/15) by yochanites Karthik Valmeekam and kstechly in Hall E2.

📢 "On the self-verification limitations of LLMs in Reasoning and Planning Tasks" arxiv.org/abs/2402.08115 (lead by Karthik Valmeekam and kstechly) 👇 Investigates LLM self-verification in three formal benchmarks--Game of 24, Graph Coloring and Planning, and shows that accuracy

Two Yochanites drove 15 hours to Austin city limits and saw this.. 🤗

A research note describing our evaluation of the planning capabilities of o1 🍓 is now on arXiv.org arxiv.org/abs/2409.13373 (thanks to Karthik Valmeekam & kstechly). As promised, here is a summary (..although you should read the whole thing..) 🧵 1/

Woo hoo.. Chain of Thoughtlessness paper will be showing up at #NeurIPS2024 🤗 Congrats to Karthik Valmeekam kstechly [details below 👇]

So Karthik Valmeekam, kstechly, Lucas Saldyt and I will be at #NeurIPS2024 starting Tuesday--to present a bunch of things👇. Easiest to catch us at 11th 11AM poster session at our "Chain of thoughtlessness" poster (East hall #3010). (I am around 10th-13th--and am happy to chat

👉 Our #NeurIPS2024 paper on Chain of Thoughtlessness @ 11AM poster session today (East Hall #3010, 11AM-2pm...). All three of us are here and looking forward to chat/answer qns.. 🙏

📢"On the Self-Verification Limitations of LLMs in Reasoning and Planning Tasks" arxiv.org/abs/2402.08115 with kstechly and Karthik Valmeekam apparently made it to #ICLR2025.. Swimming🏊 to Singapore.. 😎 [The 🧵s below give the details.. ]

📢Delighted to share that our analysis of the planning abilities of 🍓 o1 has now been accepted to #TMLR Transactions on Machine Learning Research . Joint work with Karthik Valmeekam kstechly and Atharva Gundawar. Final version, including DeepSeek R1 results, appearing soon..

PSA for #ICLR2025 authors frantically making posters: Stop worrying! Just prompt o3 and GPT-4o and you will have an AGI-ready poster in seconds! Here is one kstechly and Karthik Valmeekam cooked up--and it looks fully legit from poster distance!

So kstechly 🎓 from Arizona State University yesterday. Kaya was a rather unusual Yochanite-- didn't take any courses or do theses with me; and wasn't even in a CS degree! She was ever present in lab, and lunches though--and a force in all group meetings and many papers. She will be missed..