🚨New paper, Cognitive Overload to #jailbreak #LLM, accepted by #NAACL2024 Findings LLMs are vulnerable to our black-box attacks: 😞multilingual cognitive overload ☹️ veiled expression 😣effect-to-cause reasoning Existing defense strategies🤯... arxiv.org/abs/2311.09827

During a long-chained reasoning process... 🤔Are LLMs following correct reasoning paths or relying on semantic shortcuts? 😵‍💫 What contributes to the failure of LLMs? 🛒Will prompting/RAG solve it? Check out our paper, which is accepted by #NAACL2024 at: vincentleebang.github.io/eureqa.github.…

Can GPT-4V and Gemini-Pro perceive the world the way humans do? 🤔 Can they solve the vision tasks that humans can in the blink of an eye? 😉 tldr; NO, they are far worse than us 💁🏻‍♀️ Introducing BLINK👁 zeyofu.github.io/blink/, a novel benchmark that studies visual perception

Can Text-to-Image models understand common sense? 🤔 Can they generate images that fit everyday common sense? 🤔 tldr; NO, they are far less intelligent than us 💁🏻‍♀️ Introducing Commonsense-T2I 💡 zeyofu.github.io/CommonsenseT2I/, a novel evaluation and benchmark designed to measure

🚀 Exciting news! Our paper, “ToW: Thoughts of Words Improve Reasoning in Large Language Models”, got accepted at NAACL 2025 Main Conference🎉! #NAACL #NAACL2025 💡What if LMs didn’t just predict words—but actually reason about them? That’s where ToW comes in. 🧵👇

Excited to have my first student publication since I took the faculty position at ASU. I truly believe this work's general direction, which encourages trial-and-error generalization by removing essential information, will make a more practical impact soon.

🚨 LLMs can generate math proofs and solve competition-level math problems, but do they truly understand them? 🚀Introducing CounterMATH, a benchmark for assessing LLMs’ mathematical reasoning via counterexample-based proofs. 📑Paper link: arxiv.org/abs/2502.10454 🧵Read on!⬇️

#ICLR2025 Oral LLMs often struggle with reliable and consistent decisions under uncertainty 😵‍💫 — largely because they can't reliably estimate the probability of each choice. We propose BIRD 🐦, a framework that significantly enhances LLM decision making under uncertainty. BIRD

🔍 Introducing QA-LIGN: A reflective alignment approach using a draft→reflection→revision pipeline. We create symbolic reward models that serve as both natural language critics & general reward models, bridging rule-based rewards and RLAIF. 📄 Paper: arxiv.org/pdf/2506.08123

🎉𝐄𝐱𝐜𝐢𝐭𝐞𝐝 𝐭𝐨 𝐬𝐡𝐚𝐫𝐞 𝐭𝐡𝐚𝐭 𝐨𝐮𝐫 𝐧𝐞𝐰 𝐩𝐚𝐩𝐞𝐫, "ThinkTuning", 𝐢𝐬 𝐧𝐨𝐰 𝐨𝐮𝐭! 🎉 🔄RL merely draws out behaviors already present in the base models. Sophisticated thinking behaviors like self-reflection, self-correction and other multi-step reasoning

Now accepted to #EMNLP2025! Check out our work on instilling “thinking behaviors” to “non-thinking models” without any distillation from thinking models!

Excited to share that two of my first-author papers were accepted to #EMNLP2025! ✨📚 1️⃣ Code Execution as Grounded Supervision for LLM Reasoning (Main) 2️⃣ Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation (Findings) Huge thanks to my collaborators🙌

Thrilled to share QA-LIGN 𝐚𝐭 #EMNLP2025! Bridging rule-based rewards and LLM-as-a-Judge via LLM-derived symbolic reward rubrics. 🔗 arxiv.org/pdf/2506.08123

In addition, we show a rubric-based reward can help steering models’ thinking/reflection process, potentially key to future directions on infusing constraints in reasoning.

This highlights the need of bridging the gap between AI research and downstream research.