[LG] RSQ: Learning from Important Tokens Leads to Better Quantized LLMs Y Sung, P Yadav, J Li, J Yoon... [UNC at Chapel Hill] (2025) arxiv.org/abs/2503.01820

🚨 We introduce ✨ Symbolic-MoE ✨ which uses skill-based instance-level recruiting to dynamically combine LLMs, allowing three 7-8B LLMs to beat GPT4o-mini and Llama3.3 70B across challenging + diverse reasoning tasks (MMLU-Pro, AIME, GPQA, MedMCQA) while running on 1 GPU! Key

🚨 Introducing VEGGIE 🥦—a unified, end-to-end, and versatile instructional video generative model. Current video editing methods struggle with: 1. Understanding direct user instructions 2. Handling diverse editing skills in one model 3. balancing multiple training

Life Update: I am happy to share the news that I will be an Assistant Professor at the National Taiwan University EE department! I am very grateful for this opportunity to be back in my home country, especially at the university where I was an undergrad! 1/4

🚨Announcing TaCQ 🚨 a new mixed-precision quantization method that identifies critical weights to preserve. We integrate key ideas from circuit discovery, model editing, and input attribution to improve low-bit quant., w/ 96% 16-bit acc. at 3.1 avg bits (~6x compression)

Excited to share my first paper as first author: "Task-Circuit Quantization” 🎉 I led this work to explore how interpretability insights can drive smarter model compression. Big thank you to Elias Stengel-Eskin, Yi Lin Sung (on job market), and Mohit Bansal for mentorship and collaboration. More to come!

Introducing TaCQ: a new mixed-precision quantization method that preserves performance at low bit-widths (2–3 bits). 1⃣Keeps task-critical weights in 16-bit and quantizes the rest 2⃣Uses a novel saliency metric inspired by model editing and interpretability 3⃣Beats the strongest

🚀New paper out - We present Video-MSG (Multimodal Sketch Guidance), a novel planning-based training-free guidance method for T2V models, improving control of spatial layout and object trajectories. 🔧 Key idea: • Generate a Video Sketch — a spatio-temporal plan with

🚨Real-world retrieval is messy: queries can be ambiguous, or documents may conflict/have incorrect/irrelevant info. How can we jointly address all these problems? We introduce: ➡️ RAMDocs, a challenging dataset with ambiguity, misinformation, and noise. ➡️ MADAM-RAG, a

🚨 Introducing our Transactions on Machine Learning Research paper “Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation” W:nt UnLOK-VQA, a benchmark to evaluate unlearning in vision-and-language models—where both images and text may encode sensitive or private

Sharing some personal updates 🥳: - I've completed my PhD at UNC Computer Science! 🎓 - Starting Fall 2026, I'll be joining the Computer Science dept. at Johns Hopkins University (JHU Computer Science) as an Assistant Professor 💙 - Currently exploring options + finalizing the plan for my gap year (Aug

Thrilled to share that I’ll be joining the College of Computing and Data Science at Nanyang Technological University (NTU) (NTU Singapore) as an Assistant Professor, starting in August 2025 🇸🇬🥳 I’ll continue my research on building trustworthy and continually adaptable multimodal AI,

Excited to share Video-Skill-CoT🎬🛠️– a new framework for domain-adaptive video reasoning with skill-aware Chain-of-Thought (CoT) supervision! ⚡️Key Highlights: ➡️ Automatically extracts domain-specific reasoning skills from questions and organizes them into a unified taxonomy,

We know Attention and its linear-time variants, such as linear attention and State Space Models. But what lies in between? Introducing Log-Linear Attention with: - Log-linear time training - Log-time inference (in both time and memory) - Hardware-efficient Triton kernels

🎉 Excited to share that TaCQ (Task-Circuit Quantization), our work on knowledge-informed mixed-precision quantization, has been accepted to #COLM2025 Conference on Language Modeling! Happy to see that TaCQ was recognized with high scores and a nice shoutout from the AC – big thanks to Elias Stengel-Eskin

🎉 Very excited to see TaCQ — our work on task-conditioned mixed-precision quantization that draws on interpretability methods — accepted to Conference on Language Modeling #COLM2025 with strong scores and a nice shoutout from the AC! Kudos to Hanqi on leading this effort!

🚨Introducing Video-RTS: Resource-Efficient RL for Video Reasoning with Adaptive Video TTS! While RL-based video reasoning with LLMs has advanced, the reliance on large-scale SFT with extensive video data and long CoT annotations remains a major bottleneck. Video-RTS tackles

🤔 Can we bridge MLLMs and diffusion models more natively and efficiently, by having MLLMs produce patch-level CLIP latents already aligned with their visual encoders, while fully preserving MLLM's visual reasoning capabilities? Introducing Bifrost-1: 🌈 > High-Fidelity

📢 Introducing RotBench, which tests whether SoTA MLLMs (e.g., GPT-5, GPT-4o, o3, Gemini-2.5-pro) can identify the rotation of input images (0°, 90°, 180°, and 270°). Even frontier MLLMs struggle at this spatial reasoning task that humans solve with >98% Acc. ➡️ Models struggle