We got the #ACL2024 Outstanding Paper for “LM-Steer: Word Embeddings Are Steers for Language Models”! A big shoutout and congrats to our amazing leader Chi Han Chi Han and Heng Ji, and to our wonderful team Jialiang Xu @YiFung10 Chenkai Sun Nan Jiang Tarek!

🎖Excited that "LM-Steer: Word Embeddings Are Steers for Language Models" became my another 1st-authored Outstanding Paper #ACL2024 (besides LM-Infinite) We revealed steering roles of word embeddings for continuous, compositional, efficient, interpretable& transferrable control!

"Only-IF: Revealing the Decisive Effect of Instruction Diversity on Generalization" arxiv.org/pdf/2410.04717 We isolated 'instruction-following' ability (apart from complex reasoning like math) and designed various controlled experiments to show that -

👾 Introducing AgentOccam: Automating Web Tasks with LLMs! 🌐 AgentOccam showcases the impressive power of Large Language Models (LLMs) on web tasks, without any in-context examples, new agent roles, online feedback, or search strategies. 🏄🏄🏄 🧙 Link: arxiv.org/abs/2410.13825

I am at #EMNLP2024! I will present our work "Why Does New Knowledge Create Messy Ripple Effects in LLMs? " on Web 10:30am. Thanks to all the collaborators Heng Ji Zixuan Zhang Chi Han Manling Li Looking forward to have a chat! Paper Link: arxiv.org/pdf/2407.12828

📢Our 2nd Knowledgeable Foundation Model workshop will be at AAAI 25! Submission Deadline: Dec 1st Thanks to the wonderful organizer team Zoey Sha Li Mor Geva Chi Han Xiaozhi Wang Shangbin Feng @silingao and advising committee Heng Ji Isabelle Augenstein Mohit Bansal !

Presenting our LISA paper at NeurIPS 2024😆 - Dec. 13 at 4:30 pm (Friday afternoon) - West Ballroom A-D #5708 Fine-tuning 7B in a single GPU❔ Randomly freezing ~90% self-attention layers every 5-20 iterations allows that!🚀It is - 3x Fast - Memory-efficient - Good at

📢📈 I’m on the 2025 faculty job market! I've been incredibly grateful to work with inspiring advisors, mentors & peers. 💡My research, AI4Scientists🔬, accelerates & democratizes the research lifecycle by: 1️⃣ Few-shot scientific knowledge acquisition 2️⃣ Domain-aware scientific

👽New release: We filter out educationally valuable web data rather than using arXiv papers to continually pre-train a specialist Astro LLM. Big thanks to the first author Eric Modesitt for the project leadership! 💪 Great ideas and strong execution—you're an amazing undergrad!

Thank you so much, Heng Heng Ji! I'm incredibly grateful for your guidance and support throughout this journey and honored to receive the fellowship to continue working on exciting projects. Looking forward to more collaborations with you and Avi Sil Avi Sil @ ACL 2025 (virtually)! 🙌

🙌 Happy New Year everyone! 🤖 New preprint: TinyHelen's First Curriculum: Training and Evaluating Tiny Language Models in a Simpler Language Environment 🤖 We train and evaluate tiny language models (LMs) using a novel text dataset with systematically simplified vocabularies and

(1/4) Excited to share that our position paper "Towards LifeSpan Cognitive Systems" has been accepted to TMLR! As "agents" dominate the AI landscape in 2025, we push the boundaries by envisioning LSCS — AI systems that go beyond personal assistants to lifespan cognitive systems.

Sharing our recent study - "The Best Instruction-Tuning Data are Those That Fit". We introduce a practical, lightweight approach that selects in-distribution data for supervised fine-tuning—yielding better models with less data & compute w. surprising simplicity and efficiency!

Welcome to my #AAAI2025 Tutorial, "The Quest for A Science of LMs," today! Time: Feb 26, 2pm-3:45pm Location: Room 113A, Pennsylvania Convention Center Website: glaciohound.github.io/Science-of-LLM… Underline: underline.io/events/487/sch…

Thanks, Heng Ji, for sharing our recent work! In this paper, we investigate how LLMs achieve positional flexibility, such as understanding shuffled sentences and reading long context (a phenomenon also observed in humans). This paper uncovers the hidden computational

New Benchmark for LLM Multi-Turn-Instruct Following!🚀 Can LLMs handle multiple entangled instructions effectively? Our comprehensive Multi-Turn-Instruct dataset evaluates LLMs’ ability to 📜track, 🔄reason on, and ⚖️ resolve conflicts across multiple turns of instructions in