Poster at Compositional Learning Workshop @ NeurIPS2024 Exploring why simplicity bias in deep learning often aligns with compositionality. A fresh perspective through the lens of learning dynamics. (1/5) 🗓️Sunday, Dec 15 ⏰8:30 - 17:00 📍West Meeting Room 118-120

For those still at NeurIPS, check out my student's posters at the OPT workshop at 3pm: - Line search with constant sharpness. - Proof line search can beat acceleration. - Allowing negative step sizes? - BCD for deep nets? - Normalization in graph neural net codes makes no sense.

If you are interested in generalization in deep learning, you can find our poster in the poster session right now at #AAAI2025 : )

Happy to see sparse attention methods taking off! We've also worked on sparse attention for learning on graphs (Exphormer, Spexphormer papers) — convincing reviewers about sparse attention (and not just flash attention) wasn’t easy.

What makes Graph Transformers a perfect fit for relational data? In our latest blog, Federico López breaks down how Kumo integrates Graph Transformers to elevate predictive modeling—eliminating the need for complex pipelines and manual feature engineering. kumo.ai/research/graph…

📢Curious why your LLM behaves strangely after long SFT or DPO? We offer a fresh perspective—consider doing a "force analysis" on your model’s behavior. Check out our #ICLR2025 Oral paper: Learning Dynamics of LLM Finetuning! (0/12)

Can’t attend ICLR 🇸🇬 due to visa issues but Chandan Reddy will have the oral presentation of *LLM-SR*on Friday 👇🏻 + see our new preprint on benchmarking capabilities of LLMs for scientific equation discovery *LLM-SRBench*: arxiv.org/abs/2504.10415

🚀 New tutorial on Graph Transformers! We introduce a powerful architecture combining the strengths of GNNs and Transformers for structured data. Learn how GTs open new possibilities for graph learning. kumo.ai/research/intro… #GraphTransformers #MachineLearning #AI #GNN

Huge congrats to my labmate Yi (Joshua) Ren and my supervisor Danica Sutherland for receiving an Outstanding Paper Award at ICLR 2026 for their work on Learning Dynamics of LLM Finetuning! So proud to see their amazing research recognized 👏🔥

Thank you Graph Signal Processing Workshop 2025 for the opportunity to give the morning keynote! It was a pleasure to present my work on Exphormer and Spexphormer models to such an insightful audience.

1/ At Valence Labs, Recursion's AI research engine, we’re focused on advancing drug discovery outcomes through cutting-edge computational methods Today, we're excited to share our vision for building virtual cells, guided by the predict-explain-discover framework 🧵

Loved working on the TxPert project! It's also exciting to see my PhD work on Graph Transformers (Exphormer model) finding such meaningful application in a critical real-world task.

1/ Introducing TxPert: a new model that predicts transcriptional responses across diverse biological contexts It’s designed to generalize across unseen single-gene perturbations, novel combinations of gene perturbations, and even new cell types 🧵

What makes attention the critical component for most advances in LLMs and what holds back long-term memory modules (RNNs)? Can we strictly generalize Transformers? Presenting Atlas (A powerful Titan): a new architecture with long-term in-context memory that learns how to

امیدوارم که خانواده هاتون سالم باشن، چشمهامون اشک آلوده برای ایران، برای کودکان و ادم های بیگناه که به خاک و خون کشیده شدن. برای همه دانشجویان ایرانی در زمینه هوش مصنوعی و بیوانفرماتیک و زیست شناسی محاسباتی که دچار مشکل شدن برای ادامه تحصیل یا تحقیقاتشون، به من ایمیل بزننین برای

Can we align language models using observational data instead of costly experiments like A/B tests? In my latest research, we find that historical observational data *does* carry useful signals, but without causal care, models can learn the wrong things. Thread 🧵

We’re presenting our work today at #ICML2025, Poster Session 4 West, W-214! If you are interested in computer vision reasoning and multimodal LLMs come visit us!

Fun times at ICML. Graph learning dinner, position poster gang, theory, and graph learning hike. :)

Great attending ICML this year, really enjoyed connecting with the graph learning community! Thanks a lot to Christopher Morris for organizing the lovely dinner :)

Enjoyed giving our tutorial on Geometric & Topological Deep Learning at IEEE MLSP 2025 alongside Semih Cantürk. Loving the Istanbul vibes and the amazing food here! ✅