🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Also giving a contributed talk on the learning-theoretic complexity and optimality of ICL at the Theoretical Foundations of Foundation Models happy to share our results with the ML theory and LLM community!

Both joint work with my wonderful supervisor Taiji Suzuki

Incredibly honored and humbled by the overwhelming response to my tutorial, and thank you everyone who attended in person. Truly heartwarming to hear how much you enjoyed it. Many have been asking for a recording, and I prepared one with my own subtitles youtu.be/yBL7J0kgldU

I'm honored to receive the best paper award at Theoretical Foundations of Foundation Models on the last day of ICML Conference in Vienna!

Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics ift.tt/5ZMQJLk

Our paper is finally out on arXiv!

'High-Dimensional Probability' - Vershynin (2024, PDF): math.uci.edu/~rvershyn/pape… Full course on high-dimensional probability with 41 video lectures & 13 problem sets provided: math.uci.edu/~rvershyn/teac…

Our paper on statistical complexity and optimality of in-context learning of deep transformers has been accepted to NeurIPS! arxiv.org/abs/2408.12186

"Deep Learning Theory" by Matus Telgarsky mjt.cs.illinois.edu/dlt/index.pdf

Visit our poster #2035 NeurIPS Conference on Dec 11 (Wed) from 11 a.m. Also presenting two papers on chain-of-thought and neural IV regression M3L Workshop @ NeurIPS 2024 !

Here is a great Youtube video by Charles Riou that effectively explains our recent paper with Juno KIM on chain-of-thought accepted by ICLR2025. Check it! Kim&Suzuki: Transformers Provably Solve Parity Efficiently with Chain of Thought. ICLR2025. youtu.be/pj-GEBU2iVs?si…

Simons Institute Workshop: "Future of LLMs and Transformers": 21 talks Monday - Friday next week. simons.berkeley.edu/workshops/futu…

Graduated with my master's from the University of Tokyo!🇯🇵 I was honored to receive the graduate school dean's award for outstanding research and to deliver a speech at the ceremony! Deeply grateful to my supervisor Taiji Suzuki🙏 i.u-tokyo.ac.jp/news/topics/20…

I will be joining UC Berkeley as an EECS PhD student this fall!

Visit our poster at 10am Friday! #ICLR2025

Our paper studying dynamics of a simple Markov model for CoT reasoning has been accepted to #ICML2025 ! (reposting a nice summary↓)

I will also be presenting my ICLR oral paper at the NLP Colloquium on the 21st (talk in Japanese):

LLMs can solve complex tasks that require combining multiple reasoning steps. But when are such capabilities learnable via gradient-based training? In our new COLT 2025 paper, we show that easy-to-hard data is necessary and sufficient! arxiv.org/abs/2505.23683 🧵 below (1/10)

What if an LLM could update its own weights? Meet SEAL🦭: a framework where LLMs generate their own training data (self-edits) to update their weights in response to new inputs. Self-editing is learned via RL, using the updated model’s downstream performance as reward.