
Sunny Qin
@sunnytqin
Machine Learning PhD @ Harvard
ID: 1622819742523305984
07-02-2023 04:49:26
9 Tweet
49 Followers
87 Following

Transformer LMs get pretty far by acting like ngram models, so why do they even learn syntax? A new paper by Sunny Qin, me, and David Alvarez Melis uncovers the keys to grammar learning in a whirlwind tour of generalization, grokking, training dynamics, memorization, and random variation.



