🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Will be presenting our work on fast finite-width NTK today at #icml2022 - please come to our talk at 10:55 EDT, or the poster session at 18:30 EDT! icml.cc/virtual/2022/s…

Good morning! Another day, another chance to help African researchers get the research opportunities they deserve. We have hundreds of likes and RTs but only 45 donations so far. Any amount helps! Skip your latte this morning and give that $10 to make a difference :)

"Our work builds a foundation for future research..." Our work:

For a moment thought it was 42°C outside, but thankfully this is just my CPU temperature widget. Outside is 43°

Quadratic scaling in the number of pixels is a huge bottleneck of the NNGP/NTK. Very excited about _orders-of-magnitude_ speedups with sketching! See arxiv.org/abs/2209.04121 as well as many new nonlinearities in NT neural-tangents.readthedocs.io/en/latest/stax…

Using the excellent github.com/google/neural-… library by Roman Novak et al. we show for 2D and 3D data why and how intrinsic neural fields work. Intrinsic neural fields aggregate information on the manifold!

Due to #Russian missile strikes against civilian infrastructure, my elderly relatives (80+ y/o) have no electricity, water or heating. It's freezing cold. They barely have any food b/c local supermarkets are often closed. Mobile signal is also down, so I can't even reach them.

A Kyiv family has come to a gas station (where there is electricity) specifically to be able to plug in the inhaler their little girl needs to breathe

This paper on "Fast Neural Kernel Embeddings for General Activations" will be presented tomorrow, Tue 29 Nov 11 a.m. CST - 1 p.m. CST, Hall J #806

Today at 11am CT, Hall J #806 we are presenting our paper on infinite width neural network kernels! We have methods to compute NTK/NNGP for extended set of activations + sketched embeddings for efficient approximation (100x) for compute intensive conv kernels! See you there!

Sharing some highlights from our work on small-scale proxies for large-scale Transformer training instabilities: arxiv.org/abs/2309.14322 With fantastic collaborators Peter J. Liu, Lechao Xiao, Katie Everett, many others (see final tweet!), Jaehoon Lee, Justin Gilmer, Simon Kornblith! (1/15)

Happy to see our paper on adversarial arithmetic released! (arxiv.org/abs/2311.07587)

we have trained a humanoid transformer with large-scale reinforcement learning in simulation and deployed it to the real world zero-shot

Excited to announce our new work on using synthetic data for improving mathematical problem solving and code generation in LLMs! arxiv: arxiv.org/abs/2312.06585 A small amount of fine-tuning can lead to large gains (>6% on Hendrycks MATH with Palm-2)

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.

We recently open-sourced a relatively minimal implementation example of Transformer language model training in JAX, called NanoDO. If you stick to vanilla JAX components, the code is relatively straightforward to read -- the model file is <150 lines. We found it useful as a

We've gotten some great questions about the notion of alignment in our width-scaling parameterization paper! arxiv.org/abs/2407.05872 A deep dive into the alignment metric and intuition 🧵 [1/16]

1/5. Excited to share a spicy paper, "Rethinking conventional wisdom in machine learning: from generalization to scaling", arxiv.org/pdf/2409.15156. You might love it or dislike it! NotebookLM: notebooklm.google.com/notebook/43f11… While double-descent (generalization-centric,

Mr. President, Ukraine did not “start” this war. Russia launched an unprovoked and brutal invasion claiming hundreds of thousands of lives. The Road to Peace must be built on the Truth.🇺🇸🇺🇦 “Russia Invades Ukraine in Largest European Attack Since WWII” Fox News (February 24,

Democracy dies in cringe