Roman Novak (@aromannovak) 's Twitter Profile
Roman Novak

@aromannovak

Research Scientist @OpenAI, ex @GoogleDeepMind, Google Brain 🇺🇦

ID: 1285731025566552065

linkhttps://scholar.google.com/citations?user=LWvgl-8AAAAJ calendar_today22-07-2020 00:19:07

96 Tweet

404 Followers

267 Following

Roman Novak (@aromannovak) 's Twitter Profile Photo

Will be presenting our work on fast finite-width NTK today at #icml2022 - please come to our talk at 10:55 EDT, or the poster session at 18:30 EDT! icml.cc/virtual/2022/s…

Will be presenting our work on fast finite-width NTK today at #icml2022 - please come to our talk at 10:55 EDT, or the poster session at 18:30 EDT! icml.cc/virtual/2022/s…
Rosanne Liu (@savvyrl) 's Twitter Profile Photo

Good morning! Another day, another chance to help African researchers get the research opportunities they deserve. We have hundreds of likes and RTs but only 45 donations so far. Any amount helps! Skip your latte this morning and give that $10 to make a difference :)

Roman Novak (@aromannovak) 's Twitter Profile Photo

Quadratic scaling in the number of pixels is a huge bottleneck of the NNGP/NTK. Very excited about _orders-of-magnitude_ speedups with sketching! See arxiv.org/abs/2209.04121 as well as many new nonlinearities in NT neural-tangents.readthedocs.io/en/latest/stax…

LukasKoestler (@koestlerlukas) 's Twitter Profile Photo

Using the excellent github.com/google/neural-… library by Roman Novak et al. we show for 2D and 3D data why and how intrinsic neural fields work. Intrinsic neural fields aggregate information on the manifold!

Using the excellent github.com/google/neural-… library by <a href="/ARomanNovak/">Roman Novak</a> et al. we show for 2D and 3D data why and how intrinsic neural fields work. Intrinsic neural fields aggregate information on the manifold!
Dr Bohdana Kurylo (@bohdanakurylo) 's Twitter Profile Photo

Due to #Russian missile strikes against civilian infrastructure, my elderly relatives (80+ y/o) have no electricity, water or heating. It's freezing cold. They barely have any food b/c local supermarkets are often closed. Mobile signal is also down, so I can't even reach them.

UkraineWorld (@ukraine_world) 's Twitter Profile Photo

A Kyiv family has come to a gas station (where there is electricity) specifically to be able to plug in the inhaler their little girl needs to breathe

A Kyiv family has come to a gas station (where there is electricity) specifically to be able to plug in the inhaler their little girl needs to breathe
Amin Karbasi (@aminkarbasi) 's Twitter Profile Photo

This paper on "Fast Neural Kernel Embeddings for General Activations" will be presented tomorrow, Tue 29 Nov 11 a.m. CST - 1 p.m. CST, Hall J #806

Jaehoon Lee (@hoonkp) 's Twitter Profile Photo

Today at 11am CT, Hall J #806 we are presenting our paper on infinite width neural network kernels! We have methods to compute NTK/NNGP for extended set of activations + sketched embeddings for efficient approximation (100x) for compute intensive conv kernels! See you there!

Mitchell Wortsman (@mitchnw) 's Twitter Profile Photo

Sharing some highlights from our work on small-scale proxies for large-scale Transformer training instabilities: arxiv.org/abs/2309.14322 With fantastic collaborators Peter J. Liu, Lechao Xiao, Katie Everett, many others (see final tweet!), Jaehoon Lee, Justin Gilmer, Simon Kornblith! (1/15)

Sharing some highlights from our work on small-scale proxies for large-scale Transformer training instabilities: arxiv.org/abs/2309.14322

With fantastic collaborators <a href="/peterjliu/">Peter J. Liu</a>, <a href="/Locchiu/">Lechao Xiao</a>, <a href="/_katieeverett/">Katie Everett</a>, many others (see final tweet!), <a href="/hoonkp/">Jaehoon Lee</a>, <a href="/jmgilmer/">Justin Gilmer</a>, <a href="/skornblith/">Simon Kornblith</a>!

(1/15)
Ilija Radosavovic (@ir413) 's Twitter Profile Photo

we have trained a humanoid transformer with large-scale reinforcement learning in simulation and deployed it to the real world zero-shot

Avi Singh (@avisingh599) 's Twitter Profile Photo

Excited to announce our new work on using synthetic data for improving mathematical problem solving and code generation in LLMs! arxiv: arxiv.org/abs/2312.06585 A small amount of fine-tuning can lead to large gains (>6% on Hendrycks MATH with Palm-2)

Excited to announce our new work on using synthetic data for improving mathematical problem solving and code generation in LLMs!

arxiv: arxiv.org/abs/2312.06585

A small amount of fine-tuning can lead to large gains (&gt;6% on Hendrycks MATH with Palm-2)
Jascha Sohl-Dickstein (@jaschasd) 's Twitter Profile Photo

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.

Peter J. Liu (@peterjliu) 's Twitter Profile Photo

We recently open-sourced a relatively minimal implementation example of Transformer language model training in JAX, called NanoDO. If you stick to vanilla JAX components, the code is relatively straightforward to read -- the model file is <150 lines. We found it useful as a

Katie Everett (@_katieeverett) 's Twitter Profile Photo

We've gotten some great questions about the notion of alignment in our width-scaling parameterization paper! arxiv.org/abs/2407.05872 A deep dive into the alignment metric and intuition đź§µ [1/16]

We've gotten some great questions about the notion of alignment in our width-scaling parameterization paper! arxiv.org/abs/2407.05872

A deep dive into the alignment metric and intuition đź§µ [1/16]
Lechao Xiao (@locchiu) 's Twitter Profile Photo

1/5. Excited to share a spicy paper, "Rethinking conventional wisdom in machine learning: from generalization to scaling", arxiv.org/pdf/2409.15156. You might love it or dislike it! NotebookLM: notebooklm.google.com/notebook/43f11… While double-descent (generalization-centric,

1/5. Excited to share a spicy paper, "Rethinking conventional wisdom in machine learning: from generalization to scaling", arxiv.org/pdf/2409.15156.  
You might love it or dislike it!  
NotebookLM: notebooklm.google.com/notebook/43f11…
While double-descent (generalization-centric,
Mike Pence (@mike_pence) 's Twitter Profile Photo

Mr. President, Ukraine did not “start” this war. Russia launched an unprovoked and brutal invasion claiming hundreds of thousands of lives. The Road to Peace must be built on the Truth.🇺🇸🇺🇦 “Russia Invades Ukraine in Largest European Attack Since WWII” Fox News (February 24,

Mr. President, Ukraine did not “start” this war. Russia launched an unprovoked and brutal invasion claiming hundreds of thousands of lives. The Road to Peace must be built on the Truth.🇺🇸🇺🇦

“Russia Invades Ukraine in Largest European Attack Since WWII” <a href="/FoxNews/">Fox News</a> (February 24,