🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Humans' reading behaviors are different at the middle vs. the end of a sentence. Several theories have been proposed to explain what readers are doing on those final words to “wrap-up” the sentence. In our #acl2022 paper we take a look at this phenomenon! arxiv.org/abs/2203.17213

very excited to share our paper on reconstructing language from non-invasive brain recordings! we introduce a decoder that takes in fMRI recordings and generates continuous language descriptions of perceived speech, imagined speech, and possibly much more biorxiv.org/content/10.110…

OK, debates about the necessity or "priors" (or lack thereof) in learning systems are pointless. Here are some basic facts that all ML theorists and most ML practitioners understand, but a number of folks-with-an-agenda don't seem to grasp. Thread. 1/

New research-y project: Blueprints for Intelligence, a visual history of artificial neural networks from 1943 to 2020 philippschmitt.com/blueprints-for…

My new blog post takes a look at the state of multilingual AI. 🌍 How multilingual are current models in NLP, vision, and speech? 🏛 What are the recent contributions in this area? ⛰ What challenges remain and how we can we address them? ruder.io/state-of-multi…

Lots of folks are talking about *emergence* in Deep Learning as if it's a new thing, that happens only in large language models at scale. It's not! It has been happening for decades and in very small networks. 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵 🧵

🔥Our work has now been accepted to NeurIPS 2022 !! `Toward a realistic model of speech processing in the brain with self-supervised learning’: arxiv.org/abs/2206.01685 Let’s meet in New Orleans on Tue 29 Nov 2:30pm PST (Hall J #524). A recap of the 3 main results below 👇

Why does ChatGPT work so well? Is it “just scaling up GPT-3” under the hood? In this 🧵, let’s discuss the “Instruct” paradigm, its deep technical insights, and a big implication: “prompt engineering” as we know it may likely disappear soon:👇

Last year, we criticized LMs for performing “too well” with pathological prompts, and many papers have now shown similar results with corrupted ICL or CoT. In our new work, we find that *humans* also perform surprisingly well with irrelevant prompts! (But not misleading ones.) ⅕

The new scaling laws

LLMs such as ChatGPT and BLOOMZ claim that they are multilingual, but does this mean they can generate code-mixed data? Follow this 🧵 to find out. (1/N) Paper: arxiv.org/abs/2303.13592

Now that you’ve no doubt solved your Sunday crossword puzzle, looking to read about crosswords and linguistics? In The Atlantic theatlantic.com/science/archiv…, Scott AnderBois, Nicholas Tomlin, and I talk about what linguistics can tell us about crosswords and vice versa. Thread.

Domain experts often have intuitions about the algorithms that transformers may use to solve tasks, but do models actually use them? In new work with Thomas Serre and Brown NLP, we introduce circuit probing, a method for uncovering circuits that compute intermediate variables. (1/15)

Introducing Gemini 1.0, our most capable and general AI model yet. Built natively to be multimodal, it’s the first step in our Gemini-era of models. Gemini is optimized in three sizes - Ultra, Pro, and Nano Gemini Ultra’s performance exceeds current state-of-the-art results on

Compositional generalization is a major challenge for neural networks. In a #NeurIPS2023 spotlight paper with Thomas Serre and Brown NLP, we ask whether neural networks learn the types of representations that are a prerequisite for compositionality! (1/14)

a nyt connections game with "orca", "llama", "alpaca" and "vicuna" would such a dogwhistle

Excited to share Penzai, a JAX research toolkit from Google DeepMind for building, editing, and visualizing neural networks! Penzai makes it easy to see model internals and lets you inject custom logic anywhere. Check it out on GitHub: github.com/google-deepmin…

Why did only humans invent graphical systems like writing? 🧠✍️ In our new paper at CogSci Society, we explore how agents learn to communicate using a model of pictographic signification similar to human proto-writing. 🧵👇

THE WAY OF CODE, a project by @rickrubin in collaboration with Anthropic: