🔥New Video🔥 I delve (ha!) into Byte Latent Transformer: Patches Scale Better Than Tokens where the authors do away with tokenization and create an LLM architecture that operates on dynamically sized "patches" instead of tokens. By controlling the patch size, they gain a level

Wrote about some of my favourite papers over the past year or so and some research directions that I am excited about in 2025 As a bonus, I think it's a good overview for someone to catch up on the current state of the art :)

New research from Meta FAIR — Meta Memory Layers at Scale. This work takes memory layers beyond proof-of-concept, proving their utility at contemporary scale ➡️ go.fb.me/3lbt4m

A friend in LA just took this video

Our science team has started working on fully reproducing and open-sourcing R1 including training data, training scripts,... Full power of open source AI so that everyone all over the world can take advantage of AI progress! Will help debunk some myths I’m sure too. Thanks

We are excited to introduce Mercury, the first commercial-grade diffusion large language model (dLLM)! dLLMs push the frontier of intelligence and speed with parallel, coarse-to-fine text generation.

Meta and NVIDIA have teamed up to supercharge vector search on GPUs by integrating NVIDIA cuVS into Faiss v1.10, Meta’s open-source library for similarity search. This collaboration brings groundbreaking performance improvements: 🔹 IVF indexing: NVIDIA cuVS boosts build times

Rob Fergus is the new head of Meta-FAIR! FAIR is refocusing on Advanced Machine Intelligence: what others would call human-level AI or AGI. linkedin.com/posts/rob-ferg…

🎉 Excited to share: "𝐌𝐢𝐱𝐭𝐮𝐫𝐞-𝐨𝐟-𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 (𝐌𝐨𝐓)" has been officially accepted to TMLR (March 2025) and the code is now open-sourced! 📌 GitHub repo: github.com/facebookresear… 📄 Paper: arxiv.org/abs/2411.04996 How can we reduce pretraining costs for

Factuality and hallucination is a big problem in foundation models. We demonstrate that it’s possible to improve long form factuality by 35% without losing helpfulness/ ability to provide detailed response with long CoT, new reward for GRPO and tricks to stop reward hacking

🏆 We're thrilled to announce that Meta FAIR’s Brain & AI team won 1st place at the prestigious Algonauts 2025 brain modeling competition. Their 1B parameter model, TRIBE (Trimodal Brain Encoder), is the first deep neural network trained to predict brain responses to stimuli

New research from FAIR- Active Reading: a framework to learn a given set of material with self-generated learning strategies for generalized and expert domains(such as Finance). Absorb significantly more knowledge than vanilla finetuning and usual data augmentations strategies