🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Self-attention is actually doing kernel PCA under the hood - now we can make it robust So how does Self-attention works: it's kernel PCA in disguise, as proposed in this paper. 🤔 Original Problem: Self-attention in transformers has been developed through heuristics and

Spent the weekend hacking together Exa embeddings over 4500 NeurIPS 2024 papers - neurips.exa.ai Let's you: - do otherwise impossible searches ("transformer architectures inspired by neuroscience") - explore a 2D t-SNE plot - chat with Claude about multiple papers

Introducing An Evolved Universal Transformer Memory sakana.ai/namm Neural Attention Memory Models (NAMMs) are a new kind of neural memory system for Transformers that not only boost their performance and efficiency but are also transferable to other foundation models,

📢 Attending #NeurIPS2024 ? Come by our workshop on open-world agents! everything 👉 owa-workshop.github.io Put your questions for the panel here: forms.gle/XLumiMHAWjwydy… Our speakers & panelists lining up: Sherry Yang Tao Yu Ted Xiao Natasha Jaques Jiajun Wu

NeurIPS Conference 2024 reimagined with AI !! - summaries for instant insights 🧠 - easy-to-understand audio podcasts 🎙️ - quick links to NeurIPS Proc., Hugging Face & more 🌐 - Full papers, topic & affiliation filters 📂 All your research needs, in one hub. Dive in now! 👇

Training LLMs to Reason in a Continuous Latent Space Meta presents Coconut (Chain of Continuous Thought), a novel paradigm that enables LLMs to reason in continuous latent space rather than natural language. Coconut takes the last hidden state of the LLM as the reasoning state

Kosmas Marinakis Εδώ το βίντεο:

I think the community is excited about DeepSeek v3 not because it's yet another powerful model but because it's a story of human ingenuity in the face of constraints. Despite all the restrictions due to export control and limited budget, the humans of DeepSeek have created a

Ενώ το ραντεβού μας στις #28_Φλεβαρη για τα #Τεμπη_συγκαλυψη στους δρόμους είναι σε λίγες ώρες και το πόρισμα του #ΕΟΔΑΣΑΑΜ για τα #τεμπη_εγκλημα μίλησε για 2,5 τόνους εύφλεκτης "άγνωστης" ουσίας και ασύλληπτες και εγκληματικές παραλείψεις (#Justice_for_Tempi), με τον… 1/36

This paper is pretty cool: The Belief State Transformer Very simple technique and fast to train, makes transformers (or other seq models) better at modelling state and can additionally condition on the end! I wonder what this is like for RL, we might condition on high end reward!

Το χειροκρότημα των βολεμένων στην καρέκλα τους Βουλευτών της Νέας Δημοκρατίας για την απόρριψη της πρότασης δυσπιστίας, επισφράγισε τη συμμετοχή τους στα όσα σκανδαλώδη απο τη πρώτη στιγμή οργανώνει, μεθοδεύει και διαπράττει ο Πρωθυπουργός και οι Υπουργοί τους. Κανένας από

Introducing Mistral Small 3.1. Multimodal, Apache 2.0, outperforms Gemma 3 and GPT 4o-mini. mistral.ai/news/mistral-s…

☄️ GRPO now scales to 70B+ models with multi-node training and super-fast performance. Install the latest v0.16 version of TRL pip install trl With all these the freshest features and optimizations that we've added, you can train up to 60 times faster! More details in the

📢LLM and RL folks! 📢 No good RL algorithm for credit assignment for multi-turn LLM agents on reasoning-heavy tasks? Do not even have a good benchmark for studying it? In SWEET-RL, we give you both (a vibe coding benchmark and SWEET algorithm). A thread 🧵(1/n)

"Ας αφήσουμε τα παιδιά του Μωάμεθ να αποτελειώσουν τα παιδιά του Ροβεσπιέρου" (Παλαιών Πατρών Γερμανός, 1820): Πως αντιμετωπίστηκαν πολλοί ήρωες του 1821 απο ντόπιους αστούς, κοτζαμπάσηδες, αρχιρασοφόρους κ.α που πήραν τελικά την θέση των Οθωμανών στην εκμετάλλευση του λαού; Ο

I am pleased to share the full set of videolectures, slides, textbook, and other supporting material of the 7th offering of my Reinforcement Learning class at ASU, which was completed two days ago; check web.mit.edu/dimitrib/www/R…

Tim Rocktäschel’s keynote talk at #ICLR2025 about Open-Endedness and AI. “Almost no prerequisite to any major invention was invented with that invention in mind.” “Basically almost everybody in my lab at UCL and at DeepMind have read this book: Why Greatness Cannot Be Planned.”

Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known

Introducing Reinforcement-Learned Teachers (RLTs): Transforming how we teach LLMs to reason with reinforcement learning (RL). Blog: sakana.ai/rlt Paper: arxiv.org/abs/2506.08388 Traditional RL focuses on “learning to solve” challenging problems with expensive LLMs and

Ahahahaha, the James Webb Space Telescope continues to deliver massive L’s for astrophysics. A new paper shows that the “Cosmic Microwave Background Radiation” can be explained entirely by the energy of recently discovered Early Mature Galaxies — massive galaxies that the JWST