New paper on teaching a language model how to use tools without human supervision! 🎉

Augmented Language Models: a Survey abs: arxiv.org/abs/2302.07842

Overcoming current LLMs limitations by augmenting them with better reasoning and tools is an exciting research direction. Check out our survey on this topic!

Our paper "Cross-Domain Image Captioning with Discriminative Finetuning" was accepted to CVPR! 🥳 We finetune a captioner (Clipcap and BLIP) using a discriminative objective with a reward provided by a frozen CLIP (no human refs used during finetuning!) and improve on ood data

Investigating prompt robustness across LMs in our upcoming ICLR paper, great work led by Nathanael Carraz!

I'm excited to introduce you Camoscio: an Italian instruction-tuned LLaMA, following Stanford Alpaca. The model should provide output of similar quality to GPT text-davinci-003 and has been finetuned by translating the Alpaca dataset to Italian. github.com/teelinsan/camo… 1/3

Fauno, il più grande #LLM in Italiano. Il nostro team ha sviluppato il modello utilizzando sistemi open source e a partire da LLaMA di Metà AI. Gentilmente offerto dal team #RSTLess di DIAGSapienza di Sapienza Università di Roma.

GIANLUCA MANCINI 👋👋👋

Today I'm (remotely 🥲) presenting our work at #CVPR2023 . It's on how multimodal LMs + RL improves captioning. We cast captioning as a discrimination game and tune a captioner using a frozen CLIP. You can find the paper here arxiv.org/abs/2304.01662

Augmented Language Models: a Survey Grégoire Mialon, Roberto Dessi, Maria Lomeli et al.. Action editor: Yujia Li. openreview.net/forum?id=jh7wH… #interpreter #alms #language

Liverpool’s reaction when they rob Roma TWICE in the 2018 Champions League semi final? “Who cares” ——————- Liverpool when they are robbed of a goal vs Tottenham in matchday 7 of a 38-game league season in 2023 “Replay the game”

1/9 huggingface.co/notdiamond/not… We’re open-sourcing a lightweight preview of our router that sends queries to either GPT-3.5 or GPT-4, maximizing accuracy while drastically reducing costs and latency. Routing to Gemini, Mistral, Claude, and Llama coming soon. A few quick points:

✊

Ago vive

🟨🟥🟨🟥

Excited to release Lightplane: A highly memory-efficient differentiable NeRF renderer and feature splatter. It renders FullHD image batches using <1GB of GPU memory and greatly scales up 3D deep models such as the Large Reconstruction Model. With Ang Cao, Justin Johnson, Andrea

I believe that cracking model routing is *the* next big challenge with LLMs. Tomas Hernando Kofman and the team have just released an amazing model for this. If you are developing any advance service on top of an LLM you probably need Not Diamond and should check it out

I'm not a Queen 🎙️ fan and I'm not a fan of prisons 👎, but I'm a fan of interdisciplinary LLM projects that are not about SOTA-chasing 🤖 🎉 Check our recent preprint led by the great Gian Maria Campedelli

✨ Meet #ResiDual, a novel perspective on the alignment of multimodal latent spaces! Think of it as a spectral "panning for gold" along the residual stream. It improves text-image alignment by simply amplifying task-related directions! 🌌🔍 arxiv.org/abs/2411.00246 [1/6]

If you’ve ever spent time updating a prompt, you know the struggle. The Not Diamond crew just dropped something that'll level up your app/product/model and save you a ton of time. Huge shoutout to Tomás and the team!