
Quotient AI
@quotientai
Weโre building an advanced AI development and evaluation platform. Join our test kitchen: discord.gg/YeJzANpntv
ID: 1679732770367578115
http://www.quotientai.co 14-07-2023 06:01:36
68 Tweet
211 Followers
6 Following


Announcing our speakers for the Retrieval + Search track! โ ๏ธPSA: Tix nearly sold out, get em here: ti.to/software-3/ai-โฆโฆ Featuring: Aman, Former Founder, Harvey Jerry Liu, CEO, LlamaIndex ๐ฆ Julia Neagu, CEO, Quotient AI changhiskhan, CEO, LanceDB

Weโre heading back to AI Engineer! Deanna Emery (founding AI Engineer at Quotient AI) and Maitar Asher ๐๏ธ (Head of Eng tavily) are speaking evaluating AI search. If you're building AI search, don't miss it.

Canโt wait to be back at AI Engineer! Iโm teaming up with Maitar Asher ๐๏ธ from tavily to talk about evaluating AI search. Weโre sharing a practical eval framework, lessons from real-world deployments, and never-seen-before benchmark results. Hope to see you there!

HypoEval evaluators (github.com/ChicagoHAI/Hypโฆ) are now incorporated into judges from Quotient AI โ check it out at github.com/quotient-ai/juโฆ!

HypoEval is now available in Quotient AI's OSS judges! It uses SOTA hypothesis generation with just 30 human annotations to create decomposed rubrics, enabling LLMs to score criteria clearly. Beats fine-tuned models (w/ 3x more labels). Thanks Mingxuan (Aldous) Li for contributing!

detections go brrr One week in, Quotient AI Detections has processed 20M+ tokens, analyzed tens of thousands of logs, and caught thousands of hallucinations across real AI production apps. Still a long way to go, but we're committed to giving builders SOTA AI monitoring.



it was a pleasure speaking at AI Engineer with Maitar Asher ๐๏ธ from tavily and Deanna Emery from Quotient AI ๐ซก


retrieval + search track = best vibes AI Engineer ft Maitar Asher ๐๏ธ Deanna Emery Jerry Liu tavily Quotient AI LlamaIndex ๐ฆ



Do you need evals for your AI project? Freddie Vargus joins us this week to share his experience from Quotient AI and GitHub Co-pilot

โYou want your model hitting milestones, not minefields.โ Most AI eval talk is hand-wavy. This isnโt. Freddie Vargus (Quotient AI CTO) gets into the weeds: how to actually test tool use, avoid minefields, and build agents that donโt break. Check out the recording๐

Just shared the slides from our AI Engineer World Fair talk: Evaluating AI Search โ A Practical Framework for Augmented Systems. As more AI agents rely on real-time data (like the web!), traditional eval approaches are falling behind and don't capture what's actually


AI Engineer Looking for more resources (think: research, OSS libraries, cookbooks and more!) for AI reliability? We have that! Check out Quotient AI Alpha, our collection of tool, resources and research. more coming weekly ๐


What did Freddie Vargus see? ๐ Everyoneโs talking about context engineering now. Freddie knew months ago: context is the product.

how do i catch hallucinations? come learn to implement monitoring systems that catch AI errors as they happen in live production environments with Julia Neagu and Quotient AI if you register, you'll be sent the recording and study notes after they're done!


DMs OPEN for topics you want covered. I write my talks the night before. it's a really bad habit. it stresses out Deanna Emery
