
Rebecca Qian
@rebeccatqian
llm evals @PatronusAI, previously research eng @MetaAI
ID: 1516652840600719363
20-04-2022 05:40:30
184 Tweet
682 Followers
354 Following



Great to see Databricks use our eval benchmark FinanceBench to evaluate their new finetuning method TAO! ⚡ Test-time Adaptive Optimization (TAO) is a new finetuning method for reference-free use cases, i.e. it doesn't need labels to work, in contrast to SFT. It uses test-time

We are hosting a legal AI hackathon with Stanford University on Sunday! Thrilled to be sponsoring this event with Thomson Reuters, Bloomberg Law, LlamaIndex 🦙, and more. Come stop by our booth to say hi and see our product in action 🎉 And no, this is not an April Fools joke :) RSVP here:




Building good benchmarks is hard, and PatronusAI has released what may be the coolest agent eval yet: ✅ Realistic and objectively useful task ✅ Multilingual, multimodal, and multi-domain ✅ Easy for humans, still challenging for agents

My colleague Chris McConnell and I greatly enjoyed seeing Sky CH. Wang Darshan Deshpande Rebecca Qian Anand Kannappan bring this project to life. We’re excited to finally see it out in the world, and look forward to collaborating on the next one!

Welcome Varun Gangal to PatronusAI 🚀🚀 excited to work on eval research together





Check out the very cool work from our friends PatronusAI 🔥 work here! huggingface.co/spaces/Patronu…




Thank you, Professor zhou Yu and Berkeley Summit House, for the AI Agents in Action: Industry × Academia Exchange! Rebecca Qian, our CTO, was on a panel with Vinay Rao (Advisor at Anthropic), Shunyu Yao (Research Scientist at OpenAI), Robert Parker (Founder of Perceptix),
