🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

🚧 One of the biggest barriers to LLM evaluation today? The steep learning curve. You’re expected to: - Understand how each metric works - Interpret what the results actually mean - Compare across multiple (often conflicting) metrics And if you get it wrong? You might end up

Thrilled to announce that DeepEval now supports LiteLLM as a native model provider. Developers can now use hundreds of models without having to change the LiteLLM (YC W23) models they're already using. More to come in the following days.

Great finding DeepEval in today's AI engineer newsletter: aiengineering.beehiiv.com/p/build-and-de…

In the week of June DeepEval decided to embrace OpenAI's API messages format for evaluating multi-turn conversations - and as a result almost tripled the total number of conversational evals ran from all our users. As a result we're releasing a native OpenAI integration later

Mult-turn evals are now extremely popular for users on DeepEval.

The most neglected pages on Confident AI are the settings, organization, and project pages. Kritin Vongthongsri and I sat down over the weekend and refactored + redesigned 10k lines of code to make things more professional. Does this look better now?

Just learned Python decorators & found out you can trace your entire LLM app with just observe Most LLM evals: “Something’s broken. Somewhere.” With DeepEval: 🔹 Trace tools, retrievers, LLMs 🔹 Add custom metrics 🔹 Visualize what’s working No refactor needed.

"perfection is the enemy of good" - voltaire shipped DeepEval 1. enable tracing with hierarchy of spans LangChain agents 2. initial iteration of CrewAI integration with deepeval to trace your LLM spans 3. initial iteration of LlamaIndex 🦙 integration with deepeval..

Avi Chawla This is 🔥 Love how DeepEval makes deep LLM tracing dead simple without touching existing code.

DeepEval on trending again, seems like the recent component-level evals is definitely something people need for evaluating AI agents.

This guest post from DeepEval shows you how to build better RAG applications by combining LlamaIndex with comprehensive evaluation: 🎯 Use Answer Relevancy, Faithfulness, and Contextual Precision metrics to measure both your retriever and generator components 🔧 Set up

One of the most beautiful things about talking to users is you realize where they are struggling. We're making two major releases at DeepEval this week to address the most common issues people face when running evals.

Don’t get frustrated by writing print statements and endlessly scrolling terminal logs to debug your LangChain (and LangGraph) app. Trace your agent’s execution steps in production on Confident AI using our callback handler, with just two lines of code. Documentation:

🤖 Two LLM outputs walk into an arena… Only one leaves with the crown 👑 ⚔️ Pairwise battles ⚖️ Elo-style scoring 🙈 Blind trials 🧠 LLMs judging LLMs No complex metrics. Just ask: which one is better? confident-ai.com/blog/llm-arena…

🚀 Shipped Integration You can now trace your CrewAI apps on the Confident AI platform. By just adding 2 lines of code in your app, you can get the entire execution steps of your agent in the form of a single trace. Leverage your LLM application performance using

.@Confident_AI's DeepTeam is an open-source red teaming framework for AI agents. Test for memory leaks, goal hijacking, and decision flaws across 40+ attack types & vulnerabilities. Congrats on the launch, Jeffrey 🐬 confident-ai.com and Kritin Vongthongsri! github.com/confident-ai/d…