Open SourceAnalytics9k
DeepEval
The open-source LLM evaluation framework, like Pytest for LLMs.
DeepEval is an open-source evaluation framework from Confident AI that lets developers unit-test LLM outputs the way Pytest tests code. It ships research-backed metrics including G-Eval, hallucination, answer relevancy, faithfulness, and RAG-specific scores, and supports benchmarks, red-teaming, and component-level testing. It runs locally and plugs into CI pipelines to catch regressions before deploy.
Repository
confident-ai/deepevalLanguage
PythonWhat you'd build with it
- Write Pytest-style assertions over LLM outputs with metrics like G-Eval and hallucination
- Benchmark and red-team a model or agent before shipping it to production
- Run RAG and chatbot evals automatically in CI on every prompt or model change
Tags
evalsllm-testingred-teamingci