Llm Eval Projects

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

The platform for LLM evaluations and AI agent testing

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

The LLM Evaluation Framework

AI Observability & Evaluation

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

the LLM vulnerability scanner

Fast, flexible LLM inference

Run LLMs with MLX

Evaluation and Tracking for LLM Experiments and AI Agents

🐢 Open-Source Evaluation & Testing library for LLM Agents