AutoArena

AutoArena is an open-source tool that automates head-to-head evaluations using LLM judges to rank GenAI systems. Quickly and accurately generate leaderboards comparing different LLMs, RAG setups, or prompt variations—Fine-tune custom judges to fit your needs.

AutoArena is an open-source tool designed to automate head-to-head evaluations of GenAI systems using LLM judges. It enables users to quickly and accurately generate leaderboards comparing various LLMs, RAG setups, or prompt variations. With the ability to fine-tune custom judges, AutoArena offers tailored, efficient evaluations to meet specific needs, streamlining the process of ranking and improving AI systems.