# Ragas > Evaluation framework for your AI Application ragas-ng is a hard fork of vibrantlabsai/ragas, an open-source evaluation framework for LLM applications including RAG pipelines, AI agents, and workflows. The package and import name are still `ragas`. The fork has no LangChain dependency and `llm_factory()` routes through LiteLLM. It provides objective metrics for evaluation, test data generation, and integrations such as LlamaIndex. See the migration notes for differences from upstream. ## Getting Started - [](https://arpieb.github.io/ragas-ng/index.md) - [🚀 Get Started](https://arpieb.github.io/ragas-ng/getstarted/index.md) - [Evaluate a Simple LLM App](https://arpieb.github.io/ragas-ng/getstarted/evals/index.md) - [Run Your First Experiment](https://arpieb.github.io/ragas-ng/getstarted/experiments_quickstart/index.md) - [Installation](https://arpieb.github.io/ragas-ng/getstarted/install/index.md) - [Quick Start](https://arpieb.github.io/ragas-ng/getstarted/quickstart/index.md) - [Evaluate a Simple RAG System](https://arpieb.github.io/ragas-ng/getstarted/rag_eval/index.md) - [Testset Generation for RAG](https://arpieb.github.io/ragas-ng/getstarted/rag_testset_generation/index.md) ## Tutorials - [Tutorials](https://arpieb.github.io/ragas-ng/tutorials/index.md) - [Evaluate an AI Agent](https://arpieb.github.io/ragas-ng/tutorials/agent/index.md) - [Evaluate a prompt](https://arpieb.github.io/ragas-ng/tutorials/prompt/index.md) - [Evaluate a simple RAG system](https://arpieb.github.io/ragas-ng/tutorials/rag/index.md) - [Evaluate an AI Workflow](https://arpieb.github.io/ragas-ng/tutorials/workflow/index.md) ## Core Concepts - [📚 Core Concepts](https://arpieb.github.io/ragas-ng/concepts/index.md) - [Datasets](https://arpieb.github.io/ragas-ng/concepts/datasets/index.md) - [Experimentation](https://arpieb.github.io/ragas-ng/concepts/experimentation/index.md) - [Components Guide](https://arpieb.github.io/ragas-ng/concepts/components/index.md) - [Evaluation Dataset](https://arpieb.github.io/ragas-ng/concepts/components/eval_dataset/index.md) - [Evaluation Sample](https://arpieb.github.io/ragas-ng/concepts/components/eval_sample/index.md) - [Prompt](https://arpieb.github.io/ragas-ng/concepts/components/prompt/index.md) ## Metrics - [Metrics](https://arpieb.github.io/ragas-ng/concepts/metrics/index.md) - [List of available metrics](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/index.md) - [Agentic or Tool use](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/agents/index.md) - [Answer Correctness](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/answer_correctness/index.md) - [Answer Relevancy](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/answer_relevance/index.md) - [Aspect Critique](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/aspect_critic/index.md) - [Context Entities Recall](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/context_entities_recall/index.md) - [Context Precision](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/context_precision/index.md) - [Context Recall](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/context_recall/index.md) - [Factual Correctness](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/factual_correctness/index.md) - [Faithfulness](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/faithfulness/index.md) - [General Purpose Metrics](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/general_purpose/index.md) - [Multimodal Faithfulness](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/multi_modal_faithfulness/index.md) - [Multimodal Relevance](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/multi_modal_relevance/index.md) - [Noise Sensitivity](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/noise_sensitivity/index.md) - [Nvidia Metrics](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/nvidia_metrics/index.md) - [Rubric-Based Evaluation](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/rubrics_based/index.md) - [Semantic Similarity](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/semantic_similarity/index.md) - [SQL](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/sql/index.md) - [Summarization](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/summarization_score/index.md) - [Traditional NLP Metrics](https://arpieb.github.io/ragas-ng/concepts/metrics/available_metrics/traditional/index.md) - [Overview](https://arpieb.github.io/ragas-ng/concepts/metrics/overview/index.md) ## Test Data Generation - [Testset Generation](https://arpieb.github.io/ragas-ng/concepts/test_data_generation/index.md) - [Testset Generation for Agents or Tool use cases](https://arpieb.github.io/ragas-ng/concepts/test_data_generation/agents/index.md) - [Testset Generation for RAG](https://arpieb.github.io/ragas-ng/concepts/test_data_generation/rag/index.md) ## How-to Guides - [🛠️ How-to Guides](https://arpieb.github.io/ragas-ng/howtos/index.md) - [LLMs (llm_factory)](https://arpieb.github.io/ragas-ng/howtos/llm-adapters/index.md) - [Overview](https://arpieb.github.io/ragas-ng/howtos/observability/index.md) ## Migrating from Upstream Ragas - [Behavior changes](https://arpieb.github.io/ragas-ng/howtos/migrations/behavior_changes/index.md) - [From evaluate() to experiments](https://arpieb.github.io/ragas-ng/howtos/migrations/evaluate_to_experiment/index.md) - [From v0.1 to v0.2](https://arpieb.github.io/ragas-ng/howtos/migrations/migrate_from_v01_to_v02/index.md) - [From v0.3 to v0.4](https://arpieb.github.io/ragas-ng/howtos/migrations/migrate_from_v03_to_v04/index.md) - [Off LangChain](https://arpieb.github.io/ragas-ng/howtos/migrations/migrate_off_langchain/index.md) - [Metric score changes](https://arpieb.github.io/ragas-ng/howtos/migrations/score_changes/index.md) ## Customization Guides - [Customizations](https://arpieb.github.io/ragas-ng/howtos/customizations/index.md) - [Caching](https://arpieb.github.io/ragas-ng/howtos/customizations/caching/index.md) - [Cancelling Tasks](https://arpieb.github.io/ragas-ng/howtos/customizations/cancellation/index.md) - [Customise models](https://arpieb.github.io/ragas-ng/howtos/customizations/customize_models/index.md) - [Run Config](https://arpieb.github.io/ragas-ng/howtos/customizations/run_config/index.md) - [Storage Backends](https://arpieb.github.io/ragas-ng/howtos/customizations/storage_backends/index.md) - [Adapt Metrics to Languages](https://arpieb.github.io/ragas-ng/howtos/customizations/metrics/metrics_language_adaptation/index.md) - [Modify Prompts](https://arpieb.github.io/ragas-ng/howtos/customizations/metrics/modifying-prompts-metrics/index.md) - [Tracing and Logging Evaluations](https://arpieb.github.io/ragas-ng/howtos/customizations/metrics/tracing/index.md) - [DSPy Optimizer](https://arpieb.github.io/ragas-ng/howtos/customizations/optimizers/index.md) - [Customizing Test Data Generation](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/index.md) - [Non-English Testset Generation](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/language_adaptation/index.md) - [Persona Generation](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/persona_generator/index.md) - [Using Pre-chunked Data](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/prechunked_data/index.md) - [Custom Single-hop Query](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/testgen-custom-single-hop/index.md) - [Custom Multi-hop Query](https://arpieb.github.io/ragas-ng/howtos/customizations/testgenerator/testgen-customisation/index.md) ## Application Guides - [Applications](https://arpieb.github.io/ragas-ng/howtos/applications/index.md) - [Adding Ragas to your CI Pipeline](https://arpieb.github.io/ragas-ng/howtos/applications/add_to_ci/index.md) - [Align an LLM as a Judge](https://arpieb.github.io/ragas-ng/howtos/applications/align-llm-as-judge/index.md) - [Evaluate a New LLM](https://arpieb.github.io/ragas-ng/howtos/applications/benchmark_llm/index.md) - [Compare Embeddings for a Retriever](https://arpieb.github.io/ragas-ng/howtos/applications/compare_embeddings/index.md) - [Cost Analysis](https://arpieb.github.io/ragas-ng/howtos/applications/cost/index.md) - [Evaluate and Improve a RAG App](https://arpieb.github.io/ragas-ng/howtos/applications/evaluate-and-improve-rag/index.md) - [Evaluating Multi-turn Conversations](https://arpieb.github.io/ragas-ng/howtos/applications/evaluating_multi_turn_conversations/index.md) - [Iterate and Improve Prompts](https://arpieb.github.io/ragas-ng/howtos/applications/iterate_prompt/index.md) - [Systematic Prompt Optimization](https://arpieb.github.io/ragas-ng/howtos/applications/prompt_optimization/index.md) - [Single-hop Query Testset](https://arpieb.github.io/ragas-ng/howtos/applications/singlehop_testset_gen/index.md) - [Evaluate a Text-to-SQL Agent](https://arpieb.github.io/ragas-ng/howtos/applications/text2sql/index.md) - [Aligning Vertex AI Evaluators with Human Judgment](https://arpieb.github.io/ragas-ng/howtos/applications/vertexai_alignment/index.md) - [Comparing Vertex AI Models](https://arpieb.github.io/ragas-ng/howtos/applications/vertexai_model_comparision/index.md) - [Evaluations with Vertex AI models](https://arpieb.github.io/ragas-ng/howtos/applications/vertexai_x_ragas/index.md) ## CLI - [Ragas CLI](https://arpieb.github.io/ragas-ng/howtos/cli/index.md) - [Agent Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/agent_evals/index.md) - [LLM Benchmarking](https://arpieb.github.io/ragas-ng/howtos/cli/benchmark_llm/index.md) - [Improve RAG](https://arpieb.github.io/ragas-ng/howtos/cli/improve_rag/index.md) - [Judge Alignment](https://arpieb.github.io/ragas-ng/howtos/cli/judge_alignment/index.md) - [LlamaIndex Agent Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/llamaIndex_agent_evals/index.md) - [Prompt Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/prompt_evals/index.md) - [RAG Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/rag_eval/index.md) - [Text-to-SQL Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/text2sql/index.md) - [Workflow Evaluation](https://arpieb.github.io/ragas-ng/howtos/cli/workflow_eval/index.md) ## Integrations - [Integrations](https://arpieb.github.io/ragas-ng/howtos/integrations/index.md) - [Arize](https://arpieb.github.io/ragas-ng/howtos/integrations/arize/index.md) - [Langfuse](https://arpieb.github.io/ragas-ng/howtos/integrations/langfuse/index.md) - [LlamaIndex](https://arpieb.github.io/ragas-ng/howtos/integrations/llamaindex/index.md) - [AG-UI](https://arpieb.github.io/ragas-ng/howtos/integrations/ag_ui/index.md) - [Amazon Bedrock](https://arpieb.github.io/ragas-ng/howtos/integrations/amazon_bedrock/index.md) - [Google Gemini](https://arpieb.github.io/ragas-ng/howtos/integrations/gemini/index.md) - [Griptape](https://arpieb.github.io/ragas-ng/howtos/integrations/griptape/index.md) - [Haystack](https://arpieb.github.io/ragas-ng/howtos/integrations/haystack/index.md) - [LlamaStack](https://arpieb.github.io/ragas-ng/howtos/integrations/llama_stack/index.md) - [LlamaIndex Agents](https://arpieb.github.io/ragas-ng/howtos/integrations/llamaindex_agents/index.md) - [OCI Gen AI](https://arpieb.github.io/ragas-ng/howtos/integrations/oci_genai/index.md) - [R2R](https://arpieb.github.io/ragas-ng/howtos/integrations/r2r/index.md) - [Swarm](https://arpieb.github.io/ragas-ng/howtos/integrations/swarm_agent_evaluation/index.md) ## API Reference - [API References](https://arpieb.github.io/ragas-ng/references/index.md) - [Sync and async runs](https://arpieb.github.io/ragas-ng/references/aevaluate/index.md) - [Cache](https://arpieb.github.io/ragas-ng/references/cache/index.md) - [Embeddings](https://arpieb.github.io/ragas-ng/references/embeddings/index.md) - [metrics_experiment()](https://arpieb.github.io/ragas-ng/references/evaluate/index.md) - [Schemas](https://arpieb.github.io/ragas-ng/references/evaluation_schema/index.md) - [Executor](https://arpieb.github.io/ragas-ng/references/executor/index.md) - [Generation](https://arpieb.github.io/ragas-ng/references/generate/index.md) - [Graph](https://arpieb.github.io/ragas-ng/references/graph/index.md) - [Integrations](https://arpieb.github.io/ragas-ng/references/integrations/index.md) - [LLMs](https://arpieb.github.io/ragas-ng/references/llms/index.md) - [Metrics](https://arpieb.github.io/ragas-ng/references/metrics/index.md) - [Optimizers](https://arpieb.github.io/ragas-ng/references/optimizers/index.md) - [Prompt](https://arpieb.github.io/ragas-ng/references/prompt/index.md) - [RateLimit](https://arpieb.github.io/ragas-ng/references/rate_limit/index.md) - [RunConfig](https://arpieb.github.io/ragas-ng/references/run_config/index.md) - [Synthesizers](https://arpieb.github.io/ragas-ng/references/synthesizers/index.md) - [Schemas](https://arpieb.github.io/ragas-ng/references/testset_schema/index.md) - [Tokenizers](https://arpieb.github.io/ragas-ng/references/tokenizers/index.md) - [Transforms](https://arpieb.github.io/ragas-ng/references/transforms/index.md) ## Optional - [❤️ Community](https://arpieb.github.io/ragas-ng/community/index.md)