Schemas
Samples, messages, datasets and experiments. To score a Dataset of samples, see Scoring a dataset.
BaseSample
Bases: BaseModel
Base class for evaluation samples.
to_dict
get_features
to_string
SingleTurnSample
Bases: BaseSample
Represents evaluation samples for single-turn interactions.
Attributes:
| Name | Type | Description |
|---|---|---|
user_input |
Optional[str]
|
The input query from the user. |
retrieved_contexts |
Optional[List[str]]
|
List of contexts retrieved for the query. |
reference_contexts |
Optional[List[str]]
|
List of reference contexts for the query. |
retrieved_context_ids |
Optional[List[Union[str, int]]]
|
List of IDs for retrieved contexts. |
reference_context_ids |
Optional[List[Union[str, int]]]
|
List of IDs for reference contexts. |
response |
Optional[str]
|
The generated response for the query. |
multi_responses |
Optional[List[str]]
|
List of multiple responses generated for the query. |
reference |
Optional[str]
|
The reference answer for the query. |
rubric |
Optional[Dict[str, str]]
|
Evaluation rubric for the sample. |
persona_name |
Optional[str]
|
Name of the persona used in query generation. |
query_style |
Optional[str]
|
Style of the generated query (e.g., formal, casual). |
query_length |
Optional[str]
|
Length category of the query (e.g., short, medium, long). |
reference_contexts_metadata |
Optional[List[Dict[str, Any]]]
|
Metadata of the source document for each entry in
|
MultiTurnSample
Bases: BaseSample
Represents evaluation samples for multi-turn interactions.
Attributes:
| Name | Type | Description |
|---|---|---|
user_input |
List[Union[HumanMessage, AIMessage, ToolMessage]]
|
A list of messages representing the conversation turns. |
reference |
(Optional[str], optional)
|
The reference answer or expected outcome for the conversation. |
reference_tool_calls |
(Optional[List[ToolCall]], optional)
|
A list of expected tool calls for the conversation. |
rubrics |
(Optional[Dict[str, str]], optional)
|
Evaluation rubrics for the conversation. |
reference_topics |
(Optional[List[str]], optional)
|
A list of reference topics for the conversation. |
validate_user_input
classmethod
validate_user_input(
messages: List[
Union[HumanMessage, AIMessage, ToolMessage]
],
) -> List[Union[HumanMessage, AIMessage, ToolMessage]]
Validates the user input messages.
Source code in src/ragas/dataset_schema.py
to_messages
pretty_repr
Returns a pretty string representation of the conversation.
sample_from_dict
Build the sample a row dict describes (see :func:sample_type_for).
sample_type_for
The sample class a row dict is: multi-turn if user_input is a list of messages.
Source code in src/ragas/dataset_schema.py
Message
Bases: BaseModel
Represents a generic message.
Attributes:
| Name | Type | Description |
|---|---|---|
content |
str
|
The content of the message. |
metadata |
(Optional[Dict[str, Any]], optional)
|
Additional metadata associated with the message. |
ToolCall
Bases: BaseModel
Represents a tool call with a name and arguments.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
The name of the tool being called. |
args |
Dict[str, Any]
|
A dictionary of arguments for the tool call, where keys are argument names and values can be strings, integers, or floats. |
HumanMessage
Bases: Message
Represents a message from a human user.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[human]
|
The type of the message, always set to "human". |
Methods:
| Name | Description |
|---|---|
pretty_repr |
Returns a formatted string representation of the human message. |
ToolMessage
Bases: Message
Represents a message from a tool.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[tool]
|
The type of the message, always set to "tool". |
Methods:
| Name | Description |
|---|---|
pretty_repr |
Returns a formatted string representation of the tool message. |
AIMessage
Bases: Message
Represents a message from an AI.
Attributes:
| Name | Type | Description |
|---|---|---|
type |
Literal[ai]
|
The type of the message, always set to "ai". |
tool_calls |
Optional[List[ToolCall]]
|
A list of tool calls made by the AI, if any. |
metadata |
Optional[Dict[str, Any]]
|
Additional metadata associated with the AI message. |
Methods:
| Name | Description |
|---|---|
dict |
Returns a dictionary representation of the AI message. |
pretty_repr |
Returns a formatted string representation of the AI message. |
to_dict
Returns a dictionary representation of the AI message.
Source code in src/ragas/messages.py
pretty_repr
Returns a formatted string representation of the AI message.
Source code in src/ragas/messages.py
ragas.dataset.Dataset
Dataset(
name: str,
backend: Union[BaseBackend, str],
data_model: Optional[Type[T]] = None,
data: Optional[List[Any]] = None,
**kwargs,
)
Bases: DataTable[T]
Dataset class for managing dataset entries.
Inherits all functionality from DataTable. This class represents datasets specifically (as opposed to experiments).
Source code in src/ragas/dataset.py
ragas.experiment.Experiment
Experiment(
name: str,
backend: BaseBackend,
data_model: None = None,
data: Optional[List[Dict[str, Any]]] = None,
)
Experiment(
name: str,
backend: Union[BaseBackend, str],
data_model: Optional[Type[T]] = None,
data: Optional[List[Any]] = None,
**kwargs,
)
Bases: DataTable