LLMs
InstructorBaseRagasLLM
Bases: ABC
Base class for LLMs using the Instructor library pattern.
Provider calls are retried with exponential backoff on transient errors
(rate limits, timeouts, 5xx) per run_config (#73): the RunConfig
passed to metrics_experiment / TestsetGenerator for calls made
inside that run, else this LLM's set_run_config, else RunConfig() defaults.
A run's config never sticks to the LLM (#76).
set_run_config
set_run_config(run_config: RunConfig) -> None
generate
abstractmethod
generate(
prompt: LLMInput,
response_model: Type[InstructorTypeVar],
temperature: Optional[float] = None,
sample_index: Optional[int] = None,
) -> InstructorTypeVar
Generate a response using the configured LLM.
For async clients, this will run the async method in the appropriate event loop.
prompt is a string, sent as one user message, or a list of
OpenAI-format chat messages (content may be a list of text and
image_url parts) sent as given; system_prompt is prepended to
either unless the messages carry their own system message. Both go
through the same retries, cache, rate limit, tracing and usage
recording (#214).
temperature overrides the configured temperature for this call, for
metrics that deliberately draw several samples (strictness).
sample_index does not reach the provider; it only gives each sample
its own cache entry so a cache doesn't collapse N samples into one.
Source code in src/ragas/llms/base.py
agenerate
abstractmethod
async
agenerate(
prompt: LLMInput,
response_model: Type[InstructorTypeVar],
temperature: Optional[float] = None,
sample_index: Optional[int] = None,
) -> InstructorTypeVar
Asynchronously generate a response using the configured LLM.
See generate for temperature and sample_index.
Source code in src/ragas/llms/base.py
InstructorLLM
InstructorLLM(
client: Any,
model: str,
provider: str,
model_args: Optional[InstructorModelArgs] = None,
cache: Optional[CacheInterface] = None,
sync_client: Optional[Any] = None,
reasoning: Optional[bool] = None,
is_async: Optional[bool] = None,
**kwargs,
)
Bases: CachedCopyable, InstructorBaseRagasLLM
LLM wrapper using the Instructor library for structured outputs.
llm_factory() builds these over LiteLLM (#182). You can also construct
one directly around your own instructor client (e.g.
instructor.from_provider(...)); the provider rules below still apply,
but LiteLLM's drop_params and the LiteLLM-only defaults don't.
Source code in src/ragas/llms/base.py
generate
generate(
prompt: LLMInput,
response_model: Type[InstructorTypeVar],
temperature: Optional[float] = None,
sample_index: Optional[int] = None,
) -> InstructorTypeVar
Generate a response using the configured LLM.
For async clients, this will run the async method in the appropriate event loop.
See InstructorBaseRagasLLM.generate for temperature/sample_index.
Source code in src/ragas/llms/base.py
agenerate
async
agenerate(
prompt: LLMInput,
response_model: Type[InstructorTypeVar],
temperature: Optional[float] = None,
sample_index: Optional[int] = None,
) -> InstructorTypeVar
Asynchronously generate a response using the configured LLM.
A synchronous client is run on a worker thread, so every collections
metric (all of which are async) works with OpenAI(),
Anthropic() and other sync clients (#68).
See InstructorBaseRagasLLM.generate for temperature/sample_index.
Source code in src/ragas/llms/base.py
llm_factory
llm_factory(
model: str,
provider: str = "openai",
client: Optional[Any] = None,
adapter: str = "auto",
cache: Optional[CacheInterface] = None,
mode: Optional[Mode] = None,
rate_limit: Optional[RateLimit] = None,
**kwargs: Any,
) -> InstructorBaseRagasLLM
Create an LLM for structured output. Every call goes through LiteLLM.
Address any LiteLLM-supported provider by model name; credentials come from
the environment (OPENAI_API_KEY, ANTHROPIC_API_KEY,
GEMINI_API_KEY, ...) or from api_key= / api_base=.
Args:
model: A LiteLLM model string: "gpt-6-luna",
"anthropic/claude-sonnet-5-5", "gemini/gemini-3.6-flash",
"ollama/qwen3.8:27b", "bedrock/<model id>", ...
See https://docs.litellm.ai/docs/providers.
provider: Prefixed onto a bare model when there is no client:
llm_factory("llama3", provider="ollama") is
"ollama/llama3". "openai" adds no prefix.
client: Optional. An OpenAI / AsyncOpenAI (any
OpenAI-compatible endpoint, via base_url) or AzureOpenAI /
AsyncAzureOpenAI client, passed to LiteLLM with its settings
(a copy of it: LiteLLM changes the object it is given), or a
litellm.Router. Native SDK clients of other providers
(Anthropic(), genai.Client(), Mistral, Groq, Cohere,
boto3) are no longer accepted; use the provider's model string
instead (#182).
adapter: Deprecated and ignored: there is one path (#182).
cache: Optional cache backend for LLM responses.
mode: instructor mode. Defaults to Mode.JSON for OpenAI/Azure
clients and for Claude, otherwise instructor's default. Native
modes (Mode.ANTHROPIC_*, Mode.GENAI_*, ...) raise.
rate_limit: Optional ragas.rate_limit.RateLimit: requests (and
tokens) per minute. Every request this LLM sends waits for it,
including retries; pass the same object to LLMs that share an
API key (#171).
**kwargs: Model arguments (temperature, top_p, max_tokens,
system_prompt, reasoning, ...) and LiteLLM call
arguments (api_key, api_base, extra_headers, ...).
Returns:
An InstructorLLM with generate() and agenerate().
Examples: llm = llm_factory("gpt-6-luna") # OPENAI_API_KEY llm = llm_factory("anthropic/claude-sonnet-5-5") # ANTHROPIC_API_KEY llm = llm_factory("gemini/gemini-3.6-flash") # GEMINI_API_KEY llm = llm_factory("ollama/qwen3.8:27b") # local Ollama
# an OpenAI-compatible endpoint, with your own client settings
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
llm = llm_factory("qwen3.8:27b", client=client)
Source code in src/ragas/llms/factory.py
182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 | |