Skip to content

EvaluatorConfigDict

Union Type

EvaluatorConfigDict module-attribute

Dictionary form of EvaluatorConfig, setting exactly one variant.

Union Member Types

EvaluatorConfigLlmAsAJudgeDict

Bases: TypedDict

Attributes

llm_as_a_judge instance-attribute

The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales.

EvaluatorConfigCodeBasedDict

Bases: TypedDict

Attributes

code_based instance-attribute

Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance.

EvaluatorConfigDerivedDict

Bases: TypedDict

Attributes

derived instance-attribute

The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring.