EvaluatorConfigDict¶
Union Type¶
EvaluatorConfigDict
module-attribute
¶
EvaluatorConfigDict = Union[EvaluatorConfigLlmAsAJudgeDict | EvaluatorConfigCodeBasedDict | EvaluatorConfigDerivedDict]
Dictionary form of EvaluatorConfig, setting exactly one variant.
Union Member Types¶
EvaluatorConfigLlmAsAJudgeDict
¶
Bases: TypedDict
Attributes¶
llm_as_a_judge
instance-attribute
¶
llm_as_a_judge: LlmAsAJudgeEvaluatorConfig | LlmAsAJudgeEvaluatorConfigDict
The LLM-as-a-Judge configuration that uses a language model to evaluate agent performance based on custom instructions and rating scales.
EvaluatorConfigCodeBasedDict
¶
Bases: TypedDict
Attributes¶
code_based
instance-attribute
¶
code_based: CodeBasedEvaluatorConfig | CodeBasedEvaluatorConfigDict
Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance.
EvaluatorConfigDerivedDict
¶
Bases: TypedDict
Attributes¶
derived
instance-attribute
¶
derived: DerivedEvaluatorConfig | DerivedEvaluatorConfigDict
The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring.