Skip to content

AutomatedEvaluationConfigDict

Dictionary Type

AutomatedEvaluationConfigDict

Bases: TypedDict

Dictionary form of AutomatedEvaluationConfig.

Attributes

custom_metric_config instance-attribute

Defines the configuration of custom metrics to be used in an evaluation job.

dataset_metric_configs instance-attribute

Configuration details of the prompt datasets and metrics you want to use for your evaluation job.

evaluator_model_config instance-attribute
evaluator_model_config: EvaluatorModelConfig | EvaluatorModelConfigDict | None

Contains the evaluator model configuration details. EvaluatorModelConfig is required for evaluation jobs that use a knowledge base or in model evaluation job that use a model as judge. This model computes all evaluation related metrics.