AutomatedEvaluationConfigDict¶
Dictionary Type¶
AutomatedEvaluationConfigDict
¶
Bases: TypedDict
Dictionary form of AutomatedEvaluationConfig.
Attributes¶
custom_metric_config
instance-attribute
¶
custom_metric_config: AutomatedEvaluationCustomMetricConfig | AutomatedEvaluationCustomMetricConfigDict | None
Defines the configuration of custom metrics to be used in an evaluation job.
dataset_metric_configs
instance-attribute
¶
dataset_metric_configs: Required[Sequence[EvaluationDatasetMetricConfig | EvaluationDatasetMetricConfigDict]]
Configuration details of the prompt datasets and metrics you want to use for your evaluation job.
evaluator_model_config
instance-attribute
¶
evaluator_model_config: EvaluatorModelConfig | EvaluatorModelConfigDict | None
Contains the evaluator model configuration details.
EvaluatorModelConfig is required for evaluation jobs that use a
knowledge base or in model evaluation job that use a model as judge.
This model computes all evaluation related metrics.