LlmAsAJudgeEvaluatorConfig¶
Structure Class¶
LlmAsAJudgeEvaluatorConfig
dataclass
¶
Bases: _SerdeShape
The configuration for LLM-as-a-Judge evaluation that uses a language model to assess agent performance based on custom instructions and rating scales.
Attributes¶
instructions
instance-attribute
¶
instructions: str
The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.
model_config
instance-attribute
¶
model_config: EvaluatorModelConfig
The model configuration that specifies which foundation model to use and how to configure it for evaluation.
rating_scale
instance-attribute
¶
rating_scale: RatingScale
The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.