Skip to content

LlmAsAJudgeEvaluatorConfig

Structure Class

LlmAsAJudgeEvaluatorConfig dataclass

Bases: _SerdeShape

The configuration for LLM-as-a-Judge evaluation that uses a language model to assess agent performance based on custom instructions and rating scales.

Attributes

instructions instance-attribute
instructions: str

The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.

model_config instance-attribute
model_config: EvaluatorModelConfig

The model configuration that specifies which foundation model to use and how to configure it for evaluation.

rating_scale instance-attribute
rating_scale: RatingScale

The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.