EvaluationReferenceInputDict¶
Dictionary Type¶
EvaluationReferenceInputDict
¶
Bases: TypedDict
Dictionary form of EvaluationReferenceInput.
Attributes¶
assertions
instance-attribute
¶
assertions: Sequence[EvaluationContent | EvaluationContentDict] | None
A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.
context
instance-attribute
¶
context: Required[Context | ContextDict]
The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.
expected_response
instance-attribute
¶
expected_response: EvaluationContent | EvaluationContentDict | None
The expected response for trace-level evaluation. Built-in evaluators
that support this field compare the agent's actual response against
this value for assessment. Custom evaluators can access it through the
{expected_response} placeholder in their instructions.
expected_trajectory
instance-attribute
¶
expected_trajectory: EvaluationExpectedTrajectory | EvaluationExpectedTrajectoryDict | None
The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.