Skip to content

EvaluationReferenceInputDict

Dictionary Type

EvaluationReferenceInputDict

Bases: TypedDict

Dictionary form of EvaluationReferenceInput.

Attributes

assertions instance-attribute

A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.

context instance-attribute

The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.

expected_response instance-attribute
expected_response: EvaluationContent | EvaluationContentDict | None

The expected response for trace-level evaluation. Built-in evaluators that support this field compare the agent's actual response against this value for assessment. Custom evaluators can access it through the {expected_response} placeholder in their instructions.

expected_trajectory instance-attribute

The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.