Skip to content

start_batch_evaluation

Operation

start_batch_evaluation

start_batch_evaluation(request: StartBatchEvaluationInput | StartBatchEvaluationInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[StartBatchEvaluationInputDict]) -> StartBatchEvaluationOutput

Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.

Parameters:

Name Type Description Default
request StartBatchEvaluationInput | StartBatchEvaluationInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[StartBatchEvaluationInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
StartBatchEvaluationOutput

An instance of StartBatchEvaluationOutput.

start_batch_evaluation async

start_batch_evaluation(request: StartBatchEvaluationInput | StartBatchEvaluationInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[StartBatchEvaluationInputDict]) -> StartBatchEvaluationOutput

Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.

Parameters:

Name Type Description Default
request StartBatchEvaluationInput | StartBatchEvaluationInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[StartBatchEvaluationInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
StartBatchEvaluationOutput

An instance of StartBatchEvaluationOutput.

Input

Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.

StartBatchEvaluationInputDict

Bases: TypedDict

Dictionary form of StartBatchEvaluationInput.

Attributes

batch_evaluation_name instance-attribute
batch_evaluation_name: Required[str]

The name of the batch evaluation. Must be unique within your account.

client_token instance-attribute
client_token: str | None

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.

This field is autopopulated if not provided.

data_source_config instance-attribute

The data source configuration that specifies where to pull agent session traces from for evaluation.

description instance-attribute
description: str | None

The description of the batch evaluation.

evaluation_metadata instance-attribute
evaluation_metadata: EvaluationMetadata | EvaluationMetadataDict | None

Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.

evaluators instance-attribute
evaluators: Sequence[Evaluator | EvaluatorDict] | None

The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.

insights instance-attribute
insights: Sequence[Insight | InsightDict] | None

The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.

kms_key_arn instance-attribute
kms_key_arn: str | None

The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.

output_config instance-attribute
output_config: OutputConfig | OutputConfigDict | None

Output destination configuration.

tags instance-attribute
tags: Mapping[str, str] | None

A map of tag keys and values to associate with the batch evaluation.

StartBatchEvaluationInput dataclass

Bases: _SerdeShape

Dataclass for StartBatchEvaluationInput structure.

Attributes

batch_evaluation_name class-attribute instance-attribute
batch_evaluation_name: str | None = ...

The name of the batch evaluation. Must be unique within your account.

client_token class-attribute instance-attribute
client_token: str | None = ...

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.

This field is autopopulated if not provided.

data_source_config class-attribute instance-attribute
data_source_config: DataSourceConfig | None = ...

The data source configuration that specifies where to pull agent session traces from for evaluation.

description class-attribute instance-attribute
description: str | None = ...

The description of the batch evaluation.

evaluation_metadata class-attribute instance-attribute
evaluation_metadata: EvaluationMetadata | None = ...

Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.

evaluators class-attribute instance-attribute
evaluators: list[Evaluator] | None = ...

The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.

insights class-attribute instance-attribute
insights: list[Insight] | None = ...

The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.

kms_key_arn class-attribute instance-attribute
kms_key_arn: str | None = ...

The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.

output_config class-attribute instance-attribute
output_config: OutputConfig | None = ...

Output destination configuration.

tags class-attribute instance-attribute
tags: dict[str, str] | None = ...

A map of tag keys and values to associate with the batch evaluation.

Output

StartBatchEvaluationOutput dataclass

Bases: _SerdeShape

Dataclass for StartBatchEvaluationOutput structure.

Attributes

batch_evaluation_arn instance-attribute
batch_evaluation_arn: str

The Amazon Resource Name (ARN) of the created batch evaluation.

batch_evaluation_id instance-attribute
batch_evaluation_id: str

The unique identifier of the created batch evaluation.

batch_evaluation_name instance-attribute
batch_evaluation_name: str

The name of the batch evaluation.

created_at instance-attribute
created_at: datetime

The timestamp when the batch evaluation was created.

description class-attribute instance-attribute
description: str | None = ...

The description of the batch evaluation.

evaluators class-attribute instance-attribute
evaluators: list[Evaluator] | None = ...

The list of evaluators applied during the batch evaluation.

insights class-attribute instance-attribute
insights: list[Insight] | None = ...

The list of insight analyses applied during the batch evaluation.

kms_key_arn class-attribute instance-attribute
kms_key_arn: str | None = ...

The ARN of the KMS key used to encrypt evaluation data.

output_config class-attribute instance-attribute
output_config: OutputConfig | None = ...

The output configuration specifying where evaluation results are written.

response_metadata class-attribute instance-attribute
response_metadata: ResponseMetadata = ...

Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.

status instance-attribute
status: str

The status of the batch evaluation.

tags class-attribute instance-attribute
tags: dict[str, str] | None = ...

The tags associated with the batch evaluation.