start_batch_evaluation¶
Operation¶
start_batch_evaluation
¶
start_batch_evaluation(request: StartBatchEvaluationInput | StartBatchEvaluationInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[StartBatchEvaluationInputDict]) -> StartBatchEvaluationOutput
Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
StartBatchEvaluationInput | StartBatchEvaluationInputDict | None
|
The input as a model or dictionary. Pass this or keyword arguments, not both. |
None
|
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
**kwargs
|
Unpack[StartBatchEvaluationInputDict]
|
The input fields as keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
StartBatchEvaluationOutput
|
An instance of |
start_batch_evaluation
async
¶
start_batch_evaluation(request: StartBatchEvaluationInput | StartBatchEvaluationInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[StartBatchEvaluationInputDict]) -> StartBatchEvaluationOutput
Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
StartBatchEvaluationInput | StartBatchEvaluationInputDict | None
|
The input as a model or dictionary. Pass this or keyword arguments, not both. |
None
|
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
**kwargs
|
Unpack[StartBatchEvaluationInputDict]
|
The input fields as keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
StartBatchEvaluationOutput
|
An instance of |
Input¶
Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.
StartBatchEvaluationInputDict
¶
Bases: TypedDict
Dictionary form of StartBatchEvaluationInput.
Attributes¶
batch_evaluation_name
instance-attribute
¶
The name of the batch evaluation. Must be unique within your account.
client_token
instance-attribute
¶
client_token: str | None
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.
This field is autopopulated if not provided.
data_source_config
instance-attribute
¶
data_source_config: Required[DataSourceConfig | DataSourceConfigDict]
The data source configuration that specifies where to pull agent session traces from for evaluation.
evaluation_metadata
instance-attribute
¶
evaluation_metadata: EvaluationMetadata | EvaluationMetadataDict | None
Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.
evaluators
instance-attribute
¶
evaluators: Sequence[Evaluator | EvaluatorDict] | None
The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.
insights
instance-attribute
¶
insights: Sequence[Insight | InsightDict] | None
The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.
kms_key_arn
instance-attribute
¶
kms_key_arn: str | None
The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.
output_config
instance-attribute
¶
output_config: OutputConfig | OutputConfigDict | None
Output destination configuration.
StartBatchEvaluationInput
dataclass
¶
Bases: _SerdeShape
Dataclass for StartBatchEvaluationInput structure.
Attributes¶
batch_evaluation_name
class-attribute
instance-attribute
¶
batch_evaluation_name: str | None = ...
The name of the batch evaluation. Must be unique within your account.
client_token
class-attribute
instance-attribute
¶
client_token: str | None = ...
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.
This field is autopopulated if not provided.
data_source_config
class-attribute
instance-attribute
¶
data_source_config: DataSourceConfig | None = ...
The data source configuration that specifies where to pull agent session traces from for evaluation.
description
class-attribute
instance-attribute
¶
description: str | None = ...
The description of the batch evaluation.
evaluation_metadata
class-attribute
instance-attribute
¶
evaluation_metadata: EvaluationMetadata | None = ...
Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.
evaluators
class-attribute
instance-attribute
¶
The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.
insights
class-attribute
instance-attribute
¶
The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.
kms_key_arn
class-attribute
instance-attribute
¶
kms_key_arn: str | None = ...
The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.
output_config
class-attribute
instance-attribute
¶
output_config: OutputConfig | None = ...
Output destination configuration.
Output¶
StartBatchEvaluationOutput
dataclass
¶
Bases: _SerdeShape
Dataclass for StartBatchEvaluationOutput structure.
Attributes¶
batch_evaluation_arn
instance-attribute
¶
batch_evaluation_arn: str
The Amazon Resource Name (ARN) of the created batch evaluation.
batch_evaluation_id
instance-attribute
¶
batch_evaluation_id: str
The unique identifier of the created batch evaluation.
batch_evaluation_name
instance-attribute
¶
batch_evaluation_name: str
The name of the batch evaluation.
created_at
instance-attribute
¶
created_at: datetime
The timestamp when the batch evaluation was created.
description
class-attribute
instance-attribute
¶
description: str | None = ...
The description of the batch evaluation.
evaluators
class-attribute
instance-attribute
¶
The list of evaluators applied during the batch evaluation.
insights
class-attribute
instance-attribute
¶
The list of insight analyses applied during the batch evaluation.
kms_key_arn
class-attribute
instance-attribute
¶
kms_key_arn: str | None = ...
The ARN of the KMS key used to encrypt evaluation data.
output_config
class-attribute
instance-attribute
¶
output_config: OutputConfig | None = ...
The output configuration specifying where evaluation results are written.
response_metadata
class-attribute
instance-attribute
¶
Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.