Skip to content

create_evaluation_job

Operation

create_evaluation_job

create_evaluation_job(request: CreateEvaluationJobInput | CreateEvaluationJobInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[CreateEvaluationJobInputDict]) -> CreateEvaluationJobOutput

Creates an evaluation job.

Parameters:

Name Type Description Default
request CreateEvaluationJobInput | CreateEvaluationJobInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[CreateEvaluationJobInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
CreateEvaluationJobOutput

An instance of CreateEvaluationJobOutput.

create_evaluation_job async

create_evaluation_job(request: CreateEvaluationJobInput | CreateEvaluationJobInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[CreateEvaluationJobInputDict]) -> CreateEvaluationJobOutput

Creates an evaluation job.

Parameters:

Name Type Description Default
request CreateEvaluationJobInput | CreateEvaluationJobInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[CreateEvaluationJobInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
CreateEvaluationJobOutput

An instance of CreateEvaluationJobOutput.

Input

Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.

CreateEvaluationJobInputDict

Bases: TypedDict

Dictionary form of CreateEvaluationJobInput.

Attributes

application_type instance-attribute
application_type: str | None

Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).

client_request_token instance-attribute
client_request_token: str | None

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.

This field is autopopulated if not provided.

customer_encryption_key_id instance-attribute
customer_encryption_key_id: str | None

Specify your customer managed encryption key Amazon Resource Name (ARN) that will be used to encrypt your evaluation job.

evaluation_config instance-attribute

Contains the configuration details of either an automated or human-based evaluation job.

inference_config instance-attribute

Contains the configuration details of the inference model for the evaluation job.

For model evaluation jobs, automated jobs support a single model or inference profile, and jobs that use human workers support two models or inference profiles.

job_description instance-attribute
job_description: str | None

A description of the evaluation job.

job_name instance-attribute
job_name: Required[str]

A name for the evaluation job. Names must unique with your Amazon Web Services account, and your account's Amazon Web Services region.

job_tags instance-attribute
job_tags: Sequence[Tag | TagDict] | None

Tags to attach to the model evaluation job.

output_data_config instance-attribute

Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job.

role_arn instance-attribute
role_arn: Required[str]

The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. To learn more about the required permissions, see Required permissions for model evaluations.

CreateEvaluationJobInput dataclass

Bases: _SerdeShape

Dataclass for CreateEvaluationJobInput structure.

Attributes

application_type class-attribute instance-attribute
application_type: str | None = ...

Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).

client_request_token class-attribute instance-attribute
client_request_token: str | None = ...

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.

This field is autopopulated if not provided.

customer_encryption_key_id class-attribute instance-attribute
customer_encryption_key_id: str | None = ...

Specify your customer managed encryption key Amazon Resource Name (ARN) that will be used to encrypt your evaluation job.

evaluation_config class-attribute instance-attribute
evaluation_config: EvaluationConfig | None = ...

Contains the configuration details of either an automated or human-based evaluation job.

inference_config class-attribute instance-attribute
inference_config: EvaluationInferenceConfig | None = ...

Contains the configuration details of the inference model for the evaluation job.

For model evaluation jobs, automated jobs support a single model or inference profile, and jobs that use human workers support two models or inference profiles.

job_description class-attribute instance-attribute
job_description: str | None = ...

A description of the evaluation job.

job_name class-attribute instance-attribute
job_name: str | None = ...

A name for the evaluation job. Names must unique with your Amazon Web Services account, and your account's Amazon Web Services region.

job_tags class-attribute instance-attribute
job_tags: list[Tag] | None = ...

Tags to attach to the model evaluation job.

output_data_config class-attribute instance-attribute
output_data_config: EvaluationOutputDataConfig | None = ...

Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job.

role_arn class-attribute instance-attribute
role_arn: str | None = ...

The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. To learn more about the required permissions, see Required permissions for model evaluations.

Output

CreateEvaluationJobOutput dataclass

Bases: _SerdeShape

Dataclass for CreateEvaluationJobOutput structure.

Attributes

job_arn instance-attribute
job_arn: str

The Amazon Resource Name (ARN) of the evaluation job.

response_metadata class-attribute instance-attribute
response_metadata: ResponseMetadata = ...

Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.