create_evaluation_job¶
Operation¶
create_evaluation_job
¶
create_evaluation_job(request: CreateEvaluationJobInput | CreateEvaluationJobInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[CreateEvaluationJobInputDict]) -> CreateEvaluationJobOutput
Creates an evaluation job.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
CreateEvaluationJobInput | CreateEvaluationJobInputDict | None
|
The input as a model or dictionary. Pass this or keyword arguments, not both. |
None
|
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
**kwargs
|
Unpack[CreateEvaluationJobInputDict]
|
The input fields as keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
CreateEvaluationJobOutput
|
An instance of |
create_evaluation_job
async
¶
create_evaluation_job(request: CreateEvaluationJobInput | CreateEvaluationJobInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[CreateEvaluationJobInputDict]) -> CreateEvaluationJobOutput
Creates an evaluation job.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
CreateEvaluationJobInput | CreateEvaluationJobInputDict | None
|
The input as a model or dictionary. Pass this or keyword arguments, not both. |
None
|
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
**kwargs
|
Unpack[CreateEvaluationJobInputDict]
|
The input fields as keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
CreateEvaluationJobOutput
|
An instance of |
Input¶
Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.
CreateEvaluationJobInputDict
¶
Bases: TypedDict
Dictionary form of CreateEvaluationJobInput.
Attributes¶
application_type
instance-attribute
¶
application_type: str | None
Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).
client_request_token
instance-attribute
¶
client_request_token: str | None
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.
This field is autopopulated if not provided.
customer_encryption_key_id
instance-attribute
¶
customer_encryption_key_id: str | None
Specify your customer managed encryption key Amazon Resource Name (ARN) that will be used to encrypt your evaluation job.
evaluation_config
instance-attribute
¶
evaluation_config: Required[EvaluationConfig | EvaluationConfigDict]
Contains the configuration details of either an automated or human-based evaluation job.
inference_config
instance-attribute
¶
inference_config: Required[EvaluationInferenceConfig | EvaluationInferenceConfigDict]
Contains the configuration details of the inference model for the evaluation job.
For model evaluation jobs, automated jobs support a single model or inference profile, and jobs that use human workers support two models or inference profiles.
job_description
instance-attribute
¶
job_description: str | None
A description of the evaluation job.
job_name
instance-attribute
¶
A name for the evaluation job. Names must unique with your Amazon Web Services account, and your account's Amazon Web Services region.
job_tags
instance-attribute
¶
Tags to attach to the model evaluation job.
output_data_config
instance-attribute
¶
output_data_config: Required[EvaluationOutputDataConfig | EvaluationOutputDataConfigDict]
Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job.
role_arn
instance-attribute
¶
The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. To learn more about the required permissions, see Required permissions for model evaluations.
CreateEvaluationJobInput
dataclass
¶
Bases: _SerdeShape
Dataclass for CreateEvaluationJobInput structure.
Attributes¶
application_type
class-attribute
instance-attribute
¶
application_type: str | None = ...
Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).
client_request_token
class-attribute
instance-attribute
¶
client_request_token: str | None = ...
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.
This field is autopopulated if not provided.
customer_encryption_key_id
class-attribute
instance-attribute
¶
customer_encryption_key_id: str | None = ...
Specify your customer managed encryption key Amazon Resource Name (ARN) that will be used to encrypt your evaluation job.
evaluation_config
class-attribute
instance-attribute
¶
evaluation_config: EvaluationConfig | None = ...
Contains the configuration details of either an automated or human-based evaluation job.
inference_config
class-attribute
instance-attribute
¶
inference_config: EvaluationInferenceConfig | None = ...
Contains the configuration details of the inference model for the evaluation job.
For model evaluation jobs, automated jobs support a single model or inference profile, and jobs that use human workers support two models or inference profiles.
job_description
class-attribute
instance-attribute
¶
job_description: str | None = ...
A description of the evaluation job.
job_name
class-attribute
instance-attribute
¶
job_name: str | None = ...
A name for the evaluation job. Names must unique with your Amazon Web Services account, and your account's Amazon Web Services region.
job_tags
class-attribute
instance-attribute
¶
Tags to attach to the model evaluation job.
output_data_config
class-attribute
instance-attribute
¶
output_data_config: EvaluationOutputDataConfig | None = ...
Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job.
role_arn
class-attribute
instance-attribute
¶
role_arn: str | None = ...
The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. To learn more about the required permissions, see Required permissions for model evaluations.
Output¶
CreateEvaluationJobOutput
dataclass
¶
Bases: _SerdeShape
Dataclass for CreateEvaluationJobOutput structure.