Skip to content

invoke_model

Operation

invoke_model

invoke_model(request: InvokeModelInput | InvokeModelInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[InvokeModelInputDict]) -> InvokeModelOutput

Invokes the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. You use model inference to generate text, images, and embeddings.

For example code, see Invoke model code examples in the Amazon Bedrock User Guide.

This operation requires permission for the bedrock:InvokeModel action.

Warning

To deny all inference access to resources that you specify in the modelId field, you need to deny access to the bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream actions. Doing this also denies access to the resource through the Converse API actions (Converse and ConverseStream). For more information see Deny access for inference on specific models.

For troubleshooting some of the common errors you might encounter when using the InvokeModel API, see Troubleshooting Amazon Bedrock API Error Codes in the Amazon Bedrock User Guide

Parameters:

Name Type Description Default
request InvokeModelInput | InvokeModelInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[InvokeModelInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
InvokeModelOutput

An instance of InvokeModelOutput.

invoke_model async

invoke_model(request: InvokeModelInput | InvokeModelInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[InvokeModelInputDict]) -> InvokeModelOutput

Invokes the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. You use model inference to generate text, images, and embeddings.

For example code, see Invoke model code examples in the Amazon Bedrock User Guide.

This operation requires permission for the bedrock:InvokeModel action.

Warning

To deny all inference access to resources that you specify in the modelId field, you need to deny access to the bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream actions. Doing this also denies access to the resource through the Converse API actions (Converse and ConverseStream). For more information see Deny access for inference on specific models.

For troubleshooting some of the common errors you might encounter when using the InvokeModel API, see Troubleshooting Amazon Bedrock API Error Codes in the Amazon Bedrock User Guide

Parameters:

Name Type Description Default
request InvokeModelInput | InvokeModelInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[InvokeModelInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
InvokeModelOutput

An instance of InvokeModelOutput.

Input

Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.

InvokeModelInputDict

Bases: TypedDict

Dictionary form of InvokeModelInput.

Attributes

accept instance-attribute
accept: str | None

The desired MIME type of the inference body in the response. The default value is application/json.

body instance-attribute
body: bytes | None

The prompt and inference parameters in the format specified in the contentType in the header. You must provide the body in JSON format. To see the format and content of the request and response bodies for different models, refer to Inference parameters. For more information, see Run inference in the Bedrock User Guide.

content_type instance-attribute
content_type: str | None

The MIME type of the input data in the request. You must specify application/json.

guardrail_identifier instance-attribute
guardrail_identifier: str | None

The unique identifier of the guardrail that you want to use. If you don't provide a value, no guardrail is applied to the invocation.

An error will be thrown in the following situations.

  • You don't provide a guardrail identifier but you specify the amazon-bedrock-guardrailConfig field in the request body.

  • You enable the guardrail but the contentType isn't application/json.

  • You provide a guardrail identifier, but guardrailVersion isn't specified.

guardrail_version instance-attribute
guardrail_version: str | None

The version number for the guardrail. The value can also be DRAFT.

model_id instance-attribute
model_id: Required[str]

The unique identifier of the model to invoke to run inference.

The modelId to provide depends on the type of model or throughput that you use:

performance_config_latency instance-attribute
performance_config_latency: str | None

Model performance settings for the request.

request_metadata instance-attribute
request_metadata: str | None

Key-value pairs that you can use to filter invocation logs.

service_tier instance-attribute
service_tier: str | None

Specifies the processing tier type used for serving the request.

trace instance-attribute
trace: str | None

Specifies whether to enable or disable the Bedrock trace. If enabled, you can see the full Bedrock trace.

InvokeModelInput dataclass

Bases: _SerdeShape

Dataclass for InvokeModelInput structure.

Attributes

accept class-attribute instance-attribute
accept: str | None = ...

The desired MIME type of the inference body in the response. The default value is application/json.

body class-attribute instance-attribute
body: bytes | None = ...

The prompt and inference parameters in the format specified in the contentType in the header. You must provide the body in JSON format. To see the format and content of the request and response bodies for different models, refer to Inference parameters. For more information, see Run inference in the Bedrock User Guide.

content_type class-attribute instance-attribute
content_type: str | None = ...

The MIME type of the input data in the request. You must specify application/json.

guardrail_identifier class-attribute instance-attribute
guardrail_identifier: str | None = ...

The unique identifier of the guardrail that you want to use. If you don't provide a value, no guardrail is applied to the invocation.

An error will be thrown in the following situations.

  • You don't provide a guardrail identifier but you specify the amazon-bedrock-guardrailConfig field in the request body.

  • You enable the guardrail but the contentType isn't application/json.

  • You provide a guardrail identifier, but guardrailVersion isn't specified.

guardrail_version class-attribute instance-attribute
guardrail_version: str | None = ...

The version number for the guardrail. The value can also be DRAFT.

model_id class-attribute instance-attribute
model_id: str | None = ...

The unique identifier of the model to invoke to run inference.

The modelId to provide depends on the type of model or throughput that you use:

performance_config_latency class-attribute instance-attribute
performance_config_latency: str = ...

Model performance settings for the request.

request_metadata class-attribute instance-attribute
request_metadata: str | None = ...

Key-value pairs that you can use to filter invocation logs.

service_tier class-attribute instance-attribute
service_tier: str | None = ...

Specifies the processing tier type used for serving the request.

trace class-attribute instance-attribute
trace: str | None = ...

Specifies whether to enable or disable the Bedrock trace. If enabled, you can see the full Bedrock trace.

Output

InvokeModelOutput dataclass

Bases: _SerdeShape

Dataclass for InvokeModelOutput structure.

Attributes

body instance-attribute
body: bytes

Inference response from the model in the format specified in the contentType header. To see the format and content of the request and response bodies for different models, refer to Inference parameters.

content_type instance-attribute
content_type: str

The MIME type of the inference result.

performance_config_latency class-attribute instance-attribute
performance_config_latency: str | None = ...

Model performance settings for the request.

response_metadata class-attribute instance-attribute
response_metadata: ResponseMetadata = ...

Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.

service_tier class-attribute instance-attribute
service_tier: str | None = ...

Specifies the processing tier type used for serving the request.