Skip to content

invoke_endpoint_with_bidirectional_stream

Operation

Async only

This operation uses an event stream, so only the asynchronous client supports it.

invoke_endpoint_with_bidirectional_stream async

Invokes a model endpoint with bidirectional streaming capabilities. This operation establishes a persistent connection that allows you to send multiple requests and receive streaming responses from the model in real-time.

Bidirectional streaming is useful for interactive applications such as chatbots, real-time translation, or any scenario where you need to maintain a conversation-like interaction with the model. The connection remains open, allowing you to send additional input and receive responses without establishing a new connection for each request.

For an overview of Amazon SageMaker AI, see How It Works.

Amazon SageMaker AI strips all POST headers except those supported by the API. Amazon SageMaker AI might add additional headers. You should not rely on the behavior of headers outside those enumerated in the request syntax.

Calls to InvokeEndpointWithBidirectionalStream are authenticated by using Amazon Web Services Signature Version 4. For information, see Authenticating Requests (Amazon Web Services Signature Version 4) in the Amazon S3 API Reference.

The bidirectional stream maintains the connection until either the client closes it or the model indicates completion. Each request and response in the stream is sent as an event with optional headers for data type and completion state.

Note

Endpoints are scoped to an individual account, and are not public. The URL does not contain the account ID, but Amazon SageMaker AI determines the account ID from the authentication token that is supplied by the caller.

Parameters:

Name Type Description Default
request InvokeEndpointWithBidirectionalStreamInput | InvokeEndpointWithBidirectionalStreamInputDict | None

The input as a model or dictionary. Pass this or keyword arguments, not both.

None
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

None
**kwargs Unpack[InvokeEndpointWithBidirectionalStreamInputDict]

The input fields as keyword arguments.

{}

Returns:

Type Description
AsyncDuplexEventStream[RequestStreamEvent, ResponseStreamEvent, InvokeEndpointWithBidirectionalStreamOutput]

An AsyncDuplexEventStream for bidirectional streaming.

Input

Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.

InvokeEndpointWithBidirectionalStreamInputDict

Bases: TypedDict

Dictionary form of InvokeEndpointWithBidirectionalStreamInput.

Attributes

endpoint_name instance-attribute
endpoint_name: Required[str]

The name of the endpoint to invoke.

model_invocation_path instance-attribute
model_invocation_path: str | None

Model invocation path.

model_query_string instance-attribute
model_query_string: str | None

Model query string.

target_variant instance-attribute
target_variant: str | None

Target variant for the request.

InvokeEndpointWithBidirectionalStreamInput dataclass

Bases: _SerdeShape

Dataclass for InvokeEndpointWithBidirectionalStreamInput structure.

Attributes

endpoint_name class-attribute instance-attribute
endpoint_name: str | None = ...

The name of the endpoint to invoke.

model_invocation_path class-attribute instance-attribute
model_invocation_path: str | None = ...

Model invocation path.

model_query_string class-attribute instance-attribute
model_query_string: str | None = ...

Model query string.

target_variant class-attribute instance-attribute
target_variant: str | None = ...

Target variant for the request.

Output

This operation returns an AsyncDuplexEventStream for bidirectional streaming.

Event Stream Structure

Input Event Type

RequestStreamEvent

Output Event Type

ResponseStreamEvent

Initial Response Structure

InvokeEndpointWithBidirectionalStreamOutput dataclass

Bases: _SerdeShape

Dataclass for InvokeEndpointWithBidirectionalStreamOutput structure.

Attributes
invoked_production_variant class-attribute instance-attribute
invoked_production_variant: str | None = ...

The invoked production variant.

response_metadata class-attribute instance-attribute
response_metadata: ResponseMetadata = ...

Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.