invoke_endpoint_with_bidirectional_stream¶
Operation¶
Async only
This operation uses an event stream, so only the asynchronous client supports it.
invoke_endpoint_with_bidirectional_stream
async
¶
invoke_endpoint_with_bidirectional_stream(request: InvokeEndpointWithBidirectionalStreamInput | InvokeEndpointWithBidirectionalStreamInputDict | None = None, /, *, plugins: list[Plugin] | None = None, **kwargs: Unpack[InvokeEndpointWithBidirectionalStreamInputDict]) -> AsyncDuplexEventStream[RequestStreamEvent, ResponseStreamEvent, InvokeEndpointWithBidirectionalStreamOutput]
Invokes a model endpoint with bidirectional streaming capabilities. This operation establishes a persistent connection that allows you to send multiple requests and receive streaming responses from the model in real-time.
Bidirectional streaming is useful for interactive applications such as chatbots, real-time translation, or any scenario where you need to maintain a conversation-like interaction with the model. The connection remains open, allowing you to send additional input and receive responses without establishing a new connection for each request.
For an overview of Amazon SageMaker AI, see How It Works.
Amazon SageMaker AI strips all POST headers except those supported by the API. Amazon SageMaker AI might add additional headers. You should not rely on the behavior of headers outside those enumerated in the request syntax.
Calls to InvokeEndpointWithBidirectionalStream are authenticated by
using Amazon Web Services Signature Version 4. For information, see
Authenticating Requests (Amazon Web Services Signature Version
4)
in the Amazon S3 API Reference.
The bidirectional stream maintains the connection until either the client closes it or the model indicates completion. Each request and response in the stream is sent as an event with optional headers for data type and completion state.
Note
Endpoints are scoped to an individual account, and are not public. The URL does not contain the account ID, but Amazon SageMaker AI determines the account ID from the authentication token that is supplied by the caller.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
request
|
InvokeEndpointWithBidirectionalStreamInput | InvokeEndpointWithBidirectionalStreamInputDict | None
|
The input as a model or dictionary. Pass this or keyword arguments, not both. |
None
|
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
**kwargs
|
Unpack[InvokeEndpointWithBidirectionalStreamInputDict]
|
The input fields as keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
AsyncDuplexEventStream[RequestStreamEvent, ResponseStreamEvent, InvokeEndpointWithBidirectionalStreamOutput]
|
An |
Input¶
Pass the input as keyword arguments, a positional dictionary, or a positional model object. Pass plugins separately as a keyword argument.
InvokeEndpointWithBidirectionalStreamInput
dataclass
¶
Bases: _SerdeShape
Dataclass for InvokeEndpointWithBidirectionalStreamInput structure.
Attributes¶
endpoint_name
class-attribute
instance-attribute
¶
endpoint_name: str | None = ...
The name of the endpoint to invoke.
model_invocation_path
class-attribute
instance-attribute
¶
model_invocation_path: str | None = ...
Model invocation path.
Output¶
This operation returns an AsyncDuplexEventStream for bidirectional streaming.
Event Stream Structure¶
Input Event Type¶
Output Event Type¶
Initial Response Structure¶
InvokeEndpointWithBidirectionalStreamOutput
dataclass
¶
Bases: _SerdeShape
Dataclass for InvokeEndpointWithBidirectionalStreamOutput structure.
Attributes¶
invoked_production_variant
class-attribute
instance-attribute
¶
invoked_production_variant: str | None = ...
The invoked production variant.
response_metadata
class-attribute
instance-attribute
¶
Metadata about the response that produced this output. Use this to recover the request identifiers a service's support team needs in order to investigate a call. Members of the metadata are individually optional.