FallbackTargetDict¶
Dictionary Type¶
FallbackTargetDict
¶
Bases: TypedDict
Dictionary form of FallbackTarget.
Attributes¶
first_byte_timeout_milliseconds
instance-attribute
¶
first_byte_timeout_milliseconds: int | None
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
model
instance-attribute
¶
model: str | None
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
target_name
instance-attribute
¶
The name of the gateway target to fall back to. The target must be an inference target.
trigger_on
instance-attribute
¶
trigger_on: FallbackTriggers | FallbackTriggersDict | None
The status codes and errors from this target that cause the gateway to
send the request to the next fallback target. This setting isn't
inherited from the route. If you omit it, the gateway uses the default
triggers: 5XX status codes and the CONNECTION_ERROR,
FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.