FallbackTarget¶
Structure Class¶
FallbackTarget
dataclass
¶
Bases: _SerdeShape
Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.
Attributes¶
first_byte_timeout_milliseconds
class-attribute
instance-attribute
¶
first_byte_timeout_milliseconds: int | None = ...
The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.
model
class-attribute
instance-attribute
¶
model: str | None = ...
The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.
target_name
instance-attribute
¶
target_name: str
The name of the gateway target to fall back to. The target must be an inference target.
trigger_on
class-attribute
instance-attribute
¶
trigger_on: FallbackTriggers | None = ...
The status codes and errors from this target that cause the gateway to
send the request to the next fallback target. This setting isn't
inherited from the route. If you omit it, the gateway uses the default
triggers: 5XX status codes and the CONNECTION_ERROR,
FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.