Skip to content

FallbackTarget

Structure Class

FallbackTarget dataclass

Bases: _SerdeShape

Specifies a target to send the request to when the previous target in the fallback chain fails. Includes the target name, the model to send, and the conditions that trigger a fallback from this target to the next one.

Attributes

first_byte_timeout_milliseconds class-attribute instance-attribute
first_byte_timeout_milliseconds: int | None = ...

The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.

model class-attribute instance-attribute
model: str | None = ...

The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.

target_name instance-attribute
target_name: str

The name of the gateway target to fall back to. The target must be an inference target.

trigger_on class-attribute instance-attribute
trigger_on: FallbackTriggers | None = ...

The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.