Skip to content

FallbackTargetDict

Dictionary Type

FallbackTargetDict

Bases: TypedDict

Dictionary form of FallbackTarget.

Attributes

first_byte_timeout_milliseconds instance-attribute
first_byte_timeout_milliseconds: int | None

The maximum time, in milliseconds, from sending the request to this target until the first byte of the response body arrives. This setting isn't inherited from the route. If you don't specify a value, no timeout is enforced.

model instance-attribute
model: str | None

The model ID to send to this fallback target. If you don't specify a model, the gateway sends the model from the original request.

target_name instance-attribute
target_name: Required[str]

The name of the gateway target to fall back to. The target must be an inference target.

trigger_on instance-attribute

The status codes and errors from this target that cause the gateway to send the request to the next fallback target. This setting isn't inherited from the route. If you omit it, the gateway uses the default triggers: 5XX status codes and the CONNECTION_ERROR, FIRST_BYTE_TIMEOUT, and UNROUTABLE errors.