Skip to content

InferenceOperationConfiguration

Structure Class

InferenceOperationConfiguration dataclass

Bases: _SerdeShape

The configuration for a specific inference operation, including its request path and the models that the operation supports.

Attributes

models class-attribute instance-attribute
models: list[ModelEntry] | None = ...

The list of models supported for this operation.

path instance-attribute
path: str

The request path for this operation (for example, /v1/messages or /v1/responses).

provider_path class-attribute instance-attribute
provider_path: str | None = ...

The provider path to forward requests to, if it differs from the request path. For example, /anthropic/v1/messages when the provider expects a different path than the client-facing /v1/messages.