Skip to content

InferenceProviderTargetConfiguration

Structure Class

InferenceProviderTargetConfiguration dataclass

Bases: _SerdeShape

The configuration for a provider-based inference target. This configuration explicitly defines the endpoint, model mapping, and operations used to route requests to a large language model (LLM) provider.

Attributes

endpoint instance-attribute
endpoint: str

The HTTPS endpoint of the inference provider that the gateway forwards requests to.

model_mapping class-attribute instance-attribute
model_mapping: ModelMapping | None = ...

The configuration that translates client-facing model IDs to the model IDs expected by the provider.

operations class-attribute instance-attribute
operations: list[InferenceOperationConfiguration] | None = ...

A list of per-operation configurations that map request paths to the models supported for each operation.