InferenceProviderTargetConfigurationDict¶
Dictionary Type¶
InferenceProviderTargetConfigurationDict
¶
Bases: TypedDict
Dictionary form of InferenceProviderTargetConfiguration.
Attributes¶
endpoint
instance-attribute
¶
The HTTPS endpoint of the inference provider that the gateway forwards requests to.
model_mapping
instance-attribute
¶
model_mapping: ModelMapping | ModelMappingDict | None
The configuration that translates client-facing model IDs to the model IDs expected by the provider.
operations
instance-attribute
¶
operations: Sequence[InferenceOperationConfiguration | InferenceOperationConfigurationDict] | None
A list of per-operation configurations that map request paths to the models supported for each operation.