PromptModelInferenceConfigurationDict¶
Dictionary Type¶
PromptModelInferenceConfigurationDict
¶
Bases: TypedDict
Dictionary form of PromptModelInferenceConfiguration.
Attributes¶
max_tokens
instance-attribute
¶
max_tokens: int | None
The maximum number of tokens to return in the response.
stop_sequences
instance-attribute
¶
A list of strings that define sequences after which the model will stop generating.