InferenceConfigurationDict¶
Dictionary Type¶
InferenceConfigurationDict
¶
Bases: TypedDict
Dictionary form of InferenceConfiguration.
Attributes¶
max_tokens
instance-attribute
¶
max_tokens: int | None
The maximum number of tokens to allow in the generated response. The default value is the maximum allowed value for the model that you are using.
stop_sequences
instance-attribute
¶
A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.
temperature
instance-attribute
¶
temperature: float | None
The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.