InferenceConfigurationDict¶
Dictionary Type¶
InferenceConfigurationDict
¶
Bases: TypedDict
Dictionary form of InferenceConfiguration.
Attributes¶
max_tokens
instance-attribute
¶
max_tokens: int | None
The maximum number of tokens to generate in the model response during evaluation.
stop_sequences
instance-attribute
¶
The list of sequences that will cause the model to stop generating tokens when encountered.