Skip to content

EvaluationDataset

Structure Class

EvaluationDataset dataclass

Bases: _SerdeShape

Used to specify the name of a built-in prompt dataset and optionally, the Amazon S3 bucket where a custom prompt dataset is saved.

Attributes

dataset_location class-attribute instance-attribute
dataset_location: EvaluationDatasetLocation | None = ...

For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.

name instance-attribute
name: str

Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2.