Skip to content

TokenUsage

Structure Class

TokenUsage dataclass

Bases: _SerdeShape

The tokens used in a message API inference call.

Attributes

cache_details class-attribute instance-attribute
cache_details: list[CacheDetail] | None = ...

Detailed breakdown of cache writes by TTL. Empty if no cache creation occurred. Sorted by TTL duration (1h before 5m).

cache_read_input_tokens class-attribute instance-attribute
cache_read_input_tokens: int | None = ...

The number of input tokens read from the cache for the request.

cache_write_input_tokens class-attribute instance-attribute
cache_write_input_tokens: int | None = ...

The number of input tokens written to the cache for the request.

input_tokens instance-attribute
input_tokens: int

The number of tokens sent in the request to the model.

output_tokens instance-attribute
output_tokens: int

The number of tokens that the model generated for the request.

total_tokens instance-attribute
total_tokens: int

The total of input tokens and tokens generated by the model.