agents-features-tokenizer/ai.koog.agents.features.tokenizer.feature/CachingTokenizer

CachingTokenizer

class CachingTokenizer(tokenizer: Tokenizer) : PromptTokenizer(source)

A caching implementation of the PromptTokenizer interface that optimizes token counting by storing previously computed token counts for messages. This reduces redundant computations when the same message is processed multiple times.

Constructors

CachingTokenizer

constructor(tokenizer: Tokenizer)

Creates an instance of CachingTokenizer with a provided Tokenizer instance that performs the actual token counting.

Functions

clearCache

fun clearCache()

Clears all cached token counts from the internal cache.

tokenCountFor

open override fun tokenCountFor(prompt: Prompt): Int

Calculates the total number of tokens spent on the given prompt by summing the token usage of all messages associated with the prompt.

open override fun tokenCountFor(message: Message): Int

Retrieves the number of tokens contained in the content of the given message. This method utilizes caching to improve performance, storing previously computed token counts and reusing them for identical messages.