agents-features-tokenizer/ai.koog.agents.features.tokenizer.feature

Package-level declarations

Types

Link copied to clipboard
class MessageTokenizer(val promptTokenizer: PromptTokenizer)

The MessageTokenizer feature is responsible for handling tokenization of messages using a provided Tokenizer implementation.

Link copied to clipboard
class MessageTokenizerConfig : FeatureConfig

Configuration class for message tokenization settings.

Functions

Link copied to clipboard
fun AIAgentContext.tokenizer(): PromptTokenizer

Provides access to the PromptTokenizer instance provided by the MessageTokenizer.Feature and used within the AI agent's context.