agents-features-tokenizer/ai.koog.agents.features.tokenizer.feature/MessageTokenizerConfig

MessageTokenizerConfig

class MessageTokenizerConfig : FeatureConfig(source)

Configuration class for message tokenization settings.

This class specifies the tokenizer to be used and whether caching should be enabled for tokenization. It extends the base FeatureConfig class, allowing it to be integrated within a feature-driven system.

Constructors

constructor()

Properties

enableCaching

var enableCaching: Boolean

Indicates whether caching is enabled for tokenization processes.

messageProcessor

val messageProcessor: List<FeatureMessageProcessor>

tokenizer

var tokenizer: Tokenizer

The tokenizer property determines the strategy used for tokenizing text and estimating token counts within a message-processing feature.

Functions

addMessageProcessor

fun addMessageProcessor(processor: FeatureMessageProcessor)