prompt-executor-model/ai.koog.prompt.executor.llms/RoutingLLMPromptExecutor

RoutingLLMPromptExecutor

open class RoutingLLMPromptExecutor @JvmOverloads constructor(clientRouter: LLMClientRouter, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null) : PromptExecutor(source)

Executes prompts with load balancing across multiple LLM clients.

Delegates client selection to LLMClientRouter, which determines which client should handle each request based on the requested model. This enables load distribution strategies like round-robin, weighted routing, or health-based selection.

Parameters

clientRouter

Router responsible for selecting appropriate clients for each request

fallback

Optional fallback configuration when no client is available for the requested model

This class remains open for source and binary compatibility with existing subclasses. For new custom prompt executors, prefer extending ai.koog.prompt.executor.model.DynamicPromptExecutor. If you subclass this class, prefer overriding the ResolvedModel-based overloads. Override the LLModel-based overloads only when you intentionally take over the full model-resolution and execution flow.

Constructors

RoutingLLMPromptExecutor

@JvmOverloads

constructor(clientRouter: LLMClientRouter, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

@JvmOverloads

constructor(llmClients: Map<LLMProvider, List<LLMClient>>, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a map of providers to their client lists. Uses RoundRobinRouter for load distribution.

@JvmOverloads

constructor(llmClients: List<LLMClient>, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a list of clients. Clients are grouped by provider and routed using RoundRobinRouter.

@JvmOverloads

constructor(vararg llmClients: LLMClient, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a list of clients. Clients are grouped by provider and routed using RoundRobinRouter.

Types

FallbackPromptExecutorSettings

data class FallbackPromptExecutorSettings(val fallbackModel: LLModel)

Represents configuration for a fallback large language model (LLM) execution strategy.

Functions

open override fun close()

execute

open suspend override fun execute(prompt: Prompt, model: ResolvedModel, tools: List<ToolDescriptor> = emptyList()): Message.Assistant

Executes a given prompt using the specified tools and resolved model, and returns the assistant response. Preferred extension point for subclasses that want to preserve this executor's model-resolution flow.

open suspend override fun execute(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor> = emptyList()): Message.Assistant

Executes a given prompt using the specified tools and model, and returns the assistant response.

executeMultipleChoices

open suspend override fun executeMultipleChoices(prompt: Prompt, resolvedModel: ResolvedModel, tools: List<ToolDescriptor>): LLMChoice

Executes a given prompt using the specified tools and resolved model and returns the model choices. Preferred extension point for subclasses that want to preserve this executor's model-resolution flow.

open suspend override fun executeMultipleChoices(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): LLMChoice

Executes a given prompt using the specified tools and model and returns the model choices.

executeStreaming

open override fun executeStreaming(prompt: Prompt, resolvedModel: ResolvedModel, tools: List<ToolDescriptor> = emptyList()): Flow<StreamFrame>

Executes the given prompt with the specified resolved model and streams the response in chunks as a flow. Preferred extension point for subclasses that want to preserve this executor's model-resolution flow.

open override fun executeStreaming(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor> = emptyList()): Flow<StreamFrame>

Executes the given prompt with the specified model and streams the response in chunks as a flow.

executeStructured

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, config: StructuredRequestConfig<T>, fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

inline suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, serializer: KSerializer<T>, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

Executes a prompt with structured output, enhancing it with schema instructions or native structured output parameter, and parses the response into the defined structure.

getBasicJsonSchemaGenerator

open fun getBasicJsonSchemaGenerator(model: LLModel): BasicJsonSchemaGenerator

Basic JSON schema generator required for the given model. Return BasicJsonSchemaGenerator by default.

getStandardJsonSchemaGenerator

open fun getStandardJsonSchemaGenerator(model: LLModel): StandardJsonSchemaGenerator

Standard JSON schema generator required for the given model. Return StandardJsonSchemaGenerator by default.

models

open suspend override fun models(): List<LLModel>

Retrieves a list of available models from all LLM clients managed by this executor.

moderate

open suspend override fun moderate(prompt: Prompt, model: ResolvedModel): ModerationResult

Moderates the provided multi-modal content using the specified resolved model. Preferred extension point for subclasses that want to preserve this executor's model-resolution flow.

open suspend override fun moderate(prompt: Prompt, model: LLModel): ModerationResult

Moderates the provided multi-modal content using the specified model.

parseResponseToStructuredResponse

suspend fun <T> PromptExecutor.parseResponseToStructuredResponse(response: Message.Assistant, config: StructuredRequestConfig<T>, model: LLModel, fixingParser: StructureFixingParser? = null): StructuredResponse<T>

Parses a structured response from the assistant message using the provided structured output configuration and language model. If a fixing parser is specified in the configuration, it will be used; otherwise, the structure will be parsed directly.

resolveModel

open suspend override fun resolveModel(model: LLModel, promptExecutorOperation: PromptExecutorOperation): ResolvedModel

Resolves the LLModel to use for the given promptExecutorOperation by checking whether the router has a client for the model's provider, falling back to FallbackPromptExecutorSettings.fallbackModel when no direct client matches.