prompt-executor-model/ai.koog.prompt.executor.llms/RoutingLLMPromptExecutor

RoutingLLMPromptExecutor

open class RoutingLLMPromptExecutor @JvmOverloads constructor(clientRouter: LLMClientRouter, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null) : PromptExecutor(source)

Executes prompts with load balancing across multiple LLM clients.

Delegates client selection to LLMClientRouter, which determines which client should handle each request based on the requested model. This enables load distribution strategies like round-robin, weighted routing, or health-based selection.

Parameters

clientRouter

Router responsible for selecting appropriate clients for each request

fallback

Optional fallback configuration when no client is available for the requested model

Constructors

RoutingLLMPromptExecutor

@JvmOverloads

constructor(clientRouter: LLMClientRouter, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

@JvmOverloads

constructor(llmClients: Map<LLMProvider, List<LLMClient>>, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a map of providers to their client lists. Uses RoundRobinRouter for load distribution.

@JvmOverloads

constructor(llmClients: List<LLMClient>, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a list of clients. Clients are grouped by provider and routed using RoundRobinRouter.

@JvmOverloads

constructor(vararg llmClients: LLMClient, fallback: RoutingLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Creates executor with a list of clients. Clients are grouped by provider and routed using RoundRobinRouter.

Types

FallbackPromptExecutorSettings

data class FallbackPromptExecutorSettings(val fallbackModel: LLModel)

Represents configuration for a fallback large language model (LLM) execution strategy.

Functions

open override fun close()

execute

open suspend override fun execute(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): List<Message.Response>

Executes a given prompt using the specified tools and model, and returns a list of response messages.

executeMultipleChoices

open suspend override fun executeMultipleChoices(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): List<LLMChoice>

Executes a given prompt using the specified tools and model and returns a list of model choices.

executeStreaming

open override fun executeStreaming(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): Flow<StreamFrame>

Executes the given prompt with the specified model and streams the response in chunks as a flow.

executeStructured

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, config: StructuredRequestConfig<T>, fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

inline suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, serializer: KSerializer<T>, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

Executes a prompt with structured output, enhancing it with schema instructions or native structured output parameter, and parses the response into the defined structure.

getBasicJsonSchemaGenerator

open fun getBasicJsonSchemaGenerator(model: LLModel): BasicJsonSchemaGenerator

Basic JSON schema generator required for the given model. Return BasicJsonSchemaGenerator by default.

getStandardJsonSchemaGenerator

open fun getStandardJsonSchemaGenerator(model: LLModel): StandardJsonSchemaGenerator

Standard JSON schema generator required for the given model. Return StandardJsonSchemaGenerator by default.

models

open suspend override fun models(): List<LLModel>

Retrieves a list of available models from all LLM clients managed by this executor.

moderate

open suspend override fun moderate(prompt: Prompt, model: LLModel): ModerationResult

Moderates the provided multi-modal content using the specified model.

parseResponseToStructuredResponse

suspend fun <T> PromptExecutor.parseResponseToStructuredResponse(response: Message.Assistant, config: StructuredRequestConfig<T>, model: LLModel, fixingParser: StructureFixingParser? = null): StructuredResponse<T>

Parses a structured response from the assistant message using the provided structured output configuration and language model. If a fixing parser is specified in the configuration, it will be used; otherwise, the structure will be parsed directly.