prompt-executor-model/ai.koog.prompt.executor.llms/MultiLLMPromptExecutor

MultiLLMPromptExecutor

open class MultiLLMPromptExecutor @JvmOverloads constructor(llmClients: Map<LLMProvider, LLMClient>, fallback: MultiLLMPromptExecutor.FallbackPromptExecutorSettings? = null) : PromptExecutor(source)

MultiLLMPromptExecutor is a class responsible for executing prompts across multiple Large Language Models (LLMs). This implementation supports direct execution with specific LLM clients or utilizes a fallback strategy if no primary LLM client is available for the requested provider.

Parameters

llmClients

A map containing LLM providers associated with their respective LLMClients.

fallback

Optional settings to configure the fallback mechanism in case a specific provider is not directly available.

This class remains open for source and binary compatibility with existing subclasses. For new custom prompt executors, prefer extending ai.koog.prompt.executor.model.DynamicPromptExecutor. If you subclass this class, prefer overriding the ResolvedModel-based overloads. Override the LLModel-based overloads only when you intentionally take over the full model-resolution and execution flow.

Constructors

MultiLLMPromptExecutor

@JvmOverloads

constructor(llmClients: Map<LLMProvider, LLMClient>, fallback: MultiLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Constructs an executor instance with a map of LLM providers associated with their respective clients.

@JvmOverloads

constructor(vararg llmClients: Pair<LLMProvider, LLMClient>, fallback: MultiLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Initializes a new instance of the MultiLLMPromptExecutor class with multiple LLM clients.

@JvmOverloads

constructor(llmClients: List<LLMClient>, fallback: MultiLLMPromptExecutor.FallbackPromptExecutorSettings? = null)

Secondary constructor for MultiLLMPromptExecutor that accepts a list of LLMClient instances. The provided clients are processed to create a mapping of LLMProvider to their respective LLMClient.

@JvmOverloads

constructor(vararg llmClients: LLMClient)

Secondary constructor for MultiLLMPromptExecutor that accepts a variable number of LLMClient instances. The provided clients are processed to create a mapping of LLMProvider to their respective LLMClient.

Types

FallbackPromptExecutorSettings

data class FallbackPromptExecutorSettings(val fallbackProvider: LLMProvider, val fallbackModel: LLModel)

Represents configuration for a fallback large language model (LLM) execution strategy.

Functions

open override fun close()

execute

open suspend override fun execute(prompt: Prompt, model: ResolvedModel, tools: List<ToolDescriptor> = emptyList()): Message.Assistant

Executes a given prompt using the specified tools and resolved model, and returns the assistant response.

open suspend override fun execute(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor> = emptyList()): Message.Assistant

Executes a given prompt using the specified tools and model, and returns the assistant response.

executeMultipleChoices

open suspend override fun executeMultipleChoices(prompt: Prompt, resolvedModel: ResolvedModel, tools: List<ToolDescriptor>): LLMChoice

open suspend override fun executeMultipleChoices(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor>): LLMChoice

Executes a given prompt using the specified tools and model and returns a list of model choices.

executeStreaming

open override fun executeStreaming(prompt: Prompt, resolvedModel: ResolvedModel, tools: List<ToolDescriptor> = emptyList()): Flow<StreamFrame>

open override fun executeStreaming(prompt: Prompt, model: LLModel, tools: List<ToolDescriptor> = emptyList()): Flow<StreamFrame>

Executes the given prompt with the specified model and streams the response in chunks as a flow.

executeStructured

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, config: StructuredRequestConfig<T>, fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

inline suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

suspend fun <T> PromptExecutor.executeStructured(prompt: Prompt, model: LLModel, serializer: KSerializer<T>, examples: List<T> = emptyList(), fixingParser: StructureFixingParser? = null): Result<StructuredResponse<T>>

Executes a prompt with structured output, enhancing it with schema instructions or native structured output parameter, and parses the response into the defined structure.

getBasicJsonSchemaGenerator

open override fun getBasicJsonSchemaGenerator(model: LLModel): BasicJsonSchemaGenerator

Basic JSON schema generator required for the given model. Return BasicJsonSchemaGenerator by default.

getStandardJsonSchemaGenerator

open override fun getStandardJsonSchemaGenerator(model: LLModel): StandardJsonSchemaGenerator

Standard JSON schema generator required for the given model. Return StandardJsonSchemaGenerator by default.

models

open suspend override fun models(): List<LLModel>

Retrieves a list of available models from all LLM clients managed by this executor.

moderate

open suspend override fun moderate(prompt: Prompt, model: ResolvedModel): ModerationResult

Moderates the provided multi-modal content using the specified resolved model.

open suspend override fun moderate(prompt: Prompt, model: LLModel): ModerationResult

Moderates the provided multi-modal content using the specified model.

parseResponseToStructuredResponse

suspend fun <T> PromptExecutor.parseResponseToStructuredResponse(response: Message.Assistant, config: StructuredRequestConfig<T>, model: LLModel, fixingParser: StructureFixingParser? = null): StructuredResponse<T>

Parses a structured response from the assistant message using the provided structured output configuration and language model. If a fixing parser is specified in the configuration, it will be used; otherwise, the structure will be parsed directly.

resolveModel

open suspend override fun resolveModel(model: LLModel, promptExecutorOperation: PromptExecutorOperation): ResolvedModel

Resolves the LLModel to use for the given promptExecutorOperation by selecting a known client for the model's provider, falling back to FallbackPromptExecutorSettings.fallbackModel when no direct client matches.