prompt-executor-ollama-client

A client implementation for executing prompts using local Ollama models with limited multimodal support.

Overview

This module provides a client implementation for the Ollama API, allowing you to execute prompts using locally hosted models. Ollama enables running large language models locally on your machine. The client currently supports text and basic image processing capabilities.

Supported Models

Predefined Models (Groq)

ModelSpeedInput SupportOutput SupportRequirements
llama3-groq-tool-use:8bFastText, ToolsText, Tools8GB+ RAM
llama3-groq-tool-use:70bMediumText, ToolsText, Tools32GB+ RAM

Predefined Models (Meta)

ModelSpeedInput SupportOutput SupportRequirements
llama3.2:3bVery FastText, ToolsText, Tools4GB+ RAM
llama3.2FastText, ToolsText, Tools8GB+ RAM
llama4:latestMediumText, ToolsText, Tools16GB+ RAM

Predefined Models (Alibaba)

ModelSpeedInput SupportOutput SupportRequirements
qwen2.5:0.5bVery FastText, ToolsText, Tools2GB+ RAM
qwen3:0.6bVery FastText, ToolsText, Tools2GB+ RAM
qwqFastText, ToolsText, Tools8GB+ RAM
qwen2.5-coder:32bMediumText, ToolsText, Tools32GB+ RAM

Dynamic Models

Any model available in your local Ollama installation can be used by creating a dynamic model.

Media Content Support

Content TypeSupported FormatsMax SizeNotes
Images❌ Not implemented-API supports it but client doesn't
Audio❌ Not supported--
Video❌ Not supported--
Documents❌ Not supported-Use text extraction

Important: While the Ollama API supports images via the images field, this client implementation does not currently process MediaContent.Image. Only text content is supported.

Using in your project

Add the dependency to your project:

dependencies {
implementation("ai.koog.prompt:prompt-executor-ollama-client:$version")
}

Configure the client with your Ollama server:

val ollamaClient = OllamaLLMClient(
baseUrl = "http://localhost:11434", // Default Ollama server
)

Example of usage

suspend fun main() {
val client = OllamaLLMClient(
baseUrl = "http://localhost:11434",
)

// Text-only example
val response = client.execute(
prompt = prompt {
system("You are helpful assistant")
user("What time is it now?")
},
model = OllamaModels.Meta.LLAMA_3_2_3B
)

println(response)
}

Usage Examples

// Tool usage example
val toolResponse = client.execute(
prompt = prompt {
user("What's the weather like?")
},
model = OllamaModels.Meta.LLAMA_3_2,
tools = listOf(weatherTool)
)

// Code generation
val codeResponse = client.execute(
prompt = prompt {
user("Write a Python function to calculate fibonacci numbers")
},
model = OllamaModels.Alibaba.QWEN_CODER_2_5_32B
)

// Dynamic model usage
val dynamicModel = client.createDynamicModel("codellama:13b")
val dynamicResponse = client.execute(
prompt = prompt {
user("Explain recursion")
},
model = dynamicModel
)

// Embedding example
val embeddingModel = client.createDynamicModel("nomic-embed-text")
val embedding = client.embed(
text = "This is sample text for embedding",
model = embeddingModel
)

Setup Instructions

  1. Install Ollama: https://ollama.ai/

  2. Pull a model: ollama pull llama3.1:8b

  3. Start Ollama server: ollama serve

  4. Use the client to connect to your local instance

Packages

Link copied to clipboard
common
common