prompt-executor-ollama-client
A client implementation for executing prompts using local Ollama models with limited multimodal support.
Overview
This module provides a client implementation for the Ollama API, allowing you to execute prompts using locally hosted models. Ollama enables running large language models locally on your machine. The client currently supports text and basic image processing capabilities.
Supported Models
Predefined Models (Groq)
Model | Speed | Input Support | Output Support | Requirements |
---|---|---|---|---|
llama3-groq-tool-use:8b | Fast | Text, Tools | Text, Tools | 8GB+ RAM |
llama3-groq-tool-use:70b | Medium | Text, Tools | Text, Tools | 32GB+ RAM |
Predefined Models (Meta)
Model | Speed | Input Support | Output Support | Requirements |
---|---|---|---|---|
llama3.2:3b | Very Fast | Text, Tools | Text, Tools | 4GB+ RAM |
llama3.2 | Fast | Text, Tools | Text, Tools | 8GB+ RAM |
llama4:latest | Medium | Text, Tools | Text, Tools | 16GB+ RAM |
Predefined Models (Alibaba)
Model | Speed | Input Support | Output Support | Requirements |
---|---|---|---|---|
qwen2.5:0.5b | Very Fast | Text, Tools | Text, Tools | 2GB+ RAM |
qwen3:0.6b | Very Fast | Text, Tools | Text, Tools | 2GB+ RAM |
qwq | Fast | Text, Tools | Text, Tools | 8GB+ RAM |
qwen2.5-coder:32b | Medium | Text, Tools | Text, Tools | 32GB+ RAM |
Dynamic Models
Any model available in your local Ollama installation can be used by creating a dynamic model.
Media Content Support
Content Type | Supported Formats | Max Size | Notes |
---|---|---|---|
Images | ❌ Not implemented | - | API supports it but client doesn't |
Audio | ❌ Not supported | - | - |
Video | ❌ Not supported | - | - |
Documents | ❌ Not supported | - | Use text extraction |
Important: While the Ollama API supports images via the images
field, this client implementation does not currently process MediaContent.Image
. Only text content is supported.
Using in your project
Add the dependency to your project:
dependencies {
implementation("ai.koog.prompt:prompt-executor-ollama-client:$version")
}
Configure the client with your Ollama server:
val ollamaClient = OllamaLLMClient(
baseUrl = "http://localhost:11434", // Default Ollama server
)
Example of usage
suspend fun main() {
val client = OllamaLLMClient(
baseUrl = "http://localhost:11434",
)
// Text-only example
val response = client.execute(
prompt = prompt {
system("You are helpful assistant")
user("What time is it now?")
},
model = OllamaModels.Meta.LLAMA_3_2_3B
)
println(response)
}
Usage Examples
// Tool usage example
val toolResponse = client.execute(
prompt = prompt {
user("What's the weather like?")
},
model = OllamaModels.Meta.LLAMA_3_2,
tools = listOf(weatherTool)
)
// Code generation
val codeResponse = client.execute(
prompt = prompt {
user("Write a Python function to calculate fibonacci numbers")
},
model = OllamaModels.Alibaba.QWEN_CODER_2_5_32B
)
// Dynamic model usage
val dynamicModel = client.createDynamicModel("codellama:13b")
val dynamicResponse = client.execute(
prompt = prompt {
user("Explain recursion")
},
model = dynamicModel
)
// Embedding example
val embeddingModel = client.createDynamicModel("nomic-embed-text")
val embedding = client.embed(
text = "This is sample text for embedding",
model = embeddingModel
)
Setup Instructions
Install Ollama: https://ollama.ai/
Pull a model:
ollama pull llama3.1:8b
Start Ollama server:
ollama serve
Use the client to connect to your local instance