llama-index-llms-ollama

What it is and what it does

This package bridges LlamaIndex and Ollama, allowing you to run local language models through a standardized LLM interface. Instead of calling Ollama's API directly, you instantiate an Ollama object with a model name and interact with it via familiar LlamaIndex methods: complete() for text generation, chat() for multi-turn conversations, and stream_complete() or stream_chat() for incremental responses. It also supports JSON mode for structured outputs and Pydantic-based response schemas.

The package assumes Ollama is already running locally on your machine (typically on localhost:11434). You configure the model, request timeout, and optional JSON mode when creating the Ollama instance. It's a thin adapter layer that translates LlamaIndex's LLM protocol into Ollama API calls, making it straightforward to swap local models into LlamaIndex workflows without rewriting application code.

Use it for:

• Build RAG pipelines with local models using LlamaIndex, avoiding cloud API costs and latency.
• Stream long-form text generation from local Ollama models in real-time chat applications.
• Extract structured data (JSON objects, Pydantic models) from local LLMs for downstream processing.
• Prototype and test LLM-powered features locally before migrating to production endpoints.
• Combine Ollama with LlamaIndex's retrieval and indexing tools for document Q&A on private data.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates local Ollama language models with LlamaIndex, enabling text completion, chat, streaming, and structured JSON responses through a unified Python interface.

Yes, if you are already using LlamaIndex and want to run models locally via Ollama. Install friction is minimal, maintenance is active, and the MIT license is permissive. The package is straightforward to set up once Ollama is running. No vulnerabilities are known. The main prerequisite is having Ollama installed and a model downloaded locally.

Install

llama-index-llms-ollama on PyPI

pip install llama-index-llms-ollama

uv add llama-index-llms-ollama

poetry add llama-index-llms-ollama

Installing llama-index-llms-ollama

Before you install

Low friction: pure Python wheel with only two runtime dependencies (llama-index-core and ollama). Marked active with a recent release (147 days ago). No known vulnerabilities.

License in practice

MIT license permits commercial and private use with minimal restrictions—suitable for most projects.

Quickstart

pip install llama-index-llms-ollama

from llama_index.llms.ollama import Ollama
from llama_index.core.llms import ChatMessage

llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
resp = llm.complete("Who is Paul Graham?")
print(resp)

Requires a running local Ollama instance serving models on localhost:11434; Python 3.10 or later.

Verify before relying

• Performance characteristics and latency when streaming large responses.
• Compatibility with specific Ollama model versions beyond the examples shown.
• Memory requirements for concurrent requests or large model loads.

Package facts

Evidence: llama_index_llms_ollama-0.10.1-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading