llama-index-llms-litellm

What it is and what it does

This package bridges LiteLLM and LlamaIndex, allowing you to use multiple LLM providers through LlamaIndex's standardized interface. Instead of writing provider-specific code for OpenAI, Cohere, or other services, you instantiate a single LiteLLM object with a model identifier and call the same methods—chat, complete, stream_chat, stream_complete, and async variants—regardless of which backend you're targeting.

The integration handles the translation between LlamaIndex's message and response formats and LiteLLM's provider abstractions. You set API keys as environment variables and switch providers by changing the model string passed to the LiteLLM constructor. It supports both synchronous and asynchronous calls, as well as streaming responses.

Use it for:

• Build a RAG pipeline that can swap between OpenAI and Cohere without changing application code.
• Prototype LLM features against multiple providers to compare cost, latency, or output quality.
• Run LlamaIndex agents that fall back to alternative providers if one API is unavailable.
• Implement multi-provider chat applications where users can select their preferred LLM backend.
• Stream LLM responses in a LlamaIndex application without rewriting for each provider's API.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates LiteLLM with LlamaIndex to provide unified access to multiple LLM providers (OpenAI, Cohere, and others) through a single interface.

Yes, if you are building with LlamaIndex and need to support multiple LLM providers without provider-specific code. The low install friction, permissive license, active maintenance status, and lack of known vulnerabilities make it a straightforward choice. Install only if you actually need multi-provider flexibility; single-provider projects may not benefit.

Install

llama-index-llms-litellm on PyPI

pip install llama-index-llms-litellm

uv add llama-index-llms-litellm

poetry add llama-index-llms-litellm

Installing llama-index-llms-litellm

Before you install

Low install friction with just two runtime dependencies. Marked active with a recent release in 2026, though repository details are not publicly available.

License in practice

MIT license permits commercial and private use with minimal restrictions.

Quickstart

pip install llama-index-llms-litellm

from llama_index.llms.litellm import LiteLLM
from llama_index.core.llms import ChatMessage
import os

os.environ["OPENAI_API_KEY"] = "your-key"
message = ChatMessage(role="user", content="Hello")
llm = LiteLLM("gpt-3.5-turbo")
response = llm.chat([message])
print(response)

Requires Python 3.10 or later and valid API keys for the LLM providers you intend to use.

Verify before relying

• Whether LiteLLM's full provider roster is accessible through this integration or only a subset
• Performance characteristics and latency overhead of the abstraction layer
• Whether streaming and async features work uniformly across all supported providers

Package facts

Evidence: llama_index_llms_litellm-0.7.1-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading