npmai

What it is and what it does

npmai is a Python wrapper around a cloud-hosted LLM inference service that lets you call open-source models (Ollama, Llama, Gemma, Qwen, Mistral, and others) without installing them locally or managing API keys. It's designed to eliminate setup friction for developers wanting to experiment with or integrate multiple LLMs into applications. The package includes a RAG (retrieval-augmented generation) module that handles document ingestion—PDFs, images, video, audio, YouTube videos—converting them to text and storing them in a vectorized database, all on the cloud side with no local dependencies.

The service is free and claims to handle significant throughput (80K+ requests per 24 hours) without charging users or developers. It integrates with LangChain and other orchestration frameworks, and offers both a Python SDK and HTTP API endpoints for use from JavaScript, C++, Java, or C. Recent versions (0.1.8–0.1.9) added Supabase integration for long-term document storage and dynamic context retrieval logic to optimize RAG performance.

Use it for:

• Prototype or demo an LLM-powered application without downloading and running Ollama locally.
• Build a RAG chatbot over your own documents (PDFs, images, videos) without writing file-processing code.
• Integrate multiple open-source models into a LangChain or CrewAI workflow via a single Python API.
• Experiment with different model outputs (Llama, Gemma, Qwen, Mistral) by switching a model parameter.
• Access LLM inference from non-Python code (JavaScript, C++, Java) via the HTTP endpoint.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

npmai provides a Python interface to access open-source LLMs like Ollama and 45+ other models through a cloud API, plus RAG (retrieval-augmented generation) tools for processing documents, images, and video without local installation.

Yes, with conditions. Install if you want zero-setup access to multiple open-source LLMs and RAG tooling without local compute or API keys. The low install friction, permissive license, and active maintenance make it a reasonable experiment. However, verify uptime guarantees and data privacy for the cloud-hosted vectorized documents before using in production; the free tier's long-term reliability is unproven.

Install

npmai on PyPI

pip install npmai

uv add npmai

poetry add npmai

Installing npmai

Before you install

Low install friction: pure Python wheel with a single runtime dependency (requests). Active maintenance with recent releases; last commit 2026-07-12. No known vulnerabilities.

License in practice

MIT license is permissive—you can use, modify, and distribute npmai freely in commercial or private projects with minimal restrictions.

Quickstart

pip install npmai

from npmai import Ollama

llm = Ollama()
response = llm.invoke("What is the future of AI?", model="llama3.2")
print(response)

Requires Python 3.9 or later (up to 3.13); cloud API endpoint must be reachable.

Verify before relying

• Whether the claimed 1.2 million+ installations and 80K+ daily requests are independently verifiable or self-reported.
• Actual availability and uptime guarantees of the free cloud endpoints powering the service.
• Whether all 45+ models are equally stable or if some are experimental/unstable.
• Data retention and privacy policy for vectorized documents stored via Supabase integration.

Package facts

Evidence: npmai-0.1.9-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading