gptcache

What it is and what it does

GPTCache is a caching layer for large language model (LLM) API calls that intercepts queries and returns cached responses when an exact or semantically similar question has been asked before. It works by embedding queries into a vector space and comparing them against stored responses, allowing you to skip expensive API calls for repeated or similar questions.

The library integrates with OpenAI's API (and supports a generic get/set adapter for other LLM providers) and can be configured with different embedding models, vector databases (e.g., FAISS), and similarity evaluation strategies. It depends on numpy, cachetools, and requests for its core functionality, though additional optional dependencies are installed on demand for features like semantic embedding and vector storage.

Use it for:

• Cache exact-match queries to OpenAI's ChatCompletion API to avoid redundant API calls and reduce monthly costs.
• Implement semantic similarity matching so that paraphrased or slightly different versions of the same question return cached answers.
• Reduce latency for high-traffic chatbot applications by serving cached responses instead of waiting for LLM API responses.
• Store embeddings and responses in a persistent vector database (e.g., SQLite + FAISS) to maintain cache across application restarts.
• Control cache behavior with temperature parameters to balance between cache hits and fresh LLM responses.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

GPTCache provides semantic caching for LLM API calls, storing and retrieving responses based on exact or semantic similarity matching to reduce API costs and latency.

Yes, with conditions. GPTCache is worth installing if you run high-volume LLM applications and want to reduce API costs and latency through caching. However, be aware that the project is aging (last release August 2024, no commits since July 2025), two security vulnerabilities are known, and the API is explicitly subject to change. Verify the current status of those vulnerabilities and test thoroughly in a staging environment before production use.

Install

gptcache on PyPI

pip install gptcache

uv add gptcache

poetry add gptcache

Installing gptcache

Before you install

Low install friction with three common runtime dependencies (numpy, cachetools, requests). Maintenance status is aging—last release was 2024-08-01 and the repository has not been updated since 2025-07-11, though the project remains active with 8138 stars.

License in practice

MIT license (permissive) allows unrestricted use, modification, and distribution in both open-source and commercial projects.

Quickstart

pip install gptcache

from gptcache import cache
from gptcache.adapter import openai

cache.init()
cache.set_openai_key()
response = openai.ChatCompletion.create(
    model='gpt-3.5-turbo',
    messages=[{'role': 'user', 'content': 'your question'}]
)

Requires Python 3.8.1 or higher and an OpenAI API key set via OPENAI_API_KEY environment variable.

Verify before relying

• Current status of the two known vulnerabilities (GHSA-xfqj-4cr9-9gr5, PYSEC-2026-3468) and whether patches are available.
• Whether the API remains stable given the note that it 'may be subject to change at any time'.
• Support status for models and API shapes beyond OpenAI, given the stated policy to no longer add new model support.

Package facts

Evidence: gptcache-0.1.44-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading