{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/4"}],"enrichment":{"capability":"Python bindings for llama.cpp that enable running large language models locally with CPU or GPU acceleration, offering both low-level C API access and high-level text completion APIs compatible with OpenAI, LangChain, and LlamaIndex.","skillfed_tags":["llm-inference","local-models","gpu-acceleration"],"use_cases":["Run private language models locally without sending data to cloud APIs","Build OpenAI-compatible applications that work offline or on restricted networks","Integrate local inference into existing application frameworks for RAG or agent workflows","Deploy a local code completion server for development environments","Serve multiple models simultaneously via the built-in web server for multi-tenant inference","Accelerate inference on specific hardware (GPU, Apple Silicon) for production workloads"],"what_it_does":"llama-cpp-python wraps the llama.cpp C library to bring efficient local language model inference to Python. It provides both a low-level ctypes interface to the C API and a high-level Python API that mimics OpenAI's completion endpoints, making it easy to integrate into existing workflows. The package supports multiple hardware acceleration backends including CUDA, Metal, ROCm, Vulkan, and CPU-only inference, with pre-built wheels available to avoid compilation overhead.\n\nThe library is designed for developers who want to run large language models on their own hardware without cloud dependencies. It includes an OpenAI-compatible web server for local API access, function calling support, vision model capabilities, and multi-model serving. Installation requires a C compiler and can be complex depending on your target hardware backend, but active maintenance and comprehensive documentation help mitigate setup friction.","worth_installing":"Yes, with conditions. Install if you need local language model inference with hardware acceleration and can handle a non-trivial build process. The package is actively maintained, has no known vulnerabilities, and integrates well with popular frameworks. However, expect high install friction due to C compilation requirements\u2014use pre-built wheels where available to reduce setup complexity. Not recommended if you need a quick, zero-configuration solution."},"id":"llama-cpp-python","links":{"html":"https://skillfed.io/packages/llama-cpp-python","md":"https://skillfed.io/packages/llama-cpp-python.md","pypi":"https://pypi.org/project/llama-cpp-python/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-12","license_spdx":null,"license_treatment":"permissive","name":"llama-cpp-python","python_support":"supports_current","summary":"Python bindings for the llama.cpp library"},"popularity":{"monthly_downloads":732904,"position":5200,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.3.34"}