{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/7"},{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis/2"}],"enrichment":{"capability":"vllm-cpu provides CPU-optimized inference for large language models with automatic detection of available CPU instruction sets (AVX2, AVX-512, AMX on x86; NEON, BF16, DOTPROD on ARM), packaged as a single unified wheel.","skillfed_tags":["cpu-inference","llm-serving","arm64-support"],"use_cases":["Run inference on development laptops or CI/CD systems without GPU hardware.","Deploy LLM services on ARM-based cloud instances (AWS Graviton, Ampere Altra) with automatic NEON/BF16 detection.","Batch process text through language models on multi-socket CPU servers using NUMA optimization.","Serve an OpenAI-compatible API endpoint on edge devices or on-premises infrastructure.","Prototype and test LLM applications before scaling to GPU clusters."],"what_it_does":"vllm-cpu is a community-maintained CPU inference engine for large language models that unifies multiple CPU instruction set architectures (AVX2, AVX-512, AMX, NEON, BF16, DOTPROD) into a single wheel. It automatically detects and uses the best available instruction set at runtime, eliminating the need for manual ISA-specific builds. The package ships with fallback implementations so the same wheel works across different x86_64 and aarch64 platforms.\n\nIt enables LLM inference on servers, laptops, and edge devices without requiring a GPU, making it suitable for development, testing, and moderate-scale deployments. The package includes an OpenAI-compatible API server via FastAPI and supports batch processing through the vLLM Python API. It depends on transformers, tokenizers, safetensors for model loading, and specialized libraries like lm-format-enforcer and xgrammar for output constraints.","worth_installing":"Yes, with conditions. Install if you need CPU-only LLM inference and have a compatible processor (x86_64 with AVX2 or aarch64 with NEON). The package is actively maintained, has no known vulnerabilities, and uses a permissive license. However, be aware it is community-maintained (not official vLLM), has 63 runtime dependencies creating medium install friction, and performance will be significantly slower than GPU inference\u2014suitable for development, testing, and edge deployment rather than high-throughput production serving."},"id":"vllm-cpu","links":{"html":"https://skillfed.io/packages/vllm-cpu","md":"https://skillfed.io/packages/vllm-cpu.md","pypi":"https://pypi.org/project/vllm-cpu/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-11","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"vllm-cpu","python_support":"supports_current","summary":"A high-throughput and memory-efficient inference and serving engine for LLMs"},"popularity":{"monthly_downloads":255523,"position":8477,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.27.1"}