ipex-llm

What it is and what it does

IPEX-LLM is an acceleration library for running large language models on Intel hardware—GPUs (Arc, Flex, Max), NPUs (Core Ultra), and CPUs. It provides low-bit quantization (FP8, FP6, FP4, INT4) and integrates with popular frameworks like PyTorch, llama.cpp, Ollama, vLLM, LangChain, and HuggingFace transformers to reduce memory footprint and improve inference speed on Intel-based systems.

The library supports over 70 models including Llama, Phi, Mistral, Qwen, and DeepSeek variants. However, the project is now archived and abandoned—Intel will not provide maintenance, bug fixes, new releases, or support. The description notes known security issues exist. Users should treat this as a snapshot tool rather than an actively maintained dependency.

Use it for:

• Run quantized LLMs locally on Intel Arc GPUs without external cloud services
• Integrate low-bit quantized models into Ollama or llama.cpp workflows on Intel hardware
• Serve LLMs via vLLM with Intel GPU acceleration for batch inference
• Fine-tune models using Axolotl with Intel GPU backend
• Deploy multimodal models (Qwen-VL, Phi-3-Vision) on Intel NPU or GPU

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Accelerates large language model inference on Intel hardware (GPU, NPU, CPU) with low-bit quantization and integration into popular LLM frameworks.

No. The project is archived and abandoned; Intel provides no maintenance, bug fixes, or support. Known security issues are documented and will not be patched. While the library may still function, adopting an unmaintained LLM acceleration tool introduces technical debt and security risk. Consider maintained alternatives or direct integration with actively supported frameworks.

Install

ipex-llm on PyPI

pip install ipex-llm

uv add ipex-llm

poetry add ipex-llm

Installing ipex-llm

Before you install

Medium install friction with platform-specific wheels (manylinux2010 x86_64, Windows). Project is archived and abandoned as of the latest release; Intel will not provide maintenance, bug fixes, or support going forward.

License in practice

Apache License 2.0 (permissive) allows commercial and private use with attribution; no restrictive copyleft obligations.

Quickstart

pip install ipex-llm

from ipex_llm.transformers import AutoModel
model = AutoModel.from_pretrained('model-name', load_in_4bit=True)

Requires Intel GPU (Arc, Flex, Max), NPU (Core Ultra), or compatible CPU; no runtime dependencies listed but underlying PyTorch and model libraries are implicit.

Verify before relying

• Exact scope of 70+ verified models and which specific versions are tested
• Whether archived status means security patches will be backported or if vulnerabilities remain unfixed
• Performance benchmarks for quantization modes (FP8/FP6/FP4/INT4) relative to unquantized baseline
• Compatibility matrix for specific Intel GPU models and driver versions required

Package facts

Evidence: ipex_llm-2.2.0-py3-none-manylinux2010_x86_64.whl; ipex_llm-2.2.0-py3-none-win_amd64.whl

Tags

More Artificial Intelligence packages

Further reading