mlx-lm
LLMs with MLX and the Hugging Face Hub
Decision gist · record as of 2026-08-14
Yes, if you have Apple silicon and want to run LLMs locally. The package is actively maintained, has no known vulnerabilities, installs cleanly, and offers a mature feature set including generation, fine-tuning, quantization, streaming, and caching. The MIT license imposes no restrictions. The main blocker is hardware: you need a Mac with Apple silicon; it will not work on Intel or non-macOS systems.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires macOS with Apple silicon and MLX framework support; large models may require macOS 15.0 or higher for optimal performance.
- Low friction installation as a pure Python wheel.
- Active maintenance with recent commits and a substantial user base (top 5000 PyPI packages).
License · maintenance · safety
MIT (permissive) — MIT license is permissive; you can use, modify, and distribute this package with minimal legal restrictions.
last release 2026-04-22 (114 days) · last repo commit 2026-08-12 · 6,610 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 1,412,545 downloads/mo, #3,933 on PyPI
Alternatives
Verify before relying
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.3-4bit")
text = generate(model, tokenizer, prompt="Hello", verbose=True)- Whether all quantization formats are equally stable across different model architectures.
- Performance characteristics and memory overhead for distributed inference and fine-tuning with mx.distributed.
- Exact list of supported model architectures beyond the examples provided in the description.
- Hardware requirements and performance characteristics on different Apple silicon generations.
What it is and what it does
MLX LM is a Python package that brings large language model inference, generation, fine-tuning, and quantization to Apple silicon Macs. It wraps the MLX framework and integrates with Hugging Face Hub to let you download and run thousands of pre-built models with a single command or Python call. The package works both as a command-line tool (mlx_lm.generate, mlx_lm.chat) and as a Python module, supporting streaming generation, custom sampling, prompt caching, and rotating key-value caches for long-context work.
The package handles model quantization and can upload converted models back to Hugging Face. It supports low-rank and full model fine-tuning, including on quantized models, and offers distributed inference and fine-tuning via mx.distributed. Runtime dependencies include mlx, numpy, transformers, sentencepiece, protobuf, pyyaml, and jinja2—all standard ML and NLP libraries.
Use it for
- Run open-source LLMs locally on a Mac for chat, text generation, or code completion without cloud APIs.
- Quantize and convert models from Hugging Face to MLX format for efficient inference on Apple silicon.
- Fine-tune existing models on your own data using low-rank or full fine-tuning with quantized weights.
- Build interactive chat applications with persistent context using the Python API or command-line REPL.
- Cache long prompts to speed up repeated queries with the same context across multiple generations.
- Distribute inference and fine-tuning workloads across multiple devices using mx.distributed.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you have Apple silicon and want to run LLMs locally.
The package is actively maintained, has no known vulnerabilities, installs cleanly, and offers a mature feature set including generation, fine-tuning, quantization, streaming, and caching. The MIT license imposes no restrictions. The main blocker is hardware: you need a Mac with Apple silicon; it will not work on Intel or non-macOS systems.
Install
mlx-lm on PyPI
Before you install
Low friction installation as a pure Python wheel. Active maintenance with recent commits and a substantial user base (top 5000 PyPI packages). Seven runtime dependencies are all well-established libraries.
Requires macOS with Apple silicon and MLX framework support; large models may require macOS 15.0 or higher for optimal performance.
License in practice
MIT license is permissive; you can use, modify, and distribute this package with minimal legal restrictions.
Quickstart
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Mistral-7B-Instruct-v0.3-4bit")
text = generate(model, tokenizer, prompt="Hello", verbose=True)Verify before relying
- Whether all quantization formats are equally stable across different model architectures.
- Performance characteristics and memory overhead for distributed inference and fine-tuning with mx.distributed.
- Exact list of supported model architectures beyond the examples provided in the description.
- Hardware requirements and performance characteristics on different Apple silicon generations.
Package facts
| License | MIT permissive |
| Python support | Supports the current Python release >=3.8 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 7 packagesmlxnumpytransformerssentencepieceprotobufpyyamljinja2 |
| Maintenance | Actively maintained 114 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 1,412,545 / month, #3,933 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
Evidence: mlx_lm-0.31.3-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “llm inference on apple silicon”
- mlx-lmMLX LM loads, generates text with, fine-tunes, and quantizes large…
- mlx-vlmMLX-VLM runs vision language models and omni models (with audio and…
- mlx-audioMLX-Audio provides text-to-speech, speech-to-text, and…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
hf-xet provides chunk-based deduplication and efficient file transfer for the Hugging Face Hub, enabling faster uploads and downloads of large files with local disk caching.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
See also mlx-vlm · mlx-whisper · text-generation · lmstudio · mlx · renderers · llmcompressor · mlx-audio · llama-models · lm-eval