ctranslate2
Fast inference engine for Transformer models
Decision gist · record as of 2026-08-14
Yes. CTranslate2 is production-ready (Development Status 5), actively maintained, has no known vulnerabilities, and solves a real problem: efficient Transformer inference. Install it if you need to serve Transformer models with lower latency and memory than standard frameworks. The medium install friction is offset by pre-built wheels for common platforms and Python versions. The main gotcha is that models must be converted to CTranslate2 format first, which requires familiarity with the conversion tools.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python >=3.9; GPU support depends on CUDA 12.4 or compatible hardware; models must be converted to CTranslate2 format first using provided converters.
- Medium install friction: pre-built wheels cover Python 3.10–3.13 across macOS (ARM64 and x86_64), Linux (x86_64 and AArch64), and Windows.
- Active maintenance with last commit 2026-08-05 and a recent release 42 days ago.
License · maintenance · safety
MIT (permissive) — MIT license is permissive; you may use, modify, and distribute this package freely in commercial and open-source projects with minimal restrictions.
last release 2026-07-03 (42 days) · last repo commit 2026-08-05 · 4,619 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 10,724,102 downloads/mo, #1,440 on PyPI
Alternatives
Verify before relying
pip install ctranslate2
import ctranslate2
translator = ctranslate2.Translator(model_path)
result = translator.translate_batch(tokens)- Whether pre-built wheels include all optional backends (Intel MKL, oneDNN, OpenBLAS, Ruy, Apple Accelerate) or if some require separate installation.
- Whether AMD ROCm GPU wheels mentioned in the description are available on PyPI or only on GitHub releases.
- Performance characteristics on specific hardware configurations beyond the benchmarks shown (e.g., consumer GPUs, older CPUs).
What it is and what it does
CTranslate2 is a specialized inference runtime for Transformer models that prioritizes speed and memory efficiency over generality. It implements a custom C++ engine with performance optimizations including weights quantization (INT8, INT16, FP16, BF16, AWQ), layer fusion, batch reordering, and caching, targeting both CPU and GPU deployment. The library supports a wide range of model architectures—from translation models like BART and T5 to large language models like Llama and Mistral—and includes converters for popular training frameworks (Transformers, PyTorch, TensorFlow, Fairseq, Marian).
You use CTranslate2 when you need to serve Transformer models in production with lower latency and memory footprint than general-purpose deep learning frameworks. After converting a trained model to CTranslate2 format, you instantiate a Translator or Generator object and call batch methods to process sequences. The library handles automatic CPU backend selection, parallel execution across multiple GPUs or cores, and dynamic memory allocation. It trades model flexibility and training capability for substantial inference speedups—benchmarks show 2–3× throughput gains on CPU and comparable or better performance on GPU compared to PyTorch and TensorFlow, with quantization reducing model size by up to 4×.
Use it for
- Deploy machine translation models in production with lower latency and memory than PyTorch or TensorFlow.
- Serve large language models on resource-constrained hardware using INT8 or INT16 quantization.
- Run inference on edge devices or embedded systems where model size and memory footprint are critical.
- Batch-process text generation or translation tasks with automatic parallelization across CPU cores or multiple GPUs.
- Integrate Transformer inference into web services or microservices with minimal dependencies.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
CTranslate2 is production-ready (Development Status 5), actively maintained, has no known vulnerabilities, and solves a real problem: efficient Transformer inference. Install it if you need to serve Transformer models with lower latency and memory than standard frameworks. The medium install friction is offset by pre-built wheels for common platforms and Python versions. The main gotcha is that models must be converted to CTranslate2 format first, which requires familiarity with the conversion tools.
Install
ctranslate2 on PyPI
Before you install
Medium install friction: pre-built wheels cover Python 3.10–3.13 across macOS (ARM64 and x86_64), Linux (x86_64 and AArch64), and Windows. Active maintenance with last commit 2026-08-05 and a recent release 42 days ago.
Requires Python >=3.9; GPU support depends on CUDA 12.4 or compatible hardware; models must be converted to CTranslate2 format first using provided converters.
License in practice
MIT license is permissive; you may use, modify, and distribute this package freely in commercial and open-source projects with minimal restrictions.
Quickstart
pip install ctranslate2
import ctranslate2
translator = ctranslate2.Translator(model_path)
result = translator.translate_batch(tokens)Verify before relying
- Whether pre-built wheels include all optional backends (Intel MKL, oneDNN, OpenBLAS, Ruy, Apple Accelerate) or if some require separate installation.
- Whether AMD ROCm GPU wheels mentioned in the description are available on PyPI or only on GitHub releases.
- Performance characteristics on specific hardware configurations beyond the benchmarks shown (e.g., consumer GPUs, older CPUs).
Package facts
| License | MIT permissive |
| Python support | Supports the current Python release >=3.9 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | 3 packagessetuptoolsnumpypyyaml |
| Maintenance | Actively maintained 42 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 10,724,102 / month, #1,440 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableEnvironment :: GPU :: NVIDIA CUDA :: 12 :: 12.4Intended Audience :: DevelopersIntended Audience :: Science/ResearchProgramming Language :: Python :: 3Programming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Programming Language :: Python :: 3.9Topic :: Scientific/Engineering :: Artificial Intelligence |
Evidence: ctranslate2-4.8.1-cp310-cp310-macosx_11_0_arm64.whl; ctranslate2-4.8.1-cp310-cp310-macosx_11_0_x86_64.whl; ctranslate2-4.8.1-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; ctranslate2-4.8.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; ctranslate2-4.8.1-cp310-cp310-win_amd64.whl; ctranslate2-4.8.1-cp311-cp311-macosx_11_0_arm64.whl; ctranslate2-4.8.1-cp311-cp311-macosx_11_0_x86_64.whl; ctranslate2-4.8.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; ctranslate2-4.8.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; ctranslate2-4.8.1-cp311-cp311-win_amd64.whl; ctranslate2-4.8.1-cp312-cp312-macosx_11_0_arm64.whl; ctranslate2-4.8.1-cp312-cp312-macosx_11_0_x86_64.whl; ctranslate2-4.8.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; ctranslate2-4.8.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; ctranslate2-4.8.1-cp312-cp312-win_amd64.whl; ctranslate2-4.8.1-cp313-cp313-macosx_11_0_arm64.whl; ctranslate2-4.8.1-cp313-cp313-macosx_11_0_x86_64.whl; ctranslate2-4.8.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; ctranslate2-4.8.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; ctranslate2-4.8.1-cp313-cp313-win_amd64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “machine translation runtime”
- ctranslate2CTranslate2 is a C++ and Python library that runs Transformer models…
- sacremosesSacremoses provides tokenization, detokenization, truecasing, and…
- argostranslateArgos Translate is an open-source offline neural machine translation…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
hf-xet provides chunk-based deduplication and efficient file transfer for the Hugging Face Hub, enabling faster uploads and downloads of large files with local disk caching.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
See also faster-whisper · argostranslate · transformer-engine · transformer-engine-cu13 · transformer-engine-cu12 · optimum-quanto · auto-gptq · megatron-core · spacy-transformers · tensorrt-cu13-libs