open-clip-torch

What it is and what it does

OpenCLIP is an open-source reimplementation of CLIP that learns joint embeddings of images and text through contrastive learning. It provides a collection of pretrained models trained on datasets like LAION-2B and DataComp-1B, ranging from small ConvNext and ViT variants to large models, all loadable through a unified interface.

The package lets you encode images and text into a shared embedding space, enabling zero-shot classification, semantic search, and image-text matching without task-specific fine-tuning. You load a pretrained model, tokenize text with the appropriate tokenizer, and compute embeddings; similarity scores between image and text embeddings reveal semantic alignment. It integrates with torch, torchvision, huggingface-hub, and timm for model architectures and checkpoint management.

Use it for:

• Zero-shot image classification by encoding class names as text and comparing to image embeddings without retraining
• Image-text retrieval systems that find images matching natural language queries or vice versa
• Building semantic search engines over image collections using text queries
• Multimodal feature extraction for downstream tasks like clustering or anomaly detection
• Evaluating model robustness across datasets by computing zero-shot accuracy on ImageNet and other benchmarks

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

OpenCLIP provides open-source implementations of CLIP (Contrastive Language-Image Pre-training) models that encode images and text into a shared embedding space for zero-shot classification, retrieval, and multimodal tasks.

Yes. OpenCLIP is actively maintained, has no known vulnerabilities, installs with low friction, and offers a mature, well-documented interface to production-grade pretrained models. It is the standard open-source CLIP implementation and worth installing if you need multimodal embeddings or zero-shot classification.

Install

open-clip-torch on PyPI

pip install open-clip-torch

uv add open-clip-torch

poetry add open-clip-torch

Installing open-clip-torch

Before you install

Low install friction with a pure Python wheel. The package depends on torch, torchvision, regex, ftfy, tqdm, huggingface-hub, safetensors, and timm. Maintenance is active with recent releases and a well-maintained repository.

License in practice

MIT license is permissive, allowing commercial and private use with minimal restrictions—suitable for most projects that can include a license notice.

Quickstart

pip install open_clip_torch

import torch
import open_clip

model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')
tokenizer = open_clip.get_tokenizer('ViT-B-32')
text = tokenizer(["a dog", "a cat"])

with torch.no_grad():
    text_features = model.encode_text(text)

Requires torch and torchvision installed; GPU recommended for inference speed. Models using timm image encoders require the latest timm version to avoid 'Unknown model' errors.

Verify before relying

• Whether transformers library is required for all models or only those using transformers tokenizers
• Specific memory requirements for different model sizes during inference
• Image preprocessing and loading workflow details beyond tokenization

Package facts

Evidence: open_clip_torch-3.3.0-py3-none-any.whl

Keywords: pytorch, clip, image-text, language-image, multimodal

Tags

More Software Development packages

Further reading