laion-clap

What it is and what it does

CLAP is a PyPI package that wraps Contrastive Language-Audio Pretraining models, allowing you to generate fixed-size embeddings for both audio files and text descriptions. The embeddings are trained to align semantically—audio and text describing the same content will have similar embeddings in the learned space. You load a pretrained checkpoint (trained on AudioSet, music, speech, or combinations thereof), then call methods to embed audio from files or raw data, or to embed text strings. The package handles the model architecture, quantization, and checkpoint management internally.

The typical workflow is to extract embeddings for a corpus of audio and text, then use those embeddings for tasks like zero-shot audio classification (by comparing audio embeddings to class-label text embeddings), cross-modal retrieval, or as features for downstream supervised models. The package depends on librosa for audio loading, transformers for text encoding, and PyTorch for the model itself, plus 15 other utilities for data handling and training infrastructure.

Use it for:

• Zero-shot audio classification by embedding audio and comparing to embeddings of class labels as text.
• Cross-modal retrieval: find audio clips matching a text query or vice versa by embedding both and ranking by similarity.
• Audio-text dataset annotation: embed a large audio corpus and use text queries to discover or label relevant clips.
• Feature extraction for downstream models: use CLAP embeddings as fixed input features for audio classification or tagging tasks.
• Music or speech recognition: use pretrained checkpoints tuned for music or speech to embed domain-specific audio.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Extracts learned audio and text embeddings using contrastive language-audio pretraining, enabling cross-modal retrieval and downstream audio-text tasks.

Yes, with conditions. Install if you need audio-text embeddings and are comfortable with the 18 dependencies and aging maintenance status. The package is permissively licensed, has no known vulnerabilities, and offers multiple pretrained models for different audio domains. However, verify that the HuggingFace checkpoints you need are still available, and be aware that the last release was 467 days ago—if you encounter bugs or need updates, community support may be limited.

Install

laion-clap on PyPI

pip install laion-clap

uv add laion-clap

poetry add laion-clap

Installing laion-clap

Before you install

Low install friction with a pure Python wheel. Maintenance is aging—last commit was 2025-05-15 and the package has not been updated in 467 days—but the repository remains active and has 2250 stars. The 18 runtime dependencies are substantial and include heavy libraries like transformers, librosa, and torch-related packages.

License in practice

Released under CC0 1.0 Universal, which places the work in the public domain with no copyright restrictions. You may use, modify, and distribute it freely for any purpose, including commercial, with no attribution requirement.

Quickstart

pip install laion-clap

import laion_clap
import librosa

model = laion_clap.CLAP_Module(enable_fusion=False)
model.load_ckpt()

audio_data, _ = librosa.load('audio.wav', sr=48000)
audio_data = audio_data.reshape(1, -1)
audio_embed = model.get_audio_embedding_from_data(x=audio_data, use_tensor=False)

text_embed = model.get_text_embedding(["dog barking"])

Audio must be resampled to 48000 Hz. Requires PyTorch and librosa; model checkpoint is downloaded on first load_ckpt() call.

Verify before relying

• Whether pretrained checkpoints (630k-audioset-best.pt, music_audioset_epoch_15_esc_90.14.pt, etc.) remain reliably available from HuggingFace.
• Whether the aging maintenance status (467 days since last release) signals planned deprecation or stable maturity.
• Performance and memory footprint of the larger audio encoder models (HTSAT-base) on typical hardware.

Package facts

Evidence: laion_clap-1.1.7-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading