s3tokenizer
Reverse Engineering of Supervised Semantic Speech Tokenizer (S3Tokenizer) proposed in CosyVoice
Decision gist · record as of 2026-08-14
Yes, if you are training or fine-tuning speech language models (especially CosyVoice-based systems) and need efficient tokenization. The reverse-engineered PyTorch implementation removes a significant bottleneck in the original pipeline, and the package is actively maintained with no known vulnerabilities. The permissive Apache 2.0 license poses no restrictions. Install friction is low. Caveat: the aging maintenance status and relatively small user base (527 stars) mean fewer eyes on edge cases—verify performance on your specific hardware before committing to production.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires PyTorch and torchaudio installed; GPU recommended for practical throughput (CPU batch inference supported but slower).
- Low install friction with a pure Python wheel.
- Runtime dependencies include torch, torchaudio, numpy, and einops—all standard ML stack components.
License · maintenance · safety
Apache2.0 (permissive) — Apache 2.0 permissive license allows commercial use, modification, and redistribution with minimal restrictions, making it suitable for both research and production deployments.
last release 2025-12-22 (235 days) · last repo commit 2025-12-22 · 527 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 405,827 downloads/mo, #6,899 on PyPI
Alternatives
Verify before relying
import s3tokenizer
tokenizer = s3tokenizer.load_model("speech_tokenizer_v1").cuda()
audio = s3tokenizer.load_audio("audio.wav")
mels = s3tokenizer.log_mel_spectrogram(audio)
mels, mels_lens = s3tokenizer.padding([mels])
codes, codes_lens = tokenizer.quantize(mels.cuda(), mels_lens.cuda())- Whether the ~790x speedup claim versus the original CosyVoice pipeline is reproducible on typical hardware configurations.
- Exact memory requirements for distributed inference across multiple GPUs.
- Whether online extraction during training introduces measurable latency or gradient computation overhead.
What it is and what it does
S3Tokenizer is a PyTorch reimplementation of the Supervised Semantic Speech Tokenizer originally introduced in CosyVoice. It converts raw audio into discrete semantic tokens that preserve textual and paralinguistic information, designed for use in speech language models. The original authors released only an ONNX version and did not open-source the PyTorch implementation, so this package reverse-engineers that functionality.
The package provides three main workflows: offline batch inference (with CPU and multi-GPU support via torchrun), online token extraction during SpeechLLM training, and automatic long-audio processing for files longer than 30 seconds. It supports multiple model versions (V1 at 50Hz and 25Hz, V2 25Hz, V3 25Hz) and includes command-line tools for distributed processing. The key advantage over the original pipeline is throughput—batch inference with proper distribution can achieve significant speedups, and online extraction eliminates the need for separate preprocessing passes.
Use it for
- Batch preprocessing of large audio datasets for speech model training with multi-GPU parallelization.
- Online speech code extraction during SpeechLLM forward passes to avoid offline preprocessing bottlenecks.
- Fine-tuning CosyVoice models when batch-size-1 extraction in the original pipeline is too slow.
- Building speech-to-text or speech-to-speech systems that require semantic token representations.
- Processing long-form audio (>30 seconds) with automatic windowing and segment batching.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are training or fine-tuning speech language models (especially CosyVoice-based systems) and need efficient tokenization.
The reverse-engineered PyTorch implementation removes a significant bottleneck in the original pipeline, and the package is actively maintained with no known vulnerabilities. The permissive Apache 2.0 license poses no restrictions. Install friction is low. Caveat: the aging maintenance status and relatively small user base (527 stars) mean fewer eyes on edge cases—verify performance on your specific hardware before committing to production.
Install
s3tokenizer on PyPI
Before you install
Low install friction with a pure Python wheel. Runtime dependencies include torch, torchaudio, numpy, and einops—all standard ML stack components. Maintenance status is aging (235 days since release), but the repository remains active with recent commits and no archived status.
Requires PyTorch and torchaudio installed; GPU recommended for practical throughput (CPU batch inference supported but slower).
License in practice
Apache 2.0 permissive license allows commercial use, modification, and redistribution with minimal restrictions, making it suitable for both research and production deployments.
Quickstart
import s3tokenizer
tokenizer = s3tokenizer.load_model("speech_tokenizer_v1").cuda()
audio = s3tokenizer.load_audio("audio.wav")
mels = s3tokenizer.log_mel_spectrogram(audio)
mels, mels_lens = s3tokenizer.padding([mels])
codes, codes_lens = tokenizer.quantize(mels.cuda(), mels_lens.cuda())Verify before relying
- Whether the ~790x speedup claim versus the original CosyVoice pipeline is reproducible on typical hardware configurations.
- Exact memory requirements for distributed inference across multiple GPUs.
- Whether online extraction during training introduces measurable latency or gradient computation overhead.
Package facts
| License | Apache2.0 permissive |
| Python support | Supports the current Python release >=3.8 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 7 packagespre-commitnumpytorchonnxtqdmtorchaudioeinops |
| Maintenance | Aging 235 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 405,827 / month, #6,899 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Operating System :: OS IndependentProgramming Language :: Python :: 3Topic :: Scientific/Engineering |
Evidence: s3tokenizer-0.3.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “speech tokenization pytorch”
- s3tokenizerConverts audio into discrete semantic speech tokens using a PyTorch…
- voxcpmVoxCPM2 is a tokenizer-free text-to-speech system that generates…
- torch-stoiComputes Short Term Objective Intelligibility (STOI) as a PyTorch…
Give your agent the search over MCP, or paste the wish link into any chat.
More Scientific/Engineering packages
NumPy provides an N-dimensional array object and a comprehensive suite of mathematical, linear algebra, Fourier transform, and random number functions for scientific computing in Python.
pandas provides fast, flexible data structures (Series and DataFrame) for loading, cleaning, transforming, and analyzing labeled or relational data in Python.
scipy provides numerical algorithms for mathematics, science, and engineering—including optimization, integration, linear algebra, Fourier transforms, signal and image processing, and ODE solvers—built on numpy arrays.
scikit-learn provides a comprehensive Python library for supervised and unsupervised machine learning, including classification, regression, clustering, dimensionality reduction, and model evaluation tools built on NumPy and SciPy.
Install it if you need to train, evaluate, or deploy supervised or unsupervised learning models.
dill extends Python's pickle module to serialize and deserialize a much wider range of Python objects, including functions, lambdas, classes, and interpreter sessions, to byte streams for storage or network transmission.
Multiprocess is an enhanced fork of Python's standard multiprocessing library that uses dill for better serialization, allowing you to spawn processes with a threading-like API and share complex objects between them.
Install it if you use multiprocessing and encounter pickle serialization limits with lambdas or complex objects.
See also torchaudio · snac · voxcpm · tokie · chatterbox-tts · speechbrain · openai-whisper · torchcrepe · torchlibrosa · Resemblyzer