omnivoice

What it is and what it does

OmniVoice is a multilingual text-to-speech model built on a diffusion language model architecture that converts text to speech across over 600 languages. It operates in three modes: zero-shot voice cloning (copying a speaker's voice from a short reference clip), voice design (synthesizing speech with specified speaker attributes like gender, age, and accent), and standard TTS. The model runs on PyTorch and depends on transformers, librosa, torchaudio, and several audio utilities to handle speech generation, audio I/O, and model inference.

The package is designed for developers and researchers who need multilingual speech synthesis with fine-grained control. It supports non-verbal symbols (e.g., [laughter]), pronunciation correction via pinyin or phonemes, and claims inference speeds as low as 0.025 RTF (40x faster than real-time). Voice clones can be saved and reused across sessions. The model automatically falls back to standard attention on unsupported hardware (e.g., XPU without flash_attn) and integrates with Whisper ASR for automatic transcription of reference audio when manual transcription is not provided.

Use it for:

• Clone a speaker's voice from a 3–10 second reference clip and generate speech in any of 600+ languages.
• Synthesize speech with specific speaker characteristics (gender, age, pitch, accent) without reference audio.
• Build multilingual voice applications with fine-grained control over pronunciation and non-verbal cues.
• Integrate zero-shot TTS into production systems where low inference latency is required.
• Create cross-lingual voice clones where reference and target languages differ, accepting the accent carryover.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

OmniVoice generates speech from text in over 600 languages using a diffusion-based model, with support for voice cloning from reference audio and voice design via speaker attributes.

Yes, if you need multilingual TTS with voice cloning or design. The broad language coverage (600+), active maintenance, permissive license, and low install friction make it a strong choice. Requires upfront PyTorch setup and GPU memory; voice design mode is less stable outside Chinese and English. No known vulnerabilities.

Install

omnivoice on PyPI

pip install omnivoice

uv add omnivoice

poetry add omnivoice

Installing omnivoice

Before you install

Low install friction; pure Python wheel. Requires PyTorch and torchaudio as prerequisites (with GPU-specific variants for NVIDIA, Apple Silicon, or Intel Arc), plus 11 runtime dependencies including transformers, librosa, and torch. Active maintenance with recent releases.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions, making it suitable for production deployments.

Quickstart

pip install omnivoice

from omnivoice import OmniVoice
import soundfile as sf
import torch

model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)
audio = model.generate(
    text="Hello, this is a test.",
    ref_audio="ref.wav",
    ref_text="Transcription of reference."
)
sf.write("out.wav", audio[0], 24000)

Requires PyTorch and torchaudio pre-installed with appropriate GPU support (CUDA, Apple Silicon, or Intel Arc); Python >= 3.10 required.

Verify before relying

• Whether voice design mode is stable for languages beyond Chinese and English.
• Actual inference speed (RTF 0.025 claim) on typical hardware configurations.
• Quality of voice cloning with reference audio shorter or longer than the recommended 3–10 seconds.

Package facts

Evidence: omnivoice-0.2.1-py3-none-any.whl

Keywords: diffusion, multilingual, speech-synthesis, text-to-speech, tts, voice-cloning, zero-shot

Tags

More Artificial Intelligence packages