VibeVoice MLX
Collection
VibeVoice-{1.5b, 7b} converted to MLX โข 2 items โข Updated
How to use gafiatulin/vibevoice-1.5b-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir vibevoice-1.5b-mlx gafiatulin/vibevoice-1.5b-mlx
How to use gafiatulin/vibevoice-1.5b-mlx with VibeVoice:
import torch, soundfile as sf, librosa, numpy as np
from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor
from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference
# Load voice sample (should be 24kHz mono)
voice, sr = sf.read("path/to/voice_sample.wav")
if voice.ndim > 1: voice = voice.mean(axis=1)
if sr != 24000: voice = librosa.resample(voice, sr, 24000)
processor = VibeVoiceProcessor.from_pretrained("gafiatulin/vibevoice-1.5b-mlx")
model = VibeVoiceForConditionalGenerationInference.from_pretrained(
"gafiatulin/vibevoice-1.5b-mlx", torch_dtype=torch.bfloat16
).to("cuda").eval()
model.set_ddpm_inference_steps(5)
inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"],
voice_samples=[[voice]], return_tensors="pt")
audio = model.generate(**inputs, cfg_scale=1.3,
tokenizer=processor.tokenizer).speech_outputs[0]
sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000)MLX-converted fp16 weights for microsoft/VibeVoice-1.5B.
For inference code, benchmarks, and documentation see vibevoice-mlx.
git clone https://github.com/gafiatulin/vibevoice-mlx && cd vibevoice-mlx
uv sync
# Basic synthesis (weights download automatically)
uv run vibevoice-mlx --text "Hello, world!" --output hello.wav
# Voice cloning
uv run vibevoice-mlx \
--ref-audio speaker.wav --text "Clone this voice" --output cloned.wav
Benchmarked on M4 Max 64GB with voice cloning (~30s audio):
| Config | RTF | Gen | Peak Mem |
|---|---|---|---|
| fp16 | 1.85x | 15.5s | 8.5 GB |
| int8 | 2.65x | 10.3s | 5.7 GB |
| int4 | 2.72x | 9.7s | 4.6 GB |
| int8, no-semantic | 3.42x | 7.8s | 4.4 GB |
| int4, no-semantic | 3.92x | 6.4s | 3.3 GB |
Quantized
Base model
microsoft/VibeVoice-1.5B