# VibeVoice 声学分词器

## 概述

[VibeVoice](https://huggingface.co/papers/2508.19205) 是一个新颖的框架，通过在大型语言模型（LLM）结构中采用下一 token 扩散方法，来合成高保真、长篇幅的多说话人语音。它旨在捕捉真实对话的“氛围”，尤其适合生成播客和多参与者有声读物等音频内容。

VibeVoice 的一个关键特性是使用了两个连续的音频分词器，一个用于提取声学特征，另一个用于提取语义特征。

模型检查点可在 [microsoft/VibeVoice-AcousticTokenizer](https://huggingface.co/microsoft/VibeVoice-AcousticTokenizer) 获取

该模型由 [Eric Bezzam](https://huggingface.co/bezzam) 贡献。

## 架构

该架构采用镜像对称的编码器-解码器结构。编码器采用分层设计，包含 7 个类似 ConvNeXt 的块，这些块使用一维深度可分离因果卷积以实现高效的流式处理。六个下采样层实现了从 24kHz 输入到 3200 倍累计下采样率，从而产生每秒 7.5 个 token/帧。每个编码器/解码器组件约有 3.4 亿个参数，总计约 6.8 亿个参数。训练目标遵循 [DAC](./dac)，包括其判别器和损失函数设计。

Acoustic Tokenizer 采用了变分自编码器（VAE）的原理。编码器将输入音频映射到潜在分布的参数，即均值。结合固定的标准差，随后使用重参数化技巧采样得到潜在向量。有关更多细节，请参阅[技术报告](https://huggingface.co/papers/2508.19205)。

## 用法

以下是用于编码和解码音频的示例用法：

```python
import torch
from scipy.io import wavfile

from transformers import AutoFeatureExtractor, VibeVoiceAcousticTokenizerModel
from transformers.audio_utils import load_audio

model_id = "microsoft/VibeVoice-AcousticTokenizer"

# load model
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
model = VibeVoiceAcousticTokenizerModel.from_pretrained(model_id, device_map="auto")
print("Model loaded on device:", model.device)
print("Model dtype:", model.dtype)

# load audio
audio = load_audio(
    "https://huggingface.co/datasets/bezzam/vibevoice_samples/resolve/main/voices/en-Alice_woman.wav",
    sampling_rate=feature_extractor.sampling_rate,
)

# preprocess audio
inputs = feature_extractor(
    audio,
    sampling_rate=feature_extractor.sampling_rate,
    pad_to_multiple_of=3200,
).to(model.device, model.dtype)
print("Input audio shape:", inputs.input_values.shape)
# Input audio shape: torch.Size([1, 1, 224000])

with torch.no_grad():
    # set VAE sampling to False for deterministic output
    encoded_outputs = model.encode(inputs.input_values, sample=False)
    print("Latent shape:", encoded_outputs.latents.shape)
    # Latent shape: torch.Size([1, 70, 64])

    decoded_outputs = model.decode(**encoded_outputs)
    print("Reconstructed audio shape:", decoded_outputs.audio.shape)
    # Reconstructed audio shape: torch.Size([1, 1, 224000])

# Save audio
output_fp = "vibevoice_acoustic_tokenizer_reconstructed.wav"
wavfile.write(output_fp, feature_extractor.sampling_rate, decoded_outputs.audio.squeeze().float().cpu().numpy())
print(f"Reconstructed audio saved to : {output_fp}")
```

## 流式

对于流式 ASR 或 TTS，由于需要跟踪缓存状态，在对音频进行编码或解码时可以使用 `use_cache` 参数：

```python
import torch
from scipy.io import wavfile

from transformers import AutoFeatureExtractor, VibeVoiceAcousticTokenizerModel
from transformers.audio_utils import load_audio

model_id = "microsoft/VibeVoice-AcousticTokenizer"

# load model
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
model = VibeVoiceAcousticTokenizerModel.from_pretrained(model_id, device_map="auto")
print("Model loaded on device:", model.device)
print("Model dtype:", model.dtype)

# load audio
audio = load_audio(
    "https://huggingface.co/datasets/bezzam/vibevoice_samples/resolve/main/voices/en-Alice_woman.wav",
    sampling_rate=feature_extractor.sampling_rate,
)

# preprocess audio
inputs = feature_extractor(
    audio,
    sampling_rate=feature_extractor.sampling_rate,
    pad_to_multiple_of=3200,
).to(model.device, model.dtype)
print("Input audio shape:", inputs.input_values.shape)
# Input audio shape: torch.Size([1, 1, 224000])

# cache will be initialized after a first pass
encoder_cache = None
decoder_cache = None
with torch.no_grad():
    # set VAE sampling to False for deterministic output
    encoded_outputs = model.encode(inputs.input_values, sample=False, padding_cache=encoder_cache, use_cache=True)
    print("Latent shape:", encoded_outputs.latents.shape)
    # Latent shape: torch.Size([1, 70, 64])

    decoded_outputs = model.decode(encoded_outputs.latents, padding_cache=decoder_cache, use_cache=True)
    print("Reconstructed audio shape:", decoded_outputs.audio.shape)
    # Reconstructed audio shape: torch.Size([1, 1, 224000])

    # `padding_cache` can be extracted from the outputs for subsequent passes
    encoder_cache = encoded_outputs.padding_cache
    print("Number of cached encoder layers:", len(encoder_cache.per_layer_in_channels))
    # Number of cached encoder layers: 34
    decoder_cache = decoded_outputs.padding_cache
    print("Number of cached decoder layers:", len(decoder_cache.per_layer_in_channels))
    # Number of cached decoder layers: 34

# Save audio
output_fp = "vibevoice_acoustic_tokenizer_reconstructed.wav"
wavfile.write(output_fp, feature_extractor.sampling_rate, decoded_outputs.audio.squeeze().float().cpu().numpy())
print(f"Reconstructed audio saved to : {output_fp}")
```

## VibeVoiceAcousticTokenizerConfig[[transformers.VibeVoiceAcousticTokenizerConfig]]

#### transformers.VibeVoiceAcousticTokenizerConfig[[transformers.VibeVoiceAcousticTokenizerConfig]]

```python
transformers.VibeVoiceAcousticTokenizerConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: str | torch.dtype | None = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: Literal['regression', 'single_label_classification', 'multi_label_classification'] | None = None, channels: int = 1, hidden_size: int = 64, kernel_size: int = 7, rms_norm_eps: float = 1e-05, layer_scale_init_value: float = 1e-06, initializer_range: float = 0.01, num_filters: int = 32, downsampling_ratios: list[int] | tuple[int, ...] = (2, 2, 4, 5, 5, 8), depths: list[int] | tuple[int, ...] = (3, 3, 3, 3, 3, 3, 8), hidden_act: str = 'gelu', ffn_expansion: int = 4, vae_std: float = 0.625)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/configuration_vibevoice_acoustic_tokenizer.py#L26)

**参数：**

channels (`int`，*可选*，默认为 1)：输入通道数。

hidden_size (`int`，*可选*，默认为 64)：潜在表示的维度。

kernel_size (`int`，*可选*，默认为 7)：卷积层的核大小。

rms_norm_eps (`float`，*可选*，默认值为 `1e-05`)：rms 归一化层中使用的 epsilon。

layer_scale_init_value (`float`, *可选*, 默认为 `1e-06`) : 在自注意力层中使用的缩放比例。基础模型为 0.1，大型模型为 1e-6。设置为 0 可禁用层缩放。

initializer_range (`float`, *可选*，默认为 `0.01`) : 用于初始化所有权重矩阵的 truncated_normal_initializer 的标准差。

num_filters (`int`，*可选*，默认为 32)：初始卷积层中的滤波器数量，每次下采样后翻倍。

downsampling_ratios (`List[int]`，*可选*，默认为 `[2, 2, 4, 5, 5, 8]`)：每一层的下采样比率。

depths (`List[int]`，*可选*，默认为 `[3, 3, 3, 3, 3, 3, 8]`)：每个阶段的 ConvNeXt 块数量。

hidden_act (`str`，*可选*，默认为 `gelu`)：解码器中的非线性激活函数（函数或字符串）。例如，`"gelu"`、`"relu"`、`"silu"` 等。

ffn_expansion (`int`，*可选*，默认为 4)：前馈网络的扩展因子。

vae_std (`float`, *可选*，默认为 0.625) : 当向编码器潜变量添加噪声（`sample=True`）时，从该高斯分布中抽取每个样本噪声尺度的标准差。

这是用于存储 VibeVoiceAcousticTokenizerModel 配置的类。它根据指定的参数实例化 Vibevoice Acoustic Tokenizer 模型，并定义模型架构。使用默认值实例化配置将得到与 [microsoft/VibeVoice-1.5B](https://huggingface.co/microsoft/VibeVoice-1.5B) 类似的配置。

配置对象继承自 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig)，可用于控制模型输出。有关更多信息，请参阅 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig) 的文档。

示例：

```python
>>> from transformers import VibeVoiceAcousticTokenizerModel, VibeVoiceAcousticTokenizerConfig

>>> # Initializing a VibeVoice Acoustic Tokenizer configuration
>>> configuration = VibeVoiceAcousticTokenizerConfig()

>>> # Initializing a model (with random weights)
>>> model = VibeVoiceAcousticTokenizerModel(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config
```

## VibeVoiceAcousticTokenizerEncoderConfig[[transformers.VibeVoiceAcousticTokenizerEncoderConfig]]

#### transformers.VibeVoiceAcousticTokenizerEncoderConfig[[transformers.VibeVoiceAcousticTokenizerEncoderConfig]]

```python
transformers.VibeVoiceAcousticTokenizerEncoderConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: str | torch.dtype | None = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: Literal['regression', 'single_label_classification', 'multi_label_classification'] | None = None, channels: int = 1, hidden_size: int = 64, kernel_size: int = 7, rms_norm_eps: float = 1e-05, layer_scale_init_value: float = 1e-06, initializer_range: float = 0.01, num_filters: int = 32, downsampling_ratios: list[int] | tuple[int, ...] = (2, 2, 4, 5, 5, 8), depths: list[int] | tuple[int, ...] = (3, 3, 3, 3, 3, 3, 8), hidden_act: str = 'gelu', ffn_expansion: int = 4, vae_std: float = 0.625)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/configuration_vibevoice_acoustic_tokenizer.py#L93)

**参数：**

channels (`int`，*可选*，默认为 1)：输入通道数。

hidden_size (`int`，*可选*，默认为 64)：潜在表示的维度。

kernel_size (`int`，*可选*，默认为 7)：卷积层的核大小。

rms_norm_eps (`float`，*可选*，默认值为 `1e-05`)：rms 归一化层中使用的 epsilon。

layer_scale_init_value (`float`, *可选*, 默认为 `1e-06`) : 在自注意力层中使用的缩放比例。基础模型为 0.1，大型模型为 1e-6。设置为 0 可禁用层缩放。

initializer_range (`float`, *可选*，默认为 `0.01`) : 用于初始化所有权重矩阵的 truncated_normal_initializer 的标准差。

num_filters (`int`，*可选*，默认为 32)：初始卷积层中的滤波器数量，每次下采样后翻倍。

downsampling_ratios (`List[int]`，*可选*，默认为 `[2, 2, 4, 5, 5, 8]`)：每一层的下采样比率。

depths (`List[int]`，*可选*，默认为 `[3, 3, 3, 3, 3, 3, 8]`)：每个阶段的 ConvNeXt 块数量。

hidden_act (`str`，*可选*，默认为 `gelu`)：解码器中的非线性激活函数（函数或字符串）。例如，`"gelu"`、`"relu"`、`"silu"` 等。

ffn_expansion (`int`，*可选*，默认为 4)：前馈网络的扩展因子。

vae_std (`float`, *可选*，默认为 0.625) : 当向编码器潜变量添加噪声（`sample=True`）时，从该高斯分布中抽取每个样本噪声尺度的标准差。

这是用于存储 VibeVoiceAcousticTokenizerModel 配置的类。它根据指定的参数实例化 Vibevoice Acoustic Tokenizer 模型，并定义模型架构。使用默认值实例化配置将得到与 [microsoft/VibeVoice-1.5B](https://huggingface.co/microsoft/VibeVoice-1.5B) 类似的配置。

配置对象继承自 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig)，可用于控制模型输出。有关更多信息，请参阅 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig) 的文档。

## VibeVoiceAcousticTokenizerDecoderConfig[[transformers.VibeVoiceAcousticTokenizerDecoderConfig]]

#### transformers.VibeVoiceAcousticTokenizerDecoderConfig[[transformers.VibeVoiceAcousticTokenizerDecoderConfig]]

```python
transformers.VibeVoiceAcousticTokenizerDecoderConfig(transformers_version: str | None = None, architectures: list[str] | None = None, output_hidden_states: bool | None = False, return_dict: bool | None = True, dtype: str | torch.dtype | None = None, chunk_size_feed_forward: int = 0, is_encoder_decoder: bool = False, id2label: dict[int, str] | dict[str, str] | None = None, label2id: dict[str, int] | dict[str, str] | None = None, problem_type: Literal['regression', 'single_label_classification', 'multi_label_classification'] | None = None, channels: int = 1, hidden_size: int = 64, kernel_size: int = 7, rms_norm_eps: float = 1e-05, layer_scale_init_value: float = 1e-06, initializer_range: float = 0.01, num_filters: int = 32, downsampling_ratios: list[int] | tuple[int, ...] = (2, 2, 4, 5, 5, 8), depths: list[int] | tuple[int, ...] = (3, 3, 3, 3, 3, 3, 8), hidden_act: str = 'gelu', ffn_expansion: int = 4, vae_std: float = 0.625)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/configuration_vibevoice_acoustic_tokenizer.py#L124)

**参数：**

channels (`int`，*可选*，默认为 1)：输入通道数。

hidden_size (`int`，*可选*，默认为 64)：潜在表示的维度。

kernel_size (`int`，*可选*，默认为 7)：卷积层的核大小。

rms_norm_eps (`float`，*可选*，默认值为 `1e-05`)：rms 归一化层中使用的 epsilon。

layer_scale_init_value (`float`, *可选*, 默认为 `1e-06`) : 在自注意力层中使用的缩放比例。基础模型为 0.1，大型模型为 1e-6。设置为 0 可禁用层缩放。

initializer_range (`float`, *可选*，默认为 `0.01`) : 用于初始化所有权重矩阵的 truncated_normal_initializer 的标准差。

num_filters (`int`，*可选*，默认为 32)：初始卷积层中的滤波器数量，每次下采样后翻倍。

downsampling_ratios (`List[int]`，*可选*，默认为 `[2, 2, 4, 5, 5, 8]`)：每一层的下采样比率。

depths (`List[int]`，*可选*，默认为 `[3, 3, 3, 3, 3, 3, 8]`)：每个阶段的 ConvNeXt 块数量。

hidden_act (`str`，*可选*，默认为 `gelu`)：解码器中的非线性激活函数（函数或字符串）。例如，`"gelu"`、`"relu"`、`"silu"` 等。

ffn_expansion (`int`，*可选*，默认为 4)：前馈网络的扩展因子。

vae_std (`float`, *可选*，默认为 0.625) : 当向编码器潜变量添加噪声（`sample=True`）时，从该高斯分布中抽取每个样本噪声尺度的标准差。

这是用于存储 VibeVoiceAcousticTokenizerModel 配置的类。它根据指定的参数实例化 Vibevoice Acoustic Tokenizer 模型，并定义模型架构。使用默认值实例化配置将得到与 [microsoft/VibeVoice-1.5B](https://huggingface.co/microsoft/VibeVoice-1.5B) 类似的配置。

配置对象继承自 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig)，可用于控制模型输出。有关更多信息，请参阅 [PreTrainedConfig](/docs/transformers/v5.23.0/zh/main_classes/configuration#transformers.PreTrainedConfig) 的文档。

## VibeVoiceAcousticTokenizerFeatureExtractor[[transformers.VibeVoiceAcousticTokenizerFeatureExtractor]]

#### transformers.VibeVoiceAcousticTokenizerFeatureExtractor[[transformers.VibeVoiceAcousticTokenizerFeatureExtractor]]

```python
transformers.VibeVoiceAcousticTokenizerFeatureExtractor(feature_size = 1, sampling_rate = 24000, padding_value = 0.0, normalize_audio = True, target_dB_FS = -25, eps = 1e-06, pad_to_multiple_of = 3200, **kwargs)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/feature_extraction_vibevoice_acoustic_tokenizer.py#L29)

**参数：**

feature_size (`int`，*可选*，默认为 1)：通道数。

sampling_rate (`int`, *可选*，默认为 24000)：音频波形应被数字化采样的采样率，以赫兹（Hz）为单位。

padding_value (`float`, *可选*, 默认值为 0.0) : 用于填充的值。

normalize_audio (`bool`, *可选*, 默认为 `True`) : 是否将音频归一化到目标 dB FS。

target_dB_FS (`float`, *可选*, 默认值为 -25) : 用于归一化的目标 dB FS。

eps (`float`, *可选*，默认为 1e-06)：一个用于在归一化时避免除以零的小数值。

pad_to_multiple_of (`int`，*可选*，默认值为 3200)：将音频填充为该值的整数倍，该值应对应声学分词器的跳跃长度，即其编码器的下采样因子。

构建 VibeVoiceAcousticTokenizer 特征提取器。

#### __调用__[[transformers.VibeVoiceAcousticTokenizerFeatureExtractor.__call__]]

```python
__call__(audio: typing.Union[numpy.ndarray, ForwardRef('torch.Tensor'), collections.abc.Sequence[numpy.ndarray], collections.abc.Sequence['torch.Tensor']], sampling_rate: int | None = None, padding: bool | str | transformers.utils.generic.PaddingStrategy | None = True, pad_to_multiple_of: int | None = None, max_length: int | None = None, return_attention_mask: bool | None = True, return_tensors: str | None = 'pt', **kwargs)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/feature_extraction_vibevoice_acoustic_tokenizer.py#L72)

**参数：**

audio (`np.ndarray`、`torch.Tensor`、`list[np.ndarray]`、`list[torch.Tensor]` : 要处理的序列或序列批次。每个序列可以是 numpy 数组、torch 张量、numpy 数组列表或 torch 张量列表。

sampling_rate (`int`, *可选*) : `audio` 输入的采样率。强烈建议在 forward 调用时传入 `sampling_rate`，以避免静默错误。

padding (`bool`、`str` 或 [PaddingStrategy](/docs/transformers/v5.23.0/zh/internal/file_utils#transformers.utils.PaddingStrategy)，*可选*，默认为 `True`)：选择一种策略来填充返回的序列（根据模型的填充方向和填充索引），可选策略如下： - `True` 或 `'longest'`：填充到批次中最长的序列（如果只提供了单个序列，则不进行填充）。 - `'max_length'`：填充到由参数 `max_length` 指定的最大长度，或者如果未提供该参数，则填充到模型可接受的最大输入长度。 - `False` 或 `'do_not_pad'`（默认）：不进行填充（即可以输出包含不同长度序列的批次）。

pad_to_multiple_of (`int`, *可选*) : 如果设置，则将序列填充为所提供值的倍数。如果未提供，则默认为 `self.pad_to_multiple_of`。

## VibeVoiceAcousticTokenizerModel[[transformers.VibeVoiceAcousticTokenizerModel]]

#### transformers.VibeVoiceAcousticTokenizerModel[[transformers.VibeVoiceAcousticTokenizerModel]]

```python
transformers.VibeVoiceAcousticTokenizerModel(config: VibeVoiceAcousticTokenizerConfig)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L503)

**参数：**

config ([VibeVoiceAcousticTokenizerConfig](/docs/transformers/v5.23.0/zh/model_doc/vibevoice_acoustic_tokenizer#transformers.VibeVoiceAcousticTokenizerConfig))：模型配置类，包含模型的所有参数。通过配置文件进行初始化时，不会加载与模型关联的权重，仅加载配置。请查看 [from_pretrained()](/docs/transformers/v5.23.0/zh/main_classes/model#transformers.PreTrainedModel.from_pretrained) 方法以加载模型权重。

VibeVoice 声学分词器，包含用于连续声学 token 的编码器和解码器。

该模型继承自 [PreTrainedModel](/docs/transformers/v5.23.0/zh/main_classes/model#transformers.PreTrainedModel)。请查阅父类文档，了解该库为所有模型实现的通用方法（例如下载或保存、调整输入嵌入大小、剪枝头等）。

该模型也是 PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) 的子类。请将其作为常规的 PyTorch Module 使用，有关通用用法和行为的所有事项，请参阅 PyTorch 文档。

#### encode[[transformers.VibeVoiceAcousticTokenizerModel.encode]]

```python
encode(input_values, padding_cache = None, use_cache = None, sample = True)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L510)

**参数：**

input_values (`torch.Tensor`，形状为 `(batch_size, channels, sequence_length)`)：要编码为潜在表示的输入音频波形。

padding_cache (`VibeVoiceAcousticTokenizerConv1dPaddingCache`, *可选*) : 用于流式模式的缓存对象，用于在层间维护卷积状态。

use_cache (`bool`, *可选*) : 是否对卷积状态使用缓存。

sample (`bool`, *可选*) : 是否向潜变量添加高斯噪声（σ-VAE：噪声尺度按批次中的每个样本从 N(0, `config.vae_std`) 中抽取）。如果为 False，则直接返回编码器的输出。

#### decode[[transformers.VibeVoiceAcousticTokenizerModel.decode]]

```python
decode(latents, padding_cache = None, use_cache = False)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L537)

**参数：**

latents (`torch.Tensor`，形状为 `(batch_size, channels, sequence_length)`)：要解码回音频的输入潜在表示。

padding_cache (`VibeVoiceAcousticTokenizerConv1dPaddingCache`, *可选*) : 用于流式模式的缓存对象，用于在层间维护卷积状态。

use_cache (`bool`, *可选*) : 是否对卷积状态使用缓存。

#### forward[[transformers.VibeVoiceAcousticTokenizerModel.forward]]

```python
forward(input_values, padding_cache = None, use_cache = False, sample = True, **kwargs)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L551)

**参数：**

input_values (`torch.Tensor`，形状为 `(batch_size, channels, sequence_length)`)：要编码为潜在表示的输入音频波形。

padding_cache (`VibeVoiceAcousticTokenizerConv1dPaddingCache`, *可选*) : 用于流式模式的缓存对象，用于在层间保持卷积状态。注意仅由解码器使用。

use_cache (`bool`, *可选*) : 是否对卷积状态使用缓存。

sample (`bool`, *可选*) : 是否向潜变量添加高斯噪声（σ-VAE：噪声尺度按批次中的每个样本从 N(0, `config.vae_std`) 中抽取）。如果为 False，则直接返回编码器的输出。

[VibeVoiceAcousticTokenizerModel](/docs/transformers/v5.23.0/zh/model_doc/vibevoice_acoustic_tokenizer#transformers.VibeVoiceAcousticTokenizerModel) 的 forward 方法重写了 `__call__` 特殊方法。

尽管前向传播的实现需要在此函数中定义，但之后应调用 `Module` 实例，而不是直接调用此函数，因为前者会负责执行预处理和后处理步骤，而后者会静默地忽略这些步骤。

## VibeVoiceAcousticTokenizerEncoderModel[[transformers.VibeVoiceAcousticTokenizerEncoderModel]]

#### transformers.VibeVoiceAcousticTokenizerEncoderModel[[transformers.VibeVoiceAcousticTokenizerEncoderModel]]

```python
transformers.VibeVoiceAcousticTokenizerEncoderModel(config: VibeVoiceAcousticTokenizerEncoderConfig)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L373)

#### forward[[transformers.VibeVoiceAcousticTokenizerEncoderModel.forward]]

```python
forward(hidden_states, padding_cache = None, use_cache = False, **kwargs)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L394)

## VibeVoiceAcousticTokenizerDecoderModel[[transformers.VibeVoiceAcousticTokenizerDecoderModel]]

#### transformers.VibeVoiceAcousticTokenizerDecoderModel[[transformers.VibeVoiceAcousticTokenizerDecoderModel]]

```python
transformers.VibeVoiceAcousticTokenizerDecoderModel(config: VibeVoiceAcousticTokenizerDecoderConfig)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L466)

#### forward[[transformers.VibeVoiceAcousticTokenizerDecoderModel.forward]]

```python
forward(hidden_states, padding_cache = None, use_cache = False, **kwargs)
```

[来源](https://github.com/huggingface/transformers/blob/v5.23.0/src/transformers/models/vibevoice_acoustic_tokenizer/modeling_vibevoice_acoustic_tokenizer.py#L487)

