sucrette/nemotron-3-diarization-coreai

Nemotron 3 Diarization for Core AI (with a CAMPPlus speaker embedder)

nvidia/Nemotron-3-Diarization (revision f667ed73, OpenMDW 1.1, by NVIDIA) converted to Apple's Core AI (.aimodel, macOS 27): streaming speaker diarization for up to 8 speakers. Beside it, a CAMPPlus speaker embedder for recognising a known voice across recordings. Converted for Asa, which drives both from Swift (AsaSTT.Diarizer, AsaSTT.SpeakerEmbedder).

coreai-community also publishes Nemotron-3-Diarization-CoreAI, packaged for coreai-kit. On a 95 s meeting this graph's frame probabilities equal that bundle's exactly; this repository is a plain-graph version with the host constants as raw float files, plus the speaker embedder.

Files

file	what it is
diarizer.aimodel	the streaming Sortformer encoder and head, fp16 weights, float32 I/O
embedder_projection.f32le	float32 [512, 1024], the embedder's Linear (8 stacked log-mel frames to 512), run on the host
silence_embeds.f32le	float32 [512], the speaker cache's silence row
mel_filters_128x257.f32le	float32 [128, 257], librosa slaney filterbank (16 kHz, n_fft 512, fmax 8000)
hann_window_400.f32le	float32 [400], torch.hann_window(400, periodic=False)
speaker_embedder.aimodel	CAMPPlus speaker encoder, fp32, Kaldi fbank inside the graph

Inputs and outputs

diarizer.aimodel (main), T = 541 rows (speaker cache 264 + FIFO 264 + chunk 9 + look-ahead 4)

• packed float32 [1, 541, 512]: the embedded rows (cache, FIFO, new chunk, look-ahead)
• valid float32 [1, 541]: 1 for real rows, 0 for padding
• out logits float32 [1, 4328, 8]: per-speaker logits at 10 ms frames (apply sigmoid, then 8x pool to 80 ms)

The log-mel, the 8-frame embedder, the sigmoid and pooling, and the arrival-order speaker cache and FIFO run on the host, as transformers' Nemotron3Diarization host does. Each low-latency step is 720 ms of new audio plus 320 ms look-ahead.

speaker_embedder.aimodel (main)

• wav16 float32 [1, 48000]: 3 s of 16 kHz mono audio (tile shorter clips, average L2-normalised windows of longer ones)
• out xvector float32 [1, 192]

Checked on

Apple silicon Mac, macOS 27, coreai-core 1.0.0b3.

• Diarizer on the GPU against the eager fp32 PyTorch graph: correlation 1.000000, max logit error 0.039. Streaming in 1234-sample blocks gives the whole-file frames.
• 95.5 s five-speaker meeting: 5 speakers found, 29 of 29 scripted turns on the right label; 8.8 s to process (10.8x real time).
• Speaker embedder on the CPU (the GPU lowering fails on an MPS reshape) against Chatterbox's PyTorch CAMPPlus: cosine 1.000000 on three windows. The meeting's 29 turns take 0.6 s; each turn's nearest averaged speaker is the right one 29 of 29.

How it was made

uv run tools/export_diarizer.py OUT_DIR          # diarizer.aimodel and the .f32le constants
uv run tools/export_speaker_embedder.py          # speaker_embedder.aimodel

from Asa's tools/. Pins: coreai-core 1.0.0b3, coreai-torch 0.4.3, torch 2.13; the diarizer loads model.safetensors from nvidia/Nemotron-3-Diarization f667ed73 (sha256 checked) into a plain-PyTorch re-authoring of the per-chunk network ported from john-rocky/coreai-model-zoo conversion/nemotron3_diar @ 26241ae (BSD 3-Clause, Copyright (c) 2026 Daisuke Majima). The speaker embedder takes CAMPPlus weights from ResembleAI/chatterbox-nano's s3gen_meanflow.safetensors (chatterbox @ 5de7a54).

License

• diarizer.aimodel and the .f32le files are derived from NVIDIA's Nemotron 3 Diarization and are distributed under the OpenMDW License Agreement, version 1.1 (LICENSE, a copy of the agreement). Model by NVIDIA; see the original model card for training data, bias, privacy and safety notes.
• speaker_embedder.aimodel: CAMPPlus architecture from 3D-Speaker (Apache 2.0), weights from Resemble AI's Chatterbox (MIT, LICENSE-chatterbox).