Offline Speaker Diarization

sherpa-onnx supports offline speaker diarization through the SherpaOnnxCreateOfflineSpeakerDiarization() API. It combines a segmentation model, a speaker embedding extractor, and a clustering algorithm.

See also

SherpaOnnxCreateOfflineSpeakerDiarization, SherpaOnnxOfflineSpeakerDiarizationConfig

SherpaOnnxOfflineSpeakerDiarizationConfig config;
memset(&config, 0, sizeof(config));
config.segmentation.pyannote.model =
    "./sherpa-onnx-pyannote-segmentation-3-0/model.onnx";
// Zero uses the default window shift ratio of 0.1. Set a value in (0, 1] to
// change the sliding-window shift; smaller values mean more overlap and more
// compute.
config.segmentation.pyannote.window_shift_ratio = 0.1f;
config.embedding.model =
    "./3dspeaker_speech_eres2net_base_sv_zh-cn_3dspeaker_16k.onnx";
config.clustering.num_clusters = 4;
 
const SherpaOnnxOfflineSpeakerDiarization *sd =
    SherpaOnnxCreateOfflineSpeakerDiarization(&config);
 
const SherpaOnnxWave *wave = SherpaOnnxReadWave("./meeting.wav");
const SherpaOnnxOfflineSpeakerDiarizationResult *result =
    SherpaOnnxOfflineSpeakerDiarizationProcess(sd, wave->samples,
                                               wave->num_samples);
 
const SherpaOnnxOfflineSpeakerDiarizationSegment *segments =
    SherpaOnnxOfflineSpeakerDiarizationResultSortByStartTime(result);
int32_t n = SherpaOnnxOfflineSpeakerDiarizationResultGetNumSegments(result);
for (int32_t i = 0; i < n; ++i) {
  printf("Speaker %d: %.2f - %.2f\n", segments[i].speaker,
         segments[i].start, segments[i].end);
}
 
SherpaOnnxOfflineSpeakerDiarizationDestroySegment(segments);
SherpaOnnxOfflineSpeakerDiarizationDestroyResult(result);
SherpaOnnxFreeWave(wave);
SherpaOnnxDestroyOfflineSpeakerDiarization(sd);

Segmentation model: sherpa-onnx-pyannote-segmentation-3-0/model.onnx

Embedding model: 3dspeaker_speech_eres2net_base_sv_zh-cn_3dspeaker_16k.onnx

Example source: offline-speaker-diarization-c-api.c