sherpa-onnx C API 1.0
Public C API and C++ wrapper for sherpa-onnx
Loading...
Searching...
No Matches
Speech Enhancement / Denoising

sherpa-onnx supports both offline and online speech enhancement with three model families: GTCRN, DPDFNet and VoxLock.

See also
SherpaOnnxCreateOfflineSpeechDenoiser, SherpaOnnxCreateOnlineSpeechDenoiser

Offline GTCRN

memset(&config, 0, sizeof(config));
config.model.gtcrn.model = "./gtcrn_simple.onnx";
config.model.num_threads = 1;
config.model.provider = "cpu";
const SherpaOnnxDenoisedAudio *denoised =
SherpaOnnxOfflineSpeechDenoiserRun(sd, wave->samples, wave->num_samples,
wave->sample_rate);
SherpaOnnxWriteWave(denoised->samples, denoised->n, denoised->sample_rate,
"./clean.wav");
struct SherpaOnnxOfflineSpeechDenoiser SherpaOnnxOfflineSpeechDenoiser
Opaque offline speech denoiser handle.
Definition c-api.h:4630
void SherpaOnnxDestroyOfflineSpeechDenoiser(const SherpaOnnxOfflineSpeechDenoiser *sd)
Destroy an offline speech denoiser.
const SherpaOnnxDenoisedAudio * SherpaOnnxOfflineSpeechDenoiserRun(const SherpaOnnxOfflineSpeechDenoiser *sd, const float *samples, int32_t n, int32_t sample_rate)
Run offline speech denoising on a complete waveform.
const SherpaOnnxOfflineSpeechDenoiser * SherpaOnnxCreateOfflineSpeechDenoiser(const SherpaOnnxOfflineSpeechDenoiserConfig *config)
Create an offline speech denoiser.
int32_t SherpaOnnxWriteWave(const float *samples, int32_t n, int32_t sample_rate, const char *filename)
Write floating-point PCM to a mono 16-bit WAVE file.
void SherpaOnnxDestroyDenoisedAudio(const SherpaOnnxDenoisedAudio *p)
Destroy denoised audio returned by a speech enhancement API.
Denoised audio returned by offline or online speech enhancement APIs.
Definition c-api.h:4692
const float * samples
Definition c-api.h:4694
Configuration for offline speech denoising.
Definition c-api.h:4624
SherpaOnnxOfflineSpeechDenoiserModelConfig model
Definition c-api.h:4626
SherpaOnnxOfflineSpeechDenoiserGtcrnModelConfig gtcrn
Definition c-api.h:4610

Model file: gtcrn_simple.onnx

Example source: speech-enhancement-gtcrn-c-api.c

Offline DPDFNet

memset(&config, 0, sizeof(config));
config.model.dpdfnet.model = "./dpdfnet_baseline.onnx";
config.model.num_threads = 1;
config.model.provider = "cpu";
// ... use sd ...
SherpaOnnxOfflineSpeechDenoiserDpdfNetModelConfig dpdfnet
Definition c-api.h:4618

Model file: dpdfnet_baseline.onnx (also: dpdfnet2.onnx, dpdfnet4.onnx, dpdfnet8.onnx for 16 kHz; dpdfnet2_8khz.onnx, dpdfnet8_8khz.onnx for 8 kHz; and dpdfnet2_48khz_hr.onnx, dpdfnet8_48khz_hr.onnx for 48 kHz)

Example source: speech-enhancement-dpdfnet-c-api.c

Offline VoxLock

VoxLock is a target-speaker voice isolation model: it keeps the voice of one speaker (the caller) and removes the other voices. The model is from https://github.com/arbazsiddiqui/voxlock

memset(&config, 0, sizeof(config));
config.model.voxlock.enroll_model = "./sherpa-onnx-voxlock-int8/enroll.onnx";
config.model.voxlock.chunk_model = "./sherpa-onnx-voxlock-int8/chunk_int8.onnx";
"./sherpa-onnx-voxlock-int8/test_wavs/en-voice-sample-f.wav"; /* optional */
config.model.num_threads = 1;
config.model.provider = "cpu";
// ... use sd ...
SherpaOnnxOfflineSpeechDenoiserVoxLockModelConfig voxlock
Definition c-api.h:4620

Model files: enroll.onnx and chunk.onnx (or chunk_int8.onnx) from https://github.com/k2-fsa/sherpa-onnx/releases/tag/speech-enhancement-models

voice_sample is optional: a wave file with at least 3 seconds of the caller's voice. From a longer file, the loudest 3 seconds within its first 30 seconds are used; a file that is shorter or cannot be read is ignored. If it is empty or ignored, the offline denoiser uses the loudest 3 seconds in the first 15 seconds of the input, and the online denoiser passes the audio through unchanged until 3 seconds of speech have been heard, then uses that speech.

Example source: speech-enhancement-voxlock-c-api.c, speech-enhancement-voxlock-with-voice-sample-c-api.c

Online GTCRN

memset(&config, 0, sizeof(config));
config.model.gtcrn.model = "./gtcrn_simple.onnx";
config.model.num_threads = 1;
config.model.provider = "cpu";
int32_t frame_shift =
// Feed audio in frame_shift chunks, then flush
void SherpaOnnxDestroyOnlineSpeechDenoiser(const SherpaOnnxOnlineSpeechDenoiser *sd)
Destroy an online speech denoiser.
struct SherpaOnnxOnlineSpeechDenoiser SherpaOnnxOnlineSpeechDenoiser
Opaque online speech denoiser handle.
Definition c-api.h:4745
const SherpaOnnxOnlineSpeechDenoiser * SherpaOnnxCreateOnlineSpeechDenoiser(const SherpaOnnxOnlineSpeechDenoiserConfig *config)
Create an online speech denoiser.
int32_t SherpaOnnxOnlineSpeechDenoiserGetFrameShiftInSamples(const SherpaOnnxOnlineSpeechDenoiser *sd)
Return the recommended chunk size in samples for streaming input.
const SherpaOnnxDenoisedAudio * SherpaOnnxOnlineSpeechDenoiserFlush(const SherpaOnnxOnlineSpeechDenoiser *sd)
Flush buffered samples and reset the online denoiser.
Configuration for streaming speech denoising.
Definition c-api.h:4739
SherpaOnnxOfflineSpeechDenoiserModelConfig model
Definition c-api.h:4741

Model file: gtcrn_simple.onnx

Example source: online-speech-enhancement-gtcrn-c-api.c

Online DPDFNet

memset(&config, 0, sizeof(config));
config.model.dpdfnet.model = "./dpdfnet_baseline.onnx";
config.model.num_threads = 1;
config.model.provider = "cpu";
// ... use sd ...

Model files: dpdfnet_baseline.onnx, dpdfnet2.onnx, dpdfnet4.onnx, dpdfnet8.onnx for 16 kHz; dpdfnet2_8khz.onnx, dpdfnet8_8khz.onnx for 8 kHz; and dpdfnet2_48khz_hr.onnx, dpdfnet8_48khz_hr.onnx for 48 kHz.

Example source: online-speech-enhancement-dpdfnet-c-api.c

Online VoxLock

memset(&config, 0, sizeof(config));
config.model.voxlock.enroll_model = "./sherpa-onnx-voxlock-int8/enroll.onnx";
config.model.voxlock.chunk_model = "./sherpa-onnx-voxlock-int8/chunk_int8.onnx";
config.model.num_threads = 1;
config.model.provider = "cpu";
// ... feed audio in frame_shift chunks, then flush ...

Model files: enroll.onnx and chunk.onnx (or chunk_int8.onnx); see se_offline_voxlock for the optional voice_sample and for links to the models.

Example source: online-speech-enhancement-voxlock-c-api.c, online-speech-enhancement-voxlock-with-voice-sample-c-api.c