vits-piper-en_US-kusal-medium

Info about this model

This model is converted from https://huggingface.co/rhasspy/piper-voices/tree/main/en/en_US/kusal/medium

Model download address

https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-en_US-kusal-medium.tar.bz2

Android APK

The following table shows the Android TTS Engine APK with this model for sherpa-onnx v1.12.1

If you don't know what ABI is, you probably need to select arm64-v8a.

The source code for the APK can be found at

https://github.com/k2-fsa/sherpa-onnx/tree/master/android/SherpaOnnxTtsEngine

Please refer to the documentation for how to build the APK from source code.

More Android APKs can be found at

https://k2-fsa.github.io/sherpa/onnx/tts/apk-engine.html

C API

You can use the following code to play with vits-piper-en_US-kusal-medium with C API.

#include <stdio.h>
#include <string.h>

#include "sherpa-onnx/c-api/c-api.h"

int main() {
  SherpaOnnxOfflineTtsConfig config;
  memset(&config, 0, sizeof(config));

  config.model.vits.model = "vits-piper-en_US-kusal-medium/en_US-kusal-medium.onnx";
  config.model.vits.tokens = "vits-piper-en_US-kusal-medium/tokens.txt";
  config.model.vits.data_dir = "vits-piper-en_US-kusal-medium/espeak-ng-data";
  config.model.num_threads = 1;
  const SherpaOnnxOfflineTts *tts = SherpaOnnxCreateOfflineTts(&config);

  int sid = 0; // speaker id
  const char *text = "Friends fell out often because life was changing so fast. The easiest thing in the world was to lose touch with someone.";

  const SherpaOnnxGeneratedAudio *audio =
      SherpaOnnxOfflineTtsGenerate(tts, text, sid, 1.0);

  SherpaOnnxWriteWave(audio->samples, audio->n, audio->sample_rate,
                      "./test.wav");

  // You need to free the pointers to avoid memory leak in your app
  SherpaOnnxDestroyOfflineTtsGeneratedAudio(audio);
  SherpaOnnxDestroyOfflineTts(tts);

  printf("Saved to ./test.wav\n");

  return 0;
}

In the following, we describe how to compile and run the above C example.

Use shared library (dynamic link)

cd /tmp
git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx
mkdir build-shared
cd build-shared

cmake  -DSHERPA_ONNX_ENABLE_C_API=ON  -DCMAKE_BUILD_TYPE=Release  -DBUILD_SHARED_LIBS=ON  -DCMAKE_INSTALL_PREFIX=/tmp/sherpa-onnx/shared  ..

make
make install

You can find required header file and library files inside /tmp/sherpa-onnx/shared.

Assume you have saved the above example file as /tmp/test-piper.c. Then you can compile it with the following command:

gcc   -I /tmp/sherpa-onnx/shared/include   -L /tmp/sherpa-onnx/shared/lib   -lsherpa-onnx-c-api   -lonnxruntime   -o /tmp/test-piper   /tmp/test-piper.c

Now you can run

cd /tmp

# Assume you have downloaded the model and extracted it to /tmp
./test-piper

You probably need to run

# For Linux
export LD_LIBRARY_PATH=/tmp/sherpa-onnx/shared/lib:$LD_LIBRARY_PATH

# For macOS
export DYLD_LIBRARY_PATH=/tmp/sherpa-onnx/shared/lib:$DYLD_LIBRARY_PATH

before you run /tmp/test-piper.

Use static library (static link)

Please see the documentation at

https://k2-fsa.github.io/sherpa/onnx/c-api/index.html

Python API

Assume you have installed sherpa-onnx via

pip install sherpa-onnx

and you have downloaded the model from

https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-en_US-kusal-medium.tar.bz2

You can use the following code to play with vits-piper-en_US-kusal-medium

import sherpa_onnx
import soundfile as sf

config = sherpa_onnx.OfflineTtsConfig(
    model=sherpa_onnx.OfflineTtsModelConfig(
        vits=sherpa_onnx.OfflineTtsVitsModelConfig(
            model="vits-piper-en_US-kusal-medium/en_US-kusal-medium.onnx",
            lexicon="",
            data_dir="vits-piper-en_US-kusal-medium/espeak-ng-data",
            tokens="vits-piper-en_US-kusal-medium/tokens.txt",
        ),
        num_threads=1,
    ),
)

if not config.validate():
    raise ValueError("Please check your config")

tts = sherpa_onnx.OfflineTts(config)
audio = tts.generate(text="Friends fell out often because life was changing so fast. The easiest thing in the world was to lose touch with someone.",
                     sid=0,
                     speed=1.0)

sf.write("test.mp3", audio.samples, samplerate=audio.sample_rate)

Samples

For the following text:

Friends fell out often because life was changing so fast.
The easiest thing in the world was to lose touch with someone.

sample audios for different speakers are listed below:

Speaker 0