How to use espeak-ng with sherpa-onnx v2.x
sherpa-onnx v2.x has removed espeak-ng and piper-phonemize from its code base, see https://github.com/k2-fsa/sherpa-onnx/pull/4055. This page shows how to update your code from v1.x to v2.x if your TTS model needs espeak-ng for phonemization.
On this page
What changed
In v1.x, espeak-ng and piper-phonemize were built into sherpa-onnx.
For a TTS model using them (e.g., a piper VITS model), you downloaded an
espeak-ng-data directory and passed it via --vits-data-dir (or the
data_dir field of a model config) and sherpa-onnx phonemized the input
text internally.
In v2.x:
The text frontends of sherpa-onnx are now
lexicon — built in, no extra dependency
phonemize — you phonemize the text outside of sherpa-onnx, e.g., with piper-phonemize, and pass the phonemized result to sherpa-onnx.
data_dir, e.g.,--vits-data-dir, is deprecated and ignored. A warning is printed if you still pass it.piper-phonemize is GPL because of espeak-ng, so it stays optional and is not bundled with sherpa-onnx. sherpa-onnx itself is GPL-free now.
The error you get in v2.x
If you run a model that needs espeak-ng without a lexicon and without phonemizing the text first, you get:
phoneme_codepoints is empty, no lexicon, and no frontend available.
Please provide phoneme_codepoints via GenerationConfig or a lexicon via --vits-lexicon.
Please choose one of the two options below.
Option 1: Use a lexicon
If there is a lexicon for your language, nothing changes for you except that
you pass the lexicon instead of espeak-ng-data. For instance, for English
piper models, you can use lexicon-en-us.txt from the
tts-models
release. No external phonemizer is needed.
Option 2: Phonemize the text with piper-phonemize
Install piper-phonemize for your favorite language, phonemize the text outside of sherpa-onnx, and pass the result in. The examples below use the Python API; please see the next section for other languages.
Install piper-phonemize:
pip install piper_phonemize -f https://k2-fsa.github.io/icefall/piper_phonemize.html
Download a model:
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-en_US-amy-low.tar.bz2
tar xf vits-piper-en_US-amy-low.tar.bz2
import sherpa_onnx
from piper_phonemize import phonemize_espeak
tts_config = sherpa_onnx.OfflineTtsConfig(
model=sherpa_onnx.OfflineTtsModelConfig(
vits=sherpa_onnx.OfflineTtsVitsModelConfig(
model="./vits-piper-en_US-amy-low/en_US-amy-low.onnx",
tokens="./vits-piper-en_US-amy-low/tokens.txt",
),
debug=True,
),
)
tts = sherpa_onnx.OfflineTts(tts_config)
# The espeak-ng voice for the phonemizer is taken from the model metadata.
# Fall back to en-us if the model has no language information.
lang = tts.lang
if not lang:
lang = "en-us"
text = "Today as always, men fall into two groups: slaves and free men."
sentences = phonemize_espeak(text, lang)
phoneme_codepoints = [[ord(c) for c in sentence] for sentence in sentences]
gen_config = sherpa_onnx.GenerationConfig()
gen_config.sid = 0
gen_config.speed = 1.0
gen_config.phoneme_codepoints = phoneme_codepoints
# The text has been phonemized already, so pass an empty string here.
audio = tts.generate("", gen_config)
Please see https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/test-offline-tts-piper-phonemize.py for a complete example. There are more examples for other model families:
Other language bindings
piper-phonemize supports many languages. The idea is the same in every language:
Install piper-phonemize for your language.
Phonemize the text with it, using the espeak-ng voice of your model.
Pass the phoneme codepoints to sherpa-onnx via
GenerationConfigand generate with an empty text.
All examples below use the vits-piper-en_US-amy-low model from the
beginning of this page:
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-en_US-amy-low.tar.bz2
tar xf vits-piper-en_US-amy-low.tar.bz2
The espeak-ng-data directory shipped inside the model package is used only
by the phonemizer; sherpa-onnx itself does not read it.
C/C++
Step 1. Install piper-phonemize from a pre-built release or build it from source:
git clone https://github.com/csukuangfj/piper-phonemize
cd piper-phonemize
cmake -Bbuild -DBUILD_SHARED_LIBS=ON
cmake --build build --config Release
Step 2. Link your app with both piper-phonemize and sherpa-onnx and
phonemize the text before generating:
#include <cstdio>
#include <string>
#include "piper-phonemize/cxx-api.h"
#include "sherpa-onnx/c-api/cxx-api.h"
namespace pp = piper_phonemize::cxx;
int main() {
using namespace sherpa_onnx::cxx; // NOLINT
// espeak-ng-data is used only by piper-phonemize; sherpa-onnx itself
// does not use espeak-ng anymore.
if (pp::Initialize("./vits-piper-en_US-amy-low/espeak-ng-data") < 0) {
fprintf(stderr, "Failed to initialize piper-phonemize\n");
return -1;
}
OfflineTtsConfig config;
config.model.vits.model = "./vits-piper-en_US-amy-low/en_US-amy-low.onnx";
config.model.vits.tokens = "./vits-piper-en_US-amy-low/tokens.txt";
config.model.num_threads = 1;
config.model.debug = true;
auto tts = OfflineTts::Create(config);
std::string text =
"Today as always, men fall into two groups: slaves and free men.";
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language information.
std::string lang = tts.Lang();
if (lang.empty()) {
lang = "en-us";
}
auto result = pp::Phonemize(text, lang);
GenerationConfig gen_config;
gen_config.sid = 0;
gen_config.speed = 1.0;
gen_config.phoneme_codepoints = result.sentences;
// The text has been phonemized already; pass an empty string here.
GeneratedAudio audio = tts.Generate("", gen_config);
WriteWave("./generated.wav", {audio.samples, audio.sample_rate});
return 0;
}
If you use the C API, phonemize with PiperPhonemizeInitialize() and
PiperPhonemizeText() from piper-phonemize/c-api.h and fill
phoneme_codepoints, phoneme_codepoints_lens, and
phoneme_codepoints_num_sentences of SherpaOnnxGenerationConfig
(sherpa-onnx/c-api/c-api.h).
Java/Kotlin
Step 1. Add the piper-phonemize artifacts from JitPack. Add the
jitpack.io repository and the following dependencies to your pom.xml
(for Gradle, use the same coordinates in build.gradle):
<repositories>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>
<dependencies>
<!-- JVM core API -->
<dependency>
<groupId>com.github.csukuangfj.piper-phonemize</groupId>
<artifactId>piper-phonemize-jvm</artifactId>
<version>v1.5.9</version>
</dependency>
<!-- native lib, pick ONE for your target platform -->
<dependency>
<groupId>com.github.csukuangfj.piper-phonemize</groupId>
<artifactId>piper-phonemize-native-lib-osx-aarch64</artifactId>
<version>v1.5.9</version>
</dependency>
</dependencies>
Note that piper-phonemize has no all-in-one package: the JVM API and the native libraries are separate artifacts, so both dependencies are needed.
The native artifact is one of piper-phonemize-native-lib-osx-aarch64,
piper-phonemize-native-lib-osx-x64,
piper-phonemize-native-lib-linux-x64,
piper-phonemize-native-lib-linux-aarch64,
piper-phonemize-native-lib-win-x64, or
piper-phonemize-native-lib-win-arm64.
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx. In Java:
import com.github.csukuangfj.piper.phonemize.PiperPhonemize;
import com.k2fsa.sherpa.onnx.*;
public class Main {
public static void main(String[] args) {
OfflineTtsVitsModelConfig vits =
OfflineTtsVitsModelConfig.builder()
.setModel("./vits-piper-en_US-amy-low/en_US-amy-low.onnx")
.setTokens("./vits-piper-en_US-amy-low/tokens.txt")
.build();
OfflineTtsConfig config =
OfflineTtsConfig.builder()
.setModel(
OfflineTtsModelConfig.builder()
.setVits(vits)
.setNumThreads(1)
.setDebug(true)
.build())
.build();
OfflineTts tts = new OfflineTts(config);
// initialize() extracts the espeak-ng-data embedded in piper-phonemize.
// That data feeds only the phonemizer; sherpa-onnx does not use
// espeak-ng anymore.
PiperPhonemize.initialize();
String text =
"Today as always, men fall into two groups: slaves and free men.";
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language information.
String voice = tts.getLang();
if (voice == null || voice.isEmpty()) {
voice = "en-us";
}
int[][] phonemeCodepoints;
try (PiperPhonemize.Result result = PiperPhonemize.text(text, voice)) {
phonemeCodepoints = new int[result.getNumSentences()][];
for (int i = 0; i < result.getNumSentences(); i++) {
phonemeCodepoints[i] = result.getPhonemes(i);
}
}
GenerationConfig genConfig = new GenerationConfig();
genConfig.setSid(0);
genConfig.setSpeed(1.0f);
genConfig.setPhonemeCodepoints(phonemeCodepoints);
// The text has been phonemized already; pass an empty string here.
GeneratedAudio audio = tts.generateWithConfig("", genConfig);
audio.save("tts-piper-en-phonemize.wav");
tts.release();
}
}
In Kotlin:
import com.github.csukuangfj.piper.phonemize.PiperPhonemize
// Phonemize the text and return the phoneme codepoints, one inner array
// per sentence.
fun phonemize(tts: OfflineTts, text: String): Array<IntArray> {
PiperPhonemize.initialize()
var voice = tts.lang()
if (voice.isEmpty()) {
voice = "en-us"
}
val result =
PiperPhonemize.text(text, voice)
?: throw RuntimeException("Failed to phonemize the text")
result.use {
return Array(it.numSentences) { i -> it.getPhonemes(i) }
}
}
val genConfig = GenerationConfig(phonemeCodepoints = phonemize(tts, text))
// The text has been phonemized already; pass an empty string here.
val audio = tts.generateWithConfig(text = "", config = genConfig)
audio.save(filename = "tts-piper-en-phonemize.wav")
Go
Step 1. Add the dependencies:
go get github.com/csukuangfj/piper-phonemize-go/piper_phonemize
go get github.com/k2-fsa/sherpa-onnx-go/v2/sherpa_onnx
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
package main
import (
"log"
pp "github.com/csukuangfj/piper-phonemize-go/piper_phonemize"
sherpa "github.com/k2-fsa/sherpa-onnx-go/v2/sherpa_onnx"
)
func main() {
config := sherpa.OfflineTtsConfig{}
config.Model.Vits.Model = "./vits-piper-en_US-amy-low/en_US-amy-low.onnx"
config.Model.Vits.Tokens = "./vits-piper-en_US-amy-low/tokens.txt"
config.Model.NumThreads = 1
config.Model.Debug = 1
tts := sherpa.NewOfflineTts(&config)
defer sherpa.DeleteOfflineTts(tts)
// Initialize("") uses the espeak-ng-data embedded in
// piper-phonemize-go. It has nothing to do with sherpa-onnx;
// sherpa-onnx itself does not use espeak-ng anymore.
if ret := pp.Initialize(""); ret < 0 {
log.Fatalf("Failed to initialize piper-phonemize")
}
text := "Today as always, men fall into two groups: slaves and free men."
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language
// information.
lang := tts.Lang()
if lang == "" {
lang = "en-us"
}
result := pp.Phonemize(text, lang)
if result == nil {
log.Fatalf("Failed to phonemize the text")
}
defer pp.DeletePhonemizeResult(result)
var sentences [][]int32
for i := 0; i < result.GetNumSentences(); i++ {
var s []int32
for _, cp := range result.GetPhonemes(i) {
s = append(s, int32(cp))
}
sentences = append(sentences, s)
}
cfg := sherpa.GenerationConfig{
SilenceScale: 0.2,
Speed: 1.0,
Sid: 0,
PhonemeCodepoints: sentences,
}
// The text has been phonemized already; pass an empty string here.
audio := tts.GenerateWithConfig("", &cfg, nil)
audio.Save("./generated.wav")
}
C#/.NET
Step 1. Add the NuGet packages to your .csproj:
<ItemGroup>
<PackageReference Include="org.k2fsa.sherpa.onnx" Version="*" />
<PackageReference Include="com.github.csukuangfj.piper.phonemize" Version="*" />
</ItemGroup>
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
using PiperPhonemize;
using SherpaOnnx;
// The piper-phonemize NuGet package copies espeak-ng-data next to the
// executable. It feeds only the phonemizer; sherpa-onnx does not use
// espeak-ng anymore.
string dataDir = Path.Combine(AppContext.BaseDirectory, "espeak-ng-data");
if (PiperPhonemizeApi.Initialize(dataDir) < 0) {
throw new Exception("Failed to initialize piper-phonemize");
}
var config = new OfflineTtsConfig();
config.Model.Vits.Model = "./vits-piper-en_US-amy-low/en_US-amy-low.onnx";
config.Model.Vits.Tokens = "./vits-piper-en_US-amy-low/tokens.txt";
config.Model.NumThreads = 1;
config.Model.Debug = 1;
var tts = new OfflineTts(config);
string text = "Today as always, men fall into two groups: slaves and free men.";
// The espeak-ng voice for the phonemizer is taken from the model metadata.
// Fall back to en-us if the model has no language information.
string lang = string.IsNullOrEmpty(tts.Lang) ? "en-us" : tts.Lang;
int[][] phonemeCodepoints;
using (var result = PiperPhonemizeApi.Text(text, lang)) {
phonemeCodepoints = new int[result.NumSentences][];
for (int i = 0; i < result.NumSentences; i++) {
string ipa = result.GetPhonemesAsString(i) ?? "";
int[] cps = new int[ipa.Length];
for (int j = 0; j < ipa.Length; j++) {
cps[j] = ipa[j];
}
phonemeCodepoints[i] = cps;
}
}
var genConfig = new OfflineTtsGenerationConfig();
genConfig.Sid = 0;
genConfig.Speed = 1.0f;
genConfig.PhonemeCodepoints = phonemeCodepoints;
// The text has been phonemized already; pass an empty string here.
var audio = tts.GenerateWithConfig("", genConfig, null);
audio.SaveToWaveFile("./generated-dotnet-phonemize.wav");
Rust
Step 1. Add the crates to your Cargo.toml. The piper-phonemize crate
downloads pre-built libraries and embeds espeak-ng-data automatically:
[dependencies]
sherpa-onnx = "2.0.1"
piper-phonemize = "1.5.9"
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
use sherpa_onnx::{
GenerationConfig, OfflineTts, OfflineTtsConfig, OfflineTtsVitsModelConfig,
};
fn main() {
// Note: the espeak-ng-data used by piper-phonemize is embedded in the
// crate and initialized automatically on first use. It has nothing to
// do with sherpa-onnx; sherpa-onnx itself does not use espeak-ng
// anymore.
let config = OfflineTtsConfig {
model: sherpa_onnx::OfflineTtsModelConfig {
vits: OfflineTtsVitsModelConfig {
model: Some("./vits-piper-en_US-amy-low/en_US-amy-low.onnx".into()),
tokens: Some("./vits-piper-en_US-amy-low/tokens.txt".into()),
..Default::default()
},
num_threads: 2,
debug: false,
..Default::default()
},
..Default::default()
};
let tts = OfflineTts::create(&config).expect("Failed to create OfflineTts");
let text = "Today as always, men fall into two groups: slaves and free men.";
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language information.
let lang = {
let l = tts.lang();
if l.is_empty() {
"en-us".to_string()
} else {
l
}
};
let result =
piper_phonemize::phonemize(text, &lang).expect("Failed to phonemize the text");
let mut phoneme_codepoints = Vec::new();
for i in 0..result.num_sentences() {
let phonemes = result.get_phonemes(i).unwrap_or_default();
phoneme_codepoints.push(phonemes.into_iter().map(|c| c as i32).collect::<Vec<i32>>());
}
let gen_config = GenerationConfig {
sid: 0,
speed: 1.0,
phoneme_codepoints: Some(phoneme_codepoints),
..Default::default()
};
// The text has been phonemized already; pass an empty string here.
let audio = tts
.generate_with_config(
"",
&gen_config,
Some(|_samples: &[f32], progress: f32| -> bool {
println!("Progress: {:.1}%", progress * 100.0);
true
}),
)
.expect("Generation failed");
audio.save("./generated-vits-en-phonemize-rust.wav");
}
JavaScript (Node.js)
Step 1. Install the packages. piper-phonemize-node is the native addon;
npm install piper-phonemize installs the WASM version, which exposes the
same functions:
npm install sherpa-onnx-node piper-phonemize-node
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
const sherpa_onnx = require('sherpa-onnx-node');
const piperPhonemize = require('piper-phonemize-node');
const tts = new sherpa_onnx.OfflineTts({
model: {
vits: {
model: './vits-piper-en_US-amy-low/en_US-amy-low.onnx',
tokens: './vits-piper-en_US-amy-low/tokens.txt',
},
debug: true,
numThreads: 1,
provider: 'cpu',
},
maxNumSentences: 1,
});
const text =
'Today as always, men fall into two groups: slaves and free men.';
// The espeak-ng voice for the phonemizer is taken from the model metadata.
// Fall back to en-us if the model has no language information.
const voice = tts.lang || 'en-us';
const phonemeCodepoints = piperPhonemize.phonemize(text, voice)
// piper may emit empty sentences (e.g., a trailing segment after
// punctuation); the C API requires every sentence to be non-empty.
.filter((sentence) => sentence.length > 0);
const genConfig = new sherpa_onnx.GenerationConfig({
phonemeCodepoints,
sid: 0,
speed: 1.0,
silenceScale: 0.2,
});
// The text has been phonemized already; pass an empty string here.
const audio = tts.generate({text: '', generationConfig: genConfig});
sherpa_onnx.writeWave(
'test-piper-en-phonemize.wav',
{samples: audio.samples, sampleRate: audio.sampleRate});
Dart/Flutter
Step 1. Add the packages to your pubspec.yaml:
dependencies:
sherpa_onnx: ^2.0.1
piper_phonemize: ^1.5.9
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
import 'package:piper_phonemize/piper_phonemize.dart' as piper;
import 'package:sherpa_onnx/sherpa_onnx.dart' as sherpa_onnx;
Future<void> main() async {
await sherpa_onnx.initBindingsAsync();
await piper.initBindingsAsync();
final model = './vits-piper-en_US-amy-low/en_US-amy-low.onnx';
final tokens = './vits-piper-en_US-amy-low/tokens.txt';
// espeak-ng-data feeds only the piper phonemizer; sherpa-onnx ignores
// it. Use the copy shipped inside the model package.
final ret = piper.PiperPhonemize
.initialize('./vits-piper-en_US-amy-low/espeak-ng-data');
if (ret < 0) {
throw Exception('Failed to initialize the piper phonemizer');
}
final tts = sherpa_onnx.OfflineTts(sherpa_onnx.OfflineTtsConfig(
model: sherpa_onnx.OfflineTtsModelConfig(
vits: sherpa_onnx.OfflineTtsVitsModelConfig(
model: model,
tokens: tokens,
),
numThreads: 1,
debug: true,
),
maxNumSentences: 1,
));
const text =
'Today as always, men fall into two groups: slaves and free men.';
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language information.
final voice = tts.lang.isEmpty ? 'en-us' : tts.lang;
// piper may emit empty sentences (e.g., a trailing segment after
// punctuation); the C API requires every sentence to be non-empty.
final sentences = piper.PiperPhonemize
.phonemize(text, voice: voice)
.where((sentence) => sentence.isNotEmpty)
.toList();
final genConfig = sherpa_onnx.OfflineTtsGenerationConfig(
sid: 0,
speed: 1.0,
silenceScale: 0.2,
phonemeCodepoints: sentences,
);
// The text has been phonemized already; pass an empty string here.
final audio = tts.generateWithConfig(text: '', config: genConfig);
tts.free();
sherpa_onnx.writeWave(
filename: './generated-dart.wav',
samples: audio.samples,
sampleRate: audio.sampleRate,
);
}
Note
In a Flutter app, espeak-ng-data is a Flutter asset and cannot be used
as a path directly. On non-web platforms, extract it first:
final dataDir = await piper.extractEspeakNgData();
piper.PiperPhonemize.initialize(dataDir);
On web, the data is embedded in the WASM bundle; call
piper.PiperPhonemize.initialize('') instead.
Swift
Step 1. Add both packages to your Package.swift:
// swift-tools-version:5.9
import PackageDescription
let package = Package(
name: "MyTtsApp",
platforms: [.iOS(.v13), .macOS(.v10_15)],
dependencies: [
.package(url: "https://github.com/k2-fsa/sherpa-onnx", branch: "master"),
.package(url: "https://github.com/csukuangfj/piper-phonemize", branch: "master"),
],
targets: [
.executableTarget(
name: "MyTtsApp",
dependencies: [
.product(name: "sherpa-onnx", package: "sherpa-onnx"),
.product(name: "piper-phonemize", package: "piper-phonemize"),
]
)
]
)
Step 2. Phonemize the text and pass the codepoints to sherpa-onnx:
import Foundation
import SherpaOnnx
import piper_phonemize
@main
struct App {
static func main() {
// espeak-ng-data is used only by piper-phonemize; sherpa-onnx itself
// does not use espeak-ng anymore. The Swift package bundles the data.
guard let dataDir = piperPhonemizeBundledDataDir() else {
print("espeak-ng-data not found")
return
}
_ = piperPhonemizeInitialize(dataDir: dataDir)
let vits = sherpaOnnxOfflineTtsVitsModelConfig(
model: "./vits-piper-en_US-amy-low/en_US-amy-low.onnx",
lexicon: "",
tokens: "./vits-piper-en_US-amy-low/tokens.txt"
)
let modelConfig = sherpaOnnxOfflineTtsModelConfig(vits: vits)
var ttsConfig = sherpaOnnxOfflineTtsConfig(model: modelConfig)
let tts = SherpaOnnxOfflineTtsWrapper(config: &ttsConfig)
let text =
"Today as always, men fall into two groups: slaves and free men."
// The espeak-ng voice for the phonemizer is taken from the model
// metadata. Fall back to en-us if the model has no language information.
let voice = tts.lang.isEmpty ? "en-us" : tts.lang
guard let result = piperPhonemizeText(text: text, voice: voice) else {
print("Failed to phonemize the text")
return
}
var phonemeCodepoints: [[Int32]] = []
for i in 0..<result.numSentences {
phonemeCodepoints.append(result.getPhonemes(sentenceId: i))
}
let genConfig = SherpaOnnxGenerationConfigSwift(
speed: 1.0, sid: 0, phonemeCodepoints: phonemeCodepoints)
// The text has been phonemized already; pass an empty string here.
let audio = tts.generateWithConfig(
text: "", config: genConfig, callback: nil, arg: nil)
_ = audio.save(filename: "test-piper-phonemize.wav")
}
}
WebAssembly
Step 1. Download the piper-phonemize wasm bundle and unzip it into
sherpa-onnx/wasm/tts/piper-phonemize/:
cd sherpa-onnx/wasm/tts
wget -q https://github.com/csukuangfj/piper-phonemize/releases/download/wasm/piper-phonemize-v1.5.9-wasm-browser.zip
unzip -q piper-phonemize-v1.5.9-wasm-browser.zip
mv browser piper-phonemize
rm piper-phonemize-v1.5.9-wasm-browser.zip
Step 2. Load the scripts in your page in this order. Define Module before
loading the Emscripten glue script so the glue attaches its runtime to it:
<script src="piper-phonemize/piper-phonemize.js"></script>
<script src="app-tts.js"></script>
<!-- The Emscripten glue must be loaded last, after Module is defined -->
<script src="piper-phonemize/piper-phonemize-wasm-main.js"></script>
// Define Module before loading piper-phonemize/piper-phonemize-wasm-main.js
var Module = {
locateFile: function(path, scriptDirectory = '') {
// The glue fetches its .data with an empty scriptDirectory; prefix
// "piper-phonemize/" so the request resolves next to the bundle files.
return (scriptDirectory || 'piper-phonemize/') + path;
},
onRuntimeInitialized: function() {
piperPhonemizeInitialize(); // initialize espeak-ng
},
};
Step 3. Phonemize the text and pass the codepoints to sherpa-onnx, where
tts is created by createOfflineTts() from sherpa-onnx-tts.js:
const voice = 'en-us'; // espeak-ng voice of your model
const sentences = piperPhonemize(text, voice);
// piper may emit empty sentences (e.g., a trailing segment after
// punctuation); the C API requires every sentence to be non-empty.
const phonemeCodepoints = sentences.filter((s) => s.length > 0);
// The text has been phonemized already; pass an empty string here.
const audio = tts.generateWithConfig('', {
phonemeCodepoints: phonemeCodepoints,
sid: 0,
speed: 1.0,
});
Notes
The espeak-ng voice (e.g.,
en-us) is used only by the phonemizer, not by sherpa-onnx. Most examples read it from the model metadata and fall back toen-usif the model has no language information.piper-phonemize may return empty sentences, e.g., a trailing segment after punctuation. Please filter them out before passing the result to sherpa-onnx.
If your model uses string tokens, e.g., Chinese pinyin like
zhong1, passtokensinstead ofphoneme_codepoints; the two are mutually exclusive. Please see https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/test-offline-tts-matcha-zh-en-phonemize.pyThe
espeak-ng-datadirectory inside a model archive is used only by the external phonemizer; sherpa-onnx itself does not read it anymore. If you use a lexicon, you don’t need it at all.