nyosegawa / speech.cpp

Speech synthesis, speech recognition and voice activity detection for voice conversation, in C++ on ggml
 Models ·
 Install ·
 Quick start ·
 Documentation ·
 C API 
 English · 日本語 
speech.cpp is one command, `speech`, and one shared library, `libspeech`, behind one C API, on the CPU, Metal and
Vulkan through ggml. The command speaks text, writes the text of recordings, finds
where someone speaks in them, transcribes the microphone as someone speaks (`speech asr --live`), serves OpenAI's audio
API and its Realtime transcription over HTTP, and runs a worker process for any program that keeps models loaded between
requests, ASIST among them.
**A time to first audio of a quarter of a second or less.** On an Apple M5, Qwen3-TTS 0.6B sends its first audio
0.029 s after the text and Irodori-TTS v4.1-Small-MF 0.16 s after it, and the rest follows while it plays.
**One executable, nothing else to install.** No Python and no CUDA: `speech` fetches each model by name, one GGUF file
with its codec, and runs it on Metal, Vulkan or the CPU.
**OpenAI's API on your own computer.** `speech serve` answers OpenAI's audio API and its Realtime transcription, so an
OpenAI client works by changing its base URL.
**Checked against the official implementations.** Every stage of every model is compared with the model's official
implementation, from the same inputs.
 The page of speech serve --open : pick a model of the catalog, fetch it and try it. 
 Transcribe writes a file or a recording, a region at a time where someone speaks. 
 Live writes the microphone as you speak; the grey text is the utterance still being said. 
## Models
| Task | Name | Languages | Good at |
|---|---|---|---|
| Synthesis | `qwen3-tts-0.6b`, `qwen3-tts-1.7b` | de en es fr it ja ko pt ru zh | nine named speakers; audio that streams from its first frame of 0.08 s; the 1.7B model follows an instruction of how to speak |
| Synthesis | `irodori-tts-mf`, `irodori-tts`, `irodori-tts-large` | ja | Japanese in the voice of a reference recording, or in a voice described in words; the large model follows both more closely |
| Synthesis | `drama-3` | 83 languages | a scene of up to 16 speakers, with cues such as laughter written in the text, in voices cloned from recordings and their words; a 20.8 GB file. Built with Fish Audio |
| Recognition | `qwen3-asr-0.6b`, `qwen3-asr-1.7b` | 30 languages | finds the language itself, takes a prompt of names and terms, and up to 20 minutes at once |
| Recognition | `reazonspeech-v2` | ja | Japanese with punctuation, in recordings of many minutes |
| Recognition | `parakeet-tdt_ctc-0.6b-ja` | ja | short Japanese utterances, fast, with times |
| Recognition | `parakeet-tdt-0.6b-v3` | 25 European languages | finds the language itself, with punctuation, capitals and times |
| Detection | `silero-vad` | any | where someone speaks, to give a recognizer one utterance at a time; 1.2 MB |
A command takes a model by its name and fetches its converted file from Hugging Face, from