coqui-ai TTS: Install locally and send voice messages to Telegram

09.01.2025 iBuben Без рубрики

Back in 2001, I stumbled across a curious disc titled «Speech Recognition Systems.» The disc was filled with various STT and TTS programs for Windows, and at the time, such software seemed like a miracle of miracles, despite the fact that the voice synthesis sounded, to put it mildly, lousy.

With the advent of neural networks, voice synthesis has reached such a level that it’s sometimes difficult to distinguish it from a live speaker. However, good voice engines are mostly cloud-based and often have limitations on free use. But what if we need to synthesize voice locally? Let’s set up our own TTS server, no registration or SMS required!

After searching the web, I came across a suitable open source TTS project called coqui-ai and decided to try installing it.

The software manual describes installing it via Docker, but for some unknown reason, this option didn’t work for me. So, let’s try a different approach:
For installation we need: Debian 12, python3 and pip.

 pip install TTS

After the package has been successfully installed, you can view the list of current voice models by entering the command tts —list_models

As it turns out, there’s no dedicated voice model for Russian on the list. However, there is a multilingual model, tts_models/multilingual/multi-dataset/xtts_v2, which includes support for Russian.

Now let’s see which voices (speakers) support the model:

 tts --model_name "tts_models/multilingual/multi-dataset/xtts_v2" --list_speaker_idxs

We select any of the voices and try to generate speech:

 tts --text "Привет, дружок!" --model_name "tts_models/multilingual/multi-dataset/xtts_v2" --out_path /tmp/speech.wav --speaker_idx "Damien Black" --language_idx "ru"

In this line, we specified the text, model name, path to save the result, speaker name, and language. When this line is executed, TTS will automatically download and launch the specified model. Keep in mind that launching the model takes some time, so for further work with TTS, we’ll run it as a server with an API.

And here’s where I got a bit of a surprise. The instructions don’t directly explain how to do this correctly. After some trial and error with ChatGPT, I discovered that the server can be started like this:

 tts-server   --model_name "tts_models/multilingual/multi-dataset/xtts_v2"   --config_path "/root/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2/config.json"   --model_path "/root/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2"

Without explicitly specifying the paths, this thing wouldn’t work at all. Let’s add && to the end to force the process to go into the background. If everything is OK, the output will look like this:

  * Serving Flask app 'TTS.server.server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (::)
 * Running on http://[::1]:5002
 * Running on http://[::1]:5002

Now the server is listening on port 5002, and requests can be sent to it. And then surprise number two awaited me: there’s no clear explanation anywhere on how to actually work with the API.

ChatGPT kept pushing POST methods at me, but it turns out GET was the working method. Let’s create a test request:

http://YOUR_SERVER_IP:5002/api/tts?text=This is what a synthetic voice sounds like in Russian. Well, how do you like it? It’s not bad, really?&language_id=ru&speaker_id=Dionisio%20Schuyler

Since we are dealing with a real neural network, each new call will generate speech differently, despite the fact that we do not change the text.

However, sometimes strange artifacts creep into the synthesis, as in the following example:

The text didn’t change, but as a result, «pan» (or «pam»?) appeared from somewhere. In short, some kind of glitch…

Now, let’s try to put this into practice! My idea was to teach a Telegram bot to send voice messages to a chat. For this task, I wrote the following code example, (as always) in PHP.

For the script to work successfully, you’ll need to install ffmpeg on the server, since our audio files will need to be re-encoded into OGG format for Telegram.

 <pre class="wp-block-syntaxhighlighter-code"><?php
// Функция для генерации речи через TTS-сервер
function generateTTS($text, $language, $speaker, $ttsUrl, $outputFile) {
    // Подготовка данных для запроса
    $params = http_build_query([
        'text' => $text,
        'language_id' => $language,
        'speaker_id' => $speaker,
    ]);
    
    // Выполнение GET-запроса к TTS-серверу
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $ttsUrl . "/api/tts?" . $params);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    
    $response = curl_exec($ch);
    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);
    
    if ($httpCode === 200) {
        // Сохранение аудиофайла (например, в WAV)
        file_put_contents($outputFile, $response);
        return true;
    } else {
        echo "Ошибка при генерации речи: HTTP $httpCode\n";
        return false;
    }
}

// Функция для конвертации аудио в формат OGG
function convertToOgg($inputFile, $outputFile) {
    $command = "ffmpeg -y -i " . escapeshellarg($inputFile) . " -c:a libopus " . escapeshellarg($outputFile);
    exec($command, $output, $returnCode);
    
    if ($returnCode === 0) {
        echo "Аудиофайл успешно конвертирован в OGG: $outputFile\n";
        return true;
    } else {
        echo "Ошибка при конвертации аудио: " . implode("\n", $output) . "\n";
        return false;
    }
}

// Функция для отправки голосового сообщения в Telegram
function sendVoiceToTelegram($chatId, $voiceFile, $botToken) {
    // Подготовка данных для запроса
    $url = "https://api.telegram.org/bot$botToken/sendVoice";
    $postFields = [
        'chat_id' => $chatId,
        'voice' => new CURLFile($voiceFile)
    ];
    
    // Выполнение POST-запроса к Telegram API
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields);
    
    $response = curl_exec($ch);
    $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);
    
    if ($httpCode === 200) {
        echo "Сообщение успешно отправлено в Telegram.\n";
    } else {
        echo "Ошибка при отправке сообщения в Telegram: HTTP $httpCode\n";
        echo "Ответ: $response\n";
    }
}

// Конфигурация
$ttsUrl = "http://IP_TTS_СЕРВЕРА:5002"; // URL TTS-сервера
$inputFile = "/tmp/speech.wav"; // Временный файл для оригинального аудио
$outputFile = "/tmp/speech.ogg"; // Файл для конвертации в OGG
$text = "Хочешь, я расскажу тебе сказку, дружок?"; // Текст для генерации
$language = "ru"; // Язык синтеза
$speaker = "Dionisio Schuyler"; // Говорун
$chatId = "........."; // ID чата в Telegram
$botToken = "......................."; // Токен вашего бота Telegram

// Генерация речи
if (generateTTS($text, $language, $speaker, $ttsUrl, $inputFile)) {
    // Конвертация в формат OGG
    if (convertToOgg($inputFile, $outputFile)) {
        // Отправка в Telegram
        sendVoiceToTelegram($chatId, $outputFile, $botToken);
    }
}</pre>

Later, the source of the text became ChatGPT, but that’s a slightly different story… :)

Result:
Now I have my own TTS server. Despite the rather lengthy speech generation process, this option suits me, as I wasn’t aiming to generate speech on the fly. However, if you have such a requirement, the synthesis can be sped up by offloading the calculations from the processor to a decent graphics card with CUDA support. Since I don’t have one on my server yet, you’ll have to test the engine’s speed yourself. ;)