mirror of
https://github.com/coqui-ai/TTS.git
synced 2026-07-10 20:41:10 +02:00
a3d5801c44
* initial commit * Tortoise inference * revert path change * style fix * remove accidental remove * style fixes * style fixes * removed unwanted assests and deps * remove changes * remove cvvp * style fix black * added tortoise config and updated config and args, refactoring the code * added tortoise to api * Pull mel_norm from url * Use TTS cleaners * Let download model files * add ability to pass tortoise presets through coqui api * fix tests * fix style and tests * fix tts commandline for tortoise * Add config.json to tortoise * Use kwargs * Use regular model api for loading tortoise * Add load from dir to synthesizer * Fix Tortoise floats * Use model_dir when there are multiple urls * Use `synthesize` when exists * lint fixes and resolve preset bug * resolve a download bug and update model link * fix json * do tortoise inference from voice dir * fix * fix test * fix speaker id and remove assests * update inference_tests.yml * replace inference_test.yml * fix extra dir as None * fix tests * remove space * Reformat docstring * Add docs * Update docs * lint fixes --------- Co-authored-by: Eren Gölge <egolge@coqui.ai> Co-authored-by: Eren Gölge <erogol@hotmail.com>
3.1 KiB
3.1 KiB
Tortoise 🐢
Tortoise is a very expressive TTS system with impressive voice cloning capabilities. It is based on an GPT like autogressive acoustic model that converts input text to discritized acouistic tokens, a diffusion model that converts these tokens to melspeectrogram frames and a Univnet vocoder to convert the spectrograms to the final audio signal. The important downside is that Tortoise is very slow compared to the parallel TTS models like VITS.
Big thanks to 👑@manmay-nakhashi who helped us implement Tortoise in 🐸TTS.
Example use:
from TTS.tts.configs.tortoise_config import TortoiseConfig
from TTS.tts.models.tortoise import Tortoise
config = TortoiseConfig()
model = Tortoise.inif_from_config(config)
model.load_checkpoint(config, checkpoint_dir="paths/to/models_dir/", eval=True)
# with random speaker
output_dict = model.synthesize(text, config, speaker_id="random", extra_voice_dirs=None, **kwargs)
# cloning a speaker
output_dict = model.synthesize(text, config, speaker_id="speaker_n", extra_voice_dirs="path/to/speaker_n/", **kwargs)
Using 🐸TTS API:
from TTS.api import TTS
tts = TTS("tts_models/en/multi-dataset/tortoise-v2")
# cloning `lj` voice from `TTS/tts/utils/assets/tortoise/voices/lj`
# with custom inference settings overriding defaults.
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
file_path="output.wav",
voice_dir="TTS/tts/utils/assets/tortoise/voices/",
speaker="lj",
num_autoregressive_samples=1,
diffusion_iterations=10)
# Using presets with the same voice
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
file_path="output.wav",
voice_dir="TTS/tts/utils/assets/tortoise/voices/",
speaker="lj",
preset="ultra_fast")
# Random voice generation
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
file_path="output.wav")
Using 🐸TTS Command line:
# cloning the `lj` voice
tts --model_name tts_models/en/multi-dataset/tortoise-v2 \
--text "This is an example." \
--out_path "/data/speech_synth/coqui-tts/TTS/tests/outputs/output.wav" \
--voice_dir TTS/tts/utils/assets/tortoise/voices/ \
--speaker_idx "lj" \
--progress_bar True
# Random voice generation
tts --model_name tts_models/en/multi-dataset/tortoise-v2 \
--text "This is an example." \
--out_path "/data/speech_synth/coqui-tts/TTS/tests/outputs/output.wav" \
--progress_bar True
Important resources & papers
- Original Repo: https://github.com/neonbjb/tortoise-tts
- Faster implementation: https://github.com/152334H/tortoise-tts-fast
- Univnet: https://arxiv.org/abs/2106.07889
- Latent Diffusion:https://arxiv.org/abs/2112.10752
- DALL-E: https://arxiv.org/abs/2102.12092
TortoiseConfig
.. autoclass:: TTS.tts.configs.tortoise_config.TortoiseConfig
:members:
TortoiseArgs
.. autoclass:: TTS.tts.models.tortoise.TortoiseArgs
:members:
Tortoise Model
.. autoclass:: TTS.tts.models.tortoise.Tortoise
:members: