mirror of
https://github.com/myshell-ai/OpenVoice.git
synced 2025-12-16 08:27:48 +01:00
update docs
This commit is contained in:
60
README.md
60
README.md
@@ -7,10 +7,6 @@
|
|||||||
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
## Join Our Community
|
|
||||||
|
|
||||||
Join our [Discord community](https://discord.gg/myshell) and select the `Developer` role upon joining to gain exclusive access to our developer-only channel! Don't miss out on valuable discussions and collaboration opportunities.
|
|
||||||
|
|
||||||
## Introduction
|
## Introduction
|
||||||
As we detailed in our [paper](https://arxiv.org/abs/2312.01479) and [website](https://research.myshell.ai/open-voice), the advantages of OpenVoice are three-fold:
|
As we detailed in our [paper](https://arxiv.org/abs/2312.01479) and [website](https://research.myshell.ai/open-voice), the advantages of OpenVoice are three-fold:
|
||||||
|
|
||||||
@@ -25,12 +21,6 @@ Neither of the language of the generated speech nor the language of the referenc
|
|||||||
|
|
||||||
[Video](https://github.com/myshell-ai/OpenVoice/assets/40556743/3cba936f-82bf-476c-9e52-09f0f417bb2f)
|
[Video](https://github.com/myshell-ai/OpenVoice/assets/40556743/3cba936f-82bf-476c-9e52-09f0f417bb2f)
|
||||||
|
|
||||||
<div align="center">
|
|
||||||
<div> </div>
|
|
||||||
<img src="resources/framework-ipa.png" width="800"/>
|
|
||||||
<div> </div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
OpenVoice has been powering the instant voice cloning capability of [myshell.ai](https://app.myshell.ai/explore) since May 2023. Until Nov 2023, the voice cloning model has been used tens of millions of times by users worldwide, and witnessed the explosive user growth on the platform.
|
OpenVoice has been powering the instant voice cloning capability of [myshell.ai](https://app.myshell.ai/explore) since May 2023. Until Nov 2023, the voice cloning model has been used tens of millions of times by users worldwide, and witnessed the explosive user growth on the platform.
|
||||||
|
|
||||||
## Main Contributors
|
## Main Contributors
|
||||||
@@ -40,55 +30,16 @@ OpenVoice has been powering the instant voice cloning capability of [myshell.ai]
|
|||||||
- [Xumin Yu](https://yuxumin.github.io) at Tsinghua University
|
- [Xumin Yu](https://yuxumin.github.io) at Tsinghua University
|
||||||
- [Ethan Sun](https://twitter.com/ethan_myshell) at MyShell
|
- [Ethan Sun](https://twitter.com/ethan_myshell) at MyShell
|
||||||
|
|
||||||
## Live Demo
|
## How to Use
|
||||||
|
Please see [usage](docs/USAGE.md) for detailed instructions.
|
||||||
<div align="center">
|
|
||||||
<a href="https://www.lepton.ai/playground/openvoice"><img src="resources/lepton-hd.png" height="28"></a>
|
|
||||||
|
|
||||||
<a href="https://app.myshell.ai/bot/z6Bvua/1702636181"><img src="resources/myshell-hd.png" height="28"></a>
|
|
||||||
|
|
||||||
<a href="https://huggingface.co/spaces/myshell-ai/OpenVoice"><img src="resources/huggingface.png" height="32"></a>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
## Common Issues
|
## Common Issues
|
||||||
|
|
||||||
Please see [QnA](QA.md) for common questions and answers. We will regularly update the question and answer list.
|
Please see [QA](docs/QA.md) for common questions and answers. We will regularly update the question and answer list.
|
||||||
|
|
||||||
## Linux Installation
|
## Join Our Community
|
||||||
Clone this repo, and run
|
|
||||||
```
|
|
||||||
conda create -n openvoice python=3.9
|
|
||||||
conda activate openvoice
|
|
||||||
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
|
|
||||||
pip install -r requirements.txt
|
|
||||||
```
|
|
||||||
Download the checkpoint from [here](https://myshell-public-repo-hosting.s3.amazonaws.com/checkpoints_1226.zip) and extract it to the `checkpoints` folder
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
**1. Flexible Voice Style Control.**
|
|
||||||
Please see [`demo_part1.ipynb`](demo_part1.ipynb) for an example usage of how OpenVoice enables flexible style control over the cloned voice.
|
|
||||||
|
|
||||||
**2. Cross-Lingual Voice Cloning.**
|
|
||||||
Please see [`demo_part2.ipynb`](demo_part2.ipynb) for an example for languages seen or unseen in the MSML training set.
|
|
||||||
|
|
||||||
**3. Gradio Demo.**. We provide a minimalist local gradio demo here. We strongly suggest the users to look into `demo_part1.ipynb`, `demo_part2.ipynb` and the [QnA](QA.md) if they run into issues with the gradio demo. Launch a local gradio demo with `python -m openvoice_app --share`.
|
|
||||||
|
|
||||||
**3. Advanced Usage.**
|
|
||||||
The base speaker model can be replaced with any model (in any language and style) that the user prefer. Please use the `se_extractor.get_se` function as demonstrated in the demo to extract the tone color embedding for the new base speaker.
|
|
||||||
|
|
||||||
**4. Tips to Generate Natural Speech.**
|
|
||||||
There are many single or multi-speaker TTS methods that can generate natural speech, and are readily available. By simply replacing the base speaker model with the model you prefer, you can push the speech naturalness to a level you desire.
|
|
||||||
|
|
||||||
## Roadmap
|
|
||||||
|
|
||||||
- [x] Inference code
|
|
||||||
- [x] Tone color converter model
|
|
||||||
- [x] Multi-style base speaker model
|
|
||||||
- [x] Multi-style and multi-lingual demo
|
|
||||||
- [x] Base speaker model in other languages
|
|
||||||
- [x] EN base speaker model with better naturalness
|
|
||||||
|
|
||||||
|
Join our [Discord community](https://discord.gg/myshell) and select the `Developer` role upon joining to gain exclusive access to our developer-only channel! Don't miss out on valuable discussions and collaboration opportunities.
|
||||||
|
|
||||||
## Citation
|
## Citation
|
||||||
```
|
```
|
||||||
@@ -103,6 +54,5 @@ There are many single or multi-speaker TTS methods that can generate natural spe
|
|||||||
## License
|
## License
|
||||||
This repository is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which prohibits commercial usage. **This will be changed to a license that allows Free Commercial usage in the near future.** Stay tuned. For social responsibility and anti-misuse considerations, **MyShell reserves the ability to detect whether an audio is generated by OpenVoice**, no matter whether the watermark is added or not.
|
This repository is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which prohibits commercial usage. **This will be changed to a license that allows Free Commercial usage in the near future.** Stay tuned. For social responsibility and anti-misuse considerations, **MyShell reserves the ability to detect whether an audio is generated by OpenVoice**, no matter whether the watermark is added or not.
|
||||||
|
|
||||||
|
|
||||||
## Acknowledgements
|
## Acknowledgements
|
||||||
This implementation is based on several excellent projects, [TTS](https://github.com/coqui-ai/TTS), [VITS](https://github.com/jaywalnut310/vits), and [VITS2](https://github.com/daniilrobnikov/vits2). Thanks for their awesome work!
|
This implementation is based on several excellent projects, [TTS](https://github.com/coqui-ai/TTS), [VITS](https://github.com/jaywalnut310/vits), and [VITS2](https://github.com/daniilrobnikov/vits2). Thanks for their awesome work!
|
||||||
|
|||||||
56
docs/USAGE.md
Normal file
56
docs/USAGE.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
# Usage
|
||||||
|
|
||||||
|
## Table of Content
|
||||||
|
- [Use in MyShell](): directly use the Instant Voice Clone and TTS services.
|
||||||
|
- [Minimal Demo](): quickly try OpenVoice and do not require high quality.
|
||||||
|
- [Linux Install](): for researchers and developers only.
|
||||||
|
|
||||||
|
## Use in MyShell
|
||||||
|
|
||||||
|
For most users, the most convenient way is to directly use the free TTS and Instant Voice Clone services in MyShell.
|
||||||
|
|
||||||
|
### TTS
|
||||||
|
Go to [https://app.myshell.ai/explore](https://app.myshell.ai/explore) and follow the instructions below:
|
||||||
|
<div align="center">
|
||||||
|
<img src="../resources/tts-guide.png" width="600"/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
### Voice Clone
|
||||||
|
Go to [https://app.myshell.ai/explore](https://app.myshell.ai/explore) and follow the instructions below:
|
||||||
|
<div align="center">
|
||||||
|
<img src="../resources/voice-clone-guide.png" width="600"/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
## Minimal Demo
|
||||||
|
For users who want to quickly try OpenVoice and do not require high quality or stability, click any of the following links:
|
||||||
|
<div align="center">
|
||||||
|
<a href="https://www.lepton.ai/playground/openvoice"><img src="../resources/lepton-hd.png" height="28"></a>
|
||||||
|
|
||||||
|
<a href="https://app.myshell.ai/bot/z6Bvua/1702636181"><img src="../resources/myshell-hd.png" height="28"></a>
|
||||||
|
|
||||||
|
<a href="https://huggingface.co/spaces/myshell-ai/OpenVoice"><img src="../resources/huggingface.png" height="32"></a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
## Linux Install
|
||||||
|
This section is only for developers and researchers who are familiar with Linux, Python and PyTorch. Clone this repo, and run
|
||||||
|
```
|
||||||
|
conda create -n openvoice python=3.9
|
||||||
|
conda activate openvoice
|
||||||
|
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
|
||||||
|
pip install -r requirements.txt
|
||||||
|
```
|
||||||
|
Download the checkpoint from [here](https://myshell-public-repo-hosting.s3.amazonaws.com/checkpoints_1226.zip) and extract it to the `checkpoints` folder
|
||||||
|
|
||||||
|
**1. Flexible Voice Style Control.**
|
||||||
|
Please see [`demo_part1.ipynb`](demo_part1.ipynb) for an example usage of how OpenVoice enables flexible style control over the cloned voice.
|
||||||
|
|
||||||
|
**2. Cross-Lingual Voice Cloning.**
|
||||||
|
Please see [`demo_part2.ipynb`](demo_part2.ipynb) for an example for languages seen or unseen in the MSML training set.
|
||||||
|
|
||||||
|
**3. Gradio Demo.**. We provide a minimalist local gradio demo here. We strongly suggest the users to look into `demo_part1.ipynb`, `demo_part2.ipynb` and the [QnA](QA.md) if they run into issues with the gradio demo. Launch a local gradio demo with `python -m openvoice_app --share`.
|
||||||
|
|
||||||
|
**3. Advanced Usage.**
|
||||||
|
The base speaker model can be replaced with any model (in any language and style) that the user prefer. Please use the `se_extractor.get_se` function as demonstrated in the demo to extract the tone color embedding for the new base speaker.
|
||||||
|
|
||||||
|
**4. Tips to Generate Natural Speech.**
|
||||||
|
There are many single or multi-speaker TTS methods that can generate natural speech, and are readily available. By simply replacing the base speaker model with the model you prefer, you can push the speech naturalness to a level you desire.
|
||||||
@@ -182,7 +182,7 @@ markdown_table_v2 = """
|
|||||||
"""
|
"""
|
||||||
content = """
|
content = """
|
||||||
<div>
|
<div>
|
||||||
<strong>If the generated voice does not sound like the reference voice, please refer to <a href='https://github.com/myshell-ai/OpenVoice/blob/main/QA.md'>this QnA</a>.</strong> <strong>For multi-lingual & cross-lingual examples, please refer to <a href='https://github.com/myshell-ai/OpenVoice/blob/main/demo_part2.ipynb'>this jupyter notebook</a>.</strong>
|
<strong>If the generated voice does not sound like the reference voice, please refer to <a href='https://github.com/myshell-ai/OpenVoice/blob/main/docs/QA.md'>this QnA</a>.</strong> <strong>For multi-lingual & cross-lingual examples, please refer to <a href='https://github.com/myshell-ai/OpenVoice/blob/main/demo_part2.ipynb'>this jupyter notebook</a>.</strong>
|
||||||
This online demo mainly supports <strong>English</strong>. The <em>default</em> style also supports <strong>Chinese</strong>. But OpenVoice can adapt to any other language as long as a base speaker is provided.
|
This online demo mainly supports <strong>English</strong>. The <em>default</em> style also supports <strong>Chinese</strong>. But OpenVoice can adapt to any other language as long as a base speaker is provided.
|
||||||
</div>
|
</div>
|
||||||
"""
|
"""
|
||||||
|
|||||||
BIN
resources/tts-guide.png
Normal file
BIN
resources/tts-guide.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 277 KiB |
BIN
resources/voice-clone-guide.png
Normal file
BIN
resources/voice-clone-guide.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 240 KiB |
Reference in New Issue
Block a user