This commit is contained in:
lyuxiang.lx
2025-05-30 09:25:26 +00:00
parent 793a7fe6ad
commit 1f50ae259b

View File

@@ -163,6 +163,7 @@ for i, j in enumerate(cosyvoice.inference_zero_shot(text_generator(), '希望你
torchaudio.save('zero_shot_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)
```
#### CosyVoice2 vllm Usage
If you want to use vllm for inference, please install `vllm==v0.9.0`. Older vllm version do not support CosyVoice2 inference.
Notice that `vllm==v0.9.0` has a lot of specific requirements, for example `torch==2.7.0`. You can create a new env to in case your hardward do not support vllm and old env is corrupted.
@@ -173,6 +174,8 @@ pip install vllm==v0.9.0 -i https://mirrors.aliyun.com/pypi/simple/ --trusted-ho
conda activate cosyvoice_vllm
```
Remember to register `CosyVoice2ForCausalLM` for vllm inference at the start of the code.
```python
import sys
sys.path.append('third_party/Matcha-TTS')
@@ -184,7 +187,7 @@ ModelRegistry.register_model("CosyVoice2ForCausalLM", CosyVoice2ForCausalLM)
cosyvoice = CosyVoice2('pretrained_models/CosyVoice2-0.5B', load_jit=False, load_trt=False, load_vllm=True, fp16=False)
```
**CosyVoice Usage**
#### CosyVoice Usage
```python
cosyvoice = CosyVoice('pretrained_models/CosyVoice-300M-SFT', load_jit=False, load_trt=False, fp16=False)
# sft usage