deepgeek/CosyVoice
2
0
Fork 0
mirror of https://github.com/FunAudioLLM/CosyVoice.git synced 2026-02-04 17:39:25 +08:00
Code Issues Packages Projects Releases Wiki Activity
321 Commits 5 Branches 2 Tags
62e04e8856402c90a3af29b24b348c31ab506f46
Commit Graph

321 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
禾息
62e04e8856 Enhance CosyVoice with CUDA stream management and estimator handling 
- Introduced a queue-based system for managing CUDA streams to improve inference performance.
- Updated inference methods to utilize CUDA streams for asynchronous processing.
- Added an EstimatorWrapper class to manage TensorRT estimators, allowing for efficient execution context handling.
- Modified model loading functions to support estimator count configuration.
- Improved logging and performance tracking during inference operations.
 2025-04-16 11:16:28 +08:00
雾聪
96950745a6 Revert "mv AsyncLLMEngine init to CosyVoice2" 
This reverts commit 9b3f351496.
 2025-03-21 16:17:35 +08:00
雾聪
9b3f351496 mv AsyncLLMEngine init to CosyVoice2 2025-03-21 10:24:04 +08:00
Yabin Li
00b454cf30 Merge pull request #1053 from qi-hua/dev/use_vllm 
Dev/use vllm
 2025-03-13 14:22:50 +08:00
qihua
c0f6a474f3 fix(async_cosyvoice): 恢复原本文本令牌处理逻辑 
- 在 Frontend 中，恢复原本逐个生成文本令牌
- 在 Model 类中，移除了不必要的日志信息和断言，简化了文本令牌的处理流程
 2025-03-08 16:03:35 +08:00
qihua
ab5b8eb160 refactor(llm): 重构 vLLM 推理任务处理方式，支持多任务处理 
- 移除任务队列和单任务处理限制
- 使用 asyncio.run_coroutine_threadsafe() 在后台线程中运行推理任务
 2025-03-08 10:41:49 +08:00
qihua
b4fe05d466 docs: 添加speed_test.ipynb文件 
- 新增 speed_test.ipynb 文件，用于测试 CosyVoice2模型的性能
- 包含测试环境配置、默认情况下的使用示例、使用 vllm 加速 LLM 推理的步骤
 2025-03-08 00:41:34 +08:00
qihua
a1314e573a chore: 新增 requirements_vllm.txt 文件，指定VLLM 模型所需的依赖 2025-03-08 00:40:17 +08:00
qihua
2fbeba50ae refactor(llm): 移除未使用的异步推理方法 
- 删除了 LLM 类中的 async_llm_inference 方法
- 该方法尚未使用，且再在loop_thread之外运行后会导致 vllm 崩溃，因此将其移除
 2025-03-08 00:04:01 +08:00
qihua
d4d187bd8c refactor(llm): 重构 VLLM 推理方式 
- 新增基于队列和线程的异步推理机制
- 优化同步推理接口，使用新机制实现
 2025-03-07 23:53:50 +08:00
qihua
90b666ea20 初步合并vllm支持，异步推理的通道处理还存在bug 2025-03-07 20:26:19 +08:00
Xiang Lyu
fd45708e4b Merge pull request #977 from hanasay/main 
Convert audio to mono while extract speech token
 2025-02-16 12:51:04 +08:00
hanasay
296ed4f526 Convert audio to mono while extract speech token 
modified：     tools/extract_speech_token.py
 2025-02-14 15:25:45 +08:00
Xiang Lyu
95e99e0417 Merge pull request #940 from c4fun/fastapi-cosyvoice2 
Add a inference_instruct2 route to support and defaultly supports cosyvoice2 in fastapi server
 2025-02-07 16:30:27 +08:00
c4fun
ba6d8c07ba revert the main function to original and only preserve the inference endpoint 2025-02-07 16:28:30 +08:00
Xiang Lyu
da3f129977 Merge pull request #936 from huyyxy/main 
feat(docker): update CUDA base image to 12.4.1 for TensorRT support
 2025-02-06 11:39:53 +08:00
c4fun
2889c25863 supports and defaultly supports cosyvoice2 in fastapi server 2025-01-27 20:51:57 +08:00
huyyxy
aa65200713 feat(docker): update CUDA base image to 12.4.1 for TensorRT support 
- Upgrade base image from nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 to nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04
- Enable CUDA 12.4 runtime environment
- Ensure TensorRT dependency compatibility
- Validation steps:
  - Verify CUDA version via nvidia-smi after build
  - Test import tensorrt in container without errors

Closes #935
 2025-01-26 12:33:50 +08:00
Xiang Lyu
86e26f54c7 Merge pull request #930 from sd0ric4/main 
feat: add POST endpoints to resolve browser error about GET request wi…
 2025-01-26 10:17:15 +08:00
sd0ric4
f1c214377c fix: add POST endpoints to resolve browser error about GET request with body 2025-01-24 19:41:14 +08:00
Xiang Lyu
369ea80bd4 Merge pull request #926 from Vinkle-hzt/main 
fix bistream extra token
 2025-01-23 22:02:25 +08:00
huzetao.hzt
69518b2bde fix bistream extra token 2025-01-23 19:08:18 +08:00
Xiang Lyu
276cfa02b6 Merge pull request #925 from FunAudioLLM/dev/lyuxiang.lx 
fix pitch computation
 2025-01-23 15:45:42 +08:00
lyuxiang.lx
190840b8dc fix pitch computation 2025-01-23 15:44:03 +08:00
lyuxiang.lx
c6c3f27ecc fix typo 2025-01-23 11:27:10 +08:00
Xiang Lyu
49761d2474 Merge pull request #924 from FunAudioLLM/dev/lyuxiang.lx 
add llm bistream
 2025-01-23 10:19:21 +08:00
lyuxiang.lx
07e477519b add llm bistream 2025-01-23 10:12:06 +08:00
Xiang Lyu
41c5e8cd6d Merge pull request #887 from Wauplin/patch-1 
Fix diffusers / huggingface_hub compatibility in requirements.txt
 2025-01-15 18:23:05 +08:00
Lucain
66ceaff472 Fix diffusers / huggingface_hub compatibility in requirements.txt 
As mentioned in https://github.com/FunAudioLLM/CosyVoice/issues/516#issuecomment-2592067949 and https://github.com/FunAudioLLM/CosyVoice/issues/527#issuecomment-2592067100, it is more future-proof to upgrade `diffusers` version rather than downgrading `huggingface_hub` to an old one. This will also fix the `cannot import name 'cached_download' from 'huggingface_hub'` issue without relying on outdated packages.

Sorry again for the inconvenience 🙏
 2025-01-15 10:21:08 +01:00
Xiang Lyu
07a314767f Merge pull request #884 from FunAudioLLM/dev/lyuxiang.lx 
update
 2025-01-14 22:56:21 +08:00
lyuxiang.lx
0b75c3a03f update 2025-01-14 22:55:13 +08:00
Xiang Lyu
b4dea3d64a Merge pull request #878 from FunAudioLLM/dev/lyuxiang.lx 
update
 2025-01-13 10:31:12 +08:00
lyuxiang.lx
43f9e9ab20 update 2025-01-13 10:30:13 +08:00
Xiang Lyu
025f6f0f7f Merge pull request #875 from lsby/main 
fix docker python version
 2025-01-13 10:27:49 +08:00
Xiang Lyu
69051d11ec Merge pull request #876 from FunAudioLLM/dev/lyuxiang.lx 
fix bug
 2025-01-12 21:21:25 +08:00
lyuxiang.lx
59fa786769 fix bug 2025-01-12 21:18:41 +08:00
hbybyyang
f38f594303 fix docker python version 2025-01-12 15:59:58 +08:00
Xiang Lyu
eb4d5d053f Merge pull request #868 from FunAudioLLM/dev/lyuxiang.lx 
move prompt wav to asset
 2025-01-10 17:53:32 +08:00
lyuxiang.lx
d450c32296 update 2025-01-10 17:52:25 +08:00
lyuxiang.lx
e84d72a4d9 move prompt wav to asset 2025-01-10 17:51:21 +08:00
Xiang Lyu
06e86619c2 Merge pull request #867 from FunAudioLLM/dev/lyuxiang.lx 
Dev/lyuxiang.lx
 2025-01-10 16:46:11 +08:00
lyuxiang.lx
e257c16796 Merge branch 'dev/lyuxiang.lx' of github.com:FunAudioLLM/CosyVoice into dev/lyuxiang.lx 2025-01-10 16:44:01 +08:00
lyuxiang.lx
87475ccf41 fix conflict 2025-01-10 16:43:31 +08:00
Xiang Lyu
8a1bce6c81 Merge pull request #865 from FunAudioLLM/dev/lyuxiang.lx 
Dev/lyuxiang.lx
 2025-01-10 14:18:21 +08:00
Xiang Lyu
b1e966309d Merge branch 'main' into dev/lyuxiang.lx 2025-01-10 14:16:34 +08:00
lyuxiang.lx
b95f18909e add empty cache 2025-01-10 14:14:32 +08:00
lyuxiang.lx
1cfc5dd077 add online trt export 2025-01-10 13:55:05 +08:00
lyuxiang.lx
d2e43fe6f4 update 2025-01-08 17:01:40 +08:00
Xiang Lyu
426c4001ca Merge pull request #842 from Vinkle-hzt/main 
support online onnx to trt conversion
 2025-01-08 11:21:42 +08:00
Xiang Lyu
92f1c659b9 Merge branch 'dev/lyuxiang.lx' into main 2025-01-08 11:21:06 +08:00