禾息
369f3c2c18
Update estimator count retrieval and memory pool limit in CosyVoice
...
- Simplified estimator count retrieval in CosyVoice and CosyVoice2 classes to directly access the configs dictionary.
- Adjusted memory pool limit in the ONNX to TensorRT conversion function from 8GB to 1GB for optimized resource management.
2025-04-16 14:39:06 +08:00
禾息
7f4c9a2c64
Refactor CosyVoice inference methods to streamline CUDA stream management
...
- Removed the queue-based stream pool and integrated direct CUDA stream usage for improved performance.
- Simplified inference methods by eliminating unnecessary synchronization and stream management code.
- Enhanced logging for better tracking of synthesis operations and performance metrics.
- Updated the model class to support CUDA stream context management, ensuring efficient resource utilization during inference.
2025-04-16 14:15:14 +08:00
禾息
fd9b7d45e2
Fix logging indentation in CosyVoice TTS method for improved clarity
2025-04-16 11:24:51 +08:00
禾息
62e04e8856
Enhance CosyVoice with CUDA stream management and estimator handling
...
- Introduced a queue-based system for managing CUDA streams to improve inference performance.
- Updated inference methods to utilize CUDA streams for asynchronous processing.
- Added an EstimatorWrapper class to manage TensorRT estimators, allowing for efficient execution context handling.
- Modified model loading functions to support estimator count configuration.
- Improved logging and performance tracking during inference operations.
2025-04-16 11:16:28 +08:00
雾聪
96950745a6
Revert "mv AsyncLLMEngine init to CosyVoice2"
...
This reverts commit 9b3f351496
2025-03-21 16:17:35 +08:00
雾聪
9b3f351496
mv AsyncLLMEngine init to CosyVoice2
2025-03-21 10:24:04 +08:00
qihua
c0f6a474f3
fix(async_cosyvoice): 恢复原本文本令牌处理逻辑
...
- 在 Frontend 中，恢复原本逐个生成文本令牌
- 在 Model 类中，移除了不必要的日志信息和断言，简化了文本令牌的处理流程
2025-03-08 16:03:35 +08:00
qihua
ab5b8eb160
refactor(llm): 重构 vLLM 推理任务处理方式，支持多任务处理
...
- 移除任务队列和单任务处理限制
- 使用 asyncio.run_coroutine_threadsafe() 在后台线程中运行推理任务
2025-03-08 10:41:49 +08:00
qihua
a1314e573a
chore: 新增 requirements_vllm.txt 文件，指定VLLM 模型所需的依赖
2025-03-08 00:40:17 +08:00
qihua
2fbeba50ae
refactor(llm): 移除未使用的异步推理方法
...
- 删除了 LLM 类中的 async_llm_inference 方法
- 该方法尚未使用，且再在loop_thread之外运行后会导致 vllm 崩溃，因此将其移除
2025-03-08 00:04:01 +08:00
qihua
d4d187bd8c
refactor(llm): 重构 VLLM 推理方式
...
- 新增基于队列和线程的异步推理机制
- 优化同步推理接口，使用新机制实现
2025-03-07 23:53:50 +08:00
qihua
90b666ea20
初步合并vllm支持，异步推理的通道处理还存在bug
2025-03-07 20:26:19 +08:00
huzetao.hzt
69518b2bde
fix bistream extra token
2025-01-23 19:08:18 +08:00
lyuxiang.lx
190840b8dc
fix pitch computation
2025-01-23 15:44:03 +08:00
lyuxiang.lx
07e477519b
add llm bistream
2025-01-23 10:12:06 +08:00
lyuxiang.lx
0b75c3a03f
update
2025-01-14 22:55:13 +08:00
lyuxiang.lx
43f9e9ab20
update
2025-01-13 10:30:13 +08:00
lyuxiang.lx
87475ccf41
fix conflict
2025-01-10 16:43:31 +08:00
lyuxiang.lx
b95f18909e
add empty cache
2025-01-10 14:14:32 +08:00
lyuxiang.lx
1cfc5dd077
add online trt export
2025-01-10 13:55:05 +08:00
Xiang Lyu
92f1c659b9
Merge branch 'dev/lyuxiang.lx' into main
2025-01-08 11:21:06 +08:00
lyuxiang.lx
1e52c6071e
update gradio
2025-01-08 11:11:16 +08:00
huzetao.hzt
b6a1116d15
support online onnx to trt conversion
2025-01-07 17:20:06 +08:00
lyuxiang.lx
99ab0f4fcb
fix lint
2025-01-02 12:32:43 +08:00
lyuxiang.lx
77d8cf13a3
update
2025-01-02 10:55:59 +08:00
lyuxiang.lx
d3b1a8e352
update
2024-12-30 17:40:39 +08:00
lyuxiang.lx
b9ddcba5fd
add some instruction and assert
2024-12-30 16:41:57 +08:00
Xiang Lyu
5aa3a46d96
Merge pull request #710 from 0xCAFEBABE0/bug_cpu_hang
...
fix(bug).when generating text that contains only punctuation marks or…
2024-12-30 10:55:56 +08:00
0xCAFEBABE0
b60c37b31a
fix(bug).when generating text that contains only punctuation marks or whitespace characters, the CPU usage reaches 100%, and the process crashes.
2024-12-30 10:48:43 +08:00
Xiang Lyu
0f6ff298dd
Merge pull request #745 from garywill/patch-2
...
Rename misspelled list_avaliable_spks() to list_available_spks()
2024-12-29 17:20:17 +08:00
刘悦
178f4bbaf9
Update frontend.py
2024-12-19 21:30:20 +08:00
刘悦
5627adefb1
Update frontend.py
...
# NOTE(xcsong): 和默认参数不一致时，必须重新构图，要重新构图请务必指定 `overwrite_cache=True`
# When the parameters differ from the defaults, it is mandatory to re-compose. To re-compose, please ensure you specify `overwrite_cache=True`.
https://github.com/wenet-e2e/WeTextProcessing
2024-12-19 21:26:30 +08:00
garywill
bd4be3fc05
rename list_avaliable_spks() to list_available_spks()
...
Signed-off-by: garywill <garywill@disroot.org >
2024-12-18 11:19:43 +08:00
Xiang Lyu
0d1e562f1d
Merge pull request #736 from FunAudioLLM/dev/lyuxiang.lx
...
fix cosyvoice1.0 bug
2024-12-17 16:06:41 +08:00
lyuxiang.lx
b00d8a073c
fix cosyvoice1.0 bug
2024-12-17 16:05:37 +08:00
Xiang Lyu
8a88446858
Merge pull request #735 from FunAudioLLM/dev/lyuxiang.lx
...
add text_frontend arg
2024-12-17 14:04:45 +08:00
lyuxiang.lx
26c774098d
add text_frontend arg
2024-12-17 14:03:35 +08:00
Xiang Lyu
81edc83648
Merge pull request #728 from FunAudioLLM/dev/lyuxiang.lx
...
update
2024-12-16 15:34:26 +08:00
lyuxiang.lx
60b0416229
update
2024-12-16 15:32:30 +08:00
Xiang Lyu
32e6684025
Merge pull request #725 from FunAudioLLM/dev/lyuxiang.lx
...
add trt bash script
2024-12-16 14:39:24 +08:00
lyuxiang.lx
8ec41faf91
add trt bash script
2024-12-16 14:35:41 +08:00
Xiang Lyu
091e5c4ed8
Merge pull request #721 from FunAudioLLM/dev/lyuxiang.lx
...
update readme
2024-12-16 14:06:03 +08:00
lyuxiang.lx
1298d90e48
update readme
2024-12-16 14:05:00 +08:00
0xCAFEBABE0
bcc58cb4cb
Update common.py
2024-12-16 13:57:38 +08:00
0xCAFEBABE0
1d8d94de82
Update common.py
2024-12-16 13:56:28 +08:00
0xCAFEBABE0
0993ec5f08
Merge branch 'main' into bug_cpu_hang
2024-12-16 13:54:53 +08:00
Xiang Lyu
c4688b68eb
Merge pull request #719 from FunAudioLLM/dev/lyuxiang.lx
...
add instruct usage
2024-12-16 11:16:33 +08:00
lyuxiang.lx
d43a0171d4
add instruct usage
2024-12-16 11:15:51 +08:00
Xiang Lyu
c4c8050532
Merge pull request #717 from FunAudioLLM/dev/lyuxiang.lx
...
fix lint
2024-12-16 10:38:31 +08:00
lyuxiang.lx
3581caec76
fix lint
2024-12-16 10:37:10 +08:00
