55 Commits

Author	SHA1	Message	Date
禾息	7f4c9a2c64	Refactor CosyVoice inference methods to streamline CUDA stream management ... - Removed the queue-based stream pool and integrated direct CUDA stream usage for improved performance. - Simplified inference methods by eliminating unnecessary synchronization and stream management code. - Enhanced logging for better tracking of synthesis operations and performance metrics. - Updated the model class to support CUDA stream context management, ensuring efficient resource utilization during inference.	2025-04-16 14:15:14 +08:00
禾息	62e04e8856	Enhance CosyVoice with CUDA stream management and estimator handling ... - Introduced a queue-based system for managing CUDA streams to improve inference performance. - Updated inference methods to utilize CUDA streams for asynchronous processing. - Added an EstimatorWrapper class to manage TensorRT estimators, allowing for efficient execution context handling. - Modified model loading functions to support estimator count configuration. - Improved logging and performance tracking during inference operations.	2025-04-16 11:16:28 +08:00
qihua	90b666ea20	初步合并vllm支持，异步推理的通道处理还存在bug	2025-03-07 20:26:19 +08:00
lyuxiang.lx	07e477519b	add llm bistream	2025-01-23 10:12:06 +08:00
lyuxiang.lx	0b75c3a03f	update	2025-01-14 22:55:13 +08:00
lyuxiang.lx	b95f18909e	add empty cache	2025-01-10 14:14:32 +08:00
lyuxiang.lx	1cfc5dd077	add online trt export	2025-01-10 13:55:05 +08:00
huzetao.hzt	b6a1116d15	support online onnx to trt conversion	2025-01-07 17:20:06 +08:00
lyuxiang.lx	77d8cf13a3	update	2025-01-02 10:55:59 +08:00
lyuxiang.lx	b9ddcba5fd	add some instruction and assert	2024-12-30 16:41:57 +08:00
Xiang Lyu	0d1e562f1d	Merge pull request #736 from FunAudioLLM/dev/lyuxiang.lx ... fix cosyvoice1.0 bug	2024-12-17 16:06:41 +08:00
lyuxiang.lx	b00d8a073c	fix cosyvoice1.0 bug	2024-12-17 16:05:37 +08:00
Xiang Lyu	c4c8050532	Merge pull request #717 from FunAudioLLM/dev/lyuxiang.lx ... fix lint	2024-12-16 10:38:31 +08:00
lyuxiang.lx	3581caec76	fix lint	2024-12-16 10:37:10 +08:00
Xiang Lyu	94d6ce1006	Merge pull request #715 from FunAudioLLM/dev/lyuxiang.lx ... Dev/lyuxiang.lx	2024-12-16 09:55:27 +08:00
lyuxiang.lx	ac70560364	fix lint	2024-12-16 09:54:24 +08:00
lyuxiang.lx	2511a49a72	update	2024-12-12 18:48:25 +08:00
lyuxiang.lx	c693039d14	update	2024-12-12 16:46:28 +08:00
lyuxiang.lx	3e381002d7	add cosyvoice2	2024-12-11 16:14:19 +08:00
boostarea	3411e1f599	feat.release flow_cache_dict to prevent potential memory leaks in long-running processes.	2024-10-28 12:02:07 +08:00
Tao Liu	18b9a8c844	Update model.py ... The torch.tensor() function does not have a dim parameter	2024-10-17 11:53:09 +08:00
lyuxiang.lx	a4db3db8ed	update flow cache	2024-10-16 15:24:47 +08:00
Xiang Lyu	ace734def8	Merge pull request #455 from boji123/bj_dev_stream_fix_promptcache ... [debug] support flow cache, for sharper tts_mel output (handle prompt bug)	2024-10-16 14:12:40 +08:00
lyuxiang.lx	6b7286eb62	fix typo	2024-10-16 13:57:27 +08:00
lyuxiang.lx	7e6d60c24c	fix bug	2024-10-16 13:30:13 +08:00
lyuxiang.lx	789ee9e5e7	add hifigan train	2024-10-16 11:37:32 +08:00
boji123	c9acce1482	[debug] support flow cache, for sharper tts_mel output	2024-09-29 16:26:11 +08:00
lyuxiang.lx	ffa28e3bbd	update token args	2024-09-29 10:35:10 +08:00
lyuxiang.lx	ba3d9693da	load jit to device	2024-09-26 14:55:03 +08:00
lyuxiang.lx	06934c38c7	update vc code	2024-09-26 14:46:24 +08:00
lyuxiang.lx	72b89a52fb	update vc/tts code	2024-09-26 11:53:10 +08:00
lyuxiang.lx	f65eca6723	add speech fade in out	2024-09-19 18:02:42 +08:00
Xiang Lyu	cd26f11859	Merge pull request #379 from boji123/bj_dev_stream_fix ... [debug] fix badcase, add fade on speech output	2024-09-19 17:32:04 +08:00
lyuxiang.lx	e19e80fcd8	update tempo change	2024-09-18 16:13:40 +08:00
liubaiji	9e0b99e48e	[feature] fix badcase, add fade on speech output	2024-09-11 10:41:50 +08:00
lyuxiang.lx	122df8c420	set onnx to false as last chunk rtf unstable	2024-09-06 17:10:54 +08:00
lyuxiang.lx	90433f5373	fix lint	2024-09-05 16:15:34 +08:00
lyuxiang.lx	2ce724045b	add onnx export	2024-09-04 18:15:33 +08:00
禾息	752103a307	Merge remote-tracking branch 'origin/inference_streaming' into inference_streaming	2024-09-03 11:13:25 +08:00
禾息	a801416805	mirror modify	2024-09-03 11:07:47 +08:00
禾息	fadb22086f	export onnx	2024-09-03 11:06:24 +08:00
Xiang Lyu	ee988420f3	Merge branch 'inference_streaming' into flow_tensorrt	2024-08-30 14:20:06 +08:00
禾息	18599be8d5	mirror modify	2024-08-30 14:15:24 +08:00
zhoubofan.zbf	29408360fb	fix bug	2024-08-30 13:43:54 +08:00
禾息	6e7f5b922a	update	2024-08-30 13:14:44 +08:00
lyuxiang.lx	1ab3186799	revert trt TODO	2024-08-29 23:35:19 +08:00
zhoubofan.zbf	5f21aef786	add flow decoder tensorrt infer	2024-08-29 23:35:07 +08:00
lyuxiang.lx	1d881df8b2	fix vocoder speech overlap	2024-08-29 19:10:08 +08:00
lyuxiang.lx	f1e374a9bb	add trt script TODO	2024-08-29 10:44:04 +08:00
lyuxiang.lx	9ab298dd49	add llm export script	2024-08-28 18:21:11 +08:00