deepgeek/gradio-webrtc
2
0
Fork 0
mirror of https://github.com/HumanAIGC-Engineering/gradio-webrtc.git synced 2026-02-05 18:09:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

Rebrand to FastRTC (#60)

Browse Source 
* Add code

* add code

* add code

* Rename messages

* rename

* add code

* Add demo

* docs + demos + bug fixes

* add code

* styles

* user guide

* Styles

* Add code

* misc docs updates

* print nit

* whisper + pr

* url for images

* whsiper update

* Fix bugs

* remove demo files

* version number

* Fix pypi readme

* Fix

* demos

* Add llama code editor

* Update llama code editor and object detection cookbook

* Add more cookbook demos

* add code

* Fix links for PR deploys

* add code

* Fix the install

* add tts

* TTS docs

* Typo

* Pending bubbles for reply on pause

* Stream redesign (#63)

* better error handling

* Websocket error handling

* add code

---------

Co-authored-by: Freddy Boulton <freddyboulton@hf-freddy.local>

* remove docs from dist

* Some docs typos

* more typos

* upload changes + docs

* docs

* better phone

* update docs

* add code

* Make demos better

* fix docs + websocket start_up

* remove mention of FastAPI app

* fastphone tweaks

* add code

* ReplyOnStopWord fixes

* Fix cookbook

* Fix pypi readme

* add code

* bump versions

* sambanova cookbook

* Fix tags

* Llm voice chat

* kyutai tag

* Add error message to all index.html

* STT module uses Moonshine

* Not required from typing extensions

* fix llm voice chat

* Add vpn warning

* demo fixes

* demos

* Add more ui args and gemini audio-video

* update cookbook

* version 9

---------

Co-authored-by: Freddy Boulton <freddyboulton@hf-freddy.local>
This commit is contained in:
Freddy Boulton
2025-02-24 01:13:42 -05:00
committed by GitHub
parent 36190066ec
commit 853d6a06b5
131 changed files with 12349 additions and 4741 deletions
Download Patch File Download Diff File Expand all files Collapse all files

71
docs/advanced-configuration.md
View File

@@ -1,5 +1,9 @@
Any of the parameters for the `Stream` class can be passed to the [`WebRTC`](../userguide/gradio) component directly.
## Track Constraints
You can specify the `track_constraints` parameter to control how the data is streamed to the server. The full documentation on track constraints is [here](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints#constraints).
For example, you can control the size of the frames captured from the webcam like so:
@@ -10,21 +14,22 @@ track_constraints = {
"height": {"exact": 500},
"frameRate": {"ideal": 30},
}
webrtc = WebRTC(track_constraints=track_constraints,
modality="video",
mode="send-receive")
webrtc = Stream(
handler=...,
track_constraints=track_constraints,
modality="video",
mode="send-receive")
```
!!! warning
WebRTC may not enforce your constaints. For example, it may rescale your video
(while keeping the same resolution) in order to maintain the desired (or reach a better) frame rate. If you
really want to enforce height, width and resolution constraints, use the `rtp_params` parameter as set `"degradationPreference": "maintain-resolution"`.
WebRTC may not enforce your constraints. For example, it may rescale your video
(while keeping the same resolution) in order to maintain the desired frame rate (or reach a better one). If you really want to enforce height, width and resolution constraints, use the `rtp_params` parameter as set `"degradationPreference": "maintain-resolution"`.
```python
image = WebRTC(
label="Stream",
image = Stream(
modality="video",
mode="send",
track_constraints=track_constraints,
rtp_params={"degradationPreference": "maintain-resolution"}
@@ -36,7 +41,8 @@ webrtc = WebRTC(track_constraints=track_constraints,
You can configure how the connection is created on the client by passing an `rtc_configuration` parameter to the `WebRTC` component constructor.
See the list of available arguments [here](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/RTCPeerConnection#configuration).
When deploying on a remote server, an `rtc_configuration` parameter must be passed in. See [Deployment](/deployment).
!!! warning
When deploying on a remote server, the `rtc_configuration` parameter must be passed in. See [Deployment](../deployment).
## Reply on Pause Voice-Activity-Detection
@@ -50,19 +56,18 @@ The `ReplyOnPause` class runs a Voice Activity Detection (VAD) algorithm to dete
The following parameters control this argument:
```python
from gradio_webrtc import AlgoOptions, ReplyOnPause, WebRTC
from fastrtc import AlgoOptions, ReplyOnPause, Stream
options = AlgoOptions(audio_chunk_duration=0.6, # (1)
started_talking_threshold=0.2, # (2)
speech_threshold=0.1, # (3)
)
with gr.Blocks as demo:
audio = WebRTC(...)
audio.stream(ReplyOnPause(..., algo_options=algo_options)
)
demo.launch()
Stream(
handler=ReplyOnPause(..., algo_options=algo_options),
modality="audio",
mode="send-receive"
)
```
1. This is the length (in seconds) of audio chunks.
@@ -75,14 +80,13 @@ demo.launch()
You can configure the sampling rate of the audio passed to the `ReplyOnPause` or `StreamHandler` instance with the `input_sampling_rate` parameter. The current default is `48000`
```python
from gradio_webrtc import ReplyOnPause, WebRTC
from fastrtc import ReplyOnPause, Stream
with gr.Blocks as demo:
audio = WebRTC(...)
audio.stream(ReplyOnPause(..., input_sampling_rate=24000)
)
demo.launch()
stream = Stream(
handler=ReplyOnPause(..., input_sampling_rate=24000),
modality="audio",
mode="send-receive"
)
```
@@ -94,14 +98,13 @@ with the `output_sample_rate` and `output_frame_size` parameters.
The following code (which uses the default values of these parameters), states that each output chunk will be a frame of 960 samples at a frame rate of `24,000` hz. So it will correspond to `0.04` seconds.
```python
from gradio_webrtc import ReplyOnPause, WebRTC
from fastrtc import ReplyOnPause, Stream
with gr.Blocks as demo:
audio = WebRTC(...)
audio.stream(ReplyOnPause(..., output_sample_rate=24000, output_frame_size=960)
)
demo.launch()
stream = Stream(
handler=ReplyOnPause(..., output_sample_rate=24000, output_frame_size=960),
modality="audio",
mode="send-receive"
)
```
!!! tip
@@ -117,6 +120,10 @@ Pass any local path or url to an image (svg, png, jpeg) to the components `icon`
You can control the button color and pulse color with `icon_button_color` and `pulse_color` parameters. They can take any valid css color.
!!! warning
The `icon` parameter is only supported in the `WebRTC` component.
=== "Code"
``` python
audio = WebRTC(
@@ -148,6 +155,10 @@ You can control the button color and pulse color with `icon_button_color` and `p
You can supply a `button_labels` dictionary to change the text displayed in the `Start`, `Stop` and `Waiting` buttons that are displayed in the UI.
The keys must be `"start"`, `"stop"`, and `"waiting"`.
!!! warning
The `button_labels` parameter is only supported in the `WebRTC` component.
``` python
webrtc = WebRTC(
label="Video Chat",