mirror of
https://github.com/HumanAIGC-Engineering/gradio-webrtc.git
synced 2026-02-05 09:59:22 +08:00
Cloudflare turn integration (#264)
* Turn integration * Add code: * type hint * Fix typehint * add code * format * WIP * trickle ice * bump version * Better docs * Modify * code * Mute icon for whisper * Add code * llama 4 demo * code * OpenAI interruptions * fix docs
This commit is contained in:
Freddy Boulton
GitHub
@@ -3,27 +3,75 @@ When deploying in cloud environments with firewalls (like Hugging Face Spaces, R
|
||||
!!! tip
|
||||
The `rtc_configuration` parameter of the `Stream` class also be passed to the [`WebRTC`](../userguide/gradio) component directly if you're building a standalone gradio app.
|
||||
|
||||
## Community Server
|
||||
|
||||
Hugging Face graciously provides a TURN server for the community.
|
||||
## Cloudflare Calls API
|
||||
|
||||
Cloudflare also offers a managed TURN server with [Cloudflare Calls](https://www.cloudflare.com/en-au/developer-platform/products/cloudflare-calls/).
|
||||
|
||||
### With a Hugging Face Token
|
||||
|
||||
Cloudflare and Hugging Face have partnered to allow you to stream 10gb of WebRTC traffic per month for free with a Hugging Face account!
|
||||
|
||||
```python
|
||||
from fastrtc import Stream, get_cloudflare_turn_credentials_async
|
||||
|
||||
# Make sure the HF_TOKEN environment variable is set
|
||||
# Or pass in a callable with all arguments set
|
||||
|
||||
# make sure you don't commit your token to git!
|
||||
TOKEN = "hf_..."
|
||||
async def get_credentials():
|
||||
return await get_cloudflare_turn_credentials_async(hf_token=TOKEN)
|
||||
|
||||
stream = Stream(
|
||||
handler=...,
|
||||
rtc_configuration=get_credentials,
|
||||
modality="audio",
|
||||
mode="send-receive",
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
### With a Cloudflare API Token
|
||||
|
||||
Once you have exhausted your monthly quota, you can create a **free** Cloudflare account.
|
||||
|
||||
Create an [account](https://developers.cloudflare.com/fundamentals/setup/account/create-account/) and head to the [Calls section in your dashboard](https://dash.cloudflare.com/?to=/:account/calls).
|
||||
|
||||
Choose `Create -> TURN App`, give it a name (like `fastrtc-demo`), and then hit the Create button.
|
||||
|
||||
Take note of the Turn Token ID (often exported as `TURN_KEY_ID`) and API Token (exported as `TURN_KEY_API_TOKEN`).
|
||||
|
||||
You can then connect from the WebRTC component like so:
|
||||
|
||||
```python
|
||||
from fastrtc import Stream, get_cloudflare_turn_credentials_async
|
||||
|
||||
# Make sure the TURN_KEY_ID and TURN_KEY_API_TOKEN environment variables are set
|
||||
stream = Stream(
|
||||
handler=...,
|
||||
rtc_configuration=get_cloudflare_turn_credentials_async,
|
||||
modality="audio",
|
||||
mode="send-receive",
|
||||
)
|
||||
```
|
||||
|
||||
## Community Server (Deprecated)
|
||||
|
||||
Hugging Face graciously provides 10gb of TURN traffic through Cloudflare's global network.
|
||||
In order to use it, you need to first create a Hugging Face account by going to [huggingface.co](https://huggingface.co/).
|
||||
|
||||
Then navigate to this [space](https://huggingface.co/spaces/fastrtc/turn-server-login) and follow the instructions on the page. You just have to click the "Log in" button and then the "Sign Up" button.
|
||||
|
||||
|
||||
Then you can create an [access token](https://huggingface.co/docs/hub/en/security-tokens).
|
||||
|
||||
Then you can use the `get_hf_turn_credentials` helper to get your credentials:
|
||||
|
||||
```python
|
||||
from fastrtc import get_hf_turn_credentials, Stream
|
||||
|
||||
# Pass a valid access token for your Hugging Face account
|
||||
# or set the HF_TOKEN environment variable
|
||||
credentials = get_hf_turn_credentials(token=None)
|
||||
# Make sure the HF_TOKEN environment variable is set
|
||||
|
||||
Stream(
|
||||
handler=...,
|
||||
rtc_configuration=credentials,
|
||||
rtc_configuration=get_hf_turn_credentials,
|
||||
modality="audio",
|
||||
mode="send-receive"
|
||||
)
|
||||
@@ -31,8 +79,7 @@ Stream(
|
||||
|
||||
!!! warning
|
||||
|
||||
This is a shared resource so we make no latency/availability guarantees.
|
||||
For more robust options, see the Twilio, Cloudflare and self-hosting options below.
|
||||
This function is now deprecated. Please use `get_cloudflare_turn_credentials` instead.
|
||||
|
||||
|
||||
## Twilio API
|
||||
@@ -78,50 +125,6 @@ Stream(
|
||||
rtc_configuration = get_twilio_turn_credentials()
|
||||
```
|
||||
|
||||
## Cloudflare Calls API
|
||||
|
||||
Cloudflare also offers a managed TURN server with [Cloudflare Calls](https://www.cloudflare.com/en-au/developer-platform/products/cloudflare-calls/).
|
||||
|
||||
Create a **free** [account](https://developers.cloudflare.com/fundamentals/setup/account/create-account/) and head to the [Calls section in your dashboard](https://dash.cloudflare.com/?to=/:account/calls).
|
||||
|
||||
Choose `Create -> TURN App`, give it a name (like `fastrtc-demo`), and then hit the Create button.
|
||||
|
||||
Take note of the Turn Token ID (often exported as `TURN_KEY_ID`) and API Token (exported as `TURN_KEY_API_TOKEN`).
|
||||
|
||||
You can then connect from the WebRTC component like so:
|
||||
|
||||
```python
|
||||
from fastrtc import Stream
|
||||
import requests
|
||||
import os
|
||||
|
||||
turn_key_id = os.environ.get("TURN_KEY_ID")
|
||||
turn_key_api_token = os.environ.get("TURN_KEY_API_TOKEN")
|
||||
ttl = 86400 # Can modify TTL, here it's set to 24 hours
|
||||
|
||||
response = requests.post(
|
||||
f"https://rtc.live.cloudflare.com/v1/turn/keys/{turn_key_id}/credentials/generate-ice-servers",
|
||||
headers={
|
||||
"Authorization": f"Bearer {turn_key_api_token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
json={"ttl": ttl},
|
||||
)
|
||||
if response.ok:
|
||||
rtc_configuration = response.json()
|
||||
else:
|
||||
raise Exception(
|
||||
f"Failed to get TURN credentials: {response.status_code} {response.text}"
|
||||
)
|
||||
|
||||
stream = Stream(
|
||||
handler=...,
|
||||
rtc_configuration=rtc_configuration,
|
||||
modality="audio",
|
||||
mode="send-receive",
|
||||
)
|
||||
```
|
||||
|
||||
## Self Hosting
|
||||
|
||||
We have developed a script that can automatically deploy a TURN server to Amazon Web Services (AWS). You can follow the instructions [here](https://github.com/freddyaboulton/turn-server-deploy) or this guide.
|
||||
|
||||
267
docs/reference/credentials.md
Normal file
267
docs/reference/credentials.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# TURN Credential Utils
|
||||
|
||||
## `get_turn_credentials_async`
|
||||
|
||||
```python
|
||||
async def get_turn_credentials_async(
|
||||
method: Literal["hf", "twilio", "cloudflare"] = "cloudflare",
|
||||
**kwargs
|
||||
):
|
||||
```
|
||||
|
||||
Retrieves TURN credentials from the specified provider.
|
||||
This can be passed directly to the Stream class and it will be called for each
|
||||
unique WebRTC connection via the Gradio UI. When mounting to FastAPI, call this function
|
||||
yourself to return the credentials to the frontend client, for example, in the
|
||||
index route, you can call this function and embed the credentials in the source code of the index.html.
|
||||
See the FastRTC spaces at hf.co/fastrtc for an example.
|
||||
|
||||
Acts as a dispatcher function to call the appropriate credential retrieval
|
||||
function based on the method specified.
|
||||
|
||||
Args:
|
||||
```
|
||||
method: Literal["hf", "twilio", "cloudflare"] | None
|
||||
The provider to use. 'hf' uses the deprecated Hugging Face endpoint.
|
||||
'cloudflare' uses either Cloudflare keys or the HF endpoint.
|
||||
'twilio' uses the Twilio API. Defaults to "cloudflare".
|
||||
**kwargs:
|
||||
Additional keyword arguments passed directly to the underlying
|
||||
provider-specific function (e.g., `token`, `ttl` for 'hf';
|
||||
`twilio_sid`, `twilio_token` for 'twilio'; `turn_key_id`,
|
||||
`turn_key_api_token`, `hf_token`, `ttl` for 'cloudflare').
|
||||
```
|
||||
|
||||
Returns:
|
||||
```
|
||||
dict:
|
||||
A dictionary containing the TURN credentials from the chosen provider.
|
||||
```
|
||||
|
||||
Raises:
|
||||
```
|
||||
ValueError:
|
||||
If an invalid method is specified.
|
||||
Also raises exceptions from the underlying provider functions (see their
|
||||
docstrings).
|
||||
```
|
||||
|
||||
Example
|
||||
```python
|
||||
>>> from fastrtc import get_turn_credentials_async, Stream
|
||||
>>> credentials = await get_turn_credentials_async()
|
||||
>>> print(credentials)
|
||||
>>> # Can pass directly to stream class
|
||||
>>> stream = Stream(..., rtc_configuration=get_turn_credentials_async)
|
||||
```
|
||||
|
||||
## `get_turn_credentials`
|
||||
|
||||
```python
|
||||
def get_turn_credentials(
|
||||
method: Literal["hf", "twilio", "cloudflare"] = "cloudflare",
|
||||
**kwargs
|
||||
):
|
||||
```
|
||||
|
||||
Retrieves TURN credentials from the specified provider.
|
||||
This can be passed directly to the Stream class and it will be called for each
|
||||
unique WebRTC connection via the Gradio UI. When mounting to FastAPI, call this function
|
||||
yourself to return the credentials to the frontend client, for example, in the
|
||||
index route, you can call this function and embed the credentials in the source code of the index.html.
|
||||
See the FastRTC spaces at hf.co/fastrtc for an example.
|
||||
|
||||
Acts as a dispatcher function to call the appropriate credential retrieval
|
||||
function based on the method specified.
|
||||
|
||||
Args:
|
||||
```
|
||||
method: Literal["hf", "twilio", "cloudflare"] | None
|
||||
The provider to use. 'hf' uses the deprecated Hugging Face endpoint.
|
||||
'cloudflare' uses either Cloudflare keys or the HF endpoint.
|
||||
'twilio' uses the Twilio API. Defaults to "cloudflare".
|
||||
**kwargs:
|
||||
Additional keyword arguments passed directly to the underlying
|
||||
provider-specific function (e.g., `token`, `ttl` for 'hf';
|
||||
`twilio_sid`, `twilio_token` for 'twilio'; `turn_key_id`,
|
||||
`turn_key_api_token`, `hf_token`, `ttl` for 'cloudflare').
|
||||
```
|
||||
|
||||
Returns:
|
||||
```
|
||||
dict:
|
||||
A dictionary containing the TURN credentials from the chosen provider.
|
||||
```
|
||||
|
||||
Raises:
|
||||
```
|
||||
ValueError:
|
||||
If an invalid method is specified.
|
||||
Also raises exceptions from the underlying provider functions (see their
|
||||
docstrings).
|
||||
```
|
||||
|
||||
Example
|
||||
```python
|
||||
>>> from fastrtc import get_turn_credentials, Stream
|
||||
>>> credentials = get_turn_credentials()
|
||||
>>> print(credentials)
|
||||
>>> # Can pass directly to stream class
|
||||
>>> stream = Stream(..., rtc_configuration=get_turn_credentials_async)
|
||||
```
|
||||
|
||||
## `get_cloudflare_turn_credentials_async`
|
||||
|
||||
```python
|
||||
async def get_cloudflare_turn_credentials_async(
|
||||
turn_key_id=None,
|
||||
turn_key_api_token=None,
|
||||
hf_token=None,
|
||||
ttl=600,
|
||||
client: httpx.AsyncClient | None = None,
|
||||
):
|
||||
```
|
||||
|
||||
Asynchronously retrieves TURN credentials from Cloudflare or Hugging Face.
|
||||
|
||||
Asynchronously fetches TURN server credentials either directly from Cloudflare
|
||||
using API keys or via the Hugging Face TURN endpoint using an HF token. The HF
|
||||
token method takes precedence if provided.
|
||||
|
||||
Args:
|
||||
```
|
||||
turn_key_id (str, optional):
|
||||
Cloudflare TURN key ID. Defaults to None,
|
||||
in which case the CLOUDFLARE_TURN_KEY_ID environment variable is used.
|
||||
turn_key_api_token (str, optional):
|
||||
Cloudflare TURN key API token.
|
||||
Defaults to None, in which case the CLOUDFLARE_TURN_KEY_API_TOKEN
|
||||
environment variable is used.
|
||||
hf_token (str, optional):
|
||||
Hugging Face API token. If provided, this method
|
||||
is used instead of Cloudflare keys.
|
||||
Defaults to None, in which case the HF_TOKEN environment variable is used.
|
||||
ttl (int, optional): Time-to-live for the credentials in seconds.
|
||||
Defaults to 600.
|
||||
client (httpx.AsyncClient | None, optional): An existing httpx async client
|
||||
to use for the request. If None, a new client is created per request.
|
||||
Defaults to None.
|
||||
```
|
||||
|
||||
Returns:
|
||||
```
|
||||
dict: A dictionary containing the TURN credentials (ICE servers).
|
||||
```
|
||||
|
||||
Raises:
|
||||
```
|
||||
ValueError: If neither HF token nor Cloudflare keys (either as arguments
|
||||
or environment variables) are provided.
|
||||
Exception: If the request to the credential server fails.
|
||||
```
|
||||
|
||||
Example
|
||||
```python
|
||||
>>> from fastrtc import get_cloudflare_turn_crendials_async, Stream
|
||||
>>> credentials = await get_cloudflare_turn_credentials_async()
|
||||
>>> print(credentials)
|
||||
>>> # Can pass directly to stream class
|
||||
>>> stream = Stream(..., rtc_configuration=get_turn_credentials_async)
|
||||
```
|
||||
|
||||
|
||||
## `get_cloudflare_turn_credentials`
|
||||
|
||||
```python
|
||||
def get_cloudflare_turn_credentials(
|
||||
turn_key_id=None,
|
||||
turn_key_api_token=None,
|
||||
hf_token=None,
|
||||
ttl=600,
|
||||
client: httpx.AsyncClient | None = None,
|
||||
):
|
||||
```
|
||||
|
||||
Retrieves TURN credentials from Cloudflare or Hugging Face.
|
||||
|
||||
Fetches TURN server credentials either directly from Cloudflare using API keys
|
||||
or via the Hugging Face TURN endpoint using an HF token. The HF token method
|
||||
takes precedence if provided.
|
||||
|
||||
Args:
|
||||
```
|
||||
turn_key_id (str, optional):
|
||||
Cloudflare TURN key ID. Defaults to None,
|
||||
in which case the CLOUDFLARE_TURN_KEY_ID environment variable is used.
|
||||
turn_key_api_token (str, optional):
|
||||
Cloudflare TURN key API token.
|
||||
Defaults to None, in which case the CLOUDFLARE_TURN_KEY_API_TOKEN
|
||||
environment variable is used.
|
||||
hf_token (str, optional):
|
||||
Hugging Face API token. If provided, this method
|
||||
is used instead of Cloudflare keys.
|
||||
Defaults to None, in which case the HF_TOKEN environment variable is used.
|
||||
ttl (int, optional): Time-to-live for the credentials in seconds.
|
||||
Defaults to 600.
|
||||
client (httpx.AsyncClient | None, optional): An existing httpx async client
|
||||
to use for the request. If None, a new client is created per request.
|
||||
Defaults to None.
|
||||
```
|
||||
|
||||
Returns:
|
||||
```
|
||||
dict: A dictionary containing the TURN credentials (ICE servers).
|
||||
```
|
||||
|
||||
Raises:
|
||||
```
|
||||
ValueError: If neither HF token nor Cloudflare keys (either as arguments
|
||||
or environment variables) are provided.
|
||||
Exception: If the request to the credential server fails.
|
||||
```
|
||||
|
||||
Example
|
||||
```python
|
||||
>>> from fastrtc import get_cloudflare_turn_crendials_async, Stream
|
||||
>>> credentials = await get_cloudflare_turn_credentials_async()
|
||||
>>> print(credentials)
|
||||
>>> # Can pass directly to stream class
|
||||
>>> stream = Stream(..., rtc_configuration=get_turn_credentials_async)
|
||||
```
|
||||
|
||||
## `get_twilio_turn_credentials`
|
||||
|
||||
```python
|
||||
def get_twilio_turn_credentials(
|
||||
twilio_sid=None,
|
||||
twilio_token=None):
|
||||
```
|
||||
|
||||
Retrieves TURN credentials from Twilio.
|
||||
|
||||
Uses the Twilio REST API to generate temporary TURN credentials. Requires
|
||||
the `twilio` package to be installed.
|
||||
|
||||
Args:
|
||||
```
|
||||
twilio_sid (str, optional):
|
||||
Twilio Account SID. Defaults to None, in which
|
||||
case the TWILIO_ACCOUNT_SID environment variable is used.
|
||||
twilio_token (str, optional):
|
||||
Twilio Auth Token. Defaults to None, in which
|
||||
case the TWILIO_AUTH_TOKEN environment variable is used.
|
||||
```
|
||||
Returns:
|
||||
```
|
||||
dict:
|
||||
A dictionary containing the TURN credentials formatted for WebRTC,
|
||||
including 'iceServers' and 'iceTransportPolicy'.
|
||||
```
|
||||
|
||||
Raises:
|
||||
```
|
||||
ImportError: If the `twilio` package is not installed.
|
||||
ValueError: If Twilio credentials (SID and token) are not provided either
|
||||
as arguments or environment variables.
|
||||
TwilioRestException: If the Twilio API request fails.
|
||||
```
|
||||
Reference in New Issue
Block a user