gradio-webrtc/docs/speech_to_text_gallery.md

<style> .tag-button { cursor: pointer; opacity: 0.5; transition: opacity 0.2s ease; } .tag-button > code { color: var(--supernova); } .tag-button.active { opacity: 1; } </style>

A collection of Speech-to-Text models ready to use with FastRTC. Click on the tags below to find the STT model you're looking for!

!!! tip "Note" The model you want to use does not have to be in the gallery. This is just a collection of models with a common interface that are easy to "plug and play" into your FastRTC app. But You can use any model you want without having to do any special setup. Simply use it from your stream handler!

pytorch

<script> function filterCards() { const activeButtons = document.querySelectorAll('.tag-button.active'); const selectedTags = Array.from(activeButtons).map(button => button.getAttribute('data-tag')); const cards = document.querySelectorAll('.grid.cards > ul > li > p[data-tags]'); cards.forEach(card => { const cardTags = card.getAttribute('data-tags').split(','); const shouldShow = selectedTags.length === 0 || selectedTags.some(tag => cardTags.includes(tag)); card.parentElement.style.display = shouldShow ? 'block' : 'none'; }); } document.querySelectorAll('.tag-button').forEach(button => { button.addEventListener('click', () => { button.classList.toggle('active'); filterCards(); }); }); </script>

• 🗣️{ .lg .middle }👀{ .lg .middle } distil-whisper-FastRTC {: data-tags="pytorch"}

  ---

  Description: Distil-whisper from Hugging Face wraped in a pypi package for plug and play!

  Install Instructions

  pip install distil-whisper-fastrtc
  

  Use it the same way you would the native fastRTC TTS model!

  :octicons-arrow-right-24: Demo

  :octicons-code-16: Repository

• 🗣️{ .lg .middle }👀{ .lg .middle } Your STT Model {: data-tags="pytorch"}

  ---

  Description

  Install Instructions

  Usage

  [:octicons-arrow-right-24: Demo](Your demo here)

  [:octicons-code-16: Repository](Code here)

How to add your own STT model

1. Your model can be implemented in any framework you want but it must implement the STTModel protocol.

   class STTModel(Protocol):
       def stt(self, audio: tuple[int, NDArray[np.int16 | np.float32]]) -> str: ...
   

   • The stt method should take in an audio tuple (sample_rate, audio_array) and return a string of the transcribed text.

   • The audio tuple should be of the form (sample_rate, audio_array) where sample_rate is the sample rate of the audio array and audio_array is a numpy array of the audio data. It can be of type np.int16 or np.float32.

2. Once you have your model implemented, you can use it in your handler!

   from fastrtc import Stream, AdditionalOutputs, ReplyOnPause
   from your_model import YourModel
   
   model = YourModel() # implement the STTModel protocol
   
   def echo(audio):
       text = model.stt(audio)
       yield AdditionalOutputs(text)
   
   stream = Stream(ReplyOnPause(echo), mode="send-receive", modality="audio",
                   additional_outputs=[gr.Textbox(label="Transcription")],
                   additional_outputs_handler=lambda old,new:old + new)
   stream.ui.launch()
   

3. Open a PR to add your model to the gallery! Ideally you model package should be pip installable so other can try it out easily.