Cascaded STST

Demo for cascaded speech-to-speech translation (STST), mapping from source speech in any language to target speech in Hindi.

Demo uses OpenAI's Whisper Base model for speech translation to English, then MarianMT's opus-mt-en-hi model for translation to Hindi, and finally microsoft/speechT5 fine-tuned for Hindi on IndicTTS dataset for text-to-speech. SpeechT5 TTS model for text-to-speech:

NOTE: The goal is not to generate perfect Hindi speech or translation, but to demonstrate the cascaded STST approach using multiple models.

The model might give poor result for very short sentences (1-2 words or so). Try to send longer audio in that case.

Cascaded STST

Demo for cascaded speech-to-speech translation (STST), mapping from source speech in any language to target speech in Hindi.

Demo uses OpenAI's Whisper Base model for speech translation to English, then MarianMT's opus-mt-en-hi model for translation to Hindi, and finally microsoft/speechT5 fine-tuned for Hindi on IndicTTS dataset for text-to-speech. SpeechT5 TTS model for text-to-speech:

NOTE: The goal is not to generate perfect Hindi speech or translation, but to demonstrate the cascaded STST approach using multiple models.

The model might give poor result for very short sentences (1-2 words or so). Try to send longer audio in that case.