SOTAVerified

Speech-to-Speech Translation

Speech-to-speech translation (S2ST) consists on translating speech from one language to speech in another language. This can be done with a cascade of automatic speech recognition (ASR), text-to-text machine translation (MT), and text-to-speech (TTS) synthesis sub-systems, which is text-centric. Recently, works on S2ST without relying on intermediate text representation is emerging.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–10 of 117 papers

TitleStatusHype
Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs—0
S2ST-Omni: An Efficient and Scalable Multilingual Speech-to-Speech Translation Framework via Seamless Speech-Text Alignment and Streaming Speech Generation—0
Phi-Omni-ST: A multimodal language model for direct speech-to-speech translation—0
Dub-S2ST: Textless Speech-to-Speech Translation for Seamless Dubbing—0
Leveraging Unit Language Guidance to Advance Speech Modeling in Textless Speech-to-Speech TranslationCode0
Language translation, and change of accent for speech-to-speech task using diffusion model—0
SimulS2S-LLM: Unlocking Simultaneous Inference of Speech LLMs for Speech-to-Speech Translation—0
Using Phonemes in cascaded S2S translation pipelineCode0
Direct Speech to Speech Translation: A Review—0
Connecting Voices: LoReSpeech as a Low-Resource Speech Parallel Corpus—0
Show:102550
← PrevPage 1 of 12Next →
All datasetsTATFLEURS X-engCVSS

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hokkien→En (Two-pass decoding)ASR-BLEU (Dev)13.6—Unverified
2Hokkien→En (Two-stage)ASR-BLEU (Dev)12.5—Unverified
3Hokkien→En (Three-stage)ASR-BLEU (Dev)12.5—Unverified
4Hokkien→En (Single-pass decoding)ASR-BLEU (Dev)8.8—Unverified
5En→Hokkien (Two-pass decoding)ASR-BLEU (Dev)7.8—Unverified
6En→Hokkien (Three-stage)ASR-BLEU (Dev)7.5—Unverified
7En→Hokkien (Two-stage)ASR-BLEU (Dev)7.1—Unverified
8En→Hokkien (Single-pass decoding)ASR-BLEU (Dev)6.6—Unverified