SOTAVerified

Speech-to-Speech Translation

Speech-to-speech translation (S2ST) consists on translating speech from one language to speech in another language. This can be done with a cascade of automatic speech recognition (ASR), text-to-text machine translation (MT), and text-to-speech (TTS) synthesis sub-systems, which is text-centric. Recently, works on S2ST without relying on intermediate text representation is emerging.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 76–100 of 117 papers

TitleStatusHype
Translatotron 3: Speech to Speech Translation with Monolingual Data—0
UWSpeech: Speech to Speech Translation for Unwritten Languages—0
Scheduled Interleaved Speech-Text Training for Speech-to-Speech Translation with LLMs—0
What does it take to get state of the art in simultaneous speech-to-speech translation?—0
A Case Study on Filtering for End-to-End Speech Translation—0
A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translation—0
Analyzing Speech Unit Selection for Textless Speech-to-Speech Translation—0
Assessing Evaluation Metrics for Speech-to-Speech Translation—0
AudioPaLM: A Large Language Model That Can Speak and Listen—0
A Unit-based System and Dataset for Expressive Direct Speech-to-Speech Translation—0
Automatic Extraction of Parallel Speech Corpora from Dubbed Movies—0
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation—0
Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM—0
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data?—0
Connecting Voices: LoReSpeech as a Low-Resource Speech Parallel Corpus—0
Cross-Lingual Machine Speech Chain for Javanese, Sundanese, Balinese, and Bataks Speech Recognition and Synthesis—0
CrossVoice: Crosslingual Prosody Preserving Cascade-S2ST using Transfer Learning—0
DiffS2UT: A Semantic Preserving Diffusion Model for Textless Direct Speech-to-Speech Translation—0
Diffusion Synthesizer for Efficient Multilingual Speech to Speech Translation—0
Direct Punjabi to English speech translation using discrete units—0
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention—0
Direct Speech-to-Speech Neural Machine Translation: A Survey—0
Direct Speech to Speech Translation: A Review—0
Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features—0
Direct Text to Speech Translation System using Acoustic Units—0
Show:102550
← PrevPage 4 of 5Next →
All datasetsTATFLEURS X-engCVSS

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Hokkien→En (Two-pass decoding)ASR-BLEU (Dev)13.6—Unverified
2Hokkien→En (Two-stage)ASR-BLEU (Dev)12.5—Unverified
3Hokkien→En (Three-stage)ASR-BLEU (Dev)12.5—Unverified
4Hokkien→En (Single-pass decoding)ASR-BLEU (Dev)8.8—Unverified
5En→Hokkien (Two-pass decoding)ASR-BLEU (Dev)7.8—Unverified
6En→Hokkien (Three-stage)ASR-BLEU (Dev)7.5—Unverified
7En→Hokkien (Two-stage)ASR-BLEU (Dev)7.1—Unverified
8En→Hokkien (Single-pass decoding)ASR-BLEU (Dev)6.6—Unverified
#ModelMetricClaimedVerifiedStatus
1GenTranslateV2ASR-BLEU32.3—Unverified
2GenTranslateV1ASR-BLEU30.1—Unverified
3SeamlessM4T LargeV2ASR-BLEU29.4—Unverified
4SeamlessM4T LargeASR-BLEU25.8—Unverified
5AudioPaLM2ASR-BLEU24—Unverified
6WhisperV2ASR-BLEU23.5—Unverified
7SeamlessM4T MediumASR-BLEU20.4—Unverified
#ModelMetricClaimedVerifiedStatus
1SeamlessM4T LargeASR-BLEU36.5—Unverified
2SeamlessM4T MediumASR-BLEU28.1—Unverified