SOTAVerified

Speech-to-Text Translation

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 146 papers

TitleStatusHype
PaddleSpeech: An Easy-to-Use All-in-One Speech ToolkitCode6
StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task LearningCode5
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTCode2
SeamlessM4T: Massively Multilingual & Multimodal Machine TranslationCode2
SONAR: Sentence-Level Multimodal and Language-Agnostic RepresentationsCode2
MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text TranslationCode2
CVSS Corpus and Massively Multilingual Speech-to-Speech TranslationCode2
Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box FrameworkCode1
MEDIBENG WHISPER TINY: A FINE-TUNED CODE-SWITCHED BENGALI-ENGLISH TRANSLATOR FOR CLINICAL APPLICATIONSCode1
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language ModelsCode1
ArzEn-LLM: Code-Switched Egyptian Arabic-English Translation and Speech Recognition Using LLMsCode1
LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned ProportionsCode1
Pushing the Limits of Zero-shot End-to-End Speech TranslationCode1
End-to-End Single-Channel Speaker-Turn Aware Conversational Speech TranslationCode1
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text TranslationCode1
DUB: Discrete Unit Back-translation for Speech TranslationCode1
Back Translation for Speech-to-text Translation Without TranscriptsCode1
Pre-training for Speech Translation: CTC Meets Optimal TransportCode1
Cross-modal Contrastive Learning for Speech TranslationCode1
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo LanguagesCode1
STEMM: Self-learning with Speech-text Manifold Mixup for Speech TranslationCode1
A^3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and EditingCode1
SHAS: Approaching optimal Segmentation for End-to-End Speech TranslationCode1
Regularizing End-to-End Speech Translation with Triangular Decomposition AgreementCode1
Learning When to Translate for Streaming SpeechCode1
Lightweight Adapter Tuning for Multilingual Speech TranslationCode1
Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech TranslationCode1
End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021Code1
Learning Shared Semantic Space for Speech-to-Text TranslationCode1
End-to-end Speech Translation via Cross-modal Progressive TrainingCode1
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech TranslationCode1
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text TranslationCode1
Consecutive Decoding for Speech-to-text TranslationCode1
CoVoST 2 and Massively Multilingual Speech-to-Text TranslationCode1
CoVoST: A Diverse Multilingual Speech-To-Text Translation CorpusCode1
FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural NetworksCode1
End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data—0
S2ST-Omni: An Efficient and Scalable Multilingual Speech-to-Speech Translation Framework via Seamless Speech-Text Alignment and Streaming Speech Generation—0
Speech-to-Text Translation with Phoneme-Augmented CoT: Enhancing Cross-Lingual Transfer in Low-Resource Scenarios—0
Improving Language and Modality Transfer in Translation by Character-level Modeling—0
BeaverTalk: Oregon State University's IWSLT 2025 Simultaneous Speech Translation SystemCode0
AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translation—0
Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision—0
Balancing Speech Understanding and Generation Using Continual Pre-training for Codec-based Speech LLM—0
SparQLe: Speech Queries to Text Translation Through LLMsCode0
Speech to Speech Translation with Translatotron: A State of the Art Review—0
When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation—0
Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language UnderstandingCode0
How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?—0
Representation Purification for End-to-End Speech Translation—0
Show:102550
← PrevPage 1 of 3Next →
All datasetsMuST-C EN->DEMuST-C EN->ESMuST-C EN->FRCoVoST 2 eng-XCoVoST 2 X-engFLEURS eng-XFLEURS X-englibri-transMediBengMuST-CMuST-C EN->NL

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1Task Modulation + Multitask Learning(ASR/MT) + Data AugmentationCase-sensitive sacreBLEU28.88—Unverified
2Wav2Vec2.0+mBART+AdaptorsCase-sensitive sacreBLEU28.22—Unverified
3Transformer + Meta Learning(ASR/MT) + Data AugmentationCase-sensitive sacreBLEU27.51—Unverified
4Transformer with AdaptersCase-sensitive sacreBLEU24.63—Unverified
5Dual-decoder TransformerCase-sensitive sacreBLEU23.63—Unverified
6SpeechformerCase-sensitive sacreBLEU23.6—Unverified
7Transformer + ASR PretrainCase-sensitive sacreBLEU22.8—Unverified
8Transformer + ASR PretrainCase-sensitive sacreBLEU22.7—Unverified
#ModelMetricClaimedVerifiedStatus
1Transformer with AdaptersCase-sensitive sacreBLEU28.73—Unverified
2SpeechformerCase-sensitive sacreBLEU28.5—Unverified
3Dual-decoder TransformerCase-sensitive sacreBLEU28.12—Unverified
4Transformer + ASR Pretrain + SpecAugCase-sensitive sacreBLEU27.4—Unverified
5Transformer + ASR PretrainCase-sensitive sacreBLEU26.8—Unverified
#ModelMetricClaimedVerifiedStatus
1Dual-decoder TransformerCase-sensitive sacreBLEU33.45—Unverified
2Transformer + ASR Pretrain + SpecAugCase-sensitive sacreBLEU33.3—Unverified
3Transformer + ASR PretrainCase-sensitive sacreBLEU32.3—Unverified
#ModelMetricClaimedVerifiedStatus
1SeamlessM4T LargeBLEU30.6—Unverified
2SeamlessM4T MediumBLEU26.6—Unverified
#ModelMetricClaimedVerifiedStatus
1SeamlessM4T LargeBLEU34.1—Unverified
2SeamlessM4T MediumBLEU29.8—Unverified
#ModelMetricClaimedVerifiedStatus
1SeamlessM4T LargeBLEU21.5—Unverified
2SeamlessM4T MediumBLEU19.2—Unverified
#ModelMetricClaimedVerifiedStatus
1SeamlessM4T LargeBLEU24—Unverified
2SeamlessM4T MediumBLEU20.9—Unverified
#ModelMetricClaimedVerifiedStatus
1Transformer + ASR Pretrain + SpecAugCase-insensitive sacreBLEU17.2—Unverified
2Transformer + ASR PretrainCase-insensitive sacreBLEU16.5—Unverified
#ModelMetricClaimedVerifiedStatus
1MediBeng Whisper TinyBleu0.98—Unverified
2Whisper TinyBleu0.3—Unverified
#ModelMetricClaimedVerifiedStatus
1Transformer with AdaptersSacreBLEU26.61—Unverified
2Dual-decoder TransformerSacreBLEU25.62—Unverified
#ModelMetricClaimedVerifiedStatus
1SpeechformerCase-sensitive sacreBLEU27.7—Unverified