Speech-to-Text Translation

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 76–100 of 146 papers

Title	Date	Tasks	Status	Hype
Revisiting End-to-End Speech-to-Text Translation From Scratch	Jun 9, 2022	Decoderspeech-recognition	CodeCode Available	0
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit	May 20, 2022	AllAutomatic Speech Recognition (ASR)	CodeCode Available	6
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation	May 17, 2022	Representation LearningRetrieval	—Unverified	0
Cross-modal Contrastive Learning for Speech Translation	May 5, 2022	Contrastive LearningRetrieval	CodeCode Available	1
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages	May 2, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation	May 1, 2022	SegmentationSimultaneous Speech-to-Text Translation	—Unverified	0
NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022	May 1, 2022	SegmentationSimultaneous Speech-to-Text Translation	—Unverified	0
LibriS2S: A German-English Speech-to-Speech Translation Corpus	Apr 22, 2022	Speech-to-Speech TranslationSpeech-to-Text	CodeCode Available	0
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation	Apr 6, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
XTREME-S: Evaluating Cross-lingual Speech Representations	Mar 21, 2022	Representation LearningRetrieval	—Unverified	0
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation	Mar 20, 2022	Machine TranslationSpeech-to-Text	CodeCode Available	1
A^3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing	Mar 18, 2022	Representation LearningSpeaker Verification	CodeCode Available	1
SHAS: Approaching optimal Segmentation for End-to-End Speech Translation	Feb 9, 2022	SegmentationSpeech-to-Text Translation	CodeCode Available	1
CVSS Corpus and Massively Multilingual Speech-to-Speech Translation	Jan 11, 2022	SentenceSpeech-to-Speech Translation	CodeCode Available	2
Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement	Dec 21, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Cross-modal Contrastive Learning for Speech Translation	Dec 17, 2021	Contrastive LearningRetrieval	—Unverified	0
Improve Sinhala Speech Recognition Through e2e LF-MMI Model	Dec 1, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
An Experiment on Speech-to-Text Translation Systems for Manipuri to English on Low Resource Setting	Dec 1, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems	Oct 13, 2021	SentenceSimultaneous Speech-to-Text Translation	—Unverified	0
Learning When to Translate for Streaming Speech	Sep 15, 2021	DecoderSentence	CodeCode Available	1
Speechformer: Reducing Information Loss in Direct Speech Translation	Sep 9, 2021	Speech-to-Text TranslationTranslation	CodeCode Available	0
Infusing Future Information into Monotonic Attention Through Language Models	Sep 7, 2021	Language ModelingLanguage Modelling	CodeCode Available	0
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task	Jul 12, 2021	DecoderKnowledge Distillation	—Unverified	0
Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling	Jun 21, 2021	speech-recognitionSpeech Recognition	—Unverified	0
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR	Jun 11, 2021	Simultaneous Speech-to-Text TranslationSpeech-to-Text	—Unverified	0

Show:10 25 50

← PrevPage 4 of 6Next →

All datasets MuST-C EN->DE MuST-C EN->ES MuST-C EN->FR CoVoST 2 eng-X CoVoST 2 X-eng FLEURS eng-X FLEURS X-eng libri-trans MediBeng MuST-C MuST-C EN->NL

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Task Modulation + Multitask Learning(ASR/MT) + Data Augmentation	Case-sensitive sacreBLEU	28.88	—	Unverified
2	Wav2Vec2.0+mBART+Adaptors	Case-sensitive sacreBLEU	28.22	—	Unverified
3	Transformer + Meta Learning(ASR/MT) + Data Augmentation	Case-sensitive sacreBLEU	27.51	—	Unverified
4	Transformer with Adapters	Case-sensitive sacreBLEU	24.63	—	Unverified
5	Dual-decoder Transformer	Case-sensitive sacreBLEU	23.63	—	Unverified
6	Speechformer	Case-sensitive sacreBLEU	23.6	—	Unverified
7	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	22.8	—	Unverified
8	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	22.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer with Adapters	Case-sensitive sacreBLEU	28.73	—	Unverified
2	Speechformer	Case-sensitive sacreBLEU	28.5	—	Unverified
3	Dual-decoder Transformer	Case-sensitive sacreBLEU	28.12	—	Unverified
4	Transformer + ASR Pretrain + SpecAug	Case-sensitive sacreBLEU	27.4	—	Unverified
5	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	26.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Dual-decoder Transformer	Case-sensitive sacreBLEU	33.45	—	Unverified
2	Transformer + ASR Pretrain + SpecAug	Case-sensitive sacreBLEU	33.3	—	Unverified
3	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	32.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	30.6	—	Unverified
2	SeamlessM4T Medium	BLEU	26.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	34.1	—	Unverified
2	SeamlessM4T Medium	BLEU	29.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	21.5	—	Unverified
2	SeamlessM4T Medium	BLEU	19.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	24	—	Unverified
2	SeamlessM4T Medium	BLEU	20.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer + ASR Pretrain + SpecAug	Case-insensitive sacreBLEU	17.2	—	Unverified
2	Transformer + ASR Pretrain	Case-insensitive sacreBLEU	16.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MediBeng Whisper Tiny	Bleu	0.98	—	Unverified
2	Whisper Tiny	Bleu	0.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer with Adapters	SacreBLEU	26.61	—	Unverified
2	Dual-decoder Transformer	SacreBLEU	25.62	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Speechformer	Case-sensitive sacreBLEU	27.7	—	Unverified