Speech-to-Text Translation

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 146 papers

Title	Date	Tasks	Status
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation	Apr 6, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
XTREME-S: Evaluating Cross-lingual Speech Representations	Mar 21, 2022	Representation LearningRetrieval	—Unverified
Cross-modal Contrastive Learning for Speech Translation	Dec 17, 2021	Contrastive LearningRetrieval	—Unverified
Improve Sinhala Speech Recognition Through e2e LF-MMI Model	Dec 1, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
An Experiment on Speech-to-Text Translation Systems for Manipuri to English on Low Resource Setting	Dec 1, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems	Oct 13, 2021	SentenceSimultaneous Speech-to-Text Translation	—Unverified
Speechformer: Reducing Information Loss in Direct Speech Translation	Sep 9, 2021	Speech-to-Text TranslationTranslation	CodeCode Available
Infusing Future Information into Monotonic Attention Through Language Models	Sep 7, 2021	Language ModelingLanguage Modelling	CodeCode Available
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task	Jul 12, 2021	DecoderKnowledge Distillation	—Unverified
Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling	Jun 21, 2021	speech-recognitionSpeech Recognition	—Unverified
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR	Jun 11, 2021	Simultaneous Speech-to-Text TranslationSpeech-to-Text	—Unverified
TASK AWARE MULTI-TASK LEARNING FOR SPEECH TO TEXT TASKS	Jun 10, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
Towards Measuring Fairness in AI: the Casual Conversations Dataset	Apr 6, 2021	Age And Gender ClassificationDeepFake Detection	—Unverified
Towards the evaluation of automatic simultaneous speech translation from a communicative perspective	Mar 15, 2021	automatic-speech-translationInformativeness	—Unverified
NeurST: Neural Speech Translation Toolkit	Dec 18, 2020	Speech-to-Text TranslationTranslation	CodeCode Available
Bridging the Modality Gap for Speech-to-Text Translation	Oct 28, 2020	DecoderSpeech-to-Text	—Unverified
MAM: Masked Acoustic Modeling for End-to-End Speech-to-Text Translation	Oct 22, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
Subtitles to Segmentation: Improving Low-Resource Speech-to-Text Translation Pipelines	Oct 19, 2020	Cross-Lingual Information RetrievalInformation Retrieval	—Unverified
fairseq S2T: Fast Speech-to-Text Modeling with fairseq	Oct 11, 2020	Machine TranslationMulti-Task Learning	CodeCode Available
Contextualized Translation of Automatically Segmented Speech	Aug 5, 2020	SegmentationSentence	CodeCode Available
End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning	Jul 1, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
SimulSpeech: End-to-End Simultaneous Speech to Text Translation	Jul 1, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
Self-Supervised Representations Improve End-to-End Speech Translation	Jun 22, 2020	Cross-Lingual Transferspeech-recognition	—Unverified
Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation	Jun 9, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
Subtitles to Segmentation: Improving Low-Resource Speech-to-TextTranslation Pipelines	May 1, 2020	Cross-Lingual Information RetrievalInformation Retrieval	—Unverified

Show:10 25 50

← PrevPage 5 of 6Next →

All datasets MuST-C EN->DE MuST-C EN->ES MuST-C EN->FR CoVoST 2 eng-X CoVoST 2 X-eng FLEURS eng-X FLEURS X-eng libri-trans MediBeng MuST-C MuST-C EN->NL

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	Task Modulation + Multitask Learning(ASR/MT) + Data Augmentation	Case-sensitive sacreBLEU	28.88	—	Unverified
2	Wav2Vec2.0+mBART+Adaptors	Case-sensitive sacreBLEU	28.22	—	Unverified
3	Transformer + Meta Learning(ASR/MT) + Data Augmentation	Case-sensitive sacreBLEU	27.51	—	Unverified
4	Transformer with Adapters	Case-sensitive sacreBLEU	24.63	—	Unverified
5	Dual-decoder Transformer	Case-sensitive sacreBLEU	23.63	—	Unverified
6	Speechformer	Case-sensitive sacreBLEU	23.6	—	Unverified
7	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	22.8	—	Unverified
8	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	22.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer with Adapters	Case-sensitive sacreBLEU	28.73	—	Unverified
2	Speechformer	Case-sensitive sacreBLEU	28.5	—	Unverified
3	Dual-decoder Transformer	Case-sensitive sacreBLEU	28.12	—	Unverified
4	Transformer + ASR Pretrain + SpecAug	Case-sensitive sacreBLEU	27.4	—	Unverified
5	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	26.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Dual-decoder Transformer	Case-sensitive sacreBLEU	33.45	—	Unverified
2	Transformer + ASR Pretrain + SpecAug	Case-sensitive sacreBLEU	33.3	—	Unverified
3	Transformer + ASR Pretrain	Case-sensitive sacreBLEU	32.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	30.6	—	Unverified
2	SeamlessM4T Medium	BLEU	26.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	34.1	—	Unverified
2	SeamlessM4T Medium	BLEU	29.8	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	21.5	—	Unverified
2	SeamlessM4T Medium	BLEU	19.2	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SeamlessM4T Large	BLEU	24	—	Unverified
2	SeamlessM4T Medium	BLEU	20.9	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer + ASR Pretrain + SpecAug	Case-insensitive sacreBLEU	17.2	—	Unverified
2	Transformer + ASR Pretrain	Case-insensitive sacreBLEU	16.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MediBeng Whisper Tiny	Bleu	0.98	—	Unverified
2	Whisper Tiny	Bleu	0.3	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Transformer with Adapters	SacreBLEU	26.61	—	Unverified
2	Dual-decoder Transformer	SacreBLEU	25.62	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Speechformer	Case-sensitive sacreBLEU	27.7	—	Unverified