Speech Synthesis

Speech synthesis is the task of generating speech from some other modality like text, lip movements etc.

Please note that the leaderboards here are not really comparable between studies - as they use mean opinion score as a metric and collect different samples from Amazon Mechnical Turk.

( Image credit: WaveNet: A generative model for raw audio )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 151–175 of 1249 papers

Title	Date	Tasks	Status	Hype
RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks	Jun 14, 2022	Action SegmentationInstance Segmentation	CodeCode Available	1
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation	May 25, 2022	Representation LearningRhythm	CodeCode Available	1
End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions	May 19, 2022	Speech SynthesisStyle Transfer	CodeCode Available	1
Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis	May 9, 2022	Deep LearningSemantic Communication	CodeCode Available	1
SVTS: Scalable Video-to-Speech Synthesis	May 4, 2022	Speech Synthesis	CodeCode Available	1
Requirements and Motivations of Low-Resource Speech Synthesis for Language Revitalization	May 1, 2022	Speech Synthesis	CodeCode Available	1
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond	Apr 20, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Lip to Speech Synthesis with Visual Context Attentional GAN	Apr 4, 2022	Contrastive LearningGenerative Adversarial Network	CodeCode Available	1
ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion	Mar 29, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition	Mar 29, 2022	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
A^3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing	Mar 18, 2022	Representation LearningSpeaker Verification	CodeCode Available	1
VocBench: A Neural Vocoder Benchmark for Speech Synthesis	Dec 6, 2021	Speech Synthesis	CodeCode Available	1
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone	Dec 4, 2021	Speech SynthesisText-To-Speech Synthesis	CodeCode Available	1
Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech	Nov 7, 2021	Meta-LearningSpeech Synthesis	CodeCode Available	1
Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework	Nov 2, 2021	DecoderEEG	CodeCode Available	1
ITAcotron 2: Transfering English Speech Synthesis Architectures and Speech Features to Italian	Nov 1, 2021	Speech Synthesis	CodeCode Available	1
FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection	Oct 18, 2021	Speech SynthesisSynthetic Speech Detection	CodeCode Available	1
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing	Oct 14, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
Fine-grained style control in Transformer-based Text-to-speech Synthesis	Oct 12, 2021	Inductive BiasSpeech Synthesis	CodeCode Available	1
Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech	Oct 8, 2021	Emotion InterpretationExpressive Speech Synthesis	CodeCode Available	1
StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis	Oct 7, 2021	AttributeData Augmentation	CodeCode Available	1
Mixer-TTS: non-autoregressive, fast and compact text-to-speech model conditioned on language model embeddings	Oct 7, 2021	Language ModelingLanguage Modelling	CodeCode Available	1
EdiTTS: Score-based Editing for Controllable Text-to-Speech	Oct 6, 2021	Speech SynthesisSpeech-to-Text	CodeCode Available	1
Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme	Sep 28, 2021	Speech SynthesisVoice Conversion	CodeCode Available	1
Neural HMMs are all you need (for high-quality attention-free TTS)	Aug 30, 2021	AllSpeech Synthesis	CodeCode Available	1

Show:10 25 50

← PrevPage 7 of 50Next →

All datasets LibriTTS North American English LJSpeech Mandarin Chinese Blizzard Challenge 2013

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	PeriodWave-Turbo-L	PESQ	4.45	—	Unverified
2	BigVGAN-v2	PESQ	4.36	—	Unverified
3	EVA-GAN-big	PESQ	4.35	—	Unverified
4	PeriodWave + FreeU	PESQ	4.25	—	Unverified
5	RFWave	PESQ	4.23	—	Unverified
6	BigVSAN (w/ snakebeta)	PESQ	4.12	—	Unverified
7	BigVSAN	PESQ	4.12	—	Unverified
8	EVA-GAN-base	PESQ	4.03	—	Unverified
9	BigVGAN	PESQ	4.03	—	Unverified
10	Vocos	PESQ	3.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Tacotron 2	Mean Opinion Score	4.53	—	Unverified
2	WaveNet (Linguistic)	Mean Opinion Score	4.34	—	Unverified
3	WaveNet (L+F)	Mean Opinion Score	4.21	—	Unverified
4	Tacotron	Mean Opinion Score	4	—	Unverified
5	HMM-driven concatenative	Mean Opinion Score	3.86	—	Unverified
6	LSTM-RNN parametric	Mean Opinion Score	3.67	—	Unverified
7	means	Mean Opinion Score	0	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BDDM vocoder	Mean Opinion Score	4.48	—	Unverified
2	DiffWave LARGE	Mean Opinion Score	4.44	—	Unverified
3	Neural HMM	Mean Opinion Score	3.24	—	Unverified
4	Neural HMM Ablation with 1 state per phone	Mean Opinion Score	2.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	WaveNet (L+F)	Mean Opinion Score	4.08	—	Unverified
2	LSTM-RNN parametric	Mean Opinion Score	3.79	—	Unverified
3	HMM-driven concatenative	Mean Opinion Score	3.47	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SampleRNN (2-tier)	NLL	1.39	—	Unverified
2	SampleRNN (3-tier)	NLL	1.39	—	Unverified