Speech Synthesis

Speech synthesis is the task of generating speech from some other modality like text, lip movements etc.

Please note that the leaderboards here are not really comparable between studies - as they use mean opinion score as a metric and collect different samples from Amazon Mechnical Turk.

( Image credit: WaveNet: A generative model for raw audio )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 101–125 of 1249 papers

Title	Date	Tasks	Status	Hype
Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling	Dec 19, 2023	Contrastive LearningSpeech Synthesis	CodeCode Available	1
What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection	Dec 15, 2023	Audio Deepfake DetectionContinual Learning	CodeCode Available	1
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realism	Dec 11, 2023	Face GenerationLip Reading	CodeCode Available	1
Learning Arousal-Valence Representation from Categorical Emotion Labels of Speech	Nov 24, 2023	Dimensionality ReductionEmotion Classification	CodeCode Available	1
APNet2: High-quality and High-efficiency Neural Vocoder with Direct Prediction of Amplitude and Phase Spectra	Nov 20, 2023	Speech Synthesis	CodeCode Available	1
Improved Child Text-to-Speech Synthesis through Fastpitch-based Transfer Learning	Nov 7, 2023	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
ArTST: Arabic Text and Speech Transformer	Oct 25, 2023	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
AutoDiff: combining Auto-encoder and Diffusion model for tabular data synthesizing	Oct 24, 2023	Language ModelingLanguage Modelling	CodeCode Available	1
Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech	Oct 1, 2023	speech-recognitionSpeech Recognition	CodeCode Available	1
Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model	Sep 20, 2023	ChatbotLanguage Modeling	CodeCode Available	1
QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning	Aug 31, 2023	Representation LearningSpeech Representation Learning	CodeCode Available	1
DiffV2S: Diffusion-based Video-to-Speech Synthesis with Vision-guided Speaker Embedding	Aug 15, 2023	Speech Synthesis	CodeCode Available	1
Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation	Aug 3, 2023	DecoderQuantization	CodeCode Available	1
DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training	Jul 31, 2023	DenoisingExpressive Speech Synthesis	CodeCode Available	1
SC VALL-E: Style-Controllable Zero-Shot Text to Speech Synthesizer	Jul 20, 2023	Expressive Speech SynthesisLanguage Modelling	CodeCode Available	1
Deep Speech Synthesis from MRI-Based Articulatory Representations	Jul 5, 2023	Computational EfficiencyDenoising	CodeCode Available	1
Disentanglement in a GAN for Unconditional Speech Synthesis	Jul 4, 2023	DisentanglementGenerative Adversarial Network	CodeCode Available	1
EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech	Jun 28, 2023	Emotion RecognitionSpeech Synthesis	CodeCode Available	1
Intelligible Lip-to-Speech Synthesis with Speech Units	May 31, 2023	Lip to Speech SynthesisSpeech Synthesis	CodeCode Available	1
ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation	May 29, 2023	Speech Synthesistext-to-speech	CodeCode Available	1
Automatic Tuning of Loss Trade-offs without Hyper-parameter Search in End-to-End Zero-Shot Speech Synthesis	May 26, 2023	DecoderSpeech Synthesis	CodeCode Available	1
Multilingual Text-to-Speech Synthesis for Turkic Languages Using Transliteration	May 25, 2023	Speech Synthesistext-to-speech	CodeCode Available	1
Scaling Speech Technology to 1,000+ Languages	May 22, 2023	Automatic Speech RecognitionLanguage Identification	CodeCode Available	1
EMNS /Imz/ Corpus: An emotive single-speaker dataset for narrative storytelling in games, television and graphic novels	May 22, 2023	Expressive Speech SynthesisSpeech Synthesis	CodeCode Available	1
Diffusion-Based Mel-Spectrogram Enhancement for Personalized Speech Synthesis with Found Data	May 18, 2023	Speech EnhancementSpeech Synthesis	CodeCode Available	1

Show:10 25 50

← PrevPage 5 of 50Next →

All datasets LibriTTS North American English LJSpeech Mandarin Chinese Blizzard Challenge 2013

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	PeriodWave-Turbo-L	PESQ	4.45	—	Unverified
2	BigVGAN-v2	PESQ	4.36	—	Unverified
3	EVA-GAN-big	PESQ	4.35	—	Unverified
4	PeriodWave + FreeU	PESQ	4.25	—	Unverified
5	RFWave	PESQ	4.23	—	Unverified
6	BigVSAN (w/ snakebeta)	PESQ	4.12	—	Unverified
7	BigVSAN	PESQ	4.12	—	Unverified
8	EVA-GAN-base	PESQ	4.03	—	Unverified
9	BigVGAN	PESQ	4.03	—	Unverified
10	Vocos	PESQ	3.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Tacotron 2	Mean Opinion Score	4.53	—	Unverified
2	WaveNet (Linguistic)	Mean Opinion Score	4.34	—	Unverified
3	WaveNet (L+F)	Mean Opinion Score	4.21	—	Unverified
4	Tacotron	Mean Opinion Score	4	—	Unverified
5	HMM-driven concatenative	Mean Opinion Score	3.86	—	Unverified
6	LSTM-RNN parametric	Mean Opinion Score	3.67	—	Unverified
7	means	Mean Opinion Score	0	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BDDM vocoder	Mean Opinion Score	4.48	—	Unverified
2	DiffWave LARGE	Mean Opinion Score	4.44	—	Unverified
3	Neural HMM	Mean Opinion Score	3.24	—	Unverified
4	Neural HMM Ablation with 1 state per phone	Mean Opinion Score	2.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	WaveNet (L+F)	Mean Opinion Score	4.08	—	Unverified
2	LSTM-RNN parametric	Mean Opinion Score	3.79	—	Unverified
3	HMM-driven concatenative	Mean Opinion Score	3.47	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SampleRNN (2-tier)	NLL	1.39	—	Unverified
2	SampleRNN (3-tier)	NLL	1.39	—	Unverified