Speech Synthesis

Speech synthesis is the task of generating speech from some other modality like text, lip movements etc.

Please note that the leaderboards here are not really comparable between studies - as they use mean opinion score as a metric and collect different samples from Amazon Mechnical Turk.

( Image credit: WaveNet: A generative model for raw audio )

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 651–675 of 1249 papers

Title	Date	Tasks	Status	Hype
Assessing Evaluation Metrics for Speech-to-Speech Translation	Oct 26, 2021	Machine TranslationOpen-Ended Question Answering	—Unverified	0
DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021	Oct 25, 2021	Speech Synthesistext-to-speech	CodeCode Available	0
CaloFlow II: Even Faster and Still Accurate Generation of Calorimeter Showers with Normalizing Flows	Oct 21, 2021	Speech Synthesis	CodeCode Available	0
Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition	Oct 21, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified	0
FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection	Oct 18, 2021	Speech SynthesisSynthetic Speech Detection	CodeCode Available	1
From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translation	Oct 15, 2021	Data AugmentationSimultaneous Speech-to-Speech Translation	—Unverified	0
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention	Oct 15, 2021	Simultaneous Speech-to-Speech TranslationSpeech Synthesis	—Unverified	0
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing	Oct 14, 2021	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	CodeCode Available	1
SingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation	Oct 14, 2021	Generative Adversarial NetworkGPU	—Unverified	0
Systematic Inequalities in Language Technology Performance across the World's Languages	Oct 13, 2021	Dependency ParsingMachine Translation	CodeCode Available	0
DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding	Oct 13, 2021	Speech SynthesisVoice Conversion	—Unverified	0
Fine-grained style control in Transformer-based Text-to-speech Synthesis	Oct 12, 2021	Inductive BiasSpeech Synthesis	CodeCode Available	1
LaughNet: synthesizing laughter utterances from waveform silhouettes and a single laughter example	Oct 11, 2021	Speech Synthesis	—Unverified	0
Using multiple reference audios and style embedding constraints for speech synthesis	Oct 9, 2021	SentenceSentence Similarity	—Unverified	0
Towards Lifelong Learning of Multilingual Text-To-Speech Synthesis	Oct 9, 2021	Lifelong learningSpeech Synthesis	CodeCode Available	0
Environment Aware Text-to-Speech Synthesis	Oct 8, 2021	AttributeDisentanglement	—Unverified	0
Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech	Oct 8, 2021	Emotion InterpretationExpressive Speech Synthesis	CodeCode Available	1
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over	Oct 7, 2021	Speech Synthesistext-to-speech	—Unverified	0
StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis	Oct 7, 2021	AttributeData Augmentation	CodeCode Available	1
Cloning one's voice using very limited data in the wild	Oct 7, 2021	Speech Synthesis	—Unverified	0
Mixer-TTS: non-autoregressive, fast and compact text-to-speech model conditioned on language model embeddings	Oct 7, 2021	Language ModelingLanguage Modelling	CodeCode Available	1
GANtron: Emotional Speech Synthesis with Generative Adversarial Networks	Oct 6, 2021	Emotional Speech SynthesisSpeech Synthesis	—Unverified	0
EdiTTS: Score-based Editing for Controllable Text-to-Speech	Oct 6, 2021	Speech SynthesisSpeech-to-Text	CodeCode Available	1
Prosody-TTS: An end-to-end speech synthesis system with prosody control	Oct 6, 2021	RhythmSpeech Synthesis	—Unverified	0
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis	Oct 4, 2021	Knowledge DistillationSpeech Synthesis	—Unverified	0

Show:10 25 50

← PrevPage 27 of 50Next →

All datasets LibriTTS North American English LJSpeech Mandarin Chinese Blizzard Challenge 2013

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	PeriodWave-Turbo-L	PESQ	4.45	—	Unverified
2	BigVGAN-v2	PESQ	4.36	—	Unverified
3	EVA-GAN-big	PESQ	4.35	—	Unverified
4	PeriodWave + FreeU	PESQ	4.25	—	Unverified
5	RFWave	PESQ	4.23	—	Unverified
6	BigVSAN (w/ snakebeta)	PESQ	4.12	—	Unverified
7	BigVSAN	PESQ	4.12	—	Unverified
8	EVA-GAN-base	PESQ	4.03	—	Unverified
9	BigVGAN	PESQ	4.03	—	Unverified
10	Vocos	PESQ	3.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Tacotron 2	Mean Opinion Score	4.53	—	Unverified
2	WaveNet (Linguistic)	Mean Opinion Score	4.34	—	Unverified
3	WaveNet (L+F)	Mean Opinion Score	4.21	—	Unverified
4	Tacotron	Mean Opinion Score	4	—	Unverified
5	HMM-driven concatenative	Mean Opinion Score	3.86	—	Unverified
6	LSTM-RNN parametric	Mean Opinion Score	3.67	—	Unverified
7	means	Mean Opinion Score	0	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	BDDM vocoder	Mean Opinion Score	4.48	—	Unverified
2	DiffWave LARGE	Mean Opinion Score	4.44	—	Unverified
3	Neural HMM	Mean Opinion Score	3.24	—	Unverified
4	Neural HMM Ablation with 1 state per phone	Mean Opinion Score	2.68	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	WaveNet (L+F)	Mean Opinion Score	4.08	—	Unverified
2	LSTM-RNN parametric	Mean Opinion Score	3.79	—	Unverified
3	HMM-driven concatenative	Mean Opinion Score	3.47	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SampleRNN (2-tier)	NLL	1.39	—	Unverified
2	SampleRNN (3-tier)	NLL	1.39	—	Unverified