| PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit | May 20, 2022 | AllAutomatic Speech Recognition (ASR) | CodeCode Available | 6 | 5 |
| High-Fidelity Simultaneous Speech-To-Speech Translation | Feb 5, 2025 | DecoderSimultaneous Speech-to-Speech Translation | CodeCode Available | 5 | 5 |
| OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia | Jan 23, 2025 | Emotion RecognitionEvent Detection | CodeCode Available | 3 | 5 |
| Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation | Jun 14, 2024 | Audio-Visual Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 3 | 5 |
| MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation | Mar 1, 2023 | Audio-Visual Speech RecognitionRobust Speech Recognition | CodeCode Available | 2 | 5 |
| SONAR: Sentence-Level Multimodal and Language-Agnostic Representations | Aug 22, 2023 | DecoderMachine Translation | CodeCode Available | 2 | 5 |
| CVSS Corpus and Massively Multilingual Speech-to-Speech Translation | Jan 11, 2022 | SentenceSpeech-to-Speech Translation | CodeCode Available | 2 | 5 |
| SeamlessM4T: Massively Multilingual & Multimodal Machine Translation | Aug 22, 2023 | Automatic Speech RecognitionMachine Translation | CodeCode Available | 2 | 5 |
| Speech Model Pre-training for End-to-End Spoken Language Understanding | Apr 7, 2019 | Speech-to-TextSpoken Language Understanding | CodeCode Available | 2 | 5 |
| A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation | Jun 11, 2024 | DecoderSimultaneous Speech-to-Speech Translation | CodeCode Available | 2 | 5 |
| LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT | Oct 7, 2023 | Audio captioningAutomatic Speech Recognition | CodeCode Available | 2 | 5 |
| Pre-training for Speech Translation: CTC Meets Optimal Transport | Jan 27, 2023 | Multi-Task LearningSpeech-to-Text | CodeCode Available | 1 | 5 |
| PSST! Prosodic Speech Segmentation with Transformers | Feb 3, 2023 | SegmentationSpeech-to-Text | CodeCode Available | 1 | 5 |
| End-to-end Speech Translation via Cross-modal Progressive Training | Apr 21, 2021 | Machine TranslationSpeech-to-Text | CodeCode Available | 1 | 5 |
| OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents | Aug 6, 2024 | BenchmarkingRetrieval-augmented Generation | CodeCode Available | 1 | 5 |
| MEDIBENG WHISPER TINY: A FINE-TUNED CODE-SWITCHED BENGALI-ENGLISH TRANSLATOR FOR CLINICAL APPLICATIONS | Apr 25, 2025 | Clinical Language TranslationMachine Translation | CodeCode Available | 1 | 5 |
| One TTS Alignment To Rule Them All | Aug 23, 2021 | AllSpeech Synthesis | CodeCode Available | 1 | 5 |
| Pushing the Limits of Zero-shot End-to-End Speech Translation | Feb 16, 2024 | Speech-to-TextSpeech-to-Text Translation | CodeCode Available | 1 | 5 |
| Late reverberation suppression using U-nets | Oct 5, 2021 | DecoderSpeech Dereverberation | CodeCode Available | 1 | 5 |
| LeaPformer: Enabling Linear Transformers for Autoregressive and Simultaneous Tasks via Learned Proportions | May 18, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 1 | 5 |
| JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT | Oct 5, 2022 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| Deep Reinforcement Learning For Sequence to Sequence Models | May 24, 2018 | Abstractive Text SummarizationCaption Generation | CodeCode Available | 1 | 5 |
| DUB: Discrete Unit Back-translation for Speech Translation | May 19, 2023 | Machine TranslationSpeech-to-Text | CodeCode Available | 1 | 5 |
| A Whisper transformer for audio captioning trained with synthetic captions and transfer learning | May 15, 2023 | Audio captioningSpeech-to-Text | CodeCode Available | 1 | 5 |
| EdiTTS: Score-based Editing for Controllable Text-to-Speech | Oct 6, 2021 | Speech SynthesisSpeech-to-Text | CodeCode Available | 1 | 5 |
| Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation | May 11, 2021 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| Enhancing Speech-to-Speech Dialogue Modeling with End-to-End Retrieval-Augmented Generation | Apr 27, 2025 | RAGRetrieval | CodeCode Available | 1 | 5 |
| Kosp2e: Korean Speech to English Translation Corpus | Jul 6, 2021 | speech-recognitionSpeech Recognition | CodeCode Available | 1 | 5 |
| End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation | Nov 1, 2023 | Automatic Speech Recognitionspeech-recognition | CodeCode Available | 1 | 5 |
| ArzEn-LLM: Code-Switched Egyptian Arabic-English Translation and Speech Recognition Using LLMs | Jun 26, 2024 | ArzEn Code-switched Translation to araArzEn Code-switched Translation to eng | CodeCode Available | 1 | 5 |
| Learning Shared Semantic Space for Speech-to-Text Translation | May 7, 2021 | Machine TranslationSpeech-to-Text | CodeCode Available | 1 | 5 |
| IESTAC: English-Italian Parallel Corpus for End-to-End Speech-to-Text Machine Translation | Nov 1, 2020 | Dynamic Time WarpingMachine Translation | CodeCode Available | 1 | 5 |
| Common Voice: A Massively-Multilingual Speech Corpus | Dec 13, 2019 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| Clotho: An Audio Captioning Dataset | Oct 21, 2019 | Audio captioningDiversity | CodeCode Available | 1 | 5 |
| Indoor Air Quality Dataset with Activities of Daily Living in Low to Middle-income Communities | Jul 19, 2024 | ImputationRecommendation Systems | CodeCode Available | 1 | 5 |
| Brilla AI: AI Contestant for the National Science and Maths Quiz | Mar 4, 2024 | MathQuestion Answering | CodeCode Available | 1 | 5 |
| A Large-Scale Chinese Multimodal NER Dataset with Speech Clues | Aug 1, 2021 | named-entity-recognitionNamed Entity Recognition | CodeCode Available | 1 | 5 |
| ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation | May 24, 2023 | GPULanguage Modeling | CodeCode Available | 1 | 5 |
| CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus | Feb 4, 2020 | Speech-to-TextSpeech-to-Text Translation | CodeCode Available | 1 | 5 |
| Cross-modal Contrastive Learning for Speech Translation | May 5, 2022 | Contrastive LearningRetrieval | CodeCode Available | 1 | 5 |
| Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework | May 24, 2025 | Adversarial AttackSpeech Tokenization | CodeCode Available | 1 | 5 |
| Cross Attention Augmented Transducer Networks for Simultaneous Translation | Nov 1, 2021 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| Benchmarking Large Multimodal Models against Common Corruptions | Jan 22, 2024 | BenchmarkingImage to text | CodeCode Available | 1 | 5 |
| Denial-of-Service Poisoning Attacks against Large Language Models | Oct 14, 2024 | 16kSpeech-to-Text | CodeCode Available | 1 | 5 |
| Fine-tuning Whisper on Low-Resource Languages for Real-World Applications | Dec 20, 2024 | FormSentence | CodeCode Available | 1 | 5 |
| Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet | Jun 25, 2024 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| Back Translation for Speech-to-text Translation Without Transcripts | May 15, 2023 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities | Feb 16, 2025 | Automatic Speech RecognitionAutomatic Speech Recognition (ASR) | CodeCode Available | 1 | 5 |
| A^3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing | Mar 18, 2022 | Representation LearningSpeaker Verification | CodeCode Available | 1 | 5 |
| CoVoST 2 and Massively Multilingual Speech-to-Text Translation | Jul 20, 2020 | Machine Translationspeech-recognition | CodeCode Available | 1 | 5 |
