| Mixture of Experts Meets Prompt-Based Continual Learning | May 23, 2024 | Continual LearningMixture-of-Experts | CodeCode Available | 1 |
| Graph Sparsification via Mixture of Graphs | May 23, 2024 | Graph LearningMixture-of-Experts | CodeCode Available | 1 |
| Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models | May 23, 2024 | Mixture-of-ExpertsVisual Question Answering | CodeCode Available | 2 |
| Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast | May 23, 2024 | Computational EfficiencyGSM8K | CodeCode Available | 1 |
| Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts | May 22, 2024 | Mixture-of-Experts | —Unverified | 0 |
| xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token | May 22, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 2 |
| DirectMultiStep: Direct Route Generation for Multi-Step Retrosynthesis | May 22, 2024 | DiversityMixture-of-Experts | CodeCode Available | 1 |
| Ensemble and Mixture-of-Experts DeepONets For Operator Learning | May 20, 2024 | Mixture-of-ExpertsOperator learning | CodeCode Available | 0 |
| MeteoRA: Multiple-tasks Embedded LoRA for Large Language Models | May 19, 2024 | Mixture-of-Expertsparameter-efficient fine-tuning | CodeCode Available | 1 |
| Learning More Generalized Experts by Merging Experts in Mixture-of-Experts | May 19, 2024 | Incremental LearningMixture-of-Experts | —Unverified | 0 |
| Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts | May 18, 2024 | Mixture-of-ExpertsVisual Question Answering | CodeCode Available | 5 |
| Many Hands Make Light Work: Task-Oriented Dialogue System with Module-Based Mixture-of-Experts | May 16, 2024 | Dialogue State TrackingMixture-of-Experts | —Unverified | 0 |
| M^4oE: A Foundation Model for Medical Multimodal Image Segmentation with Mixture of Experts | May 15, 2024 | Image SegmentationMixture-of-Experts | CodeCode Available | 1 |
| A Mixture of Experts Approach to 3D Human Motion Prediction | May 9, 2024 | Human motion predictionMixture-of-Experts | CodeCode Available | 0 |
| A Mixture-of-Experts Approach to Few-Shot Task Transfer in Open-Ended Text Worlds | May 9, 2024 | Few-Shot LearningMixture-of-Experts | —Unverified | 0 |
| EWMoE: An effective model for global weather forecasting with mixture-of-experts | May 9, 2024 | Mixture-of-ExpertsWeather Forecasting | CodeCode Available | 1 |
| CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts | May 9, 2024 | Image CaptioningInstruction Following | CodeCode Available | 2 |
| SUTRA: Scalable Multilingual Language Model Architecture | May 7, 2024 | Computational EfficiencyHallucination | —Unverified | 0 |
| DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model | May 7, 2024 | Language ModelingLanguage Modelling | CodeCode Available | 9 |
| MEET: Mixture of Experts Extra Tree-Based sEMG Hand Gesture Identification | May 6, 2024 | Electromyography (EMG)Gesture Recognition | —Unverified | 0 |
| WDMoE: Wireless Distributed Large Language Models with Mixture of Experts | May 6, 2024 | Mixture-of-Experts | —Unverified | 0 |
| Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training | May 6, 2024 | Language ModelingLanguage Modelling | —Unverified | 0 |
| Mixture of partially linear experts | May 5, 2024 | Mixture-of-Experts | —Unverified | 0 |
| MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-Experts | May 2, 2024 | Combinatorial OptimizationMixture-of-Experts | CodeCode Available | 3 |
| Hierarchical mixture of discriminative Generalized Dirichlet classifiers | May 2, 2024 | Mixture-of-ExpertsSpam detection | —Unverified | 0 |