SOTAVerified

Mixture-of-Experts

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 1312 papers

TitleStatusHype
DeepSeek-V3 Technical ReportCode16
Qwen2.5 Technical ReportCode13
Qwen2 Technical ReportCode13
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and ApplicationsCode9
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language ModelCode9
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code IntelligenceCode9
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal UnderstandingCode9
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated ParametersCode9
Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM TrainingCode7
HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion TransformerCode7
MiniMax-01: Scaling Foundation Models with Lightning AttentionCode7
MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning AttentionCode7
MoE-LLaVA: Mixture of Experts for Large Vision-Language ModelsCode7
MoBA: Mixture of Block Attention for Long-Context LLMsCode7
OpenMoE: An Early Effort on Open Mixture-of-Experts Language ModelsCode5
Parrot: Multilingual Visual Instruction TuningCode5
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsCode5
Kimi-VL Technical ReportCode5
Jamba-1.5: Hybrid Transformer-Mamba Models at ScaleCode5
Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of ExpertsCode5
Comet: Fine-grained Computation-communication Overlapping for Mixture-of-ExpertsCode5
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-ExpertsCode5
Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language ModelCode5
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-BudgetCode5
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingCode5
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of ExpertsCode5
Rethinking LLM Language Adaptation: A Case Study on Chinese MixtralCode5
Aria: An Open Multimodal Native Mixture-of-Experts ModelCode5
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by TencentCode5
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented ScaleCode4
JetMoE: Reaching Llama2 Performance with 0.1M DollarsCode4
OLMoE: Open Mixture-of-Experts Language ModelsCode4
MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation ExpertsCode4
Mixtral of ExpertsCode4
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-FreeCode4
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of ExpertsCode4
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language ModelsCode4
Training Sparse Mixture Of Experts Text Embedding ModelsCode4
Fast Inference of Mixture-of-Experts Language Models with OffloadingCode4
MoH: Multi-Head Attention as Mixture-of-Head AttentionCode4
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language ModelsCode4
Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural Radiance FieldsCode3
MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-ExpertsCode3
Generalizing Motion Planners with Mixture of Experts for Autonomous DrivingCode3
FlashDMoE: Fast Distributed MoE in a Single KernelCode3
Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts ModelsCode3
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of ExpertsCode3
MoAI: Mixture of All Intelligence for Large Language and Vision ModelsCode3
MoE-Mamba: Efficient Selective State Space Models with Mixture of ExpertsCode3
LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge DistillationCode3
Show:102550
← PrevPage 1 of 27Next →

No leaderboard results yet.