SOTAVerified

Mixture-of-Experts

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 1312 papers

TitleStatusHype
DeepSeek-V3 Technical ReportCode16
Qwen2.5 Technical ReportCode13
Qwen2 Technical ReportCode13
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated ParametersCode9
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code IntelligenceCode9
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language ModelCode9
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and ApplicationsCode9
DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal UnderstandingCode9
Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM TrainingCode7
MoE-LLaVA: Mixture of Experts for Large Vision-Language ModelsCode7
MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning AttentionCode7
HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion TransformerCode7
MoBA: Mixture of Block Attention for Long-Context LLMsCode7
MiniMax-01: Scaling Foundation Models with Lightning AttentionCode7
Parrot: Multilingual Visual Instruction TuningCode5
Aria: An Open Multimodal Native Mixture-of-Experts ModelCode5
Jamba-1.5: Hybrid Transformer-Mamba Models at ScaleCode5
Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of ExpertsCode5
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-ExpertsCode5
Kimi-VL Technical ReportCode5
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by TencentCode5
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of ExpertsCode5
Rethinking LLM Language Adaptation: A Case Study on Chinese MixtralCode5
Comet: Fine-grained Computation-communication Overlapping for Mixture-of-ExpertsCode5
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-BudgetCode5
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingCode5
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language ModelsCode5
Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language ModelCode5
OpenMoE: An Early Effort on Open Mixture-of-Experts Language ModelsCode5
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented ScaleCode4
JetMoE: Reaching Llama2 Performance with 0.1M DollarsCode4
OLMoE: Open Mixture-of-Experts Language ModelsCode4
MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation ExpertsCode4
Training Sparse Mixture Of Experts Text Embedding ModelsCode4
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-FreeCode4
Mixtral of ExpertsCode4
Fast Inference of Mixture-of-Experts Language Models with OffloadingCode4
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language ModelsCode4
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language ModelsCode4
MoH: Multi-Head Attention as Mixture-of-Head AttentionCode4
Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of ExpertsCode4
BlackMamba: Mixture of Experts for State-Space ModelsCode3
Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural Radiance FieldsCode3
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts AdaptersCode3
MVMoE: Multi-Task Vehicle Routing Solver with Mixture-of-ExpertsCode3
A Survey on Mixture of ExpertsCode3
A Survey on Inference Optimization Techniques for Mixture of Experts ModelsCode3
AnyGraph: Graph Foundation Model in the WildCode3
Generalizing Motion Planners with Mixture of Experts for Autonomous DrivingCode3
MoAI: Mixture of All Intelligence for Large Language and Vision ModelsCode3
Show:102550
← PrevPage 1 of 27Next →

No leaderboard results yet.