360-LLaMA-Factory: Plug & Play Sequence Parallelism for Long Post-Training
2025-10-08Code Available0· sign in to hype
Haosheng Zou, Xiaowei Lv, Shousheng Jia, Lin Li, Xiaochun Gong, Xiangzheng Zhang
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/qihoo360/360-llama-factoryOfficialIn paper★ 605
Abstract
Adding sequence parallelism into LLaMA-Factory, we open-sourced 360-LLaMA-Factory at https://github.com/Qihoo360/360-LLaMA-Factory. 360-LLaMA-Factory has received wide recognition and used in models such as Light-R1 arXiv:2503.10460, TinyR1 arXiv:2503.04872, Kaggle AIMO math models and also in large companies' training frameworks. This technical report delves deeper into the different sequence parallel modes behind 360-LLaMA-Factory and discusses our implementation insights.