OpenGrok: Enhancing SNS Data Processing with Distilled Knowledge and Mask-like Mechanisms
Lumen AI, Zaozhuang No. 28 Middle School, Shihao Ji, Zihui Song, Fucheng Zhong, Jisen Jia, Zhaobo Wu, Zheyi Cao, Tianhao Xu
Code Available — Be the first to reproduce this paper.
ReproduceCode
- github.com/Lumen-Laboratory/OpenGrokOfficialpytorch★ 1
Abstract
This report details Lumen Labs' novel approach to processing Social Networking Service (SNS) data. We leverage knowledge distillation, specifically a simple distillation method inspired by DeepSeek-R1's CoT acquisition, combined with prompt hacking, to extract valuable training data from the Grok model. This data is then used to fine-tune a Phi-3-mini model, augmented with a mask-like mechanism specifically designed for handling the nuances of SNS data. Our method demonstrates state-of-the-art (SOTA) performance on several SNS data processing tasks, outperforming existing models like Grok, Phi-3, and GPT-4. We provide a comprehensive analysis of our approach, including mathematical formulations, engineering details, ablation studies, and comparative evaluations.
Tasks
Benchmark Results
| Dataset | Model | Metric | Claimed | Verified | Status |
|---|---|---|---|---|---|
| Text-To-SQL | Orange-mini | 0-shot MRR | 74.17 | — | Unverified |