| Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code | Mar 30, 2024 | Continual PretrainingLanguage Modelling | —Unverified | 0 |
| PARAMANU-AYN: Pretrain from scratch or Continual Pretraining of LLMs for Legal Domain Adaptation? | Mar 20, 2024 | Abstractive Text SummarizationContinual Pretraining | —Unverified | 0 |
| Yi: Open Foundation Models by 01.AI | Mar 7, 2024 | AttributeChatbot | CodeCode Available | 9 |
| Investigating Continual Pretraining in Large Language Models: Insights and Implications | Feb 27, 2024 | Continual LearningContinual Pretraining | —Unverified | 0 |
| Data Engineering for Scaling Language Models to 128K Context | Feb 15, 2024 | 4kContinual Pretraining | CodeCode Available | 3 |
| Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts | Feb 12, 2024 | Continual PretrainingGSM8K | CodeCode Available | 2 |
| Continual Learning for Large Language Models: A Survey | Feb 2, 2024 | Continual LearningContinual Pretraining | —Unverified | 0 |
| RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization | Jan 25, 2024 | Continual PretrainingSentiment Analysis | CodeCode Available | 0 |
| PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment | Nov 11, 2023 | Action Quality AssessmentContinual Pretraining | CodeCode Available | 0 |
| Effective Long-Context Scaling of Foundation Models | Sep 27, 2023 | Continual PretrainingLanguage Modeling | CodeCode Available | 2 |
