Meta-Learning for Efficient Fine-Tuning of Large Language Models

Code

• github.com/mobambas/NLP-Finetuning-Using-LLMs
  none★ 1

Abstract

In this project, we developed a parameter-efficient text generator model that generates text in the same way as a Reddit TIFU post. We used the Reddit TIFU dataset, which contains long-form text posts with titles and summaries. We explored the data and preprocessed it for training. We then designed and trained a BERT-based text generation model using PyTorch and the Hugging Face Transformers library. We tuned the hyperparameters to improve the model's performance and evaluated it using multiple metrics. Our final model achieved a high level of accuracy and can be used for various natural language processing tasks.

Tasks

Reproductions