MaLei at the PLABA Track of TAC-2024: RoBERTa for Task 1 -- LLaMA3.1 and GPT-4o for Task 2

Code

• github.com/hecta-uom/plaba2024
  OfficialIn paperpytorch★ 0

Abstract

This report is the system description of the MaLei team (Manchester and Leiden) for shared task Plain Language Adaptation of Biomedical Abstracts (PLABA) 2024 (we had an earlier name BeeManc following last year). This report contains two sections corresponding to the two sub-tasks in PLABA 2024. In task one, we applied fine-tuned ReBERTa-Base models to identify and classify the difficult terms, jargon and acronyms in the biomedical abstracts and reported the F1 score. Due to time constraints, we didn't finish the replacement task. In task two, we leveraged Llamma3.1-70B-Instruct and GPT-4o with the one-shot prompts to complete the abstract adaptation and reported the scores in BLEU, SARI, BERTScore, LENS, and SALSA. From the official Evaluation from PLABA-2024 on Task 1A and 1B, our much smaller fine-tuned RoBERTa-Base model ranked 3rd and 2nd respectively on the two sub-task, and the 1st on averaged F1 scores across the two tasks from 9 evaluated systems. Our LLaMA-3.1-70B-instructed model achieved the highest Completeness score for Task-2. We share our fine-tuned models and related resources at https://github.com/HECTA-UoM/PLABA2024

Tasks

Reproductions