Atyaephyra at SemEval-2025 Task 4: Low-Rank Negative Preference Optimization

Code

• github.com/XelfXendr/peft_unlearning
  Officialpytorch★ 3

Abstract

We present a submission to the SemEval 2025 shared task on unlearning sensitive content from LLMs. Our approach employs negative preference optimization using low-rank adaptation. We show that we can utilize this combination to efficiently compute additional regularization terms, which help with unlearning stabilization. The results of our approach significantly exceed the shared task baselines.

Reproductions