SOTAVerified

Task Arithmetic

A task vector specifies a direction in the weight space of a pre-trained model, such that movement in that direction improves performance on the task. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–61 of 61 papers

TitleStatusHype
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task ArithmeticCode2
Merging Multi-Task Models via Weight-Ensembling Mixture of ExpertsCode1
Concrete Subspace Learning based Interference Elimination for Multi-task Model FusionCode1
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization—0
Task Arithmetic with LoRA for Continual Learning—0
Model Merging by Uncertainty-Based Gradient MatchingCode1
Parameter Efficient Multi-task Model Fusion with Partial LinearizationCode1
AdaMerging: Adaptive Model Merging for Multi-Task LearningCode1
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained ModelsCode1
An Empirical Study of Multimodal Model MergingCode1
Editing Models with Task ArithmeticCode2
Show:102550
← PrevPage 3 of 3Next →

No leaderboard results yet.