SOTAVerified

Task Arithmetic

A task vector specifies a direction in the weight space of a pre-trained model, such that movement in that direction improves performance on the task. We build task vectors by subtracting the weights of a pre-trained model from the weights of the same model after fine-tuning on a task. We show that these task vectors can be modified and combined together through arithmetic operations such as negation and addition, and the behavior of the resulting model is steered accordingly.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 61 papers

TitleStatusHype
Task Singular Vectors: Reducing Task Interference in Model MergingCode2
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task ArithmeticCode2
Localizing Task Information for Improved Model Merging and CompressionCode2
Editing Models with Task ArithmeticCode2
Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task ArithmeticCode1
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model MergingCode1
Model Merging by Uncertainty-Based Gradient MatchingCode1
Knowledge Composition using Task Vectors with Learned Anisotropic ScalingCode1
AdaMerging: Adaptive Model Merging for Multi-Task LearningCode1
Merging Multi-Task Models via Weight-Ensembling Mixture of ExpertsCode1
Localize-and-Stitch: Efficient Model Merging via Sparse Task ArithmeticCode1
Parameter Efficient Multi-task Model Fusion with Partial LinearizationCode1
Concrete Subspace Learning based Interference Elimination for Multi-task Model FusionCode1
NegMerge: Consensual Weight Negation for Strong Machine UnlearningCode1
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained ModelsCode1
An Empirical Study of Multimodal Model MergingCode1
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers—0
ATM: Improving Model Merging by Alternating Tuning and Merging—0
BADTV: Unveiling Backdoor Threats in Third-Party Task Vectors—0
Beyond Task Vectors: Selective Task Arithmetic Based on Importance Metrics—0
Bias Vector: Mitigating Biases in Language Models with Task Arithmetic Approach—0
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging—0
CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning—0
Disentangling Task Interference within Neurons: Model Merging in Alignment with Neuronal Mechanisms—0
DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic—0
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging—0
Ethos: Rectifying Language Models in Orthogonal Parameter Space—0
FedRPCA: Enhancing Federated LoRA Aggregation Using Robust PCA—0
HPE-CogVLM: Advancing Vision Language Models with a Head Pose Grounding Task—0
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization—0
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge—0
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing—0
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts—0
Task Arithmetic Through The Lens Of One-Shot Federated Learning—0
Task Arithmetic with LoRA for Continual Learning—0
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse—0
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models—0
Transferring Visual Explainability of Self-Explaining Models through Task Arithmetic—0
What Matters for Model Merging at Scale?—0
When Domain Generalization meets Generalized Category Discovery: An Adaptive Task-Arithmetic Driven Approach—0
MCU: Improving Machine Unlearning through Mode Connectivity—0
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic—0
Neural Networks Remember More: The Power of Parameter Isolation and Combination—0
On Fairness of Task Arithmetic: The Role of Task Vectors—0
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion—0
OpenThaiGPT 1.6 and R1: Thai-Centric Open Source and Reasoning Large Language Models—0
Scalable Strategies for Continual Learning with Replay—0
Single-Input Multi-Output Model Merging: Leveraging Foundation Models for Dense Multi-Task Learning—0
Soup to go: mitigating forgetting during continual learning with model averaging—0
Subspace-Boosted Model Merging—0
Show:102550
← PrevPage 1 of 2Next →

No leaderboard results yet.