SOTAVerified

L2 Regularization

See Weight Decay.

$L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{2}$ Norm of the weights:

$$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{w^{T}w}$$

where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).

Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–75 of 128 papers

TitleStatusHype
Gram Regularization for Multi-view 3D Shape Retrieval—0
Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks—0
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation—0
Implicit Filter Sparsification In Convolutional Neural Networks—0
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks—0
L2 Regularization versus Batch and Weight Normalization—0
Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing—0
Learning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent—0
Learning Sparse Low-Precision Neural Networks With Learnable Regularization—0
Low-rank bias, weight decay, and model merging in neural networks—0
Maintaining Plasticity in Continual Learning via Regenerative Regularization—0
Maximum margin learning of t-SPNs for cell classification with filtered input—0
Multi-branch fusion network for hyperspectral image classification—0
Multimodal Bearing Fault Classification Under Variable Conditions: A 1D CNN with Transfer Learning—0
On Implicit Filter Level Sparsity in Convolutional Neural Networks—0
On sparse regression, Lp-regularization, and automated model discovery—0
On the utility and protection of optimization with differential privacy and classic regularization techniques—0
Overcoming catastrophic forgetting in neural networks—0
Parkinson's Disease Diagnosis Through Deep Learning: A Novel LSTM-Based Approach for Freezing of Gait Detection—0
Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data—0
Pricing Football Players using Neural Networks—0
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning—0
Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling—0
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation—0
Regularization techniques for fine-tuning in neural machine translation—0
Show:102550
← PrevPage 3 of 6Next →

No leaderboard results yet.