SOTAVerified

L2 Regularization

See Weight Decay.

$L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{2}$ Norm of the weights:

$$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{w^{T}w}$$

where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).

Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 128 papers

TitleStatusHype
Gram Regularization for Multi-view 3D Shape Retrieval—0
Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks—0
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation—0
Implicit Filter Sparsification In Convolutional Neural Networks—0
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks—0
L2 Regularization versus Batch and Weight Normalization—0
Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing—0
Learning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent—0
Learning Sparse Low-Precision Neural Networks With Learnable Regularization—0
Low-rank bias, weight decay, and model merging in neural networks—0
Maintaining Plasticity in Continual Learning via Regenerative Regularization—0
Maximum margin learning of t-SPNs for cell classification with filtered input—0
Multi-branch fusion network for hyperspectral image classification—0
Multimodal Bearing Fault Classification Under Variable Conditions: A 1D CNN with Transfer Learning—0
On Implicit Filter Level Sparsity in Convolutional Neural Networks—0
On sparse regression, Lp-regularization, and automated model discovery—0
On the utility and protection of optimization with differential privacy and classic regularization techniques—0
Overcoming catastrophic forgetting in neural networks—0
Parkinson's Disease Diagnosis Through Deep Learning: A Novel LSTM-Based Approach for Freezing of Gait Detection—0
Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data—0
Pricing Football Players using Neural Networks—0
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning—0
Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling—0
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation—0
Regularization techniques for fine-tuning in neural machine translation—0
Regularized Policy Iteration—0
Regularized Training of Nearest Neighbor Language Models—0
Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis—0
Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling—0
Reverse Engineering Deep ReLU Networks An Optimization-based Algorithm—0
Revisiting Activation Regularization for Language RNNs—0
Robust method for finding sparse solutions to linear inverse problems using an L2 regularization—0
Self-Distillation Amplifies Regularization in Hilbert Space—0
Semantic segmentation for building houses from wooden cubes—0
Sequence Length is a Domain: Length-based Overfitting in Transformer Models—0
Improved error rates for sparse (group) learning with Lipschitz loss functions—0
The Ant Swarm Neuro-Evolution Procedure for Optimizing Recurrent Networks—0
The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial—0
Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modification—0
Tight Sample Complexity of Large-Margin Learning—0
To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout—0
Towards a Better Understanding of Predict and Count Models—0
Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit—0
Understand the Effect of Importance Weighting in Deep Learning on Dataset Shift—0
Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus—0
Weight decay induces low-rank attention layers—0
Super-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning—0
A Bayesian encourages dropout—0
A Bayesian traction force microscopy method with automated denoising in a user-friendly software package—0
Achieving Strong Regularization for Deep Neural Networks—0
Show:102550
← PrevPage 2 of 3Next →

No leaderboard results yet.