SOTAVerified

L2 Regularization

See Weight Decay.

$L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{2}$ Norm of the weights:

$$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{w^{T}w}$$

where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).

Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 51–100 of 128 papers

TitleStatusHype
Globally Gated Deep Linear Networks—0
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks—0
On the utility and protection of optimization with differential privacy and classic regularization techniques—0
Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data—0
Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks—0
A Note on the Regularity of Images Generated by Convolutional Neural Networks—0
A Closer Look at Rehearsal-Free Continual Learning—0
How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic CharacterizationCode0
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning—0
Disturbing Target Values for Neural Network RegularizationCode0
Regularized Training of Nearest Neighbor Language Models—0
Sequence Length is a Domain: Length-based Overfitting in Transformer Models—0
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity—0
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation—0
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process PerspectiveCode0
Learning with Hyperspherical UniformityCode0
Effect of the regularization hyperparameter on deep learning-based segmentation in LGE-MRI—0
Gram Regularization for Multi-view 3D Shape Retrieval—0
Exponentially Weighted l_2 Regularization Strategy in Constructing Reinforced Second-order Fuzzy Rule-based Model—0
An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning—0
A Bayesian traction force microscopy method with automated denoising in a user-friendly software package—0
Data-dependent Gaussian Prior Objective for Language Generation—0
Correlated Initialization for Correlated Data—0
Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modification—0
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation—0
Empirical Study on Airline Delay Analysis and Prediction—0
Data and Model Dependencies of Membership Inference AttackCode0
Self-Distillation Amplifies Regularization in Hilbert Space—0
Customers Churn Prediction in Financial Institution Using Artificial Neural Network—0
A Comparative Study of Neural Network Compression—0
Improved error rates for sparse (group) learning with Lipschitz loss functions—0
Understanding and Stabilizing GANs' Training Dynamics with Control TheoryCode0
The Ant Swarm Neuro-Evolution Procedure for Optimizing Recurrent Networks—0
Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus—0
The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial—0
Emergence of Implicit Filter Sparsity in Convolutional Neural Networks—0
Implicit Filter Sparsification In Convolutional Neural Networks—0
A MAX-AFFINE SPLINE PERSPECTIVE OF RECURRENT NEURAL NETWORKS—0
Analysis of overfitting in the regularized Cox model—0
Learning a smooth kernel regularizer for convolutional neural networksCode0
Deep Optimization model for Screen Content Image Quality Assessment using Neural Networks—0
Adaptive Estimators Show Information Compression in Deep Neural Networks—0
Multi-branch fusion network for hyperspectral image classification—0
Construction of Differentially Private Empirical Distributions from a low-order Marginals Set through Solving Linear Equations with l2 Regularization—0
What is the Effect of Importance Weighting in Deep Learning?Code0
On Implicit Filter Level Sparsity in Convolutional Neural Networks—0
dynamic Long Short-Term Memory Neural-Network-Based Indict Remaining-Useful-Life Prognosis for Satellite Lithium-ion Battery—0
Edge-adaptive l2 regularization image reconstruction from non-uniform Fourier data—0
Gradient-Coherent Strong Regularization for Deep Neural Networks—0
Learning Sparse Low-Precision Neural Networks With Learnable Regularization—0
Show:102550
← PrevPage 2 of 3Next →

No leaderboard results yet.