SOTAVerified

Second-order methods

Use second-order statistics to process data.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1–50 of 181 papers

TitleStatusHype
NysAct: A Scalable Preconditioned Gradient Descent using Nystrom ApproximationCode0
KOALA++: Efficient Kalman-Based Optimization of Neural Networks with Gradient-Covariance Products—0
Accelerated Training of Federated Learning via Second-Order Methods—0
Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information AnalysisCode2
Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology—0
FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration—0
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian ApproximationCode1
Online Covariance Matrix Estimation in Sketched Newton MethodsCode0
A Survey of Optimization Methods for Training DL Models: Theoretical Perspective on Convergence and Generalization—0
Distributed Quasi-Newton Method for Fair and Fast Federated Learning—0
Preconditioners for the Stochastic Training of Neural Fields—0
GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning—0
A Learn-to-Optimize Approach for Coordinate-Wise Step Sizes for Quasi-Newton Methods—0
Gradient Norm Regularization Second-Order Algorithms for Solving Nonconvex-Strongly Concave Minimax Problems—0
Don't Be So Positive: Negative Step Sizes in Second-Order Methods—0
Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor GeneralizationCode0
Improving Stochastic Cubic Newton with Momentum—0
Second-Order Min-Max Optimization with Lazy Hessians—0
Unlocking FedNL: Self-Contained Compute-Optimized Implementation—0
Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility—0
FLeNS: Federated Learning with Enhanced Nesterov-Newton SketchCode0
Alternating Iteratively Reweighted _1 and Subspace Newton Algorithms for Nonconvex Sparse OptimizationCode0
Fed-Sophia: A Communication-Efficient Second-Order Federated Learning Algorithm—0
Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement LearningCode1
Adaptive and Optimal Second-order Optimistic Methods for Minimax Optimization—0
Stochastic Newton Proximal Extragradient Method—0
Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks—0
A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial NetworksCode0
Inverse-Free Fast Natural Gradient Descent Method for Deep Learning—0
SGD with Partial Hessian for Deep Neural Networks OptimizationCode0
Second Order Methods for Bandit Optimization and Control—0
The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks—0
Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order PerspectiveCode1
On The Temporal Domain of Differential Equation Inspired Graph Neural Networks—0
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate—0
Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFACCode1
Adapting Newton's Method to Neural Networks through a Summary of Higher-Order DerivativesCode0
Adaptive Optimization Algorithms for Machine Learning—0
A Computationally Efficient Sparsified Online Newton MethodCode0
AdaSub: Stochastic Optimization Using Second-Order Information in Low-Dimensional SubspacesCode0
Studying K-FAC Heuristics by Viewing Adam through a Second-Order LensCode0
Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function—0
Jorge: Approximate Preconditioning for GPU-efficient Second-order Optimization—0
A Homogenization Approach for Gradient-Dominated Stochastic Optimization—0
Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold—0
Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods—0
Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oracles—0
Error Feedback Can Accurately Compress PreconditionersCode0
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 UpdatesCode1
Minibatching Offers Improved Generalization Performance for Second Order Optimizers—0
Show:102550
← PrevPage 1 of 4Next →

No leaderboard results yet.