| Automatic Gradient Descent: Deep Learning without Hyperparameters | Apr 11, 2023 | Deep LearningSecond-order methods | CodeCode Available | 2 | 5 |
| Towards Practical Second-Order Optimizers in Deep Learning: Insights from Fisher Information Analysis | Apr 26, 2025 | Computational Efficiencyimage-classification | CodeCode Available | 2 | 5 |
| Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization | Jun 9, 2020 | Bayesian OptimizationSecond-order methods | CodeCode Available | 2 | 5 |
| Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization | Feb 7, 2020 | Second-order methods | CodeCode Available | 1 | 5 |
| SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation | Feb 25, 2025 | Second-order methods | CodeCode Available | 1 | 5 |
| MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates | Jun 2, 2023 | Second-order methods | CodeCode Available | 1 | 5 |
| M-FAC: Efficient Matrix-Free Approximations of Second-Order Information | Jul 7, 2021 | Network PruningSecond-order methods | CodeCode Available | 1 | 5 |
| MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation | Mar 27, 2020 | Bilevel OptimizationNeural Architecture Search | CodeCode Available | 1 | 5 |
| Near out-of-distribution detection for low-resolution radar micro-Doppler signatures | May 12, 2022 | Contrastive LearningGeometry-aware processing | CodeCode Available | 1 | 5 |
| Symmetry Teleportation for Accelerated Optimization | May 21, 2022 | Second-order methods | CodeCode Available | 1 | 5 |
| Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning | Jan 24, 2022 | Federated LearningSecond-order methods | CodeCode Available | 1 | 5 |
| Second-Order Neural ODE Optimizer | Sep 29, 2021 | image-classificationImage Classification | CodeCode Available | 1 | 5 |
| ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning | Jun 1, 2020 | BIG-bench Machine LearningSecond-order methods | CodeCode Available | 1 | 5 |
| Second-Order Stochastic Optimization for Machine Learning in Linear Time | Feb 12, 2016 | BIG-bench Machine LearningSecond-order methods | CodeCode Available | 1 | 5 |
| Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC | Dec 9, 2023 | Second-order methods | CodeCode Available | 1 | 5 |
| Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning | Jun 5, 2024 | reinforcement-learningReinforcement Learning | CodeCode Available | 1 | 5 |
| Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective | Feb 5, 2024 | Second-order methods | CodeCode Available | 1 | 5 |
| Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates | Dec 3, 2019 | Second-order methods | CodeCode Available | 0 | 5 |
| SGD with Partial Hessian for Deep Neural Networks Optimization | Mar 5, 2024 | image-classificationImage Classification | CodeCode Available | 0 | 5 |
| Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning | Dec 26, 2018 | BIG-bench Machine LearningSecond-order methods | CodeCode Available | 0 | 5 |
| A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks | Apr 10, 2024 | DiversityImage Generation | CodeCode Available | 0 | 5 |
| Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning | Jun 30, 2017 | BIG-bench Machine Learningregression | CodeCode Available | 0 | 5 |
| Studying K-FAC Heuristics by Viewing Adam through a Second-Order Lens | Oct 23, 2023 | Computational EfficiencySecond-order methods | CodeCode Available | 0 | 5 |
| Online Second Order Methods for Non-Convex Stochastic Optimizations | Mar 26, 2018 | Second-order methods | CodeCode Available | 0 | 5 |
| Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches | Jun 6, 2022 | Second-order methods | CodeCode Available | 0 | 5 |
| Adapting Newton's Method to Neural Networks through a Summary of Higher-Order Derivatives | Dec 6, 2023 | Second-order methods | CodeCode Available | 0 | 5 |
| Newtonian Monte Carlo: single-site MCMC meets second-order gradient methods | Jan 15, 2020 | Second-order methodsVariational Inference | CodeCode Available | 0 | 5 |
| SGD momentum optimizer with step estimation by online parabola model | Jul 16, 2019 | Second-order methods | CodeCode Available | 0 | 5 |
| Adaptive Consensus Optimization Method for GANs | Apr 20, 2023 | Image GenerationSecond-order methods | CodeCode Available | 0 | 5 |
| Sharpened Lazy Incremental Quasi-Newton Method | May 26, 2023 | Second-order methods | CodeCode Available | 0 | 5 |
| Nonlinear matrix recovery using optimization on the Grassmann manifold | Sep 13, 2021 | Riemannian optimizationSecond-order methods | CodeCode Available | 0 | 5 |
| LocoProp: Enhancing BackProp via Local Loss Optimization | Jun 11, 2021 | Second-order methods | CodeCode Available | 0 | 5 |
| Statistical Inference of Constrained Stochastic Optimization via Sketched Sequential Quadratic Programming | May 27, 2022 | Second-order methodsStochastic Optimization | CodeCode Available | 0 | 5 |
| AdaSub: Stochastic Optimization Using Second-Order Information in Low-Dimensional Subspaces | Oct 30, 2023 | ManagementSecond-order methods | CodeCode Available | 0 | 5 |
| NysAct: A Scalable Preconditioned Gradient Descent using Nystrom Approximation | Jun 10, 2025 | Second-order methods | CodeCode Available | 0 | 5 |
| Generalized Optimistic Methods for Convex-Concave Saddle Point Problems | Feb 19, 2022 | Second-order methods | CodeCode Available | 0 | 5 |
| FOSI: Hybrid First and Second Order Optimization | Feb 16, 2023 | Audio ClassificationLanguage Modelling | CodeCode Available | 0 | 5 |
| Improving SGD convergence by online linear regression of gradients in multiple statistically relevant directions | Jan 31, 2019 | regressionSecond-order methods | CodeCode Available | 0 | 5 |
| Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation | Oct 11, 2019 | Binary ClassificationSecond-order methods | CodeCode Available | 0 | 5 |
| Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization | Nov 12, 2024 | Second-order methods | CodeCode Available | 0 | 5 |
| A Computationally Efficient Sparsified Online Newton Method | Nov 16, 2023 | Second-order methods | CodeCode Available | 0 | 5 |
| Limitations of the Empirical Fisher Approximation for Natural Gradient Descent | May 29, 2019 | Second-order methods | CodeCode Available | 0 | 5 |
| FLeNS: Federated Learning with Enhanced Nesterov-Newton Sketch | Sep 23, 2024 | Dimensionality ReductionEdge-computing | CodeCode Available | 0 | 5 |
| ISAAC Newton: Input-based Approximate Curvature for Newton's Method | May 1, 2023 | Second-order methods | CodeCode Available | 0 | 5 |
| Differentially Private Image Classification from Features | Nov 24, 2022 | Classificationimage-classification | CodeCode Available | 0 | 5 |
| Alternating Iteratively Reweighted _1 and Subspace Newton Algorithms for Nonconvex Sparse Optimization | Jul 24, 2024 | Second-order methods | CodeCode Available | 0 | 5 |
| LIBS2ML: A Library for Scalable Second Order Machine Learning Algorithms | Apr 20, 2019 | BIG-bench Machine LearningSecond-order methods | CodeCode Available | 0 | 5 |
| Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training | Jun 16, 2020 | Second-order methods | CodeCode Available | 0 | 5 |
| On the Promise of the Stochastic Generalized Gauss-Newton Method for Training DNNs | Jun 3, 2020 | Second-order methods | CodeCode Available | 0 | 5 |
| Error Feedback Can Accurately Compress Preconditioners | Jun 9, 2023 | ClassificationSecond-order methods | CodeCode Available | 0 | 5 |
