SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Showing 51–100 of 1387 papers

TitleStatusHype
Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image ClassificationCode1
Bi-level Score Matching for Learning Energy-based Latent Variable ModelsCode1
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic OptimizationCode1
Reinforcement Learning for Low-Thrust Trajectory Design of Interplanetary MissionsCode1
Stochastic Optimization ForestsCode1
Randomized Automatic DifferentiationCode1
A Better Alternative to Error Feedback for Communication-Efficient Distributed LearningCode1
Personalized Federated Learning with Moreau EnvelopesCode1
Stochastic Optimization for Performative PredictionCode1
Kernel Distributionally Robust OptimizationCode1
Quadruply Stochastic Gaussian ProcessesCode1
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine LearningCode1
An Analysis of the Adaptation Speed of Causal ModelsCode1
Distributionally Robust Neural NetworksCode1
Progressive Identification of True Labels for Partial-Label LearningCode1
PolyFold: an interactive visual simulator for distance-based protein foldingCode1
Adaptivity of Stochastic Gradient Methods for Nonconvex OptimizationCode1
PACOH: Bayes-Optimal Meta-Learning with PAC-GuaranteesCode1
Self-Directed Online Machine Learning for Topology OptimizationCode1
ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization ProblemsCode1
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case GeneralizationCode1
Federated Learning over Wireless Networks: Convergence Analysis and Resource AllocationCode1
On the Variance of the Adaptive Learning Rate and BeyondCode1
Lookahead Optimizer: k steps forward, 1 step backCode1
ADMM for Efficient Deep Learning with Global ConvergenceCode1
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep NetworksCode1
Large Batch Optimization for Deep Learning: Training BERT in 76 minutesCode1
Cyclical Stochastic Gradient MCMC for Bayesian Deep LearningCode1
Agnostic Federated LearningCode1
Learning concise representations for regression by evolving networks of treesCode1
Spectral Inference Networks: Unifying Deep and Spectral LearningCode1
Adafactor: Adaptive Learning Rates with Sublinear Memory CostCode1
Averaging Weights Leads to Wider Optima and Better GeneralizationCode1
Stochastic Hyperparameter Optimization through HypernetworksCode1
Shampoo: Preconditioned Stochastic Tensor OptimizationCode1
Training Deep Networks without Learning Rates Through Coin BettingCode1
Online Learning Rate Adaptation with Hypergradient DescentCode1
Deep Generalized Canonical Correlation AnalysisCode1
SGDR: Stochastic Gradient Descent with Warm RestartsCode1
Revisiting Distributed Synchronous SGDCode1
Second-Order Stochastic Optimization for Machine Learning in Linear TimeCode1
Variational Inference: A Review for StatisticiansCode1
Optimizing Neural Networks with Kronecker-factored Approximate CurvatureCode1
Adam: A Method for Stochastic OptimizationCode1
First-order methods for stochastic and finite-sum convex optimization with deterministic constraints—0
Convergence of Momentum-Based Optimization Algorithms with Time-Varying Parameters—0
The Sample Complexity of Parameter-Free Stochastic Convex Optimization—0
Underage Detection through a Multi-Task and MultiAge Approach for Screening Minors in Unconstrained Imagery—0
"What are my options?": Explaining RL Agents with Diverse Near-Optimal Alternatives (Extended)—0
Distribution free M-estimation—0
Show:102550
← PrevPage 2 of 28Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24—Unverified
2AdaShiftAccuracy81.12—Unverified
3Adam (eps-adjusted)Accuracy81.04—Unverified
4SGDAccuracy80.95—Unverified
5AdamWAccuracy79.87—Unverified
6AdaBoundAccuracy77.24—Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36—Unverified
2AvaGradAccuracy96.2—Unverified
3SGDAccuracy96.14—Unverified
4AdaShiftAccuracy95.92—Unverified
5AdamWAccuracy95.89—Unverified
6AdaBoundAccuracy94.6—Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55—Unverified
2LookaheadAccuracy95.27—Unverified
3SGDAccuracy95.23—Unverified
4ADAMAccuracy94.84—Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51—Unverified
2SGDTop 1 Accuracy75.99—Unverified
3AdamWTop 1 Accuracy72.9—Unverified
4AdaBoundTop 1 Accuracy72.01—Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86—Unverified
2AdaShiftBit per Character (BPC)1.27—Unverified
3AdamWBit per Character (BPC)1.23—Unverified
4AvaGradBit per Character (BPC)1.18—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85—Unverified
2Resnet34Accuracy (max)86.14—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48—Unverified
2Resnet34Accuracy (max)54.5—Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15—Unverified
2LookaheadTop 1 Accuracy75.13—Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49—Unverified
2SGDTop 1 Accuracy75.15—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34—Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05—Unverified