SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Showing 9511000 of 1387 papers

TitleStatusHype
A Note On The Popularity of Stochastic Optimization Algorithms in Different Fields: A Quantitative Analysis from 2007 to 20170
Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning0
DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERMCode0
Topic Modeling via Full Dependence MixturesCode0
Optimal Convergence for Stochastic Optimization with Multiple Expectation Constraints0
Online Forecasting of Total-Variation-bounded SequencesCode0
Reducing the variance in online optimization by transporting past gradientsCode0
Universal Boosting Variational InferenceCode0
A Generic Acceleration Framework for Stochastic Composite OptimizationCode0
Data-Pooling in Stochastic OptimizationCode0
Beyond Volumetric Albedo -- A Surface Optimization Framework for Non-Line-Of-Sight Imaging0
ADMM for Efficient Deep Learning with Global ConvergenceCode1
Don't Forget Your Teacher: A Corrective Reinforcement Learning Framework0
Exploiting Uncertainty of Loss Landscape for Stochastic OptimizationCode0
On stochastic gradient Langevin dynamics with dependent data streams: the fully non-convex case0
Sample Complexity of Sample Average Approximation for Conditional Stochastic Optimization0
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep NetworksCode1
A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization0
AI-CARGO: A Data-Driven Air-Cargo Revenue Management System0
Cloud Storage for Multi-Service Battery Operation (Extended Version)0
Approximation of the objective insensitivity regions using Hierarchic Memetic Strategy coupled with Covariance Matrix Adaptation Evolutionary Strategy0
Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization0
Trajectory-Based Off-Policy Deep Reinforcement LearningCode0
Non-cooperative Aerial Base Station Placement via Stochastic OptimizationCode0
On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization0
A Contrastive Divergence for Combining Variational Inference and MCMCCode0
Stochastic Iterative Hard Thresholding for Graph-structured Sparsity OptimizationCode0
Smoothing Policies and Safe Policy Gradients0
A Latent Variational Framework for Stochastic Optimization0
Accelerating first order optimization algorithms0
signSGD via Zeroth-Order Oracle0
A Walk with SGD: How SGD Explores Regions of Deep Network Loss?0
A preconditioned accelerated stochastic gradient descent algorithm0
Rethinking learning rate schedules for stochastic optimization0
On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics0
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least SquaresCode0
Beyond Adaptive Submodularity: Approximation Guarantees of Greedy Policy with Adaptive Submodularity Ratio0
On the Convergence of Adam and BeyondCode0
Constrained Deep Networks: Lagrangian Optimization via Log-Barrier ExtensionsCode0
Adaptive Sequential Machine Learning0
Large Batch Optimization for Deep Learning: Training BERT in 76 minutesCode1
An Upper Bound for Minimum True Matches in Graph Isomorphism with Simulated Annealing0
Online Variance Reduction with MixturesCode0
On the Influence of Bias-Correction on Distributed Stochastic Optimization0
Stochastic Optimization of Sorting Networks via Continuous RelaxationsCode0
Traversing the noise of dynamic mini-batch sub-sampled loss functions: A visual guide0
The importance of better models in stochastic optimizationCode0
Distributed stochastic optimization with gradient tracking over strongly-connected networks0
Inefficiency of K-FAC for Large Batch Size Training0
DeepOBS: A Deep Learning Optimizer Benchmark SuiteCode0
Show:102550
← PrevPage 20 of 28Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24Unverified
2AdaShiftAccuracy81.12Unverified
3Adam (eps-adjusted)Accuracy81.04Unverified
4SGDAccuracy80.95Unverified
5AdamWAccuracy79.87Unverified
6AdaBoundAccuracy77.24Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36Unverified
2AvaGradAccuracy96.2Unverified
3SGDAccuracy96.14Unverified
4AdaShiftAccuracy95.92Unverified
5AdamWAccuracy95.89Unverified
6AdaBoundAccuracy94.6Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55Unverified
2LookaheadAccuracy95.27Unverified
3SGDAccuracy95.23Unverified
4ADAMAccuracy94.84Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51Unverified
2SGDTop 1 Accuracy75.99Unverified
3AdamWTop 1 Accuracy72.9Unverified
4AdaBoundTop 1 Accuracy72.01Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86Unverified
2AdaShiftBit per Character (BPC)1.27Unverified
3AdamWBit per Character (BPC)1.23Unverified
4AvaGradBit per Character (BPC)1.18Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85Unverified
2Resnet34Accuracy (max)86.14Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48Unverified
2Resnet34Accuracy (max)54.5Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15Unverified
2LookaheadTop 1 Accuracy75.13Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49Unverified
2SGDTop 1 Accuracy75.15Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05Unverified