SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Showing 11011150 of 1387 papers

TitleStatusHype
Towards Understanding Acceleration Tradeoff between Momentum and Asynchrony in Nonconvex Stochastic Optimization0
Probabilistic Joint Face-Skull Modelling for Facial Reconstruction0
A PID Controller Approach for Stochastic Optimization of Deep NetworksCode0
Kernel embedding of maps for sequential Bayesian inference: The variational mapping particle filterCode0
Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization0
Zeroth-Order Stochastic Variance Reduction for Nonconvex OptimizationCode0
Learning to Optimize Contextually Constrained Problems for Real-Time Decision-Generation0
Universal discriminative quantum neural networks0
Robust 6D Object Pose Estimation with Stochastic Congruent Sets0
Dyna: A Method of Momentum for Stochastic Optimization0
Robust Utility Maximization in Discrete-Time Markets with Friction0
Optimal Virtualization Framework for Cellular Networks with Downlink Rate Coverage Probability Constraints0
Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization0
Simultaneous shot inversion for nonuniform geometries using fast data interpolation0
Nonparametric Stochastic Compositional Gradient Descent for Q-Learning in Continuous Markov Decision ProblemsCode0
Adafactor: Adaptive Learning Rates with Sublinear Memory CostCode1
Towards Training Probabilistic Topic Models on Neuromorphic Multi-chip Systems0
Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalentsCode0
Byzantine Stochastic Gradient Descent0
Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates0
On the insufficiency of existing momentum schemes for Stochastic OptimizationCode0
Averaging Weights Leads to Wider Optima and Better GeneralizationCode1
The Challenge of Crafting Intelligible Intelligence0
A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization0
Sever: A Robust Meta-Algorithm for Stochastic OptimizationCode0
Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimization0
ADMM-based Networked Stochastic Variational Inference0
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency AnalysisCode0
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization0
Shampoo: Preconditioned Stochastic Tensor OptimizationCode1
Stochastic Hyperparameter Optimization through HypernetworksCode1
Distributed Stochastic Optimization via Adaptive SGD0
A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization0
Superposition-Assisted Stochastic Optimization for Hawkes Processes0
Stochastic Variance-Reduced Hamilton Monte Carlo Methods0
Online Variance Reduction for Stochastic OptimizationCode0
Robust and Sparse Regression in GLM by Stochastic Optimization0
DeepType: Multilingual Entity Linking by Neural Type System EvolutionCode0
When Does Stochastic Gradient Algorithm Work Well?0
Asynchronous Stochastic Variational Inference0
SpectralNet: Spectral Clustering using Deep Neural NetworksCode0
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods0
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients0
Third-order Smoothness Helps: Even Faster Stochastic Optimization Algorithms for Finding Local Minima0
Wasserstein Distributionally Robust Optimization and Variation Regularization0
Stochastic Particle Gradient Descent for Infinite Ensembles0
Proximodistal Exploration in Motor Learning as an Emergent Property of Optimization0
Comparative analysis of criteria for filtering time series of word usage frequencies0
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks0
ADMM without a Fixed Penalty Parameter: Faster Convergence with New Adaptive Penalization0
Show:102550
← PrevPage 23 of 28Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24Unverified
2AdaShiftAccuracy81.12Unverified
3Adam (eps-adjusted)Accuracy81.04Unverified
4SGDAccuracy80.95Unverified
5AdamWAccuracy79.87Unverified
6AdaBoundAccuracy77.24Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36Unverified
2AvaGradAccuracy96.2Unverified
3SGDAccuracy96.14Unverified
4AdaShiftAccuracy95.92Unverified
5AdamWAccuracy95.89Unverified
6AdaBoundAccuracy94.6Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55Unverified
2LookaheadAccuracy95.27Unverified
3SGDAccuracy95.23Unverified
4ADAMAccuracy94.84Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51Unverified
2SGDTop 1 Accuracy75.99Unverified
3AdamWTop 1 Accuracy72.9Unverified
4AdaBoundTop 1 Accuracy72.01Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86Unverified
2AdaShiftBit per Character (BPC)1.27Unverified
3AdamWBit per Character (BPC)1.23Unverified
4AvaGradBit per Character (BPC)1.18Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85Unverified
2Resnet34Accuracy (max)86.14Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48Unverified
2Resnet34Accuracy (max)54.5Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15Unverified
2LookaheadTop 1 Accuracy75.13Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49Unverified
2SGDTop 1 Accuracy75.15Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05Unverified