SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 701–750 of 1387 papers

TitleStatusHype
Stochastic Gradient Variance Reduction by Solving a Filtering ProblemCode1
Learning from History for Byzantine Robust OptimizationCode1
Stochastic Compositional Gradient Descent under Compositional Constraints—0
Strong overall error analysis for the training of artificial neural networks via random initializations—0
Online Stochastic Optimization with Wasserstein Based Non-stationarity—0
Quality-Diversity Optimization: a novel branch of stochastic optimizationCode1
Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance—0
Large-scale Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image ClassificationCode1
Learning with risks based on M-locationCode0
Sampling from a k-DPP without looking at all items—0
The Devil is in the Detail: A Framework for Macroscopic Prediction via Microscopic Models—0
Escaping Saddle-Point Faster under Interpolation-like Conditions—0
Adam^+: A Stochastic Method with Adaptive Variance Reduction—0
Optimal Algorithms for Convex Nested Stochastic Composite Optimization—0
State-of-charge Estimation of a Li-ion Battery using Deep Learning and Stochastic Optimization—0
Mixing ADAM and SGD: a Combined Optimization MethodCode0
Improving Offline Contextual Bandits with Distributional Robustness—0
Towards Optimal Problem Dependent Generalization Error Bounds in Statistical Learning Theory—0
Non-local Optimization: Imposing Structure on Optimization Problems by Relaxation—0
Neural Network Compression Via Sparse Optimization—0
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering—0
Finite-Time Convergence Rates of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning—0
Stochastic Optimization with Laggard Data Pipelines—0
Graph-Homomorphic Perturbations for Private Decentralized Learning—0
How Data Augmentation affects Optimization for Linear Regression—0
Progressive Batching for Efficient Non-linear Least SquaresCode0
Dual Averaging is Surprisingly Effective for Deep Learning Optimization—0
Bi-level Score Matching for Learning Energy-based Latent Variable ModelsCode1
AlphaZero Based Post-Storm Repair Crew Dispatch for Distribution Grid Restoration—0
Regret minimization in stochastic non-convex learning via a proximal-gradient approach—0
Three-Dimensional Swarming Using Cyclic Stochastic Optimization—0
Online Learning and Distributed Control for Residential Demand Response—0
Practical Precoding via Asynchronous Stochastic Successive Convex ApproximationCode0
Learning the Step-size Policy for the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno Algorithm—0
Expectigrad: Fast Stochastic Optimization with Robust Convergence PropertiesCode0
Deep FPF: Gain function approximation in high-dimensional setting—0
Variance-Reduced Methods for Machine Learning—0
Adaptive Online Estimation of Piecewise Polynomial Trends—0
Data augmentation as stochastic optimization—0
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic OptimizationCode1
f-Divergence Variational Inference—0
Escaping Saddle-Points Faster under Interpolation-like Conditions—0
Scheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning—0
Stochastic Learning-Based Robust Beamforming Design for RIS-Aided Millimeter-Wave Systems in the Presence of Random Blockages—0
Finding Acceptable Parameter Regions of Stochastic Hill functions for Multisite Phosphorylation Mechanism—0
Variance-Reduced Off-Policy Memory-Efficient Policy Search—0
Generalized Multi-Output Gaussian Process Censored RegressionCode0
GTAdam: Gradient Tracking with Adaptive Momentum for Distributed Online Optimization—0
Cost-aware Feature Selection for IoT Device Classification—0
Robust, Accurate Stochastic Optimization for Variational Inference—0
Show:102550
← PrevPage 15 of 28Next →
All datasetsCIFAR-100 WRN-28-10 - 200 EpochsCIFAR-10 WRN-28-10 - 200 EpochsCIFAR-10 ResNet-18 - 200 EpochsImageNet ResNet-50 - 90 EpochsPenn Treebank (Character Level) 3x1000 LSTM - 500 EpochsCIFAR-10CIFAR-100ImageNet ResNet-50 - 50 EpochsImageNet ResNet-50 - 60 EpochsAG NewsCoLAMNIST

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24—Unverified
2AdaShiftAccuracy81.12—Unverified
3Adam (eps-adjusted)Accuracy81.04—Unverified
4SGDAccuracy80.95—Unverified
5AdamWAccuracy79.87—Unverified
6AdaBoundAccuracy77.24—Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36—Unverified
2AvaGradAccuracy96.2—Unverified
3SGDAccuracy96.14—Unverified
4AdaShiftAccuracy95.92—Unverified
5AdamWAccuracy95.89—Unverified
6AdaBoundAccuracy94.6—Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55—Unverified
2LookaheadAccuracy95.27—Unverified
3SGDAccuracy95.23—Unverified
4ADAMAccuracy94.84—Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51—Unverified
2SGDTop 1 Accuracy75.99—Unverified
3AdamWTop 1 Accuracy72.9—Unverified
4AdaBoundTop 1 Accuracy72.01—Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86—Unverified
2AdaShiftBit per Character (BPC)1.27—Unverified
3AdamWBit per Character (BPC)1.23—Unverified
4AvaGradBit per Character (BPC)1.18—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85—Unverified
2Resnet34Accuracy (max)86.14—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48—Unverified
2Resnet34Accuracy (max)54.5—Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15—Unverified
2LookaheadTop 1 Accuracy75.13—Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49—Unverified
2SGDTop 1 Accuracy75.15—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34—Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05—Unverified