Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 501–525 of 1387 papers

Title	Date	Tasks	Status
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization	Jun 20, 2017	Stochastic Optimization	—Unverified
Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimization	Aug 7, 2019	Multi-agent Reinforcement LearningReinforcement Learning	—Unverified
Fast Power Control Adaptation via Meta-Learning for Random Edge Graph Neural Networks	May 2, 2021	Graph Neural NetworkMeta-Learning	—Unverified
Fast Rates for Bandit Optimization with Upper-Confidence Frank-Wolfe	Feb 22, 2017	BIG-bench Machine LearningStochastic Optimization	—Unverified
Fast Rates for Bandit PAC Multiclass Classification	Jun 18, 2024	ClassificationPAC learning	—Unverified
Fast Rates for Exp-concave Empirical Risk Minimization	Dec 1, 2015	regressionStochastic Optimization	—Unverified
Distributed Evolution Strategies for Black-box Stochastic Optimization	Apr 9, 2022	Evolutionary AlgorithmsStochastic Optimization	—Unverified
Fast Stochastic Algorithms for Low-rank and Nonsmooth Matrix Problems	Sep 27, 2018	Stochastic Optimization	—Unverified
Distributed Delayed Stochastic Optimization	Dec 1, 2011	Distributed OptimizationStochastic Optimization	—Unverified
f-Divergence Variational Inference	Sep 28, 2020	Stochastic OptimizationVariational Inference	—Unverified
Distributed and Stochastic Optimization Methods with Gradient Compression and Local Steps	Dec 20, 2021	Stochastic Optimization	—Unverified
Dissimilarity Coefficient based Weakly Supervised Object Detection	Nov 25, 2018	Objectobject-detection	—Unverified
A Unified Analysis of Stochastic Optimization Methods Using Jump System Theory and Quadratic Constraints	Jun 25, 2017	Stochastic Optimization	—Unverified
A Unified Analysis on the Subgradient Upper Bounds for the Subgradient Methods Minimizing Composite Nonconvex, Nonsmooth and Non-Lipschitz Functions	Aug 30, 2023	Stochastic Optimization	—Unverified
Disentangled Representation Learning and Generation with Manifold Optimization	Jun 12, 2020	DisentanglementRepresentation Learning	—Unverified
Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base	Mar 30, 2025	Semantic SimilaritySemantic Textual Similarity	—Unverified
Augmentation Invariant Manifold Learning	Nov 1, 2022	Data AugmentationRepresentation Learning	—Unverified
A General Stochastic Optimization Framework for Convergence Bidding	Oct 12, 2022	SchedulingStochastic Optimization	—Unverified
Direct Acceleration of SAGA using Sampled Negative Momentum	Jun 28, 2018	Stochastic Optimization	—Unverified
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning	Sep 29, 2021	Reinforcement Learning (RL)Stochastic Optimization	—Unverified
Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimization	Mar 6, 2018	Dimensionality ReductionStochastic Optimization	—Unverified
Gamifying the Vehicle Routing Problem with Stochastic Requests	Nov 14, 2019	Atari GamesDecision Making	—Unverified
A Generalized Version of Chung's Lemma and its Applications	Jun 9, 2024	LEMMAStochastic Optimization	—Unverified
Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning	Nov 29, 2021	BIG-bench Machine LearningDeep Learning	—Unverified
Accelerating SGD for Distributed Deep-Learning Using Approximated Hessian Matrix	Sep 15, 2017	Deep LearningSecond-order methods	—Unverified

Show:10 25 50

← PrevPage 21 of 56Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified