Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1076–1100 of 1387 papers

Title	Date	Tasks	Status
Accelerating Stochastic Probabilistic Inference	Mar 15, 2022	Second-order methodsStochastic Optimization	—Unverified
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization	Aug 3, 2023	Model CompressionNetwork Pruning	—Unverified
A Class of Short-term Recurrence Anderson Mixing Methods and Their Applications	Sep 29, 2021	image-classificationImage Classification	—Unverified
A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret	Jan 21, 2023	Federated LearningStochastic Optimization	—Unverified
A Communication Efficient Stochastic Multi-Block Alternating Direction Method of Multipliers	Dec 1, 2019	Stochastic Optimization	—Unverified
A Comparison of Monte Carlo Tree Search and Mathematical Optimization for Large Scale Dynamic Resource Allocation	May 21, 2014	ManagementStochastic Optimization	—Unverified
A Computational Efficient Pumped Storage Hydro Optimization in the Look-ahead Unit Commitment and Real-time Market Dispatch Under Uncertainty	Apr 7, 2023	Stochastic Optimization	—Unverified
A Continuous-time Stochastic Gradient Descent Method for Continuous Data	Dec 7, 2021	Stochastic OptimizationVariational Inference	—Unverified
A Convergence Analysis for A Class of Practical Variance-Reduction Stochastic Gradient MCMC	Sep 4, 2017	Stochastic Optimization	—Unverified
A Coreset-based, Tempered Variational Posterior for Accurate and Scalable Stochastic Gaussian Process Inference	Nov 2, 2023	Stochastic Optimization	—Unverified
Adam^+: A Stochastic Method with Adaptive Variance Reduction	Nov 24, 2020	Automatic Speech RecognitionAutomatic Speech Recognition (ASR)	—Unverified
On the Trend-corrected Variant of Adaptive Stochastic Optimization Methods	Jan 17, 2020	Computational EfficiencyStochastic Optimization	—Unverified
Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning	Nov 29, 2021	BIG-bench Machine LearningDeep Learning	—Unverified
Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems	May 19, 2020	Stochastic Optimization	—Unverified
Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizes	Mar 23, 2021	Stochastic Optimization	—Unverified
Adaptive Learning Rates for Faster Stochastic Gradient Methods	Aug 10, 2022	DiversityStochastic Optimization	—Unverified
Adaptive Learning Rate via Covariance Matrix Based Preconditioning for Deep Neural Networks	May 31, 2016	Stochastic Optimization	—Unverified
Adaptive Online Estimation of Piecewise Polynomial Trends	Sep 30, 2020	2kregression	—Unverified
Adaptive Sampling Quasi-Newton Methods for Derivative-Free Stochastic Optimization	Oct 29, 2019	reinforcement-learningReinforcement Learning	—Unverified
Adaptive Sampling Quasi-Newton Methods for Zeroth-Order Stochastic Optimization	Sep 24, 2021	Reinforcement Learning (RL)Stochastic Optimization	—Unverified
Adaptive Sampling Strategies for Stochastic Optimization	Oct 30, 2017	regressionStochastic Optimization	—Unverified
Adaptive Sequential Machine Learning	Apr 4, 2019	BIG-bench Machine LearningModel Selection	—Unverified
Adaptive Shells for Efficient Neural Radiance Field Rendering	Nov 16, 2023	Novel View SynthesisStochastic Optimization	—Unverified
Adaptive Single-Pass Stochastic Gradient Descent in Input Sparsity Time	Jan 1, 2021	Second-order methodsStochastic Optimization	—Unverified
Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent	Nov 28, 2023	image-classificationImage Classification	—Unverified

Show:10 25 50

← PrevPage 44 of 56Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified