Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1126–1150 of 1387 papers

Title	Date	Tasks	Status	Hype
Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimization	Mar 6, 2018	Dimensionality ReductionStochastic Optimization	—Unverified	0
ADMM-based Networked Stochastic Variational Inference	Feb 27, 2018	ArticlesDocument Classification	—Unverified	0
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis	Feb 26, 2018	Deep LearningNeural Architecture Search	CodeCode Available	0
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization	Feb 26, 2018	Stochastic Optimization	—Unverified	0
Shampoo: Preconditioned Stochastic Tensor Optimization	Feb 26, 2018	Stochastic Optimization	CodeCode Available	1
Stochastic Hyperparameter Optimization through Hypernetworks	Feb 26, 2018	BIG-bench Machine LearningHyperparameter Optimization	CodeCode Available	1
Distributed Stochastic Optimization via Adaptive SGD	Feb 16, 2018	Stochastic Optimization	—Unverified	0
A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization	Feb 14, 2018	Bayesian InferenceDimensionality Reduction	—Unverified	0
Superposition-Assisted Stochastic Optimization for Hawkes Processes	Feb 13, 2018	DiversityRecommendation Systems	—Unverified	0
Stochastic Variance-Reduced Hamilton Monte Carlo Methods	Feb 13, 2018	Stochastic Optimization	—Unverified	0
Online Variance Reduction for Stochastic Optimization	Feb 13, 2018	Stochastic Optimization	CodeCode Available	0
Robust and Sparse Regression in GLM by Stochastic Optimization	Feb 9, 2018	regressionStochastic Optimization	—Unverified	0
DeepType: Multilingual Entity Linking by Neural Type System Evolution	Feb 3, 2018	Entity DisambiguationEntity Embeddings	CodeCode Available	0
When Does Stochastic Gradient Algorithm Work Well?	Jan 18, 2018	General Classificationregression	—Unverified	0
Asynchronous Stochastic Variational Inference	Jan 12, 2018	Stochastic OptimizationVariational Inference	—Unverified	0
SpectralNet: Spectral Clustering using Deep Neural Networks	Jan 4, 2018	ClusteringStochastic Optimization	CodeCode Available	0
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods	Dec 27, 2017	Stochastic Optimization	—Unverified	0
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients	Dec 20, 2017	Stochastic OptimizationUncertainty Quantification	—Unverified	0
Third-order Smoothness Helps: Even Faster Stochastic Optimization Algorithms for Finding Local Minima	Dec 18, 2017	Stochastic Optimization	—Unverified	0
Wasserstein Distributionally Robust Optimization and Variation Regularization	Dec 17, 2017	Stochastic Optimization	—Unverified	0
Stochastic Particle Gradient Descent for Infinite Ensembles	Dec 14, 2017	Ensemble LearningStochastic Optimization	—Unverified	0
Proximodistal Exploration in Motor Learning as an Emergent Property of Optimization	Dec 14, 2017	Stochastic Optimization	—Unverified	0
Comparative analysis of criteria for filtering time series of word usage frequencies	Dec 10, 2017	Stochastic OptimizationTime Series	—Unverified	0
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks	Dec 8, 2017	Stochastic Optimization	—Unverified	0
ADMM without a Fixed Penalty Parameter: Faster Convergence with New Adaptive Penalization	Dec 1, 2017	Stochastic Optimization	—Unverified	0

Show:10 25 50

← PrevPage 46 of 56Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified