Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1251–1300 of 1387 papers

Title	Date	Tasks	Status	Hype
SEBOOST - Boosting Stochastic Learning Using Subspace Optimization Techniques	Sep 2, 2016	Stochastic Optimization	CodeCode Available	0
Data Dependent Convergence for Distributed Stochastic Optimization	Aug 30, 2016	Stochastic Optimization	—Unverified	0
SGDR: Stochastic Gradient Descent with Warm Restarts	Aug 13, 2016	EEGElectroencephalogram (EEG)	CodeCode Available	1
Learning Structured Predictors from Bandit Feedback for Interactive NLP	Aug 1, 2016	Machine TranslationStochastic Optimization	—Unverified	0
Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization	Jul 5, 2016	Binary ClassificationGeneral Classification	—Unverified	0
Accelerate Stochastic Subgradient Method by Leveraging Local Growth Condition	Jul 4, 2016	Stochastic Optimization	—Unverified	0
Convolutional Sketch Inversion	Jun 9, 2016	Stochastic Optimization	CodeCode Available	0
Distributed stochastic optimization via matrix exponential learning	Jun 3, 2016	Stochastic Optimizationvalid	—Unverified	0
Learning Weight Uncertainty With Stochastic Gradient MCMC for Shape Classification	Jun 1, 2016	General ClassificationStochastic Optimization	—Unverified	0
Simultaneous Clustering and Model Selection for Tensor Affinities	Jun 1, 2016	Clusteringmodel	—Unverified	0
CYCLADES: Conflict-free Asynchronous Machine Learning	May 31, 2016	BIG-bench Machine LearningStochastic Optimization	CodeCode Available	0
Adaptive Learning Rate via Covariance Matrix Based Preconditioning for Deep Neural Networks	May 31, 2016	Stochastic Optimization	—Unverified	0
Stochastic Optimization for Large-scale Optimal Transport	May 27, 2016	Stochastic Optimization	—Unverified	0
Kronecker Determinantal Point Processes	May 26, 2016	Point ProcessesStochastic Optimization	CodeCode Available	0
Faster Eigenvector Computation via Shift-and-Invert Preconditioning	May 26, 2016	Stochastic Optimization	—Unverified	0
Effective Blind Source Separation Based on the Adam Algorithm	May 25, 2016	blind source separationStochastic Optimization	—Unverified	0
NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization	May 25, 2016	Stochastic Optimization	—Unverified	0
Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks	May 23, 2016	Model-based Reinforcement Learningreinforcement-learning	CodeCode Available	0
Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds	May 23, 2016	Riemannian optimizationStochastic Optimization	—Unverified	0
The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response	May 18, 2016	Stochastic Optimization	—Unverified	0
Reducing the Model Order of Deep Neural Networks Using Information Theory	May 16, 2016	General ClassificationNetwork Pruning	—Unverified	0
Barzilai-Borwein Step Size for Stochastic Gradient Descent	May 13, 2016	Stochastic Optimization	CodeCode Available	0
Nonconvex Sparse Learning via Stochastic Optimization with Progressive Variance Reduction	May 9, 2016	parameter estimationSparse Learning	—Unverified	0
Distributed stochastic optimization for deep learning (thesis)	May 7, 2016	Deep Learningimage-classification	—Unverified	0
Algorithms for stochastic optimization with functional or expectation constraints	Apr 13, 2016	Stochastic Optimization	—Unverified	0
Efficient Globally Convergent Stochastic Optimization for Canonical Correlation Analysis	Apr 7, 2016	Stochastic Optimization	—Unverified	0
Revisiting Distributed Synchronous SGD	Apr 4, 2016	Stochastic Optimization	CodeCode Available	1
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods	Mar 18, 2016	Stochastic Optimization	—Unverified	0
Without-Replacement Sampling for Stochastic Gradient Methods: Convergence Results and Application to Distributed Optimization	Mar 2, 2016	Distributed OptimizationLearning Theory	—Unverified	0
Scalable Metric Learning via Weighted Approximate Rank Component Analysis	Mar 1, 2016	Metric LearningPerson Re-Identification	—Unverified	0
Second-Order Stochastic Optimization for Machine Learning in Linear Time	Feb 12, 2016	BIG-bench Machine LearningSecond-order methods	CodeCode Available	1
Improved Dropout for Shallow and Deep Learning	Feb 6, 2016	Deep LearningStochastic Optimization	—Unverified	0
Variance-Reduced and Projection-Free Stochastic Optimization	Feb 5, 2016	Stochastic Optimization	—Unverified	0
A Kronecker-factored approximate Fisher matrix for convolution layers	Feb 3, 2016	Stochastic Optimization	CodeCode Available	0
On Deep Multi-View Representation Learning: Objectives and Optimization	Feb 2, 2016	Representation LearningStochastic Optimization	CodeCode Available	0
SCOPE: Scalable Composite Optimization for Learning on Spark	Jan 30, 2016	Stochastic Optimization	CodeCode Available	0
Variational Inference: A Review for Statisticians	Jan 4, 2016	Stochastic OptimizationVariational Inference	CodeCode Available	1
Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization	Dec 25, 2015	Stochastic Optimization	CodeCode Available	0
Efficient Distributed SGD with Variance Reduction	Dec 9, 2015	Stochastic Optimization	—Unverified	0
Variance Reduction for Distributed Stochastic Gradient Descent	Dec 5, 2015	Stochastic Optimization	—Unverified	0
A Projection Free Method for Generalized Eigenvalue Problem With a Nonsmooth Regularizer	Dec 1, 2015	Image SegmentationSemantic Segmentation	—Unverified	0
Adaptive Stochastic Optimization: From Sets to Paths	Dec 1, 2015	Stochastic Optimization	—Unverified	0
Local Smoothness in Variance Reduced Optimization	Dec 1, 2015	Stochastic Optimization	—Unverified	0
Fast Rates for Exp-concave Empirical Risk Minimization	Dec 1, 2015	regressionStochastic Optimization	—Unverified	0
Asynchronous stochastic convex optimization: the noise is in the noise and SGD don't care	Dec 1, 2015	Stochastic Optimization	—Unverified	0
Constant Time EXPected Similarity Estimation using Stochastic Optimization	Nov 17, 2015	Anomaly DetectionStochastic Optimization	—Unverified	0
Joint Training of Generic CNN-CRF Models with Stochastic Optimization	Nov 16, 2015	Stochastic Optimization	—Unverified	0
Large-Scale Approximate Kernel Canonical Correlation Analysis	Nov 15, 2015	Representation LearningStochastic Optimization	—Unverified	0
Seeing the Unseen Network: Inferring Hidden Social Ties from Respondent-Driven Sampling	Nov 13, 2015	Stochastic OptimizationTime Series	—Unverified	0
Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?	Nov 11, 2015	AstronomyStochastic Optimization	—Unverified	0

Show:10 25 50

← PrevPage 26 of 28Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified