Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 901–950 of 1387 papers

Title	Date	Tasks	Status	Hype
Adaptive Sampling Quasi-Newton Methods for Derivative-Free Stochastic Optimization	Oct 29, 2019	reinforcement-learningReinforcement Learning	—Unverified	0
Proactive Optimization with Machine Learning: Femto-caching with Future Content Popularity	Oct 29, 2019	BIG-bench Machine LearningStochastic Optimization	—Unverified	0
Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation	Oct 29, 2019	Federated LearningPrivacy Preserving	CodeCode Available	1
Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching	Oct 27, 2019	Bayesian OptimizationHyperparameter Optimization	CodeCode Available	0
An Adaptive and Momental Bound Method for Stochastic Learning	Oct 27, 2019	Stochastic Optimization	CodeCode Available	0
Gradient Sparification for Asynchronous Distributed Training	Oct 24, 2019	Federated LearningStochastic Optimization	—Unverified	0
Katyusha Acceleration for Convex Finite-Sum Compositional Optimization	Oct 24, 2019	Stochastic Optimization	—Unverified	0
The Practicality of Stochastic Optimization in Imaging Inverse Problems	Oct 22, 2019	DeblurringImage Deblurring	—Unverified	0
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks	Oct 19, 2019	Stochastic Optimization	—Unverified	0
Robust Learning Rate Selection for Stochastic Optimization via Splitting Diagnostic	Oct 18, 2019	DiagnosticStochastic Optimization	—Unverified	0
Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach	Oct 13, 2019	Stochastic Optimization	—Unverified	0
Projection-free nonconvex stochastic optimization on Riemannian manifolds	Oct 9, 2019	Stochastic Optimization	—Unverified	0
Variance-Reduced Decentralized Stochastic Optimization with Gradient Tracking -- Part II: GT-SVRG	Oct 8, 2019	Stochastic Optimization	—Unverified	0
FIS-GAN: GAN with Flow-based Importance Sampling	Oct 6, 2019	Density EstimationStochastic Optimization	—Unverified	0
Tightening Bounds for Variational Inference by Revisiting Perturbation Theory	Sep 30, 2019	Gaussian ProcessesStochastic Optimization	—Unverified	0
Model Predictive Control-Based Battery Scheduling and Incentives to Manipulate Demand Response Baselines	Sep 26, 2019	Model Predictive ControlScheduling	—Unverified	0
Statistical Adaptive Stochastic Optimization	Sep 25, 2019	SchedulingStochastic Optimization	—Unverified	0
Localizing and Amortizing: Efficient Inference for Gaussian Processes	Sep 25, 2019	Gaussian ProcessesStochastic Optimization	—Unverified	0
Accelerating First-Order Optimization Algorithms	Sep 25, 2019	Stochastic Optimization	—Unverified	0
Learning to Anneal and Prune Proximity Graphs for Similarity Search	Sep 25, 2019	Stochastic Optimization	—Unverified	0
A Non-asymptotic comparison of SVRG and SGD: tradeoffs between compute and speed	Sep 25, 2019	Computational Efficiencyregression	—Unverified	0
α^α-Rank: Practically Scaling α-Rank through Stochastic Optimisation	Sep 25, 2019	Stochastic Optimization	—Unverified	0
Geometry, Computation, and Optimality in Stochastic Optimization	Sep 23, 2019	Stochastic Optimization	—Unverified	0
Using Statistics to Automate Stochastic Optimization	Sep 21, 2019	Stochastic Optimization	—Unverified	0
Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games	Sep 18, 2019	reinforcement-learningReinforcement Learning	—Unverified	0
Communication-Censored Distributed Stochastic Gradient Descent	Sep 9, 2019	QuantizationStochastic Optimization	CodeCode Available	0
Distributionally Robust Optimization with Correlated Data from Vector Autoregressive Processes	Sep 8, 2019	Stochastic Optimization	—Unverified	0
Multi-Level Composite Stochastic Optimization via Nested Variance Reduction	Aug 29, 2019	Stochastic Optimization	—Unverified	0
Stochastic Optimization for Non-convex Inf-Projection Problems	Aug 26, 2019	Stochastic Optimization	—Unverified	0
RATQ: A Universal Fixed-Length Quantizer for Stochastic Optimization	Aug 22, 2019	QuantizationStochastic Optimization	—Unverified	0
On the Variance of the Adaptive Learning Rate and Beyond	Aug 8, 2019	image-classificationImage Classification	CodeCode Available	1
Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimization	Aug 7, 2019	Multi-agent Reinforcement LearningReinforcement Learning	—Unverified	0
Gradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimization	Aug 2, 2019	BIG-bench Machine LearningDistributed Optimization	—Unverified	0
From low probability to high confidence in stochastic convex optimization	Jul 31, 2019	Stochastic OptimizationVocal Bursts Intensity Prediction	—Unverified	0
Optimal Dynamic Multi-Resource Management in Earth Observation Oriented Space Information Networks	Jul 30, 2019	Earth ObservationManagement	—Unverified	0
Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization	Jul 29, 2019	Stochastic Optimization	—Unverified	0
The Wang-Landau Algorithm as Stochastic Optimization and Its Acceleration	Jul 27, 2019	Stochastic Optimization	—Unverified	0
DEAM: Adaptive Momentum with Discriminative Weight for Stochastic Optimization	Jul 25, 2019	Stochastic Optimization	—Unverified	0
Multi-Objective Optimization for Drone Delivery	Jul 24, 2019	SchedulingStochastic Optimization	—Unverified	0
Re-route Package Pickup and Delivery Planning with Random Demands	Jul 24, 2019	Stochastic Optimization	—Unverified	0
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization	Jul 24, 2019	Stochastic Optimization	—Unverified	0
An introduction to decentralized stochastic optimization with gradient tracking	Jul 23, 2019	BIG-bench Machine LearningStochastic Optimization	—Unverified	0
Hyperlink Regression via Bregman Divergence	Jul 22, 2019	Graph EmbeddingLink Prediction	—Unverified	0
When can we improve on sample average approximation for stochastic optimization?	Jul 19, 2019	Portfolio OptimizationStochastic Optimization	—Unverified	0
Lookahead Optimizer: k steps forward, 1 step back	Jul 19, 2019	Image ClassificationMachine Translation	CodeCode Available	1
SVGD: A Virtual Gradients Descent Method for Stochastic Optimization	Jul 9, 2019	Stochastic Optimization	CodeCode Available	0
Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization	Jul 9, 2019	Stochastic Optimization	CodeCode Available	0
Variational Inference MPC for Bayesian Model-based Reinforcement Learning	Jul 8, 2019	Bayesian InferenceModel-based Reinforcement Learning	—Unverified	0
A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization	Jul 8, 2019	Stochastic Optimization	—Unverified	0
The Role of Memory in Stochastic Optimization	Jul 2, 2019	Stochastic Optimization	—Unverified	0

Show:10 25 50

← PrevPage 19 of 28Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified