Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 326–350 of 1387 papers

Title	Date	Tasks	Status	Hype
Momentum Provably Improves Error Feedback!	May 24, 2023	Stochastic Optimization	—Unverified	0
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model	May 24, 2023	AllLanguage Modeling	CodeCode Available	0
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training	May 23, 2023	Language ModelingLanguage Modelling	CodeCode Available	2
Offline Primal-Dual Reinforcement Learning for Linear MDPs	May 22, 2023	Offline RLreinforcement-learning	—Unverified	0
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent	May 20, 2023	Generalization BoundsLearning Theory	—Unverified	0
Constrained Environment Optimization for Prioritized Multi-Agent Navigation	May 18, 2023	Stochastic Optimization	—Unverified	0
Lyapunov-Driven Deep Reinforcement Learning for Edge Inference Empowered by Reconfigurable Intelligent Surfaces	May 18, 2023	CPUData Compression	—Unverified	0
Provable Multi-instance Deep AUC Maximization with Stochastic Pooling	May 14, 2023	GPUStochastic Optimization	CodeCode Available	0
Towards Understanding the Generalization of Graph Neural Networks	May 14, 2023	Stochastic OptimizationTransductive Learning	—Unverified	0
Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression	May 12, 2023	Stochastic Optimization	—Unverified	0
MoMo: Momentum Models for Adaptive Learning Rates	May 12, 2023	Recommendation SystemsStochastic Optimization	CodeCode Available	1
Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization	May 9, 2023	Combinatorial OptimizationDeep Reinforcement Learning	—Unverified	0
UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization	May 9, 2023	Stochastic OptimizationVocal Bursts Type Prediction	—Unverified	0
A Variational Perspective on Solving Inverse Problems with Diffusion Models	May 7, 2023	DenoisingImage Restoration	CodeCode Available	1
Projection-Free Online Convex Optimization with Stochastic Constraints	May 2, 2023	Stochastic Optimization	—Unverified	0
Stochastic MPC for energy hubs using data driven demand forecasting	Apr 24, 2023	Demand ForecastingGaussian Processes	—Unverified	0
Near-Optimal Decentralized Momentum Method for Nonconvex-PL Minimax Problems	Apr 21, 2023	Stochastic Optimization	—Unverified	0
Debiasing Conditional Stochastic Optimization	Apr 20, 2023	Causal InferenceStochastic Optimization	—Unverified	0
Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One	Apr 20, 2023	Stochastic Optimization	CodeCode Available	1
MAC, a novel stochastic optimization method	Apr 14, 2023	CPUStochastic Optimization	—Unverified	0
Estimate-Then-Optimize versus Integrated-Estimation-Optimization versus Sample Average Approximation: A Stochastic Dominance Perspective	Apr 13, 2023	Stochastic Optimization	CodeCode Available	0
Sample Average Approximation for Black-Box VI	Apr 13, 2023	Stochastic Optimization	—Unverified	0
Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box	Apr 11, 2023	Probabilistic ProgrammingStochastic Optimization	CodeCode Available	0
μ^2-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism	Apr 9, 2023	Stochastic Optimization	—Unverified	0
Gradient Sparsification for Efficient Wireless Federated Learning with Differential Privacy	Apr 9, 2023	Federated LearningScheduling	—Unverified	0

Show:10 25 50

← PrevPage 14 of 56Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified