Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1351–1387 of 1387 papers

Title	Date	Tasks	Status
Natural Gradient Hybrid Variational Inference with Application to Deep Mixed Models	Feb 27, 2023	Stochastic OptimizationVariational Inference	CodeCode Available
Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning	Feb 18, 2025	Combinatorial OptimizationDeep Reinforcement Learning	CodeCode Available
Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed	Jun 6, 2024	Stochastic Optimization	CodeCode Available
Adversarial Subspace Generation for Outlier Detection in High-Dimensional Data	Apr 10, 2025	feature selectionOne-Class Classification	CodeCode Available
Near-Optimal Algorithms for Making the Gradient Small in Stochastic Minimax Optimization	Aug 11, 2022	Stochastic Optimization	CodeCode Available
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model	May 24, 2023	AllLanguage Modeling	CodeCode Available
Towards Faster Decentralized Stochastic Optimization with Communication Compression	May 30, 2024	Stochastic Optimization	CodeCode Available
High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise	Jun 10, 2021	Stochastic Optimization	CodeCode Available
Stochastic Optimization of Sorting Networks via Continuous Relaxations	Mar 21, 2019	Stochastic Optimization	CodeCode Available
Binary Search and First Order Gradient Based Method for Stochastic Optimization	Jul 27, 2020	Stochastic Optimization	CodeCode Available
Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems	Oct 1, 2021	BIG-bench Machine LearningBilevel Optimization	CodeCode Available
Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks	May 28, 2021	Deep LearningStochastic Optimization	CodeCode Available
Stochastic Gradient Descent with Biased but Consistent Gradient Estimators	Jul 31, 2018	Stochastic Optimization	CodeCode Available
Graph Pattern Mining and Learning through User-defined Relations (Extended Version)	Sep 14, 2018	Graph Neural NetworkStochastic Optimization	CodeCode Available
Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks	May 25, 2023	Stochastic Optimization	CodeCode Available
Greedy Step Averaging: A parameter-free stochastic optimization method	Nov 11, 2016	Stochastic Optimization	CodeCode Available
Practical Precoding via Asynchronous Stochastic Successive Convex Approximation	Oct 3, 2020	Stochastic Optimization	CodeCode Available
Byzantine-Robust Decentralized Stochastic Optimization over Static and Time-Varying Networks	May 12, 2020	Stochastic Optimization	CodeCode Available
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization	May 25, 2018	Material ClassificationStochastic Optimization	CodeCode Available
Stochastic Optimization under Distributional Drift	Aug 16, 2021	Stochastic Optimizationvalid	CodeCode Available
Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes	Sep 22, 2023	Bayesian InferenceGaussian Processes	CodeCode Available
AmCLR: Unified Augmented Learning for Cross-Modal Representations	Dec 10, 2024	Contrastive LearningDiversity	CodeCode Available
A Kronecker-factored approximate Fisher matrix for convolution layers	Feb 3, 2016	Stochastic Optimization	CodeCode Available
Balancing Forecast Accuracy and Switching Costs in Online Optimization of Energy Management Systems	Jun 29, 2024	Decision Makingenergy management	CodeCode Available
Noise Stability Optimization for Finding Flat Minima: A Hessian-based Regularization Approach	Jun 14, 2023	Data Augmentationimage-classification	CodeCode Available
Predictor-corrector algorithms for stochastic optimization under gradual distribution shift	May 26, 2022	Object TrackingStochastic Optimization	CodeCode Available
Adaptive Consensus Gradients Aggregation for Scaled Distributed Training	Nov 6, 2024	Stochastic Optimization	CodeCode Available
Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks	Sep 25, 2024	Stochastic Optimization	CodeCode Available
Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function	Jul 19, 2021	Stochastic OptimizationTransfer Learning	CodeCode Available
Joint control variate for faster black-box variational inference	Oct 13, 2022	Stochastic OptimizationVariational Inference	CodeCode Available
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale	Sep 12, 2023	GPUStochastic Optimization	CodeCode Available
Stochastic optimization with arbitrary recurrent data sampling	Jan 15, 2024	Stochastic Optimization	CodeCode Available
Prior Specification for Bayesian Matrix Factorization via Prior Predictive Matching	Oct 27, 2019	Bayesian OptimizationHyperparameter Optimization	CodeCode Available
Universal Boosting Variational Inference	Jun 4, 2019	Stochastic OptimizationVariational Inference	CodeCode Available
A Kernel Mean Embedding Approach to Reducing Conservativeness in Stochastic Programming and Control	Jan 28, 2020	Stochastic Optimization	CodeCode Available
Non-cooperative Aerial Base Station Placement via Stochastic Optimization	May 10, 2019	Stochastic Optimization	CodeCode Available
SpectralNet: Spectral Clustering using Deep Neural Networks	Jan 4, 2018	ClusteringStochastic Optimization	CodeCode Available

Show:10 25 50

← PrevPage 28 of 28Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified