Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 1301–1325 of 1387 papers

Title	Date	Tasks	Status
Task-based End-to-end Model Learning in Stochastic Optimization	Mar 13, 2017	BIG-bench Machine LearningScheduling	CodeCode Available
MDPGT: Momentum-based Decentralized Policy Gradient Tracking	Dec 6, 2021	Multi-agent Reinforcement LearningPolicy Gradient Methods	CodeCode Available
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization	Jun 12, 2020	BIG-bench Machine LearningStochastic Optimization	CodeCode Available
Bridging the Gap between Community and Node Representations: Graph Embedding via Community Detection	Dec 17, 2019	Community DetectionGraph Embedding	CodeCode Available
Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning	Jun 9, 2021	Continual LearningFederated Learning	CodeCode Available
Convolutional Sketch Inversion	Jun 9, 2016	Stochastic Optimization	CodeCode Available
A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Data	Nov 12, 2024	FairnessFederated Learning	CodeCode Available
Stochastic Differentially Private and Fair Learning	Oct 17, 2022	Binary ClassificationDecision Making	CodeCode Available
Convex Optimization: Algorithms and Complexity	May 20, 2014	Stochastic Optimization	CodeCode Available
f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization	Dec 6, 2023	FairnessStochastic Optimization	CodeCode Available
Stochastic First-Order Learning for Large-Scale Flexibly Tied Gaussian Mixture Model	Dec 11, 2022	Stochastic Optimization	CodeCode Available
RFFNet: Large-Scale Interpretable Kernel Methods via Random Fourier Features	Nov 11, 2022	Stochastic OptimizationVariable Selection	CodeCode Available
Outlier-robust neural network training: variation regularization meets trimmed loss to prevent functional breakdown	Aug 4, 2023	Numerical IntegrationStochastic Optimization	CodeCode Available
SIM-Shapley: A Stable and Computationally Efficient Approach to Shapley Value Approximation	May 13, 2025	Computational EfficiencyExplainable artificial intelligence	CodeCode Available
Minimax AUC Fairness: Efficient Algorithm with Provable Convergence	Aug 22, 2022	Decision MakingFairness	CodeCode Available
MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients	Jun 21, 2020	Image ClassificationMachine Translation	CodeCode Available
Breaking the Heavy-Tailed Noise Barrier in Stochastic Optimization Problems	Nov 7, 2023	Stochastic Optimization	CodeCode Available
Mining the manifolds of deep generative models for multiple data-consistent solutions of ill-posed tomographic imaging problems	Feb 10, 2022	Generative Adversarial NetworkStochastic Optimization	CodeCode Available
Training Deep Neural Networks via Branch-and-Bound	Apr 5, 2021	Object RecognitionStochastic Optimization	CodeCode Available
Understanding the Role of Momentum in Stochastic Gradient Methods	Oct 30, 2019	Stochastic Optimization	CodeCode Available
Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization	Jan 16, 2025	Stochastic Optimization	CodeCode Available
Fixed-Mean Gaussian Processes for Post-hoc Bayesian Deep Learning	Dec 5, 2024	Computational EfficiencyDeep Learning	CodeCode Available
Mixing ADAM and SGD: a Combined Optimization Method	Nov 16, 2020	Document ClassificationStochastic Optimization	CodeCode Available
Rieoptax: Riemannian Optimization in JAX	Oct 10, 2022	CPUGPU	CodeCode Available
Boosting Adversarial Transferability through Augmentation in Hypothesis Space	Jan 1, 2025	Stochastic Optimization	CodeCode Available

Show:10 25 50

← PrevPage 53 of 56Next →

All datasets CIFAR-100 WRN-28-10 - 200 Epochs CIFAR-10 WRN-28-10 - 200 Epochs CIFAR-10 ResNet-18 - 200 Epochs ImageNet ResNet-50 - 90 Epochs Penn Treebank (Character Level) 3x1000 LSTM - 500 Epochs CIFAR-10 CIFAR-100 ImageNet ResNet-50 - 50 Epochs ImageNet ResNet-50 - 60 Epochs AG News CoLA MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Accuracy	81.24	—	Unverified
2	AdaShift	Accuracy	81.12	—	Unverified
3	Adam (eps-adjusted)	Accuracy	81.04	—	Unverified
4	SGD	Accuracy	80.95	—	Unverified
5	AdamW	Accuracy	79.87	—	Unverified
6	AdaBound	Accuracy	77.24	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Adam (eps-adjusted)	Accuracy	96.36	—	Unverified
2	AvaGrad	Accuracy	96.2	—	Unverified
3	SGD	Accuracy	96.14	—	Unverified
4	AdaShift	Accuracy	95.92	—	Unverified
5	AdamW	Accuracy	95.89	—	Unverified
6	AdaBound	Accuracy	94.6	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD - cosine LR schedule	Accuracy	95.55	—	Unverified
2	Lookahead	Accuracy	95.27	—	Unverified
3	SGD	Accuracy	95.23	—	Unverified
4	ADAM	Accuracy	94.84	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AvaGrad	Top 1 Accuracy	76.51	—	Unverified
2	SGD	Top 1 Accuracy	75.99	—	Unverified
3	AdamW	Top 1 Accuracy	72.9	—	Unverified
4	AdaBound	Top 1 Accuracy	72.01	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	AdaBound	Bit per Character (BPC)	2.86	—	Unverified
2	AdaShift	Bit per Character (BPC)	1.27	—	Unverified
3	AdamW	Bit per Character (BPC)	1.23	—	Unverified
4	AvaGrad	Bit per Character (BPC)	1.18	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	86.85	—	Unverified
2	Resnet34	Accuracy (max)	86.14	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Resnet18	Accuracy (max)	58.48	—	Unverified
2	Resnet34	Accuracy (max)	54.5	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	SGD	Top 5 Accuracy	92.15	—	Unverified
2	Lookahead	Top 1 Accuracy	75.13	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Lookahead	Top 1 Accuracy	75.49	—	Unverified
2	SGD	Top 1 Accuracy	75.15	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	93.99	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Bert	Accuracy (max)	86.34	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MLP	NLL	0.05	—	Unverified