SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Showing 51100 of 1387 papers

TitleStatusHype
SOUL: Unlocking the Power of Second-Order Optimization for LLM UnlearningCode1
Spectral Inference Networks: Unifying Deep and Spectral LearningCode1
A Variational Perspective on Solving Inverse Problems with Diffusion ModelsCode1
Stochastic Gradient Descent Captures How Children Learn About PhysicsCode1
An Analysis of the Adaptation Speed of Causal ModelsCode1
Stochastic Optimization for Performative PredictionCode1
The Acquisition of Physical Knowledge in Generative Neural NetworksCode1
Time-Causal VAE: Robust Financial Time Series GeneratorCode1
Adaptive Semantic Token Communication for Transformer-based Edge InferenceCode1
Variational Inference: A Review for StatisticiansCode1
Why Do We Need Weight Decay in Modern Deep Learning?Code1
A Better Alternative to Error Feedback for Communication-Efficient Distributed LearningCode1
A Novel Unified Parametric Assumption for Nonconvex OptimizationCode1
Averaging Weights Leads to Wider Optima and Better GeneralizationCode1
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic OptimizationCode1
ATD: Augmenting CP Tensor Decomposition by Self SupervisionCode1
Cyclical Stochastic Gradient MCMC for Bayesian Deep LearningCode1
Learning from History for Byzantine Robust OptimizationCode1
BinaryViT: Pushing Binary Vision Transformers Towards Convolutional ModelsCode1
Combinatorial Optimization enriched Machine Learning to solve the Dynamic Vehicle Routing Problem with Time WindowsCode1
Adaptivity of Stochastic Gradient Methods for Nonconvex OptimizationCode1
Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic OptimizationCode1
Deep Generalized Canonical Correlation AnalysisCode1
Self-Directed Online Machine Learning for Topology OptimizationCode1
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case GeneralizationCode1
Efficient approximation of Jacobian matrices involving a non-uniform fast Fourier transform (NUFFT)Code1
Exploiting Explainable Metrics for Augmented SGDCode1
Federated Learning over Wireless Networks: Convergence Analysis and Resource AllocationCode1
Adafactor: Adaptive Learning Rates with Sublinear Memory CostCode1
ADMM for Efficient Deep Learning with Global ConvergenceCode1
Revisiting Distributed Synchronous SGDCode1
Lookahead Optimizer: k steps forward, 1 step backCode1
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine LearningCode1
Monte Carlo Policy Gradient Method for Binary OptimizationCode1
Adam: A Method for Stochastic OptimizationCode1
Online Learning Rate Adaptation with Hypergradient DescentCode1
ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization ProblemsCode1
PACOH: Bayes-Optimal Meta-Learning with PAC-GuaranteesCode1
Personalized Federated Learning with Moreau EnvelopesCode1
Sequential Manipulation Planning on Scene GraphCode1
Adapting to Mixing Time in Stochastic Optimization with Markovian DataCode1
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm PerformanceCode1
Quality-Diversity Optimization: a novel branch of stochastic optimizationCode1
Randomized Automatic DifferentiationCode1
Adaptive Single-Pass Stochastic Gradient Descent in Input Sparsity Time0
Adaptive Shells for Efficient Neural Radiance Field Rendering0
A Communication-Efficient Adaptive Algorithm for Federated Learning under Cumulative Regret0
Adaptive Sequential Machine Learning0
Accelerated Reinforcement Learning0
A Latent Variational Framework for Stochastic Optimization0
Show:102550
← PrevPage 2 of 28Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24Unverified
2AdaShiftAccuracy81.12Unverified
3Adam (eps-adjusted)Accuracy81.04Unverified
4SGDAccuracy80.95Unverified
5AdamWAccuracy79.87Unverified
6AdaBoundAccuracy77.24Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36Unverified
2AvaGradAccuracy96.2Unverified
3SGDAccuracy96.14Unverified
4AdaShiftAccuracy95.92Unverified
5AdamWAccuracy95.89Unverified
6AdaBoundAccuracy94.6Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55Unverified
2LookaheadAccuracy95.27Unverified
3SGDAccuracy95.23Unverified
4ADAMAccuracy94.84Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51Unverified
2SGDTop 1 Accuracy75.99Unverified
3AdamWTop 1 Accuracy72.9Unverified
4AdaBoundTop 1 Accuracy72.01Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86Unverified
2AdaShiftBit per Character (BPC)1.27Unverified
3AdamWBit per Character (BPC)1.23Unverified
4AvaGradBit per Character (BPC)1.18Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85Unverified
2Resnet34Accuracy (max)86.14Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48Unverified
2Resnet34Accuracy (max)54.5Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15Unverified
2LookaheadTop 1 Accuracy75.13Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49Unverified
2SGDTop 1 Accuracy75.15Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05Unverified