SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Showing 150 of 1387 papers

TitleStatusHype
MARS: Unleashing the Power of Variance Reduction for Training Large ModelsCode4
Benchopt: Reproducible, efficient and collaborative optimization benchmarksCode4
Conformal Symplectic Optimization for Stable Reinforcement LearningCode2
Smoothing Methods for Automatic Differentiation Across Conditional BranchesCode2
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingCode2
Riemannian Adaptive Optimization MethodsCode2
Accurate, Large Minibatch SGD: Training ImageNet in 1 HourCode2
Adaptive Semantic Token Communication for Transformer-based Edge InferenceCode1
JaxSGMC: Modular stochastic gradient MCMC in JAXCode1
A Novel Unified Parametric Assumption for Nonconvex OptimizationCode1
Time-Causal VAE: Robust Financial Time Series GeneratorCode1
Training-free Diffusion Model Alignment with Sampling DemonsCode1
Randomized Physics-Informed Neural Networks for Bayesian Data AssimilationCode1
SOUL: Unlocking the Power of Second-Order Optimization for LLM UnlearningCode1
The Acquisition of Physical Knowledge in Generative Neural NetworksCode1
Why Do We Need Weight Decay in Modern Deep Learning?Code1
Monte Carlo Policy Gradient Method for Binary OptimizationCode1
BinaryViT: Pushing Binary Vision Transformers Towards Convolutional ModelsCode1
Neural Inventory Control in Networks via Hindsight Differentiable Policy OptimizationCode1
MoMo: Momentum Models for Adaptive Learning RatesCode1
A Variational Perspective on Solving Inverse Problems with Diffusion ModelsCode1
Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of OneCode1
Combinatorial Optimization enriched Machine Learning to solve the Dynamic Vehicle Routing Problem with Time WindowsCode1
Variational Linearized Laplace Approximation for Bayesian Deep LearningCode1
Optimal Planning of Hybrid Energy Storage Systems using Curtailed Renewable Energy through Deep Reinforcement LearningCode1
End-to-End Stochastic Optimization with Energy-Based ModelCode1
Stochastic Gradient Descent Captures How Children Learn About PhysicsCode1
Sequential Manipulation Planning on Scene GraphCode1
Communication-Efficient Adaptive Federated LearningCode1
Exploiting Explainable Metrics for Augmented SGDCode1
A Framework for Improving the Reliability of Black-box Variational InferenceCode1
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm PerformanceCode1
Adapting to Mixing Time in Stochastic Optimization with Markovian DataCode1
Reinforcement Learning with Dynamic Convex Risk MeasuresCode1
BCD Nets: Scalable Variational Approaches for Bayesian Causal DiscoveryCode1
Efficient approximation of Jacobian matrices involving a non-uniform fast Fourier transform (NUFFT)Code1
slimTrain -- A Stochastic Approximation Method for Training Separable Deep Neural NetworksCode1
ATD: Augmenting CP Tensor Decomposition by Self SupervisionCode1
Scaling Up Graph Neural Networks Via Graph CoarseningCode1
Differentiable Quality DiversityCode1
Efficient Stochastic Optimal Control through Approximate Bayesian Input InferenceCode1
Stochastic Optimization of Areas Under Precision-Recall Curves with Provable ConvergenceCode1
A Stochastic Optimization Framework for Fair Risk MinimizationCode1
Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-TrainingCode1
Parameter-free Stochastic Optimization of Variationally Coherent FunctionsCode1
Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic OptimizationCode1
LIRA: Learnable, Imperceptible and Robust Backdoor AttacksCode1
Stochastic Gradient Variance Reduction by Solving a Filtering ProblemCode1
Learning from History for Byzantine Robust OptimizationCode1
Quality-Diversity Optimization: a novel branch of stochastic optimizationCode1
Show:102550
← PrevPage 1 of 28Next →

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24Unverified
2AdaShiftAccuracy81.12Unverified
3Adam (eps-adjusted)Accuracy81.04Unverified
4SGDAccuracy80.95Unverified
5AdamWAccuracy79.87Unverified
6AdaBoundAccuracy77.24Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36Unverified
2AvaGradAccuracy96.2Unverified
3SGDAccuracy96.14Unverified
4AdaShiftAccuracy95.92Unverified
5AdamWAccuracy95.89Unverified
6AdaBoundAccuracy94.6Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55Unverified
2LookaheadAccuracy95.27Unverified
3SGDAccuracy95.23Unverified
4ADAMAccuracy94.84Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51Unverified
2SGDTop 1 Accuracy75.99Unverified
3AdamWTop 1 Accuracy72.9Unverified
4AdaBoundTop 1 Accuracy72.01Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86Unverified
2AdaShiftBit per Character (BPC)1.27Unverified
3AdamWBit per Character (BPC)1.23Unverified
4AvaGradBit per Character (BPC)1.18Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85Unverified
2Resnet34Accuracy (max)86.14Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48Unverified
2Resnet34Accuracy (max)54.5Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15Unverified
2LookaheadTop 1 Accuracy75.13Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49Unverified
2SGDTop 1 Accuracy75.15Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05Unverified