SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1151–1200 of 1387 papers

TitleStatusHype
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure—0
Stochastic Mirror Descent in Variationally Coherent Optimization Problems—0
Adaptive SVRG Methods under Error Bound Conditions with Unknown Growth Parameter—0
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms—0
Random gradient extrapolation for distributed and stochastic optimization—0
Stochastic Submodular Maximization: The Case of Coverage Functions—0
Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression—0
Accelerate RNN-based Training with Importance Sampling—0
Adaptive Sampling Strategies for Stochastic Optimization—0
SGDLibrary: A MATLAB library for stochastic gradient descent algorithmsCode0
Gradient Sparsification for Communication-Efficient Distributed Optimization—0
Accelerated Reinforcement Learning—0
Function Norms and Regularization in Deep Networks—0
Convergence diagnostics for stochastic gradient descent with constant step size—0
A deep generative model for single-cell RNA sequencing with application to detecting differentially expressed genes—0
Decentralized Online Learning with Kernels—0
SGD for robot motion? The effectiveness of stochastic optimization on a new benchmark for biped locomotion tasksCode0
Stochastic Nonconvex Optimization with Large Minibatches—0
Accelerating SGD for Distributed Deep-Learning Using Approximated Hessian Matrix—0
Causality-Aided Falsification—0
A Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism, Composite Objectives, and Variational Bounds—0
A deep generative model for gene expression profiles from single-cell RNA sequencingCode0
A Convergence Analysis for A Class of Practical Variance-Reduction Stochastic Gradient MCMC—0
Learning Implicit Generative Models Using Differentiable Graph Tests—0
Hydroassets Portfolio Management for Intraday Electricity Trading from a Discrete Time Stochastic Optimization Perspective—0
Weighted parallel SGD for distributed unbalanced-workload training system—0
Style2Vec: Representation Learning for Fashion Items from Style SetsCode0
Non-stationary Stochastic Optimization under L_p,q-Variation Measures—0
Stochastic Optimization with Bandit Sampling—0
Thompson Sampling Guided Stochastic Searching on the Line for Deceptive Environments with Applications to Root-Finding Problems—0
Follow the Moving Leader in Deep Learning—0
Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence—0
Dynamic Stochastic Approximation for Multi-stage Stochastic Optimization—0
Unsupervised Submodular Rank Aggregation on Score-based PermutationsCode0
Temporal Action Co-Segmentation in 3D Motion Capture Data and Videos—0
A Unified Analysis of Stochastic Optimization Methods Using Jump System Theory and Quadratic Constraints—0
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations—0
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization—0
On the convergence of mirror descent beyond stochastic convex programming—0
Accurate, Large Minibatch SGD: Training ImageNet in 1 HourCode2
Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory—0
Biased Importance Sampling for Deep Neural Network TrainingCode0
Training Deep Networks without Learning Rates Through Coin BettingCode1
Limited-Memory Matrix Adaptation for Large Scale Black-box OptimizationCode0
Comments on the proof of adaptive submodular function minimization—0
Accelerating Stochastic Gradient Descent For Least Squares Regression—0
Stochastic Optimization from Distributed, Streaming Data in Rate-limited Networks—0
Bandit Structured Prediction for Neural Sequence-to-Sequence LearningCode0
Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case—0
Importance Sampled Stochastic Optimization for Variational Inference—0
Show:102550
← PrevPage 24 of 28Next →
All datasetsCIFAR-100 WRN-28-10 - 200 EpochsCIFAR-10 WRN-28-10 - 200 EpochsCIFAR-10 ResNet-18 - 200 EpochsImageNet ResNet-50 - 90 EpochsPenn Treebank (Character Level) 3x1000 LSTM - 500 EpochsCIFAR-10CIFAR-100ImageNet ResNet-50 - 50 EpochsImageNet ResNet-50 - 60 EpochsAG NewsCoLAMNIST

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24—Unverified
2AdaShiftAccuracy81.12—Unverified
3Adam (eps-adjusted)Accuracy81.04—Unverified
4SGDAccuracy80.95—Unverified
5AdamWAccuracy79.87—Unverified
6AdaBoundAccuracy77.24—Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36—Unverified
2AvaGradAccuracy96.2—Unverified
3SGDAccuracy96.14—Unverified
4AdaShiftAccuracy95.92—Unverified
5AdamWAccuracy95.89—Unverified
6AdaBoundAccuracy94.6—Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55—Unverified
2LookaheadAccuracy95.27—Unverified
3SGDAccuracy95.23—Unverified
4ADAMAccuracy94.84—Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51—Unverified
2SGDTop 1 Accuracy75.99—Unverified
3AdamWTop 1 Accuracy72.9—Unverified
4AdaBoundTop 1 Accuracy72.01—Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86—Unverified
2AdaShiftBit per Character (BPC)1.27—Unverified
3AdamWBit per Character (BPC)1.23—Unverified
4AvaGradBit per Character (BPC)1.18—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85—Unverified
2Resnet34Accuracy (max)86.14—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48—Unverified
2Resnet34Accuracy (max)54.5—Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15—Unverified
2LookaheadTop 1 Accuracy75.13—Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49—Unverified
2SGDTop 1 Accuracy75.15—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34—Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05—Unverified