SOTAVerified

Stochastic Optimization

Stochastic Optimization is the task of optimizing certain objective functional by generating and using stochastic random variables. Usually the Stochastic Optimization is an iterative process of generating random variables that progressively finds out the minima or the maxima of the objective functional. Stochastic Optimization is usually applied in the non-convex functional spaces where the usual deterministic optimization such as linear or quadratic programming or their variants cannot be used.

Source: ASOC: An Adaptive Parameter-free Stochastic Optimization Techinique for Continuous Variables

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 1151–1200 of 1387 papers

TitleStatusHype
SpectralNet: Spectral Clustering using Deep Neural NetworksCode0
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods—0
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients—0
Third-order Smoothness Helps: Even Faster Stochastic Optimization Algorithms for Finding Local Minima—0
Wasserstein Distributionally Robust Optimization and Variation Regularization—0
Proximodistal Exploration in Motor Learning as an Emergent Property of Optimization—0
Stochastic Particle Gradient Descent for Infinite Ensembles—0
Comparative analysis of criteria for filtering time series of word usage frequencies—0
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks—0
Stochastic Mirror Descent in Variationally Coherent Optimization Problems—0
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure—0
Adaptive SVRG Methods under Error Bound Conditions with Unknown Growth Parameter—0
ADMM without a Fixed Penalty Parameter: Faster Convergence with New Adaptive Penalization—0
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms—0
Random gradient extrapolation for distributed and stochastic optimization—0
Stochastic Submodular Maximization: The Case of Coverage Functions—0
Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression—0
Accelerate RNN-based Training with Importance Sampling—0
Adaptive Sampling Strategies for Stochastic Optimization—0
SGDLibrary: A MATLAB library for stochastic gradient descent algorithmsCode0
Gradient Sparsification for Communication-Efficient Distributed Optimization—0
Accelerated Reinforcement Learning—0
Function Norms and Regularization in Deep Networks—0
Convergence diagnostics for stochastic gradient descent with constant step size—0
A deep generative model for single-cell RNA sequencing with application to detecting differentially expressed genes—0
Decentralized Online Learning with Kernels—0
SGD for robot motion? The effectiveness of stochastic optimization on a new benchmark for biped locomotion tasksCode0
Stochastic Nonconvex Optimization with Large Minibatches—0
Accelerating SGD for Distributed Deep-Learning Using Approximated Hessian Matrix—0
Causality-Aided Falsification—0
A Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism, Composite Objectives, and Variational Bounds—0
A deep generative model for gene expression profiles from single-cell RNA sequencingCode0
Learning Implicit Generative Models Using Differentiable Graph Tests—0
A Convergence Analysis for A Class of Practical Variance-Reduction Stochastic Gradient MCMC—0
Hydroassets Portfolio Management for Intraday Electricity Trading from a Discrete Time Stochastic Optimization Perspective—0
Weighted parallel SGD for distributed unbalanced-workload training system—0
Style2Vec: Representation Learning for Fashion Items from Style SetsCode0
Non-stationary Stochastic Optimization under L_p,q-Variation Measures—0
Stochastic Optimization with Bandit Sampling—0
Thompson Sampling Guided Stochastic Searching on the Line for Deceptive Environments with Applications to Root-Finding Problems—0
Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence—0
Follow the Moving Leader in Deep Learning—0
Dynamic Stochastic Approximation for Multi-stage Stochastic Optimization—0
Unsupervised Submodular Rank Aggregation on Score-based PermutationsCode0
Temporal Action Co-Segmentation in 3D Motion Capture Data and Videos—0
A Unified Analysis of Stochastic Optimization Methods Using Jump System Theory and Quadratic Constraints—0
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations—0
A Unified Approach to Adaptive Regularization in Online and Stochastic Optimization—0
On the convergence of mirror descent beyond stochastic convex programming—0
Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory—0
Show:102550
← PrevPage 24 of 28Next →
All datasetsCIFAR-100 WRN-28-10 - 200 EpochsCIFAR-10 WRN-28-10 - 200 EpochsCIFAR-10 ResNet-18 - 200 EpochsImageNet ResNet-50 - 90 EpochsPenn Treebank (Character Level) 3x1000 LSTM - 500 EpochsCIFAR-10CIFAR-100ImageNet ResNet-50 - 50 EpochsImageNet ResNet-50 - 60 EpochsAG NewsCoLAMNIST

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1AvaGradAccuracy81.24—Unverified
2AdaShiftAccuracy81.12—Unverified
3Adam (eps-adjusted)Accuracy81.04—Unverified
4SGDAccuracy80.95—Unverified
5AdamWAccuracy79.87—Unverified
6AdaBoundAccuracy77.24—Unverified
#ModelMetricClaimedVerifiedStatus
1Adam (eps-adjusted)Accuracy96.36—Unverified
2AvaGradAccuracy96.2—Unverified
3SGDAccuracy96.14—Unverified
4AdaShiftAccuracy95.92—Unverified
5AdamWAccuracy95.89—Unverified
6AdaBoundAccuracy94.6—Unverified
#ModelMetricClaimedVerifiedStatus
1SGD - cosine LR scheduleAccuracy95.55—Unverified
2LookaheadAccuracy95.27—Unverified
3SGDAccuracy95.23—Unverified
4ADAMAccuracy94.84—Unverified
#ModelMetricClaimedVerifiedStatus
1AvaGradTop 1 Accuracy76.51—Unverified
2SGDTop 1 Accuracy75.99—Unverified
3AdamWTop 1 Accuracy72.9—Unverified
4AdaBoundTop 1 Accuracy72.01—Unverified
#ModelMetricClaimedVerifiedStatus
1AdaBoundBit per Character (BPC)2.86—Unverified
2AdaShiftBit per Character (BPC)1.27—Unverified
3AdamWBit per Character (BPC)1.23—Unverified
4AvaGradBit per Character (BPC)1.18—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)86.85—Unverified
2Resnet34Accuracy (max)86.14—Unverified
#ModelMetricClaimedVerifiedStatus
1Resnet18Accuracy (max)58.48—Unverified
2Resnet34Accuracy (max)54.5—Unverified
#ModelMetricClaimedVerifiedStatus
1SGDTop 5 Accuracy92.15—Unverified
2LookaheadTop 1 Accuracy75.13—Unverified
#ModelMetricClaimedVerifiedStatus
1LookaheadTop 1 Accuracy75.49—Unverified
2SGDTop 1 Accuracy75.15—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)93.99—Unverified
#ModelMetricClaimedVerifiedStatus
1BertAccuracy (max)86.34—Unverified
#ModelMetricClaimedVerifiedStatus
1MLPNLL0.05—Unverified