Application of Monte Carlo Stochastic Optimization (MOST) to Deep Learning

Abstract

In this paper, we apply the Monte Carlo stochastic optimization (MOST) proposed by the authors to a deep learning of XOR gate and verify its effectiveness. Deep machine learning based on neural networks is one of the most important keywords driving innovation in today's highly advanced information society. Therefore, there has been active research on large-scale, high-speed, and high-precision systems. For the purpose of efficiently searching the optimum value of the objective function, the author divides the search region of a multivariable parameter constituting the objective function into two by each parameter, numerically finds the integration of the two regions by the Monte Carlo method, compares the magnitude of the integration value, and judges that there is an optimum point in a small region. In the previous paper, we examined the problem of the benchmark in the optimization method. This method is applied to neural networks of XOR gate, and compared with the results of weight factor optimization by Adam and genetic algorithm. As a result, it was confirmed that it converged faster than the existing method.

Tasks

Reproductions