A Deep Architecture for Log-Linear Models

Abstract

We present a novel perspective on deep learning architectures using a partial order structure, which is naturally incorporated into the information geometric formulation of the log-linear model. Our formulation provides a different perspective of deep learning by realizing the bias and weights as different layers on our partial order structure. This formulation of the neural network does not require any gradients and can efficiently estimate the parameters using the EM algorithm.

Tasks

Reproductions