Markov Chain Concentration with an Application in Reinforcement Learning

Abstract

Given X_1, ,X_N random variables whose joint distribution is given as we will use the Martingale Method to show any Lipshitz Function f over these random variables is subgaussian. The Variance parameter however can have a simple expression under certain conditions. For example under the assumption that the random variables follow a Markov Chain and that the function is Lipschitz under a Weighted Hamming Metric. We shall conclude with certain well known techniques from concentration of suprema of random processes with applications in Reinforcement Learning

Tasks

Reproductions