Finite-Time Error Analysis of Online Model-Based Q-Learning with a Relaxed Sampling Model

Abstract

Reinforcement learning has witnessed significant advancements, particularly with the emergence of model-based approaches. Among these, Q-learning has proven to be a powerful algorithm in model-free settings. However, the extension of Q-learning to a model-based framework remains relatively unexplored. In this paper, we delve into the sample complexity of Q-learning when integrated with a model-based approach. Through theoretical analyses and empirical evaluations, we seek to elucidate the conditions under which model-based Q-learning excels in terms of sample efficiency compared to its model-free counterpart.

Tasks

Reproductions