XCS as a reinforcement learning approach to automatic test case prioritization

Code

• github.com/LagLukas/XCS_TESTING
  none★ 5

Abstract

Testing is a crucial part in the development of new products. With the rise of test automation methods, companies start relying on an even higher number of tests. Sometimes it is not feasible to run all tests and the goal is to determine which tests are crucial and which are less important. This prioritization problem has just recently gotten into the focus of reinforcement learning. A neural network combined with prioritized experience replay (ER) was used to identify critical tests. We are the first to apply XCS classifier systems (XCS) for this use case and reveal that XCS is not only suitable for this problem, but can also be superior to the aforementioned neural network and leads to more stable results. In this work, we adapt XCS's learning mechanism to the task by introducing a batch update which is based on Monte Carlo control. Further, we investigate if prioritized ER has the same positive effects on XCS as on the neural network for this test prioritization problem. Our experiments show that in general this is not the case for XCS.

Tasks

Reproductions