Learning-Process Evaluation to Select Spatial Abstractions in Reinforcement Learning

Reinforcement Learning is a general approach to learn acting optimally in a stochastic environment; however, because of a trial-and-error strategy, learning may take prohibitive long times. An effective method to accelerate the learning process is to make use of spatial abstractions to produce a generalization of experiences. Therefore, how to automatically design abstractions is an important question in Reinforcement Learning. We formalize the problem of designing abstractions as the problem of selecting abstractions, and we propose to evaluate the learning process as a surrogate of the quality of a subset of abstractions. Because evaluating the learning process is costly, we present an algorithm, the SEPO algorithm, that construct a ranking of abstractions efficiently. To evaluate the efficiency of the SEPO algorithm, experiments are done in a soccer simulator.