Based on my reading of the paper, the central research question is:
Can an ensemble method involving agents learning distributionally in separate environments lead to more robust and efficient reinforcement learning?
Specifically, the paper proposes and studies an extension of categorical distributional reinforcement learning called "Ensemble Categorical Control" (ECC). The goal is to have an ensemble of agents learn distributionally in separate environments, and aggregate their distributional information to create more robust learning targets.
The central hypothesis is that the distributional and ensemble approach of ECC will:
- Lead to more robust initial learning compared to regular CDRL
- Result in stronger individual agent performance
- Be efficient in terms of performance per sample
The paper empirically evaluates ECC on a set of Atari games, comparing it to standard CDRL and an ensemble CDRL baseline. The experimental results are intended to validate the benefits of the distributional ensemble approach of ECC.
In summary, the central research question is whether the proposed ECC algorithm can enhance robustness, individual performance, and sample efficiency through its combination of distributional and ensemble learning across separate environments. The hypothesis is that ECC will demonstrate clear improvements in these areas over regular CDRL.