We study the problem of imitation learning via inverse reinforcement learning where the agent attempts to learn an expert’s policy from a dataset of collected state, action tuples. We derive a new Robust model-based Offline Imitation Learning method (ROIL) that mitigates covariate shift by avoiding estimating the expert’s occupancy frequency. Frequently in offline settings, there is insufficient data to reliably estimate the expert’s occupancy frequency and this leads to models that do not generalize well. Our proposed approach, ROIL, is a method that is guaranteed to recover the expert’s occupancy frequency and is efficiently solvable as an LP. We demonstrate ROIL’s ability to achieve minimal regret in large environments under covariate shift, such as when the state visitation frequency of the demonstrations does not come from the expert.
@article{roil,title={ROIL: Robust Offline Imitation Learning},author={Doko, Gersi and Yang, Guang and Brown, Daniel and Petrik, Marek},year={2024},month=may,institution={RLC},journal={Reinforcement Learning Journal},publisher={RLJ},volume={1},number={1}}
RCR
Deep Reinforcement Learning based Optimization of an Island Energy-Water Microgrid System
Roozbeh Ghasemi, Gersi Doko, Marek Petrik, and 4 more authors
@article{deeprl-island,title={Deep Reinforcement Learning based Optimization of an Island
Energy-Water Microgrid System},author={Ghasemi, Roozbeh and Doko, Gersi and Petrik, Marek and Wosnik, Martin and Lu, Zhongming and Foster, Diane and Mo, Weiwei},year={2024},month=oct,journal={Resources, Conservation and Recycling},publisher={RCR},volume={1},number={1}}