Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems
Authors: Kunal Pattanayak, Vikram Krishnamurthy; Volume 24, Issue 52, Pages 1-64, 2023
Abstract
This paper introduces a framework for inverse reinforcement learning (IRL) in Bayesian stopping time problems. The goal is to determine whether the actions taken by a Bayesian decision maker are consistent with optimizing a given cost function. In a Bayesian setting with partial observations, the IRL algorithm can only identify optimality based on the observed strategies. To accomplish this, we leverage concepts from Bayesian revealed preferences in microeconomics to identify optimality and construct set-valued estimates of the cost function. We demonstrate the effectiveness of our proposed IRL scheme through two important examples: sequential hypothesis testing and Bayesian search. Additionally, we apply our method to predict user engagement in online multimedia platforms using a YouTube dataset, achieving high accuracy. Finally, we propose an IRL detection algorithm for finite datasets and provide finite sample bounds on its error probabilities.
[abs]