Making Reinforcement Learning Understandable: A Case Study on Surgical Recovery

[Submitted on 25 Aug 2023]

Download a PDF of the paper titled “Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recovery” by Patrick Emedom-Nnamdi and three other authors here.

Abstract: We propose a nonparametric additive model for estimating interpretable value functions in reinforcement learning. Our method addresses the challenge of learning effective adaptive clinical interventions based on digital phenotyping features, particularly in the context of spine surgery and post-operative recovery recommendations. While reinforcement learning has shown success in various domains, existing methods often rely on black-box neural networks, hindering interpretability. Our novel approach combines local kernel regression and basis expansion to estimate the action-value function without explicit parametric assumptions. This allows us to examine the contribution of individual and joint features in producing the final suggested decision. Through simulation and application to spine disease, we validate our approach and uncover recovery recommendations aligned with clinical knowledge.