Deep reinforcement learning (DRL) has demonstrated great potential in achieving human-level autonomy in various domains such as robotics, computer vision, and computer games. This has led to widespread interest and enthusiasm in both academia and industry. However, the focus of the community has primarily been on the development phase of DRL systems, with little attention given to their deployment. This paper presents an empirical study conducted on Stack Overflow (SO), the leading Q&A forum for developers, to explore and comprehend the challenges faced by practitioners when deploying DRL systems.
The study categorizes relevant SO posts according to the platforms used for deployment, namely server/cloud, mobile/embedded systems, browser, and game engine. After filtering and manual analysis, a total of 357 SO posts related to DRL deployment were examined to investigate the current state and identify the challenges associated with deploying DRL systems. The prevalence and difficulty of these challenges were also investigated.
The results indicate a growing interest in DRL deployment, highlighting the relevance and importance of this study. Furthermore, the findings reveal that deploying DRL systems is more challenging compared to other aspects of DRL. A taxonomy consisting of 31 unique challenges in deploying DRL to different platforms was developed. Among all platforms, challenges related to the RL environment were found to be the most common, while communication-related challenges were identified as the most difficult for practitioners.
The aim of this study is to inspire future research and assist the community in overcoming the most common and difficult challenges encountered when deploying DRL systems.