An Interpretation of the Bellman Equation for Risk-Informed Decision Making

Traditionally, the Bellman Equation is an equation iteratively solved in support of finding the optimal policy of a Markov Decision Process (MDP). Once determined, optimal policies can be followed for any system state to maximize the reward obtained by the system. Thus, the implementation of MDP policies constitutes optimal autonomous control of a system or plant. However, the reward function which is optimized is designed by hand, typically with a large degree of arbitrary tuning to generate the desired decisions. In this work, it is demonstrated that rather than maximize an arbitrary reward, the dynamic evolution of the core damage frequency (cdf) measure of risk used in probabilistic risk assessment by the nuclear industry can be calculated using a value iteration scheme within an MDP. An optimal policy of operational actions to minimize system risk can then be determined. As such, this work presents a first step towards using a measure accepted within the nuclear community as the objective of a machine learning (ML) approach. This work thus supports moving one step closer to unraveling issues of transparency and interpretability that keep risk-informed ML decision making methods from experiencing large scale implementation in nuclear applications.