Reinforcement Learning for Anomaly Detection in Nuclear Power Plant Operation and Maintenance

In nuclear power plants (NPPs), timely identification of sensor and human errors is critical to ensure safe and efficient plant operations. Anomaly detection models can be employed for this task. However, traditional anomaly detection approaches may have high dependency on labeled data sets and struggle with adaptability in complex, dynamic environments. Reinforcement learning (RL) has demonstrated significant potential in fault diagnosis and anomaly detection; however, its application to anomaly detection in NPPs remains a relatively underexplored research direction. Hence, to address this gap, in this study, we present a novel physics-informed RL model, PIRL-AD (physics-informed reinforcement learning for anomaly detection), that integrates domain knowledge from calorimetric equations into the RL framework for enhanced sensor and human error anomaly detection. We evaluate the performance of PIRL-AD against a nonphysics-informed RL benchmark and a support vector machine (SVM) on data collected from a forced-flow loop testbed. Experimental results suggest that PIRL-AD outperforms other baselines on a range of anomalous data sets that include both sensor and human-induced anomalies across key performance metrics, statistically outperforming the RL and SVM benchmarks with respect to geometric mean (respectively, 92.96% versus 91.06% versus 83.01%) and F1 score (respectively, 89.23% versus 86.98% versus 77.01%). The findings suggest the potential of physics-integrated RL models for enhanced anomaly detection performance in NPPs.