Root Cause Correlation Analysis of Software Failures via Orthogonal Defect Classification and Natural Language Processing

Systems theoretic process analysis (STPA) is becoming an increasingly popular technique to assess
how complex digital software systems can fail. Rather than defining failures by their observable
failure events, which may be sparse especially for safety rated nuclear digital instrumentation and
control systems (DI&C), failures are defined as postulated unsafe actions under specific contextual conditions. This
permits a top-down analysis of system hazards and identifies whether imposed constraints and requirements can
sufficiently address undesirable hazards. However, STPA is a qualitative approach at identifying inadequacies in
the development process and cannot currently be
used to quantify unsafe action likelihoods for probabilistic risk assessment. Therefore, in this work,
we examine the root causes of software failure and explore whether a consistent correlation can be
linked to specific unsafe action classes. We implement Lbl2Vec, an unsupervised document classification and
retrieval algorithm, on a database of 4,096 software defect reports acquired from various open-source software
systems. By analyzing sentence structure, embedded labels, and word vectors, we show that certain defect types
positively correlate to specific unsafe action classes over
others. The correlations developed can be used to estimate the failure probability of safety intended
DI&C systems which provides a licensing basis for nuclear plant modernization efforts.