Exploring the psychometrics of common post-scenario human factors questionnaires of workload, situation awareness, and perceived difficulty

In the United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Plant Modernization pathway, the role of human factors engineering (HFE) continues to have an integral role in ensuring new advanced digital technologies that are part of the overall plant modernization strategy support optimized human-system performance and do not introduce new human failure modes. Regulatory guidance highlights human-system performance is a complex and multifaceted construct that requires a hierarchical set of measures including aspects of workload, situation awareness (SA), and task performance. Since workload, SA, and task difficulty are not directly observable, an accepted practice is to use subjective measures such as survey instrumentations. However, the time allotted for survey administration and post-scenario discussion during HFE activities can be significantly limited depending on operator availability and/or simulator availability. This paper explores the psychometrics of common survey instruments that measure workload, SA, and perceived task difficulty in nuclear power plant operator-in-the-loop studies with a goal of informing development of a consolidated survey instrument to reduce administration time. A set of exploratory analyses uncovered two principal components from the individual items combined from these surveys. A prototype survey instrument was thus developed as an early concept to illustrate how these two principals components could be expanded into a survey tool with fewer questions. A detailed discussion of the exploratory analyses and new prototype survey are discussed in this paper.