A Causal Approach to Model Validation and Calibration

All current developed methods for model validation are based on standard statistical analysis (to measure statistical differences between data populations) or machine learning (to identify response surface from data) methods. Both methods have to be purely data driven, that is they provide quantitative comparison measures between data sets (e.g., simulated vs. measured data) without explicitly considering the hypothesis behind them (e.g., boundary conditions) and the structure of the employed models. This can generate the erroneous conclusion that, when two data populations are close enough, the models that have generated them are similar. In addition, when simulated and experimental data differ beyond the acceptance criteria, model calibration techniques are used to tweak simulation model parameters to reduce the gap between simulated and experimental data. This gives the false expectation that a simulation model matches reality. The goal of this paper is to move away from current purely data-driven methods for validation and calibration toward more robust model-driven methods based on causal inference. The presented methods capture the causal relationships between data elements (e.g., simulated and experimental data) rather than looking at their associations, and they employ these relationships to measure differences between simulated and measured data. These causal differences then directly inform the calibration process rather than relying on the analyst educated guess.