Nonidentifiability in Model Calibration and Implications for Medical Decision Making
Published in Medical Decision Making, 2018
Recommended citation: Alarid-Escudero F, MacLehose RF, Peralta Y, Kuntz KM, Enns EA. Non-identifiability in model calibration and implications to medical decision making. Medical Decision Making, 2018;38(7):810-21. https://doi.org/10.1177/0272989X18792283 https://doi.org/10.1177/0272989X18792283
Abstract
Background
Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability.
Methods
We illustrate nonidentifiability by calibrating a 3-state Markov model of cancer relative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer.
Results
When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile.
Conclusions
In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.