You are here:
Repeated holdout Cross-Validation of Model to Estimate Risk of Lyme Disease by Landscape Attributes
Citation:
HILBORN, E. D., D. G. CATANZARO, L. Tran, AND L. JACKSON. Repeated holdout Cross-Validation of Model to Estimate Risk of Lyme Disease by Landscape Attributes. INTERNATIONAL JOURNAL OF ENVIRONMENTAL HEALTH RESEARCH. Carfax Publishing Limited, Basingstoke, Uk, DOI: 10.1080/(09603123.2011.5):1-11, (2011).
Impact/Purpose:
We previously modeled Lyme Disease (LD) risk at the landscape scale using roadbounded analysis units to aggregate land-cover and disease surveillance data. In this study, we further evaluated the model's performance using a holdout validation technique.
Description:
We previously modeled Lyme disease (LD) risk at the landscape scale; here we evaluate the model's overall goodness-of-fit using holdout validation. Landscapes were characterized within road-bounded analysis units (AU). Observed LD cases (obsLD) were ascertained per AU. Data were randomly subset 2,000 times. Of 514 AU, 411 (80%) were selected as a training dataset to develop parameter estimates used to predict observations in the remaining 103 (20%) AU, the validation subset. Predicted values were subtracted from obsLD to quantify accuracy across iterations. We calculated the percentage difference of over- and under-estimation to assess bias. Predictive ability was strong and similar across iterations and datasets; the exact number of obsLD cases per AU were predicted almost 60% of the time. However, the three highest obsLD AU were under-predicted. Our model appears to be accurate and relatively unbiased, however is conservative at high disease incidence.
