You are here:
Model application niche analysis: An approach for assessing the transferability and generalizability of ecological models
Citation:
Moon, J., Ted DeWitt, M. Errend, Randy Bruins, M. Kentula, S. Chamberlain, M. Fennessy, AND K. Naithani. Model application niche analysis: An approach for assessing the transferability and generalizability of ecological models. Ecological Society of America Annual Meeting, OR, Portland, August 06 - 11, 2017.
Impact/Purpose:
Frequently, environmental decision makers, managers, planners, and scientists need estimates of ecological stocks or processes at sites where little or no primary data or models exist, and they use existing data or models (i.e., that were measured at or developed for other locations) to obtain those estimates (e.g., they transfer those estimates or models to the new site). However, no standard methodology currently exists to assess whether those transfers are justifiable. Scientists at NHEERL Western Ecology Division in Newport, OR, have developed a framework to assess the transferability of ecological estimates or ecological models, and demonstrate the application of this methodology with an empirical model for plant biodiversity (e.g., an aspect of wetland ecological condition) transferred to wetlands across the US, based on applications of the model to Pennsylvania wetlands. The abstract contributes to SHC Project 2.61, Task 3.
Description:
A 30-year review of predictive models used in regulatory decision-making, revealed that transferring models to contexts other than that for which the models were developed was one of the biggest vulnerabilities to their legal defensibility. The use and transfer of models by ecologists, to inform environmental management and decision-making, has grown exponentially in the past 50 years. Given this trend, and the importance of public confidence in the decisions that are being made based on models, model users need better ways to evaluate the possibility of misuse when transferring models to new contexts. We present one approach, a model application niche analysis, where ecologists synthesize information from databases and past studies to create model performance curves and decision landscape plots. These visualization tools characterize a model’s application niche as a function of model performance and uncertainty across dimensions of context, as a means to evaluate both model transferability and generalizability. To demonstrate the utility of this approach, we evaluated an empirical model developed to predict the mean coefficient of conservatism (i.e., ecological condition) of plant communities in wetlands across Pennsylvania for transfers across the contiguous U.S. The model predictors include surface soil pH and the landscape development index, a measure of anthropogenic activity in the wetland’s surrounding landscape. Using model performance curves and decision landscape plots along latitude and elevation gradients, we show that this model is transferable to locations in the eastern, southeastern and Pacific Northwest regions of the U.S. where high forest cover, rainfall and geology contribute to acidic soil formation. We also show that model performance is weakest, and the model structure is questionable, in regions of the U.S. with basic soil conditions, such as the west and the tips of Florida and Louisiana. Our goal is to invoke further inquiries into the development of consistent and transparent practices for model selection when transferring ecological models.