You are here:
Model application niche analysis: Assessing the transferability and generalizability of ecological models
Citation:
Moon, J., Ted DeWitt, M. Errend, Randy Bruins, M. Kentula, S. Chamberlain, M. Fennessy, AND K. Naithani. Model application niche analysis: Assessing the transferability and generalizability of ecological models. Ecosphere. ESA Journals, 8(10):e01974, (2017).
Impact/Purpose:
A team of EPA and university scientists are developing a methodology to assess the reliability of using ecological models (such as those used to estimate ecosystem service stocks and production, or ecosystem condition) applied to locations or contexts different from where a model was originally developed. Developing new models of a site of interest may be prohibitively expensive, and thus analysts frequently use existing models when developing estimates of an ecological quantity, process, or service; however, model transfers may not be appropriate (and thus inaccurate) because of contextual differences between the development site and the application site. In this paper, the team develops reviews past considerations of model generality and transferability, develops a conceptual framework centered on comparison of the ecological contexts of new site and previous locations where the model was applied, and demonstrates a methodology for assessing the transferability based on the performance of the model as a function of the similarity of the contexts of the new and previous sites. This methodology will be useful for identifying which environmental models are most appropriate to be used at sites where data are limited.
Description:
The use of models by ecologist and environmental managers, to inform environmental management and decision-making, has grown exponentially in the past 50 years. Due to logistical, economical and theoretical benefits, model users are frequently transferring preexisting models to new sites where data is scarce. Modelers have made significant progress in understanding how to improve model transferability and generalizability during the model development stage. However, models will always be both imperfect representations of systems and limited by the contextual frameworks that were used during model construction. Thus, model users need better ways to evaluate the risks of misuse when transferring models to new sites. Given the diversity of models and model practices in ecology, there are numerous challenges to providing a common framework for a model transfer risk assessment. We propose adopting a meta-analytic approach to assessing this risk that we call a model application niche meta-analysis. This approach synthesizes the sources of information provided by past model transfers to inform model selection, which can be visualized with model performance curves and decision landscape plots. These diagrams characterize a model’s application niche as a function of model performance and uncertainty across multiple dimensions of context, and are a means to evaluate a model’s transferability and generalizability. To highlight the utility of this analysis (i.e., optimal data conditions) we demonstrate the approach using a simple empirical plant nativity model developed for wetlands across Pennsylvania, and its transferability to application sites across the conterminous United States from data collected during the U.S. Environmental Protection Agency’s (EPA) National Wetland Condition Assessment (NWCA) in 2011. With this methodology we set the stage for the development of consistent and transparent practices for deciding the risks of transferring a model to new sites.
