You are here:
Correlated Metrics Yield Multimetric Indices with Inferior Performance
Citation:
VAN SICKLE, J. Correlated Metrics Yield Multimetric Indices with Inferior Performance. TRANSACTIONS OF THE AMERICAN FISHERIES SOCIETY. American Fisheries Society, Bethesda, MD, 139:1802-1817, (2010).
Impact/Purpose:
Multimetric indices (MMIs) are widely used to assess the ecological health of freshwater ecosystems.
Description:
Multimetric indices (MMIs) are widely used to assess the ecological health of freshwater ecosystems. An MMI is a sum of 5-15 standardized, numeric variables (or “metrics”), each representing a different attribute of a biological assemblage. Many researchers believe that highly correlated metrics should not be included in the same MMI because they convey redundant information. To seek evidence for (or against) this belief, I compared the performances of 1000 MMIs created for each of 8 existing data sets, by randomly resampling metrics from sets of previously-identified candidates. An MMI’s performance was measured by its ability to differentiate between assemblages sampled in independently-assessed “reference” and “impaired” streams. For 7 of 8 data sets, multiple linear regressions fitted to each set of 1000 MMIs predicted a decrease in MMI performance as the mean correlation magnitude between metrics increased, after adjusting for the average responsiveness of individual metrics to the distinction between reference and impaired conditions. However, similar regressions showed that the size of the largest correlation between any 2 metrics in an MMI had no effect on its performance. Thus, minimizing the mean of metric correlations is more effective than the widespread practice of setting an upper correlation limit, when optimal MMI performance is desired. Finally, an MMI had originally been built for each data set by selecting one set of individually “best” metrics. In 15 of 16 cases, 5-57% of the randomly-selected MMIs outperformed the original MMI. Because individually-best metrics rarely yielded a best-performing MMI, I recommend assessing multiple candidate MMI’s, rather than just multiple candidate metrics.