You are here:
Model Performance Evaluation and Scenario Analysis (MPESA) Tutorial
Citation:
Mohamoud, Y. Model Performance Evaluation and Scenario Analysis (MPESA) Tutorial. U.S. EPA Office of Research and Development, Washington, DC, EPA/600/B-16/080, 2016.
Impact/Purpose:
Manual for model performance evaluation metrics and model diagnostics.
Description:
This tool consists of two parts: model performance evaluation and scenario analysis (MPESA). The model performance evaluation consists of two components: model performance evaluation metrics and model diagnostics. These metrics provides modelers with statistical goodness-of-fit measures that capture magnitude only, sequence only, and combined magnitude and sequence errors. The performance measures include error analysis, coefficient of determination, Nash-Sutcliffe efficiency, and a new weighted rank method. These performance metrics only provide useful information about the overall model performance. Note that MPESA is based on the separation of observed and simulated time series into magnitude and sequence components. The separation of time series into magnitude and sequence components and the reconstruction back to time series provides diagnostic insights to modelers. For example, traditional approaches lack the capability to identify if the source of uncertainty in the simulated data is due to the quality of the input data or the way the analyst adjusted the model parameters. This report presents a suite of model diagnostics that identify if mismatches between observed and simulated data result from magnitude or sequence related errors. MPESA offers graphical and statistical options that allow HSPF users to compare observed and simulated time series and identify the parameter values to adjust or the input data to modify. The scenario analysis part of the tool provides quantitative metrics on how model simulated scenario results differ from the baseline results and what impact these differences may have on aquatic organisms and their habitats (e.g., increased flood and drought frequency).