Many environmental regulatory agencies depend on models to organize, understand, and utilize the information for regulatory decision making. A general analytical protocol was developed to quantify prediction error associated with commonly used surface water quality models. Its application is demonstrated by comparing water quality models configured to represent different levels of spatial, temporal, and mechanistic complexity. This comparison can be accomplished by fitting the models to a benchmark data set. Once the models are successfully fitted to the benchmark data, the prediction errors associated with each application can be quantified using the Monte Carlo simulation techniques.