Grantee Research Project Results
2020 Progress Report: Estimation of Spatially Explicit Water Quality Benefits throughout River Systems: Development of Next Generation Stated Preference Methods Using National Probability Samples and Online Labor Pools
EPA Grant Number: R836167Title: Estimation of Spatially Explicit Water Quality Benefits throughout River Systems: Development of Next Generation Stated Preference Methods Using National Probability Samples and Online Labor Pools
Investigators: Johnston, Robert J , Wollheim, Wil , Moeltner, Klaus
Institution: Clark University , University of New Hampshire , Virginia Tech
EPA Project Officer: Packard, Benjamin H
Project Period: April 1, 2016 through March 31, 2019 (Extended to March 31, 2022)
Project Period Covered by this Report: April 1, 2020 through March 31,2021
Project Amount: $799,919
RFA: Water Quality Benefits (2015) RFA Text | Recipients Lists
Research Category: Water
Objective:
This project will develop novel, modular approaches to stated preference valuation designed to address the challenges of use/nonuse value estimation for complex, temporally/spatially explicit aquatic ecosystem change with heterogeneous benefits for different user and nonuser groups. The approach, denoted Free-form Choice Experiments (FCEs), will restructure the way that willingness to pay (WTP) is elicited and estimated in survey-based valuation, hybridizing methods from contingent valuation, choice experiments, revealed preference modeling, online labor pool sampling, and Bayesian econometrics. The approach will be developed to estimate WTP for water quality and ecosystem service improvements throughout river networks. FCEs are designed to be transformative across multiple dimensions, including: (a) novel ways to elicit stated preferences and design surveys, with scenarios coupled directly to ecological models, (b) flexible value elicitation and modeling that generates benefit functions linked to temporally/spatially explicit effects, (c) an ability to evaluate the relevance of many ecological indicators to different user/nonuser groups, (d) novel modeling of data using Bayesian econometrics, and (e) new approaches to stated preference sampling using online labor pools. Biogeochemical forecasts for valuation scenarios will be projected using FrAMES (Framework for Aquatic Modeling of the Earth System), a process-based spatially explicit water quality model.
Progress Summary:
We have completed Tasks I - VII outlined in the proposal. Tasks VIII, IX and X are in progress. Survey development has been completed, included development of water quality scenarios and indicators, bid design, choice questions around scenarios, and additional questionnaire materials. Questionnaires were pretested and revised with input from focus groups and cognitive interviews, with additional feedback from external experts in stated preference methodology. This input was combined to develop final survey templates. We jointly developed scenario design with Bayesian econometric models suitable for model estimation and have evaluated model performance using simulated response data. We completed GIS map zooming and tracking functionality to enable respondents to view different water quality changes at any desired scale and resolution within the study area, with the resulting “map tracking” data available to populate the econometric model. The questionnaire was pilot tested on AWS Mechanical Turk (abbreviated as “MTurk”) platform and Qualtrics online panel. The first round of final sampling was conducted via Qualtrics panel. The final questionnaire sampling wave, implemented via address-based random population sampling (ABS), is in the field and will be completed by June 2021. Econometric analysis, hypothesis testing and development of benefit functions for water quality improvements is ongoing. We are also evaluating differences between online panel and ABS results.
Water Quality Indicators and Scenarios: Details of water quality indicators are provided in prior reports. An aquatic life indicator was developed based on a range of chloride concentrations that affect aquatic organism survival. A water safety indicator for drinking and recreation was developed based on fecal coliform count guidelines. An indicator for overall water quality was developed as a combined metric using all three modeled solute concentrations, compared to reference conditions for the case study watersheds. These indicators were quantified using FrAMES predictions across three scenarios for the Northeast US: (1) contemporary conditions with current environmental management practices, (2) a future climate scenario, with no updates to current environmental management, and (3) a future scenario with each proposed program for management. Grounded in this model, we developed 41 policy scenarios that included combinations of region-wide management actions to generate management-oriented scenarios of water quality change for the valuation survey.
Questionnaire Design, Pretesting and Implementation: Complete versions of questionnaires were developed, pretested, finalized, and are hosted on Qualtrics. Spatially explicit levels for water quality indicators were mapped across the study area. Valuation scenarios compare outcomes for these indicators with and without proposed management alternatives. These were combined with bid levels designed with input from focus groups, cognitive interviews and survey pretests, reflecting the presented annual cost per household in taxes and fees ($30, $60, $120, $240, $480, $720, $960, $1200). Within the survey, each respondent considers a single, hypothetically binding, binary referendum question designed to meet conditions for incentive compatibility. All scenarios are illustrated for a case study area of approximately 95,800 miles of rivers and streams in Connecticut, Massachusetts, Rhode Island, Vermont, New Hampshire and Maine, over 71,992 square miles of land. All maps have GIS zoom capability, enabling respondents to view water quality baselines and changes at any desired magnification and local area. All map interactions are captured and recorded to a DynamoDB JavaScript Object Notation database. Seven focus groups with non-expert respondents were used for survey pretesting, with additional one-on-one pretest interviews. Four final pilot tests were also conducted prior to final survey implementation (Mturk, N=200 and 136; Qualtrics panel, N=336 and 60). Final survey implementation occurred via online panel (Qualtrics convenience panel) and a mixed-mode, push-to-web, address-based probability sample (ABS) of New England residents. The first main wave was implemented via Qualtrics panel during November 2020 – February 2021 (N=3,364). Respondents were drawn from residents of the six New England states, with quotas for age, gender, income, and education to approximate regional Census distributions. Data quality screens were used to ensure response validity. The mixed-mode, address-based push-to-web sample was drawn from comprehensive ABS mailing list covering all New England states, with 7,167 names/addresses drawn from each state (total mailing sample of N=43,002). Participation was solicited through personalized letters with unique ID numbers and passwords for each respondent. The initial invitation letter was followed by a reminder post card and letter. The final mailings occurred during May - June 2021 and responses are still being received.
Bayesian Econometric Models and Preliminary Results: Econometric tasks completed this past project year included: (i) analysis of map tracking data by respondents, (ii) estimation of Bayesian logit models with and without model search, using actual field data (collected by Qualtrics), and (iii) incorporating (i) into (ii). Initial results of econometric modeling suggest that respondents primarily valued overall changes in water safety, as shown on the overview page for a given cleanup scenario, but were also able to discern quality changes in their home county and zip code from the maps. Additional analysis is ongoing.
Future Activities:
Work during year six will focus on the completion of all remaining tasks described in the proposal. Remaining tasks include estimating and comparing results from the online panel and probability samples (Task VIII), evaluating model performance and testing hypotheses related to WTP for water quality change (Task IX), and producing benefit functions able to predict per household and aggregate WTP for water quality improvements (Task X), and writing/submitting associated publications.
Journal Articles:
No journal articles submitted with this report: View all 6 publications for this projectSupplemental Keywords:
Media: water, watersheds; Ecosystem Protection: ecosystem, indicators, aquatic, habitat; Public Policy: decision making, cost-benefit, non-market valuation, contingent valuation, survey, preferences, public good, Bayesian, willingness-to-pay; Disciplines: social science, economics, ecology, hydrology; Methods/Techniques: modeling, analytical, surveys; Geographic Areas: Northeast, EPA Region 1; Other: ecosystem service, choice modeling, choice experiment, nonuse value, welfare analysis, water quality.Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.
Project Research Results
- Final Report
- 2019 Progress Report
- 2018 Progress Report
- 2017 Progress Report
- 2016 Progress Report
- Original Abstract
1 journal articles for this project