Grantee Research Project Results
2001 Progress Report: Continued Development of Methods for Characterizing and Ranking Health, Safety, and Environmental Risks
EPA Grant Number: R827920Title: Continued Development of Methods for Characterizing and Ranking Health, Safety, and Environmental Risks
Investigators: Morgan, M. Granger , Dekay, Michael L. , Fischbeck, Paul S.
Institution: Carnegie Mellon University
EPA Project Officer: Chung, Serena
Project Period: December 15, 1999 through December 14, 2001
Project Period Covered by this Report: December 15, 2000 through December 14, 2001
Project Amount: $235,504
RFA: Decision-Making and Valuation for Environmental Policy (1999) RFA Text | Recipients Lists
Research Category: Environmental Justice
Objective:
The specific methods that have been employed in the majority of current risk-ranking exercises have received little attention. In the case of many of the U.S. Environmental Protection Agency (EPA)-sponsored regional ranking exercises, this has not been a serious problem because the primary objective has been to promote dialog among stakeholders. However, if regulatory agencies are going to use risk-ranking results in support of risk management decisionmaking, they must be based on normatively justifiable terms. Researchers in the Department of Engineering and Public Policy at Carnegie Mellon have begun the development, empirical testing, and refinement of such a method for ranking health and safety risks. This research project is designed to complete this work and extend it to include environmental risks.
Progress Summary:
We continued to make excellent progress on all phases of the work, and we completed two studies that extend previous research in environmental psychology, ecology, conservation, and comparative risk assessment on attributes for use in characterizing ecological risks.
In the first study, we used a systematic process to develop candidate attributes to describe a comprehensive, multi-media set of activities and stressors. Aggregate-level factor of analysis data collected from laypeople has confirmed many previous findings from the literature about the perception of ecological risks. However, our results suggest that the factors underlying perception of ecological risks are not uncorrelated, as previously assumed. Moreover, multiple-regression results suggest that aesthetic impacts affect judgments of ecological risk, even when impacts on humans, other species, and habitats are held constant.
In the second study, we used a subset of attributes, activities, and stressors to examine these relationships among members of four stakeholder groups (laypeople, environmentalists, and ecological risk assessors from government and industry). Results indicate that these four groups view the risks similarly (the group-level factor structures are very similar), but the groups differ in that the factors are related to risk judgments (e.g., compared to the other groups, laypeople place more emphasis on aesthetics and scientific understanding than on the impacts of species and habitats). Compared to group membership, differences in participants’ worldviews played only minor roles in determining the relationships between the factors and risk judgments. Differences between aggregate- and individual-level analyses, differences between hazard- and participant-based analyses, and the common confound between these two distinctions also are considered. This research project has provided us with a firm basis on which to incorporate ecological risks and their attributes into our risk-ranking method.
In parallel with these efforts, we defined a new testbed for experimental risk-ranking studies. Our previous testbed allowed us to study only risks to health and safety. We subsequently defined a new testbed and developed a new set of 10 risk summary sheets to describe risks in the United States. Copies of these are being posted on the Web site. Creating these sheets was very labor intensive because they required a great deal of risk-specific information, and because they were subjected to intensive review and refinement. After developing the risk summary sheets, we used them in a number of risk-ranking exercises with risk managers and laypeople. Specifically, we used the new summary sheets in short courses at Harvard's School of Public Health and at an annual meeting of the Society of Environmental Toxicology and Chemistry (SETAC), and with three groups of laypeople at Carnegie Mellon. Results indicate that our risk-ranking method can be used successfully to simultaneously compare health, safety, and environmental risks. More specifically, measures of the consistency of the rankings produced by the holistic and multi-attribute procedures, explicit and implicit measures of participants’ satisfaction with the process and the resulting rankings, and levels of agreement among individuals and groups are similar to results from our earlier studies that used only health and safety risks. Notably, all groups ranked some ecological attributes above some health and safety attributes, and ranked some purely ecological risks above some purely health and safety risks.
In addition to the work on extending our risk-ranking method to cover ecological risks and attributes, we also have continued work on refining the previously developed method for ranking health and safety risks. We have completed a study examining the relative effects of different types of risk information on the ranking results. Individual laypeople ranked our full set of 22 health and safety risks in each of four conditions: simple verbal descriptions only, anonymous (e.g., Risk A, Risk B) numerical attribute tables, verbal descriptions plus numerical attribute tables, and full risk summary sheets. The results were consistent with those from an earlier pilot study in which individual undergraduates ranked 12 of the risks in these conditions. Compared to the condition that included verbal descriptions, there was more consistency among rankings (and between individual rankings and a ranking based only on expected mortality) in the three conditions that only included the numerical attribute table. These results strongly suggest that laypeople are very responsive to quantitative risk information when it is presented in a coherent, easy-to-understand format. Risk judgments were much more idiosyncratic (i.e., there was very little consistency among participants) when such information was omitted from the descriptions of the hazards.
We have reconsidered the relative appeal of several multi-attribute models based on participants' rankings of the relative importance of the various attributes (e.g., rank-order-centroid, reciprocal-of-the-rank, and rank-sum models for inferring risk rankings from attribute rankings). Although we have suggested that the reciprocal model worked well for the attributes of health and safety risks, recent papers and feedback at presentations have encouraged us to reconsider the rank-order-centroid model. In addition, the larger number of attributes required for the description of ecological risks, with the constraints of risk-ranking exercises (e.g., the desire for simple spreadsheet implementation, transparency for the participants, and a more even-weighting profile) caused us to reconsider the rank-sum model. In pilot studies with health, safety, and ecological risks, these models implied rankings that were reasonably consistent with participants' holistic rankings; we opted for the simpler rank-sum model in our most recent risk-ranking exercises. We are evaluating these models on the basis of the many risk-ranking exercises that we have conducted with risk managers and laypeople.
Finally, we have argued that simply reporting the results of a risk-ranking was not sufficient, that decisionmakers would need a "thick description" which highlights sources of agreement and disagreement, and notes risks that are likely to be problematic. Draft materials are currently under development. These will serve as the basis for an experimental study to determine the nature and structure of summary materials that people find most useful.
To assure the quality of the results, we will continue to use state-of-the-art methods for developing social science research materials, and for analyzing the results of our experiments. Results have been, and will continue to be, submitted to peer-reviewed journals for publication.
Future Activities:
We will focus on analysis of the data that have been collected and on the publication of these results. We also will continue to explore possibilities for a real-world application, either in North America or in Europe.
Journal Articles on this Report : 2 Displayed | Download in RIS Format
| Other project views: | All 26 publications | 7 publications in selected types | All 5 journal articles |
|---|
| Type | Citation | ||
|---|---|---|---|
|
|
Florig HK, Morgan MG, Morgan KM, Jenni KE, Fischhoff B, Fischbeck PS, DeKay ML. A deliberative method for ranking risks (I): overview and test bed development. Risk Analysis 2001;21(5):913-922. |
R827920 (2000) R827920 (2001) R827920 (Final) |
|
|
|
Morgan KM, DeKay ML, Fischbeck PS, Fischhoff B, Morgan MG, Florig HK. A deliberative method for ranking risks (II): evaluation of validity and agreement among risk managers. Risk Analysis 2001;21(5):923-938. |
R827920 (2000) R827920 (2001) R827920 (Final) |
|
Supplemental Keywords:
risk ranking, ecological risk, attributes of risk, measurement of ecological risk, risk characterization, risk assessment, ecosystem protection, ecosystem indicators, methods/techniques, exposure, public policy, community-based, preferences, psychological, modeling, survey., RFA, Scientific Discipline, Economic, Social, & Behavioral Science Research Program, Ecosystem Protection/Environmental Exposure & Risk, Ecology, Applied Math & Statistics, Ecosystem/Assessment/Indicators, Ecosystem Protection, Health Risk Assessment, Ecological Effects - Environmental Exposure & Risk, Ecological Effects - Human Health, Ecological Risk Assessment, Ecology and Ecosystems, decision-making, Sociology, Ecological Indicators, Psychology, Social Science, Economics & Decision Making, ecological exposure, deliberative policy, multi-objective decision making, valuation, risk reduction, risk characterization, environmental risks, decision analysis, risk management, multi-attribute utility, health and safety ranking, environmental decision-making, risk ranking, environmental policy, value-based judgement, multi-criteria, human health risk, stakeholderRelevant Websites:
http://www.epp.cmu.edu/research/risk_ranking.html Exit
Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.