Impact/Purpose:

Initiate a dialogue within the Agency about Expert Elicitation and to facilitate development of Expert Elicitation guidance for EPA.

Description:

The EPA is frequently called upon to make decisions on complex environmental issues, and to make these decisions, analyses are required from a broad range of disciplines. In all of these analyses, uncertainty and variability exist in estimates of the values of key parameters and in the choices of models. In some cases, critical data may be unavailable or inadequate. While the presence of uncertainty is a complicating factor in environmental decision making processes, EPA must still make timely environmental decisions. Thus, the influence of uncertainty must be addressed in all decisions. There are numerous methods to quantitatively characterize uncertainty. Expert elicitation (EE) has long been recognized as a powerful and legitimate tool for quantitative characterization of uncertainty and for providing distributions where there are data gaps and it is not feasible to conduct additional experimental research. There has been a recent increase in interest in the use of expert elicitation as a tool to be used in regulatory analyses and support of EPA decision making. A recent National Academy of Sciences report (2002) recommended that EPA use expert elicitation to improve its characterization of uncertainties in its health benefits analyses for air pollution regulations. More recently, the Office of Management and Budget (OMB) suggested using expert elicitation as a tool in addressing Circular A-4 requirements (OMB, 2004) for probabilistic uncertainty analysis and as part of their proposed risk assessment bulletin (OMB, 2006). Provisions for expert elicitation were also included in the recently revised cancer guidelines (EPA, 2005). EPA’s experience conducting EEs has been limited, and there are no clear Agency guidelines available to assist in the conduct and use of such techniques for regulatory analyses or other purposes. Given the limited EPA experience in using this methodology, there is a need for a thoughtful discussion to investigate the conduct and use of EE to support EPA regulatory and non-regulatory analyses and decision-making. With the increased identification of EE to meet various uncertainty requirement there will be a desire to move forward with such efforts. Any early efforts have the potential to set precedents on how analyses may be conducted, and more importantly how these are used to support decision-making. Such precedents may have unintended consequences on seemingly unrelated tasks. Therefore, a dialogue on the regulatory, legal, statutory implications in addition to the technical aspects of such efforts is important to minimize such unintended consequences. Therefore, the EPA Science Policy Council formed the Expert Elicitation Task Force to initiate a dialogue within the Agency about these methods and to facilitate development of EE guidance for EPA. The Task Force facilitated a series of discussions to explore the potential utility of using EE and to evaluate and address issues that may arise from using this tool. Based on those discussions, the Task Force has developed a set of findings and recommendations concerning 1) when it is appropriate to use EE (i.e., what constitutes “good practice” in deciding whether to conduct an EE), 2) how to plan and conduct such assessments, and 3) how to present and use the results of such assessments. The Task Force also has identified various steps that it recommends be taken to facilitate future development and application of this tool within EPA. This White Paper is not intended to present official EPA guidelines or policy since input from a range of stakeholders within and outside the Agency were not formally solicited. However, one of the Task Force’s recommendations in this White Paper is that such guidance or policy be developed. It is hoped that this White Paper will help guide individual offices and regions in their early efforts or applications of EE until such final and official EPA policy on EE is developed and that this White Paper may be used to facilitate the development of future EE guidance or policy.

Record Details: