You are here:
Systematic Review and Weight of Evidence are Integral to Ecological and Human Health Assessments: They need an Integrated Framework
Citation:
Suter, G., J. Nichols, E. Lavoie, AND S. Cormier. Systematic Review and Weight of Evidence are Integral to Ecological and Human Health Assessments: They need an Integrated Framework. Integrated Environmental Assessment and Management. Allen Press, Inc., Lawrence, KS, 16(5):718-728, (2020). https://doi.org/10.1002/ieam.4271
Impact/Purpose:
When environmental assessors assemble information to answer a question, they often obtain information from multiple studies; each study may provide a piece of evidence (e.g., a cancer test that provides evidence of carcinogenicity), multiple pieces of evidence (e.g., a positive cancer test that also provides mechanistic evidence), or information that contributes to a piece of evidence (e.g., a positive cancer test that is used with exposure estimates as evidence of causal sufficiency). In human health and ecological assessments of environmental contaminants, two practices have applied different approaches to assess information from multiple studies: weight of evidence (WoE) and systematic review (SR). The terms WoE and SR are used to refer to both practices that have different histories and traditions and to approaches for assembling and drawing inferences from information. The differences are substantial enough that the European Food Safety Authority has separate guidance for WoE and SR (EFSA 2010, 2017). Although SR practices have largely been distinct, in some organizations and contexts, including the U.S. Environmental Protection Agency, WoE and SR have begun to overlap (NRC 2018; USEPA 2018). The premise of this paper is that, the convergence should be encouraged and driven by an understanding of what each practice has to offer. This paper defines the background in terms of the development of the practices and their different approaches to assessment. The paper also defines where they have been successful and suggests an integrated framework. Our scope does not include planning processes such as problem formulation, scoping, or definition of the assessment question, because they are peripheral to the topic and should be performed for the assessment as a whole, not just WoE or SR. Rather, our focus is on the differences between SR and WoE practices and how traditional practices have led to differences in approaches to the assembly of information and use of information as evidence to make an inference. That analysis of the practices leads us to propose an explicitly integrated approach that we term Systematic Assembly and Weighing of Evidence.
Description:
Scientific assessments synthesize the various results of scientific research for policy and decision making. Synthesizing evidence in environmental assessments can involve either or both of two systems: systematic review (SR) and weight of evidence (WoE). SR was developed to systematically assemble results of clinical trials to be combined by meta-analysis. Weight of evidence (WoE) approaches have evolved from jurisprudence to make inferences from diverse bodies of evidence in various fields. Our objectives are to describe the similarities and differences between SR and WoE and suggest how their best practices can be combined into a general framework that is applicable to human health and ecological assessments. Integrating SR and WoE is based on the recognition that two processes are required, assembling evidence and making an inference. SR is characterized by methodical literature searching, screening, and data extraction, originally for meta-analysis but now for various inferential methods. WoE is characterized by systematically relating heterogeneous evidence to considerations appropriate to the inference and making the inference by weighing the evidence. SR enables the unbiased assembly of evidence from literature, but methods for assembling other information must be considered as well. If only one type of quantitative study estimates the assessment endpoint, meta-analysis is appropriate for inference. Otherwise, the heterogeneous evidence must be weighed. A framework is presented that integrates best practices into a methodical assembly and weighing of evidence. A glossary of terms for the combined practice and a history of the origins of SR and WoE are provided in supplemental material.
URLs/Downloads:
DOI: Systematic Review and Weight of Evidence are Integral to Ecological and Human Health Assessments: They need an Integrated Framework