Grantee Research Project Results
Final Report: Corporate Environmental Behavior: Examining the Effectiveness of Government Interventions and Voluntary Initiatives
EPA Grant Number: R831035Title: Corporate Environmental Behavior: Examining the Effectiveness of Government Interventions and Voluntary Initiatives
Investigators: Morgenstern, Richard , Pizer, Billy , Shih, Jhih-Shyang
Institution: Resources for the Future
EPA Project Officer: Hahn, Intaek
Project Period: June 1, 2003 through May 31, 2006
Project Amount: $359,895
RFA: Corporate Environmental Behavior: Examining the Effectiveness of Government Interventions and Voluntary Initiatives (2002) RFA Text | Recipients Lists
Research Category: Environmental Justice
Objective:
Voluntary programs have been a key part of U.S. climate change policy since the early 1990s. Such programs figured prominently in President Bush’s 2002 climate change policy announcement, referencing recent agreements with the semi-conductor and aluminum industries and leading to the creation of the Climate Leaders and Climate Vision programs. They were also the centerpiece of President Clinton’s 1993 Climate Change Action Plan, which included Energy Star, Rebuild America, Green Lights, Motor Challenge, the Voluntary Reporting of Greenhouse Gases Program (required under Section 1605(b) of the Energy Policy Act of 1992), and Climate Wise. In fact, a number of these programs were initiated in the George W. Bush Administration. A 2005 survey identified 87 voluntary programs at the U.S. Environmental Protection Agency (EPA), up from 54 in 1999 and 28 in 1996 (U.S. EPA 2005). In fiscal year 2006, voluntary programs comprised 1.6% of EPA’s operating budget. Dozens more programs operate at the U.S. Department of Energy (DOE) and other federal agencies and at the state level. While many of these programs focus on climate change and energy, others cover waste, water, toxics, and agriculture. Voluntary programs have figured prominently in the national climate policies of other countries as well, and continue to play a leading role in Japan.
The prominence of voluntary programs begs the obvious question of whether or not these programs are effective in achieving the stated goals. For example, following the 2002 announcement by President Bush of voluntary efforts to achieve an 18% improvement in greenhouse gas intensity, recent data indicating that we are track to meet that 18% goal has been cited as evidence of the voluntary program’s success (White House 2006). But is the data really evidence of the voluntary program’s success, or does it simply reflect other coincident events?
The key issue considered in this research is whether or not these programs actually work as advertised. That is, do voluntary programs deliver the promise of significant environmental gains without the burdens associated with mandatory regulation? Do they improve environmental and conservation outcomes relative to a realistic baseline, or do they pave the way for other actions that do? Quantitatively, how large are the likely gains? Can such approaches serve as a substitute for mandatory requirements or should only modest gains be expected from these efforts? Unfortunately, the existing literature—which primarily emphasizes the motivation of firms to participate rather than the environmental accomplishments of the programs—provides only limited answers to these questions.
As is well known, program evaluation in the absence of randomized experiments is difficult because the decision to participate may not be random and, in particular, may be correlated with the outcomes. The present research is designed to overcome these problems by measuring the environmental effectiveness of two voluntary climate change programs -- EPA’s Climate Wise program, and DOE’s Voluntary Reporting of Greenhouse Gases Program, 1605(b) -- with particular attention to the participation decision and how various assumptions affect estimates of program effects. For both programs, the analysis focuses on manufacturing firms and uses confidential Census data to create a comparison group as well as measure outcomes (expenditures on fuel and electricity).
Although these programs are not the most recent voluntary initiatives, and did not benefit from potential improvements in voluntary program design that occurred over the past decade, the relatively long histories of these programs make them particularly amenable to statistical analysis. While 1605(b) is strictly a reporting program, both programs emphasize flexibility for the participants. Arguably, Climate Wise offers more ‘carrots’ in the form of technical assistance and public acknowledgement. It also imposes more ‘sticks’ in the sense of at least an implied expectation that participating firms will make larger emission reductions. Climate Wise is oriented entirely to non-electric utility firms, 1605(b) is open to a broader range of entities. However, because of the nature of our matching sample, (described below), our analysis of both Climate Wise and 1605(b) is limited to firms in the manufacturing sector.
As part of our effort to develop a credible baseline, we are fortunate to have access to confidential plant-level data files for the manufacturing sector collected by the U.S. Census Bureau. We consider two alternative approaches to evaluating outcomes, attempting to control for self-selection in joining the voluntary programs. In one approach we consider a model where program participation depends on both observed and unobservable variables that may be correlated with the outcome. In the other approach, we match participants to appropriate non-participants and consider pairwise differences. The latter method is referred to as propensity score matching.
Understanding the true effectiveness of these programs is important. Protagonists and antagonists of the trend toward voluntary approaches are increasingly at odds, sometimes drawing opposite conclusions about the same program. Protagonists, typically on the side of industry, see voluntary programs as a more practical, flexible approach to regulation. Antagonists, including some environmental advocates, often see voluntary programs as an obstacle to more stringent, mandatory programs. This polarization may be partly a consequence of poor information. While intuition and anecdotes may provide some reason for believing that a given program has or has not had a beneficial environmental impact, careful empirical analysis with peer review is much more convincing. The goal of this research is to help fill that void.
Summary/Accomplishments (Outputs/Outcomes):
Thus far, the rigorous assessment of the environmental performance of voluntary programs, especially climate-related programs, has been quite limited. The key challenge is to measure performance relative to a realistic baseline. The present research, which examines both a DOE- and an EPA-sponsored program, relies on confidential plant-level data for the manufacturing sector collected by the U.S. Census Bureau to develop such a baseline based on a comparable set of non-participant controls, focusing on activities through 2001 when available Census data ends. We consider two alternative approaches to evaluating outcomes, attempting to control for selection in joining the programs based on both observable as well as unobservable characteristics. In one approach we consider a structural model where program participation depends on both observed and unobservable variables that may be correlated with the outcome. In the other approach, we match participants to appropriate non-participants based on observable characteristics only, and consider pairwise differences – a method known as propensity score matching. The results are sobering.
In contrast to the claims of relatively large emission reductions reported by the sponsoring agencies, our analysis suggests more modest reductions are attributable to the programs studied. Overall, we find that that the effects from Climate Wise and 1605(b) on fuel and electricity expenditures are no more than 10% and likely less than 5%. There is no evidence of reductions in direct emissions from fossil fuels attributable to the voluntary programs; however, there is some statistically significant impacts on the use of electricity. In particular, there is some statistical evidence that participation in 1605(b) lead to a slight decrease in electricity expenditures, on the order of 4-8 percent. This decrease persists for at least three years. The statistically significant evidence on Climate Wise is that the program may be associated with a slight increase in electricity expenditures, although that effect vanishes after two years. Given the limitations of the analysis, we tend to discount these findings and conclude, instead, that in all likelihood, participation in Climate Wise has at most a negligible effect on emissions.
The findings of modest, albeit statistically significant, reductions in electricity expenditures for 1605(b) reporters may have implications for other government-sponsored voluntary programs as well. Recall the EIA observation that most of the entities reporting under 1605(b) are also affiliated with one or more other government-sponsored programs. Thus, the observed emission reductions for the 1605(b) reporters may reflect the influence not only of the 1605(b) program itself but also that of other programs. While our separate assessment of Climate Wise suggests that participation in that program is not likely associated with significant emission reductions, other larger programs, e.g., EPA’s Energy Star Program, may be more effective. Unfortunately, the EIA reporting form does not require disclosure of the name of any of the other individual programs in which the firm participates.
Methodologically, our research highlights the inevitable complexity of assessing voluntary programs. Our research reinforces the work of others in emphasizing the importance of distinguishing between the participation decision and the environmental outcomes achieved. Our work also points to the value of working with micro-level data, and the particular need to take special care in matching otherwise disparate samples to obtain a credible control group. This process is all the more difficult in our case, where the samples were not coded via a uniform system. In terms of estimation, we have applied two distinct methods to evaluating outcomes. One based on the work of Heckman and Hotz (1985), assumes that program participation depends on both observed and unobservable variables that may be correlated with the outcome. The other, propensity score matching, based on the work of Rosenbaum and Rubin (1983), matches participants to appropriate non-participants and considers pairwise differences. Because the Heckman and Hotz approach requires both a correct specification and identification of one or more excluded variables, it is more demanding than the Rosenbaum and Rubin approach which relaxes the specification assumption and does not require excluded variables (but does not allow for correlated, unobserved errors in the selection and outcome model). Because of our difficulty identifying excluded variables in the former method, our results seem more plausible with use of the latter approach, and we think such an approach may have wider application in the future evaluation of voluntary programs.
Overall, the evaluation of environmental programs seeks to determine what works and what does not. Our findings of at most a small effect should not be all that surprising. Energy-related greenhouse gas emissions are quite different than many other types of emissions, e.g., un-priced industrial byproducts such as toxics with no near-term localized effects whose existence was widely ignored until the 1980s and 1990s. With no practical opportunity for end-of-pipe abatement, reductions in energy-related greenhouse gas emissions often amount to reductions in energy use itself – something that has been picked over for some time. Given the underlying positive price on energy, there is always an incentive to reduce energy use. The existence of such underlying incentives, in turn, implies a far greater challenge for government in designing effective voluntary programs for industry.
Journal Articles:
No journal articles submitted with this report: View all 16 publications for this projectSupplemental Keywords:
voluntary programs, participation models, emission levels, environmental behavior, environmental performance, plant emissions, plant-level analysis, policy analysis, Longitudinal Research Database, Manufacturing Energy Consumption Survey, 1605(b), Climate Wise programs,, RFA, Scientific Discipline, Economic, Social, & Behavioral Science Research Program, Air, climate change, Air Pollution Effects, Corporate Performance, Economics and Business, decision-making, Atmosphere, Social Science, Economics & Decision Making, environmental performance, air pollution policy, emission levels, policy analysis, environmental monitoring, impact of federal policy instruments, participation model, government intervention, plant emissions, business-led environmental management, decision analysis, decision making, environmental decision making, environmental compliance determinants, environmental behavior, management participation, pollution prevention, plant-level analysis, corporate environmental behaviorRelevant Websites:
Progress and Final Reports:
Original AbstractThe perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.