Step 4: Design Implementation Program and Assemble the Watershed Plan (cont.)

Create an Evaluation Framework
You should review your watershed program for two primary reasons. First, you want to be able to prove, or demonstrate, that by implementing the management measures, you are achieving your water quality and other environmental goals. Second, you want to be able to continually improve your program in terms of efficiency and quality. This adaptive management process should be built into your program before implementation so that you ask the right questions and use the answers to strengthen your program. Collecting information does no good if you don’t use the information to improve your watershed program.

In general, you will evaluate three major parts of your watershed implementation program to be able to demonstrate progress and make improvements in your program. You need to structure your evaluation framework to consider all three components and develop indicators that will measure each. The components are inputs, outputs, and outcomes. When filling in these components, you’ll work backward, starting with your desired outcomes (goals) and working toward identifying the specific inputs needed to achieve those outcomes.

1. Inputs: the process used to implement your program. This includes inputs to your program such as resources of time and technical expertise, organizational structure and management, and stakeholder participation.
  Sample evaluation questions:
• Are the human and monetary resources allocated sufficient to carry out the tasks?
• Did stakeholders feel they were well represented in the process?
2. Outputs: the tasks conducted and the products developed. This includes the implementation activities such as installing management practices, developing brochures, holding workshops, preparing fact sheets, and so forth.
  Sample evaluation questions:
• Are we meeting our implementation schedule?
• Are we meeting our milestones?
• Did we meet our milestones sooner than expected?
• Did we reach the appropriate target audiences with our I/E materials?
3. Outcomes: the results or outcomes seen from implementation efforts. This includes increased awareness and behavior changes among the watershed community, and environmental improvements such as water quality, habitat, and physical changes. Outcomes can be further broken down into short-term outcomes and long-term outcomes.
  Sample evaluation questions:
• Did the target audience increase its awareness of watershed issues?
• Did the behaviors of the target audience change as a result of implementing the watershed plan?
• Are we meeting our interim targets for pollutant load reductions?
• Are pollutant loads being reduced?

Many programs use a logic model to evaluate their programs. A logic model is a visual representation of your program showing the inputs, outputs and outcomes. This type of model is useful because it puts all this information in one place and summarizes a complex program in a simple picture. It also provides a to do list for evaluation, signaling what needs to be evaluated and when. The sample logic model shown here is for a program aimed at restricting cattle from a creek to reduce sediment and nutrient loading.

In addition to the logic model, you can use many others methods and tools to evaluate your watershed program, such as baseline surveys, focus groups, direct measurements, and stakeholder interviews. The important point is to determine what methods you will use before your implement your program. Identifying these methods will help make sure you are collecting information that will directly relate to your program. For example, if you wish to do any before-and-after comparisons, you should have baseline information with which you can compare the final results. The methods will be used to measure the indicators you have selected. For each indicator selected, you will identify the method for measuring the indicator.

Remember that you’ll need to evaluate your program before you’ve started to implement it, during implementation, after activities have been completed, and after the entire project is complete.