Evaluating the Process of Creating and Promoting a Quality Report

An evaluation of the process you went through to develop, release, and promote your report can help you identify which aspects worked well. Just as important, you can learn which ones did not work as well and what could or should be done differently.

A good process evaluation is not designed to get you a “grade.” It is designed to help you learn and improve by answering a number of important questions.

Questions About the Planning Process

• How well did your original plan work out?
  • Did you leave out key steps?
  • Did you leave enough time?
  • Was there enough money in the budget?
  • Were you able to adapt your plan to respond to any emerging issues?
• Did you engage with the right partners at the right time?
• How effectively did you handle disagreements, conflicts, and controversies?

Questions About the Development of the Report

• What did you learn about your audience? Did you learn enough to target your report to their needs and preferences? Did you use what you learned?
• Did the measures you chose resonate with your audience? Was there enough variation in scores to get people’s attention?
• Did you design the report based on the best available evidence?
• Did you develop graphs and charts that people could understand easily and quickly?
• Did your explanatory material make sense to people?
• Was your report designed explicitly to motivate people to examine and use the data?

Questions About the Distribution and Promotion of Your Report

• If you did a print report, did your distribution methods get it to as many people in your audience as possible?
• If you did a Web report, did you design, place, and promote it so people could and did easily find it?
• Did you get the kind and amount of media attention you wanted? Did it seem to make a difference?
• Were you able to get ads placed where you wanted, when you wanted, at a reasonable price or for free? What messages worked? Which didn’t?
• Did your outreach efforts find those in your audience who are not necessarily literate or Web literate?
  • Did they feel as if the report was “for them?”
  • Did you build long-term relationships with your outreach partners?

Evaluation in the Early Stages

To refine your efforts, you can also evaluate the process in its early stages, which would include both the design and development stages as well as the first few months after the report is released. Evaluators refer to this early-stage investigation as a formative evaluation. The audience for a formative evaluation is typically the people doing the work and those who oversee them directly. While a formative evaluation should focus on your processes, it is also helpful to try to get early indicators of your results.

You might want to undertake a formative evaluation to address these questions:

• Is your plan being carried out?
• How is the process affecting your stakeholders in the short term?
• Did you choose the right audience and audience segment?
• Are people seeing your report?
• Is your report easy to understand and use?
• Did you get initial media attention or ongoing media attention?
• Are different advertising and outreach strategies working and which are working best?

Also in "What You Can Evaluate"

• Evaluating the Process
• Evaluating Results
• Evaluating Cost-Effectiveness