3. How do we monitor the intervention and measure outcomes?

Questions To Consider in Developing an Antimicrobial Stewardship Program To Reduce C. Difficile Infection

At all points in this process you will need to monitor appropriate processes and outcomes of the stewardship efforts. The frequency that data are collected, compiled, and analyzed will vary by site. Perhaps your facility has internal committees or boards who require regular submission of data; perhaps your State has a reporting requirement (e.g., C. difficile is reportable in New York State).

It will be important to know the requirements at your facility so that you can make the proper preparations in terms of reporting mechanisms. In deciding how to monitor your intervention and outcomes, you should consider six questions. Some of the tools and resources from earlier sections will help you in this process.

3.1. How do we measure rates of C. difficile over time?

Many facilities are already tracking patient cases and rates of C. difficile. If your facility (or State/local health department) does not have a formal system, it will be easier to use or adapt from CDC's NHSN system rather than creating your own definitions and surveillance. The following are questions to consider:

  • Whom might we need help from?
  • What is key information to be aware of?
  • How should we track C. difficile cases?
  • Is it important to know whether C. difficile cases are acquired in our facility or in the community?
  • Are we mandated to report C. difficile? Are there Web resources to help with C. difficile surveillance?

Resources and Tools

Ø 3A RESOURCE: Hospital-Acquired Infection (HAI) Rates in New York State Hospitals

Ø 3B RESOURCE: About NHSN

Ø 3C RESOURCE: NHSN Manuals and Protocols

Ø 3D RESOURCE: NHSN Patient Safety Component, MDRO/CDI Module (customizable reporting forms, including Laboratory-Identified MDRO or CDI Event) (PDF File; Plugin Software Help)

Ø 3E RESOURCE: Instructions for Completion of the Laboratory-Identified MDRO or CDI Event Form

3.2. How do we obtain, measure, and analyze antibiotic data?

One of the biggest challenges in antimicrobial stewardship in general and specifically in measuring the potential effect of an intervention is obtaining needed data on antibiotic volume for more than one patient at a time. Once the data are obtained, challenges arise in cleaning, aggregating, summarizing, and comparing the data in a meaningful way. It is beyond the scope of this toolkit to address every potential issue, but the following questions may be helpful in guiding your process:

  • What are possible antibiotic data sources?
  • What are some common antibiotic data cleaning and aggregation challenges?
  • What are some antibiotic metrics we can consider using?

Resources and Tools

Ø 2B RESOURCE: A Comparison of Antibiotic Data Sources

Ø 2C RESOURCE: Possible Methods for Evaluating AntibioticUse

Ø 2G RESOURCE: A Review of Antimicrobial Stewardship Interventions With Suggested Process/Monitoring, Antibiotic Use, and Additional Measures of Impact

Ø 3F RESOURCE: World Health Organization Defined Daily Dose Definition and General Considerations (links on left side of page go to DDD lists)

3.3. How can we monitor the intervention and why should we?

It is important to monitor the interventions for multiple reasons. First, you need to be able to verify that the interventions are occurring. Next, you need to look for early and tangible signs of success (antibiotic metrics may be slow to show effects). Finally, you need to identify barriers to success and ways to improve the intervention, assess whether additional interventions are needed, assess whether the interventions are sustainable, and determine whether the intervention could be an effective way to affect other outcomes, such as C. difficile and antibiotic resistance.

This last point is important, because many stewardship interventions have been successful in reducing targeted antibiotic use and cost but have fallen short in reducing more concrete outcomes such as C. difficile rates and antibiotic resistance. If these outcomes are not achieved, monitoring the intervention is important to distinguish whether the intervention did not work because the activities were not completed; the intervention could not affect prescribing practices, antibiotic use, or outcomes; or insufficient time has elapsed to affect C. difficile rates.

Resources and Tools

Ø 2G RESOURCE: A Review of Antimicrobial Stewardship Interventions With Suggested Process/Monitoring, Antibiotic Use, and Additional Measures of Impact

3.4. What other processes do we need to monitor and measure?

It is also important to ensure that your interventions do not have unintended consequences for your patients. Thus, looking at rates of reinstitution of broad antibiotic therapy, missed or delayed antibiotic doses, or readmission (especially for infection) should be part of ongoing surveillance. In addition, your interventions may have other positive outcomes (e.g., reduced length of stay for patients receiving the intervention) that you will need to monitor and track.

Further, your institution's infection control, isolation precautions, and environmental cleaning policies and practices should be monitored throughout your intervention as changes in these can also affect C. difficile rates. Finally, it is important to assess how the intervention affects your stewardship team and other prescribers. Qualitative data collection such as prescriber surveys, informal discussions, or focus groups may enrich your understanding of the ASP and activities at your facility.

Resources and Tools

Ø 1L TOOL: Survey of Staff Attitudes Toward ASP and Current Practices

Ø 3G/3H RESOURCE: Environmental Cleaning and Infection Prevention Checklists

3.5. How do we analyze financial data?

It will be important to conduct a financial impact analysis at your facility to estimate the costs associated with running an ASP, including staffing, software, and equipment; estimate the cost savings from reduced antimicrobial use; and understand the potential reimbursement impact of reducing infections. In addition, you may want to estimate indirect savings, including savings on isolation equipment or estimated savings from meeting external quality measures, such as those from the Centers for Medicare & Medicaid Services.

You may choose to examine antibiotic purchasing data if those data are available, or you may need to use industry sources to estimate costs associated with antibiotic purchasing. Data from the hospital should be available as they are routinely compiled for internal hospital purposes or required external use (e.g., billing). In the current climate of competing demands, leadership is frequently interested in the ways patient safety initiatives translate into financial savings.

3.6. How do we assess the overall impact of our interventions? How do we decide which interventions have been the most successful (and which interventions were not) and why?

Factors to consider include:

  • Ability to implement the intervention (cost, staffing, IT support, timeframe).
  • Ability to potentially affect a large enough burden of target antibiotic and then reduce that antibiotic use (and not increase use of more worrisome antibiotic combinations).
  • Acceptability to stewardship team (does not impede other needed stewardship activities).
  • Acceptability to prescribers.
  • Effect on surrogate/process measure, long-term outcomes (such as C. difficile if possible), and lack of negative consequences.
  • Sustainability of the intervention.

Resources and Tools

Ø 2H RESOURCE: Specific Intervention Examples From ERASE C. difficile Project

Current as of September 2012
Internet Citation: 3. How do we monitor the intervention and measure outcomes?: Questions To Consider in Developing an Antimicrobial Stewardship Program To Reduce C. Difficile Infection. September 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/quality-patient-safety/patient-safety-resources/resources/cdifftoolkit/cdiff-questions3.html