Advanced Methods in Delivery System Research - Planning, Executing, Analyzing, and Reporting Research on Delivery System Improvement

Webinar #2: Statistical Process Control (Slide Presentation)

This slide presentation was presented on May 14, 2013.

Presenter: Jill Marsteller, PhD, MPP
Discussant: Stephen Alder, PhD
Moderator: Cindy Brach, MPP

Sponsored by AHRQ's Delivery System Initiative in partnership with the AHRQ PCMH program.

Slide 1

Advanced Methods in Delivery System Research –Planning, Executing, Analyzing, and Reporting Research on Delivery System Improvement 

Webinar #2: Statistical Process Control

Text description is shown below.

Presenter: Jill Marsteller, PhD, MPP
Discussant: Stephen Alder, PhD
Moderator: Cindy Brach, MPP

Sponsored by AHRQ's Delivery System Initiative in partnership with the AHRQ PCMH program.

May 14, 2013

Slide 2

Speaker Introductions

Text description is shown below. .

Jill Marsteller, PhD, MPP is currently an Associate Professor of Health Policy and Management at the Johns Hopkins Bloomberg School of Public Health.

Dr. Marsteller’s presentation today will draw on her paper with Mimi Huizinga and Lisa Cooper on Statistical Process Control. This AHRQ PCMH Research Methods Brief is posted on the AHRQ PCMH website. Details will be provided at the end of this webinar.

Stephen C. Alder, PhD serves as chief of the  Division of Public Health in the University of Utah Department of Family and Preventive Medicine. He is an associate professor of Family and Preventive Medicine.

Dr. Alder is currently working on an AHRQ-funded demonstration grant on “Primary Care Practice Redesign – Successful Strategies.” His presentation today is based on one part of the research conducted under that grant.

Images of Dr. Marsteller and Dr. Alder are also shown on this slide.

Slide 3

Statistical Process Control--Possible Uses to Monitor and Evaluate Patient-Centered Medical Home Models

Text description is shown below.

Jill A. Marsteller, PhD, MPP
Center for Health Services and Outcomes Research, Johns Hopkins Bloomberg School of Public Health
and Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine

With Thanks to Mimi Huizinga, MD, Melissa Sherry, MPH and Lisa Cooper, MD

Slide 4

Statistical Process Control

Text description is shown below.

  • Typically used for quality control:
    • Developed in 1920s at Bell Telephone Laboratories by Walter Shewart to aid in the production of telephone components that were of uniform quality.
    • Based on theory of variation.
    • Long history of use within manufacturing.
    • Gaining popularity in health care:
      • The Joint Commission uses SPC to analyze hospital performance.
  • A key SPC tool is the control chart, which is the focus of this presentation:
    • Combines time-series analysis with graphical representation of data.

Slide 5

Control Charts Are a Primary Tool of SPC

Text description is shown below.

  • Allows determination of system’s “control”:
    • Wide fluctuations = out-of-control systems.
    • Out-of-control indicates opportunity to improve reliability.
  • Distinguishes between common- and special-cause variation:
    • Common-cause variation = normal, random variation.
    • Special-cause variation:
      • Changes in the pattern of data that can be assigned to a specific cause.
      • Cause may or may not be beneficial, intentional.

To the right of the text are images of two line graphs. One is titled "Common-Cause Variation" and the other one "Special-Cause Variation".

Slide 6

Features of an SPC Chart

Text description is shown below.

Image of SPC chart titled Example: 30 Day All Cause Readmission Rate. Certain features are highlighted on the chart: upper control limit, lower control limit, mean of all data points, benchmark based on trends for similar populations.

 

Slide 7

The Type of Control Chart Is Based on Your Data and Needs

Text description is shown below.

Image of a flow chart. The source cited is Radiographics, 2012 Nov-Dec;32(7):2113-26.

Slide 8

Why Should We Consider Using a Control Chart?

Text description is shown below.

  • Differentiates true change from random noise.
  • Emphasizes early detection of meaningful change.
  • Visualization can engage additional stakeholders.
  • Allows timing and degree of intervention impact to be detected.

Below the text is the image of a control chart titled "Did the intervention really decrease the use of CT scans?" The source cited is Radiographics, 2012 Nov-Dec;32(7):2113-26.

Slide 9

Application to Health Care

Text description is shown below.

  • Reducing variation in the delivery of health care is core tenet of highly reliable care.
  • Most often used for Quality Improvement and practice management.
  • Also useful as an easily interpretable approach for evaluating health care delivery system interventions.

Statistical process control (SPC) is a branch of statistics that combines rigorous time-series analysis methods with graphical presentation of data, often yielding insights into the data more quickly and in a way more understandable to lay decision makers.

—JC Benneyan et al., Qual Saf Health Care 2003;12:458–464

Slide 10

Using a Control Chart to Evaluate an Intervention

Text description is shown below.

  • Establish common-cause variation in a stable period:
    • Observe process or outcome variables over time in the absence of an intervention.
  • Monitor data for evidence of special-cause variation after intervention is introduced:
    • This indicates meaningful change.
    • Can be used to examine implementation or impact variables.

Slide 11

Methodology: Key Steps

Text description is shown below.

  1. Identify process(es) or outcome(s) of interest.
  2. Identify measurable attributes.
  3. Select appropriate control chart given your variable of choice.
  4. Use SPC software to generate chart type and compute mean value over time period of interest.
  5. Characterize natural variation using upper and lower control limits (± 3 SDs around mean).
  6. Track variable to observe patterns.
  7. Determine whether changes in variable over time meet criteria indicating special cause.

Slide 12

Methodology: Special-Cause Variation Criteria*

Text description is shown below.

  • One value outside control limits.
  • 2 of 3 consecutive values above or below mean and >2 SDs away from mean.
  • ≥ 8 values above or below mean, OR
  • ≥ 6 values in a row steadily increasing or decreasing.
  • Four out of five successive points more than 1SD from the mean on the same side of the center line.
  • Obvious cyclical behavior.
  • If these rules apply, the chance that changes seen are due to circumstances beyond regular variation is 99.7% (Benneyan et al. 2003).

* There are several special-cause variation criteria sets.

Slide 13

Identifying Significance in an SPC Chart: Examples

Text description is shown below.

Image of an SPC chart titled "Example: 30 Day All Cause Readmission Rate". Certain features on the chart are highlighted: four out of five successive data points more than 150 from the mean on the same side of the center line, a data point outside of upper or lower limit, six successive data points increasing or decreasing.

 

Slide 14

Uses of Control Charts

Text description is shown below.

  • Monitor process measures.
  • Identify early signs of correlation between processes and outcomes.
  • Identify differences across groups.
  • Aid self-management interventions:
    • Monitor changes in individual patients (e.g., clinical outcomes, patient experience, financial measures).
  • Determine time from implementation to effect.

Slide 15

Example: Control Chart of Appointment Access Satisfaction

Text description is shown below.

Image of a control chart showing progress over several years in appointment access satisfaction.

Slide 16

Example: Control Chart of Infectious Waste

Text description is shown below.

Image of a control chart showing two line graphs. The first graph shows the baseline average pounds of waste per day and the second chart shows the average pounds per day after changes were implemented.

Slide 17

Limitations (1)

Text description is shown below.

  • Requires frequent measurement:
    • Less data than traditional regression analysis (e.g., fewer sites or subjects), but control charts are only useful with data over many time periods.
  • Involves some degree of autocorrelation:
    • Problem amplified with more frequent measurement (e.g., hourly vs. daily).
    • Can reduce by using measurements 3 to 4 periods apart.

Slide 18

Limitations (2)

Text description is shown below.

  • Will not work in every situation:
    • For example, seasonal variability can impact control chart’s usability
    • Must understand process and goals of improvement before using control charts
  • Requires expert consultation for initial use.

Slide 19

Thanks! Questions?

Text description is shown below.

Slide 20

Statistical Process Control in Primary Care and Practice Redesign

Text description is shown below.

AHRQ Delivery System Initiative & AHRQ Primary Care Initiative
Statistical Process Control
Advanced Methods Webinars

Stephen C. Alder, Ph.D.
Chief, Division of Public Health
Family and Preventive Medicine

May 14, 2013

Slide 21

Overview

Text description is shown below.

  • Statistical Process Control in Primary Care
  • Disease surveillance and the 2002 SLC Olympics
  • Point-of-Care Testing as an Influenza Surveillance Tool
  • Use of SPC in Practice Redesign- Colon Cancer Screening Example

Slide 22

Division of Public Health

Text description is shown below.

Image of the logo for the University of Utah School of Medicine Department of Family & Preventive Medicine Division of Public Health. Underneath the logo is the caption "The Community is our Patient".

Slide 23

First Observations of Primary Care System Shifts

Text description is shown below.

Joseph Lynn Lyon, MD, MPH – mid-1990s, observed that patient loads in urgent care clinics increased as the influenza epidemic emerged.

Traditional influenza surveillance provided an epidemic post mortem rather than providing preventive benefit.

Slide 24

Public Health Surveillance 2002 Olumpics

Text description is shown below.

Images of the 2002 Salt Lake City Utah Olympics.

Slide 25

Olympic Preparation Post 2001

Text description is shown below.

  • September 11, 2001: Terrorist attacks on World Trade Center (NYC), Pentagon (Washington, DC) and Shanksville (Pennsylvania).
  • September 17 – November 20, 2001: Anthrax letters attack.
  • Athletes begin arriving in Salt Lake City mid-January (as the annual influenza epidemic is emerging).

Slide 26

U. (University of Utah) to monitor athletes for bioterror symptoms computers will look for any medical quirks

Text description is shown below.

By Norma Wagner
 Deseret News staff writer
Friday, Jan. 11 2002

‘U. researchers since mid-October have been developing a surveillance system that analyzes data from electronic medical records to continuously monitor for abnormal patterns in patients' symptoms that could flag a bioterrorism attack’

Slide 27

Syndromic Surveillance: Increased Gastro-intestinal Distress

Text description is shown below.

  • Paper-Pencil surveillance shows significant increase in clinically identified GI distress.
  • SPC-based surveillance using University of Utah Community Clinics EMR system allows near real-time capacity to both detect and investigate significant system shifts.
  • Quickly identified GI involvement in culture/rapid test-confirmed influenza.

Slide 28

Point-of-Care Testing as an Influenza Surveillance Tool: Methodology and Lessons Learned from Implementation

Text description is shown below.

Lisa H. Gren, Christina A. Porucznik, Elizabeth A. Joy, Joseph L. Lyon, Catherine J. Staes, and Stephen C. Alder

Influenza Research and Treatment (2013), Article 242970

Slide 29

Images of 2 Line Graphs

Text description is shown below.

Image of 2 line graphs showing the proportion of visits and ILI cases from 2007-2008 and percentage of positive rapid tests.

Slide 30

Image of Line Graph

Text description is shown below.

Image of line graph showing a timeline of percentage of visits with positive rapid test.

Slide 31

2004-2005 to 2007-2008 Influenza Seasons

Text description is shown below.

Box and whiskers plot showing notification days relative to epidemic onset at CDC, UDOH (3.6%), UDOH (1.7%), and UUPCRN.

Slide 32

Improving Colon Cancer Screening Rates in Primary Care: A Pilot Study Emphasizing the Role of the Medical Assistant

Text description is shown below.

A N  Baker, M Parsons,  S M Donnelly, L Johnson, J Day, A Mervis, B James, R Burt, M K Magill

Qual Saf Health Care 2009:18:355-350. doc10.1136/qshc.2008.027649

Slide 33

Table 1 Medical Assistant Screening Script

Text description is shown below.

I notice that you may be due for colon cancer screening. Have you had a colonoscopy?

  "Yes"

     Let me see if I have a copy of that report.

       If you have a copy of that report, update health maintenance tab or make a note of date on progress note.

       If you do not have a copy, ask when/where it was done and give patient a record release to send for report.

  "No"

     The American Cancer Society recommends colon cancer screening for everyone age 50. Have you thought about having a colonoscopy?

      Enter preliminary order for physician review and decision or give patient information, depending on their response.

      If patient has had a flexible sigmoidoscopy or completed stool cards, make a note.

Slide 34

Table 2 Manual Colon Cancer Screening Log

Text description is shown below.

Review ActivityOrder activity
No pop-up (no screening needed)Had colonoscopy (records in EpicCare® but need to update Health Maintenance section of the record)Had colonoscopy (no records in EpicCare®)Sent releaseDue for colonscopy (patient refused)No documentation (forgot to ask)Ordered colonoscopyOrdered flexible sigmoidoscopyOrdered faecal occult blood testOther

Slide 35

Figure 1. Colonoscopy Referrals for Patients Not Current at Encounter, EMR, Electronic Medical Record; MA, Medical Assistant

Text description is shown below.

Image of chart showing expanded role of medical assistants to give colonoscopy referrals due to EMRs from 2003 to 2005.

Slide 36

Conclusion

Text description is shown below.

Electronic decision support tolls alone do not increase CRC screening referral rated. Facilitators, IT support staff and system changes were all necessary to effect change. The greatest barrier to CRC screening for providers seemed to be competing demands during a short patient visit. Adding redesigned clinical workflow, particularly an expanded role for the MA, appeared to increase referrals for CRC screening.

Slide 37

Conclusion (continued)

Text description is shown below.

Statistical Process Control provides an important tool for monitoring variation in a system and identifying when transitions occur. Application of this approach can be at the patient, provider and population levels. Identifying transitions is a start – impact requires understanding the cause and adapting the system to reduce variation (improve the control) and improve performance.

Slide 38

Thank You for Attending!

Text description is shown below.

Webinar #3: Logic Models

Presenter: Dana Petersen, PhD
Discussant: Todd Gilmer, PhD
Moderator: Michael I. Harrison, PhD

Sponsored by AHRQ’s Delivery System Initiative in partnership with the AHRQ PCMH program

June 4, 2013 1 p.m.

Register for this and other events in our series on advanced methods in delivery system research here: http://bit.ly/EconometricaAHRQ

Slide 39

For More Information About the AHRQ PCMH Research Methods Briefs, Please Visit:

Text description is shown below.

http://www.pcmh.ahrq.gov/portal/server.pt/community/pcmh__home/1483/pcmh_evidence___evaluation_v2

Current as of July 2013
Internet Citation: Advanced Methods in Delivery System Research - Planning, Executing, Analyzing, and Reporting Research on Delivery System Improvement: Webinar #2: Statistical Process Control (Slide Presentation). July 2013. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/prevention-chronic-care/improve/coordination/webinar02/spcslides.html