Implementing the GRADE Method in Guideline Development: Real-World Experiences

Contemplation Stage: To GRADE or Not to GRADE?

On September 15, 2009, Sheila A. Agyeman, MHA made this presentation at the 2009 Annual Conference.

Slides

Slide 1

Slide 1. Contemplation Stage: To GRADE or Not to GRADE?

Implementing the GRADE Method in Guideline Development: Real-World Experiences
Contemplation Stage: To GRADE or Not to GRADE?

Sheila A. Agyeman, MHA
Director of Evidence Based Medicine
American Gastroenterological Association Institute
September 15, 2009

 

Slide 2

Slide 2. AGA Institute Guideline Development Process Prior to 2007: Searching for and appraising the evidence

AGA Institute Guideline Development Process Prior to 2007

Searching for and appraising the evidence

  • Authors performed literature searches
  • Table below was provided to aide authors in organizing findings but not a requirement
  • Evidence summarized in Technical Review document
  • Technical review (intended for academics) summarized into Medical Position Statement (intended for practicing clinicians)

 

Slide 3

Slide 3. AGA Institute Guideline Development Process Prior to 2007: Inconsistencies identified in Pre-2007 Process

AGA Institute Guideline Development Process Prior to 2007

Inconsistencies identified in Pre-2007 Process

  • Technical review and medical position statement not adequately evidence based
  • Disconnect between technical review and medical position statement
    • Not guiding readers in interpreting the data
    • Not giving clear clinical recommendations
  • Concept of quality of evidence and strength of recommendations not well emphasized

Slide 4

Slide 4. AGA Institute Guideline Development Current Process: Modifications made to the process

AGA Institute Guideline Development Current Process

Modifications made to the process

  • Construction of specific clinical questions to be addressed in technical review
  • Implementation of U.S. Preventive Services Task Force (USPSTF) Ratings - Management of GERD guideline
  • Composition of Medical Position Panel
    • Patient/patient advocate
    • Payer
    • Gastroenterologist in community practice
    • Gastroenterologist with knowledge in health services research
    • Primary care physician, surgeon or pathologist (included as needed based on topic)

 

Slide 5

Slide 5. AGA Institute Guideline Development Current Process: Challenges of current process

AGA Institute Guideline Development Current Process

Challenges of current process

  • USPSTF grading system primarily designed for preventive services
  • USPSTF grading system changed after first guideline was already in the final stages (mid 2007)
  • USPSTF grading system caused confusion amongst guideline authors (Dysplasia in IBD guideline) how to grade evidence and recommendations (e.g., diagnostic tests)

Slide 6

Slide 6. AGA Institute Guideline Development Current Process: What made the AGA Institute decide to consider switching from USPSTF to using GRADE?

AGA Institute Guideline Development Current Process

What made the AGA Institute decide to consider switching from USPSTF to using GRADE?

  • Invitations to collaborate with other GI and non-GI societies
  • Establish uniformity across all GI societies
    • Further invitations for collaboration led to the discovery that more societies are using or also considering GRADE
    • Considered using GRADE to reduce the confusion in different grading schemes used in various guidelines
    • Joining effort to establish an international standard

 

Slide 7

Slide 7. AGA Institute Guideline Development Current Process: Benefits of using GRADE

AGA Institute Guideline Development Current Process

Benefits of using GRADE we considered:

  • A very rigorous and formalized process that will ensure evidence-based guidelines necessary for value-based health care initiatives (e.g., P4P)
  • Already closely resembles AGA Institute's current process
  • Provides increased transparency of judgments of quality of evidence and strength of recommendations
  • Investment into GRADE evidence profiles in the long-term could be future asset at the time of guidelines update Availability of GRADE methodologists who could assist in the process

 

Slide 8

Slide 8. AGA Institute Guideline Development Current Process: Initial reservations about using GRADE

AGA Institute Guideline Development Current Process

Initial reservations about using GRADE

  • Higher costs?
    • Need to hire full time methodologist?
      • Answer: GRADE not necessarily more complex than other grading systems.
    • Increased training of staff and authors?
      • Answer: Any grading system would require some form of training
  • Requires more time and lengthens the development process?
    • Answer: No. The most time consuming and costly phase of the guideline development process is the systematic review of the evidence. This is independent from the grading system used.

Slide 9

Slide 9. AGA Institute Guideline Development Current Process: Next Steps

AGA Institute Guideline Development Current Process

Next Steps

  • Develop an implementation plan
    • Train staff in GRADE process
    • Train committee members in GRADE process
    • Include GRADE experts on committee
  • Include in budget cycle for 2010-2011
Current as of February 2009
Internet Citation: Implementing the GRADE Method in Guideline Development: Real-World Experiences: Contemplation Stage: To GRADE or Not to GRADE?. February 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2009/agyeman/index.html