Implementing the GRADE Method in Guideline Development: Real-World Experiences

Contemplation Stage: To GRADE or Not to GRADE?

On September 15, 2009, Sheila A. Agyeman, MHA made this presentation at the 2009 Annual Conference. Select to access the PowerPoint® presentation (1.7 MB) (Plugin Software Help).

Slide 1

Implementing the GRADE Method in Guideline Development: Real-World Experiences
Contemplation Stage: To GRADE or Not to GRADE?

Sheila A. Agyeman, MHA
Director of Evidence Based Medicine
American Gastroenterological Association Institute
September 15, 2009

Slide 2

AGA Institute Guideline Development Process Prior to 2007

Searching for and appraising the evidence

• Authors performed literature searches
• Table below was provided to aide authors in organizing findings but not a requirement
• Evidence summarized in Technical Review document
• Technical review (intended for academics) summarized into Medical Position Statement (intended for practicing clinicians)

Slide 3

AGA Institute Guideline Development Process Prior to 2007

Inconsistencies identified in Pre-2007 Process

• Technical review and medical position statement not adequately evidence based
• Disconnect between technical review and medical position statement
  • Not guiding readers in interpreting the data
  • Not giving clear clinical recommendations
• Concept of quality of evidence and strength of recommendations not well emphasized

Slide 4

AGA Institute Guideline Development Current Process

Modifications made to the process

• Construction of specific clinical questions to be addressed in technical review
• Implementation of U.S. Preventive Services Task Force (USPSTF) Ratings - Management of GERD guideline
• Composition of Medical Position Panel
  • Patient/patient advocate
  • Payer
  • Gastroenterologist in community practice
  • Gastroenterologist with knowledge in health services research
  • Primary care physician, surgeon or pathologist (included as needed based on topic)

Slide 5

AGA Institute Guideline Development Current Process

Challenges of current process

• USPSTF grading system primarily designed for preventive services
• USPSTF grading system changed after first guideline was already in the final stages (mid 2007)
• USPSTF grading system caused confusion amongst guideline authors (Dysplasia in IBD guideline) how to grade evidence and recommendations (e.g., diagnostic tests)

Slide 6

AGA Institute Guideline Development Current Process

What made the AGA Institute decide to consider switching from USPSTF to using GRADE?

• Invitations to collaborate with other GI and non-GI societies
• Establish uniformity across all GI societies
  • Further invitations for collaboration led to the discovery that more societies are using or also considering GRADE
  • Considered using GRADE to reduce the confusion in different grading schemes used in various guidelines
  • Joining effort to establish an international standard

Slide 7

AGA Institute Guideline Development Current Process

Benefits of using GRADE we considered:

• A very rigorous and formalized process that will ensure evidence-based guidelines necessary for value-based health care initiatives (e.g., P4P)
• Already closely resembles AGA Institute's current process
• Provides increased transparency of judgments of quality of evidence and strength of recommendations
• Investment into GRADE evidence profiles in the long-term could be future asset at the time of guidelines update Availability of GRADE methodologists who could assist in the process

Slide 8

AGA Institute Guideline Development Current Process

Initial reservations about using GRADE

• Higher costs?
  • Need to hire full time methodologist?
    • Answer: GRADE not necessarily more complex than other grading systems.
  • Increased training of staff and authors?
    • Answer: Any grading system would require some form of training
• Requires more time and lengthens the development process?
  • Answer: No. The most time consuming and costly phase of the guideline development process is the systematic review of the evidence. This is independent from the grading system used.

Slide 9

AGA Institute Guideline Development Current Process

Next Steps

• Develop an implementation plan
  • Train staff in GRADE process
  • Train committee members in GRADE process
  • Include GRADE experts on committee
• Include in budget cycle for 2010-2011