Brief Summary of the GRADE Framework (Text Version)

Slide Presentation from the AHRQ 2011 Annual Conference

Brief Summary of the GRADE Framework

Slide Presentation from the AHRQ 2011 Annual Conference

On September 20, 2011, Holger Schünemann made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (620 KB). Plugin Software Help.

Slide 1

Brief Summary of the GRADE Framework

Holger Schünemann, MD, PhD
Chair and Professor, Department of Clinical Epidemiology & Biostatistics
Professor of Medicine
Michael Gent Chair in Healthcare Research
McMaster University, Hamilton, Canada

AHRQ, Washington
September 19, 2011

Slide 2

Image: A map of the northeast section of the United States and Canada is shown. Over the map is an image of McMaster University.

Slide 3

Disclosure

• Co-chair GRADE Working Group.
• Work with various guideline groups using GRADE.
• No direct personal for profit payments for work related to the topic area.
• American College of Physicians (ACP) Clinical Practice Guidelines Committee.
• WHO: Expert Advisory Panel on Clinical Practice Guidelines and Clinical Research Methods and Ethics & chair of various guideline panels.

Slide 4

Healthcare Problem

Image: A large gray box. To the upper left of the box are words "Healthcare problem" and an arrow pointing to the top left corner of the box. An arrow pointing from the lower right corner of the box goes to the word "recommendation". On the upper right corner of the box is a text balloon with the words: "Healthy people", Herd immunity", "Long term perspective", "Few RCTs", and "Lots of other things".

Slide 5

Simple Hierarchies Are (Too) Simplistic

Study Design:

• Randomized Controlled Trials.
• Cohort Studies and Case Control Studies.
• Case Reports and Case Series, Non-systematic observations.

To the right of the text is a large red triangle. At the top of the triangle is the word "Bias". To the right triangle are the words "Expert Opinion".

Schünemann & Bone, 2003

Slide 6

GRADE: Recommendations & Quality of (A Body of) Evidence

Clear separation, but judgments required:

1. Recommendation: 2 grades—conditional (aka weak) or strong (for or against an action)?
   • Balance of benefits and downsides, values and preferences, resource use and quality of evidence.
2. 4 categories of quality of evidence:
   ++++(High), +++(Moderate), ++(Low), +(Very low)?
   • Methodological quality of evidence.
   • Likelihood of bias related to recommendation.
   • By outcome and across outcomes

*www.GradeWorking-Group.org

Slide 7

Evidence Based Healthcare Decisions

Image: A Venn diagram. The three variables: State and circumstances, Population/societal values and preferences, and Evidence about effects merge into Expertise.

Haynes et al. 2002

Slide 8

GRADE Quality of Evidence

In the context of making recommendations:

• The quality of evidence reflects the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation.

Slide 9

Likelihood Of and Confidence in an Outcome

Image: A cartoon is shown. There are two men in an office with the words "Weather Bureau" on the door talking and one man says to the other "I figure there's a 40% chance of showers, and a 10% chance we know what we're talking about."

Figure 1. Belief and confidence: a two-dimensional weather report. (Reprinted by permission from the Wall Street Journal).

Slide 10

Determinants of quality

• RCTs ++++.
• Observational studies ++.
• 5 factors that can lower quality:
  1. Limitations in detailed design and execution (risk of bias criteria).
  2. Inconsistency (or heterogeneity).
  3. Indirectness (PICO and applicability).
  4. Imprecision (number of events and confidence intervals).
  5. Publication bias.
• 3 factors can increase quality:
  1. Large magnitude of effect.
  2. Plausible residual bias or confounding.
  3. Dose-response gradient.

Slide 11

Quality Assessment Criteria

Slide 12

Interpretation of Grades of Evidence

• ++++/A/High: We are very confident that the true effect lies close to that of the estimate of the effect.
• +++'B/Moderate: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.
• ++C/Low: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect.
• +/D/Very low: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.

Slide 13

Interpretation of Grades of Evidence (continued)

• ++++/A/High: Further research is very unlikely to change confidence in the estimate of effect.
• +++/B/Moderate: Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate.
• ++/C/Low: Further research is very likely to have an important impact on confidence in the estimate of effect and is likely to change the estimate.
• +/D/Very low: We have very little confidence in the effect estimate: Any estimate of effect is very uncertain.

Slide 14

Image: A flow chart of the Systematic Review process is shown.

Slide 15

GRADE Uptake

• World Health Organization.
• Allergic Rhinitis in Asthma Guidelines (ARIA).
• American Thoracic Society.
• American College of Physicians (ACP).
• Canadian Task Force for the Preventive Services.
• European Respiratory Society.
• European Society of Thoracic Surgeons.
• British Medical Journal.
• Infectious Disease Society of America.
• UpToDate®.
• National Institutes of Health and Clinical Excellence (NICE).
• Scottish Intercollegiate Guideline Network (SIGN).
• Cochrane Collaboration.
• Clinical Evidence.
• Agency for Health Care Research and Quality (AHRQ).
• Partner of GIN.
• Over 50 major organizations (over 250 members).

Current as of December 2011

Internet Citation:

Brief Summary of the GRADE Framework. Slide Presentation from the AHRQ 2011 Annual Conference (Text Version). December 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualconf11/berkman_schunemann/schunemann.htm