Assessing the Impact of the IOM Report on the Future of the National Guideline Clearinghouse™

Slide Presentation from the AHRQ 2011 Annual Conference

On September 19, 2011, Richard N. Shiffman made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (4.3 MB). Plugin Software Help.

Slide 1

Assessing the Impact of the IOM Report on the Future of the National Guideline Clearinghouse™

Assessing the Impact of the Institute of Medicine (IOM) Report on the Future of the National Guideline Clearinghouse™

Richard N. Shiffman, MD, MCIS
Yale School of Medicine
New Haven, Connecticut, USA

Slide 2

Overview

Overview

  • IOM process.
  • Proposed standards for developing trustworthy guideline recommendations.
  • Recommendations to AHRQ.
  • Rich discussion!

Slide 3

What is the Institute of Medicine?

What is the Institute of Medicine?

  • Independent, non-profit, non-governmental organization.
  • Health component of the National Academy of Sciences.
  • Aim: To help those in government and the private sector make informed health decisions by providing evidence upon which they can rely.
  • Expertise in constituting panels of stakeholders and creating summary reports.
  • Provides unbiased and authoritative advice to decisionmakers and the public.

Slide 4

Develop evidence-based methodological standards for SRs and CPGs

Image: A series of images and logos are shown.

Slide 5

 

A New Definition for Clinical Practice Guidelines (CPGs)

  • Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (IOM 1990).

Slide 6

To Be Trustworthy, Guidelines Should . . .

To Be Trustworthy, Guidelines Should...

  • Be based on a systematic review of the existing evidence.
  • Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups.
  • Consider important patient subgroups and patient preferences as appropriate.
  • Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest.
  • Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations.
  • Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

Slide 7

IOM's Two Types of Advice

IOM's Two Types of Advice

  • Standard = "a process, action, or procedure for developing CPGs that is deemed essential to producing scientifically valid, transparent, and reproducible results".
  • Recommendation.

Slide 8

"AHRQ should require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines submitted to it adhere to standards for trustworthiness."

"AHRQ should require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines submitted to it adhere to standards for trustworthiness."

  • Review the 8 standards for trustworthy guidelines.
  • ECRI = AHRQ's contractor for NGC.
  • How can NGC/ECRI monitor compliance?

Slide 9

Establishing Transparency

Establishing Transparency

  • 1.1 The processes by which a clinical practice guideline (CPG) is developed and funded should be detailed explicitly and publicly accessible.

Slide 10

A Transparent Development Process Makes Clear…

A Transparent Development Process Makes Clear...

  • Who (expertise, bias).
  • How authors weighed:
    • Evidence.
    • Pathophysiologic reasoning (first principles).
    • Expert experience.
    • Patients' and society's values.
  • Allows users to judge reasonableness of recommendations.

Slide 11

Management of Conflict of Interest

Management of Conflict of Interest

  • 2.1 Prior to selection declare all interests and activities.
  • 2.2 Disclose COIs within GDG.
  • 2.3 Divest financial investments of panelists and their family members and not participate in marketing activities or advisory boards.
  • 2.4 Members with COIs should be a minority of the GDG:
    • The chair or co-chairs should not have COI.
    • Funders should have no role in CPG development.

Slide 12

Composition of Guideline Development Group (GDG)

Composition of Guideline Development Group (GDG)

  • 3.1 The GDG should be multidisciplinary and balanced.
  • 3.2 Include a patient and a patient advocate or patient/consumer organization representative at least at the time of clinical question formulation and draft CPG review.
  • 3.3 Adopt strategies to increase effective participation of patient and consumer representatives.

Slide 13

Intersection of Clinical Practice Guideline and Systematic Review

Intersection of Clinical Practice Guideline and Systematic Review

  • 4.1 Use systematic reviews that meet standards set by the IOM Committee on Standards for Systematic Reviews.
  • 4.2 The GDG and systematic review team should interact.

Slide 14

Establishing Evidence Foundations and Rating Strength of Recommendations (Summary)

Establishing Evidence Foundations and Rating Strength of Recommendations (Summary)

5.1 For each recommendation provide:

  • A summary of relevant available evidence (and evidentiary gaps), description of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence.
  • A clear description of potential benefits and harms.
  • An explanation of the part played by values, opinion, theory, and clinical experience in deriving the recommendation.
  • A description of any differences of opinion regarding the recommendation.
  • A rating of the level of confidence in the evidence.
  • A rating of the strength of the recommendation.

Slide 15

Articulation of Recommendations

Articulation of Recommendations

  • 6.1 Articulate recommendations in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed.
  • 6.2 Word strong recommendations so that compliance can be evaluated.

Slide 16

External Review

External Review

  • 7.1 External reviewers should comprise a full spectrum of relevant stakeholders.
  • 7.2 The authorship of external reviews should be kept confidential unless that protection has been waived.
  • 7.3 The GDG should consider all external reviewer comments and keep a record of the rationale for modifying or not modifying in response to reviewers' comments.
  • 7.4 Make a draft of the CPG–prior to the final draft–available to the general public for comment.

Slide 17

Updating

Updating

  • 8.1 Document the CPG publication date, date of systematic evidence review, and proposed date for future review in the CPG.
  • 8.2 Monitor the literature to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.
  • 8.3 Update CPGs when new evidence suggests the need.

Slide 18

Recommendation

Recommendation

  • To be trustworthy, a clinical practice guideline should comply with proposed standards 1-8.
  • Optimally, CPG developers should adhere to these proposed standards and CPG users should adopt CPGs compliant with these proposed standards.

Slide 19

Additional Recommendations

Additional Recommendations

  • The Secretary of HHS should establish a public-private mechanism to examine—at the request of developer organizations—the procedures they use to produce their clinical practice guidelines.
  • And to certify whether these organizations' CPG development processes comply with standards for trustworthy CPGs.

Slide 20

NICE (UK) Certifies Organizations

NICE (UK) Certifies Organizations

  • Reviews procedures (and products) from applicant organizations using AGREE.
  • Internal and external reviewers.
  • Draft decision posted on web with public consultation.
  • Organizations that meet accreditation requirements and agree to maintain the approved processes during a 3-year accreditation period.
  • Receive a mark to be placed on future CPGs of organizations.
  • Accreditor may review organizational procedures at any point during the accreditation period and, if non-compliance with accreditation requirements is detected, withdraw accreditation.

Slide 21

Recommendations to AHRQ

Recommendations to AHRQ

  • The committee heard testimony that the NGC does not set sufficiently high standards to assure users that poor-quality guidelines are not admitted.
  • NGC should eliminate CPGs for which trustworthiness cannot be determined, and identify the trustworthiness of those retained.
  • Guidelines that have not included a thorough systematic review of the relevant scientific evidence base should be excluded from the NGC.
    • Findings of no scientific evidence resulting from an SR should not preclude listing of the CPG in the NGC.

Slide 22

Additional Recommendations

Additional Recommendations

  • NGC should prominently identify guidelines originating from CPG developers certified by the designated mechanism as trustworthy.
  • CPGs from an organization that requested and failed review by the certifying mechanism should also be identified in a special category, with standards met and shortcomings specified.
  • The proposed standards will require additional NGC effort as current NGC abstraction does not require review of development process data adequate to meet the requirements of the proposed standards.
  • NGC needs to be funded at a sufficient level for it to improve the quality, timeliness, and trustworthiness of its CPGs and other products.

Slide 23

Finally, AHRQ should . . .

Finally, AHRQ should...

  • Conduct research on the causes of inconsistent CPGs and strategies to encourage their harmonization.
  • Assess the strengths and weaknesses of IOM standards by pilot-test.
  • Estimate the validity and reliability of the proposed standards.
  • Evaluate the effectiveness of interventions to encourage standards implementation.
  • Evaluate the effects of standards on CPG development, healthcare quality, and patient outcomes.

Slide 24

Trustworthy v. Untrustworthy Faces

Image: Two faces are shown. The first is of features of a untrustworthy face, the second is of features that appear trustworthy.

Page last reviewed March 2012
Internet Citation: Assessing the Impact of the IOM Report on the Future of the National Guideline Clearinghouse™. March 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2011/schiffman/index.html