Assessing the Impact of the IOM Report on the Future of the National Guideline Clearinghouse™

On September 19, 2011, Richard N. Shiffman made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (4.3 MB). Plugin Software Help.

Slide 1

Assessing the Impact of the Institute of Medicine (IOM) Report on the Future of the National Guideline Clearinghouse™

Richard N. Shiffman, MD, MCIS
Yale School of Medicine
New Haven, Connecticut, USA

Slide 2

Overview

• IOM process.
• Proposed standards for developing trustworthy guideline recommendations.
• Recommendations to AHRQ.
• Rich discussion!

Slide 3

What is the Institute of Medicine?

• Independent, non-profit, non-governmental organization.
• Health component of the National Academy of Sciences.
• Aim: To help those in government and the private sector make informed health decisions by providing evidence upon which they can rely.
• Expertise in constituting panels of stakeholders and creating summary reports.
• Provides unbiased and authoritative advice to decisionmakers and the public.

Slide 4

Image: A series of images and logos are shown.

Slide 5

A New Definition for Clinical Practice Guidelines (CPGs)

• Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (IOM 1990).

Slide 6

To Be Trustworthy, Guidelines Should...

• Be based on a systematic review of the existing evidence.
• Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups.
• Consider important patient subgroups and patient preferences as appropriate.
• Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest.
• Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations.
• Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

Slide 7

IOM's Two Types of Advice

• Standard = "a process, action, or procedure for developing CPGs that is deemed essential to producing scientifically valid, transparent, and reproducible results".
• Recommendation.

Slide 8

"AHRQ should require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines submitted to it adhere to standards for trustworthiness."

• Review the 8 standards for trustworthy guidelines.
• ECRI = AHRQ's contractor for NGC.
• How can NGC/ECRI monitor compliance?

Slide 9

Establishing Transparency

• 1.1 The processes by which a clinical practice guideline (CPG) is developed and funded should be detailed explicitly and publicly accessible.

Slide 10

A Transparent Development Process Makes Clear...

• Who (expertise, bias).
• How authors weighed:
  • Evidence.
  • Pathophysiologic reasoning (first principles).
  • Expert experience.
  • Patients' and society's values.
• Allows users to judge reasonableness of recommendations.

Slide 11

Management of Conflict of Interest

• 2.1 Prior to selection declare all interests and activities.
• 2.2 Disclose COIs within GDG.
• 2.3 Divest financial investments of panelists and their family members and not participate in marketing activities or advisory boards.
• 2.4 Members with COIs should be a minority of the GDG:
  • The chair or co-chairs should not have COI.
  • Funders should have no role in CPG development.

Slide 12

Composition of Guideline Development Group (GDG)

• 3.1 The GDG should be multidisciplinary and balanced.
• 3.2 Include a patient and a patient advocate or patient/consumer organization representative at least at the time of clinical question formulation and draft CPG review.
• 3.3 Adopt strategies to increase effective participation of patient and consumer representatives.

Slide 13

Intersection of Clinical Practice Guideline and Systematic Review

• 4.1 Use systematic reviews that meet standards set by the IOM Committee on Standards for Systematic Reviews.
• 4.2 The GDG and systematic review team should interact.

Slide 14

Establishing Evidence Foundations and Rating Strength of Recommendations (Summary)

5.1 For each recommendation provide:

• A summary of relevant available evidence (and evidentiary gaps), description of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence.
• A clear description of potential benefits and harms.
• An explanation of the part played by values, opinion, theory, and clinical experience in deriving the recommendation.
• A description of any differences of opinion regarding the recommendation.
• A rating of the level of confidence in the evidence.
• A rating of the strength of the recommendation.

Slide 15

Articulation of Recommendations

• 6.1 Articulate recommendations in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed.
• 6.2 Word strong recommendations so that compliance can be evaluated.

Slide 16

External Review

• 7.1 External reviewers should comprise a full spectrum of relevant stakeholders.
• 7.2 The authorship of external reviews should be kept confidential unless that protection has been waived.
• 7.3 The GDG should consider all external reviewer comments and keep a record of the rationale for modifying or not modifying in response to reviewers' comments.
• 7.4 Make a draft of the CPG�prior to the final draft�available to the general public for comment.

Slide 17

Updating

• 8.1 Document the CPG publication date, date of systematic evidence review, and proposed date for future review in the CPG.
• 8.2 Monitor the literature to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.
• 8.3 Update CPGs when new evidence suggests the need.

Slide 18

Recommendation

• To be trustworthy, a clinical practice guideline should comply with proposed standards 1-8.
• Optimally, CPG developers should adhere to these proposed standards and CPG users should adopt CPGs compliant with these proposed standards.

Slide 19

Additional Recommendations

• The Secretary of HHS should establish a public-private mechanism to examine—at the request of developer organizations—the procedures they use to produce their clinical practice guidelines.
• And to certify whether these organizations' CPG development processes comply with standards for trustworthy CPGs.

Slide 20

NICE (UK) Certifies Organizations

• Reviews procedures (and products) from applicant organizations using AGREE.
• Internal and external reviewers.
• Draft decision posted on web with public consultation.
• Organizations that meet accreditation requirements and agree to maintain the approved processes during a 3-year accreditation period.
• Receive a mark to be placed on future CPGs of organizations.
• Accreditor may review organizational procedures at any point during the accreditation period and, if non-compliance with accreditation requirements is detected, withdraw accreditation.

Slide 21

Recommendations to AHRQ

• The committee heard testimony that the NGC does not set sufficiently high standards to assure users that poor-quality guidelines are not admitted.
• NGC should eliminate CPGs for which trustworthiness cannot be determined, and identify the trustworthiness of those retained.
• Guidelines that have not included a thorough systematic review of the relevant scientific evidence base should be excluded from the NGC.
  • Findings of no scientific evidence resulting from an SR should not preclude listing of the CPG in the NGC.

Slide 22

Additional Recommendations

• NGC should prominently identify guidelines originating from CPG developers certified by the designated mechanism as trustworthy.
• CPGs from an organization that requested and failed review by the certifying mechanism should also be identified in a special category, with standards met and shortcomings specified.
• The proposed standards will require additional NGC effort as current NGC abstraction does not require review of development process data adequate to meet the requirements of the proposed standards.
• NGC needs to be funded at a sufficient level for it to improve the quality, timeliness, and trustworthiness of its CPGs and other products.

Slide 23

Finally, AHRQ should...

• Conduct research on the causes of inconsistent CPGs and strategies to encourage their harmonization.
• Assess the strengths and weaknesses of IOM standards by pilot-test.
• Estimate the validity and reliability of the proposed standards.
• Evaluate the effectiveness of interventions to encourage standards implementation.
• Evaluate the effects of standards on CPG development, healthcare quality, and patient outcomes.

Slide 24

Image: Two faces are shown. The first is of features of a untrustworthy face, the second is of features that appear trustworthy.