Assessing the Impact of the IOM Report on the Future of the National Guideline Clearinghouse™
On September 19, 2011, Richard N. Shiffman made this presentation at the 2011 Annual Conference. Select to access the PowerPoint® presentation (4.3 MB). Plugin Software Help.
Slide 1
Assessing the Impact of the Institute of Medicine (IOM) Report on the Future of the National Guideline Clearinghouse™
Richard N. Shiffman, MD, MCIS
Yale School of Medicine
New Haven, Connecticut, USA
Slide 2
Overview
- IOM process.
- Proposed standards for developing trustworthy guideline recommendations.
- Recommendations to AHRQ.
- Rich discussion!
Slide 3
What is the Institute of Medicine?
- Independent, non-profit, non-governmental organization.
- Health component of the National Academy of Sciences.
- Aim: To help those in government and the private sector make informed health decisions by providing evidence upon which they can rely.
- Expertise in constituting panels of stakeholders and creating summary reports.
- Provides unbiased and authoritative advice to decisionmakers and the public.
Slide 4
Image: A series of images and logos are shown.
Slide 5
A New Definition for Clinical Practice Guidelines (CPGs)
- Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (IOM 1990).
Slide 6
To Be Trustworthy, Guidelines Should...
- Be based on a systematic review of the existing evidence.
- Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups.
- Consider important patient subgroups and patient preferences as appropriate.
- Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest.
- Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations.
- Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.
Slide 7
IOM's Two Types of Advice
- Standard = "a process, action, or procedure for developing CPGs that is deemed essential to producing scientifically valid, transparent, and reproducible results".
- Recommendation.
Slide 8
"AHRQ should require the National Guideline Clearinghouse (NGC) to provide a clear indication of the extent to which clinical practice guidelines submitted to it adhere to standards for trustworthiness."
- Review the 8 standards for trustworthy guidelines.
- ECRI = AHRQ's contractor for NGC.
- How can NGC/ECRI monitor compliance?
Slide 9
Establishing Transparency
- 1.1 The processes by which a clinical practice guideline (CPG) is developed and funded should be detailed explicitly and publicly accessible.
Slide 10
A Transparent Development Process Makes Clear...
- Who (expertise, bias).
- How authors weighed:
- Evidence.
- Pathophysiologic reasoning (first principles).
- Expert experience.
- Patients' and society's values.
- Allows users to judge reasonableness of recommendations.
Slide 11
Management of Conflict of Interest
- 2.1 Prior to selection declare all interests and activities.
- 2.2 Disclose COIs within GDG.
- 2.3 Divest financial investments of panelists and their family members and not participate in marketing activities or advisory boards.
- 2.4 Members with COIs should be a minority of the GDG:
- The chair or co-chairs should not have COI.
- Funders should have no role in CPG development.
Slide 12
Composition of Guideline Development Group (GDG)
- 3.1 The GDG should be multidisciplinary and balanced.
- 3.2 Include a patient and a patient advocate or patient/consumer organization representative at least at the time of clinical question formulation and draft CPG review.
- 3.3 Adopt strategies to increase effective participation of patient and consumer representatives.
Slide 13
Intersection of Clinical Practice Guideline and Systematic Review
- 4.1 Use systematic reviews that meet standards set by the IOM Committee on Standards for Systematic Reviews.
- 4.2 The GDG and systematic review team should interact.
Slide 14
Establishing Evidence Foundations and Rating Strength of Recommendations (Summary)
5.1 For each recommendation provide:
- A summary of relevant available evidence (and evidentiary gaps), description of the quality (including applicability), quantity (including completeness), and consistency of the aggregate available evidence.
- A clear description of potential benefits and harms.
- An explanation of the part played by values, opinion, theory, and clinical experience in deriving the recommendation.
- A description of any differences of opinion regarding the recommendation.
- A rating of the level of confidence in the evidence.
- A rating of the strength of the recommendation.
Slide 15
Articulation of Recommendations
- 6.1 Articulate recommendations in a standardized form detailing precisely what the recommended action is and under what circumstances it should be performed.
- 6.2 Word strong recommendations so that compliance can be evaluated.
Slide 16
External Review
- 7.1 External reviewers should comprise a full spectrum of relevant stakeholders.
- 7.2 The authorship of external reviews should be kept confidential unless that protection has been waived.
- 7.3 The GDG should consider all external reviewer comments and keep a record of the rationale for modifying or not modifying in response to reviewers' comments.
- 7.4 Make a draft of the CPG–prior to the final draft–available to the general public for comment.
Slide 17
Updating
- 8.1 Document the CPG publication date, date of systematic evidence review, and proposed date for future review in the CPG.
- 8.2 Monitor the literature to identify the emergence of new, potentially relevant evidence and to evaluate the continued validity of the CPG.
- 8.3 Update CPGs when new evidence suggests the need.
Slide 18
Recommendation
- To be trustworthy, a clinical practice guideline should comply with proposed standards 1-8.
- Optimally, CPG developers should adhere to these proposed standards and CPG users should adopt CPGs compliant with these proposed standards.
Slide 19
Additional Recommendations
- The Secretary of HHS should establish a public-private mechanism to examine—at the request of developer organizations—the procedures they use to produce their clinical practice guidelines.
- And to certify whether these organizations' CPG development processes comply with standards for trustworthy CPGs.
Slide 20
NICE (UK) Certifies Organizations
- Reviews procedures (and products) from applicant organizations using AGREE.
- Internal and external reviewers.
- Draft decision posted on web with public consultation.
- Organizations that meet accreditation requirements and agree to maintain the approved processes during a 3-year accreditation period.
- Receive a mark to be placed on future CPGs of organizations.
- Accreditor may review organizational procedures at any point during the accreditation period and, if non-compliance with accreditation requirements is detected, withdraw accreditation.
Slide 21
Recommendations to AHRQ
- The committee heard testimony that the NGC does not set sufficiently high standards to assure users that poor-quality guidelines are not admitted.
- NGC should eliminate CPGs for which trustworthiness cannot be determined, and identify the trustworthiness of those retained.
- Guidelines that have not included a thorough systematic review of the relevant scientific evidence base should be excluded from the NGC.
- Findings of no scientific evidence resulting from an SR should not preclude listing of the CPG in the NGC.
Slide 22
Additional Recommendations
- NGC should prominently identify guidelines originating from CPG developers certified by the designated mechanism as trustworthy.
- CPGs from an organization that requested and failed review by the certifying mechanism should also be identified in a special category, with standards met and shortcomings specified.
- The proposed standards will require additional NGC effort as current NGC abstraction does not require review of development process data adequate to meet the requirements of the proposed standards.
- NGC needs to be funded at a sufficient level for it to improve the quality, timeliness, and trustworthiness of its CPGs and other products.
Slide 23
Finally, AHRQ should...
- Conduct research on the causes of inconsistent CPGs and strategies to encourage their harmonization.
- Assess the strengths and weaknesses of IOM standards by pilot-test.
- Estimate the validity and reliability of the proposed standards.
- Evaluate the effectiveness of interventions to encourage standards implementation.
- Evaluate the effects of standards on CPG development, healthcare quality, and patient outcomes.
Slide 24
Image: Two faces are shown. The first is of features of a untrustworthy face, the second is of features that appear trustworthy.