New and Improved: Registries for Evaluating Patient Outcomes and HIT (Text Version)

On September 27, 2010, Elise Berliner, Rich Gliklich, and Nancy Dreyer made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (2 MB). Free PowerPoint® Viewer (Plugin Software Help).

Slide 1

New and Improved: Registries for Evaluating Patient Outcomes and HIT

Elise Berliner, Ph.D.
Director, Technology Assessment Program
Center for Outcomes and Evidence

Slide 2

Agenda

• Introduction to the Second Edition
  • Elise Berliner, Ph.D., Task Order Officer
• Review of major changes and new sections
  • Richard Gliklich, M.D., Senior Editor
  • Nancy Dreyer, Ph.D., M.P.H., Senior Editor
• Planning for the Third Edition
  • Audience discussion, led by Dr. Berliner

Slide 3

Background

• First edition, published in 2007, has been widely used as a reference for designing, operating, and evaluating patient registries.
  • Numerous citations (> 60) in literature and in significant government publications (e.g., Federal Register, FCC Report to the President, etc.)
• As registries continue to evolve, many new methodological and practical issues have arisen.

Slide 4

Purpose of the Second Edition

• Update and expand the first edition of the User's Guide with new information gathered from recent publications and reported experiences encountered by researchers and other professionals who utilize registries for research.
• Expound on selected topics in the original guide and add new topics that deserve further in-depth discussion.

Slide 5

Process for Creating the Guide

• Topics identified based on public comments received for the first edition.
• New sections:
  • Author and reviewer teams assembled with similar balance to first edition.
  • Posted for public comment and revision.
• Original chapters:
  • Original authors and reviewers were invited to participate. Additions made for new topic areas when necessary.
  • Full, revised handbook posted for public comment.
• Open call for case examples.

Slide 6

Second Edition Collaborators

• 55 contributors from industry, academia, health plans, physician societies, and government.
• 49 invited peer reviewers, plus public comment.
• 38 case studies illustrate challenges and solutions (including 20 new examples).
• Senior Editors: R. Gliklich; N. Dreyer. Managing Editor, M. Leavy. Outcome DEcIDE Center.

Slide 7

Second Edition: Table of Contents

1. Patient Registries (Overview)
2. Planning a Registry
3. Registry Design (includes Planning for the End of a Patient Registry)
4. Use of Registries in Product Safety Assessment
5. Data Elements for Registries
6. Data Sources for Registries
7. Linking Registry Data: Technical and Legal Considerations
8. Principles of Registry Ethics, Data Ownership, and Privacy
9. Recruiting and Retaining Participants in the Registry
10. Data Collection and Quality Assurance
11. Interfacing Registries With Electronic Health Records
12. Adverse Event Detection, Processing, and Reporting
13. Analysis and Interpretation of Registry Data To Evaluate Outcomes
14. Assessing Quality

Slide 8

Registries for Evaluating Patient Outcomes:
A User's Guide
Second Edition

Richard Gliklich, M.D., Senior Editor
Nancy Dreyer, Ph.D., M.P.H., Senior Editor

Slide 9

Review of Major Changes

• Changes to Sections on Creating and Operating Registries:
  • All chapters were updated. Some chapters had more significant additions.
  • Multiple new case studies.
• Review of "Linking Registry Data.".
• Review of "Interfacing Registries with EHRs.".
• Highlighted Case Examples.

Slide 10

Updates to 1st Edition Chapters

• "Planning a Registry" (Chapter 2) now discusses public-private partnerships.
• "Registry Design" (Chapter 3) and "Analysis and Interpretation" (Chapter 13) were reorganized to align more closely.
• "Data Elements" (Chapter 5) was expanded to discuss new data standards.

Slide 11

Updates to 1st Edition Chapters

• "Principles of Registry Ethics, Data Ownership, and Privacy" (Chapter 8) addresses new legislation:
  • Patient Safety and Quality Improvement Act of 2005 (PSQIA)
  • Genetic Information Nondiscrimination Act of 2008 (GINA)
  • Health Information Technology for Economic and Clinical Health Act (HITECH Act)
• "Adverse Event Detection, Processing, and Reporting" (Chapter 12) discusses risk evaluation and mitigation strategies (REMs).

Slide 12

Case Examples (new examples highlighted)

1. Using Registries To Understand Rare Diseases
2. Creating a Registry To Fulfill Multiple Purposes and Using a Publications Committee To Review Data Requests
3. Using a Registry To Track Emerging Infectious Diseases
4. Using a Collaborative Approach To Plan and Implement a Registry
5. Using a Scientific Advisory Board To Support Investigator Research Projects
6. Determining When To Stop an Open-Ended Registry
7. Designing a Registry for a Health Technology Assessment
8. Assessing the Safety of Products Used During Pregnancy
9. Designing a Registry To Study Outcomes
10. Analyzing Clinical Effectiveness and Comparative Effectiveness in an Observational Study
11. Using a Registry To Assess Long-Term Product Safety
12. Using a Registry To Monitor Long-Term Product Safety

Slide 13

Case Examples (new examples highlighted)

13. Identifying and Responding to Adverse Events Found in a Registry Database
14. Selecting Data Elements for a Registry
15. Using Performance Measures To Develop a Dataset
16. Developing and Validating a Patient-Administered Questionnaire
17. Understanding the Needs and Goals of Registry Participants
18. Using Validated Measures To Collect Patient-Reported Outcomes
19. Integrating Data From Multiple Sources With Patient ID Matching
20. Linking Registries at the International Level
21. Linking a Procedure-Based Registry With Claims Data To Study Long-Term Outcomes
22. Linking Registry Data To Examine Long-Term Survival
23. Considering the Institutional Review Board Process During Registry Design
24. Issues With Obtaining Informed Consent
25. Building Value as a Means To Recruit Hospitals

Slide 14

Case Examples (new examples highlighted)

26. Using Registry Tools To Recruit Sites
27. Using Proactive Awareness Activities To Recruit Patients for a Pregnancy Exposure Registry
28. Using Reimbursement as an Incentive for Participation
29. Data Collection Challenges in Rare Disease Registries
30. Managing Care and Quality Improvement for Chronic Diseases
31. Developing a Performance-Linked Access System
32. Using Audits To Monitor Data Quality
33. Challenges in Creating Electronic Interfaces Between Registries and Electronic Health Records
34. Creating a Registry Interface To Incorporate Data From Multiple Electronic Health Records
35. Technical and Security Issues in Creating a Health Information Exchange
36. Developing a New Model for Gathering and Reporting Adverse Events
37. Using Registry Data To Evaluate Outcomes by Practice
38. Using Registry Data To Study Patterns of Use and Outcomes

Slide 15

New Chapter: Linking Registry Data

Rationale: Increasingly, statistical methods are used to link data from multiple de-identified sources, including registries. For these projects:

• What is the risk of identifying patients by combining data from multiple registries?
• What are the legal and ethical requirements for researchers to ensure patient privacy and confidentiality?

Slide 16

Linking Registry Data: Overview

• Chapter is divided into 2 sections: Technical Aspects and Legal Considerations.
• Technical Aspects: What is a feasible technical approach to linking the data?
• Legal Considerations: Is the linkage legally feasible under the permissions, terms, and conditions that applied to the original compilations of each data set?

Slide 17

Table 10: Technical Planning Questions

Slide 18

Table 9: Legal Planning Questions

Slide 19

Table 9: Legal Planning Questions (continued)

Note: HIPAA is Health Insurance Portability and Accountability Act
IRB=institutional review board. NIH=National Institutes of Health.

Slide 20

Linking Registry Data: Case Example

• Goal: Study long-term patient outcomes for diagnostic cardiac catheterizations and percutaneous coronary interventions (PCI).
• Data Sources: CathPCI registry lacks long-term follow-up. Medicare data lacks detailed procedure information.
• Legal Approach: After review of HIPAA, Common Rule, project team determined that linkage required use of limited datasets only and IRB approval for linkage project.
• Technical Approach: CathPCI data were linked with Medicare data using probabilistic matching techniques.

Slide 21

New Chapter: Interfacing Registries with Electronic Health Records (EHRs)

Rationale: With national efforts to invest in EHRs, and to advance the evidence base in areas such as effectiveness, safety, and quality through registries and other studies, it is clear that interfacing registries with EHRs will become increasingly important to reduce data collection burden.

This chapter explores issues of interoperability and a pragmatic "building-block approach" towards a functional, open-standards based solution.

Slide 22

Interfacing Registries & EHRs: Overview

• Chapter covers 4 main topics:
  • Role of EHRs and patient registries in health care.
  • Vision of EHR-registry interoperability.
  • Interoperability challenges.
  • Partial and potential solutions.

Slide 23

Interfacing Registries & EHRs: Definitions

• EHR: an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians and staff, across more than one health care organization.
  • Individual focused
• Patient registry: an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical or policy purposes.
  • Population focused

Slide 24

Interoperability Challenges

• Syntactic interoperability (communication): the ability of heterogeneous health information systems to exchange data.
  • Wiring, application protocol, standard messaging protocol.
  • Most easily solved.
• Semantic interoperability (content): implies that the systems understand the data that has been exchanged at the level of defined domain concepts.
  • Depends on standard vocabulary and shared data elements.
  • More difficult to solve; efforts include CDASH, ASTM Continuity of Care Record (CCR), HL7 Continuity of Care Document.
• Other issues: managing patient identifiers and authenticating users across multiple applications.

Slide 25

Partial and Potential Solutions

• We lack a single, complete interoperability model, but, several open standard components provide a reasonably good start which can be improved with further development, testing, and adoption of additional open standard building blocks.
  • Signing, Privacy, Encryption
  • Data standards (e.g. HL7 and CDISC)
  • Content profile (e.g. CRD)
  • Integration profile (e.g. RFD)
  • Web services: http(s), Web browsers
  • Physical network connection

Slide 26

Building-Block Approach: Retrieve Form for Data Capture/HITSP-TP50

Image: "Figure 4: Retrieve Form for Data Capture (RFD) Diagram" is shown. A box captioned "Form Filler" is positioned above another box captioned "Form Manager." The two boxes are connected by two lines and two downward pointing arrows labeled "Retreive Forms (ITI-a)" and "Query Form Manager (ITI-b). To the right of "Form Filler" and also connected to it by lines are a data bin captioned "Ex: Document Vault" and a box captioned "Form Receiver." Above "Ex: Document Vault" is a downward pointing arrow labeled "Archive Form (ITI-d)." Between "Ex: Document Vault" and "Form Receiver" is a downward pointing arrow labeled "Submit Form (ITI-c)."

Slide 27

Summary

• EHR-registry interoperability will be increasingly important as adoption of EHRs and patient registries increases significantly.
• Interoperability requires accurate and consistent data exchange and use of the information that has been exchanged.
• While a complete interoperability solution does not yet exist, enough open standard building blocks do exist today to enable 'functional interoperability' which while imperfect, significantly improves workflow and reduces duplication of effort.

Slide 28

Interfacing Registries & EHRs: Case Examples

• A registry focused on effectiveness in pain management was made interoperable with a commercial EHR using RFD communication.
• ASTER project: interoperability was achieved for the purpose of reporting adverse event information to the FDA.
• A commercial EHR was made interoperable with a quality reporting initiative for the American College of Rheumatology and with a Physician Quality Reporting Initiative Registry for reporting data to the Centers for Medicare and Medicaid Services.

Slide 29

Review of Major Changes

• Review of "Planning for the End of a Registry.".
• Review of "Use of Registries for Product Safety Assessment.".
• Highlighted Case Examples.
• Changes to Section III, Evaluating Registries.

Slide 30

New Section: When to End a Registry

Rationale: Registries may be developed without specific endpoints, making it difficult to determine when to end data collection. A more clear explanation of recommendations for stopping a particular registry is useful from several perspectives, including costs.

This section addresses how to determine when a registry has met its objectives and should be closed.

Slide 31

Stopping Decisions and Registry Goals

• Registry protocol should include specific and measurable endpoints against which to judge whether the project should continue or stop.
• Measurable goals for endpoints will make it possible to determine whether the registry has achieved a core purpose and may lead to a reasonable stopping point.
• Conversely, a registry that fails to meet measurable goals and appears to be unable to meet them in a reasonable time is also a candidate to be stopped.
• Other reasons for stopping:
  • Incomplete or poor quality data.
  • Registry has outlived the question it was created to answer.
  • Funding and/or staffing is no longer available.

Slide 32

What Happens When a Registry Stops?

• Stopping a registry could mean:
  • Ceasing all data collection and issuing a final report.
  • Ceasing to accrue new patients while continuing to collect data on existing patients.
• Regardless, archiving rules should be checked and followed.
• No clear ethical obligation to participants to continue a registry that has outlived its scientific usefulness.

Slide 33

What Happens When a Registry Stops?

• Registries with one sponsor who has decided to stop the registry should be encouraged to engage other stakeholders in discussions of potential transitioning of the registry to other owners.
  • Issues of data ownership, property, confidentiality, and patient privacy would need to be satisfactorily addressed to make such transitions possible.
• Reasons to consider preserving registry data:
  • If data may be capable of producing a recognized public health benefit that will continue if the registry does.
  • If registry has historical importance, such as a registry that tracks the outbreak of a novel infectious disease that may provide insight into the transmission of the disease if not now, then sometime in the future.
  • If longitudinal data may be useful for hypothesis generation.

Slide 34

Ending a Registry: Case Example

• Bupropion Pregnancy Registry:
  • Open-ended, observational exposure-registration and followup study to monitor prenatal exposure to bupropion and detect any major teratogenic effect.
• Registry collected data on over 1,500 exposed pregnant women over 10 years.
• In 2007, advisory committee recommended discontinuation of the registry, based on its conclusion that sufficient information had accumulated to meet the scientific objective of the registry.

Slide 35

New Chapter: Use of Registries for Product Safety Assessments

Rationale: Registry populations often include patients that are significantly different from patients studied in clinical trials (age, comorbidities, etc.) and therefore are of particular interest in post-approval safety monitoring.

• How should data be monitored for adverse events?
• How is the "expected" rate calculated?
• How often should the data be analyzed?
• What is the legal and ethical responsibility of the registry for reporting, informing physicians and institutions of potential problems?

Slide 36

Use of Registries for Product Safety Assessments: Overview

• Chapter divided into 4 sections:
  • Registries specifically designed for safety assessment.
  • Registries designed for purposes other than safety.
  • Signal detection in registries.
  • Potential obligations for registry developers in reporting safety issues.

Slide 37

Registries Designed for Safety Assessments

• Registries are well suited to identify effects that can only be observed in a large and diverse population over an extended period of time (drug-drug interactions, genetic differences in drug metabolism, etc.).
• When designing a registry for the purposes of safety, the size of the registry, the enrolled population, and duration of follow-up are all critical to ensure validity of the inferences made based on the data collected.
• Some other challenges:
  • Recruiting a meaningful patient population.
  • Evaluating the utility of a registry when the entire population-at-risk has not been included (a common scenario.)
  • Understanding timing of treatments, switching of treatments, multiple therapies, dose effects, delayed effects, and patient compliance.

Slide 38

Registries Designed for Purposes Other than Safety

• Registries examining comparative effectiveness, the natural history of a disease, evidence in support of national coverage decisions, or quality improvement efforts may gather and report AE data, but may not be able to reliably detect all events.
  • Facilitate safety reporting rather than evaluate it.

Slide 39

Use of Registries for Product Safety Assessment: Case Example

• British Society for Rheumatology Biologics Register (BSRBR):
  • Prospective observational study to monitor the routine clinical use and long-term safety of biologics in patients with severe rheumatoid arthritis and other rheumatic conditions.
• Registry enrolls patients on anti-TNF agents and a control cohort of patients on DMARDs.
• Data from registry have been used to determine whether an increased risk of tuberculosis existed in patients treated with anti-TNF therapy. Analysis found a differential risk among the three anti-TNF agents (Dixon et al., Annal Rheum Dis 2010).

Slide 40

Evaluating Registries

• Tables in this chapter were re-organized to group the good practices by concept.
  • Research Quality: Basic Elements and Potential Enhancements of Good Practice for Establishing and Operating Registries.
  • Evidence Quality: Indicators of Good Evidence Quality and Indicators of Enhanced Good Evidence Quality for Registries.
• Chapter was updated to include good practices related to the new handbook sections.

Slide 41

Table 20: Research Quality—Basic Elements of Good Practices for Establishing and Operating Registries

Slide 42

Table 21; Research Quality—Potential Enhancements to Good Practices for Establishing and Operating Registries

Slide 43

Table 22: Evidence Quality—Indicators of Good Evidence Quality for Registries

Slide 44

Table 23: Evidence Quality—Indicators of Enhanced Good Evidence Quality for Registries

Slide 45

Plan for the Third Edition

• Development begins in Fall 2010.
• Plan is to include 11 new chapters, plus case examples.
• 7 topics identified through public comments on second edition.

Slide 46

Topics for the Third Edition

1. Patient identity management
2. Protection of data from litigation
3. Data protection concerns
4. Public-private partnerships
5. Statistical techniques for analyzing combined data
6. Pregnancy registries
7. Registry transitions

Slide 47

Additional Topics

For discussion: What other topics should be addressed in the third edition?

Slide 48

Contact

Elise Berliner
elise.berliner@ahrq.hhs.gov
301-427-1612