NOPR 2006: Results and Lessons Learned (Text Version)

Slide presentation from the AHRQ 2010 conference.

On September 29, 2010, Bruce Hillner made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (415 KB). Free PowerPoint® Viewer (Plugin Software Help).


Slide 1

Slide 1. NOPR 2006: Results and Lessons Learned

NOPR [National Oncologic PET Registry] 2006: Results and Lessons Learned

2010 Annual AHRQ Conference

Bruce E. Hillner, M.D.
Eminent University Scholar and Professor
Virginia Commonwealth University
Richmond, VA

Slide 2

Slide 2. NOPR Results

NOPR Results

  • Overall Impact on Patient Management:
    • Diagnosis, Staging, Restaging, Recurrence
    • Data on 22,975 scans from May 8, 2006—May 7, 2007
    • J Clin Oncol 2008; 26:2155-61
  • Impact on Patient Management by Cancer Type:
    • Confirmed Cancers
    • Staging, Restaging, Recurrence
    • Data on 40,863 scans from May 8, 2006—May 7, 2008
    • J Nucl Med 2008; 49:1928-35
  • Treatment Monitoring:
    • Data on 10,447 scans from May 8, 2006—Dec 31, 2007
    • Cancer 2009:115:410-18

Slide 3

Slide 3. PET Changed Intended Management in 36.5% of Cases

PET Changed Intended Management in 36.5% of Cases

Pre-Pet PlanPost-PET PlanClinical Indication for PET Study (Percent)All
n=22,975
Dx
n=5,616
Staging
n=6,464
Restaging
n=5,607
Recurrence
n=5,388
TreatSame16.046.515.820.425.5
Non-TreatSame52.914.048.040.737.9
Non-TreatTreat23.231.628.629.228.3
TreatNon-Treat7.97.97.59.78.2
Patients with change post-PET (%)31.139.536.139.036.5

Hillner et al., J Clin Oncol 2008.

Slide 4

Slide 4. Changes in Intended Management (%) Stratified by Pre-PET Plan

Changes in Intended Management (%) Stratified by Pre-PET Plan

Pre-PET Plan

Post-PET PlanImage
n=9,518
Biopsy
n=3,552
Watch
n=2,199
Treatment
n=7,706
Image5.86.04.63.0
Biopsy9.524.09.06.8
Watch37.233.662.315.6
Same RxNANANA42.4
New or Major Change in Rx47.636.324.18.7
Minor change RxNANANA23.5

Hillner et al., J Clin Oncol 2008.

Slide 5

Slide 5. Change in Management by Cancer Type

Change in Management by Cancer Type

Cancer TypeStagingRestagingSuspected Recurrence
Prostate32.0
(1,491)
34.0
(1,477)
39.4
(1,790)
Ovary43.2
(378)
37.7
(1,971)
44.5
(2,160)
Bladder39.9
(1,461)
36.4
(1,239)
36.7
(878)
Pancreas39.2
(1,491)
38.3
(1,021)
39.3
(802)
Small Cell Lung43.3
(1,082)
40.8
(1,357)
38.1
(544)

Hillner et al., J Nucl Med 2008.

Slide 6

Slide 6. Global Summary

Global Summary

  • Change in intended management associated with PET in previously non-covered cancers was similar to that reported in single-institution studies of covered cancers.
  • ~1/3 of older patients undergoing PET for cancer types covered under Medicare's CED policy had a major change in intended management, including type of treatment.
  • The relative impact of PET on intended management was observed across the full spectrum of indications for PET.

Hillner et al., J Clin Oncol 2008.

Slide 7

Slide 7. Strengths of the NOPR Data

Strengths of the NOPR Data

  • "Real world" data.
  • Timely data.
  • Very large patient cohorts.
  • Current technology (≥ 85% PET/CT).
  • Good observational studies usually match controlled studies in magnitude and direction of effect.
  • Results similar to more tightly managed single-institution studies (e.g., Hillner, J Clin Oncol 22:4147, 2004) and to Australian studies with outcome validation (Scott, J Nucl Med 49:1451, 2008),

Slide 8

Slide 8. NOPR Limitations (1)

NOPR Limitations (1)

  • Data "quality."
  • Potential that physicians may have been influenced by the knowledge that future Medicare reimbursement might be influenced by their responses.
  • Collected change in "intended" management, not actual management.
  • Unknown if management changes were in the correct direction or improve long-term outcomes.
  • Defining the relevant long-term outcomes for a diagnostic (instead of therapeutic) procedure is controversial.

Slide 9

Slide 9. NOPR Limitations (2)

NOPR Limitations (2)

  • NOPR does not address:
    • Whether PET should be used in lieu of or as a complement to other imaging techniques.
    • The optimal sequencing of CT, MRI and PET.
    • How much 'better' PET is than next best method.

Slide 10

Slide 2. Lessons Learned 1: Must Do Even if Painful

Lessons Learned 1: Must Do Even if Painful

  • Preparing a formal "operations manual" similar in structure to a clinical trial protocol document.
  • Project annual registry enrollment for relevant time frame.
  • Prepare a statistical plan even if multiple definitions of 'meaningful' change in endpoints are considered.
  • If registry includes multiple diseases (e.g. different types of cancer) or sub-groups (e.g. past myocardial infarctions vs. angina), define the priority areas for first analysis.

Slide 11

Slide 11. Lessons Learned 2

Lessons Learned 2

  • Design your case report forms such that all data fields must be complete before accepting record.
  • Web-based entry of data to 'center' minimizes costs and assists data integrity.
  • No evidence that being the treating physician was associated with higher rates of change in the post-PET (treatment) strategies.

Slide 12

Slide 12. Lessons Learned 3

Lessons Learned 3

  • The primary endpoint of 'intended' vs. "actual" management was a compromise.
  • Define and concurrently implement 'validation' strategy from the onset of the registry:
    • Prospective claims collection.
    • Prospective selected chart audits ($).
  • Consider how proximate imaging is to preferred 'hard endpoint'.

Slide 13

Slide 13. CMS Decision Memo, CAG-00181R, April 3, 2009

CMS Decision Memo
CAG-00181R
April 3, 2009

New Framework differentiates PET imaging into uses informing initial treatment strategies from uses guiding subsequent treatment after completion of initial treatment.

Slide 14

Slide 14. IOM Priority 17/100

IOM Priority 17/100

"Compare the effectiveness of imaging technologies in diagnosing, staging, and monitoring positron emission tomography (PET), magnetic resonance imaging (MRI) and computed tomography (CT)."

Image: Cover of Institute of Medicine (IOM) report titled Initial National Priorities for Comparative Effectiveness Research.

Slide 15

Slide 15. Prospective Concurrent Claims Database of Registry Participants is Necessary but Not Necessarily Sufficient for Proving Better Outcomes

Prospective Concurrent Claims Database of Registry Participants is Necessary but Not Necessarily Sufficient for Proving Better Outcomes

  • Relevant outcome rarely proximate to diagnostic test.
  • Discordance between intended vs. actual management is more likely to vary depending on the type of intended management than on the value of the test information.
  • Defining control group is problematic.

Slide 16

Slide 16. The Challenge to Registry-based Studies: Defining appropriate comparison control groups

The Challenge to Registry-based Studies: Defining appropriate comparison control groups

Options:

  1. Historical controls to Non-PET care when PET not available.
  2. Contemporary controls to Non-PET when PET was available.

Both face:

  • Indication Bias:
    • Differ in presentation.
    • Differ in probability of metastasis.
    • Differ in potential extent of metastasis.
  • Provider Bias (MDs and hospital):
    • Patterns of care by referring MDs and hospitals using PET likely to differ from non-PET users.
  • Spectrum Bias: For non-PET imaging, clinical indication not available.

Slide 17

Slide 17. 2010 CER Challenge

2010 CER Challenge

  • Such 'comparative effectiveness' evaluations must move beyond the "if" to the "how" by addressing the relative value of:
    • Sequencing
    • Frequency
    • Timing (during treatment monitoring)
    • Combinations of PET, MRI and CT
  • Comparisons between imaging types less likely to benefit from registry design.
  • Complementary prospective and retrospective studies.

Slide 18

Slide 18. Final Comments

Final Comments

  • It has taken 20-30 years for one "knowledge turn" to show that PET has unique value in cancer management.
  • NOPR has shown the feasibility of performing large-scale, policy-relevant imaging research that is minimally intrusive to patients and imagers.
  • Going forward the policy and economic questions for advanced imaging are when, how often, and in what sequence should advanced imaging be used in patients with suspected and confirmed cancer.
  • Prospective multi-center investigator-initiated evaluations are needed to confirm 'relative' comparative value.
Current as of December 2010
Internet Citation: NOPR 2006: Results and Lessons Learned (Text Version). December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2010/hillner/index.html