Using Value of Information to Prioritize Future Research: A Case Study

Slide presentation from the AHRQ 2010 conference.

On September 29, 2010, Gillian Sanders made this presentation at the 2010 Annual Conference. Select to access the PowerPoint® presentation (215 KB). Free PowerPoint® Viewer (Plugin Software Help).


Slide 1

Slide 1. Using Value of Information to Prioritize Future Research: A Case Study

Using Value of Information to Prioritize Future Research: A Case Study

Gillian D. Sanders Ph.D.
Duke Evidence-Based Practice Center
Duke University

On bottom left of every page is the organization Duke Clinical Research Institute.

Slide 2

Slide 2. Duke EPC Collaborators

Duke EPC Collaborators

  • Evan Myers M.D., M.P.H
  • Laura Havrilesky M.D.
  • David Matchar M.D.
  • Benjamin Powers M.D.
  • Matthew Crowley M.D.
  • Ravi Dhurjati Ph.D.
  • Gregory Samsa Ph.D.
  • Amanda McBroom Ph.D.
  • Michael Musty B.A.
  • Rebecca Gray Ph.D.

Slide 3

Slide 3. Overview of Project

Overview of Project

  • Investigating role of value of information analysis (VOI) in helping prioritize research gaps identified as part of EPC reviews
    • Systematic review of priority setting methods
    • Survey of existing priority setting methods
    • Two case studies of evidence reports with models
      • Comparative effectiveness of ACEI/ARBs in patients with ischemic heart disease
      • Comparative effectiveness of management strategies for uterine fibroids

Slide 4

Slide 4. Background

Background

  • Uterine fibroids are the most common, non-cancerous tumors in women of childbearing age and the second most common reason women of childbearing age undergo surgery.
  • Despite the prevalence and possible complications of uterine fibroids, the currently available evidence-based literature on their natural history and optimal management is insufficient.
  • DECIDE project to develop research priorities for comparative effectiveness research for fibroids:
    • Grew out of evidence gaps identified in systematic reviews conducted by EPCs
    • Used modified delphi/nominal group methodology
  • Current project investigating use of modeling in the uterine fibroids evidence gap prioritization process.

Slide 5

Slide 5. Methods

Methods

  • Updating model developed for original Duke Evidence Report on Fibroids
  • VOI for subset of priority questions identified through DeCIDE project
  • Comparison of VOI results to results of DeCIDE project
  • Presentation and discussion of VOI results with subset of stakeholders who participated in DeCIDE project
    • Value of value of information? (VOVOI!)
    • Optimal timing relative to qualitative process

Slide 6

Slide 6. Questions

Questions

  • Is VOI feasible for a condition like fibroids where the evidence gaps are so large?
  • Can VOI complement other priority-setting methods?
  • If so, what is best timing of VOI?
  • Model focuses on subset of research gaps identified in DeCIDE project
    • Further model development needed
    • Results interesting, but preliminary
    • Focus is on whether this process could be helpful, not these specific results

Slide 7

Slide 7. Model Schematic

Model Schematic

Image: A model schematic for the treatment of fibroids shows the following process: Symptomatic fibroids → Treatment → Short-term complications → Improved symptoms → Long-term complications, Improved symptoms, or Pregnancy → No complications or Pregnancy complications.

Slide 8

Slide 8. Model

Model

  • Interventions:
    • Myomectomy (surgical removal of fibroid)
    • Uterine artery embolization (UAE-angiographic occlusion of uterine blood supply)
    • MRI-guided focused ultrasound (FUS-ultrasonic energy applied directly to fibroid)
  • All 3 preserve uterus:
    • Varying amount, quality of data on recurrence
      • FUS—shortest recovery time, highest recurrence:
    • Limited data on pregnancy outcomes with UAE, FUS (both discouraged in women seeking pregnancy)

Slide 9

Slide 9. Model

Model

  • Patient population:
    • Fibroids more common in African-American women
      • At younger ages, more severe disease
    • Age and racial distribution taken from 3000+ patient registry of women undergoing UAE:
      • Mean age 40, 48% African-American
    • Assumed 25% planned pregnancy
      • Based on pre-procedure data from registry
      • Approximately 5% reported pregnancy in follow-up
      • Age distribution modeled separately based on desire for pregnancy

Slide 10

Slide 10. Model

Model

  • Age- and race-specific probabilities of fertility, miscarriage, preterm delivery:
    • Older women less likely to get pregnant, higher risk of miscarriage or preterm birth.
    • African-American women higher risk for preterm birth
  • This version of model assumes no difference in pregnancy outcomes between treatments:
    • Key area of uncertainty
    • Left out for purposes of simplifying analysis/presentation for stakeholders

Slide 11

Slide 11. Model

Model

  • Specific treatments/outcomes after recurrent symptoms not modeled:
    • Simplification
    • Subjects stayed in "retreatment" state until end of simulation
    • Utility of "retreatment" varied widely
  • Model run for 3 years or until patient reached age 45
    • Spontaneous pregnancy highly unlikely after 45
    • Evidence for interaction between declining ovarian function and treatment efficacy:
      • Necessary next step, but out of scope

Slide 12

Slide 12. Model Output

Model Output

  • Cumulative probability of events:
    • Assumes 25% of patients attempting pregnancy
OutcomeUAEMyomectomyFUS
MeanSDMeanSDMeanSD
Recurrence

0.095

0.924

0.082

0.275

0.225

0.412

Pregnant

0.087

0.282

0.097

0.296

0.085

0.228

Live birth

0.055

0.228

0.063

0.243

0.055

0.222

Preterm birth

0.012

0.110

0.014

0.111

0.012

0.104

(% livebirth)

22.4%

 

21.8%

 

22.0%

 

  • 2500 women actively trying to get pregnant within 6 months of procedure required to 800-900 pregnancies, 500-600 live births within 3 years.
  • Sample size requirements to determine differences in reproductive outcomes between procedures daunting.

Slide 13

Slide 13. Acceptability Curve

Acceptability Curve

Image: The Acceptability Curve is shown as a line graph, comparing probability of cost-effectiveness against willingness to pay for myomectomy, UAE, and focused ultrasound. The myomectomy line remains at 0.1 probability from $0k to over $90K. The UAE line begins at just above 0.2 probability and rises slightly to 0.3 by over $90K. The focused ultrasound line begins at 0.65 probability and decreases to 0.5 by over $90K.

  • Graph shows the proportion of simulations (n=10,000) that a given strategy was preferred at different thresholds of cost-effectiveness.
  • FUS has shorter length of stay, faster return to work, lower short-term complications than other procedures → lower costs.

Slide 14

Slide 14. Individual Parameters

Individual Parameters

ParameterEVPPI
Utility after retreatment$1053
Time to return to work$1047
Time before trying to get pregnant$1046
Relative recurrence rates$1043
Time between recurrence and retreatment$1038
Cost of complicated cases$1037
Length of stay (all)$1030
Cost of uncomplicated cases$9.50
  • Highest VOI related to recurrence-both probability and impact on QOL.
  • Peri-procedural factors with high uncertainty or high potential contribution to differences (cost of complications, time to return to work) also high VOI.

Slide 15

Slide 15. Different Patient Populations

Different Patient Populations

  • Relatively small differences when run separately for white and African-American patients:
    • Main known differences between groups are younger age, higher risk of adverse pregnancy outcomes in black women.
    • Differences likely greater if age-specific treatment effects or treatment effects on reproductive outcomes included.

Images: Two line graphs depicting the partial expected value of perfect information (EVPI) vs WTP Curve for Whites and African-Americans are shown. For Whites, EVPI begins at $1010 at $50K threshold for cost-effectiveness and rises evenly to $1130 at $65K, $1260 at $80K, and $1360 at $95K. For African-Americans, EVPI begins at $1050 at $50K threshold for cost-effectiveness and rises evenly to $1175 at $65K, $1300 at $80K, and $1450 at $95K.

Slide 16

Slide 16. Different Patient Populations

Different Patient Populations

  • In this version of model, pregnancy parameters are fixed.
  • Pregnancy is a competing risk with recurrence, and assume QOL if pregnant same as improved symptoms:
    • Fewer recurrences, higher overall QOL
  • Again, potential importance of age-dependent recurrence risk, differential effects of treatment on pregnancy.

Images: Two line graphs depicting the partial EVPI vs WTP Curve for women desiring pregnancy and not desiring pregnancy are shown. For women desiring pregnancy, EVPI begins at $565 at $50K threshold for cost-effectiveness and rises evenly to $590 at $65K, $610 at $80K, and $630 at $95K. For women not desiring pregnancy, EVPI begins at $1160 at $50K threshold for cost-effectiveness and rises evenly to $1310 at $65K, $1460 at $80K, and $1610 at $95K.

Slide 17

Slide 17. (VERY) Cautious Interpretations of VOI Results

(Very) Cautious Interpretations of VOI Results

  • Highest priority topics by VOI related to durability of treatment and effect on QOL:
    • Aligns with #1 priority from DeCIDE qualitative process
    • Unlikely to change with additional model refinement
  • Importance of impact of treatments on pregnancy outcomes population-dependent
    • Pregnancy outcomes important, but small proportion of overall patient population
    • Sample size issues in addressing this research gap

Slide 18

Slide 18. Feedback from Stakeholders

Feedback from Stakeholders

  • All found results provocative, interesting, helpful in thinking about research priorities.
  • Some found VOI results most helpful after qualitative exercise:
    • Qualitative process narrowed options
      Easier to understand VOI when only a few parameters varied
  • Others preferred before or parallel to qualitative process
    • Would have helped focus discussion
      Potential way to resolve persistent differences between stakeholders

Slide 19

Slide 19. General Lessons/Next Steps

General Lessons/Next Steps

  • Even with pre-existing model as basis, more work than anticipated:
    • Would add significantly to resources needed if done as part of CER.
      Some of this may be attributable to relatively poor state of evidence and relative complexity of modeling outcomes (symptoms And pregnancy) for this clinical area.
  • Stakeholders generally very positive about use of quantitative analysis
    • One major challenge is need to educate on both VOI approach in general And content-specific application.
      Could consider shared resource on methods.

Slide 20

Slide 20. General Lessons/Next Steps

General Lessons/Next Steps

  • Optimal timing still not clear:
    • Need to develop better method for testing
  • Fibroids model/VOI promising:
    • Adding uncertainty about relative treatment effects on reproductive outcomes
    • Adding age-specific effects on ovarian function and interaction with treatment effects on symptoms
    • More specific modeling of retreatment options
    • Move decision point earlier in process to evaluate nonsurgical treatment (medication, watchful waiting)
Current as of December 2010
Internet Citation: Using Value of Information to Prioritize Future Research: A Case Study. December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2010/sanders/index.html