How Can One Examine a Trial for Heterogeneity of Treatment Effect?

Weiss Slide Presentation -Text Version- AHRQ 2010 Conference, How Can One Examine a Trial for Heterogeneity of Treatment Effect? The Example of the BARI trial for CABG vs PTCA.

On September 28, 2010, Carlos Weiss made this presentation at the 2010 Annual Conference (Microsoft PowerPoint file Microsoft PowerPoint version - 408.9 KB ). Free PowerPoint® Viewer (Plugin Software Help).


Slide 1

WeisSlide 1. How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)? The Example of the BARI trial for CABG vs PTCA

Accounting for Clinical Heterogeneity in Comparative Effectiveness Research

How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)? The Example of the BARI trial for CABG vs PTCA

September 28, 2010
Carlos Weiss, MD, MHS

Slide 2

Weiss2. AHRQ DEcIDE Project: Methods to Study the Heterogeneity of Treatment Effects in Comparative Effectiveness Research

AHRQ DEcIDE Project: Methods to Study the Heterogeneity of Treatment Effects in Comparative Effectiveness Research

PI: Ravi Varadhan, PhD
Co-I: Jodi Segal, MD, MPH; Cynthia Boyd, MD, MPH; Al Wu, MD, MPH

Consultant: David Kent, MD, MPH
Technical Experts: Curt Furberg, MD, PhD; Bruce Psaty, MD, PhD
Task Order Officer: Parivash Nourjah, PhD

Slide 3

Weiss 3. Image: The cover of the New England Journal of Medicine from July 25, 1996

Image: The cover of the New England Journal of Medicine from July 25, 1996 with the title "Comparison of Coronary Bypass Surgery with Angioplasty in Patients with Multivessel Disease."

Slide 4

Weiss 4. BARI Clinical Question

BARI Clinical Question

Population targeted: "Multivessel disease" with severe angina or ischemia.
Intervention: PTCA (a form of PCI).
Comparator: CABG.
Outcome: 5-yr Mortality.

Slide 5

Weiss 5. Questions to Audience-Set 1

Questions to Audience—Set 1

What are sources of HTE?

How would pre-specification of analyses affect interpretation of results?

Slide 6

Weiss 6. BARI Design for HTE

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:

  • Angina severity

Slide 7

Weiss 7. BARI Design for HTE

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:

  • Angina severity
  • Left ventricular function
  • Number of diseased vessels
  • Complex lesions

Slide 8

Weiss 8. BARI Clinical Question: Sources of HTE in CABG vs PTCA

BARI Clinical Question: Sources of HTE in CABG vs PTCA

Slide 9

Weiss 9. BARI Clinical Question: Sources of HTE in PTCA v CABG

BARI Clinical Question: Sources of HTE in PTCA v CABG

  • Patients:
    • Baseline risk
    • Competing risks
    • Risk of treatment harms
    • Treatment responsiveness

Ideas drawn from Kravitz, Duan & Braslow, 2004, Milbank Quarterly.

Slide 10

Weiss10. BARI Clinical Question: Sources of HTE in PTCA v CABG

BARI Clinical Question: Sources of HTE in PTCA v CABG

  • Patients:
    • Baseline risk
    • Competing risks
    • Risk of treatment harms
    • Treatment responsiveness
  • Treatment
  • Providers
  • Environments

Slide 11

Weiss11. Image: Graph depicts how the sources of HTE.

Image: Graph depicts how the sources of HTE—Patients, Treatment, Providers, and Environments—can interact with each other.

Slide 12

Weiss12. BARI Results

BARI Results

5-yr Mortality: Overall, no clinically significant nor statistically significant difference.

Slide 13

Weiss13. Image: A chart depicts the 4 subgroup analyses for the BARI trial at 5 Years

Image: A chart depicts the 4 subgroup analyses for the BARI trial at 5 years. There is also a 5th subgroup: treated diabetes that is absent or present.

Part way through the trial the safety and data monitoring board requested an analysis of diabetic patients on the basis of a subgroup analysis in a study of aggressive PTCA vs PTCA only if clearly needed. That analysis suggested an increased risk of death in patients with a history of diabetes AND no prior MI. Here, the line says history of diabetes but the variable is actually treated diabetes, defined as diabetes involving the use of insulin or oral hypoglycemic agents at entry into the study.

For the treated DM subgroup, the CABG arm did approximately 15% better than the PTCA arm with respect to 5-year survival.

Wider confidence intervals of 99 percent and 99.5 percent for the a priori and treated diabetes subgroups, respectively, were used to correct for multiple comparisons made here.

Slide 14

Weiss14. Image: A graph depicting survival over time is shown for four subgroups according to treatment arm and presence or absence of treated diabetes.

Image: A graph depicting survival over time is shown for four subgroups according to treatment arm and presence or absence of treated diabetes, with survival presented on the y-axis and time on the x-axis. The survival curve for patients with treated diabetes who received PTCA is lower than all the other curves, and approaches 60% survival at 6 years. The log-rank test for difference in survival between treatment arms, among patients with treated diabetes, has a p-value of 0.003, suggesting that CABG is superior to PTCA in this group.

Slide 15

Weiss15. Questions to Audience-Set 2

Questions to Audience—Set 2

When should one be worried that a subgroup result is an error (Type I or Type II) ?

What can be done to lower error probabilities?

Slide 16

Weiss16. Proposed General Approach to Examining a Trial for HTE

Proposed General Approach to Examining a Trial for HTE

  1. HTE hypotheses pre-specified?
  2. Design and measurement quality?
  3. Modeling pre-specified?

Slide 17

Weiss 17. Proposed General Approach to Examining a Trial for HTE

Proposed General Approach to Examining a Trial for HTE

  1. HTE hypotheses pre-specified?
  2. Design and measurement quality?
  3. Modeling pre-specified?
  4. If No to 1, 2 or 3: Validation study available?

Slide 18

Weiss18. Proposed General Approach to Examining a Trial for HTE

Proposed General Approach to Examining a Trial for HTE

  1. HTE hypotheses pre-specified?
  2. Design and measurement quality?
  3. Modeling pre-specified?
  4. If No to 1, 2 or 3: Validation study available?
  5. a. If frequentist, test of interaction performed?
  6. b. If Bayesian, pre-specified priors and variance acceptable?

Slide 19

Weiss 19. (Blank Slide)

This slide is blank.

Slide 20

Weiss20. Extra Slides

Extra Slides

Slide 21

Weiss 21. What is Heterogeneity of Treatment Effect?

What is Heterogeneity of Treatment Effect?

Non-random variability in the direction or magnitude of a treatment effect.

Slide 22

Weiss 22. Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

HTE is present on the absolute and/or relative risk scales of treatment effect. Treatment effect cannot be homogeneous in both scales (Figure 1), unless baseline risk is constant.

Figure 1. Heterogeneity of Treatment Effect Is Present in Absolute And/Or Relative Scales. Figure 1 Legend: Absolute baseline risk of the primary outcome is on the x-axis and is the source of HTE in this example. Risk if treated is on the y-axis and the dotted line indicates no treatment effect. Treatment effects according to quintiles of baseline risk are presented for 2 studies (closed, open circles). Treatment effect can be calculated according to absolute risk (solid line) or a relative risk (dashed line) or both. Corresponding effect models are that the absolute risk reduction equals 'a' and relative risk equals 'b'. Treatment effect cannot be homogeneous on both scales. Heterogeneity may be apparent in one effect model and not in the other effect model. The study represented by closed circles would not show HTE on a relative risk scale, but would show HTE related to baseline risk on an absolute risk scale; the study represented by open circles would not appear to have HTE on an absolute risk scale, but would on a relative risk scale.

Suppose that the treatment effect is constant on the absolute risk scale, i.e. Prob(Yi(2)=1) - Prob(Yi(1)=1) = a, for all individuals i, then the individual treatment effect on the relative risk scale is equal to 1 + (a/ Prob(Yi(1)=1)) and varies with individual's baseline risk. Conversely, suppose that the individual treatment effect is constant on the relative risk scale, i.e. Prob(Yi(2)=1)/ Prob(Yi(1)=1) = b, for all individuals i, then the individual treatment effect on the absolute risk scale is equal to (1-b) * Prob(Yi(1)=1), which also varies with individual's baseline risk.

Slide 23

Weiss 23. Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

Go to detailed overview of the graphs on Slide 22.

Slide 24

Weiss 24. Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

Image: A graph depicts Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale.

Go to detailed overview of the graphs on Slide 22.

Current as of December 2010
Internet Citation: How Can One Examine a Trial for Heterogeneity of Treatment Effect?. December 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/news/events/conference/2010/weiss/index.html