| Harry Burke
|| George Washington University
|| [The Peer Reviewer sent a draft of a manuscript submitted for publication for consideration]
|| Thank you for your contribution. We agree that a distinction between prognosis and prediction is important. The methods section of the TA describes the epidemiological, methodological and clinical characteristics of eligible studies. The submitted manuscript is not eligible for inclusion in the TA.
| John Hermanek
|| [The comment referred to more recent information on panitumumab and KRAS, published after the last update of the draft TA.]
|| The TA has been updated to include publications on KRAS through March 2010. Also, according to the eligibility criteria only full publications are eligible.
| Gregory D. Pawelski
|| [The Peer Reviewer presented an overall interpretation of translational research in personalized medicine and cancer]
|| Thank you for you contribution. The TA used a systematic review approach to address the posed key questions. The Methods section describes the approach, and the epidemiological, methodological and clinical characteristics of eligible studies, as well as the statistical analyses performed.
|August J. Salvado
[The Peer Reviewer makes clarifying comments also with respect to the MEDCAC. Two points that could be relevant to the report are brought up:]
1. […] It warrants mention that CML is the only hematological malignancy
covered by the technology assessment, and as such it should be recognized
that physicians utilize different techniques to measure treatment response
and clinical efficacy than are typically used to monitor solid tumors. PCR
monitoring of BCR-ABL is the least invasive and most sensitive test currently
available to hematologists to monitor treatment response and clinical outcomes
2. The second point of clarification focuses on the scope of the technology
assessment in reviewing the utility of BCR-ABL testing to predict treatment
response based upon polymorphisms of point mutations. We concur with the
technology assessment’s conclusion that there is little evidence at the current
time supporting the predictive value of point mutations in the BCR-ABL1 gene
to determine differences in clinical benefit of one TKI agent over another.
However, one exception to this point that should be explicitly indicated
in the technology assessment is that the T315I mutation predicts a decreased
clinical response to all currently commercially available TKI therapies.
In addition, when this mutation is present, physicians should prepare for
an aggressive treatment regimen, potentially including stem cell therapy
or experimental treatments, in the event that patients with T315I mutations
fail currently available TKI therapies.
Thank you for your comments. We note that the MEDCAC had a broader set of questions that the TA. The key questions of the TA are listed in the methods section. With respect to the 2 clarification points:
First, the TA states clearly that tumor load monitoring is outside its scope.
Only mutation testing is in the scope of the TA and all our analyses pertain
to mutations only.
Second, the TA clearly singles out T315I and performs separate analyses
for this mutation. However, the TA does not make recommendations for treatment
or the management of patients with specific mutations.
||American Society of Hematology (ASH)
||General comments on BCR-ABL1 mutation testing
(This is a distillation, see pdf) The comments are the ASH answers to the key questions of the TA.
1. Does BCR-ABL1 mutation testing predict response to TKI therapy?
Yes. There is enough evidence, both in vitro and in vivo that different
mutations have different sensitivity to different inhibitors and that the
in vitro sensitivity correlates well with the clinical response of patients
[…].In view of the recognized value of mutations analysis in predicting for
response, this assay is recommended by a panel of international experts sponsored
by the European LeukemiaNet in all patients after failure or suboptimal response
to imatinib, and before changing therapy.
2. What patient- and disease-relevant factors affect the test results,
their interpretation or their predictive response to therapy?
The most important factor is the response to prior therapy […]
3. How does the gene testing impact the therapeutic choice?
As mentioned earlier, patients with mutations of intermediate or low sensitivity
to a certain tyrosine kinase inhibitor should be offered the alternative
4. What are the benefits and harms or adverse effects for patients
when managed with gene testing?
5. The obvious benefit is the possible selection of an agent that may
offer the best probability of response. There are no known adverse events
other than those implicated with the venipuncture. However, the test is usually
obtained at the time other routine monitoring tests are obtained.
Thank you for your comments, which are a narrative (non-systematic) review of the positions of the ASH.
The TA used a systematic review approach to address the posed key questions.
The Methods section describes the approach, and the epidemiological, methodological
and clinical characteristics of eligible studies, as well as the statistical
Studies that evaluated in-vitro sensitivity of BCR-Abl1 mutations to TKIs
do not meet our inclusion criteria; thus, we did not assess these studies.
Also, we did not assess the three studies reporting the association between
in-vitro sensitivity of the mutations to TKIs and clinical outcomes (Hughes
2009, Muller 2009, and Jabbour 2009), which could be viewed as a way to lump
some specific mutations together to perform subgroup analysis, because they
were published after our literature search.
||American Society of Clinical Oncology
First, on p. S-1, the authors list Key Question (KQ) 4 as “What are the benefits and harms or adverse effects for patients when managed with gene testing?” They
note that no studies were identified that could be used to answer this question.
Elsewhere (p. 3), however, the authors provide a broader explication of
KQ4 that creates some confusion: “What are the benefits and harms or adverse
effects for patients when managed with gene testing? Any cognitive, behavioral
or other health effects of testing with the three tests of interest. These
may be direct effects of the process of testing (e.g., increased anxiety)
or downstream effects stemming from treatment decisions informed by testing.”
Given this broader framework for KQ4, the statement that no studies were
identified that could be used to answer this question is problematic. Thus,
in the KRAS section the authors concluded that, “When treated with anti-EGFR
antibodies, patients with KRAS mutations were less likely to experience treatment
benefit, compared to patients whose tumors were wild-type for KRAS mutations,
for all outcomes assessed. These results were confirmed in several RCT-based
analyses of progression-free survival that demonstrated a significant treatment-by-KRAS
mutation interaction in three out of the four cases where such analyses were
These results, which arguably reflect downstream effects stemming from treatment
decisions informed by KRAS testing, would seem to address KQ4, although the
authors do note that “Most of the latter [the downstream effects] would be
captured by Key Question 1.”
Can the authors clarify this? Would it be simpler to restrict KQ4 to the
benefits and harms of the testing itself, defined as “any cognitive, behavioral
or other health effects of testing,? vs. clouding things by referring to
the downstream effects element”
|We appreciate the comment. We agree that there is the potential for confusion, and we have rephrased the methods section to enhance clarity. Indeed KQ1 captures the downstream effects of testing, and KQ4 captures other (additional) benefits and harms that are related to testing and are beyond those covered in KQ1.
| Mark Somerfield
||American Society of Clinical Oncology
p. S-2, the authors note that “It is questionable whether pharmacogenetic
testing of germline (heritable) variations in CYP2D6 can predict differential
response to adjuvant tamoxifen in women with non-metastatic breast cancer.
Further, evidence is severely limited for tamoxifen-treated women with metastatic
disease. Our conclusions are in accordance with the 2009 American Society
of Clinical Oncology (ASCO) practice guideline update.”
The ASCO guideline that is referenced here is an update of a guideline on
breast cancer risk reduction. The ASCO guideline update did consider CYP2D6
and concluded that, “Given the limited evidence, CYP2D6 testing is currently
not recommended in the preventive setting.” However, it is not accurate
in our view to state that the conclusion of the AHRQ systematic review are “in
accordance with” the ASCO guideline; the ASCO guideline concerns the very
different setting of risk reduction or chemoprevention, and the AHRQ report
focuses on the adjuvant and metastatic settings.
Later, in the body of the report (p. 27), the authors do note indirectly
that the ASCO guideline concerned the risk reduction or prevention setting:
?Our conclusions are in accordance with the relevant 2009 ASCO practice guideline
update, which states ?Given the limited evidence, CYP2D6 testing is currently
not recommended in the preventive setting.?? However, as many may read only
the executive summary of the technology assessment, this point should be
made in that summary as well. Ideally, the authors would change the phrasing
from ?in accordance with? to ?[our conclusions] are analogous to the conclusion
of the ASCO guideline that addressed the role of CYP2D6 in the breast cancer
risk reduction setting.?
|Thank you for providing this clarification. We rephrased the pertinent parts for clarity, both in the Executive summary and in the body of the TA adopting some of the Peer Reviewer’s wording.
|Anonymous Reviewer 1
||Utilization of pharmacogenetic information to prescribe medications on the basis of the needs of the individual patients can save money and lives. Knowing that Tamoxifen will not work for you, if you were a cancer patient, is something you would like to know before taking the medication for years just to learn that your cancer is back. Money and time are wasted when a patient takes a drug that has no effect because of variants in a patient’s genetic make-up! Pharmacogenetics is a tool that physicians can use in many areas of medicine to better and more safely help patients. More knowledge is always better for any disease state.
|| Thank you for your comments. No reply needed.
|Anonymous Reviewer 2
||Friends of Cancer Research
[The letter contains several general thoughts and three more specific mentions, listed below:]
1. We believe that it is premature to accurately determine impact of CYP2D6
polymorphisms on response to tamoxifen based solely this literature review
Currently, multiple trials are underway or have been completed that will
prospectively examine the role of CYP2D6 genotype on the clinical effect
of tamoxifen treatment in both the metastatic and adjuvant setting.1 In addition,
a study conducted by Schroth et. al., published after the timeframe of the
technology assessment, retrospectively examined 1,325 patients treated with
adjuvant tamoxifen for early stage breast cancer. The study concluded that, “Among
women with breast cancer treated with tamoxifen, there was an association
between CYP2D6 variation and clinical outcomes, such that the presence of
two functional CYP2D6 alleles was associated with better clinical outcomes
and the presence of nonfunctional or reduced-function alleles with worse
outcomes.” While this study is not a randomized controlled trial, of which
it may be several years until clinical results will demonstrate prospectively
what the actual impact of altered drug metabolism is, the conclusions of
this study are indicative of the complexity of the topic and weakness of
the currently available evidence. Therefore, it would be difficult to conclude
that pharmacogenetic testing for variations in CYP2D6 should not be used
by physicians and patients with breast cancer to determine course of treatment.
2. Perhaps one of the most notable and recent changes in the practice oncology
is the utilization of KRAS testing to aid therapeutic decision making for
colorectal cancer patients. The conclusions described in the technology assessment
are in accordance with the body of clinical evidence that has prospectively
demonstrated, in several cases, the predictive value of testing for KRAS
mutational status prior to delivering the anti-EGFR therapies, cetuximab
and panitumumab, to colorectal cancer patients. However, the technology assessment
suggests that the data are not being used in therapeutic decision making
(S-5: “In all three examples, we found no evidence on whether testing impacts
therapeutic decisions…”). […]
3. In the case of evaluating how variations in BCR-ABL1 impact response
to three drugs used for treatment of chronic myeloid leukemia (CML), the
technology assessment concludes that “The presence of any BCR-ABL1 mutation
does not appear to predict differential response to treatment…with any of
the three tyrosine kinase inhibitor (TKI)-based regimens.” While this is
an accurate summary of existing published evidence, there is a rapidly developing
body of new clinical evidence that will provide additional information on
this complex topic, including over a hundred studies examining the use of
imatinib, dasatinib or nilotinib in chronic myeloid leukemia.4 However, to
date BCR-ABL pharmacogenetic testing has not been shown to be a predictive
factor to guide first-line treatment decisions. But the value of molecular
evaluation in this case should not be dismissed due to the existence of molecular
methods, such as RQ-PCR transcript testing, that can be used to monitor treatment
Thank you for your comments. The TA used a systematic review approach to address the posed key questions. The Methods section describes the approach, and the epidemiological, methodological and clinical characteristics of eligible studies, as well as the statistical analyses performed.
1. CYP2D6: The TA summarized the published evidence and does not make recommendations
for practice. The final version of the TA, based on an updated search and
data extraction, includes the Schroth paper (among others), and reaches the
2. The Methods section describes exactly what is meant by each key question,
which is quite specific.
As described in the TA there were no eligible studies that quantified the
impact of testing on therapeutic decisions in the form of how many times
the physicians changed treatment, etc. This is qualitatively different from
the comments made by the Peer Reviewer.
3. The TA clearly states that only BCR-ABL1 mutations are in the scope of
the pertinent systematic review. As per our reply to other Peer Reviewers,
transcript levels are not examined in the TA. This was predetermined in out
review protocol and clearly stated in the methods section of the TA.
|Anonymous Reviewer 3
||Association for Molecular Pathology
||ABCR-ABL1 – General
[comment put into numbered paragraphs:]
1. We very much appreciate the authors’ statistical expertise of this technology
assessment report; however, we note a number of shortcomings that compromise
the relevance of the report’s conclusions. We believe that many of these
shortcomings could have been avoided had there been prior input from clinicians
and molecular pathologists intimately familiar with the performance and clinical
utilization of these tests
This lack of clinical input is immediately evident in the definition of
genetic test adopted by the study:
“The analysis of human DNA, RNA, chromosomes,
proteins, and certain metabolites in order to detect heritable disease-related
genotypes, mutations, phenotypes, or karyotypes for clinical purposes. Such
purposes include predicting risk of disease, identifying carriers, establishing
prenatal and clinical diagnosis or prognosis. Prenatal, newborn, and carrier
screening, as well as testing in high-risk families, are included. Tests
for metabolites are covered only when they are undertaken with high probability
that an excess or deficiency of the metabolite indicates the presence of
heritable mutations in single genes.”
Two of the three tests evaluated in the study do not fulfill this definition,
highlighting a superficial understanding of the biology underlying these
tests and how they are used clinically. Acknowledging that pharmacogenomic
tests can be a special type of genetic tests, it is noteworthy that the authors
fail to appreciate that of the three tests evaluated in the report, only
the CYP2D6 qualifies as being heritable. Alterations in the KRAS gene and
in the BCR-ABL translocation are not heritable, but are tumor specific, intrinsic
to the neoplastic process. This distinction is not moot. Polymorphisms that
influence drug metabolism can be identified in healthy individuals and can
have bearing on dosing or drug selection of numerous therapeutic agents.
Tumor specific genetic changes, in contrast, have significance beyond simple
choice of drug, influencing disease recognition, disease prognosis, tumor
aggressiveness, and potential response to multiple and combinational chemotherapeutic
agents. Therefore, the value of a genetic test in specific malignancy is
more than for the selection of one specific chemotherapeutic agent. These
genetic changes need to be considered in the clinical context of the specific
tumor for each patient. The clinical decision to treat or not treat with
a specific agent takes into account all of these factors and is not made
on the basis of a single test result.
Thank you for your comment. Please, see point to point replies:
1. We now clarify in the methods the nebulous text (which was a verbatim
quotation). Please also see our replies to the peer-Peer Reviewers.
The definition includes somatic mutations. (there is an “or” connecting
several attributes of what falls under genetic testing, and somatic mutations
are not explicitly excluded). In any case we make clear in the report.
As described in the Methods of the TA, the introductions, and results of
the specific parts and the Discussion of the methodological topics, we are
cognizant of the distinction between germline and somatic variations.
Both the draft and the final report provide a detailed discussion of methodological
issues unique to heritable vs somatic genetic variation. We selected dasatinib
as a second-generation TKI after discussion with AHRQ and CMS.
|2. The naivete’ of the concept of “one bioanalyte - one drug” becomes apparent
in consideration of the drug dasatinib, one of the drugs used in the setting
of Bcr-Abl+ leukemias resistant or intolerant to prior therapy. There is
evidence that this drug also has activity against Src family kinases as well
as Flt3 and c-Kit. (Corey, et al, Clin Cancer Res 16:1149-58, 2010).
2. The Methods section presents detailed criteria for study eligibility.
Also, note that contrary to narrative reviews, a systematic review uses
structured and rigorous methodologies to provide answers to *specific*
|3. Additionally, pre-analytic issues are critical to the performance of
each assay, and must be given consideration. For example, the choice of method
used for the detection of a mutation will have a major impact the sensitivity
of the assay, with limits of detection ranging from 1 cell (or less) in one
million for a PCR approach targeting the mutant allele, to the requirement
that greater than 15-20% of cells contain the mutation for most sequencing
methods. The selection of pure tumor cells prior to sample processing can
further exaggerate apparent variations in analytic sensitivity, so that a
study utilizing relatively insensitive conventional sequencing, along with
selection for tumor cells, will likely vastly underestimate the true occurrence
of the mutation in a case series being studied. These “false negative” results
will lead to an inaccurate assessment of the clinical correlation or clinical
||3. As described in the methods section, analytic validity was not in the
scope of the TA. Please, also refer to the replies above.
|4. In January 2009, AMP published laboratory practice guidelines for detecting
and reporting BCR-ABL drug resistance mutations in CML and ALL. Those guidelines
effectively discussed the state of knowledge regarding BCR-ABL mutation testing
not only in considering analytical factors, but also in the clinical contexts
for which such testing has import (Jones, et al, J Molec Diag 11:4 ? 11,
2009). We asked Dr. Dan Jones, one of the authors of that report, to comment
(in qutoes below) on the technology assessment’s conclusions regarding BCR-ABL
||4. Replies to quotations by Dr Jones in points 5-8 below:
|5. KQ1: "The commentary in Key Question 1 is fair. However, the literature
on CML and mutations is pretty vast right now and some studies have been
omitted. Therefore, some qualifications on the conclusions reached in that
Key Question is recommended. The Authors need to emphasize that there are
big differences in the incidence of mutations (particularly T315I) and the
therapy responses depending on the phase of disease (chronic, accelerated
and blast phase) and lumping all together as "CML" is probably not useful
for interpretation of test results."
||5. Thank you for your comments. The Methods section presents an explanation
of the analyses performed in the TA. Briefly, the TA did not perform a meta-analysis
(did not “lump”) studies exactly because of clinical heterogeneity (dissimilarity).
We agree that the incidence of mutations is heterogeneous, which is already
described in the Results section of the TA. However, we disagree with the
comment that the incidence of the T315I mutation depends on the phase of
disease. This is not necessarily the case in our results (see Figure 13).
We agree that response to therapy depends upon the phase of disease and did
not “lump” different disease phases together. Further, the general methods
section has a brief explanation of graphical (“qualitative”) analyses of
published studies for readers who are not familiar with evidence-based methods
or clinical research methodology.
|6. KQ2: "There is some gathering data on levels of drug metabolizing genes
on responses to TKIs but agree that this question is not really relevant
to interpretation of BCR-ABL testing."
||6. It is unclear to us what the comment means. The key questions were refined
by AHRQ, CMS and Tufts EPC, influenced by the methodological framework of
the EGAPP initiative. The methods section includes a description of eligible
studies. In particular, no study performed interaction analyses. The TA includes
a description of the rational for focusing on interaction analyses to identify
modifiers of the pharmacogenetic effect.
7. KQ3: "I would encourage the Authors to include the reference Jabbour
E, Jones D, Kantarjian HM, O'Brien S, Tam C, Koller C, Burger JA, Borthakur
G, Wierda WG, Cortes J. Long-term outcome of patients with chronic myeloid
leukemia treated with second-generation tyrosine kinase inhibitors after
imatinib failure is predicted by the in vitro sensitivity of BCR-ABL kinase
domain mutations. Blood. 2009 Sep 3;114 (10):2037-43 which does show (retrospectively)
that if mutations are matched to the Kd for in vitro inhibition of second
(or third) TKIs that there are differences in outcome in chronic phase
CML. This would contradict the general statement in the first line of 3.4
Given the already extensive data on in vitro responses to particular TKIs,
a prospective study is unlikely to be done in CML to randomize treatment
choice based on mutation result. However, the European LeukemiaNet guidelines
(Baccarani M, Cortes J, Pane F, Niederwieser D, Saglio G, Apperley J et
al. Chronic Myeloid Leukemia: An Update of Concepts and Management Recommendations
of European LeukemiaNet. J Clin Oncol 2009 November) are an attempt to
codify current clinical practice on how detection of T315I impacts choice
The homoharringtonine clinical trial (published in abstract form, Khoury
HJ, Michallet M, Facon T, Guilhot F, Jones D, Hochaus A, Benichou A-C,
Schwartz R, Cortes J. Safety and efficacy study of subcutaneous homoharringtonine
(SC HHT) in imatinib-resistant chronic myeloid leukemia (CML) with the
T315I BCR-ABL kinase domain mutation ? initial report of a Phase II trial.
Blood 110(11):318a, 2007.) and to some extent the TKI-MK-457 clinical trial
(Blood, 15 January 2007, Vol. 109, No. 2, pp. 500-502) use presence of
the T315I mutation as enrollment criteria, based on the selective responses
of those particular agents against that mutation."
7. The methods section explains what is sought after in key question
4. Briefly, eligible studies are those that report the number of times
that the treatment decision changed before compared to after testing.
Regarding the first sentence of Section 3.4 (“In our systematic review
of the literature, presence of any BCR-ABL1 mutation does not appear to
predict differential response to treatment in CML patients treated with
imatinib-, dasatinib-, or nilotinib-based regimens.”): This is not pertinent
to KQ3. Further, we feel that the statement is accurate statement, as it
refers to having “any mutation”. This is based on the results described
Regarding the Jones paper: This was published after the last search. Further,
it is not in the scope of KQ3, since this paper does not describe the data
on treatment decision impact of the test (i.e., how frequently treatments
were changed after performing the test).In addition, T315I mutation testing
to predict treatment outcomes of homoharringtonine alone or TKI-MK-457
is beyond the scope of this technology assessment review.
|8. Key question 4 "Different 2nd and 3rd-generation TKIs (and non-KI therapies)
have different toxicity profiles so the use of BCR-ABL mutation data to influence
choice of a particular TKI will have benefits and harms to patients."
8. The Methods section of the TA describes the epidemiological, methodological
and clinical characteristics of eligible studies, as well as what is sought
after in KQ4.
None of the eligible studies provided clinical data for KQ4. No studies
provided specific data on clinical benefit or harm defined in details in
the general method section.
|9. In summary, we believe that if the authors had access to appropriately
qualified clinical and technical input, the value of their study would have
been markedly enhanced. Certainly, surveying the literature at a single point
in time for a rapidly growing field suffers the danger of being irrelevant
by the time the results are analyzed. This deficit would be very apparent
to anyone with true clinical experience. As it is, the conclusions can only
be regarded as having limited marginal value. We offer the Guidelines published
by AMP in 2009 as an example of a rational, coherent approach to assessment
of test efficacy and utility that recognizes that such an assessment must
be a dynamic, clinically relevant process. We strongly urge that future meta
analyses of published reports include appropriate scientific and clinical
expertise to better design the inquiries and better assess the outcomes for
reasonableness. We further urge that any such technical assessments be presented
in the appropriate clinical contexts. We believe that introducing these elements
will significantly enhance the validity and utility of future studies.
9. Thank you for the suggestions.As detailed in the Methods, the Results
and Discussion sections, we did not perform a meta-analysis in our systematic
review. A meta-analysis was performed only in the KRAS section of the TA.
No further reply necessary.
|Anonymous Reviewer 4
||College of American Pathologists
[the comments are presented as a numbered list to facilitate point to point replies]
Thank you for your comments. Please find point to point replies below:
1. The College commends the authors at the Tufts Evidence-Based Practice
Center for a well written and insightful report. It has a consistent format
throughout and explains the statistical analyses clearly. However, the
College is concerned about the performance of meta-analyses and other literature
reviews divorced from an understanding of the clinical use of the tests
which can result in the wrong questions being asked or incorrect framing
of the questions.
The College believes the questions should be more nuanced than stated
in the report. Question 1 asks “Does the genetic test result predict response
to therapy?” Testing and treatment are complicated, and reducing the
issue to a simple one bioanalyte-one drug (i.e. companion diagnostic) issue
is naive. The article by Jones et al. provides a very cogent review of
the status of BCR-ABL mutation monitoring which does address the complex
clinical contexts in which that question of imatinib resistance arises
and provides a useful model for this type of review. Tests may answer questions
important to the treatment of the patient but not directly related to response
to therapy including the following:
a. Is mutation testing performed simply to select a drug, or is it also
informative about likely disease aggressiveness and potential response
to any therapy?
b. Are there differences in degree of resistance to the various drugs?
c. After therapy has failed (for whatever reason), is testing used differently
than upon initiation of therapy?
1. The General Methods Section presents the rational behing the Key
Questions asked and the epidemiological, methodological and clinical characteristics
of eligible studies. We make the following clarifications: first, the eligibility
criteria of studies per key question are listed in the Mehods Section (general
and topic-specific). Further, the Methods section clearly defines what
is sought after in each question of the systematic review (population,
intervention, comparator, outcome, design). Please refer to the methods
section for a clear description of eligible research.
The key questions of the report were refined by the Tufts EPC in discussions
with AHRQ and CMS. Further, the TA did not perform meta-analyses in BCR-ABL1.
This is described in the methods, the results and the discussion.
Also, the TA used predefined and explicit definitions of what research
designs/clinical settings would be analyzed. This information is presented
in the Methods section of the BCR-ABL topic.
With respect to 1a-1c:
The posed questions span too broad a range of decisional contexts. The
TA has a clearly defined decisional context, as detailed in the Methods
Section of the BCR-ABL1 section.
2. Evidence-based medicine requires published evidence, and for diagnostic tests there may be sufficient data to implement an assay using intermediate outcomes rather than waiting for studies of long term outcome or survival. Diagnostics are critical to patient care but are also just one piece of medical decision-making processes, with many other clinicopathologic and socioeconomic variables contributing to patient outcomes.
We are troubled that the investigators were unable to find any studies that
answered Key Questions 2, 3, and 4. For example, the whole point of BCR-ABL
mutation testing is to understand the basis for treatment failure in a very
complex clinical context; yet they report no patient or disease-related factors
were found that affect the test results. Also, it does not make sense that
the authors found no information on Key Question 3: How does the gene testing
impact the therapeutic choice? Clearly K-RAS testing impacts therapeutic
2. The General Methods section and the specific methods sections of the
TA presents the inclusion and exclusion criteria employed as well as what
we considered as relevant evidence regarding each Key Question.
We believe that the TA is accurate in stating that none of the identified studies fulfilled the eligibility criteria for KQ2-4.
|3. On Key Question 4: What are the benefits and harms or adverse effects
for patients when managed with gene testing? It is not clear how one defines "benefits
or harms" in the context of genetic testing ? i.e. long term versus short term
or surrogate outcomes, or a prospective study in which patients are managed
with or without the genetic test. According to this report, none of the included
studies reported benefits of testing. Similarly, no study reported harms or
adverse effects of testing. We believe a greater understanding of the clinical
contexts in which this testing is used might allow an appropriate search for
the benefits and harms and adverse effects of inappropriate therapy for CML
for example. The effectiveness of various treatment regimens has been well
documented in multiple clinical studies and therefore unnecessary to include
in studies of this testing. In studies addressing treatment failure, of which
BCR-ABL1 mutations are only one cause, the evaluation of outcomes was not a
primary goal, so it is not surprising that these issues were not specifically
addressed. This is a deficiency of the very narrow approach taken.
||3. The Methods section clarifies what is meant under KQ4. Note that it includes direct data on benefits and harms not summarized under KQ1.
|4. In the conclusion the authors note that most studies analyze only treated
patients, "effectively assuming that effects in untreated patients are zero." The
assumption of the authors is not necessarily correct; the reason the studies
do not have untreated patient arms is that we are treating human beings, not
mice. To not offer some form of treatment in cancer therapy has potential ethical
implications. Although an untreated arm may be necessary for good data collection,
clinician cannot do this when treating patients.
||4. The methodological discussion of cross-cutting issues provides a detailed explanation behind the rational of requiring interaction tests between treated and untreaded groups. We kindly note that for the CYP2D6 and the KRAS topic randomized controlled trials were available and included in the review. The interpretation of our statement is out of context. We will rephrase to enhance clarity.
Specific comments on the Summary and Introduction:
5. In Section 3 of the Summary (page S-3), we suggest the removal of the
second sentence ?The presence of any BCR-ABL1 mutation does not appear
to predict differential response to treatment? and begin the summary with
the third sentence ?There is consistent evidence that presence of the [most
common] T315I mutation can predict TKI treatment failure?..? The second
sentence as currently written implies that no BCR-ABL mutations are associated
with treatment response, whereas it should read that if you query for the
presence of ANY mutation, treatment response cannot be predicted, likely
due to the fact that different mutations confer differential resistance
to TKIs. The fourth and fifth sentences summarize the effect of the presence
of ANY BCR-ABL mutation more clearly.
|5. Indeed the TA refers to the presence of any mutation. We revisited the
phrasing to enhance clarity: “The presence of any BCR-ABL1 mutation (all
mutations considered together) does not appear to predict differential response
to tyrosine kinase inhibitor (TKI) treatments (defined as imatinib-, dasatinib-,
and nilotinib-based regimens).”
|6. In the introduction the authors choose to use a definition of genetic
tests from National Human Genome Research Institute (NHGRI) which includes
only heritable mutations; then proceed to review studies of two tests that
do not meet the definition provided on page one of the report. We do not have
an issue with the definition provided, but suggest that the report be internally
||6. Thanks for pointing out the discrepancy with the direct quotation. We have rephrased for clarity.
Specific comments on Section 1:
7. We suggest that the omission of a recently published and relevant study
by Schroth et al. in JAMA , due to the timing of the review process, highlights
the difficulty in taking a snapshot in time approach to review of a rapidly
changing field. The paper by Schroth et al. was published in October 2009.
|7. This section has been updated in the final report to include 3 additional papers including the suggested one. The conclusions do not change.
Specific comments on Section 2:
8. Other factors not reviewed also impact the quality of the studies reviewed.
The College notes that the analytic validity of assays should be mentioned
as a factor impacting the quality of KRAS studies. Pre-analytic specimen
preparation is also critical for somatic mutation tests such as KRAS (e.g.
adequate tumor cell percentage). Though the test performance characteristics
such as precision (reproducibility), analytic sensitivity, etc are important
for evaluating the utility of these test but we understand that this topic
is beyond the scope of this review. Quality is often assumed because CAP
accreditation and proficiency programs provide successful oversight for consistent
|8. As described in the methods section, analytic validity was not in the scope of the TA. We also note that in the current version of the TA the studies are not penalized for issues arising from less than perfect analytic validity (we do not account for measurement error).
|9. On page 29, this document should describe HUGO Gene Nomenclature Committee-approved
official gene symbols along with alias in parenthesis. For example: Replace
c-erbB-1 with ERBB1, and c-erbB-2 with ERBB2, etc. (Correct symbols and colloquial
names for every gene and its encoded protein can be found at www.genenames.org.)
Gene symbols are in italics (KRAS) while protein symbols are not (KRAS).
||9. Thank you for the pointer on gene names. In the Methods section we state that the TA refers to gene symbols in italics and protein symbols in regular font. We have verified that the convention is followed throughout.
Specific comments on Section 3:
10. We suggest a change to the verbiage throughout the document referring
to the BCR-ABL1 T315I mutation as ?relatively rare,? as T315I is one of the
most commonly identified BCR-ABL mutations. (see Jones, et al. J Mol Diagn
|10. We have defined what we mean as (relatively) rare. We do not change the verbiage. In our review, the frequency of T315I is not necessarily the highest of all identified mutations across the included studies (see out Figure 13).
|11. On the top of page 130 Section 3-4, we suggest the following addition,
?A limitation of this systematic review is that all BCR-ABL1 mutations were
lumped together, potentially restricting our ability to identify all but the
strongest associations between a given mutation and drug resistance. Further
complicating the process was the variability in study design, including varying
indications for testing and varying spectra of analytes tested. Specific mutations
beyond T315I that predict TKI resistance cannot be ruled out. Generally, it
was found that patients who developed mutations during treatment experienced
higher imatinib resistance compared with those with no mutations detected during
||11. Please refer to the discussion section of BCR-ABL1. Thank you for the suggestions. We have revised the initial part of the summary to clarify the points made.
|12. It is striking to know that all the mutation tests performed so far
have not shown to be predictive to patient outcomes except T315I. The authors
may suggest research designs for future studies to answer key questions and
items important for future data mining such as the detection sensitivity of
an assay for future analysis.
||12. This is not what the TA results were nor what the TA concludes. The Methods Section describes what was studied, how, and in which clinical context. We have already proposed that an appropriate next step would be a collaborative international patient registry or patient-level meta-analyses using standardized definitions of clinical context, disease stage, follow-up, and outcome assessment to identify particular mutations beyond T315I to predict treatment outcomes.