Table 2. Public Reviewer Comments

Disposition of Comments

Comments received from Systematic Reviews on Selected Pharmacogenetic Tests for Cancer Treatment.

Table 2. Public Review Comments

Reviewer NameReviewer AffiliationSectionReviewer CommentsAuthor Response
Harry BurkeGeorge Washington UniversityGeneral �[The Peer Reviewer sent a draft of a manuscript submitted for publication for consideration]Thank you for your contribution. We agree that a distinction between prognosis and prediction is important. The methods section of the TA describes�the epidemiological, methodological and clinical characteristics of eligible studies. The submitted manuscript is not eligible for inclusion in the TA.
John HermanekAmgen�KRAS[The comment referred to more recent information on panitumumab and KRAS, published after the last update of the draft TA.]The TA has been updated to include publications on KRAS through March 2010. Also, according to the eligibility criteria only full publications are eligible.
Gregory D. PawelskiNoneGeneral �[The Peer Reviewer presented an overall interpretation of translational research in personalized medicine and cancer]�Thank you for you contribution. The TA used a systematic review approach to address the posed key questions. The Methods section describes the approach, and the epidemiological, methodological and clinical characteristics of eligible studies, as well as the statistical analyses performed.
August J. Salvado�Novartis Pharmaceuticals�BCR-ABL1

[The Peer Reviewer makes clarifying comments also with respect to the MEDCAC. Two points that could be relevant to the report are brought up:]

1. [�] It warrants mention that CML is the only hematological malignancy covered by the technology assessment, and as such it should be recognized that physicians utilize different techniques to measure treatment response and clinical efficacy than are typically used to monitor solid tumors. PCR monitoring of BCR-ABL is the least invasive and most sensitive test currently available to hematologists to monitor treatment response and clinical outcomes in CML.�

2. The second point of clarification focuses on the scope of the technology assessment in reviewing the utility of BCR-ABL testing to predict treatment response based upon polymorphisms of point mutations. We concur with the technology assessment�s conclusion that there is little evidence at the current time supporting the predictive value of point mutations in the BCR-ABL1 gene to determine differences in clinical benefit of one TKI agent over another. However, one exception to this point that should be explicitly indicated in the technology assessment is that the T315I mutation predicts a decreased clinical response to all currently commercially available TKI therapies. In addition, when this mutation is present, physicians should prepare for an aggressive treatment regimen, potentially including stem cell therapy or experimental treatments, in the event that patients with T315I mutations fail currently available TKI therapies.

Thank you for your comments. We note that the MEDCAC had a broader set of questions that the TA. The key questions of the TA are listed in the methods section. With respect to the 2 clarification points:

First, the TA states clearly that tumor load monitoring is outside its scope. Only mutation testing is in the scope of the TA and all our analyses pertain to mutations only.

Second, the TA clearly singles out T315I and performs separate analyses for this mutation. However, the TA does not make recommendations for treatment or the management of patients with specific mutations.

Lawrence SolbergAmerican Society of Hematology (ASH)General comments on BCR-ABL1 mutation testing�

(This is a distillation, see pdf) The comments are the ASH answers to the key questions of the TA.

1. Does BCR-ABL1 mutation testing predict response to TKI therapy?

Yes. There is enough evidence, both in vitro and in vivo that different mutations have different sensitivity to different inhibitors and that the in vitro sensitivity correlates well with the clinical response of patients [�].In view of the recognized value of mutations analysis in predicting for response, this assay is recommended by a panel of international experts sponsored by the European LeukemiaNet in all patients after failure or suboptimal response to imatinib, and before changing therapy.

2. What patient- and disease-relevant factors affect the test results, their interpretation or their predictive response to therapy?

The most important factor is the response to prior therapy [�]

3. How does the gene testing impact the therapeutic choice?

As mentioned earlier, patients with mutations of intermediate or low sensitivity to a certain tyrosine kinase inhibitor should be offered the alternative agent [�].

4. What are the benefits and harms or adverse effects for patients when managed with gene testing?

5. The obvious benefit is the possible selection of an agent that may offer the best probability of response. There are no known adverse events other than those implicated with the venipuncture. However, the test is usually obtained at the time other routine monitoring tests are obtained.

Thank you for your comments, which are a narrative (non-systematic) review of the positions of the ASH.

The TA used a systematic review approach to address the posed key questions. The Methods section describes the approach, and the epidemiological, methodological and clinical characteristics of eligible studies, as well as the statistical analyses performed.

Studies that evaluated in-vitro sensitivity of BCR-Abl1 mutations to TKIs do not meet our inclusion criteria; thus, we did not assess these studies. Also, we did not assess the three studies reporting the association between in-vitro sensitivity of the mutations to TKIs and clinical outcomes (Hughes 2009, Muller 2009, and Jabbour 2009), which could be viewed as a way to lump some specific mutations together to perform subgroup analysis, because they were published after our literature search.

Mark SomerfieldAmerican Society of Clinical Oncology�KRAS

First, on p. S-1, the authors list Key Question (KQ) 4 as �What are the benefits and harms or adverse effects for patients when managed with gene testing?� They note that no studies were identified that could be used to answer this question.

Elsewhere (p. 3), however, the authors provide a broader explication of KQ4 that creates some confusion: �What are the benefits and harms or adverse effects for patients when managed with gene testing? Any cognitive, behavioral or other health effects of testing with the three tests of interest. These may be direct effects of the process of testing (e.g., increased anxiety) or downstream effects stemming from treatment decisions informed by testing.�

Given this broader framework for KQ4, the statement that no studies were identified that could be used to answer this question is problematic. Thus, in the KRAS section the authors concluded that, �When treated with anti-EGFR antibodies, patients with KRAS mutations were less likely to experience treatment benefit, compared to patients whose tumors were wild-type for KRAS mutations, for all outcomes assessed. These results were confirmed in several RCT-based analyses of progression-free survival that demonstrated a significant treatment-by-KRAS mutation interaction in three out of the four cases where such analyses were reported.�

These results, which arguably reflect downstream effects stemming from treatment decisions informed by KRAS testing, would seem to address KQ4, although the authors do note that �Most of the latter [the downstream effects] would be captured by Key Question 1.�

Can the authors clarify this? Would it be simpler to restrict KQ4 to the benefits and harms of the testing itself, defined as �any cognitive, behavioral or other health effects of testing,? vs. clouding things by referring to the downstream effects element�

We appreciate the comment. We agree that there is the potential for confusion, and we have rephrased the methods section to enhance clarity. Indeed KQ1 captures the downstream effects of testing, and KQ4 captures other (additional) benefits and harms that are related to testing and are beyond those covered in KQ1.
�Mark SomerfieldAmerican Society of Clinical Oncology��CYP2D6

p. S-2, the authors note that �It is questionable whether pharmacogenetic testing of germline (heritable) variations in CYP2D6 can predict differential response to adjuvant tamoxifen in women with non-metastatic breast cancer. Further, evidence is severely limited for tamoxifen-treated women with metastatic disease. Our conclusions are in accordance with the 2009 American Society of Clinical Oncology (ASCO) practice guideline update.�

The ASCO guideline that is referenced here is an update of a guideline on breast cancer risk reduction. The ASCO guideline update did consider CYP2D6 and concluded that, �Given the limited evidence, CYP2D6 testing is currently not recommended in the preventive setting.� However, it is not accurate in our view to state that the conclusion of the AHRQ systematic review are �in accordance with� the ASCO guideline; the ASCO guideline concerns the very different setting of risk reduction or chemoprevention, and the AHRQ report focuses on the adjuvant and metastatic settings.

Later, in the body of the report (p. 27), the authors do note indirectly that the ASCO guideline concerned the risk reduction or prevention setting: ?Our conclusions are in accordance with the relevant 2009 ASCO practice guideline update, which states ?Given the limited evidence, CYP2D6 testing is currently not recommended in the preventive setting.?? However, as many may read only the executive summary of the technology assessment, this point should be made in that summary as well. Ideally, the authors would change the phrasing from ?in accordance with? to ?[our conclusions] are analogous to the conclusion of the ASCO guideline that addressed the role of CYP2D6 in the breast cancer risk reduction setting.?�

Thank you for providing this clarification. We rephrased the pertinent parts for clarity, both in the Executive summary and in the body of the TA adopting some of the Peer Reviewer�s wording.
Anonymous Reviewer 1NAGeneral �Utilization of pharmacogenetic information to prescribe medications on the basis of the needs of the individual patients can save money and lives. Knowing that Tamoxifen will not work for you, if you were a cancer patient, is something you would like to know before taking the medication for years just to learn that your cancer is back. Money and time are wasted when a patient takes a drug that has no effect because of variants in a patient�s genetic make-up! Pharmacogenetics is a tool that physicians can use in many areas of medicine to better and more safely help patients. More knowledge is always better for any disease state.��Thank you for your comments. No reply needed.
Anonymous Reviewer 2Friends of Cancer ResearchGeneral�

[The letter contains several general thoughts and three more specific mentions, listed below:]

1. We believe that it is premature to accurately determine impact of CYP2D6 polymorphisms on response to tamoxifen based solely this literature review Currently, multiple trials are underway or have been completed that will prospectively examine the role of CYP2D6 genotype on the clinical effect of tamoxifen treatment in both the metastatic and adjuvant setting.1 In addition, a study conducted by Schroth et. al., published after the timeframe of the technology assessment, retrospectively examined 1,325 patients treated with adjuvant tamoxifen for early stage breast cancer. The study concluded that, �Among women with breast cancer treated with tamoxifen, there was an association between CYP2D6 variation and clinical outcomes, such that the presence of two functional CYP2D6 alleles was associated with better clinical outcomes and the presence of nonfunctional or reduced-function alleles with worse outcomes.� While this study is not a randomized controlled trial, of which it may be several years until clinical results will demonstrate prospectively what the actual impact of altered drug metabolism is, the conclusions of this study are indicative of the complexity of the topic and weakness of the currently available evidence. Therefore, it would be difficult to conclude that pharmacogenetic testing for variations in CYP2D6 should not be used by physicians and patients with breast cancer to determine course of treatment.

2. Perhaps one of the most notable and recent changes in the practice oncology is the utilization of KRAS testing to aid therapeutic decision making for colorectal cancer patients. The conclusions described in the technology assessment are in accordance with the body of clinical evidence that has prospectively demonstrated, in several cases, the predictive value of testing for KRAS mutational status prior to delivering the anti-EGFR therapies, cetuximab and panitumumab, to colorectal cancer patients. However, the technology assessment suggests that the data are not being used in therapeutic decision making (S-5: �In all three examples, we found no evidence on whether testing impacts therapeutic decisions��). [�]

3. In the case of evaluating how variations in BCR-ABL1 impact response to three drugs used for treatment of chronic myeloid leukemia (CML), the technology assessment concludes that �The presence of any BCR-ABL1 mutation does not appear to predict differential response to treatment�with any of the three tyrosine kinase inhibitor (TKI)-based regimens.� While this is an accurate summary of existing published evidence, there is a rapidly developing body of new clinical evidence that will provide additional information on this complex topic, including over a hundred studies examining the use of imatinib, dasatinib or nilotinib in chronic myeloid leukemia.4 However, to date BCR-ABL pharmacogenetic testing has not been shown to be a predictive factor to guide first-line treatment decisions. But the value of molecular evaluation in this case should not be dismissed due to the existence of molecular methods, such as RQ-PCR transcript testing, that can be used to monitor treatment response

Thank you for your comments. The TA used a systematic review approach to address the posed key questions. The Methods section describes the approach, and the epidemiological, methodological and clinical characteristics of eligible studies, as well as the statistical analyses performed.

1. CYP2D6: The TA summarized the published evidence and does not make recommendations for practice. The final version of the TA, based on an updated search and data extraction, includes the Schroth paper (among others), and reaches the same conclusions.

2. The Methods section describes exactly what is meant by each key question, which is quite specific.

As described in the TA there were no eligible studies that quantified the impact of testing on therapeutic decisions in the form of how many times the physicians changed treatment, etc. This is qualitatively different from the comments made by the Peer Reviewer.

3. The TA clearly states that only BCR-ABL1 mutations are in the scope of the pertinent systematic review. As per our reply to other Peer Reviewers, transcript levels are not examined in the TA. This was predetermined in out review protocol and clearly stated in the methods section of the TA.

Anonymous Reviewer 3Association for Molecular PathologyABCR-ABL1 � General

[comment put into numbered paragraphs:]

1. We very much appreciate the authors� statistical expertise of this technology assessment report; however, we note a number of shortcomings that compromise the relevance of the report�s conclusions. We believe that many of these shortcomings could have been avoided had there been prior input from clinicians and molecular pathologists intimately familiar with the performance and clinical utilization of these tests

This lack of clinical input is immediately evident in the definition of genetic test adopted by the study:

�The analysis of human DNA, RNA, chromosomes, proteins, and certain metabolites in order to detect heritable disease-related genotypes, mutations, phenotypes, or karyotypes for clinical purposes. Such purposes include predicting risk of disease, identifying carriers, establishing prenatal and clinical diagnosis or prognosis. Prenatal, newborn, and carrier screening, as well as testing in high-risk families, are included. Tests for metabolites are covered only when they are undertaken with high probability that an excess or deficiency of the metabolite indicates the presence of heritable mutations in single genes.�

Two of the three tests evaluated in the study do not fulfill this definition, highlighting a superficial understanding of the biology underlying these tests and how they are used clinically. Acknowledging that pharmacogenomic tests can be a special type of genetic tests, it is noteworthy that the authors fail to appreciate that of the three tests evaluated in the report, only the CYP2D6 qualifies as being heritable. Alterations in the KRAS gene and in the BCR-ABL translocation are not heritable, but are tumor specific, intrinsic to the neoplastic process. This distinction is not moot. Polymorphisms that influence drug metabolism can be identified in healthy individuals and can have bearing on dosing or drug selection of numerous therapeutic agents. Tumor specific genetic changes, in contrast, have significance beyond simple choice of drug, influencing disease recognition, disease prognosis, tumor aggressiveness, and potential response to multiple and combinational chemotherapeutic agents. Therefore, the value of a genetic test in specific malignancy is more than for the selection of one specific chemotherapeutic agent. These genetic changes need to be considered in the clinical context of the specific tumor for each patient. The clinical decision to treat or not treat with a specific agent takes into account all of these factors and is not made on the basis of a single test result.

Thank you for your comment. Please, see point to point replies:

1. We now clarify in the methods the nebulous text (which was a verbatim quotation). Please also see our replies to the peer-Peer Reviewers.

The definition includes somatic mutations. (there is an �or� connecting several attributes of what falls under genetic testing, and somatic mutations are not explicitly excluded). In any case we make clear in the report.

As described in the Methods of the TA, the introductions, and results of the specific parts and the Discussion of the methodological topics, we are cognizant of the distinction between germline and somatic variations.

Both the draft and the final report provide a detailed discussion of methodological issues unique to heritable vs somatic genetic variation. We selected dasatinib as a second-generation TKI after discussion with AHRQ and CMS.

2. The naivete� of the concept of �one bioanalyte - one drug� becomes apparent in consideration of the drug dasatinib, one of the drugs used in the setting of Bcr-Abl+ leukemias resistant or intolerant to prior therapy. There is evidence that this drug also has activity against Src family kinases as well as Flt3 and c-Kit. (Corey, et al, Clin Cancer Res 16:1149-58, 2010).

2. The Methods section presents detailed criteria for study eligibility.

Also, note that contrary to narrative reviews, a systematic review uses structured and rigorous methodologies to provide answers to *specific* key questions

3. Additionally, pre-analytic issues are critical to the performance of each assay, and must be given consideration. For example, the choice of method used for the detection of a mutation will have a major impact the sensitivity of the assay, with limits of detection ranging from 1 cell (or less) in one million for a PCR approach targeting the mutant allele, to the requirement that greater than 15-20% of cells contain the mutation for most sequencing methods. The selection of pure tumor cells prior to sample processing can further exaggerate apparent variations in analytic sensitivity, so that a study utilizing relatively insensitive conventional sequencing, along with selection for tumor cells, will likely vastly underestimate the true occurrence of the mutation in a case series being studied. These �false negative� results will lead to an inaccurate assessment of the clinical correlation or clinical utility.3. As described in the methods section, analytic validity was not in the scope of the TA. Please, also refer to the replies above.
4. In January 2009, AMP published laboratory practice guidelines for detecting and reporting BCR-ABL drug resistance mutations in CML and ALL. Those guidelines effectively discussed the state of knowledge regarding BCR-ABL mutation testing not only in considering analytical factors, but also in the clinical contexts for which such testing has import (Jones, et al, J Molec Diag 11:4 ? 11, 2009). We asked Dr. Dan Jones, one of the authors of that report, to comment (in qutoes below) on the technology assessment�s conclusions regarding BCR-ABL mutation testing:4. Replies to quotations by Dr Jones in points 5-8 below:
5. KQ1: "The commentary in Key Question 1 is fair. However, the literature on CML and mutations is pretty vast right now and some studies have been omitted. Therefore, some qualifications on the conclusions reached in that Key Question is recommended. The Authors need to emphasize that there are big differences in the incidence of mutations (particularly T315I) and the therapy responses depending on the phase of disease (chronic, accelerated and blast phase) and lumping all together as "CML" is probably not useful for interpretation of test results."5. Thank you for your comments. The Methods section presents an explanation of the analyses performed in the TA. Briefly, the TA did not perform a meta-analysis (did not �lump�) studies exactly because of clinical heterogeneity (dissimilarity). We agree that the incidence of mutations is heterogeneous, which is already described in the Results section of the TA. However, we disagree with the comment that the incidence of the T315I mutation depends on the phase of disease. This is not necessarily the case in our results (see Figure 13). We agree that response to therapy depends upon the phase of disease and did not �lump� different disease phases together. Further, the general methods section has a brief explanation of graphical (�qualitative�) analyses of published studies for readers who are not familiar with evidence-based methods or clinical research methodology.
6. KQ2: "There is some gathering data on levels of drug metabolizing genes on responses to TKIs but agree that this question is not really relevant to interpretation of BCR-ABL testing."6. It is unclear to us what the comment means. The key questions were refined by AHRQ, CMS and Tufts EPC, influenced by the methodological framework of the EGAPP initiative. The methods section includes a description of eligible studies. In particular, no study performed interaction analyses. The TA includes a description of the rational for focusing on interaction analyses to identify modifiers of the pharmacogenetic effect.

7. KQ3: "I would encourage the Authors to include the reference Jabbour E, Jones D, Kantarjian HM, O'Brien S, Tam C, Koller C, Burger JA, Borthakur G, Wierda WG, Cortes J. Long-term outcome of patients with chronic myeloid leukemia treated with second-generation tyrosine kinase inhibitors after imatinib failure is predicted by the in vitro sensitivity of BCR-ABL kinase domain mutations. Blood. 2009 Sep 3;114 (10):2037-43 which does show (retrospectively) that if mutations are matched to the Kd for in vitro inhibition of second (or third) TKIs that there are differences in outcome in chronic phase CML. This would contradict the general statement in the first line of 3.4 Discussion.�

Given the already extensive data on in vitro responses to particular TKIs, a prospective study is unlikely to be done in CML to randomize treatment choice based on mutation result. However, the European LeukemiaNet guidelines (Baccarani M, Cortes J, Pane F, Niederwieser D, Saglio G, Apperley J et al. Chronic Myeloid Leukemia: An Update of Concepts and Management Recommendations of European LeukemiaNet. J Clin Oncol 2009 November) are an attempt to codify current clinical practice on how detection of T315I impacts choice of therapy.

The homoharringtonine clinical trial (published in abstract form, Khoury HJ, Michallet M, Facon T, Guilhot F, Jones D, Hochaus A, Benichou A-C, Schwartz R, Cortes J. Safety and efficacy study of subcutaneous homoharringtonine (SC HHT) in imatinib-resistant chronic myeloid leukemia (CML) with the T315I BCR-ABL kinase domain mutation ? initial report of a Phase II trial. Blood 110(11):318a, 2007.) and to some extent the TKI-MK-457 clinical trial (Blood, 15 January 2007, Vol. 109, No. 2, pp. 500-502) use presence of the T315I mutation as enrollment criteria, based on the selective responses of those particular agents against that mutation."

7. The methods section explains what is sought after in key question 4. Briefly, eligible studies are those that report the number of times that the treatment decision changed before compared to after testing.

Regarding the first sentence of Section 3.4 (�In our systematic review of the literature, presence of any BCR-ABL1 mutation does not appear to predict differential response to treatment in CML patients treated with imatinib-, dasatinib-, or nilotinib-based regimens.�): This is not pertinent to KQ3. Further, we feel that the statement is accurate statement, as it refers to having �any mutation�. This is based on the results described in KQ1.

Regarding the Jones paper: This was published after the last search. Further, it is not in the scope of KQ3, since this paper does not describe the data on treatment decision impact of the test (i.e., how frequently treatments were changed after performing the test).In addition, T315I mutation testing to predict treatment outcomes of homoharringtonine alone or TKI-MK-457 is beyond the scope of this technology assessment review.

8. Key question 4 "Different 2nd and 3rd-generation TKIs (and non-KI therapies) have different toxicity profiles so the use of BCR-ABL mutation data to influence choice of a particular TKI will have benefits and harms to patients."

8. The Methods section of the TA describes the epidemiological, methodological and clinical characteristics of eligible studies, as well as what is sought after in KQ4.

None of the eligible studies provided clinical data for KQ4. No studies provided specific data on clinical benefit or harm defined in details in the general method section.

9. In summary, we believe that if the authors had access to appropriately qualified clinical and technical input, the value of their study would have been markedly enhanced. Certainly, surveying the literature at a single point in time for a rapidly growing field suffers the danger of being irrelevant by the time the results are analyzed. This deficit would be very apparent to anyone with true clinical experience. As it is, the conclusions can only be regarded as having limited marginal value. We offer the Guidelines published by AMP in 2009 as an example of a rational, coherent approach to assessment of test efficacy and utility that recognizes that such an assessment must be a dynamic, clinically relevant process. We strongly urge that future meta analyses of published reports include appropriate scientific and clinical expertise to better design the inquiries and better assess the outcomes for reasonableness. We further urge that any such technical assessments be presented in the appropriate clinical contexts. We believe that introducing these elements will significantly enhance the validity and utility of future studies.

9. Thank you for the suggestions.As detailed in the Methods, the Results and Discussion sections, we did not perform a meta-analysis in our systematic review. A meta-analysis was performed only in the KRAS section of the TA.

No further reply necessary.

Anonymous Reviewer 4College of American PathologistsBCR-ABL1 General

[the comments are presented as a numbered list to facilitate point to point replies]

Thank you for your comments. Please find point to point replies below:

1. The College commends the authors at the Tufts Evidence-Based Practice Center for a well written and insightful report. It has a consistent format throughout and explains the statistical analyses clearly. However, the College is concerned about the performance of meta-analyses and other literature reviews divorced from an understanding of the clinical use of the tests which can result in the wrong questions being asked or incorrect framing of the questions.

The College believes the questions should be more nuanced than stated in the report. Question 1 asks �Does the genetic test result predict response to therapy?� Testing and treatment are complicated, and reducing the issue to a simple one bioanalyte-one drug (i.e. companion diagnostic) issue is naive. The article by Jones et al. provides a very cogent review of the status of BCR-ABL mutation monitoring which does address the complex clinical contexts in which that question of imatinib resistance arises and provides a useful model for this type of review. Tests may answer questions important to the treatment of the patient but not directly related to response to therapy including the following:

a. Is mutation testing performed simply to select a drug, or is it also informative about likely disease aggressiveness and potential response to any therapy?

b. Are there differences in degree of resistance to the various drugs?

c. After therapy has failed (for whatever reason), is testing used differently than upon initiation of therapy?

1. The General Methods Section presents the rational behing the Key Questions asked and the epidemiological, methodological and clinical characteristics of eligible studies. We make the following clarifications: first, the eligibility criteria of studies per key question are listed in the Mehods Section (general and topic-specific). Further, the Methods section clearly defines what is sought after in each question of the systematic review (population, intervention, comparator, outcome, design). Please refer to the methods section for a clear description of eligible research.

The key questions of the report were refined by the Tufts EPC in discussions with AHRQ and CMS. Further, the TA did not perform meta-analyses in BCR-ABL1. This is described in the methods, the results and the discussion.

Also, the TA used predefined and explicit definitions of what research designs/clinical settings would be analyzed. This information is presented in the Methods section of the BCR-ABL topic.

With respect to 1a-1c:

The posed questions span too broad a range of decisional contexts. The TA has a clearly defined decisional context, as detailed in the Methods Section of the BCR-ABL1 section.

2. Evidence-based medicine requires published evidence, and for diagnostic tests there may be sufficient data to implement an assay using intermediate outcomes rather than waiting for studies of long term outcome or survival. Diagnostics are critical to patient care but are also just one piece of medical decision-making processes, with many other clinicopathologic and socioeconomic variables contributing to patient outcomes.

We are troubled that the investigators were unable to find any studies that answered Key Questions 2, 3, and 4. For example, the whole point of BCR-ABL mutation testing is to understand the basis for treatment failure in a very complex clinical context; yet they report no patient or disease-related factors were found that affect the test results. Also, it does not make sense that the authors found no information on Key Question 3: How does the gene testing impact the therapeutic choice? Clearly K-RAS testing impacts therapeutic choice.

2. The General Methods section and the specific methods sections of the TA presents the inclusion and exclusion criteria employed as well as what we considered as relevant evidence regarding each Key Question.

We believe that the TA is accurate in stating that none of the identified studies fulfilled the eligibility criteria for KQ2-4.

3. On Key Question 4: What are the benefits and harms or adverse effects for patients when managed with gene testing? It is not clear how one defines "benefits or harms" in the context of genetic testing ? i.e. long term versus short term or surrogate outcomes, or a prospective study in which patients are managed with or without the genetic test. According to this report, none of the included studies reported benefits of testing. Similarly, no study reported harms or adverse effects of testing. We believe a greater understanding of the clinical contexts in which this testing is used might allow an appropriate search for the benefits and harms and adverse effects of inappropriate therapy for CML for example. The effectiveness of various treatment regimens has been well documented in multiple clinical studies and therefore unnecessary to include in studies of this testing. In studies addressing treatment failure, of which BCR-ABL1 mutations are only one cause, the evaluation of outcomes was not a primary goal, so it is not surprising that these issues were not specifically addressed. This is a deficiency of the very narrow approach taken.3. The Methods section clarifies what is meant under KQ4. Note that it includes direct data on benefits and harms not summarized under KQ1.
4. In the conclusion the authors note that most studies analyze only treated patients, "effectively assuming that effects in untreated patients are zero." The assumption of the authors is not necessarily correct; the reason the studies do not have untreated patient arms is that we are treating human beings, not mice. To not offer some form of treatment in cancer therapy has potential ethical implications. Although an untreated arm may be necessary for good data collection, clinician cannot do this when treating patients.4. The methodological discussion of cross-cutting issues provides a detailed explanation behind the rational of requiring interaction tests between treated and untreaded groups. We kindly note that for the CYP2D6 and the KRAS topic randomized controlled trials were available and included in the review. The interpretation of our statement is out of context. We will rephrase to enhance clarity.

Specific comments on the Summary and Introduction:

5. In Section 3 of the Summary (page S-3), we suggest the removal of the second sentence ?The presence of any BCR-ABL1 mutation does not appear to predict differential response to treatment? and begin the summary with the third sentence ?There is consistent evidence that presence of the [most common] T315I mutation can predict TKI treatment failure?..? The second sentence as currently written implies that no BCR-ABL mutations are associated with treatment response, whereas it should read that if you query for the presence of ANY mutation, treatment response cannot be predicted, likely due to the fact that different mutations confer differential resistance to TKIs. The fourth and fifth sentences summarize the effect of the presence of ANY BCR-ABL mutation more clearly.

5. Indeed the TA refers to the presence of any mutation. We revisited the phrasing to enhance clarity: �The presence of any BCR-ABL1 mutation (all mutations considered together) does not appear to predict differential response to tyrosine kinase inhibitor (TKI) treatments (defined as imatinib-, dasatinib-, and nilotinib-based regimens).�
6. In the introduction the authors choose to use a definition of genetic tests from National Human Genome Research Institute (NHGRI) which includes only heritable mutations; then proceed to review studies of two tests that do not meet the definition provided on page one of the report. We do not have an issue with the definition provided, but suggest that the report be internally consistent.6. Thanks for pointing out the discrepancy with the direct quotation. We have rephrased for clarity.

Specific comments on Section 1:

7. We suggest that the omission of a recently published and relevant study by Schroth et al. in JAMA , due to the timing of the review process, highlights the difficulty in taking a snapshot in time approach to review of a rapidly changing field. The paper by Schroth et al. was published in October 2009.

7. This section has been updated in the final report to include 3 additional papers including the suggested one. The conclusions do not change.

Specific comments on Section 2:

8. Other factors not reviewed also impact the quality of the studies reviewed. The College notes that the analytic validity of assays should be mentioned as a factor impacting the quality of KRAS studies. Pre-analytic specimen preparation is also critical for somatic mutation tests such as KRAS (e.g. adequate tumor cell percentage). Though the test performance characteristics such as precision (reproducibility), analytic sensitivity, etc are important for evaluating the utility of these test but we understand that this topic is beyond the scope of this review. Quality is often assumed because CAP accreditation and proficiency programs provide successful oversight for consistent laboratory performance.

8. As described in the methods section, analytic validity was not in the scope of the TA. We also note that in the current version of the TA the studies are not penalized for issues arising from less than perfect analytic validity (we do not account for measurement error).
9. On page 29, this document should describe HUGO Gene Nomenclature Committee-approved official gene symbols along with alias in parenthesis. For example: Replace c-erbB-1 with ERBB1, and c-erbB-2 with ERBB2, etc. (Correct symbols and colloquial names for every gene and its encoded protein can be found at Gene symbols are in italics (KRAS) while protein symbols are not (KRAS).9. Thank you for the pointer on gene names. In the Methods section we state that the TA refers to gene symbols in italics and protein symbols in regular font. We have verified that the convention is followed throughout.

Specific comments on Section 3:

10.�� We suggest a change to the verbiage throughout the document referring to the BCR-ABL1 T315I mutation as ?relatively rare,? as T315I is one of the most commonly identified BCR-ABL mutations. (see Jones, et al. J Mol Diagn 2009;11:4-11).

10.�� We have defined what we mean as (relatively) rare. We do not change the verbiage. In our review, the frequency of T315I is not necessarily the highest of all identified mutations across the included studies (see out Figure 13).
11.�� On the top of page 130 Section 3-4, we suggest the following addition, ?A limitation of this systematic review is that all BCR-ABL1 mutations were lumped together, potentially restricting our ability to identify all but the strongest associations between a given mutation and drug resistance. Further complicating the process was the variability in study design, including varying indications for testing and varying spectra of analytes tested. Specific mutations beyond T315I that predict TKI resistance cannot be ruled out. Generally, it was found that patients who developed mutations during treatment experienced higher imatinib resistance compared with those with no mutations detected during the follow-up.?11.�� Please refer to the discussion section of BCR-ABL1. Thank you for the suggestions. We have revised the initial part of the summary to clarify the points made.
12.�� It is striking to know that all the mutation tests performed so far have not shown to be predictive to patient outcomes except T315I. The authors may suggest research designs for future studies to answer key questions and items important for future data mining such as the detection sensitivity of an assay for future analysis.12.�� This is not what the TA results were nor what the TA concludes. The Methods Section describes what was studied, how, and in which clinical context. We have already proposed that an appropriate next step would be a collaborative international patient registry or patient-level meta-analyses using standardized definitions of clinical context, disease stage, follow-up, and outcome assessment to identify particular mutations beyond T315I to predict treatment outcomes.

Return to Contents


Current as of October 2010
Internet Citation: Table 2. Public Reviewer Comments: Disposition of Comments. October 2010. Agency for Healthcare Research and Quality, Rockville, MD.