Public Review Comments Appendix: Response to Amgen

Disposition of Comments

Comments received from draft review on The Impact of Pre-transplant Red Blood Cell Transfusions in Renal Allograft Rejection.

The Impact of Pre-transplant Red Blood Cell Transfusions in Renal Allograft Rejection

Public Review Comments Appendix: Response to Amgen

Note: Responses to comments in this appendix have been slightly reformatted to comply with federal accessibility requirements. All responses are indented and marked with [Response:] to signify responses by comment reviewers.

General Thoughts

[Response:] We would like to thank Amgen for the detailed review and comments on the technology assessment. In order to address the comments systematically, we have listed all our responses to the comments in the format that Amgen provided us. Please refer to the final report for specific changes.

Executive Summary

As the executive summary is an abbreviated version of text from other sections of the technology assessment, please refer to Amgen’s detailed comments in the individual sections. Below is a high-level summary of Amgen’s comments.

The objective of the draft technology assessment was to evaluate the evidence regarding the impact of transfusions on renal transplant outcomes. However, the technology assessment has serious limitations and therefore should not be used in its current form to inform policy decisions. In this document, Amgen provides a review of the technology assessment and identifies its key limitations. The limitations have been grouped into two categories – design limitations and analysis limitations.

Design limitations

1. The questions formulated to address the stated objective in the technology assessment were inappropriately narrow as they failed to include the impact of transfusions on eligibility for organ and transplant wait times, which are critical renal transplant outcomes that significantly affect the lives of chronic kidney disease (CKD) patients. Exposing transplant candidates to transfusions and their risk of allosensitization, may prolong organ wait time and/or preclude them from receiving a suitable organ, and thus, may relegate them to lifelong dependency on dialysis [1-3]. This can be even more pronounced among African Americans, who have a higher likelihood of being sensitized by transfusions and a lower probability of finding a suitable matching organ, which unnecessarily disadvantages their opportunity to receive a transplant [4-7].

[Response:] We understand the concern that the reviewer had. The key questions were posed by CMS and we were asked to answer them in a rigorous and unbiased manner. Although the outcomes of interest discussed above are valid points, they are tangential to the key questions that we are charged to answer. However, please understand that there are very important limitations to answering a key question when you need to use one data set to establish a link between an intervention and a surrogate outcome and another data set to show a link between the surrogate and the outcome as we discuss at length in the background section response. In addition, the assertion of increased likelihood of sensitization by transfusion in African American is not supported by current available data, in which the literature revealed that African Americans tended to be transfused inappropriately given that the indifference of their hemoglobin reference range to the standard range. Also, it has been reported from historical observational studies that worse renal transplant outcomes have been observed in African American although etiologies for such is unknown.

2. The technology assessment missed key publications from the peer review nephrology literature, and data from the US transplant registry, that are relevant to the use of therapeutic blood transfusions in CKD patients. Importantly, the US transplant registry captures data on the entire US wait-listed and transplanted patient populations. A selected list of these publications is provided in Appendix A.

[Response:] With the list of 27 publications provided by the reviewer, only three studies meet the inclusion criteria of the TA. [Alarif et al, 1987 (2), d’Apice et al, 1982 (7), Opelz et al, 2005 (19)] Please see Appendix A for the rationale of exclusion for the other 24 publications. The inclusion and exclusion criteria for the technology assessment were listed clearly in the report. The inclusion criteria allows only for controlled studies (both clinical and observational) published in peer reviewed literature to be included. Although we recognize the availability of US transplant registry data, we opted not to include such data in our evaluation since the data were not published as peer reviewed original controlled trial. Clearly, peer reviewed longitudinal cohort studies would meet the criteria.

3. The technology assessment failed to acknowledge the important distinction between transfusion for the purpose of immunomodulation (eg, donor-specific transfusions [DSTs]) and therapeutic transfusions for the management of chronic anemia in CKD [8]. The differences in the clinical purpose, sources of blood, and volume of blood make these specific types of transfusions distinct; consequently, outcomes associated with immunomodulation transfusion strategies are not generalizable to those associated with therapeutic transfusions. For the purpose of this technology assessment, combining the evidence for these distinct types of transfusions confounds the conclusions rendering them invalid as a basis for clinical decision making.

[Response:] We recognize the distinction between different types of transfusion, and their clinical purpose of immunomodulation. We now separate the analyses for DST and therapeutic transfusion in Key Question 1, and evaluate the impact of transfusion on renal allograft outcomes in the 2 subgroups of analyses. The results of the subgroup transfusion are not markedly different from our original analyses, and they do not change the overall conclusion of the TA. Please refer to Tables 14-19 of the final report for the results of these subgroup analyses.

Analysis limitations

4. Inappropriate weighting: All studies were weighted equally in the evaluation of the literature.

a) Due to the vote counting methodology employed, results from studies of markedly different sample sizes (eg, 37,000 versus 732) were weighted equally. This approach for summarizing evidence is prone to bias, considered the least robust method, and is not the recommended approach for systematic reviews [9].

[Response:] We understand the reviewer’s concern in this aspect. With the poor literature base available, this was the best we can do, and in fact, we did address this limitation in the discussion section of our draft report: “This approach has limitations because analyses of varying quality and sample size were evaluated together but it provides that only type of independent qualitative analyses that can be done on such a literature base.”

b) Results from studies in the pre- and post-cyclosporine era were weighted equally. The development of cyclosporine and the later development of multi-drug immunosuppressive regimens transformed transplant medicine. Given that the vast majority of studies in the technology assessment pre-date these advancements, the conclusions drawn in the technology assessment are not relevant to current medical practice.

[Response:] We account for the changes in practice by comparing the analyses in different time periods and display what happened in all of the available studies. It is clear that most of the data is older in nature, as illustrated in our report. We cannot do anything about the nature of the evidence we have to review. Amgen can help to fill in the research gap by funding future studies of reasonable quality that provides insight into these key questions that obviously is important to CMS. All we can do is evaluate the available literature in the best available manner and to report it with transparency and rate the strength of evidence as such. We hope Amgen understands that we are not saying that transfusions are beneficial in transplant. We believe that the available evidence is incredibly weak and that firm conclusions cannot really be drawn. However, we feel it is disingenuous to suggest that the data for harm from using transfusions is strong.

5. Selection bias: Transplant candidates who receive transfusions may become allosensitized and consequently be precluded from receiving a kidney transplant. This was most clearly demonstrated in studies of DSTs where up to 30% of patients became allosensitized to their donor and were not able to receive the donor organ [10-13]. Preclusion from transplant is a clinically relevant renal transplant outcome and the exclusion of these patients from the technology assessment analysis introduces a significant selection bias. The failure to acknowledge this inherent selection bias, and the failure to account for it in the evaluation of the evidence, undermines the validity of the conclusions drawn regarding the impact of transfusions on transplant outcomes.

[Response:] This is a valid concern, and a very important limitation to this data set. We tried to be quite transparent in the discussion section alerting people to this very large confounder. To account for this, we have now added evaluations on the impact of sensitization on eligibility for transplantation in transfused patients (see pages 69-73, Table 43). We found that a proportion of patients who were sensitized from transfusion were precluded from their planned kidney transplantation, and the graft outcomes, if any, for this particular population was unclear. Thus, we cannot be sure whether transfusions have a beneficial to neutral effect on outcomes in this subgroup of patients.

6. Factual errors: Errors were made reporting data from the original articles that contributed to misclassification of results and inaccurate conclusions.

[Response:] Given the limitations in the body of evidence, we evaluated and summarized the data sufficiently to show what the literature suggests and that the strength of evidence is low to insufficient. We appreciate the review that Amgen provided and looked at each carefully and made alterations where necessary. None of the “Factual Errors” or “Transparency Issues” in any way alters the conclusions of our Technology Assessment. We feel that with the number of studies included and the number of endpoints evaluated, that we are proud of our ability to accurately extract and categorize the data.

7. Lack of transparency: Insufficient detail was provided throughout the technology assessment, preventing reproducibility of the analyses.

[Response:] See above

As a result of these limitations, the technology assessment did not adequately address the stated objective and the conclusions drawn in the technology assessment are not fully and appropriately informed by the totality of the available evidence. The conclusions in the technology assessment contradict current practice and evidence-based clinical guidelines, and have the potential to adversely impact patient care.

Introduction and Background

In this section of the technology assessment, there are two major areas of concern. One relates to the source of the information cited as guiding the conceptualization of the research question. The second relates to statements that contradict current clinical viewpoints regarding the adverse consequences of transfusions on transplant outcomes.

The technology assessment’s review of the field of renal transplantation cites two sources:

  • Pharmacotherapy: Principles & Practice, 2nd Edition
  • The Organ Procurement and Transplant Network (OPTN) website.

There was no reference to authoritative textbooks of transplant medicine, kidney transplant medicine, or transfusion medicine. The textbook that the technology assessment did reference (Pharmacotherapy) and directly quotes (page 2), provides in subsequent paragraphs useful information regarding the assessment of pre-transplant immune risk factors of transplant recipients. The specific text reads as follows:

“To avoid acute or chronic rejection, assessment of pre-transplant immune risk factors of recipients plays an important role in the prevention of immune-mediated allograft injuries. Evaluation of the presence or absence of alloantibodies and T cell activities to HLA antigens plays a significant role in individualization of immunosuppressive therapy. Patients with a high panel level of reactive antibodies (PRA) have a greater risk of immune-mediated injuries to the transplanted allograft. The PRA test measures the recipient’s mismatches and pre-formed antibodies against 50 to 60 different individuals (not donor). If 25 cells react, it is considered 50% reactive (PRA of 50%). Patients with higher PRAs and pre-formed antibodies have lower long-term allograft survival.” (p.
943, Ch. 55 Solid Organ Transplantation, In: Pharmacotherapy: Principles &
Practice, 2nd Edition)

This text clearly states that allosensitization (PRA) is an important predictor of renal transplant outcomes. Thus, the impact of transfusions on allosensitization should have been considered when evaluating the impact of transfusions on renal transplant outcomes.

[Response:] We chose the textbook we did because we wanted to reference a text that would concisely present an overview of the topic in a manner understandable to many stakeholders including healthcare policymakers and patients. The introduction was to introduce the topic not to report the opinions of experts in the field that had written book chapters. That is not what systematic review is about. In addition, we were not charged with deriving the key questions. Key questions were provided to us by CMS. We were asked to answer the key questions that were asked in a transparent and scientifically rigorous manner. That being said, I think there can be a rigorous defense made by CMS for the key questions they posed. Their questions have an advantage of limiting the dataset to those studies that directly evaluate the link between transfusion and outcomes. There are very important limitations to looking at two steps which may or may not be related. Here we are referring to the link between transfusions and sensitization using one data set and then the link between people who are sensitized (whether by transfusion or a host of other reasons) and outcomes in another set of data. The AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews is clear that direct evidence is far superior to indirect evidence.

The second concern relates to the statements made in the “Evolution of transfusion in renal transplantation” section (p. 3). The technology assessment makes the following statement: “the practice of blood transfusions has been shifting back and forth for the last few decades.” This is not a proper characterization of the evidence. Therapeutic transfusions have well-known risks, including the development of allosensitization, and thus were curtailed once other therapeutic options became available for the treatment of chronic anemia in CKD patients [15, 57].

[Response:] We have removed this sentence from the background section.

Below is a listing of specific comments for consideration. Please note this list does not include a number of minor editorial or factual errors that were not included for brevity.

1. The second paragraph, last sentence on page 1 states that “The use of hemodialysis, transplant, transplantation wait list, or peritoneal dialysis in patients covered by CMS in 2006 is provided in Table 2.” A more complete presentation of the wait-listed population would have included data on the distribution of panel reactive antibody (PRA) levels, which are known to impact wait-times. Sensitized patients have longer wait times on the transplant list and remain on dialysis resulting in greater morbidity and mortality than in transplanted patients. The OPTN reports that the median transplant wait time for patients with a peak PRA of 0%-9% was 1,276 in 1999-2000 and 1,329 days in 2001 compared with 4,059 days and 3,448 days, respectively during those timeframes, for patients with a PRA of = 80%. Amgen recommends that the authors consider updating this table to include that information.

[Response:] The causes of sensitization in transplant candidates were not captured in OPTN/UNOS database (i.e. sensitized for any reason, not just from transfusion). The data you propose be used is not limited to those sensitized from transfusions but rather sensitized for any reason and as such, the ability directly extrapolate to outcomes is problematic. Our report does bring to light that issue in the discussion section.

2. Page 2, Allograft Rejection, Acute Rejection, first sentence states that “Acute rejection is a cell mediated process that generally occurs within 5 to 90 days after a transplant, although it can rarely occur after this time.” The source for this information does not conclude that acute rejection “rarely” occurs after 90 days. Amgen recommends removing “rarely” from this sentence.

[Response:] We removed the word “rarely” from the report to comply with this comment.

3. Page 2, Allograft Rejection, Acute Rejection, fifth sentence states that “Pretransplant assessment for the presence or absence of alloantibodies and T cell activities to HLA antigens is touted to reduce the risk of acute rejection.” The overwhelming evidence in transplant nephrology suggests that pre-formed alloreactive anti-HLA antibodies sufficient to cause a positive cross match are a contraindication to kidney transplantation (Patel and Terasaki P. NEJM 1969[60]; Bergentz et al 1970[70]). Therefore, Amgen recommends the sentence be modified to “Pretransplant assessment for the presence or absence of alloantibodies and T cell activities to HLA antigens reduces the risk of acute rejection.”

[Response:] We have not systematically evaluated this data set and rated the strength of evidence using an objective measure. The data you quote is from 1969 to 1970 and its applicability to contemporary practice cannot be determined. Does it still have the same importance as it did before induction, cyclosporine, etc? Therefore, we would rather not make a definitive statement such as the one Amgen would like.

Methods

The limitations of this technology assessment are classified as 1) design limitations and 2) analysis limitations. As a result of these limitations, the technology assessment did not adequately address the stated objective and the conclusions drawn in the technology assessment contradict current practice and evidence-based clinical guidelines, and have the potential to adversely impact patient care. Below, each of these limitations is summarized.

Design Limitations

The translation of the stated objective into the specification of the research question was inappropriately narrow and therefore did not permit a full evaluation of the effect of transfusions on transplant outcomes among transplant candidates. The clinically relevant underlying question is “should transplant candidates receive RBC transfusions for the management of chronic anemia?” To comprehensively address this question, the technology assessment needed to examine the effect of transfusions on eligibility for suitable organs and the time a transplant candidate spends on the wait list, in addition to an assessment of the impact of transfusions on graft survival among transplants that do occur. The technology assessment fails to acknowledge that waiting time for transplant and not receiving a transplant are transplant outcomes. To an individual patient, eligibility and wait-times for an organ are critically important because the longer a patient is on a transplant wait-list, the higher the likelihood the patient will die on dialysis rather than receive a transplant. Therefore, the scope of the question that the technology assessment evaluated was too narrow to adequately inform on the effect of transfusions on the range of relevant and important kidney transplant outcomes.

[Response:] We understand the concern that the reviewer had. The key questions were posed by CMS and we were asked to answer them in a rigorous and unbiased manner. Although the outcomes of interest discussed above are valid points, they are tangential to the key questions that we are charged to answer. However, please understand that there are very important limitations to answering a key question when you need to use one data set to establish a link between an intervention and a surrogate outcomes and another data set to show a link between the surrogate and the outcome as we discuss at length in the background section response.

The technology assessment missed key publications from the peer review nephrology literature, and data from the US transplant registry, that are relevant to the use of therapeutic blood transfusions in CKD patients. Importantly, the US transplant registry captures data on the entire US wait-listed and transplanted patient populations. A selected list of these publications is provided in Appendix A.

[Response:] With the list of 27 publications provided by the reviewer, only three studies meet the inclusion criteria of the TA. [Alarif et al, 1987 (2), d’Apice et al, 1982 (7), Opelz et al, 2005 (19)] Please see Appendix A for the rationale of exclusion for the other 24 publications. The inclusion and exclusion criteria for the technology assessment were listed clearly in the report. Although we recognize the availability of US transplant registry data, we opted not to include such data in our evaluation since the data were not published as an original controlled trial.

The technology assessment failed to acknowledge the important distinction between transfusion for the purpose of immunomodulation (eg, donor-specific transfusions [DSTs]) and therapeutic transfusions for the management of chronic anemia in CKD [8]. The differences in the clinical purpose, sources of blood, and volume of blood make these specific types of transfusions distinct; consequently, outcomes associated with immunomodulation transfusion strategies are not generalizable to those associated with therapeutic transfusions. For the purpose of this technology assessment, combining the evidence for these distinct types of transfusions confounds the conclusions rendering them invalid as a basis for clinical decision making.

[Response:] We recognize the distinction between different types of transfusion, and their clinical purpose of immunomodulation. We now separate the analyses for DST and therapeutic transfusion in Key Question 1, and evaluate the impact of transfusion on renal allograft outcomes in the 2 subgroups of analyses. The results of the subgroup transfusion are not markedly different from our original analyses, and they do not change the overall conclusion of the TA. Please refer to Tables 14-19 of the final report for the results of these subgroup analyses.

Analysis Limitations

When summarizing the evidence and drawing conclusions, the authors used a vote-counting methodology, that is, classifying studies as positive, neutral, or negative and summarizing the literature based on counting studies in each category. This is not a preferred approach [9, 66] even for qualitative systematic literature reviews because it gives equal weight to all studies irrespective of their sample size or the time period in which they were conducted. For example, a study of 100 patients would be considered equivalent to a study of 10,000 patients, and a study conducted in the 1980’s would be given the same weight as a study conducted in the 2000’s, even if medical practice has changed. Moreover, conclusions drawn from analyses using vote-counting can differ substantively from conclusions drawn using more robust methods, and the conclusions can become more misleading as the amount of evidence (the number of studies) increases [66], in contrast to other statistical methods.

[Response:] We understand the reviewer’s concern in this aspect. With the poor literature base available, this was the best we can do, and in fact, we did address this limitation in the discussion section of our draft report: “This approach has limitations because analyses of varying quality and sample size were evaluated together but it provides that only type of independent qualitative analyses that can be done on such a literature base.”

Below are two examples from the technology assessment that illustrate the problem of equal weighting of studies. The first occurs in the assessment of the multivariate analyses examining the affect of red blood cell (RBC) transfusions on renal graft outcomes and the potential impact by the number of transfusions, the number of units of blood, and/or the number of donors (Question 1bii). The following table summarizes the results provided in Table 23 (p. 32-34) of the technology assessment:

Table 1. Illustration of the bias introduced by vote counting: Summary of studies examining effect of transfusion on graft survival included in Table 23 of the technology assessment

 Sample Size (# of patients studied)Percent of total sample
Significantly worse graft survival11,24074%
Neutral effect (not statistically significant)3,81025%
Significantly improved graft survival1431%


The technology assessment summary of these results states the following: “Transfusions were not an independent predictor of rejection, graft survival or patient survival in either direction in a large number of analyses.” The methodology used to arrive at this conclusion is not scientifically valid when 74% of the evidence is conclusive in one direction, 1% is conclusive in the other direction, and yet the technology assessment concludes that there is no effect in either direction.

The second example occurs in the summary of the analyses evaluating “how useful are PRA assays in predicting sensitization, and renal transplant rejection/survival --- especially in the setting of question 2a?” (Question 2b). The summary states “The 1-year graft survival, where the direction of effect regardless of significance was assessed, for the lower PRA groups had higher graft survival in five of eight (62.5 percent)33,101,132,151 analyses and lower survival in three of eight (37.5 percent)12,29,153 analyses.” For the eight analyses used to evaluate this outcome
(Table 33 [p. 49-51] in the technology assessment), the total sample size of the analyses
showing statistically significant worse 1-year graft survival with higher PRA levels included ~37,000 patients (7 analyses) while the total sample size showing a trend towards better graft survival with higher PRA levels included 732 patients (1 analysis). Notably, in the technology assessment, the results of the Bucin et al (1988) study were misinterpreted and the results of the Opelz et al study (1972) were incorrectly reported from the original article. The technology assessment concluded in the discussion section that “lower PRA generally has a beneficial to neutral effect on outcomes”, which does not appear to incorporate the overwhelming evidence showing a beneficial effect of lower PRA levels.

Conclusions did not distinguish between the pre-cyclosporine and modern era

Prior to the advances in immunosuppression, acute rejection was a major obstacle and
contributed to a significantly lower 1-year graft survival compared to current survival rates (65% vs. 96%) [1, 67, 68]. Thus, in the 1980s, the potential for immunomodulation offered by transfusions may have outweighed the concerns about sensitization and preclusion from transplantation [7]. However, with the availability of better immunosuppressive regimens (eg, cyclosporine) and the significant improvement in early graft survival, any potential benefits of immunomodulating transfusions were no longer considered to outweigh their risks [15].

Consequently, transfusions are now avoided and this change in the clinical management of transplant candidates is described in the authoritative textbook on kidney transplant medicine, Medical Management of Kidney Transplantation:

“Blood transfusions: Early in the history of solid organ transplantation, the “transfusion effect” was observed by Opelz et al. (21) and others (22) when they demonstrated a benefit on graft outcome if preoperative blood transfusion were given in combination with immunosuppressive drugs or x-radiation (23,24). … over the years it has also become clear that blood transfusion may also induce sensitization, and by the late 1990s the previously noted beneficial “transfusion effect” had given way to a deleterious effect, with worsening graft survival associated with the greater numbers of transfusions in sensitized and nonsensitized patients (26).”

The technology assessment acknowledges the substantial changes in the outcomes of renal allograft outcomes following the introduction of cyclosporine in 1984, and further notes that practices changed after 1992 with the introduction of multi-drug immunosuppression regimens. However, less than 10% of the studies reviewed in the technology assessment report outcomes of transplanted patients treated in the current era. Importantly, the more recent data show no benefit for the use of therapeutic transfusions on renal graft survival.

[Response:] We could not pool data due to the poor and heterogeneous nature of the studies. It seems that Amgen is suggesting that we should not give a general overview for what the results are in a qualitative sense through counting. We believe that we would be left with just putting together one very large table with every study in it and not saying anything summative at all if we listened to this advice. This would be completely noninformative and believe strongly that our approach of summarizing what is there in a qualitative way with a correspondingly low to insufficient strength of evidence is the appropriate way to go.

We feel confident that looking at significant results and then looking at direction/magnitude of effect gives people a general feel for where the data is going. In so doing, we account for studies that might have been underpowered to show significant effects in another analysis. It seems disingenuous to us that some of the limitations are being pointed out because if we show that in 85-100% of studies that they are beneficial to neutral effects and the magnitude shows beneficial to neutral effects that slicing and dicing will not yield disparate results. We believe that is exactly what we found when we did our previous subgroup analyses and still show with our newly added subgroup analyses. It certainly will not change the quality of the data and strength of evidence for the analyses which are low to insufficient. We account for the changes in practice by comparing the analyses in different time periods and display what happened in all of the available studies. It is transparent that most of the data is older in nature, it is clearly in our report. We cannot do anything about the nature of the evidence we have to review but Amgen could have. Amgen had the ability to fund studies of reasonable quality that provided insight into these key questions that obviously is important to CMS. All we can do is evaluate the available literature in the best available manner and to report it with transparency and rate the strength of evidence as such. We hope Amgen understands that we are not saying that transfusions are beneficial in transplant. We believe that the available evidence is incredibly weak and that firm conclusions cannot really be drawn. However, we feel it is disingenuous to suggest that the data for harm from using transfusions is strong.

Preclusion from transplantation due to transfusion-related allosensitization (selection bias) confounds the results and conclusions

In the technology assessment’s review of the literature, many of the studies that examined the impact of transfusion on transplant outcomes also provided data on the patients who became allosensitized following transfusion and were precluded from receiving the donor organ. For example, in Akiyama et al. [1984; citation # 112], 12% of patients who received a DST became allosensitized and did not receive the donor kidney; in Glass et al., [1985, citation # 114], 20% of patients who received a DST became allosensitized and did not receive the donor kidney; and in Cochrum et al., [1981, citation # 20], 30% of patients who received a DST became allosensitized and did not receive the planned donor kidney. These were studies in the setting of DSTs, and thus it is reasonable to conclude that a similar or greater sensitization effect would apply to therapeutic transfusions.

Preclusion from transplant is a clinically relevant renal transplant outcome related to transfusion and the exclusion of these patients from the analysis introduces a significant selection bias.

Failure to include the outcomes of these patients in the technology assessment undermines the validity of the conclusions drawn regarding the impact of transfusions on transplant outcomes.

To illustrate this selection bias, consider a situation where 100 transplant candidates being evaluated for the impact of transfusions on their transplant outcomes, where 70 patients received a transplant and 30 did not due to allosensitization. The approach taken in the technology assessment was to examine the effect of transfusions only in the 70 patients who ultimately received the transplant, ignoring the impact of transfusions on the 30 patients who were never transplanted due to transfusion-related allosensitization. Since allosensitization is a determinant of who is selected for transplantation, excluding this consequence of transfusion exposure biases toward benefit the assessment of exposure to transfusions on transplant outcomes [69].

[Response:] This is a valid concern, and a very important limitation to this data set. We tried to be quite transparent in the discussion section alerting people to this very large confounder. To account for this, we have now added evaluations on the impact of sensitization on eligibility for transplantation in transfused patients (see pages 69-73, Table 43 of the final report). We found that a proportion of patients who were sensitized from transfusion were precluded from their planned kidney transplantation, and the graft outcomes, if any, for this particular population was unclear. Thus, we cannot be sure whether transfusions have a beneficial to neutral effect on outcomes in this subgroup of patients.

In addition to the above limitations, the search strategy differed between Medline and EMBASE in the technology assessment. Medline was used to identify all English and non-English studies. The EMBASE search strategy was designed to exclude English-language studies and non-trial data. In doing so, the English and non-English studies were not treated equally. Since EMBASE and Medline do not completely overlap in journal coverage, searching both using the same strategy is preferred.

[Response:] In a comprehensive search, we must balance precision and recall. We conducted a broad search in Medline and Cochrane Central databases. This, in addition to citation tracking, meets the criteria for a comprehensive search in the AHRQ CER Methods Guide. While we did do a targeted rather than broad search of EMBASE, remember that Cochrane Central includes most of the pertinent medical journals included in EMBASE. As such, a full exhaustive search of Cochrane Central and EMBASE would have reduced precision considerably without impacting recall in a realistic way. Amgen has not substantiated that an exhaustive search of EMBASE would have yielded a treasure trove of additional missing studies of high quality. We are confident that it would, in no way, have changed the strength of evidence, or the general conclusions since those conclusions are so firmly in the beneficial to neutral camp. This can be illustrated by the list of references that Amgen provided. We have evaluated all the references that were listed by Amgen in the Appendix, and have now included the eligible studies in our final report. The conclusion and strength of evidence of our report is not impacted by the addition of these studies.

Results

Given the substantial flaws in the methodology employed in this technology assessment, both in the design and analysis, the validity of the results and conclusions are questionable. Amgen requests the authors revisit the evidence in light of the comments provided. Beyond the design and analysis limitations that impact all results and conclusions, there are two additional general issues in the Results section: factual errors and lack of transparency.

[Response:] Given the limitations in the body of evidence, we evaluated and summarized the data sufficiently to show what the literature suggests and that the strength of evidence is low to insufficient. None of the “Factual Errors” or “Transparency Issues” in any way alters the conclusions of our Technology Assessment. We appreciate the review that Amgen provided and looked at each carefully and made alterations where necessary. We feel that with the number of studies included and the number of endpoints evaluated, that we are proud of our ability to accurately extract and categorize the data.

Factual errors

There are factual errors in the document that impact the conclusions drawn by the technology assessment. These include errors in the reporting of data from the original papers. For example, as described above, the results of the Opelz et al (1972)[35] study were reported incorrectly (graft survival at 12 months [%] among PRA < 5 reported as 36% and > 5 reported as 55%; however, the reverse is what is reported in the article) and the results of the Bucin et al (1988) study were misinterpreted (no data on PRA were reported in the paper, but were listed in the technology assessment showing no effect of higher PRA). The incorrect reporting of results in the technology assessment was used to support the conclusion that the evidence regarding the relationship between PRA and graft outcomes was mixed. In fact, seven of the eight studies evaluated in the technology assessment showed that higher PRA levels are associated with increased graft failure.

[Response:] The results of the Opelz et al (1972) study are now reported appropriately. Regarding the results of the Bucin et al (1988) study, even though the study did not specifically state that the presence of antibodies was assessed by PRA, we consulted our key informant and confirmed that the presence of antibodies described in table 1 of the study was some sort of PRA test (even though the specific of the test was not described). We have put in a footnote to account for the ambiguity present in the study and changed the terminology used in our table to reflect what was reported in the study.

Transparency: Insufficient detail was provided to enable reproducibility

In high-quality technology assessments, it is important for the peer-review process that the evaluation methods are sufficiently detailed and documented to enable appropriately trained and skilled individuals to follow the rationale and reproduce the results. For key questions 2a and 2b, specifically, the details of the search strategy including the search terms used and the databases queried were not provided. Moreover, throughout the document, tables which summarize directionality of results based on analyses within studies do not identify the source of each analysis.

[Response:] As specified in the method section, the search strategy was designed to answer both key questions 1 and 2b, and the search strategy is available in Appendix A. Since this technology assessment was commissioned to evaluate the transfusion effects on renal allograft outcomes, and the correlation of PRA to renal allograft outcomes in the transfused patients, the search terms we used should capture a majority of the studies, and we also performed citation tracking as well. In addition, our method section described clearly that key question 2a was intended to provide an overview of the use of PRA in renal transplant patients, and thus, it was not conducted systematically. References have now been added to each section to clarify the source of analyses.

Below is a listing of specific comments. Please note this list does not include a number of minor editorial or factual errors that were not included for brevity.

Question 1a (pages 8-18)

1. Multivariate Analysis, page 11, 2nd paragraph, 6th sentence states that “The other analyses found that prior transplantations were not independent predictors of graft survival in either direction.” There may be an error in the statement where “transplantations” should be replaced with “transfusions”.

[Response:] Thank you for pointing out the textual error. Change was made accordingly.

Question 1b.i (pages 19-21)

1. In Table 11 on page 20, “Insight into body of literature: Donor-specific transfusions (KQ 1bi)”: In the validity of studies column, there is a total of 11 studies. However, there are 12 studies covered in this section of the assessment. Eleven of the unique studies were included and referenced in the first sentence (Page 19). The Jin DC (1996)[71] reference is missing in the text, but is listed in Table 15 on page 21. Amgen recommends citing the specific studies aligned to the evidence presented in the appropriate tables for greater clarity.

[Response:] The eleven unique studies refer to the studies with a clearly defined experimental group and a control group that reported univariate results. We have added a heading in each section to clarify this. The final study is a multivariate analysis and would not be included in this section of the text. We have now provided more references in the final report.

2. The third sentence of the paragraph under Rejection (Page 19), states that “to be considered for this analysis, studies had to provide a p-value, 95% CI, or explicitly state whether or not statistical significance was achieved.” There were 11 studies reviewed in the assessment of question 1b.i. A review of these studies indicates that only 1 (Glass NR [1985][12]) meets these criteria; therefore, it is unclear why the others were included. Amgen recommends either removing those studies that do not meet these criteria or providing rationale for their inclusion.

[Response:] The statement quoted by the reviewer refers to the section which we reported that “three analyses found either a significant reduction in rejection or no significant effect associated with DST” (a sentence before the quoted sentence). The following publications are the 3 analyses that we reported in the report:

1. The study by Reed A et al, 1991 reported that significantly fewer patients in the DST group had a rejection episode (50%) in one year as compared to the patients in random transfusion group (75%) (p=0.0008) [Other rejection outcomes also show similar significant results] (p.383 of the publication)

2. Table 2 of the study by Casadei R et al, 1987 specified that the number of patients with rejection in DST group and non-DST group were not statistically significant (footnote of table 2 in the Casadei R, 1987 study).

3. Table 2 of the study by Jovicic S et al, 2010 reported the number of patients with acute rejection, the p-values were specified for each of the intergroup comparisons (DST versus No transfusion, DST versus random transfusion). All of them showed significant reduction of acute rejection in DST group. (footnote of table 2 in the Jovicic S et al, 2010 study)

Note: Unlike the study conducted by Jovicic S et al, 2010, the significance of the results for acute rejection reported in the study by Glass NR et al, 1985 (Table 6) provided the overall p-value of the intergroup comparison (i.e. DST versus Imuran+DST versus HLA-identical). Thus, we opted not to include the significance of the results in our analysis.

3. In Table 13 on page 21. “Impact of DST on graft and patient survival outcomes (KQ 1b)”: The specific studies and analyses evaluated in this section (Table 13) are not directly identified in the table, and hence, it is not possible to reproduce the results. Furthermore, it is unclear what types of transfusion were evaluated in each of these studies. Amgen recommends that additional information be provided to enable reproducibility of the analysis.

[Response:] The references for the specific studies have now been added to the main body of the report. All the analyses included in this section evaluated the impact of DST versus non-DST or any other types of transfusions.

Question 1b.ii (pages 22-35)

1. Key Question 1bii, page 22 asks “Is any such impact of red blood cell transfusions on renal transplant outcomes altered by the number of transfusions, the number of units of blood, and/or the number of donors?” The answer to this question and subsequent conclusion may be biased because of the variability in the way transfusion exposure was assessed and reported. The distinction between “transfusions” and “units of transfusion” are unclear as they may be referencing similar quantities, yet they have been categorized into 4 distinct groups for this analysis. This lack of clarity about the specific assessment of exposure to blood transfusions potentially confounds the assessment of the studies and the subsequent interpretations. Amgen recommends that this limitation be noted in the report.

[Response:] We understand the reviewer’s concern. While the definitions for “number of transfusions” and “units of transfusions” were poorly defined in the weak literature base, we classified analyses in the 2 categories based on the terminology that was used in the studies. We excluded the analyses that were ambiguous. Each analysis could only be included in either “number of transfusions” or “units of transfusion” evaluation.

2. Key Question 1bii, page 22, second paragraph, first sentence states that “Thirty-six unique studies were included in the evaluation of the impact of different number of transfusions on renal allograft outcomes.” As noted throughout this review, the technology assessment does not provide adequate detail to determine how the authors assigned each article and/or study to the groups. Amgen recommends that the authors provide sufficient information to ensure reproducibility.

[Response:] See response above

3. Graft Survival, page 23, Number of transfusions, paragraph 1, sentence 4 states that “None of the analyses found transfusion to have a significant negative impact on ... graft survival.” Data reported by Chavers et al (reference 121) contradict this statement. Chavers et al found that “the risk of graft failure was increased in LD and CAD in recipients who received > 5 pre-transplant transfusions”. Amgen recommends modifying the statement to read “Chavers et al found repeated transfusion to have a significant negative impact on graft survival.”

[Response:] Corrected. Please refer to the report for the changes.

4. Graft Survival, page 23, Number of transfusions, paragraph 2, sentence 2 states that “In all cases the 1-year data was the maximum duration of follow-up for graft survival.” There are a number of articles that report graft survival beyond 1 year. Feduska et al (reference 29) evaluated graft survival up to 5 years and Andrus et al (reference 120) evaluated graft survival up to 2 years. Amgen recommends that the statement be corrected.

[Response:] Corrected. Please refer to the report for the changes.

5. Patient Survival, page 25, Number of transfusions, paragraph 3 states that “Eight and seven analyses performed evaluations of the magnitude of different transfusion intensities (1-5, 5-10, or =10) compared to no transfusion on 1-year and maximum duration patient survival, respectively. All analyses found either a >10 percent increase or a small change within 10 percent in either direction in 1-year and maximum duration patient survival (Table 20). We concluded that there was a large beneficial to neutral effect of different number of transfusions versus no transfusion on patient survival and we graded the strength of the body of evidence as low.” Amgen does not agree with the conclusion because there are no data for 1-year patient survival and maximum duration patient survival for all 4 categories (1-5 vs. 0, 5-10 vs. 0, or =10 vs. 0). Amgen recommends that the statement be modified to reflect this insufficient evidence.

[Response:] There were analyses for different number of transfusions reported for 1-year and maximum duration patient survival, and the grading for the strength of evidence should remain the same. The confusion came from the mislabeled table number. Instead of Table 20, it should refer to Table 18 in the draft report (or Tables 31-32 in the final report). We also reorganized the tables in this section to increase the readability.

6. Table 22 and 23, pages 32-34 and Table 24, page 34 includes a number of references that are not listed on page 22 and in the reference section at the conclusion of the Assessment. The conclusions are based on an incomplete data set that discounts literature that suggests that pre-transplant transfusion is associated with poorer graft survival and poorer patient survival.

• The following publications were identified by the technology assessment but were not among the citations listed in the evaluation on page 22: Higgins RM (Table 22); Tang H, Park YH, Bunnarpradist S, Agarwal SK, Montagnino G, Poli F, Sautner T (Table 23); Herget-Rosenthal S, Agarwal S (Table 24). It is unclear whether the data from these papers were included in the analysis and the conclusions drawn. However, it appears that they were not included because the results in these articles contradict the conclusions.

[Response:] We separated the discussion into univariate and multivariate sections. Page 22 of the draft report includes citations that reported univariate analyses, so citations of the studies referred to by the reviewer would not appear there. It should be obvious to every unbiased observer, that we had no preconceived notions and performed a completely objective evaluation of the data. We accepted the key questions presented from CMS, assured that we understood what they were asking, identified an outside expert without financial conflict of interest, defined a priori the methods that we would use given our horizon scan, and then steadfastly adhered to the methods. The relevant citations you are interested in are in the Multivariate Analysis section as are their results.

• There are references published between 1994-2008 that were not used, that provide more contemporary information, and report that transfusions worsen graft survival (Park et al, Bunnarpradist et al, Sautner et al) [72-74] and patient survival (Herget-Rosenthal et al, Tang et al) [75, 76]. When drawing conclusions on unviariate analyses, it appears that the authors do not include univariate analyses that appear in articles that contain multivariate analyses. These omissions impact the conclusions drawn. For example, Herget-Rosenthal et al, 2003 [75] (not cited on page 22, or listed in the Reference list) is noted in Table 24 with multivariate analysis showing that transfusion of > 40 units worsens patient survival. In this article, a univariate analysis also notes this worsened patient survival. It is uncertain if this result is included in the Patient Survival section (page 25) where the AHRQ report states that transfusion has a small impact on patient survival. Sautner et al, 1994 [72] (not cited on page 22, or listed in the Reference list) is listed in Table 23 with multivariate analysis showing that 5-10 or > 10 pre-transplant transfusions increases risk of graft failure. However, the univariate analysis from this study also demonstrates an increased risk of 1 or >1 transfusion versus 0 transfusions (P = 0.002). Amgen recommends that the authors revisit this literature to ensure that all relevant univariate analyses are included.

[Response:] Although the univariate analysis included in Herget-Rosenthal et al [2003] reported the negative effect of >40 units transfusions, we decided not to include this result since the range used in the study did not fit into the predefined categories of the units of transfusions. While > 40 technically fits under >10, the group to whom it was being compared could have had a substantial number of patients with 10-39 transfusions and would therefore not meet our criteria. It should be apparent why we would not have wanted to do this when we decided on our approach to analyzing this data.

Regarding the study by Sautner et al [1994], Amgen seems to be referring to prior transplants rather than pretransplant transfusions. The data that was quoted by Amgen (p=0.002) could not be matched to any results in the study regarding transfusions. However, there is some data regarding blood transfusions in Table 1 as well. We understand that honest mistakes sometimes happen, especially with very large reports including over a hundred studies with many variables. They performed a 3X2 Chi squared analysis of the impact of blood transfusions and found a p-value of 0.01 (with no post-hoc 2X2 evaluations) for primary non-functioning grafts. This is not measuring overall graft failure but rather only one type of graft failure, namely primary non-functioning of the graft. As such, we did not originally include it in univariate analyses. However, in retrospect, we see some value in its inclusion and now add it in as requested. We had to generate our own post-hoc analyses to include it and the results came out as follows (0 versus 1-5 transfusions: p=0.627, 0 versus >5 transfusions: p=0.016, 1-5 versus >5: p=0.003, >5 versus <5: p<0.001). As far as the other studies quoted by Amgen (Park et al, Bunnarpradist et al, and Tang et al), none of them reported univariate analysis results on transfusion effect on survival outcomes.

In Tables 22 and 23, Chavers et al [77] is listed with multivariate analysis although it is uncertain if these are multivariate analyses. In Tables 22, 23, 24, most of the analyses types are not listed and are specified as NR, though the analysis types may be found in the articles (logistic regression, for example, in Higgins et al [78] and Sautner et al [72]) that casts doubt on the attention to detail in the report and the reported conclusions. Amgen recommends a re-examination of the statistical methods in the papers reported in tables 22, 23 and 24 followed by an update to the tables.

[Response:] Changes were made accordingly. Please refer to the final report.

Question 1b.iii (pages 35-37)

1. In Table 25 on page 37, “Insight into body of literature: Leukocyte-depleted blood (KQ 1bii), the second column “Rejection” it states that there are no Clinical Control Trials (CCTs) or Prospective Observational Studies (POBs). However, of the cited references evaluating graft rejection, one (67) is a CCT and one (135) is a POB. Amgen recommends that the authors include these references in Table 25.

[Response:] We classified reference 67 (Nubé et al, 1983) as a POBS. To be classified as a CCT, the authors need to be instrumental in the logical allocation of patients to the different treatment groups, either in a randomized or non-randomized design. Here, the authors were prospectively observing patients who were receiving protocol or random blood transfusions.

In reference 67 (Nubé et al, 1983), the authors discuss rejection only in reference to graft loss. They describe the number of grafts lost to irreversible rejection. There is no description of rejection episodes experienced in the entire patient population, regardless of graft loss. As a result, the precise number of rejection episodes is not reported. In reference 135 (Persijn et al, 1981), there is a similar situation where rejection is discussed but only in the context of graft loss. There is no accounting for rejection throughout the patient population.

2. In Table 25, on page 37, “Insight into body of literature: Leukocyte-depleted blood (KQ 1bii), under the third column, the Max time for graft survival in CCT should be noted as 2 years rather than 1 year. Amgen recommends this error be corrected.
In Table 36 of the final report, column 3, there is no specific time listed for the Max Time. For graft survival, the CCT listed provides graft survival for 1-year, as well as the Max time. The time frame for Max Time reported is variable in the included studies.

3. In Table 25, on page 37, “Insight into body of literature: Leukocyte-depleted blood (KQ 1bii): the POBS times for graft survival are incorrect as currently listed in the third column. They should be listed as follows: 2 year (1 study), 6 months (1 study), Max time (2 year; 2 studies). Also, in the fourth column of Table 25, the POBS only included one patient survival analysis with Max time (ie, study 137) rather than the two analyses currently listed. Amgen recommends correcting these errors.

[Response:] Number of analyses for each outcome was reported in Table 25 of the draft report (or Table 36 of the final report) rather than the number of studies. This was done to account for studies that included more than one analysis. The study by Nubé et al, 1983 (reference 67 of the draft report) reported 2 analyses on the impact of leukocyte-depleted blood transfusion versus no transfusion on 1-year and 2-year (max duration) graft survival, and 2 analyses on the impact of leukocyte-depleted blood transfusion versus therapeutic transfusion on 1-year and 2-year graft survival were also evaluated within the same study. For the POBS times for graft survival listed in the table, two POBS studies (references 67 and 135 of the draft report) reported three analyses on 1-year graft survival, and 3 analyses on max duration graft survival.

For the patient survival analysis, reference 67 provides two analyses, each of which provide data at a Max time followup (i.e. 2 years) only. There was no data provided at 1 year patient survival.

4. In Table 25, on page 37, the specific analyses in the 4 studies included in the table are not specified. Amgen recommends incorporating this information.

[Response:] The specific references utilized in answering Key Question 1biii are now detailed in the text of the report.

Question 1b.iv-v (pages 38-44)

1. Under Graft Survival section, page 39, second paragraph, fourth sentence states that “The conclusion was that regardless of the time period, transfusion has a beneficial to neutral effect on graft survival. We graded the strength of the body of evidence to be low.” As opposed to the earlier studies (pre-1984) none of the studies reviewed in the technology assessment for the period 1992 to present demonstrated significant benefit for the use of therapeutic transfusions on renal graft survival, which contradicts the authors’ conclusions.

[Response:] What we were trying to say is that every study from before 1984 to present found either a beneficial or neutral effect. We clearly stated in the sentence before the offending sentence that the trend was towards neutral but we now alter the concluding sentence to comply as follows: “The conclusion was that regardless of the time period, transfusion has either a beneficial or neutral effect on graft survival with a shifting away from beneficial to solidly neutral in more contemporary practice.” With this in mind, we also graded the strength of evidence for this conclusion to be low.

Question 2b (pages 47-53)

1. Key Question 2b, page 47, Rejection, first and second sentences state that “Two studies with two analyses evaluated the impact of PRA on graft rejection (Table 32). In both analyses, the risk of rejection was not significantly elevated for the higher PRA group but was qualitatively lower when lower PRA groups were compared with higher PRA groups.”

The authors reviewed two older studies, but did not consider a more recent study by Opelz et al [2005] of ~160,000 transplanted patients [32]. In this study, elevated PRA levels were associated with significantly reduced graft survival. Amgen recommends that the statement be expanded. For example, the authors may want to include the following statement: “A more recent study of ~160,000 transplant patients evaluated the impact of PRA on graft rejection that demonstrated the association between elevated PRAs and the risk of graft failure. Additionally, a meta-analysis concluded that patients with an elevated PRA at the time of transplant had shorter graft half-lives.” Amgen recommends this article be added to Table 32 and included in their analysis and subsequent conclusions.

[Response:] We are a bit confused with Amgen’s comment here. The specific section of the TA that they commented on was specifically evaluating the impact of PRA on graft rejection. While the Opelz et al [2005] study only reported outcomes on graft survival, the recommendation of adding a statement from Opelz et al [2005] study in this section would be inappropriate. The results of Opelz et al [2005] are included in the analysis for graft survival of key question 2b.

In addition, we are not sure which meta-analysis that Amgen is referring to in that statement, and the results of a meta-analysis should not be used in answering the key question since it is not an original controlled trial and would either duplicate studies already included or would allow studies not deemed to be included into our search strategy into the results.

2. Key Question 2b, page 47, Graft Survival, second and third paragraphs state that “The 1-year graft survival was significantly better with lower versus higher PRA levels in three of five (60.0 percent) analyses that assessed for significance and not significantly different in the other analyses. The 1-year graft survival, where the direction of effect regardless of significance was assessed, for the lower PRA groups had higher graft survival in five of eight (62.5 percent) analyses and lower survival in three of eight (37.5 percent) analyses.”

“The maximum duration graft survival was significantly better with lower versus higher PRA levels in one of nine (11.1 percent) analyses that assessed for significance and not significantly different in the other analyses. The maximum duration graft survival, where the direction of effect regardless of significance was assessed, for the lower PRA groups had higher graft survival in 11 of 14 (71.4 percent) analyses and lower survival in 3 of 14 [bd1] (28.6 percent) analyses.” This statement omits adjusted, pooled data (2002-2007) from the USRDS 2009 Annual Data Report (ADR) on transplant outcomes [57]. These data show, for deceased donor transplants, a 30% increased hazard of all-cause graft failure for transplant recipients with a pre-transplant PRA > 50% compared with transplant recipients with a pre-transplant PRA =50% (hazard ratio [HR] = 1.3; 95% CI 1.18-1.41; P < 0.0001). For living donor transplants, there was a 50% increased hazard of all-cause graft failure for transplant recipients with a pre-transplant PRA > 50% compared with transplant recipients with a pre-transplant PRA = 50% (HR = 1.5; 95% CI 1.25-1.74; P < 0.0001). For completeness, Amgen recommends that these data be included in the report.

[Response:] The data from USRDS 2009 Annual Data Report will not be incorporated into this key question as the technology assessment included only original peer reviewed studies. However, we have now incorporated this type of ancillary information in the discussion section of the final report. It should be noted that including 6 book chapters and an annual data report using the same data sets over similar time periods does not mean that there are multiple individual studies showing the same thing. I think that there needs to be greater attention paid to the overlap in the sources of the data so as not to triple or quadruple count.

Discussion/Conclusion

Discussion

The comments presented here summarize what has been included in previous sections.
Amgen notes that there is a substantial body of literature informing the field that was not consulted in this technology assessment. Our review identifies several serious limitations in the design and analysis of this technology assessment which undermine the validity of the conclusions in the technology assessment. These include:

  • The questions formulated to address the stated objective were inappropriately narrow as they failed to include the impact of transfusions on organ access and transplant wait time, critically important renal transplant outcomes.

[Response:] We did not formulate these key questions; rather, CMS posed the key questions and we were asked to answer them. We did so in a completely unbiased and transparent way and the level of transparency has been enhanced even more in this revised draft and appreciates Amgen’s review and their suggestions for improvement.

  • Key publications and data from the US Transplant Registry that are relevant to the use of therapeutic blood transfusion in CKD patients were not included
  • Inappropriate aggregation of evidence regarding the use of transfusions for immunomodualtion and transfusions for chronic anemia management
  • The vote counting methodology used in this assessment is recognized as among the weakest in quantitative methods, and weights all studies equally (regardless of sample size or medical era) in the evaluation of the evidence.
  • Failure to account for the selection bias due to transfusion-related allosensitization resulting in the preclusion from receiving a transplant, a critically important renal transplant outcome.

[Response:] In the previous sections above we defended the use of our methods. I believe Amgen fails to understand that we are not making this literature base out to be something that it is not. We gave the lowest strength of evidence ratings to these analyses because the literature base is poor. Pooling this data would have been completely inappropriate and summing data which would allow for larger studies to have greater weight without accounting for variance is even worse. We simply had a poor literature base and the result is a Technology Assessment that cannot confidently provide answers to the key questions posed. We did include a few of the studies that Amgen recommended that would have fit our inclusion and exclusion criteria and reconsidered a couple of choices we made for a couple of additional studies. We had already alerted readers to the huge confounder which is that allosensitization precluded some people from receiving a transplant including a statement in the conclusions. We have now have added two tables explicitly showing where and to what extent this occurred (Table 43 and Appendix D in the final report). We also include book chapter and annual report data in the discussion but could not add it to the methods and the results because of the reasons stated in those sections.

  • Factual errors

[Response:] Even in your review of our report, there were several factual errors in your comments. This is to be expected in such a large report with so many lower quality studies and we do not hold it against you. We have examined all of your suggested changes and made alterations where appropriate. Such factual errors do not impact the conclusions or the strength of evidence.

  • Lack of transparency preventing reproducibility of the analyses

[Response:] We were very transparent, using two different counted methods to allow the largest possible evaluation and writing a very lengthy report with numerous subgroup analyses and explicit tables. We understand that you wanted references for each outcome of the key questions and now provide them. Understand though that at a certain number of pages of text that the report becomes inaccessible to end users given its density and the overall conclusions can be lost. So we need to balance that in this revision.

Below is a listing of specific comments. Please note this list does not include a number of minor editorial or factual errors that were not included for brevity.

1. Paragraph 3, page 55: The technology assessment arrives at the following conclusion: “So the literature, weak as it is, supports a neutral to positive effect resulting from transfusion and does not support a detrimental effect resulting from transfusion of a larger number of transfusions.” This conclusion does not reflect the totality of evidence or appropriate analysis, and therefore lacks validity and contradicts evidence-based guidelines about the use of transfusions. This highlights the need for this technology assessment to be peer-reviewed by experts in transplant nephrology. Amgen disagrees with the statement that the evidence regarding the use of transfusions in kidney transplant candidates is weak because a large body of evidence which was not included (see list of citations) provide fair to good evidence that transfusions cause allosensitization, which effects access to available organs and prolongs time on the transplant wait-list, in addition to significantly increases graft failure.

[Response:] Amgen’s purported good evidence would not be rated highly using objective measures. Observational studies not published in the peer reviewed literature, where the funding sources are not explicitly reported, without demographic comparisons or explicit methods, and a lack of confounder control to not yield high strength of evidence. In addition, they want to use data that predominantly shows that transfusions increase sensitization and then use other data in people who were sensitized by any cause (one of which could be transfusion) that shows patient harm and then state that the literature is conclusive that transfusions cause harm.

2. Paragraph 4, page 55: Amgen disagrees with the statement that “lower PRA generally has a beneficial to neutral effect on outcomes.” The available evidence including the Opelz et al. (2005) [32] study of 160,000 patients (not included in the technology assessment) show that higher PRA levels are associated with significantly shorter graft survival compared to low PRA levels. The statement that the effect is neutral is not consistent with the preponderance of available evidence. Amgen recommends inclusion of this definitive study before finalizing the technology assessment.

[Response:] We included Opelz et al 2005 in graft survival.

Future Research Directions

Amgen comments on the two recommendations offered in the technology assessment.

1. Recommendation # 1 (page 55): “We believe that additional adequately powered studies should be conducted. In these studies we believe that they should be multi-institutional because individual center practices and procedures are so variable, have adequate reporting of demographics and either use statistical means to account for confounders (propensity score adjustment or matching) or use of randomization, have standard definitions of outcomes, and have a standard follow-up time of at least 1-year. Patients receiving or being randomized to no transfusions should be screened to assure that this not only includes transfusions within the dialysis or transplant center but other transfusions as well.”

In order to randomize a subject to a treatment with the potential for harm, several conditions must be met. Among these are equipoise, the presence of genuine uncertainty in the expert community as to whether a treatment is beneficial, and the potential for patient benefit from the proposed intervention. Both of these elements are lacking with respect to this question by the expert community practicing in this field.

[Response:] In order to answer this question with good quality evidence, adopting more of these techniques would lead you there. Continuing to generate observational data with little to no reported methods, no control for confounders, and little to no demographics will not get you there.

2. Recommendation #2 Page 55: “We believe that standard PRA testing should be supplanted with updated CPRA testing so that specific HLA antigen sensitivities resulting from transfusions can be identified and perhaps correlated with outcomes.”

The inclusion of this recommendation lacks relevance as it does not bear upon current clinical practice. The calculated PRA (CPRA) has been almost universally adopted since 2009 as noted in their reference 137. The specific antigen specificities detected by the CPRA are used to preclude transplantation of kidneys when antibodies to kidney’s antigens are found, also noted in reference 137. This is because it is well accepted that the presence of these specific antibodies results in a high frequency of acute rejection.

[Response:] We feel that Amgen misunderstands the point. Yes, everyone is doing CPRA now because it is so much better, but the available literature evaluating transfusions and final health outcomes did not use CPRA. As such, future studies should use the newer CPRA and PRA data would be of less value.

Conclusion

The technology assessment has numerous critical design and analysis limitations that undermine the validity of its conclusions. Importantly, these conclusions also contradict current evidence based guidelines and clinical practice. Therefore, Amgen strongly recommends the technology assessment receive peer-review by transplant nephrology experts prior to finalization. To let the current conclusions stand misinforms decision making bodies and is a true disservice to the affected population. When the totality of the evidence is considered with inclusion of all pertinent transplant outcomes, the modern of immunosuppressive medications, the differences between transfusions for immunomodulation and transfusions for anemia management, the conclusion is clear: the evidence does not support a beneficial impact of transfusions on transplant outcomes. Indeed, the evidence is to the contrary. Transfusions can cause allosensitization, which is associated with longer time on the wait-list, and can preclude transplantation, and transfusions are also associated with worse graft survival.

Finally, the overall conclusion of the technology assessment was:

“…transfusions generally have a beneficial to neutral effect on renal allograft outcomes.” (p. 55)

In light of limitations of the technology assessment and the additional important literature cited herein, Amgen strongly disagrees with this conclusion and believes that the evidence is clear that in the contemporary setting therapeutic transfusion is detrimental to renal transplant outcomes. Given the serious limitations of the draft technology assessment it should not be used to inform policy decisions.

[Response:] We believe that our conclusions are explicit and conservative. We clearly identify that there is low or insufficient evidence. This means that we have low confidence that the evidence reflects the true effect. Further research is likely to change our confidence in the estimate of effect and is likely to change the estimate. Ultimately, that is the point that Amgen is missing. We are not saying that the main result is that transfusions have beneficial to neutral effects on renal allograft outcomes but rather there is low to insufficient evidence from which to draw conclusions. Given the available evidence, it is hard to say that people who receive transfusions have worse renal allograft outcomes. While we were not charged with answering this key question, the literature suggests that it limits the number of allografts offered to patients using the older PRA system and that staying on dialysis longer is associated with untoward effects but I do not think you can honestly say the literature shows that those who receive transfusions and undergo transplantation in the studies that have been conducted show worse outcomes.

Tables

There are a number of factual inaccuracies in the tables of the technology assessment. For illustrative purposes, please find below a sample of these inaccuracies.

1. On page 10 of the technology assessment, under Key Question 1a. Specific references are not listed for many of the studies in Tables 9; additionally, there some of the studies seem to be mis-categorized or omitted. For example, Tang H (2008) appears in the multivariate tables; the univariate analysis in the study by Tang H (2008) shows significant decreases in graft survival associated with 6 or greater transfusions, yet no papers in the univariate analyses are categorized as showing a significant decrease in graft survival associated with increased use of transfusions (Table7).

[Response:] We are confused with the reviewer’s comments, as there is no mention of Table 9 on page 10. We listed all the first authors’ last names and publication years for the included studies in Table 9. As far as we could tell, there was no univariate analysis reported in the study by Tang H et al [2008], and therefore the reviewer’s comments regarding Table 7 are not applicable.

2. In Table 8 on page 14, from the citation Reed A (1991), the sample size of “N=127” is for the full study, not the subset in the multivariate analysis. Further, Amgen believes the N should be reported 119, not 127 as stated in the technology assessment.

[Response:] The N represented in Table 8 is indicative of the number of patients assessed in the analysis of transfusion and its effects. As a result, we feel the accurate N for the above cited study is 127.

3. In Table 9 on page 15, it was not entirely clear why the citation Jin (1996), which appears in Table 15 on DSTs and transfusion does not also appear on Table 9. Because there is a covariate “blood transfusion” in that multivariate analysis in addition to the covariate “DST” that is used in table 15.

[Response:] In Jin, 1996, the precise number of patients receiving non-specific transfusions within the non-DST group is not reported. As a result, the data regarding these transfusions were not used in alternate analyses.

4. In Table 9 on Page 15, the column heading describing the outcome evaluated is “graft survival”, but the presentation of data from the studies is not consistent with respect to the outcome evaluated, and is confusing. Further, the authors should evaluate if the relative risk estimates were interpreted correctly from the original studies.

[Response:] We listed the outcome evaluated as the manner in which the data were presented in the original study. The reported significant effect is derived from the original study.

5. In Table 9 on page 15, Amgen believes t is not be appropriate to consider the population in the citation Peters TG (1995) as a “separate analysis”, given that the 2 subsequent analyses presented are derived from the same study population.

[Response:] We agree, and have removed the two sub-analyses from the table. Only the results from the overall multivariate analysis are included.

6. In Table 9 on page 17, the analysis type on the Sautner T (1994) study should be “stepwise logistic regression” rather than not reported (NR) as presently stated.

[Response:] We have changed the analysis type as suggested.

7. In Table 15 on page 21, under the heading of “Multivariate Analysis” It is noteworthy that significance was determined at an alpha of 6% rather than the conventional 5% level in the one analysis that suggested DST as being an independent predictor in benefiting graft survival (Sanfilippo [1990]).

8. In Table 22 on page 32, the Higgins (2004) citation should have its analysis type identified as “multiple logistic regression”, as opposed to “not reported” (NR)

[Response:] We have changed the analysis type as suggested.

9. In table 22, on page 32, in Chavers BM (1997) they state “Proportional Hazards Regression” but it would be more appropriate to be more specific regarding the analysis.

[Response:] The authors do not provide any further detail regarding the analysis methods used.

10. In table 23, on page 32, in Tang H (2008) graft survival should be changed to graft failure in the 3 instances that occur in the table

[Response:] We use graft survival as this is how the study authors report the data in their tables.

11. On page 47, under the heading of Graft Survival, in the first sentence, citation #154 should have been included.

[Response:] Citation included

12. On page 47, under the heading of Graft Survival, in the third paragraph, “the maximum duration graft survival was significantly better with lower versus higher PRA levels in one of nine” should be “the maximum duration graft survival was significantly better with lower versus higher PRA levels in two of ten”. The percentage of studies should be changed to 20% from the current 11.1%. Citation #152 should have been included in the body of this text, and citation 101 should have been cited at the end of the sentence.

[Response:] Citations were updated accordingly

The following sentence, “….for the lower PRA groups had higher graft survival in 11 of 14” should be corrected to: “for the lower PRA groups had higher graft survival in 12 of 14”. Hence, the percentage of studies should now change from 71.4% to 85.7%. The citation #153 should be included in this sentence. In the same paragraph, “analyses and lower survival in 3 of 14 (28.6 percent)” should be corrected to: “analyses and lower survival in 2 of 14 (14.3 percent)”. Citation #153 should not be cited in this sentence.

[Response:] Citations were updated accordingly

13. In Table 33 on page 50, for Bucin (1988a), “N=116” should be “N=79”, updated based on Table 1.

[Response:] The N represented in Table 33 is indicative of the number of study population. As a result, we feel the accurate N for the above cited study is 116.

14. In Table 33 on page 50, for Garvin, 1983a, “N=118” should be “N=37”. Black cohort, only 37 patients had PRA available (Table 2 in the paper).

[Response:] The N represented in Table 33 is indicative of the number of study population. As a result, we feel the accurate N for the above cited study is 118.

15. On page 51, for Opelz (1973), under Graft survival “P=NR” (not reported) should be correctly stated as “P=0.001”. The sentence in the Results column should be corrected by deleting the words “with no statistical analysis”. It would correctly read as: “Graft survival better for PRA <5”.

[Response:] Change was made accordingly.

Figures

Not applicable

Appendix A (from Amgen’s Comments)

[Response:] Of the following list of references, only 3 publications meet the inclusion criteria of the report (in bold), the rationales for exclusion are listed below for each of the citation.

1. Ahmed Z and T. PI, Effect of transfusions, in Clinical Transplants., P. Terasaki., Editor. 1991, UCLA Tissue Typing Laboratory: Los Angeles. p. 305-312.

[Response:] This publication included data from the UCLA database, and we were not confident that the results of this publication included a unique patient population since we suspected a potential overlap with one of the publications included in the TA (Takiff H et al, 1988). Since the study by Takiff H et al was published in a peer-review journal with more robust methodology, it was included in our TA instead of the study by Ahmed et al.

2. Alarif, L., et al., Influence of pretransplant antibodies on early renal allograft rejection. Transplant Proc, 1987. 19(1 Pt 1): p. 771-2.

[Response:] Added to the final report. Please refer to section key question 2b.

3. Cecka, J.M. and L. Cho, Sensitization, in Clinical Transplants, P. Terasaki, Editor. 1988, UCLA Tissue Typing Laboratory. p. 365-373.

[Response:] The concern we had in including publications from book chapters was that many of these publications may have included overlapping population from other publications. The vast majority of the book chapters did not provide sufficient information on the methodology (i.e. no information on patient demographic, study design, use of controlled group) to allow us distinguish one publication from the other. In order to avoid duplicate reporting, we opted to exclude book chapters. The publication suggested by the reviewer above (Ahmed et al) serves as an example of our concerns.

4. Cheigh, J.S., et al., Declining transplantability of prospective kidney transplant recipients. JAMA, 1981. 246(2): p. 135-9.

[Response:] This is listed in our original search. It was excluded due to no outcomes of interest for the KQs. (Note: Reported transfusion effect on sensitization, but no graft outcomes reported)

5. Collins, A.J., R.N. Foley, and C. Herzog,. US Renal Data System 2010 Annual Data Report: Transplantation Am J Kidney Dis, 2011. 57(1 Suppl 1): p. e311-e324.

[Response:] Since the publication from the UNOS database is not a peer reviewed study, and had insufficient description of methodology, it did not meet our inclusion criteria. However, we have now included the information in the discussion section of the report.

6. Collins, A.J., et al., US Renal Data System 2010 Annual Data Report. Am J Kidney Dis, 2010. 57(1 Suppl 1): p. S281-S294.

[Response:] Please refer to comments above (#5).

7. d'Apice, A.J. and B.D. Tait, An elective transfusion policy: sensitization rates, patient transplantability, and transplant outcome. Transplantation, 1982. 33(2): p. 191-5.

[Response:] Added to the final report. Please refer to section key question 2b.

8. Hardy, S., S.-H. Lee, and P.I. Terasaki, Sensitization 2001., C.a.T.E. Clinical Transplants 2001, UCLA Immunogenetics Center, UCLA, Editor. 2001.

[Response:] Book chapter. See above (#3)

9. Iwaki, Y. and P.I. Terasaki, Sensitization effect, in Clinical Transplants, P. Terasaki, Editor. 1986, UCLA Tissue Typing Center: Los Angeles. p. 257-265.

[Response:] Book chapter. See above (#3)

10. Jayes, R.L., Jr. and A.S. Levey, Donor specific transfusions or cyclosporinee for related-donor kidney transplantation? Kidney Int, 1989. 36(3): p. 485-96.

[Response:] It was listed in our original search. It was excluded since there were no outcomes of interest. The study is a decision analysis comparing the decision of DST versus cyclosporine use.

11. Karpinski, M., et al., Leukocyte reduction of red blood cell transfusions does not decrease allosensitization rates in potential kidney transplant candidates. J Am Soc Nephrol, 2004. 15(3): p. 818-24.

[Response:] It was listed in our original search. It was excluded due to no outcomes of interest for the KQs. (Note: Reported transfusion effect on sensitization, but no graft outcomes reported)

12. Katznelson, S., S. Bhaduri, and J.M. Cecka, Clinical aspects of sensitization in Clinical Transplants, J.M. Cecka and P. Terasaki, Editors. 1997, UCLA Tissue Typing Laboratory: Los Angeles.

[Response:] Book Chapter. See above (#3)

13. Lefaucheur, C., et al., Preexisting donor-specific HLA antibodies predict outcome in kidney transplantation. J Am Soc Nephrol, 2010. 21(8): p. 1398-406.

[Response:] This study did not evaluate the direct effect of transfusion on graft outcomes, and thus it was not included.

14. Lefaucheur, C., et al., Clinical relevance of preformed HLA donor-specific antibodies in kidney transplantation. Am J Transplant, 2008. 8(2): p. 324-31.

[Response:] This study did not evaluate the direct effect of transfusion on graft outcomes, and thus it was not included.

15. Leone, M.R., et al., A comparison of 2 protocols for living-related renal transplantation in children: donor-specific transfusions versus cyclosporinee. J Urol, 1990. 144(3): p. 721-3.

[Response:] It was listed in our original search. It was excluded because the study groups were not in direct comparison to each other (DST versus cyclosporine).

16. Mackie, F., Donor-specific transfusions. Nephrology, 2010. 15: p. S101-S105.

[Response:] It is excluded because it is a guideline, not a controlled trial.

17. Meier-Kriesche, H.U. and B. Kaplan, Waiting time on dialysis as the strongest modifiable risk factor for renal transplant outcomes: a paired donor kidney analysis. Transplantation, 2002. 74(10): p. 1377-81.

[Response:] Transfusion effect on graft outcomes was not evaluated in the study. Outcomes (i.e. waiting time on dialysis) reported in the study were tangential to the key questions of the TA.

18. Opelz, G., Improved kidney graft survival in nontransfused recipients. Transplant Proc, 1987. 19(1 Pt 1): p. 149-52.

[Response:] We were not confident that the population included in the study consisted of an unique dataset, and there was a suspected overlap of population with other publications from the Collaborative Transplant Study that were included in the technology assessment.

19. Opelz, G., Non-HLA transplantation immunity revealed by lymphocytotoxic antibodies. Lancet, 2005. 365(9470): p. 1570-6.

[Response:] Added to the final report. Please refer to section key question 2b.

20. Otsuka, M., et al., Long-term results of donor-specific blood transfusion with cyclosporinee in living related kidney transplantation. Nephron, 2001. 88(2): p. 144-8.

[Response:] It was listed in our original search, but it was excluded because the type of transfusion evaluated in the study (i.e. buffy coat DST) was not red cell component transfusion.

21. Raftery, M.J., et al., Controlled trial of azathioprine and cyclosporine to prevent anti-HLA antibodies due to third-party transfusion. Nephrol Dial Transplant, 1988. 3(5): p. 671-6.

[Response:] This publication did not meet the inclusion criteria because it did not have controlled group comparison.

22. Scornik, J.C., et al., Assessment of the risk for broad sensitization by blood transfusions. Transplantation, 1984. 37(3): p. 249-53.

[Response:] It was listed in our original search, but was excluded due to no outcomes of interest. (Reported transfusion effect on sensitization only, no graft outcomes reported)

23. Keogan, M., et al., Causes of sensitisation in patients awaiting renal transplantation in Ireland. Ir Med J., 2003. 96(4): p. 109-12.

[Response:] It was excluded because the outcomes reported in the study (i.e. effect of sensitization on transplant waiting time) were not of interest in answering the key questions of the technology assessment.

24. van den Berg-Loonen, P.M., et al., B cell antibodies after planned transfusions. Transplant Proc, 1987. 19(1 Pt 1): p. 743.

[Response:] There is no comparison between transfusion and no transfusion.

25. Vella, J.P., et al., Sensitization to human leukocyte antigen before and after the introduction of erythropoietin. Nephrol Dial Transplant, 1998. 13(8): p. 2027-32.

[Response:] It was listed in the original search, but it was excluded since it did not evaluate the direct link between transfusion effect, sensitization rate and allograft outcomes.

26. Wolfe, R.A., et al., Differences in access to cadaveric renal transplantation in the United States. Am J Kidney Dis, 2000. 36(5): p. 1025-33.

[Response:] It was excluded because the outcomes reported in the study (i.e. factors affecting access to transplantation) were not of interest in answering the key questions of the technology assessment.

27. Zou, S., et al., Prevalence, incidence, and residual risk of human immunodeficiency virus and hepatitis C virus infections among United States blood donors since the introduction of nucleic acid testing. Transfusion, 2010. 50(7): p. 1495-504.

[Response:] The focus of the study was not relevant to kidney transplantation, and none of the outcomes reported was of the interest to the topic. Thus, it was excluded.

Return to Contents

 

Page last reviewed August 2014
Internet Citation: Public Review Comments Appendix: Response to Amgen: Disposition of Comments. August 2014. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/findings/ta/renal-transplant/appendix.html