Methods

The Center for Medicare Management of the Centers for Medicare and Medicaid Services (CMS) requested this report from The Technology Assessment Program (TAP) at the Agency for Healthcare Research and Quality (AHRQ). AHRQ assigned this report to the following Evidence-based Practice Center: ECRI Institute EPC (Contract Number: 290-2007-10063).The purpose of this review is to provide information to CMS for consideration in HCPCS coding decisions. The review will facilitate CMS' evaluation of HCPCS coding for NPWT devices by providing CMS with relevant studies and information for consideration of coding changes, as required by the MIPPA legislation. CMS will use this review in its assessment of whether existing HCPCS codes adequately represent the technology and comparative benefits of NPWT devices.

The EPC Program of AHRQ contracts with organizations to perform scientific reviews of a variety of topics. The ECRI Institute EPC is one of four EPCs with a focus on assessments for CMS.

The process of systematic review as practiced by the EPC Program follows specific prescribed steps:

1. The investigators start with formulated "key" questions. These questions test hypotheses and are structured using the "PICO" framework: patients, intervention of interest, comparator, and outcomes. EPC are encouraged to focus on outcomes that are relevant and important to patients (patient-oriented outcomes). The framework is depicted visually in the "analytic framework" that the EPC program uses to show the relationship between the key questions and the outcomes used to address these questions. (Go to Figure 3)
2. Inclusion and exclusion criteria for studies to be used in the review are determined based on the specific questions to be addressed. Criteria may vary for each question in the review.
3. Next, an objective and comprehensive search of the medical literature and "gray literature," (i.e., reports, monographs and studies produced by government agencies, educational facilities and corporations that do not appear in the peer-reviewed literature) is conducted. The reference lists of included studies are examined for any studies not identified by electronic searches.
4. Studies are compared to the inclusion criteria developed prior to examining the evidence, and those included in the review are then critically appraised, noting features of the design and conduct of the studies that create potential for bias. Bias, in this context, is a study feature that could impact whether the treatment being studied is responsible for the outcomes observed. Studies with a low potential for bias are typically described as being of "high quality," whereas those with high potential for bias are described as being of "low" or "poor" quality, and those of moderate quality as having intermediate potential for bias. The degree to which a study protects against bias is referred to as "internal validity." Following this appraisal, data are extracted from the included studies and analyzed or summarized as appropriate.

The following is a detailed explanation of the methods followed in this review.

Key Questions

1. After receiving the work assignment for this review in December, 2008, we developed the following Key Questions Does any single NPWT system have a significant therapeutic distinction in terms of wound healing outcomes compared to any other NPWT system for the treatment of acute or chronic wounds?
2. Does any component of a NPWT system have a significant therapeutic distinction in terms of wound healing outcomes compared to any other similar component of a NPWT system for the treatment of acute or chronic wounds?
3. What are the reported occurrences of pain, bleeding, infection, other complications, and mortality for NPWT systems?
4. Do patients being treated with one NPWT system have a significant therapeutic distinction in terms of less pain, bleeding, infection, other complications, or mortality than other NPWT systems?

For the purpose of addressing these Key Questions we will use the following definitions:

• Any NPWT system or component commercially marketed within the past 20 years will be considered in this report. Restricting inclusion to currently commercially available NPWT systems would eliminate research performed with devices that have been discontinued as new models replaced them. In-house developed/produced/created/built devices (what might be considered "home-made" negative pressure devices) were excluded.
• Components of a NPWT system include the pump, the tubing, the dressing kits, and the services (education, clinical care, special treatment protocols, staff intervention, clinical support, etc.) provided as part of the NPWT system.
• Acute wounds: present for ≤30 days.
• Chronic wounds: present for >30 days.

This report also provides an overview of the clinical research evaluating NPWT systems. However, this report does not address whether NPWT systems provide a better wound care alternative compared to non-NPWT wound care therapies.

Return to Contents

Analytic Framework

The analytic framework below (Figure 3) graphically depicts the events that individuals with chronic or acute wounds experience as they are treated with negative pressure wound therapy. This figure portrays the pathway of events that patients experience, starting from when they are first identified (the far left of the figure), to the treatments they receive, and to patient-oriented outcomes. As such, patients in the population of interest are identified and "enter" the pathway at the left of the figure. Each of the questions is represented in the framework by a circled number.

According to a guidance document prepared by the FDA in 2006, clinical outcomes associated with the use of a wound-treatment product or device can be broadly grouped into two categories—improved wound healing and improved wound care.(61) A number of outcomes or endpoints fall into these two categories. The most important outcomes to consider under the category of improved wound healing are percent of patients with complete wound closure and time to complete wound healing. The FDA defines complete wound healing as skin closure without drainage or dressing requirements.(61) Facilitation of surgical wound closure by partial healing is also a clinically important measure of improved wound healing.

Improvements in wound care can potentially reduce the occurrence of conditions, such as infection, that can interfere with proper wound healing.(61) Thus, measuring the impact of NPWT on the occurrence or healing of infections, as well as its impact on the incidence of other problems, such as sepsis, edema, or amputation, is important. We consider all of these outcomes in our evaluation of the evidence and the claim of significant therapeutic distinction. In addition to these outcomes, we consider other outcomes important to patients, such as quality of life, satisfaction with treatment, duration of treatment, and survival. (Note: process indicators such as improved compliance, convenience and personal preference (and patient-oriented outcomes such as quality of life or satisfaction with treatment) are considered by CMS to be significant distinctions only to the extent that they result in demonstrably improved clinical outcomes.)

In some cases, wound healing technologies are not expected to result in complete wound closure. Rather, the treatment may be intended to advance the wound to a stage where healing is possible. We consider these goals to represent intermediate treatment outcomes. If the overall treatment strategy is successful, the benefit of these intermediate treatment outcomes will be reflected in improved rates of complete healing. Intermediate outcome states are represented by the following outcomes: time to 50% reduction of wound volume, percent change in wound volume, and improved wound condition. Outcome assessment should also include measurement of adverse events that result from the treatment or natural history of the disorder. We consider adverse events in Key Questions 3 and 4. The adverse events include: pain, bleeding, infection/bacterial load, mortality, and other complications.

Figure 3 . Analytic Framework

[D] Select for Text Description

Notes:

• In keeping with guidance provided by the FDA, in this report we considered improved wound healing and improved wound care to be the most important clinical outcomes. The most important outcomes to consider under the category of improved wound healing are percent of patients with complete wound closure and time to complete healing (particularly when NPWT is used to prepare the wound for surgical closure). Please note: process indicators such as improved compliance, convenience and personal preference (and patient oriented outcomes such as quality of life or satisfaction with treatment) are considered by CMS to be significant distinctions only to the extent that they result in demonstrably improved clinical outcomes.
• Improved wound condition as presented in the Analytic Framework is defined as a reduction in wound exudate and infectious materials; the promotion of granulation tissue formation and perfusion; an improvement in graft appearance; a reduction in odor; and a greater rate of epithelialization.

Return to Contents

Inclusion Criteria

We used the following criteria to determine which studies identified by our searches and submitted by invited manufacturers and professional organizations would be included in our analysis. These criteria were developed prior to any review of the clinical literature or materials sent by interested parties. Inclusion and exclusion criteria were developed to specify the types of studies appropriate for addressing each of the Key Questions.

Population

1. Results for patients with different wound etiologies (diabetic ulcers, pressure wounds, vascular ulcers, surgical wounds, trauma wounds, etc.) must be reported separately.
   Time to heal and the frequency and characteristics of adverse events can be expected to vary depending on the underlying cause of the wounds.
2. Study must have enrolled human subjects.
   Studies of animals are outside the scope of this assessment. Evidence-based reports, for the purpose of policy or clinical decision making, rarely rely on non-clinical evidence (studies using animals, cell culture, cadavers, etc.) to address the effectiveness of treatments. While animal studies may lead to important discoveries that ultimately prove valuable in human applications, experts have cautioned that fewer than a third of highly-cited animal studies translate into human RCTs showing the same results of treatment.(62) Animal studies also seldom use study design procedures such as randomization, concealment of allocation, and blinding of outcome assessment that would limit the potential for bias.(63) Publication bias, the preferential publication of studies with positive results, may be especially common with animal studies.(64) In addition, positive results in animal studies may not translate well to the clinical setting. Investigators can control the severity of the wound in animals to a greater extent than in human studies. Animal subjects are likely to be younger and healthier than humans with wounds. Animals in such studies may not have co-morbid health conditions or exposure to concurrent medical interventions, in contrast to human subjects with wounds. An additional problem with animal studies of wound healing is determining which of the human wound etiologies (pressure ulcers, diabetic foot ulcers, venous leg ulcers, burns, sternal wound infections, or trauma-induced wounds) the animal model represents.

Intervention

3. Study must evaluate the efficacy and/or safety of a NPWT system or components of an NPWT system commercially marketed within the past 20 years.
   In-house developed/produced/created/built devices (what might be considered "home-made" negative pressure devices) are outside the scope of this assessment.

Study Design

4. Studies must have included five or more patients per treatment group..
   The results of smaller studies and especially case reports are often not applicable to the general population.
5. For Key Question 1, 2, and 4, study must have been a controlled study comparing one NPWT system or components of a system to another NPWT system or components..
   Randomization to a NPWT system group was not required. For Key Question 2, studies were required that compared different dressing sets, tubing, or pumps while maintaining identical components for the other parts of an NPWT system. In other words, both groups in the study would need to be receiving NPWT. For Key Question 3, no control group was required, because the focus of the question was simply to identify adverse events rather than compare rates across systems or components. However, because of the potential for bias in case series studies, no analyses were performed using adverse event data from these studies.
6. If a study employed a cross-over design, data from the second half of the study were excluded.
   Because there may be a lingering treatment effect from the first treatment applied, we exclude data from the second half of cross-over trials. Studies that did not report data from the two different periods separately were excluded.

Outcomes

7. The reliability and validity of all instruments measuring relevant outcomes such as quality of life or pain must have been addressed in the published literature..
   However, if a study did not use a validated instrument, then the entire study was not necessarily excluded for all outcomes—only its data from instruments in which the psychometric properties were not reported in the published literature were excluded.
8. Study must have reported on at least one of the outcomes of interest for one or more of the Key Questions.
9. For all outcomes, we only considered time points for which at least 50% of the enrolled participants contributed data.

Publication Type

10. Study must have been published in English..
    Moher et al. have demonstrated that exclusion of non-English language studies from meta-analyses has little impact on the conclusions drawn.(65) Juni et al. found that non-English studies typically were of lower methodological quality and that excluding them had little effect on effect size estimates in the majority of meta-analyses they examined.(66) Although we recognize that in some situations exclusion of non-English studies could lead to bias, we believe that the few instances in which this may occur do not justify the time and cost typically necessary for translation of studies to identify those of acceptable quality for inclusion in our reviews.
11. Study was reported as a full-length article. Abstracts were only considered if they contained new or previously unreported data from a published full article..
    Published abstracts and letters alone do not include sufficient details about experimental methods to permit verification and evaluation of study design.(67,68) We only included data from an abstract if it reported additional outcomes from a study and patient group that had been reported in a full-length article that met all inclusion criteria.(69) For this report, publication also includes non-confidential transmission of a study report to ECRI Institute or non-confidential information on a study with sufficient detail to permit an evaluation of the study.
12. When several sequential reports from the same study center were available, we included outcome data from only the largest, most recent or most complete report. However, we used relevant data from earlier and smaller reports if the report presented pertinent data not presented in the larger, more recent report..
    This criterion prevents double-counting of patients.

Table 13 in Appendix A lists the reasons for exclusion for all excluded studies.

Return to Contents

Search Strategy

To identify relevant information on the benefits and harms of NPWT systems, we employed the following search strategies:

• Systematic search of 13 external and internal electronic databases, including CINAHL, EMBASE, and MEDLINE from 1950 (MEDLINE)/1980 (EMBASE)/1982 (CINAHL) to the present for fully published primary clinical studies. A detailed search strategy is presented in Appendix A. Articles were retrieved for further review if they 1) evaluated the efficacy and/or harms of an NPWT system or components of a system; 2) reported outcomes for human patients; and 3) were published in English. Excluded from further consideration were animal studies, cell culture studies, and studies that focused on the mechanisms of NPWT. Also excluded were editorials, letters, comments, and meeting/poster abstracts. We only considered abstracts if they contained new or previously unreported data from a published full article. Go to Appendix A for a full explanation of our electronic database search.
• Systematic search of the following databases unlimited by date for secondary publications (e.g., systematic reviews, Health Technology Assessments): The Cochrane Database of Systematic Reviews (Cochrane Reviews), Database of Abstracts of Reviews of Effects (DARE), and Health Technology Assessment and Database (HTA).
• Search for additional published and unpublished studies, which included the following steps:
  • Manual search of bibliographies listed in fully published studies
  • Search and written inquiry to regulatory agencies, including the U.S. Food and Drug Administration (FDA) and Centers for Medicare & Medicaid Services (CMS)
  • Search of http://clinicaltrials.gov and http://www.controlled-trials.com for ongoing clinical trials.
• In the interest of being certain that our searches identified all relevant studies, we invited manufacturers and professional organizations to submit the following (Go to Appendix A for a list of all organizations contacted):
  • A current product label (requested of industry stakeholders only)
  • Published randomized controlled trials, observational studies, or other compelling clinical evidence that uses NPWT devices to impact relevant clinical outcomes.
  • Unpublished randomized controlled trials, observational studies, or other compelling clinical evidence that uses NPWT devices to impact relevant clinical outcomes.

The materials received were then assessed against the a priori inclusion criteria for each Key Question. Over 1,400 individual items were submitted by interested stakeholders for possible inclusion in the report. All items were reviewed for their relevance to the key questions. None of the submissions were studies directly comparing different NPWT devices or systems. We identified one additional systematic review,(70) two comparison studies evaluating NPWT vs. a comparator treatment,(71,72) and 23 uncontrolled case series(73-95)that met the inclusion criteria for consideration but which had not been identified by our searches. In addition, we included one unpublished case series submitted by Smith and Nephew(96) giving us a total of 24 additional case series included in this report. Figure 4 is an attrition diagram that provides a visualization of the disposition of materials as they were evaluated for possible inclusion in the report.

Figure 4 . Disposition of Articles Identified by Internal Searches and Outside Submissions

[D] Select for Text Description

The most common reasons for exclusion of submitted materials were

• personal statements of support for specific NPWT systems that did not include data relevant to this review
• animal studies
• studies not relevant to negative pressure wound therapy
• narrative reviews
• poster presentations
• case studies (fewer than five patients)
• publications that duplicated an already included study

A listing of individual stakeholders with included and excluded submissions including reasons for exclusion are provided in Appendix D.

Return to Contents

Study Quality Assessment

After determining which of the publications identified in our searches and materials submitted by interested parties met the inclusion criteria for this report, we assessed the potential for bias in these studies. The potential for bias in each study was assessed using a quality assessment instrument developed by ECRI Institute for comparative studies.

A poorly designed study may contain biases with the potential to artificially alter how effective a technology appears to be. In this sense, a bias is an error introduced into sampling or testing. In well-constructed studies, biases are minimized by design and conduct, and changes in outcomes and differences in outcomes between groups are definitively attributed to the treatment of interest. For these reasons, high-quality studies are ones in which study design and conduct eliminate or greatly reduce the potential for bias. The degree to which a study protects against bias is referred to as "internal validity." Evaluating study quality is a means of assessing the risk that bias, whether systematic or nonsystematic, has obscured the true treatment effect of the interventions under study and lowered the study's internal validity. Assessing study quality is therefore an essential part of making judgments about the overall strength of a body of evidence that addresses a key question.

To aid in assessing the quality of each of the studies included in this report, we used the quality assessment instruments developed by ECRI Institute for comparative studies as shown in Appendix B. Studies that did not meet the inclusion criteria were not assessed for quality. The ECRI Institute instrument examines different factors of study design that have the potential to reduce the validity of the conclusions that can be drawn from a study. In brief, the tool was designed so that a study attribute that, in theory, protects a study from bias receives a "Yes" response. If the study clearly does not contain that attribute it receives a "No" response. If poor reporting precludes assigning a "Yes" or "No" response for an attribute, then "NR" is recorded (NR = not reported).

To estimate the quality of an individual study, we computed a normalized score so that a perfect study received a score of 10, a study for which the answers to all items was "No" received a score of 0, and a study for which the answers to all questions was "NR" was 5.0. We then classified the overall quality of the evidence base by taking the median quality score. Quality scores were converted to categories as shown in Table 4 below. Studies with a low potential for bias are typically described as being of "high quality," whereas those with high potential for bias are described as being of "low" or "poor" quality, and those of moderate quality as having intermediate potential for bias.

Table 4. Study Quality Categories by Overall Quality of Evidence Base

	Low	Moderate	High
Median Overall Quality Score of the evidence base	≤6.0	>6.0 but <8.5	≥8.5

Return to Contents

Data Synthesis

The most appropriate study to address Key Questions 1, 2, and 4 is one that would directly compare the efficacy and/or safety of one NPWT system or components of a system to another NPWT system or its components. If the evidence base included two or more studies comparing one NPWT system to another, and when 75% or more of the available study data for an outcome could have been used in this analysis, we would have attempted to reach a quantitative conclusion using a random-effects meta-analysis and calculated a summary effect size estimate. Meta-analysis is a statistical technique that can be used to maximize the information obtained from the available evidence. Sometimes, individual studies are too small to determine even the direction of a possible effect. Using well-developed techniques, meta-analysis involves an efficient pooling of the data to possibly enable an evidence-based conclusion. Studies are not weighted equally, but instead larger studies tend to be weighted more heavily due to the increased precision of effect size estimates. An effect size is a measure of the size of a relationship between two treatments and is usually expressed as the difference between treatment results or as the ratio of treatment results. In a random-effects meta-analysis the study weights are determined not only by within-study variation, but also by between-study variation (which is also referred to as heterogeneity).(97)

If applicable, heterogeneity would have been assessed using the I² statistic, with an I² greater than or equal to 50% as evidence of substantial heterogeneity among study results.(97,98) Substantial heterogeneity among studies may have indicated that the studies being pooled are measuring different treatment effects. If at least five studies were used in a meta-analysis, we would have performed a meta-regression in an attempt to explain the heterogeneity using the permutation test p-value as described by Higgins and Thompson.(99) The following variables would have been used in a meta-regression: size of wound, duration of wound, patient comorbidities, use of ancillary treatments, and intensity of treatment. We would have attempted to obtain a quantitative summary effect estimate from an evidence base with unexplained heterogeneity. Individual studies may have undue influence in a meta-analysis and may be the sole reason a summary effect size is significant. Therefore, we would have tested homogeneous meta-analyses for robustness and the influence of single studies by the removal and replacement of each separate study, and by performing cumulative meta-analysis by publication date.

In the event that a quantitative conclusion were not possible, we would have entered all available data into a random effects meta-analysis to determine the robustness of a qualitative (i.e., direction of effect) conclusion. We would have performed the same sensitivity analyses as described above (removal of individual studies and performing cumulative meta-analysis). The data would have been considered robust if the summary effect size remained statistically significant and the direction of the effect size did not change (go from positive to negative or negative to positive) during the analysis.

In keeping with the methods of the EPC Program, we would have performed "adjusted indirect comparisons" if no studies directly comparing NPWT systems or components of a system were available.(100,101) In adjusted indirect comparisons, the comparison of the intervention of interest (i.e., different NPWT systems) would have been adjusted by the results of their direct comparison with a common control group (e.g., standard wound therapy).(102) The validity of an adjusted indirect comparison depends on the internal validity and similarity of the included trials. Thus, to be considered for inclusion in an indirect comparison, studies would have been similar in terms of quality, similar for factors related to applicability (population, interventions, and settings), and similar in measurement of outcomes including the incidence of adverse events.(102-105)

Evidence for indirect comparisons would have been obtained from randomized controlled trials (RCTs) of NPWT systems versus a common comparator. Evidence from non-randomized studies would not have been considered because indirect analyses are only recommended when RCTs are available. Non-randomized controlled trials may lead to biased estimates of treatment effects due to differences in baseline characteristics between groups within different studies. Even when patient characteristics are similar, other aspects of non-randomized controlled trials may vary, such as use of ancillary treatments, other aspects of patient care, or application of the actual intervention.(102) Even when RCTs are available for indirect comparisons, the conclusions must be framed cautiously because of the difficulty in assuring that the trial features are truly similar enough.

The validity of an adjusted indirect comparison depends on the internal validity and similarity of the included trials.(102) Thus, to be considered for inclusion in an indirect comparison, studies would have provided sufficient information to determine their comparability in terms of patient characteristics, patient exclusion/inclusion criteria, methodological quality, outcome definitions, outcome measures, and methods used in the comparison condition. Patients in studies on NPWT would have been similar in terms of age, comorbidities, use of ancillary treatments, type of wound, and severity of wound. The comparator condition would have been similar in terms of products (e.g., dressings), dosage, frequency of administration, and method of application. The studies considered in an indirect comparison would also have been similar with regard to important methodological criteria, such as concealment of allocation, proper randomization, blinding, follow-up times, and completion rates. In an attempt to locate appropriate studies to include in an indirect comparison, we abstracted and catalogued all relevant data from full-length controlled trials on NPWT systems. These data are reported in Appendix C.

Return to Contents
Proceed to Next Section