Expanding Task 2. Application of a Modified Algorithm at Denver Health
In addition to the activities described in previous chapters, the infection control team at Denver Health (DH) sought to further adapt, tailor, and validate the electronic detection algorithm created in Chapter 2 for use in everyday surveillance of surgical site infections at DH, to reduce the burden of chart review while also identifying a high percentage of SSI. The mandate for the Expansion algorithm was to maximize sensitivity at the expense of specificity, while realizing a meaningful reduction in the chart review burden experienced by infection control staff. “Meaningful” was loosely estimated as at least a 50 percent reduction while maintaining at least 95 percent specificity. The team focused specifically on the loose algorithm rule, as it was considered the more sensitive model in the main project.
DH's Infection Prevention Data Manager, Bryan Knepper, generated a retrospective cohort of procedures, including associated SSI as defined by NHSN definitions, using DH surveillance data from 2007-2010. Procedures included hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy.
The suggested algorithm components generated in Chapter 2 were reevaluated in a DH-specific context. Mr. Knepper, who was not an original member of the project team, met with a group of clinical professionals to discuss the loose rule parameters that would be most useful—given the DH patient population, clinical practices and prescribing and ordering tendencies—to determine which variables were most easily and reliably obtained through the DH centralized data warehouse. As a result of these discussions, along with subsequent validation of each parameter using an expanded dataset from DH, some factors from the main project's loose algorithm were incorporated faithfully into the Expansion model while other parameters were deleted, altered or expanded. “NE_N”, “ESR”, and “postopabx” were deleted. “Postopcx” was incorporated faithfully. The cut-point for “wbc” was changed from 9,000 cells/ml to 10,000 cells/ml. The biggest change to the models generated through the main project was to the parameter “postopadmit”. The Expansion algorithm was trained to look for both outpatient visits and inpatient admissions. Large gains were realized in sensitivity, with substantial decreases to specificity. To offset decreased specificity, only visits/admissions associated with a specific list of ICD-9 codes76 were searched for. The listed ICD-9 codes are generally associated with infection, with some directly related to surgical site infection (go to Exhibit 51).
Variables included in the DH modified algorithm were leukocytosis (white blood cell count > 10,000 cells/mL), a culture (regardless of result), or a followup visit associated with any of a list of SSI-related ICD-9 codes (go to Exhibit 51).
From this, 2,179 procedures were included in the cohort. Sixty procedures were associated with SSI after manual chart review using NHSN methodology (Exhibit 52).
Results. The modified algorithm flagged 804 procedures (37 percent of total charts) for review. The percent of total procedures flagged for review varied by procedure type, and ranged from 15 percent (herniorrhaphy) to 62 percent (craniotomy). The modified algorithm achieved 100 percent sensitivity and 72 percent specificity in detecting SSI validated on 4 years of our manual SSI surveillance data using NHSN methodology.
Potential for savings. Over the four year period, 1,375 unnecessary chart reviews would have been avoided without sacrificing detection of a single SSI. Assuming 20 minutes per chart for manual review, 57 full (8-hr.) days of chart review would have been eliminated using the algorithm for surveillance of SSI in hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy.
Conclusions. DH was able to successfully adapt, tailor, and validate the electronic detection algorithm (generated in Chapter 2) to determine SSI rates for an expanded set of surgical procedures, including hip and knee arthroplasty, abdominal and vaginal hysterectomy, spinal fusion, craniotomy, and herniorrhaphy at Denver Health. The modified algorithm was tailored to our setting, to be 100 percent sensitive while still reducing overall chart review burden by 63 percent.Over a 4-year period, this would have saved 57 full days of chart review at our institution, allowing more time for education and other active infection prevention interventions. The successful adaptation of the Chapter 2 algorithm to maximize sensitivity for utilization with both inpatient and outpatient visits instead of solely for postoperative admissions by an individual who was uninvolved in its development demonstrates the potential for translation into practice on a broad scale, which was one of the goals of this project.
Expanding Task 3. Testing additional risk factors using uniquely available Intermountain Healthcare data
As an expansion of Subtask 3.5, we identified SSI risk factors using only the dataset from Intermountain that included eight additional potential risk factors. Since 87 percent of the data came from Intermountain, we also analyzed the Intermountain data alone to see if we could detect any differences in risk factors for SSI. The same statistical methods were used as for the previous analysis that included all the data from the three other facilities plus the Intermountain data. However, this analysis also included eight additional potential risk factors that were electronically available in the Intermountain EDW (Exhibit 24), but not available at all of the other three facilities. Thus, 41 potential risk factors were included.
E3.5.1. Multivariate analysis of the datasets, including each procedure as a binary variable
During the univariate analyses of the derivation dataset, 11 different risk factors were included in the model compared to 13 for the total dataset. As before, that analysis also included each of the five different procedures as a binary variable (yes/no). Each of those 11 risk factors was then included in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 53, Exhibit 54, and Exhibit 55). Again, the significance of each of the 11 potential risk factors changed during each test using the three different datasets. For the derivation dataset, eight of the 11 univariate risk factors remained significant in the model compared to five in the validation set and then 9 when both derivation and validation sets were combined. Not only were history of MRSA infection and a postoperative admission within 30 days significant in all three tests as in the previous analysis, but chronic kidney disease, preop hemoglobin, and herniorrhapy were also significant. Two of the eight new risk factors made it into the multivariate analysis—number of surgeons and preop glucose. While preop glucose was not found significant in any of the three analyses, number of surgeons was significant in the derivation and combined analyses. Compared to the previous combined analyses with the total dataset, the combined analysis with only the Intermountain data identified additional risk factors including emergency surgery, being male, number of surgeons, preop hemoglobin, surgery duration, and herniorrhaphy, while number of procedures, postop hematocrit, and CABG surgery were no longer significant. As before, postoperative admission was indicative for admission due to a postoperative wound.
E3.5.2. Multivariate analysis of the datasets, including only CABG surgery
During the univariate analyses of the derivation dataset, 14 different risk factors were included in the model compared to only 7 for the previous total dataset. Each of those 14 risk factors was then included in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 56, Exhibit 57, and Exhibit 58). The significance of each of the 14 potential risk factors changed during each test using the three different datasets. For the derivation dataset, nine of the 14 univariate risk factors remained significant in the model compared to only two in the validation set and four when both derivation and validation sets were combined. In this case, only postoperative admission within 30 days and BMI was significant in all three tests. Three of the eight new risk factors made it into the multivariate analysis but only number of surgeons was significant in the derivation analysis. Compared to the previous combined analyses with the total dataset, the combined analysis with only the Intermountain data only identified postop hemoglobin as an additional risk factor along with BMI, history of MRSA and postop admission which were included in the previous all site analysis. Surgery duration was the only significant risk factor from the previous analysis not included in the Intermountain data only. Thus the significant risk factors from the analysis with only the Intermountain data were very similar to those in the analysis with total dataset. This is not surprising since the Intermountain data contributed over 95 percent of the CABG surgeries in the total dataset. The difference would be attributed to the 78 CABG surgeries from the VA and the inclusion of the eight new risk factors.
E3.5.3. Multivariate analysis of the datasets including only herniorrhaphy.
During the univariate analyses of this derivation dataset, six different risk factors were included in the model compared to 7 for the previous total dataset. Each of those six risk factors was then included in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 59, Exhibit 60, and Exhibit 61). The significance of each of the six potential risk factors changed during each test using the three different datasets. For the derivation and validation datasets, only postop admission remained significant in the model. In the combined dataset, emergency surgery was significant in addition to postop admission. Only preop glucose from the eight new risk factors made it into the model, but was not found to be significant in any of the three separate analyses. Compared to the previous combined analyses with the total dataset, the combined analysis with only the Intermountain data only identified only emergency surgery in addition to postop admission while postop admission was the only risk factor identified in the previous total dataset. Thus, while the risk factors included in the model were mostly different, only emergency surgery was different in the list of significant risk factors. Since Intermountain only contributed 57 percent of the herniorrhaphy data, this similar result was not due to a dominance of Intermountain data in this case.
E3.5.4. Multivariate analysis of the datasets including only total hip surgery.
During the univariate analyses of the derivation dataset, 12 different risk factors were included in the model compared to eight for the previous total dataset. Each of those 12 risk factors was then included in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 62, Exhibit 63, and Exhibit 64). The significance of each of the 12 potential risk factors changed during each test using the three different datasets. For the derivation dataset, only three of the 12 univariate risk factors remained significant in the model compared to only one in the validation set and then four when both derivation and validation sets were combined. Only postoperative admission within 30 days was significant in all three tests. Only two, Charlson score and preop glucose, of the eight new risk factors made it into the multivariate analysis but neither was significant in any of the three analyses. Of interest for total hip surgery, although not all of the risk factors identified during the univariate analyses were the same, the same four risk factors were significant in the logistic regression combined analyses using the Intermountain data alone and the previous combined analyses with the total dataset. Total hip surgeries from Intermountain contributed to 85 percent of the total dataset.
E3.5.5. Multivariate analysis of the datasets including only total knee surgery.
During the univariate analyses of this derivation dataset, eight different risk factors were included in the model compared to 5 for the previous total dataset. Each of those eight risk factors was then included in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 65, Exhibit 66, and Exhibit 67). The significance of each of the six potential risk factors changed during each test using the three different datasets. For the derivation dataset, Charlson score, male, history of MRSA and postop admission remained significant, only history of MRSA and postop admission were significant in the validation dataset and male was included along with history of MRSA and postop admission in the combined dataset. While it was included in many of the other Intermountain analyses, this was the first time Charlson score was found to be significant. Compared to the previous combined analyses with the total dataset, the combined analysis with only the Intermountain data only identified of the five risk factors identified in the previous total dataset. Number of procedures and preop hematocrit were additionally found significant in the total dataset. The Intermountain total knee data contributed to 90 percent of the total dataset.
E3.5.6. Multivariate analysis of the dataset including only appendectomy surgery at Intermountain healthcare
Although all the appendectomy data in the total dataset was from Intermountain, we analyzed the appendectomy data again with the eight new potential risk factors included. During the univariate analyses of the derivation dataset, 10 different risk factors were included in the model compared to only seven for the previous dataset. Each of those 10 risk factors was then included again in three different logistic regression analyses using a 60 percent derivation set, a 40 percent validation set and then the combined datasets (Exhibit 68, Exhibit 69, and Exhibit 70). The significance of each of the 10 potential risk factors changed during each test using the three different datasets. For the derivation dataset, only three of the 10 univariate risk factors remained significant in the model compared to three in the validation set and three when both derivation and validation sets were combined. Postop hematocrit along with postoperative admission within 30 days was significant in all three tests. While Charlson score and preop glucose were the only two of the eight new risk factors that made it into the multivariate analysis, only Charlson score was significant in the derivation analysis. The inclusion of the eight new potential risk factors in this analysis did impact the list of univariate risk factors included in the logistic analysis and the significance of each. Postop hemoglobin was significant in the analysis that included the new risk factors in addition to the same other two, postop admission and postop hematocrit, that were significant in the previous analysis without them.
