Metrics Tool

Project RED (Re-Engineered Discharge) Training Program

This document describes several metrics used to implement and measure the impact of a re-engineered discharge program.

Metrics


This document describes several metrics you may wish to use as you implement and measure the impact of your re-engineered discharge program.


Patient Care Plan Audit

The Project RED Patient Care Plan should have the following components:

  • Date of discharge and name and contact information for physician and Discharge Advocate (DA).
  • Medications the patient should continue to take at home (name, reason, time and frequency, dose).
  • Pending tests and results.
  • Follow-up appointments scheduled (or to be scheduled if patient must do this).
  • Other orders related to patient's self care (diet, activity, etc.).
  • Information about disease or condition.
  • Signs and symptoms that warrant a phone call to the physician.
  • Signs and symptoms that warrant a visit to the emergency department.
  • Form for patient to record questions to ask at a follow-up appointment.

Consider conducting a periodic audit to evaluate the completeness of the patient care plans. Each component in the list above should be addressed on the patient care plan. You may want to consider giving final copies of patient care plans to Quality Department staff members to evaluate care plans against the list.

To calculate the percent of care plans that includes each component, follow these steps:

  1. Count the total number of care plans reviewed (the denominator).
  2. For each section, count the number of care plans that includes that section's content.
  3. For each section, divide the total number of care plans (step 1 above) by the total number of care plans that included that section's content (step 2 above).
  4. Multiply the result of step 3, by 100 to calculate the percent of charts that include each section's content.

Example

Step 1—10 patient care plans were audited.
Step 2—Medications: 6 of the care plans included full information about the patient's medications.
Step 3—6 divided by 10 equals .60
Step 4—.60 times 100 equals 60 percent. 60 percent of the audited patient care plans included full content for the medications.

These steps are repeated for each patient care plan section.

This audit provides a general review of the completeness of your Project RED Patient Care Plans and highlights content areas that may warrant analysis for why the content was not added to the plan.

Time Metrics

Time metrics are used to analyze the timeliness of activities that are part of the re-engineered discharge program. The DA can keep track of dates and times associated with meeting the patient and completing Project RED's various activities. The time metrics of interest may include:

  1. Average elapsed time between admission of patient and notification of DA.
  2. Average elapsed time between DA notification and DA's first visit with the patient.
  3. Average elapsed time between discharge order and patient discharge (hours).
  4. Average elapsed time between patient departure and sending discharge summary to primary care provider.
  5. Average elapsed time between patient departure and placement of follow-up phone call to patient.

To calculate the average times above, the DA needs to collect specific time data. A log sheet can be created, as shown below, that includes the time data elements listed in the first column and subsequent columns representing individual patients. The DA notes all times in the patient's column.

Data to be CollectedPatient 1Patient 2Patient 3
Patient Name   
Patient Location   
Date and Time of Admission   
Date and Time DA Notified   
Patient Meets All Inclusion Criteria (Yes/No)   
Date and Time of Initial DA Visit With patient   
Date and Time of Daily DA Visits With Patient (Note All)   
Date and Time of Discharge   
Date and Time Care Plan Faxed to Primary Care Physician   
Date and Time of Post-Discharge Call   

To calculate the elapsed time between two activities, the time of the second activity is subtracted from the time of the first activity. For example, if the patient is admitted at 10 a.m. on June 24 and the DA is notified at 8 a.m. on June 25, the elapsed time is 22 hours. To calculate the average elapsed time for a metric across all included patients, add the elapsed hours for all included patients and then divide by the total number of included patients.

Ideally, the DA should collect initial data for the first 10 patients and then measure again after 3 and 6 months, using 10 patients each time.

Project RED Patient Experience Phone Survey

This brief questionnaire asks patients how prepared they were for discharge. It can easily be administered over the phone. This call should not be confused with the Project RED follow-up call, which is a clinical call focused on assessing the patient's condition and compliance with treatment plan.

All questions are posed as "yes" or "no" questions with yes as the desired answer. You can record answers on a chart, as shown below, marking a 1 for yes and a 0 for no.

To calculate the percent of yes responses, add the numbers for all patients then divide by the total number of patients interviewed. Multiple that amount by 100 to get the percent.

Each question is reviewed separately to identify specific areas of discharge preparation that staff may need to improve.

QuestionPatient 1
Yes = 1
No = 0
Patient 2
Yes = 1
No = 0
Patient 3
Yes = 1
No = 0
Total
Add the columns with each patient's answer to the question
Percent
Yes
Divide the total by the number of patients that responded to the question then multiply by 100
When you left the hospital, did you know the names of the medications your doctor prescribed for you?     
Do you know and understand the purpose of each medication?     
Do you know how and when to take each medication?     
Do you know when you should call your doctor (e.g., if you noticed changes in your condition)?     
Do you know how to reach your doctor?     
Do you know when to seek emergency care (e.g., if you noticed changes in your condition)?     
When you left the hospital, were you given written information about
  • Your medications?
  • Your specific disease or condition?
  • When to seek medical attention?
     
Were you able to follow these instructions when you got home? If no, please explain.     
Were you told about any tests you still needed to have completed?     
Were you told if any test results had not been completed yet?     
If any tests or test results were still incomplete, were you told if you were supposed to do anything about this?     
Were you told about follow-up appointments your doctor wanted you to have?     
Were these follow-up appointments scheduled for you before you left the hospital?     
If no, have you made your appointments yet?     
If the appointments were scheduled for you, were they made with your input as to whether the timing was good for you?     
If your input was not considered, were you able to make the appointment at the time scheduled for you? If no, why not?     
Did you get answers to all your questions before leaving the hospital?     
Was there enough time to ask your questions?     
What else could we have done to better prepare you take care of yourself at home? (List answers.)     

If you choose to use this questionnaire, collect baseline and post-implementation data. Before implementing the re-engineered discharge program, use this questionnaire with a sample of patients who have been recently discharged with the diagnosis of your targeted patient population and from the unit where you plan to pilot the new process. This will give you baseline information about your current discharge program. Following implementation, interview patients who received the Project RED intervention to compare the percent "yes" for each question to the baseline percentage. If desired, you can perform statistical tests to evaluate the significance of any change.

Project RED Frontline Staff Survey

This brief questionnaire is a tool to help analyze staff perceptions of the discharge program. Questions are answered using a Likert scale of 1 to 5, with 1 indicating "strongly disagree" and 5 indicating "strongly agree," with "strongly agree" as the desired answer. You can administer the questionnaire either by providing a paper copy to each staff member or by assigning someone to interview staff members. You may record answers on a chart, as shown below, recording a 1, 2, 3, 4, or 5 to reflect the staff member's response. Each question is analyzed separately to identify specific areas of discharge preparation that may need to be improved, according to staff perception. To calculate the average score for each question, add the numbers for all staff then divide by the number of staff members who responded to the question. Questions with high averages suggest the activity addressed by the question is working well. Questions with low averages suggest areas that may warrant review for improvement.

QuestionStaff 1
Staff 2Staff 3Total
Add the columns for each staff's answers to the question
Average
Divide Total of columns by total number of staff
1. Staff members on this unit think the discharge process is effective at preparing patients for discharge.     
2. The discharge process on my unit is organized and efficient.     
3. Team members' communication of discharge information is clear and direct.     
4. On our unit, we begin discharge planning on the day of admission.     
5. We have a process, such as discharge planning rounds, that communicates clear and concise information about the discharge plan.     
6. We check that our patients can follow the discharge instructions given.     
7. Our patients have discharge follow-up appointments scheduled prior to leaving the hospital.     
8. Our patients' primary care physicians have the information about patients' hospital visits immediately after discharge.     
9. Our patient/family knows and understands medications (dose, route, procedure, schedule, etc) prior to discharge.     
10. Our unit calls the patient after discharge to see how he or she is doing.     

If you choose to use this questionnaire, you should plan to collect baseline and post-implementation data. Before implementing the new process, interview a sample of staff members (nurses, physicians, case managers, etc.) who work on the unit where the new discharge program will be implemented. This will give you baseline information about the staff's opinion of the current discharge program. After implementation, interview staff members again and compare those results to the baseline results to assess the effect of the change. If desired, you can perform statistical tests to evaluate the significance of any change.

Project RED Primary Care Physician Survey

This brief questionnaire is a tool to help analyze primary care physician (PCP) perceptions of the discharge program. Questions are answered using a Likert scale of 1 to 5, with 1 indicating "strongly disagree" and 5 indicating "strongly agree," with "strongly agree" as the desired answer. The questionnaire can be administered either by providing a paper or e-mailed copy of the questionnaire to each PCP or by assigning someone to interview the PCP. You may record answers on a chart, as shown below, recording a 1, 2, 3, 4, or 5 to reflect the PCP's response. Each question is analyzed separately to identify specific areas of discharge preparation that may need to be improved, according to PCP perception. To calculate the average score for each question, add the numbers for all PCPs then divide by the total number of PCPs who responded to the question. Questions with a high averages suggest the activity addressed by the question is working well. Questions with low averages suggest areas that may warrant review for improvement.

QuestionPCP 1
PCP 2PCP 3Total
Add the columns for each PCP answer to the question
Average
Divide Total by number of patients
1. As a referring physician, I think the hospital's discharge process is effective at preparing my patients for discharge.     
2. The hospital sends me discharge information within 24-72 hours of my patient's discharge.     
3. The care plans given to my patients at discharge provide information that is clear and direct.     
4. When my patients come to me for their follow-up appointment, they can describe why they were admitted and can follow the discharge instructions given by the hospital.     
5. Our patients have follow-up appointments scheduled prior to leaving the hospital.     
6. Our patient or family knows and understands the medications (dose, route, procedure, schedule, etc.) upon discharge.     

If you choose to use this questionnaire, collect baseline and post-implementation data. Before implementing the new process, interview a sample of PCPs to give you baseline information about their opinion of the current discharge program. Following implementation of Project RED, complete the process again and compare the results to the baseline results to assess the impact of the change. If desired, statistical tests can also be conducted to analyze the significance of any change.

Return to Contents

Current as of August 2011
Internet Citation: Metrics Tool: Project RED (Re-Engineered Discharge) Training Program. August 2011. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/professionals/systems/hospital/red/metrictool.html