Sections from the data collection instruments developed and used by AHRQ Estimating Costs grantees are included here as examples for researchers who are considering how to best collect information to measure the costs of primary care transformation efforts. Please contact the appropriate principal investigators for permission to use these instruments and for full versions of the tools. Contact information is provided with each tool.
Example 1. Pre-Interview Questionnaire
Principal Investigator: Sarah Shih
Institution: New York City Department of Health
Contact Information: sshih@health.nyc.gov
The following questionnaire was used as a pre-interview preliminary assessment. The first page (describing the purpose of the questionnaire) and a sample page of the questionnaire are presented below. The full questionnaire contains 38 questions assessing the practice site, quality improvement activities, participation in incentive programs, patient information systems, care coordination, language services, self-care support and community resources, patient population and volume, finances, and respondent information.
Purpose: The goal of this pre-interview questionnaire and interview is to understand the resources used and activities conducted by primary care practices in order to meet Patient Centered Medical Home (PCMH) standards. PCMH, also referred to as advanced primary care or comprehensive primary care, is an increasingly popular model of care delivery.
Why you were asked: You are invited because your practice has obtained PCMH recognition in the past. The pre-interview questionnaire is administered by PCIP.
Benefits of participating: There are no direct benefits to participating in this pre-interview questionnaire. If you choose to participate, you will receive $100 as a thank you.
Risks or discomforts: No risks or discomforts are anticipated. You may skip any questions that you do not want to answer, or mark not applicable (NA).
Confidentiality/How the findings will be used: Participation is completely voluntary and will not affect the services you receive from PCIP, nor your relationship with NCQA. Your answers will only be shared internally amongst PCIP members. The findings will potentially influence future incentive or reimbursement policies. Summary results may be presented at conferences and in publications. We may use individual quotations, such as "a small practice provider reported...", but no names and/or identifying information will be shared.
How long will it take: The pre-interview questionnaire should take about 25–30 minutes. If you have concerns, please contact the Principle Investigator, Sarah Shih.
By clicking "Next" at the bottom of the page, you acknowledge that you have read this information and agree to participate.
14. About how often do you think your patients experience the following at this practice/clinic site?
| Usually (75%– 100% of the time) |
Often (50%– 74% of the time) |
Sometimes (25%– 49% of the time) |
Rarely (1%–24% of the time) |
Never | |
|---|---|---|---|---|---|
| a. Patients' appointments are scheduled with their personal clinician versus another clinician | |||||
| b. Patients are able to receive a same- or next-day appointment when they request one | |||||
| c. Patients can get telephone advice on clinical issues during office hours | |||||
| d. Patients can get telephone advice on clinical issues on weekends or after regular office hours | |||||
| e. Patients can email clinicians about clinical issues |
Example 2. Interview Guide
Principal Investigator: Ming Tai-Seale
Institution: Palo Alto Medical Foundation Research Institute
Contact Information: Tai-sealeM@pamfri.org
The interview guide presented below was used to conduct interviews with key informants/individuals who were involved in the target interventions. Through the interviews, the study team obtained details about practice personnel and resources used in the intervention, changes to workflow, and financial information.
Primary Care Transformation Costs Interview Guide
Project Beginnings
- How did the project start?
- Any additional time spent on the project outside of work hours (e.g., thinking/brainstorming)?
- How much time did these beginning activities take?
- Could we perform an Outlook calendar/email search to give a more accurate representation?
- How much time did these beginning activities take?
- Any additional time spent on the project outside of work hours (e.g., thinking/brainstorming)?
- When did the project start (year/month)?
- Could we perform an Outlook calendar/email search to give a more accurate representation?
- When did the intervention roll out for clinical practice?
- Did the intervention expand to other sites/departments, and if so, when?
Individual Contribution
- When did you start working on the project?
- Tell me about your role in the intervention.
- What were your responsibilities?
- Describe in detail a typical day of doing tasks specific to the implementation.
- How much time did it take to complete each responsibility?
- Could we perform an Outlook calendar/email search to give a more accurate representation?
- What were your responsibilities?
- How long were you a part of the initiative?
Key Personnel
- Who were the key personnel involved in the project?
- Is there a personnel list?
- Did the key personnel change over time?
- Who else was involved in the intervention?
- Who would you recommend we talk to in order to learn more about how this project was developed and implemented, specifically regarding costs (leadership and clinical level)?
Budget
- Outline what has happened with the budget over time—from early development to clinic implementation:
- When did the budget begin?
- What was your involvement in the creation of the budget?
- What was the decision process for deciding how much money would be allocated to the project?
- Was there a budget proposal?
- Were other data/ documents used?
- What kind of budgeting practice is used (e.g., bottom line)?
- How accurately did it reflect what actually occurred with the project?
- If budget modifications occurred when project was implemented:
- Who determines modifications to the budget?
- What is the decision process for making a change?
- Were data/documents needed?
- If NO budget, how did you cover the costs?
- What are the overall fiscal implications of the implementation?
Resources/Organizational Expenses
- What were some of the resources/organizational expenses associated with the project?
- Purchasing equipment/hardware/software?
- Hiring personnel/reallocating job responsibilities?
- Information technology work?
- Building/reallocating facility space?
- Staff trainings (days and hours)?
- Anything else?
- Please specify specific costs of each.
- If you do not know, who has the knowledge and/or documentation of each?
What else should we know? What did we not ask that we should have? What more do you have to add?
Example 3. Organizational Survey
Principal Investigator: John Kralewski
Institution: University of Minnesota
Contact Information: krale001@umn.edu
The screenshots below are excerpts of the practice organizational survey developed by Kralewski et al. This survey was distributed at the time of the interviews and was used to collect data on clinic structure and organization as well as cost data. Data elements collected through this survey included clinical and support staffing levels (by type, job category, and specialty); patient workload (patient visits per provider per year, adjusted for illness level); unique patients, patient encounters, and procedures per FTE physician by specialty; medical revenue; cost data pertaining to transformation components described during interviews, such as new equipment, additional staff, and training costs, and changes in patient level costs. In addition to the practice organizational survey, the research team also used an organizational culture assessment. This survey is not featured in the Practice Guide but is available upon request.
Medical Group Practice Organizational Survey
Clinic Identification Code: _________________________________________________________
Please provide the following information about your group practice.
| 1. Number of physicians | Number of FTEs |
| a) Primary care: _____ | _____ |
| b) Surgery specialty: _____ | _____ |
| c) Medical specialty:_____ | _____ |
| d) Pediatrics:_____ | _____ |
| e) OB/GYN_____ | _____ |
2. Indicate how many primary care physicians in the following categories work at your clinic.
a) Less than 10 hours a week ____
b) 10 to 24 hours a week ____
c) 25 to 35 hours a week ____
d) More than 35 hours a week ____
3. Please complete the following regarding the number of health professional staff in your clinic and their FTEs:
| Staff Title | # Staff | FTEs | Staff Title | # Staff | FTEs |
|---|---|---|---|---|---|
| Nurse practitioner | Psychologist | ||||
| Registered nurse | Optometrist | ||||
| Medical assistant/LPN | Social worker | ||||
| Social worker | Speech-language pathologist | ||||
| Chiropractor | Respiratory therapist | ||||
| Dietician | Other (please specify) | ||||
| Occupational therapist | |||||
| Physician assistant | |||||
| Psycho-geriatric | |||||
| Pharmacist | |||||
| Physiotherapist |
6. Average number of patients scheduled per primary care physician and NP/PA during a normal day:
Physician ____ NP/PA _____
7. Average number of patients scheduled per primary care physician and NP/PA per clinic hour:
Physician ____ NP/PA _____
8. Is someone in your practice designated to monitor patients with chronic illnesses?
Yes ____ No _____
9. Total support staff FTE per FTE physician
| Number | |
|---|---|
| Total business operations support staff FTE per FTE physician | |
| Total front office support staff FTE per FTE physician | |
| Total clinical support staff FTE per FTE physician | |
| Total ancillary support staff FTE per FTE physician | |
| Primary care physicians per FTE physician | |
| Nonsurgical physicians per FTE physician | |
| Surgical specialty physicians per FTE physician | |
| Total NPPs per FTE physician |
10. Cost
| Cost | |
|---|---|
| Total operating cost per FTE physician | |
| Total operating and NPP cost per FTE physician | |
| Total operating cost as a percent of total medical revenue | |
| Total operating and NPP cost as a percent of total medical revenue |
11. Revenue
| Revenue | |
|---|---|
| Total medical revenue per FTE physician | |
| Total medical revenue after operating cost per FTE physician | |
| Total medical revenue after operating and NPP cost per FTE physician |
12. At your clinic, for follow up of people with chronic illnesses (e.g., COPD, diabetes, heart failure), do you (doctors or clinic staff):
| Always | Often | Sometimes | Rarely | Never | |
|---|---|---|---|---|---|
| Use a tracking system to remind patients about needed visits or services? | |||||
| Offer to follow up with patients between visits by telephone? | |||||
| Use recognized practice guidelines as the basis for their treatment plans? | |||||
| Assist patients in setting and attaining self-management goals (e.g., participation of patient in management of their care)? | |||||
| Refer patients to someone within your clinic for education about their chronic illness? | |||||
| Refer patients to someone outside your clinic for education about their chronic illness? | |||||
| Have flow sheets (checklists) in medical records to track critical elements of care? |
13. What are the roles and functions of the nurse practitioners on your medical team? (Check all that apply)
| There are no nurse practitioners in our clinic | |
| Triage of walk-in patients | |
| Counseling on tobacco use, diet, and physical activity | |
| Patient education (e.g., blood glucose testing, blood pressure measurement) | |
| Follow up of specific patient groups (e.g., chronic diseases, age group) | |
| Sexually transmitted and blood borne infections (STBI) counseling | |
| Prescribe diagnostic examinations (e.g., radiography, blood tests) | |
| Liaison and coordination with LTCF, hospitals, and other clinics | |
| Support for medical activities (blood pressure, weight, injections and vaccinations) | |
| Participation in clinical decisions | |
| Conducting clinical activities as part of a delegated medical act | |
| Prescribe medications and other substances |
14. During the last year, approximately how many patients received primary care from your clinic? Please count each patient only once, no matter how much care he or she received (your best estimate will do). Number of patients ____
15. Is your clinic currently accepting new patients for management and followup? Check a single answer only.
| Our clinic accepts all new patients who ask | |
| Our clinic selectively accepts a limited number of new patients | |
| Our clinic is not accepting new patients |
16. Is your clinic currently accepting new patients with the following health insurance?
| Yes | No | |
|---|---|---|
| Health insurance | ||
| Medicare | ||
| Medicaid | ||
| Patients with high deductible commercial insurance | ||
| Minnesota Care |
17. At your clinic, how much time is scheduled for the following patients?
a) A routine visit for a new patient? _____ minutes
b) A routine visit for an established patient? ______ minutes
18. If a patient requests an appointment for a non-urgent condition, how long will the patient need to wait before being seen? ____ days
19. Do you offer open-access scheduling? ____ yes ____ no
20. Please indicate whether your clinic offers patient the option to:
| Yes | No | Don't Know | |
|---|---|---|---|
| Request appointments or referrals online | |||
| Email a medical question or concern | |||
| Request refills for prescriptions online | |||
| View test results on a secure Web site |
Example 4. Web-Based Data Collection Tool: Cost Assessment of Collaborative Healthcare (CoACH)
Principal Investigator: Benjamin Miller
Institution: University of Colorado Denver
Contact Information: Benjamin.Miller@ucdenver.edu
The screenshots below show the CoACH tool developed and used by Miller et al. This tool consists of an online questionnaire and integration activities graphic workflow. The intake questionnaire asks about practice demographic information; number, FTEs, salary, and benefit information for all providers and staff involved in integrated care delivery; time spent on integration activities by each provider and staff type; and operational costs directly related to integration, including new space, computers, software, and other materials. Based on information provided in the intake questionnaire, the tool estimates the total cost of integration activities and displays a graphic workflow and a table summarizing minutes per activity per personnel. Respondents can then edit workflow assumptions to model the total cost of different scenarios; for example, adjusting the number of minutes per activity or adding new activities.
Interactive workflow:
