Appendix: Evaluation Items This worksheet can be used as a guide to developing and tracking your program outcomes. The topics and questions listed here are not intended to be prescriptive or inclusive. Instead, they are intended to be items to consider and track before and after implementation of an initiative or program. You may select specific areas in which you are most interested.OverallProgram goals and objectivesResearch Center SupportDo you have: Staff/systems to prepare budgets for grants/contractsStaff/systems to prepare invoices for funded projectsStaff/systems to keep track of funded project expenses and produce monthly expense reportsStaff/systems to identify and disseminate information about funding opportunitiesStaff/systems to provide access to previous proposalsStaff/systems to review proposals or approve proposalsInformation technology (IT)/computer equipment, software adequate for completing researchStaff/equipment/systems to facilitate literature searches and access to literatureAdministrative/secretarial supportCoordinate grant and contract activities within the organization (so that two different groups would not be competing with each other for the same project)Graduate or other research assistants to support your workOffice or laboratory spacePI and Staff CharacteristicsPI titlePI role(s) in organization (e.g. program chair, research faculty, academic faculty, clinical)PI salary Has the PI received a raise or promotion [include any increases greater than annual standard-of-living increases (~3-5% per year)]?Number of PIs' prior projects as a senior investigator (PI, co-PI, co-investigator, project director [PD], or deputy PD) Average per year in 5 years before and after initiativeAmount of PI research funding support Average per year in 5 years before and after initiativeHealth-services related funding supportAmount of staff funding support Average per year in 5 years before and after initiativeHealth-services related funding supportDid the PI leave the organization during the program? When did the PI leave the program?What was the reason for the departure?How long before the PI was replaced?Is the former PI currently doing health services research?How many staff or participants left the organization during the program? How long (months or years) was each staff person part of the program?What was the reason for the departure?Was this staff's role filled after the departure?How long before the staff was replaced?Is the former staff person currently doing health services research?PI advancement (list promotions, receipt of tenure, awards)Staff advancement (list promotions, receipt of tenure, awards)Building Research Skills: Training and MentoringDid program staff provide any mentoring to individuals? How many staff provided mentoring?What was the structure of the mentoring?How many mentees did the staff support?Did program staff receive any mentoring? How many staff?What was the structure of the mentoring?Did project staff provide any training/education? How many hours/sessions?What were the topics?Did project staff receive any training/education? How many hours/sessions?What were the topics?What was the format for training?Who provided the training?Did the project staff attend any health services related conferences?Building Research Skills: Individual Research ProjectsTotal number of individual health services research projectsContent areas and topics of research projects Do the projects cover similar content areas?None (0%); Some (1-60%) or Most (61-100%).Research methods of projects Do the projects use similar research methodology?None (0%); Some (1-60%) or Most (61-100%).Community impacts of research Were any health care services rendered as a part of or as a result of your project or program?Was any health-related information disseminated to the public?Were any government policies (at any level) influenced by findings from your research?Did your organization's program have any other community impacts?PartnershipsList each partner. For each partner, include: Current status of partnership (ongoing formal, ongoing informal or ended)How successful the partnership has beenHow likely the partnership is to continue in the next 3-5 yearsResearch CulturePrior levels of research funding overall (in 5 yrs before and after initiative). Average per year? http://www.nsf.gov/statistics/What types of research does your organization receive funding for?Are there any “research centers” in your organization? Are any of the research centers specifically related to health services research?Are there any formal organizational documents, such as mission or vision statements or strategic plans or policies and practices that explicitly supported or encouraged research?Are there any staff positions or committees that support research? (e.g., VP of Research, Staff Research Council).Does your organization have it have its own Institutional Review Board (IRB) or a formal agreement to use another organization's IRB?To what extent is conducting and disseminating research — that is, getting funding, making professional presentations and publishing in journals — considered by your organization in performance evaluations and decisions about raises, promotions and tenure?How much verbal support and encouragement does staff get from supervisors for conducting research activities?How much tangible support (e.g., financial support, release time, staff to assist) is available from supervisors for conducting research activities?How difficult is it to get the help needed from people within your organization? BiostatisticsGrant/Proposal writing and reviewJournal article writing and reviewResearch methodology issues (e.g., study design, designing a data collection instrument)How difficult is it to get the help needed from people outside of your organization? BiostatisticsGrant/Proposal writing and reviewJournal article writing and reviewResearch methodology issues (e.g., study design, designing a data collection instrument)How easy or difficult is it to obtain the following: Organizational approval to apply for a health services research grantStaff labor or time to assist with preparing health services research proposalsStaff labor or time to support conducting health services researchFinancial support from your organization to write proposals or to conduct health services researchTime off or release time to write proposals or conduct health services researchWhat barriers has your program faced; that is, what things have interfered with your ability to conduct health services research? How did you solve the problem?What has made your project run more smoothly or otherwise made it easier for you to conduct health services research?Communicating and ReportingPI: Total number of publications Average per year in five years before and after initiative?Articles in peer-reviewed journalsFirst author articlesHealth services research related articlesArticles in leading health services journalsSubmitted articlesStaff: Total number of publications Average per year in five years before and after initiative?Articles in peer-reviewed journalsFirst author articlesHealth services research related articlesArticles in leading health services journalsSubmitted articlesPI: Total number of completed posters, papers and presentations made at conferences. Average per year in five years before and after initiative?First author presentationsHealth services research related presentationsStaff: Total number of completed posters, papers and presentations made at conferences. Average per year in five years before and after initiative?First author presentationsHealth services research related presentationsDid you share the results of your research with the public? In what format? Current as of October 2012 Internet Citation: Appendix: Evaluation Items. October 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/funding/other/education/resources/hsrguide/hsrguideap.html
Back
