The document below is an amended transcript of a Web chat that took place on September 20, 2001, among the AHRQ staff and potential applicants to the CAHPS® II RFA. Questions have been numbered for reference. In some cases, the response has been edited for clarity or expanded to include additional information that may be helpful to applicants. Even those who participated in the Web chat may find it helpful to review these responses. Applicants continue to E-mail us with questions, so we will continue to post these new questions and their responses as they are received.
Moderator: Welcome to the Web Chat. Dr. Crofton, do you have any opening comments?
CAHPS: Yes, I'd like to say hello to everybody and thanks for joining us. With me today is Chuck Darby, and Al Deal from our Grants office. We also have two new members of the CAHPS® Team—Kathy Crosson and Carla Zema.
1. I have tried to access the TalkingQuality Web site (www.TalkingQuality.gov) that was referenced in the RFA, but the site is password-protected. How can I access this?
Unfortunately, the TalkingQuality Web site is not available at this time. The URL takes you to the beta site that was tested
earlier this year. If you are interested in seeing the beta site, please
contact Maria Wey at MWey@ahrq.gov for access information.
2. Will grantees have automatic access
to NCBD (National CAHPS® Benchmarking Database) data and reports for analytic and testing
purposes? For example, will the grantees be able to
test the NCBD reports with providers?
Regarding the NCBD data, there are established
rules and procedures for access to this data set. Ready access is
provided to the NCBD data as long as those wanting access follow a standard set
of procedures for maintaining confidentiality of the data. Regarding
testing NCBD reports with providers, this is a possibility. Once we assemble
the grantee team, we will decide together which reporting projects will take
priority. This will be done through the steering committee.
3. What are the minimum details required for the
letter of intent?
Organization and the name of the Principal
Investigator and their alternate. If you feel like providing us with more information
about team members, that would be helpful, but not necessary. More
information is on page 22 of the RFA.
4. Page 8 of the RFA discusses convening experts in a
variety of areas, including "information" design.
Could you elaborate the types of skills entailed in the
area of "information" design?
Information design refers to ways of formatting
electronic and print documents in ways that maximize the reader's ability to
understand and use the information. The reference in the RFA to Ginny Redish's is a
good source of information.
5. Can I submit an application which addresses some, but
not all, of the activities listed in the RFA?
As it states in the RFA, applications that do not
describe the applicant's ability to respond to all elements of the RFA will be returned to the
authors as nonresponsive. We attribute much of the success that CAHPS® has experienced to
our great network of grantees, their subcontractors, the SUN (Survey User's Network) contractor, Federal
partners like CMS (the Centers for Medicare & Medicaid Services) and OPM (the Office of Personnel Management), accreditation organizations like NCQA (the National Committee for Quality Assurance), sponsors who have agreed
to participate in field testing and evaluating CAHPS® products, and many others. But
managing a network of this size is often challenging and always time-consuming. Our
experience in CAHPS® I convinced us that the most efficient and productive way to
organize and monitor the work was to consolidate those who receive CAHPS® funds into
"grantee consortia." This limits the "official" points of contact for the Project
Officer to each grantee's Principal Investigator and alternate while ensuring that, across all
grantees, we have all the necessary expertise to perform the work. One consequence for applicants
is that Principal Investigators be excellent managers as well as content experts.
6. One of the report templates listed on Page 7 of the
RFA under Report Development and Testing is "CAHPS® for
children with special health care needs." The instrument was developed with leadership from the
Foundation for Accountability (FACCT). Is it the
intent of the RFA that applicants work with FACCT
framework or is the template to be largely based on
the CAHPS® framework?
The CAHPS® instrument for children with special
health care needs (CSN) was developed in collaboration with the CAHMI (Child
and Adolescent Health Measurement Initiative). FACCT played a major role in the CAHMI and was
critical to the successful production of the instrument. Before we can say
whether our collaboration with FACCT will continue, we need to know who our
grantees are, so we'll address that question as a team after the awards are made.
7. We're wanting to use
CAHPS® statewide in public health for quality
improvement purposes. Would it be best for us to
partner with a CAHPS® vendor or with a university if we
applied for the grant?
You need to review the requirements as specified in
the RFA and choose the organization which has the best mix of skills
to assist you in addressing this part of the project.
8. Do the current CAHPS® grantees have an advantage over
new applicants for an award?
No. CAHPS® products have been available for more
than five years and many organizations apart from the CAHPS® I grantees have
had the opportunity to use them and learn their strengths and weaknesses. We expect
that some of these organizations will consider applying for CAHPS® II funds. But as
the RFA states, neither experience with CAHPS® products nor status as a CAHPS® I grantee is a
prerequisite for application, though applicants must have experience developing and
testing consumer instruments and reports. In order to level the playing field as much as
possible, we have made available through the RFA a
variety of documents related to CAHPS®. Many of these documents have already been published or have been made available to the public in other ways. Some of them are "working notes" to which only the CAHPS® I team has had access in the past. Our intent is to inform potential applicants as thoroughly as possible about each ongoing or completed CAHPS® I project. Also, the CAHPS® II RFA includes two new areas, quality improvement studies and a greatly expanded reporting task. Since these activities were not included in CAHPS® I, CAHPS® I grantees have no advantage here.
9. Do you foresee that the G-CAHPS® will replace the standard
plan-level CAHPS® in meeting health plan accreditation requirements any time soon?
The use of G-CAHPS® for accreditation purposes and other uses will be advised by the committee of users that will assist the CAHPS® developers in deciding the application of the CAHPS® instruments. The health plan level CAHPS® adoption for use as an accrediting instrument came after
considerable consultation with CAHPS® users. Adoption of G-CAHPS for accreditation purposes is also dependent on the accrediting organizations such as NCQA and JCAHO (The Joint Commission on
Accreditation of Healthcare Organizations).
10. When do you anticipate that there will be enough use
made of the G-CAHPS® to have a useful amount of data to
employ for benchmarking?
The determination of sufficient data in the
National CAHPS® Benchmarking Database of G-CAHPS® data will be based on the availability of data in
different sectors, including the privately insured and publicly insured.
This decision will be made in consultation with
the CAHPS® grantees and SUN contractor.
11. On page 6 of the RFA, one area
listed under questionnaire development and testing is
"health plans, services and providers." How are you
distinguishing providers from services? Also, how
broadly should we construe the term "services?"
The term "services" refers to things that happen in
interaction with providers and things that happen in interactions with plans.
12. The RFA places the QI (Quality Indicators) work in Phase
II. Is it your intention that the proposed QI
activities would not begin until year 3 of the project?
You've hit on an area (timing of work) that was
tough for us as we wrote the RFA! We think that we'll need to review the timing of
lots of different project activities when we meet with the grantees at the beginning of the
project. We're also aware that we haven't asked people to "sign up" sites at
which these QI projects would be done. So, we're assuming that we'll need a little time up
front so that the grantees can do that.
13. Is it important for grantees to obtain letters from
NRHCHDR (the National Rehabilitation Hospital Center for Health and Disability Research) and NIDRR (the National Institute on Disability and Rehabilitation Research) indicating their willingness to work
with them or is this assumed regardless of the grantee?
Assume that NRHCHDR and NIDRR will work with us on
development of products for people with mobility impairments (PWMI).
14. Must applicants have recruited an organization who
will work with them as test sites and obtain
confirmation letters from them as part of the
application?
No, you don't have to sign them up in advance. We
do want to know that you're able to recruit sites that you collaborate with and have
done so in the past.
15. What are the most frequently
requested languages for the CAHPS® instruments, other
than Spanish?
It's really varied. In California, they need
translations of instruments in different Asian languages. People have also asked for
translations in Russian and other dialects of Spanish. We hate to sound like a broken record, but we'll
need to consult as a team at the beginning of the project to get some consensus on which
languages we'll target. Also, this is a perfect example of where users give us excellent
input on their needs.
16. Unlike nursing homes, I didn't see any specific
mention of hospitals in the questionnaire development
section; are they included under the term "providers?"
When we developed the RFA, we did not specifically
target the development of questionnaires of hospitals, but we continue to
get requests for hospital survey instruments. We can't say this is a dead issue—issues tend to
recycle in this project, but we're not asking for applicants to describe their ability to
do hospital surveys as part of their response.
17. What is a good working assumption for the number of
members of the advisory committee and the expert panel?
Assume 12 for the advisory committee and 12 for the
expert panel. But, as you've heard us say before, these are issues we'll talk about at
the beginning of the project.
18. What is the Grantee Advisory Committee?
The Grantee Advisory Committee will be composed of
representatives of the key stakeholders affected by the implementation of
CAHPS® products. Since this group's role is to advise the grantees (rather than AHRQ), the
grantees are responsible for selecting members of this group, setting meeting agendas,
making all meeting arrangements and convening the group as often as necessary.
19. In section 4
under budget and related items, it states that the
applicant should devote 1/3 effort to each of three
areas. I am not clear on what the areas are.
This budget guidance refers to the amount of
money you would spend on further development of
instruments currently being developed. It means that approximately 1/3 of that budget
would be spent on G-CAHPS and Nursing Home CAHPS®, 1/3 on PWMI CAHPS®, and 1/3 on
individual provider level CAHPS®. Each of these extends over the first two years.
20. In the RFA you use the term "cultural
comparability"; how are you distinguishing it from "cultural
competence?"
In our terminology, cultural comparability refers
to the development of survey instruments which are neutral to response
tendencies of different cultural groups. In contrast, cultural competence refers to the
capability of a health service facility to address cultural differences in their clients.
21. The RFA notes testing among those with poor cognitive
skills. Is it up to the grantees to propose a method
of assessing cognitive status?
Let's distinguish between assessment of cognitive
skills to see if people are able to fill out a
questionnaire, versus testing of reports and other materials to determine
comprehension and ease of use. We were really thinking about the latter throughout
most of the RFA. We started CAHPS® I by
thinking that a somewhat small segment of the population would require materials
for people with poor cognitive skills. When we saw the results of the National Adult Literacy
Survey that showed that only eighteen percent of adult Americans can correctly
use a bus schedule, we began to realize that there are a lot more people out there with poor
cognitive skills than we thought. So, we need to keep this fact in mind as we prepare
reports and other materials.
22. The CAHPS® Steering Committee includes PIs and "their
alternates." Are the Co-PIs the likely alternates or
is this open to other possibilities?
In grants terminology, there really is no such
thing as a Co-PI, but we need an alternate for each PI with similar skills. So, it's the PI
and alternate who will serve on the steering committee.
23. In light of recent events, is it possible to
consider some video-based conferencing as a substitute
for some of the cross-grantee meetings?
We would love it!
The issue is, not everyone has access to the
videoconferencing equipment. So, once again, we'll talk about this after the awards
are made.
24. The format and range of the RFA does not fit the
format of the typical PHS 398 grant. Has any thought
been given to allowing another response format?
At this time, we have not given any additional
thought to allowing another response format. However, format and range questions should be
directed to our Review Office, specifically Jerry
Calderone at JCaldero@ahrq.gov.
25. The RFA states that one goal is to avoid having senior
level staff perform tasks that a person with less
experience could perform. On the flip side, what is
the minimum level of commitment (% time) to CAHPS® II
expected from PI and co-PIs?
This is a judgment call. The level of effort to
be committed for PIs and Co-PIs should be determined by
the grantee's assessment of the project requirements.
26. Re: your comment on the 1/3 budget
distribution: are your providing any guidance about
what portion of the total budget should go to the
development of those 4 instruments?
You should review the requirements and make your
best judgment as to the resources needed for developing these instruments.
27. Two free registrations for the CAHPS® User Meeting
per grantee are awarded. How much is the registration
fee expected to be for other grantee reps?
The final determination has not been made regarding
the registration fee. For purposes of budgeting, assume $300.
28. Is "prospective questionnaires" equivalent to
longitudinal panel surveys? The RFA refers to "feasibility studies to determine
whether prospective questionnaires are desired by
consumers."
In the RFA, prospective questionnaires is used to describe CAHPS questionnaires that may be developed in the future.
29. Does the scope of
the work involve putting all three CAHPS® products into
one RFA? That is, is it the responsibility of one PI
to get teams together to address the work involved in
all three CAHPS® products (NH, group practices and
Persons with Mobility) into one grant proposal? Or, do you mean that a complete competitive
proposal is one that addresses one product (for example
NH CAHPS®) and its various components (instrumentation,
reports, QI and evaluation studies)?
An application must address all requirements of the
RFA and not be confined to just one area such as nursing homes.
30. What platforms (PC, Mac, or both) will be supported
for the 5 online workshops each year?
The online workshops will be Internet-based, and will support both PC and MAC platforms.
31. What is your anticipated award date?
Spring 2002.
32. What is the purpose of the steering committee,
and what prompted you to include it in CAHPS® II?
The Steering Committee (composed of the AHRQ
Project Officers, at least one representative from each of the organizations who
join us as funding partners, each Principal Investigator and their alternate) will
meet at least annually to consider work for the upcoming year and to decide which products will
be developed. During CAHPS® I, funding and/or research partners like CMS and
others approached us during different points in our work to discuss collaborations.
These partnerships were extremely productive and ultimately enriched CAHPS® products
and extended the project's "reach." But we felt that the work could be managed more
effectively and dollars spent more efficiently if we considered the universe of potential products
and tasks in a more organized fashion and at the beginning of each project year.
We hope that the Steering Committee allows us to do
that.
33. What start date should we use for
budgeting purposes?
Assume May 1, 2002.
34. It wasn't clear whether work on the reporting and
QI activities could begin in Phase I or whether you
intended those to wait until Phase II. Please clarify.
The reporting activities will begin with a research
plan that will span all 5 years of the project. So we need to start talking about that at the
beginning of the project. I think we covered your
question about timing of the QI projects earlier. So, please check the transcript and let us know if
you need further clarification. Also, take a look at section 5 timetable in the RFA.
35. Please clarify how we can get additional info.
The AHRQ Web site contains extensive background
information about CAHPS® and supplementary information that may be
useful in preparing a response to the request for proposals. It is not mandatory
to review this information. Information about CAHPS® I as related to the RFA can be found at
http://www.ahrq.gov/fund/grantix.htm and is entitled
"Supplemental Information."
36. Are applications feasible that address
subgroups of the topics and tasks? For example, CAHPS®
MI more than other components.
If your main interest is in addressing a subgroup
or only one of the topics, your best strategy is to
partner with a group that can address the broad range of
the work.
37. Can AHRQ give us additional information on the level of effort for each of the five objectives listed on page 1 of the RFA? Can AHRQ provide guidance regarding the relative level of effort expected on the "research" vs. the "template preparation" components of the reporting task?
While it is difficult to be specific about the balance of research and development activities within a specific contract objective, we can provide estimates of the level of effort expected for the five major contract components:
- Maintain existing CAHPS® products (10 percent).
- Enhance 3 CAHPS® products nursing homes, group practice and persons with mobility impairment (20 percent).
- Develop new products, including reporting strategies (40 percent).
- Assess usefulness of CAHPS® data for quality improvement (20 percent).
- Evaluate the effectiveness of CAHPS® in applied settings (10 percent).
Moderator: Thank you for participating in today's chat. Since we could not answer all your questions,
Chris and Chuck will take questions via E-mail.
Please submit them to CDarby@ahrq.gov and
CCrofton@ahrq.gov. We will posting any additional Q's
and As to the Web site.
Chris, do you have any closing remarks?
CAHPS: Thanks a lot! This was really interesting for us
and you asked some good questions. We're interested in knowing how this Web chat
format worked for you. Send us an E-mail and let us know:
CCrofton@ahrq.gov or CDarby@ahrq.gov.
Looking forward to your applications by November
14, 2001.
Current as of October 2001
Internet Citation:
Web Chat Transcript: CAHPS® II. October 2001. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/fund/tran920.htm
540 Gaither Road Rockville, MD 20850