On July 9, 2009, Ernie Moy, Foster Gesten, and Keely Cofrin
Allen presented a Web conference on the AHRQ 2008 State Snapshots
Tool. This is the transcript of the event's presentation (Part
2 of 2).
July 9, 2009
1:00-2:30 PM ET
(Continued)
Margie Shofer: I think that is about our time for questions
for now, but we will get to the rest of these questions, or try to get to as
many of them as possible, in the next Q & A session. It's now time for
us to move on to the next presentation, I'm going to turn this over to Foster
Gesten.
Foster Gesten: Thank you, Margie, and good day to everyone.
Probably after the last presentation, it should be clearer why and what the
opportunities were for us in New York, and I would say two things. One is that
seeing Ernie go through the demonstration of some of the new features, I think
that it's, to use a clinical term, "way cool." Two, in terms of
our process, when we first started looking at these, we had to go through the
steps of denial and anger about what the data was. But marching through those
steps we found it to be very useful in terms of our thinking about how to prioritize
and that's what I'm going to talk about a little bit and how we used it in
a specific reform effort in Medicaid.
Like other States and other folks, we do a lot of measurement and a lot of
public reporting specifically related to health plans. We also do benchmarking
to national benchmarks to the degree that they exist for HEDIS and other databases,
but we really lacked and welcomed the idea of being able to compare and look
at how we're doing as a State.
Our use of the State Snapshots this past year is within the context
of fairly aggressive and comprehensive Medicaid reform over the past 2 years
in which we've been looking at moving investments from the in-patient to the
out-patient ambulatory care side. The reason will become clearer when you look
at some of the data, but the history was that there was a tremendous imbalance
and the consequences that one might expect from that imbalance.
Our broad objectives were to promote broad promotion of primary and preventive
care focusing on those services and those programs that had high value. We've
been using the Prevention Quality Indicators [PQIs] in other contexts, including
other grants that we had, in which we were able to see where New York was in
comparison to a few other participating States. We understood that we had some
issues and some challenges relative particularly to preventable hospitalizations
for folks with chronic disease and most especially, asthma and diabetes. So,
the State Snapshots and the use of those for benchmarking came at
a serendipitous time for us as we were looking at how we start to turn this
around and how specifically we wanted to create support for a new benefit related
to certified diabetes and asthma educators.
I deliberately decided to keep the 2007 data and not update this, as you saw
the updated data from Ernie, because this was really the information we were
looking at as we made our decisions. We looked at where we were in terms of
all measures and saw that we were average, and this is based on 2007 going
in the wrong direction. You can see where we are in terms of a regional comparison
average and average is better than below, but I think nobody strives to be
average and certainly we didn't.
This slide really shows the Dashboard in a different configuration than Ernie
showed previously, but again, this helped us to think about the areas that
we specifically wanted to focus on and that were problematic for us. Using
this data, we clearly found support for focusing on areas related to chronic
care and both hospital and ambulatory measures in particular.
The next slide shows it was no accident that we focused on asthma and diabetes
given where we were in terms of performance in these two clinical areas, with
respiratory diseases counting for more than asthma. There are a significant
number of measures where we are below the national average in the area of asthma.
This is just another way of looking at it or getting underneath the hood about
which are the weakest measures. Looking at this we clearly knew we were in
the right direction in terms of focusing on hospitalizations and the potential
for avoidable hospitalizations around diabetes and asthma and the data that
Ernie showed goes along with this.
This is my one defensive slide to show that actually there is at least one
area in New York in which we're doing well. Our challenge, as we thought about
this, was that we had large health care investments in New York in public programs,
but in health care in general we had average to weak results. It corroborated
other national reports, such as those from The Commonwealth Fund and elsewhere,
that showed in many ways we're in the middle of the pack and it didn't jive
with where our investments were.
We saw specific gaps in chronic care related to diabetes and asthma and particularly
focused on preventable admissions, and although the data is not Medicaid specific—most
of the data isn't specifically Medicaid—we still viewed this as relevant
in looking at the entire population. We see that some of the newer reports
do have more of a focus in looking at disparities or looking at low income.
Despite some of those limitations we were able to really use this information
to advance the argument to all of the folks that we had to convince, both in
the executive and the budget and legislative arenas, to support the specific
new benefit related to diabetes and asthma.
Again it was particularly important for us to gather this information and
to summarize it. We knew at the time that the clinical evidence for some of
these benefits is less than clear, but the State Snapshots and the
benchmarking data allowed us to really focus and understand that this was clearly
a priority area in which we needed to act. The benchmarking helped us focus
in areas and prioritize the data that we had relative to disparities, which
again pointed to the direction of a specific need for low income population
Medicaid and the potential, even though at the time we did this there were
not specific calculations. At that time, relative to costs for Medicaid, we
knew from other data that reductions in preventable hospitalizations and improvements
in care had major cost implications as well, so the combination of being able
to use the data for that was extremely useful.
Just to wrap up, I'll mention some of the next steps for us. Part of
this is really a conversation l with folks at AHRQ and a better understanding
of what is behind the differences on measures. Somebody asked about whom to
contact, and the reality I think in our State and probably for other States,
is there is not one person but teams of people throughout State agencies that
might be able to give some information. Trying to figure out a way to organize
the information, how to understand why, in the examples that we saw, we see
such big differences between nursing home care between Utah and New York, is
a struggle. Is it contextual factors? Is it programmatic features? Are there
benefit issues? I think many of us struggle with trying to understand what
has gone into this when we see the differences. We try to understand the relationship
between measures and the contextual factors. Which of these measures relate
to one another, which are either measuring the same thing or different things
and what is the relationship between some of the structural or process measures
and some of the outcomes? In terms of prioritizing action, it is important
to try to sort out which of the measures really had the greatest impact. I
think some of the cost calculations are going to be very helpful.
I'm pleased to see in the newest iteration that there's much more integration
of disparities data in the State Snapshots. I think it's a very
important area and dimension to keep in mind as States start thinking about
the important priority areas. As you can see, some of the data that was shown
for New York relative to Black, White, and Hispanic-White differences presents
some opportunities for us. Lastly, a question was posed by someone in the audience
about being able to customize peer groups. Certainly we looked at national
averages, we looked at regional averages, but there are certain States that
we look at and think of as being similar peer groups in terms of their population
mix or contextual factors that Ernie went into, based on size or type of benefits
or programs. It would be helpful to be able to create that sort of a menu in
which one could compare and have averages across whatever might be a self-defined
peer group.
I think that's the last slide, and I'll be happy to answer questions at the
end of the presentation. I'm going to turn the presentation over to my colleague
from Utah, Keely Cofrin Allen, Director of the Office of Healthcare Statistics.
Keely Cofrin Allen: Hi, thanks. I am Keely Cofrin Allen and
I am the Director of Health Care Statistics out here at the Utah Department
of Health. Good morning or afternoon depending on where you are. I'm looking
forward to sharing what Utah has been doing with all of you on the call today.
Just a little bit about Utah. We have about 2.5 million people in the population
and 76 percent live along the big lake that you see on the map, on the Wassatch
Front. We have a young population and the highest birth rate, so that affects
our health measures. And we have about a 10 percent uninsured rate, Medicaid
enrolled about 240,000, and median household income puts us at 13th overall.
The good news is that we have a relatively young and healthy population, smoking
and drinking are not as common here as in other States; however we do have
our share of issues around obesity given our that obesity rates are increasing
faster than the national average. That's something we need to keep an eye on
as well.
A little bit about our office. We were brought into being in 1990 by the Health
Data Authority Act. This act formed the Health Data Committee and our office
is staff to that committee and I'll tell you a little bit more about that committee
in a moment. The purpose is still very relevant today even though we were formed
18 years ago. We have a long history of doing quality health data analysis
work.
Here is just a quick look at our Health Data Committee. It has 13 members
who were appointed by the governor's office and who serve 6-year terms. We
like to talk about the five P's in our office to conceptualize our customers.
Providers and payers were the very first users of our data that purchase our
data set for internal analysis, patients and purchasers have been the focus
in more recent years, and policymakers are really our newest users as we navigate
this very complex area of health care reform that is of interest both at the
national level and at the State level. I'm going to be focusing on policymaking
in this presentation today, but I just wanted you to see that we do have a
variety of consumers who use our data.
As I said before, we were established in 1990 as staff to the Health Data
Committee. We began collecting hospital in-patient data in 1993 and in 1996
began collecting data on ambulatory surgery centers and emergency departments.
That year also saw us beginning to report HEDIS and CAHPS data. We're one of
the earliest States to do many of these things and this year we'll become one
of the first States to report CAHPS data PPOs [Preferred Provider Organizations]
and our newest project is the all-payer database. We'll be creating a State-wide
database of claims and enrollment data that allows us to look at cost of episode
of care. We're in final testing and we're expecting to report on that beginning
in the fall.
In 2007 we released the Challenges in Utah Health Care Report—you can
see a screen shot of that on the slide—which was a policymakers' manual
for the State of the State and health care. We see our job in this office as
presenting the big picture to policymakers. They need to understand two important
issues: where does our State stand relative to other States—both other
States in the Nation and other States in the Mountain Region—and the
direction that we're moving. Are we getting better on things or worse on things?
Obviously the health care system is complex so we need to tell a story that
includes multiple health settings. It could be about Medicaid or hospital systems,
home health care which we do very well in, wellness and preventive care, which
as you saw we don't do quite as well in, or primary care.
It's very important for States to track the health priorities in their particular
State and tie their message to those, since that's what policymakers will be
tracking. For us, those will be the priorities of the Department of Health
Director, the governor's office, and the legislature. Ideally they will be
on the same message but they might be slightly different and in any case, they
are going to approach solutions to the problem differently. They play different
roles so you have to tailor your message to fit the role and the approach that
particular policymaker is going to be taking. And lastly you want to make policymakers
aware of emerging issues. We see it as our job in this office to track things
not only at State level but at a national level and this gives us an idea of
things that might become State priorities down the road. When policymakers
come to us asking about this new thing we can have something ready for them
to understand the emerging issue.
We released the Challenges in Utah Health Care Report on the same
day as AHRQ released their National Quality Report, which came out
in June 2007. We did a joint release with AHRQ and it generated a lot of interest
in Utah for national ranking. We presented where we are as well as where we
are going on a variety of measures related to cost, quality, and access.
The report indicators that were included in that challenge were 16 summary
indicators. Some were from State Snapshots, some were from other AHRQ
tools such as HCUPnet (the Healthcare Cost and Utilization Project), and some
were from a special analysis that was done with in-kind assistance from Anne
Elixhauser from AHRQ.
Here is Utah's overall performance. This is the dashboard meter that we used
in the report so you'll see us being in the "strong" category,
which is where we were when that report came out. As you can see in the more
recent demonstration that Ernie did, we've slipped a little bit. This is a
really nice dashboard measure and something that policymakers like. It's easy
to understand, it's clear, and it's memorable. It's very visual, it's got colors,
and it ties into things that people already understand with the green to the
red. This is something that when you have literally 30 seconds with the legislator,
you can focus and give them something that they can use as a take away message.
We highlighted strengths and weaknesses. These are dashboard measures from
last year, so low performing measures can be targeted for improvement, average
measures should be watched for trends that may be going up and may be going
down. Average is not always bad if it's an improvement from last year. Of course,
as you saw earlier, the home health care measures are things that show what
we're doing right. It's important to distinguish those things that you're doing
right because of process—and I think home health care falls into that
category—and those that look good perhaps because of demographics. Because
of our young age, we tend to be lower on measures of cancer, for example, and
that's not necessarily something that we are doing right. It may be something
that has to do with our demographic.
Here is a table that we included in the challenges report. Notice that we've
organized the information to help readers absorb it quickly: three categories
of quality, access, and cost. Those are words policymakers should be familiar
with and trends show measures that are steady in this particular table. We
had three tables: one for improving measures, one for steady measures, and
another for measures going down, and we included a page number so readers could
quickly flip to the measures of greater interest.
The same year, The Commonwealth Fund released its Aiming Higher report,
and you can see our rankings on this. There are two interesting things about
Utah in this report. First, Utah was ranked much lower in this particular report
than in any other report, including the State Snapshots, and second
with very large variability in rankings across measures, Utah was the only
State with rankings that were this disparate.
Here is our Department of Health Director, Dr. David Sundwall. He led an investigation
and he came to me and asked the Office of Health Care Statistics staff to look
at the data used for the measures both by The Commonwealth Fund and in the State
Snapshots to really deconstruct them to give him and other people an idea
of the methods used to collect the data and the indicators and how they were
put together. He convened an informal meeting of the Utah Medical Ethics Committee
in August of 2007. We met at the cabana at his condo and had a pot luck and
just a chance to discuss the utility of measurements of health care quality,
how best to incorporate what we learned from these sorts of rankings of both State
Snapshots and other ranking reports that come out.
The summary from that meeting was that there is an important difference between
outcome in how the patient fares in the setting and the process, how the care
is delivered. If you measure at different levels you'll see different things,
so it's important to look at both and see how they interact with each other.
For instance, if the outcome you get doesn't match the process, one is good
and one is bad, this is a good indication that perhaps you don't have things
linked up in the way you'd like to. We need to continue to work on validating
measures so we're certain we're measuring what we think we're measuring: We
can't take for granted that we are.
There were several questions about measurement, where the measures come from,
and what the baseline measure was. I really encourage everybody who wants to
use the State Snapshots tool to drill down into those tables, really
see what year data we are looking at and where the data come from, and take
the time to go through the tables. In our office, we are very happy to refer
to ourselves as data geeks, and we really like crawling through the tables
in that fashion and getting a deep understanding of the types of things that
can be found in there.
So the take home message from Utah is that first you have to be in touch with
your policymakers to understand their priorities and interests, otherwise you'll
look irrelevant and be confusing and give them information that they don't
necessarily want or they aren't necessarily tracking at this particular time.
It's important to really speak back and forth with your policymakers and understand
what their priorities are. Second, is to fully explore the Snapshots for
your State as I already said and make sure you or somebody who has a data background
explores the methodology behind each measure so you can respond to questions
or concerns from policymakers. One of the questions we got over and over was
why we look good on this report and not on this one, but if you drill down
and look at the individual measures, the same message comes out of all of them.
We do very well here in Utah in home health measures, we're not doing as well
in nursing home measures, and we need to improve there. We have preventive
care weaknesses and cancer screenings and prenatal care, and when you looked
beyond just the summary which is useful, you saw that the same data emerged
over and over. That was comforting to policymakers because the initial anxiety
of why we look different here versus there was eased. We were able to say the
measures were put together differently, the composites are constructed differently,
and that's why we're getting a little bit of what looks like variation but
the same message comes through.
I also encourage you to work with the AHRQ staff and play with AHRQ's other
tools such as HCUPnet so you can get additional data that's of interest to
policymakers in your State. We have found in Utah AHRQ staff can be very helpful
in receiving feedback on measures and also to help us construct some additional
measures that would be of use to us. With that I'm going to turn it over to
the question slide and turn the presentation back over to Margie.
Margie Shofer: Thanks, Keely. We have the remaining time
for Q & A.
Question: Does AHRQ have any plans to show metrics broken
down by public payer versus private payer versus total State?
Ernie Moy: Well I guess that's for me. Let's say that we
have an interest and we actually do have a couple of document reports that
focus on different payers at the national level. It's something that we are
interested in and we could probably receive some additional push to go down
that pathway, with a couple caveats. When we look at different payers, payer
status means something very different in the survey information that we gather
from patients (What kind of insurance do you have?) than it means to another
big contributor to the State Snapshots, hospital-based information,
where the question is about the primary expected source of payment. We observe
that the two often do not line up very well. If people could live with that
caveat, that difference in definition, it's something that we could potentially
explore.
Question: How are the measures selected?
Ernie Moy: That's pretty straightforward. The measures that
are used in the State Snapshots are a subset of all of the measures
we use in the National Healthcare Quality and Disparities Reports.
They are pretty much all measures for which there is State information. The
measures in the Quality Report and Disparities Report were
selected through a very elaborate process which occurred about 7 years ago,
when we were first asked to do the National Healthcare Quality and Disparities
Reports. That process included getting recommendations for measurement
of quality from multiple different organizations and an Institute of Medicine
committee giving us guidance. As part of that, they took public testimony.
There was a Federal Register announcement and there were some stakeholder
meetings held around the country to get input about quality measures and recommendations
for quality measurement. All those quality measures were then compiled into
a mega-list and a Department of Health and Human Services group then
went through that and honed in on those they thought were the most significant,
the most scientifically sound, and the most actionable, and that constitutes
the core set that we track in the Reports as a whole. Again, the State
Snapshots are that subset for which there are State data.
Question: We have a question for New York for Foster. Can
you explain how you use the AHRQ data to secure payment? Did it result in legislation
and if so please explain the process and what the legislation required.
Foster Gesten: Sure thing. I should have made that clear.
The end of the story was that the information that we had as backdrop to support
defining the need and why we needed to invest in this area did result in legislation.
That allowed us to add to the benefits for the Medicaid program certified diabetes
educators and certified asthma educators, with an evaluation component. We
are intent on being able to evaluate both the uptick and also the eventual
impact of this. It may take some time, particularly in the asthma area because
of the dearth of certified asthma educators. We used the data essentially to
craft our argument for why this was necessary and what the value was of the
interventions and how it related to important priorities for New York.
Question: I have a question for both Foster and Ernie. This
is from someone in Maine who asks, "Our work in Maine shows remarkable
in-State variation. Most health care is local, regional, or at a sub-State
level. I'd like to see sub-State breakdowns of much of this data. Is that possible?"
Ernie Moy: Okay. Again, there is a great deal of interest
and very little data or at least very limited data but we'll take that as additional
encouragement to move down that particular developmental path. We agree when
we look at sub-State variation, it's enormous. It swamps the State variation,
and so I think that it's a very important area. It's more a matter of how much
we can say at the sub-State level.
Foster Gesten: All I have to add is that I agree and we see
the variations in New York just as you see in Maine. However, sometimes when
we adjust for other factors the regional differences are explained by things
other than the region. It's certainly helpful for us at the State level to
be able to dig down once we see there's a problem and try to understand whether
it's population driven, whether it's geography driven, whether it's insurance
type driven, and so on. And very often the answers to all those questions are
"yes," "yes," and "yes," as
well as unknowns about why the variation exists.
Question: How is "no data" handled? Is it simply
not included in an aggregate measure or is the aggregate measure discounted
somehow for missing data?
Ernie Moy: I think that it's just considered average and
therefore would not influence your score.
Question: Is there a section where you track medical errors?
Ernie Moy: Yes, actually, one of the new additions to this
year's State Snapshots which I probably should have emphasized
more was the addition of measures related to adverse events. You can find them
under "hospital," and they relate to the Patient Safety Indicators
developed by AHRQ. These are things like number of respiratory complications
during surgery, numbers of post-operative sepsis, etc. If you look under the
hospital section you'll see these new adverse events that we track in the State
Snapshots.
Question: Is there a way to evaluate whether the changes
seen between baseline and current are just noise or standard variation versus
some real change?
Ernie Moy: I think there is some variability in terms of
what is tracked in the Reports. The things that I would typically
look for to see if a change is real or not real is to look at the individual
measures and if you see a number of related measures headed in the same direction
then I think that might be real. If, on the other hand, I see one measure that's
driving that dramatic change then I would potentially say, maybe that's just
some problem with that particular measure for that given year. I think the
answer is "yes" but look at it on a case-by-case basis.
Question: Who is the right contact person for the software?
Ernie Moy: It depends on what you want to do with the software.
I guess if it's a content question about the State Snapshots then
I'm the appropriate contact person. If you're interested, as some States in
the past have expressed, in developing a system like this and wanted to get
into the nitty gritty of the XML and the programming that's involved with it,
that would be our contractor, Thomson Reuters, and Rosanna is the contact.
Question: Was there a baseline description as to what the
ideal State health care would be?
Ernie Moy: Again, because everything in the State Snapshots is
defined in relative terms, I don't think that there is a standard absolute
definition of optimal health care. Rather, what would constitute optimal health
care for each measure in the State Snapshots product would be the
State that performs the best for that measure, either having the highest rates
of desirable care or the lowest rates of undesirable care, but it's a theoretical
construct. No State even comes close to that.
Question: When a specific type of insurance is not mentioned
should we assume that the data is a sample of all types of residents: Medicare,
Medicaid, private, and the uninsured?
Ernie Moy: Yes, that would be the appropriate assumption.
There are potentially some restrictions, Much of our survey information relates
only to noninstitutionalized individuals but besides that, it should cover
all payers.
Question: Are the rates reported for the various measures
an aggregate representation of those who have different types of coverage and
are uninsured? How is data from the NCQA [National Committee for Quality Assurance]
used for the Snapshots if at all?
Ernie Moy: I think the first question was similar to the
last one, so the rates represent all payers unless it's specifically stated
otherwise. In terms of NCQA data, some of the measures that you'll see there
are HEDIS measures coming from the HMOs and those relate to patient centers
and are measures of patient provider communication but we don't use any of
the clinical HEDIS measures in the reports.
Question: Is there any plan to incorporate a rate of change
calculation, for example, change over year, since there are no different base
years?
Ernie Moy: We have not thought about doing that, let's put
it that way. We find that rate of change tends to be very variable. Just learning
from the national report, there is some bouncing around over time. We're actually,
this year, moving to regressing it to try to establish rates of change as opposed
to just using difference between 2 years. If you are really interested in this,
you can write to us and encourage us to do it and it might be something that
we'll pick up. I am concerned about the variability that often exists from
year to year and quantifying that as a rate of change.
Question: Do you provide insights into how States should
utilize the various State health care report cards such as The Commonwealth
Fund, United Health Foundation, etc? How do States decide which is best and
most applicable?
Ernie Moy: Well I can put in my two cents but maybe I'll
defer to my colleagues from the States to see how they use it. From my perspective,
more information is better and as you know, each of the State report cards
that exist have their strengths and weaknesses; there are certainly differences
among them. I think when there are areas of agreement and we match up with
them—and we see that there are typically more areas of agreement than
disagreement—that gives the greatest motivation and greatest confidence
for a State to invest in a particular area. That being said, I can understand
State policymakers do not like to see conflicting data and I'm interested to
see how New York or Utah deals with that issue.
Foster Gesten: This is Foster from New York. For better or
for worse, I think there is a great amount of agreement between the two major
report cards. I think the biggest area of disagreement we have is if we focus
only on the managed care population. Our numbers look a whole lot better. When
trying to jive why it looks better inside NCQA numbers versus other ones, again,
the devil's in the details of exactly who is included in the numbers
and what rates and what years and so on. But our experience has been that in
the areas that matter in a broad way, the arrows tend to be going in the same
direction. We have not had the experience that was described earlier from Utah
where we had one report that said we're wonderful and another one said that
we were mediocre or challenged. Both of them showed similar areas or problems.
In Ernie's camp, more information and more data are generally better and you
look for those areas where there's great overlap and consistency.
Question: Do you have data on eye diseases such as cataracts,
glaucoma, etc.?
Ernie Moy: I think the only measure that we have is the diabetic
screening, the diabetic eye exam measure. That's the only measure that we have
that has State information. In the Report itself we do track a couple
of measures related to vision but we don't have State information for that
particular measure.
Margie Shofer: I see that a couple of people have asked if
we're going to have a recording of this and the answer is "yes," and
you can send an e-mail to an address that will be on the very last slide if
you'd like to be notified when that's available. As you know it takes us a
little bit of time to put that up. Otherwise, we'll probably be sending out
an e-mail letting folks know when that's up, generally through AHRQ's .gov
delivery e-mail list, the list which was used to notify some of you about this
Web conference. Also, those of you who have been part of our ongoing technical
assistance for the State Quality Tools project, we'll let you know through
that list.
I think we are at the end of our questions and the timing is perfect, because
we are at the end of the Web conference. First of all I want to thank our speakers
who did a fabulous job, thanks Ernie, Foster, and Keely. And I want to thank
you, the audience, for your great questions, and your participation in this
Web conference. I think that we answered everybody's questions but if
for some reason we didn't get to your question, we promise we'll answer it
over the next few days. We hope this discussion was helpful to you, and if
you do have a question about follow-on technical assistance opportunities,
please do not hesitate to submit them to the AHRQ quality tools e-mail address.
You see the e-mail address on this slide here. If you have any questions or
comments about the tool, please send an e-mail to the same address.
Also, after the event closes, you will see an evaluation, so if you have a
few minutes, please fill that out. It's really important; it really helps us
know how this presentation helped you and helps us plan for future events.
So, thanks again, and this concludes the Web conference and we look forward
to hearing you from. Goodbye everyone.
Return to Contents
Internet Citation:
Web Conference: 2008 State Snapshots: Transcript. July 2009.
Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/kt/webinars/stsnaptrans0709_pt1.htm
540 Gaither Road Rockville, MD 20850 