July 22, 2009 (continued)

Afternoon Session 

Jeffrey Schiff: We are actually going to move now from this process around talking about what we mean by validity and feasibility, which I think I will compliment Rita on getting through in a fairly quick amount of time, to really talking about some of the specific measures that we scored before. Rita will explain how we are going through and talk about them with regard to feasibility and validity. And I'll just say in general what we did is we identified ones that were—I do not want to say that were on the cusp.

Rita Mangione-Smith: So first, we are going to look at the validity scoring for all of the measures, and like Jeff alluded to at the very beginning when he was going over the structure of our process, if we had a measure or measures that clearly did not make the validity cut, and to us that was 5 or below on validity—we are not going to be talking about them today. If there were measures that clearly made the cut, and to us that was a score of 8 or 9, we are not talking about them today. What we are talking about are the ones that got a 6 or a 7, and we are particularly drawing your attention to the ones that got a 6 where there was almost nobody among us who graded them in the lower four categories—so 1, 2, 3, or 4. So the first one, that 6.5 is a really good example. Out of—there were 18 people who returned score sheets, and of 18 people, only two people put validity in those bottom two categories. And you can see in category two and category four—and you can see it was at 6.5, and at 7, it would have made it in. So that is our rationale for how we kind of went through this. We looked at measures where there seems to be maybe a sense that although it did not make it, maybe it should have based on scoring.

Then the blue ones, same thing, it is a 6 or a 7, but a fair number of people put themselves in those lower categories. And by a fair number, we defined it as measures with a median of 6 or 7 where four or more subcommittee members ranked the measure in the bottom four categories. The pink ones are measures that had a median of 6, where two or fewer subcommittee members put them in the bottom four.

So the point of this exercise is to look at each of these measures and think about them as a group. If you strongly believe this is not a valid measure, speak up. If you strongly believe this is a valid measure, speak up. And then we are all going to go back home and re-score. So we are not going to come to a consensus over whether these should make it or not today, but we are going to hear what everybody has to say and then go back and do the Delphi process again just on these measures, okay, and all the additional measures that people have been kindly writing up.

Jeffrey Schiff: Rita, there are just some logistics issues here I just want to make sure people are aware of.

There are actually two sets of measures.

Rita Mangione-Smith: One is validity.

Jeffrey Schiff: One is validity, and one is feasibility. And I think as we go through them, if you just state why we included it—

Rita Mangione-Smith: I will.

Jeffrey Schiff:—why we included each particular one, I think that will help folks out.

Female Voice: Even if you do not want them to rate importance, it just is a very simple [cross-talking]—

Rita Mangione-Smith: So that is for tomorrow. We will be doing the Delphi here on importance tomorrow after we have established what our importance criteria are. Those are the measures that made it. They were the clear 7 to 9 on validity and 4 and up on feasibility.

And after you go back home and redo these for validity and feasibility, if they make the cut, we will then, through an e-mail, online process, get you to rank importance on these if they make it, okay?

Jeffrey Schiff: Right.

Rita Mangione-Smith: Does that make sense to everybody? Questions? Yes?

Female Voice: [inaudible]

Rita Mangione-Smith: So we need to discuss. Okay, so some people rated this measure, but you can see there are clearly not 18 scores up there. Some people said, "This measure does not exist. I'm not going to grade it." So can we—I'm going to make a suggestion. I forgot about this, Jeff, I do not have my little cheat sheet up here. We had a particular order we were going to go through this in—

Jeffrey Schiff: You bet. Right.

Rita Mangione-Smith:—and we are not going to actually go top to bottom because we thought some places might be easier to start to kind of cut our teeth on, and then we will move to the harder ones like that.

Jeffrey Schiff: Yes, we actually talked about doing the coverage one last because for exactly that reason. So I think what we would like to start on are the ones that are—the ones we wanted to start on were the acute and chronic conditions—

Jeffrey Schiff: Right. So if you will trust the way in which we looked at variation in reporting, then you really just need to look at the ones that have some sort of a shading on them and that the shading extends into the distribution score.

Rita Mangione-Smith: Which is treatment and management—

Jeffrey Schiff: Of acute and chronic conditions, and diabetes is the first one we have there.

Rita Mangione-Smith: Yes. So our first questionable one was diabetes management care. It was a question, although some States are doing diabetes measures, this was one of those measures where we were given really no specification. We are not sure it is being measured in children in any States, and we could also consider doing it as a gestational diabetes measure.

This is the frustrating one, I think, because we all know that there are valid diabetes management measures, but we have no idea what is meant by this. So it scored a 7. So a lot of people thought it was on the valid side, but it gets flagged because four people put it in the bottom four categories. And I do not know if that is because of the lack of specificity of what is written here or we, you know—so anyway, discuss.

Jeffrey Schiff: And our purpose of this discussion is actually to inform future Delphi voting. We do not have to—we are not going to come to a consensus agreement about these. I just want to make—

Rita Mangione-Smith: So Jeff, I'm going to count on you to call on people.

Jeffrey Schiff: Okay, you got it. Okay, let's do the 10 thing again, but I—Marlene is already raising her hand.

Marlene Miller: So I mean I guess I'll start, but if there are not any specifications, and no sense of it being measured in children, why are we even discussing it right now? We have a lot of other things.

Rita Mangione-Smith: I'm not positive it is not being measured in children.

Female Voice: It is being measured but [cross-talking]

Marlene Miller: But we do not know what the specs are. So when we do the environmental scan, we will see if anybody—or if anybody knows around the table whether it is being used for kids or not.

Mary McIntyre: I mean we are using the diabetes measures in children but we ended up—and this is the thing about specifications—we ended up involving what we call domain experts that will involve pediatric endocrinologists and others because of the whole issue with type 1/type 2 diabetes, which is one of the things that you are going to have to look at because based on the standards and when certain things are done is really based on how long they have had the diagnosis or time since diagnosis. So we ended up with consensus from the group of experts that we had an order in how we could put them in and what age to start looking at them. Certain measures, they were okay with the diagnosis regardless of whether it was type 1, type 2, or the age of the child. Other measures, they consistently wanted them to be 16 years old or older before we started looking at that particular measure. So that was the issue with these. Because it is being measured, I think—I know we are—but even within Medicaid in Alabama, we ended up into some issues about, well, where do we start measuring, what age groups do we look at, and then the type 1 versus type 2 issue.

Rita Mangione-Smith: Okay.

Female Voice: Well, that is terrific, and you know one of the things we wanted from everybody today was do you know of other measures being used or if these measures are being used. If you could send me that, the specs—

Mary McIntyre: We can. It is all part of TFQ (Together for Quality) and we can send it.

Female Voice:—because then I think we will have the information for our next round.

Rita Mangione-Smith: So when you go to re-score, we promise any additional information we get about specification between now and then will be included so that you can take that into consideration when you re-score.

Jeffrey Schiff: Denise asked me earlier, and I failed to do this—we need the input of this whole group and the public to identify the specifications for any measures that are out there, and you saw that this is really a communal effort to search or if you know about this, it is people's responsibility, please, as you nominate things or as you try to refine them, to send us that information. So it is part collection right at this point or during the interval as well.

Female Voice: There is no comprehensive set of pediatric [cross-talking] measures in use by Medicaid and CHIP or anybody.

Jeffrey Schiff: Okay, so more comments on this particular issue or measure?

Rita Mangione-Smith: Cathy.

Catherine Hess: Sort of riffing off this, I'm actually stretching the definition a little bit and asking a larger question because outside there is a handout from I guess the presentation we are going to have tomorrow on the National Quality Forum (NQF), and that there are currently 55 approved pediatric measures in various NQF sets. One of them is hemoglobin A1c for pediatric patients. So it does not tell you it is in use by the Medicaid or CHIP program, but it is an approved measure. It has technical specifications if it has been through this process. And I'm just wondering, are all of these 55 pediatric measures in our list somewhere or how do—?

Jeffrey Schiff: Nope.

Female Voice: Unless we got to know somehow through our triangulation process that they were being used by Medicaid and CHIP, they are not in here. So we would need—

Catherine Hess: But again, to look back, I thought the legislation was clear that they were in use anywhere, and it is not a criterion of used by Medicaid and CHIP.

Female Voice: But our guidance was to start with those in use by Medicaid and CHIP, which is what we did. Our internal guidance for practical purposes, primarily because since there is no comprehensive list of who is using what and what is out there, we would never get done by January 1st.

Rita Mangione-Smith: Okay, so I think that is a really—

Female Voice: [indiscernible].

Rita Mangione-Smith:—it is a good point and at certainly some place, it looks like we should think about going to NQF for their specifications, if they have specifications for these measures.

Female Voice: Yes. Well, they would not get approved through the process at NQF unless they were specified.

Rita Mangione-Smith: Okay, forgive me, I'm not familiar with the process.

Female Voice: Exactly.

Female Voice: [indiscernible] for all specs for all measures, that is needed.

Rita Mangione-Smith: Right. Great.

Female Voice: That might make our job easier with specification problems.

Rita Mangione-Smith: Absolutely.

Female Voice: Yes, it would.

Rita Mangione-Smith: The more we can get, the better. That is great. Who is next, Jeff?

Jeffrey Schiff: Cathy?

Cathy Caldwell: Well, I think Mary already brought up one of the questions I had, was this juvenile or type 1 diabetes or type 2, or both, and could they be split and—but then you said something, all right, it is on here but during pregnancy, and I did not know whether that meant, well, you know, you have got girls from, say, 13 on who get pregnant. And was gestational diabetes related to that group, or were you trying to say let's go back and look and see whether the mothers of children on Medicaid or CHIP had had a diagnosis of that, and was it managed? I did not know what to make of "during pregnancy."

Rita Mangione-Smith: Well, the question marks are to say that some States are using—and a number of States, and we have those numbers—some States are using a diabetes care management measure. That is all we knew.

The questions are, are they using it for children? Are they using it for pregnant women? And the pregnant women, because we did not get to the eligibility criteria by age, which was the question you asked earlier. But yes, girls on Medicaid and CHIP do get pregnant. I'm not a clinician or an endocrinologist. I do not know if they can get gestational diabetes or not. So yes, a lot of [cross-talking]

Female Voice: So that was what it was meant to be—

Rita Mangione-Smith: The perinatal measures.

Female Voice:—as opposed to the mother of children who might have had—in other words—okay, I want to rule out women who are not children who get gestational diabetes, but their kids are on Medicaid. You are not concerned with their mothers.

Rita Mangione-Smith: Okay, here is—the legislative language says, and I bet Marina had something to do with this, "services to promote healthy birth, and services to prevent prematurity."

Female Voice: Okay.

Rita Mangione-Smith: So it can be both the mother if she is an adolescent enrolled in Medicaid or CHIP or the mother who is on Medicaid and delivering in prenatal care or not, as the case may be.

Male Voice: Okay. Thank you.

Female Voice: [Cross-talking] And the CHIP program does have pregnant women over the age of childhood enrolled. So—

Rita Mangione-Smith: That is true.

Female Voice: But Medicaid [indiscernible].

Female Voice:—yes, but—

Rita Mangione-Smith: But basically any woman who is enrolled in Medicaid and CHIP who is pregnant could be measured with this measure.

Female Voice: Okay. That helps, yes.

Rita Mangione-Smith: Okay?

Female Voice: Okay.

Rita Mangione-Smith: That was the thinking.

Jeffrey Schiff: I think this one is particularly challenging because there is so little specificity and—

Female Voice: Right.

Jeffrey Schiff:—so before we get too far into this, sort of becomes an abstract conversation a little bit around this, but—George?

George Oestreich: That was a point that I was going to try to make, too, on this and several of the others. As a rolled-up opportunity, are we measuring it for the specific quality metrics of measuring diabetes care? We are measuring it and including children in those measurements. So for the purposes of the discussion, are we looking at it as an aggregate measurement of some type of quality metric or would we want to then have the level of specificity of which of those would be included as a reportable [indiscernible]?

Rita Mangione-Smith: I think based on our prior feasibility conversation before lunch that we would want to take actual measures that are being applied to children. So if hemoglobin A1c is being applied—

George Oestreich: Okay, so this should be broken down into four or five different metrics.

Rita Mangione-Smith: It needs, I think, in the next round before you guys score again, it sounds like there are measures out there that are being used that are much more specific than just saying diabetes care, and we will work really hard to get that information so that when you go to re-score it is there, okay?

George Oestreich: Right.

Rita Mangione-Smith: I'm going to move this on to the next one if that is okay.

Jeffrey Schiff: Yes.

Rita Mangione-Smith: Okay, so it will be Initiation and Engagement of Alcohol and Other Drug Dependence Treatment. That is actually a Healthcare Effectiveness Data Information Set (HEDIS) measure, so it does have specifications and there were five committee members who put it in the lower four categories, although it does make the cut. But it is one that we thought maybe you should have some discussion because a fair number of people graded its validity on the lower side.

Jeffrey Schiff: So—Xavier?

Xavier Sevilla: Yes, I think this measure, looking at it from at least a practicing pediatricians' standpoint, I think this is something that again lacks a lot of the specs, and what exactly are you looking at. We always screen for alcohol and drug use in adolescents, but we do very little in terms of the actual treatment in the pediatric office. So I just wanted to ask the group, you know, any more guidance on what this measure actually is looking at.

Rita Mangione-Smith: So it might be helpful if I actually read you the specs because it tells you sort of the numerator and the denominator on the measure. It is the percentage of adolescent and adult members on a health plan with a new episode of alcohol or other drug dependence who received initiation of treatment, and that is defined as percentage of members who initiate treatment through an inpatient alcohol and drug dependence admission, outpatient visit, intensive outpatient encounter, or partial hospitalization within 14 days of being identified as somebody who needs that.

The second part is the engagement—so that is initiation—engagement is engagement of alcohol and drug treatment is defined as the percentage of members who initiated treatment and who had two or more additional services with an alcohol or other drug (AOD)—that is what they call it—diagnosis within 30 days of the initiation visit. So it is basically looking for a first visit within 14 days and then two or more visits within the subsequent 30 days. So that is a little bit more specificity of what that means.

Jeffrey Schiff: So this is a health plan measure?

Female Voice: This is a HEDIS measure.

Jeffrey Schiff: Right.

Female Voice: Does anyone have any data on the link to the outcomes differences based on this? I do not know of any.

Male Voice: Do you have that?

Female Voice: Evidence.

Jeffrey Schiff: I think—Doreen, do you a comment about that?

Doreen Cavanagh: Hi, I was on the committee that identified and specified that measure for the National Committee for Quality Assurance (NCQA). So it was a measure that was developed by an expert panel. It falls into that level of evidence. There have been additional studies subsequently looking at the feasibility of it. The feasibility is really easy because it is a claims-based measure. There is no question about the fact that you can do it, but the validity has also been examined primarily by researchers at the Heller School at Brandeis University, and there are papers available to look at which I can produce. Again, most of the research has been done on adults, but some research has been done on adolescents.

Rita Mangione-Smith: Are there any other comments on this measure before we move on?

Jeffrey Schiff: I had a question for Doreen. Are there results from the studies that had been done by the [indiscernible] from the school, from Brandeis, as far as the linked outcomes or—?

Doreen Cavanagh: I do not believe there is a link to outcomes for the adolescents' paper as far as I'm aware of.

Jeffrey Schiff: Okay. Is there a link for adults?

Doreen Cavanagh: This is sort of a commonsense measure. It is a commonsense measure inasmuch as it was primarily designed for specialty care—to answer your question. It was designed to look at if someone comes in to specialty care and gets identified with a substance abuse problem, does that youth initiate treatment because our big problem in substance abuse is just getting somebody to start and stay in treatment. So it really is that basic. It is if you walk in the door and you are assessed and you are identified with having a substance abuse problem, do you get to the first visit? And if you do, do you get two followup visits within 30 days?

In the substance abuse field, things are very, very basic. We have a long way to go in substance abuse treatment in my opinion, and this is sort of where we are starting. This is our benchmark of at least getting you in the door and getting you engaged in treatment. And I would of course strongly advocate for it. It is the only measure for substance abuse in our set, that is number one, but it is a measure which a national expert committee who met for 2 to 2-1/2 years decided that this is where we would start. And also the NCQA accepted this measure as part of their HEDIS set.

Rita Mangione-Smith: Which means there was some degree of assessment as far as its scientific soundness, its meaningfulness, all of the HEDIS attributes. The CPM, the Committee on Performance Management at NCQA, looks at all candidate measures that are developed and kind of puts them through a pretty rigorous look in terms of those desirable attributes. So I would think that at least for the adult population, they must have looked at the evidence base, and there must have been something that convinced them that it was scientifically sound.

Female Voice: Can you clarify for me what you meant when you said this was based—you said something to the effect that this was built for specialty care? You said this was built, this measure was intended for specialty care. So are you meaning that if you go to see a substance abuse provider, this is the followup for that? This is not Xavier's followup.

Female Voice: That is correct.

Male Voice: No.

Rita Mangione-Smith: So it is requiring that somebody actually go see a specialist?

Female Voice: It would not preclude this being used in primary care, but the purpose of it when it was designed was the group's thinking was looking at the specialty care sector because we have so many youth who come in and have one contact, and we never see them again in the specialty care sector.

Jeffrey Schiff: All righty, Cathy, and then Phyllis, and then Mary.

Rita Mangione-Smith: And then we are going to have to move on.

Cathy: One generic question. There are essentially four measures buried in this. Initiation is one, engagement is one, alcohol is one, and whatever goes into that drug category. So I do not know whether in some cases like this, we do better to see four separate things spelled out rather than such a conglomerate that would be measured in different ways. So that it is kind of a generic question about these aggregate measures that might be hard to think about when some things would be more feasible than others and so forth.

On the substance about the specialty care and all, I did not pick that up. But what would happen in circumstances in which a Medicaid provider picked up something like this and then referred somebody to what amounts to a private or like a county or a city program for, say, alcohol dependence or something like that, and the referral was made, and for all we know it was a terrific success in the sense that somebody stayed with it but is completely outside the Medicaid and that system or the State Children's Health Insurance Program (SCHIP) system. And so that is why I was wondering whether separating out initiation from engagement and defining engagement might be useful for us. I do not know.

Female Voice: Can I respond to that? In the NCQA measures set, there are two measures. Initiation is a measure by itself, and engagement is a measure by itself. However, we accept any alcohol or drug diagnosis quote.

Cathy: Okay.

Female Voice: So we have split it by two rather than by four.

Cathy: Okay, got it.

Female Voice: So Denise, maybe when we put these through for the second round, we can separate those out into two measures.

Jeffrey Schiff: But I think to be clear we are severing out the two measures, we are not separating out the other ones.

Rita Mangione-Smith: That is right. It sounds just like two, initiation and engagement.

Jeffrey Schiff: Yes.

Cathy: That is fine, mine is a generic question.

Jeffrey Schiff: Okay, I think Phyllis was—

Phyllis Sloyer: Yes, I'm going to follow up on the comment about the provider not necessarily being in the plan's network. We actually, in our reform sites, are collecting this information. It is very difficult. It is not claims-driven either. Much of this is a carve-out, capitated, and frequently the treatment is not necessarily within that carve-out. It is referred to a county organization and the treatment is there. There are issues of sharing of information. So this one, because I know we collect this, it is very difficult.

Rita Mangione-Smith: Okay, I think we are going to have to close comment on this one because we have a fair number more to get through, but thank you for all the comments.

Jeffrey Schiff: Right.

Female Voice: Can I just say one thing?

Rita Mangione-Smith: Oh, sure go ahead.

Female Voice: I need to at least be clear. If we are going to be re-voting on this, are we saying this is for specialty people or there is some sense that we are trying to broaden it to be everybody, which is not the intent of NCQA?

Rita Mangione-Smith: Can you clarify for us? Thank you. Sarah is from NCQA.

Sarah Hudson Scholle: Good afternoon, I'm sorry what is the question?

Rita Mangione-Smith: For the initiation and engagement of alcohol and drug dependence treatment, is the spec written such that any outpatient visit within 14 days counts, or does it have to be a visit to a subspecialist who deals with substance abuse?

Sarah Hudson Scholle: For subspecialists, these measures—remember HEDIS is for health plans.

Rita Mangione-Smith: Right.

Sarah Hudson Scholle: The measures that you are looking at are HEDIS measures that are specified for reporting for the population that belongs, that is enrolled in a health plan. And so this measure, the initiation/engagement measure, is looking to see whether people that have these identified problems get into care and stay in care for a certain amount of time. This is not a measure. We may have specified this for physician- or practice-level reporting, but I'm not sure about this one because it would apply primarily to people that are being treated in a specialty setting.

Now our other measures in general have been specified for physician-level reporting—so, for example, the immunization measure and other measures. And I think that might be something that needs to be clarified. Is this reporting for the population or is it reporting at a practice- or physician- or organization-level? That might be helpful for some of the discussion?

Rita Mangione-Smith: Right.

Jeffrey Schiff: Thanks, Sarah. Okay—

Rita Mangione-Smith: So I think, just so that we are clear, it is obviously for specialty care, and I think this particular measure is, right?

Jeffrey Schiff: Right.

Male Voice: You know, I just went to the AHRQ Web site and looked at the National Quality Measures Clearinghouse information, and they point you to an article that actually looks at primary care treatment, and that it is successful in a brief intervention. So I would be careful about just constraining this to specialty care because—

Jeffrey Schiff: That will be a different measure then. That will be a different measure to nominate, I would think.

Rita Mangione-Smith: Well, yes. If we are going to go with this specification as it exists, you are right.

Male Voice: But should it not be initiation of treatment by any kind of health care provider? Is that not the quality standard we are trying to get at?

Female Voice: We would probably try to get at, and what is actually available, are two different things. So, on the wish list we might want to put provider—

Rita Mangione-Smith:—provider measure.

Female Voice:—primary care provider.

Rita Mangione-Smith: Yes.

Jeffrey Schiff: Okay.

Rita Mangione-Smith: Right.

Jeffrey Schiff: Yes.

Rita Mangione-Smith: Okay, I'm going to move us on to the next one because we are—yes [cross-talking] much time and we have a whole lot to get through, okay?

Jeffrey Schiff: Right.

Rita Mangione-Smith: Again, just take all this in and when you go to re-vote, you have got the information that you are hearing.

Jeffrey Schiff: Right.

Rita Mangione-Smith: And certainly some new information that we will just be providing before you do that for the next Delphi.

Female Voice: Well, the more you can provide, the more we can provide.

Rita Mangione-Smith: The better, right. So dental and oral treatment under that one, there is—I'm not sure—months? Is this supposed to be members? Members who received any dental treatment other than diagnosis and preventive services in the past year?

Male Voice: It is actually this and two of the other measures that are in here are actually part of the CMS-416 Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) reporting requirements, and so they are the specifications, the total the number of eligibles who receive something beyond diagnostic and preventive services. And the other two related sort of measures are the percent of eligibles who receive any dental service in a preceding 12-month period and the percent of total eligibles that receive any preventive dental service.

Rita Mangione-Smith: Got it. Okay, so as we come to those, we will maybe try to correct the language that is clearer.

Male Voice: Okay. And just related—I notice you have all 50 States reporting on those three measures because they are part of the CMS-416 and so it—

Female Voice: It is EPSDT [indiscernible] reporting.

Male Voice:—exactly, yes.

Jeffrey Schiff: Thank you.

Return to Contents
Proceed to Next Section