Technical Assistance Conference Call Summary (continued)

Coordinator: Michael Rupp?

M. Rupp: I wanted a little bit more clarification on the role of the outside independent contractor that will be doing the evaluation. The RFA states—I know we already talked about this but, again, it's unclear to me. The RFA indicates that applicants must offer a proposed evaluation methodology and even a proposed cost for that evaluation. Presumably, that refers to the evaluation that the outside independent contractor will employ? Is that correct? I'm kind of at a lost to propose both the budget, as well as the methodology that an outside evaluator will use or should use to apparently pull together the results of, for example, my project and all the other projects in this report to Congress.  So I still don't understand that person's role in what the RFA is requesting of us with respect to that person's role.

J. White: That's a great question. The answer is we know that this information that is generated under the work from these grants will have to be brought together and presented to Congress in a report. Therefore, applicants need to think about their work in that context. The way we're asking to do that is to think about that if you were to try to pull together information from these grants, how would you do it and how much do you think it would cost? I don't think we're necessarily asking for—we're not looking at auditing your tax returns or anything along those lines. I don't want applicants to give us wonderful applications that don't have any thought given to how this information then feeds into what the ultimate product of this must be, which is a report to congress. So that's the reasoning behind having that requirement in there. And it is a requirement that's in there so it has to be done.

M. Friedman: This is Maria. We're trying to ask the applicants to think about this because the evaluation contractor will be working with the grantees. And it's very helpful for us to understand on such short notice that we're going to have to do in such a short time frame that we have under the MMA to understand what you think is a good way to do this. This will help jump-start the process and just facilitate the whole thing all the way down the line.

J. White: I do want to point out that the estimate for the evaluation is not part of your budget to actually perform the work that you're proposing under the grant application. It's to be a separate issue. Again, it's more just to give some thought to you what would it take to bring all of this together.

M. Rupp: Actually, that helps a lot, Jon. Thank you, and Maria.

Coordinator: Daniel Sands, you may ask your question.

D. Sands: I have questions related to testing of standards that weren't answered that we have submitted earlier. So, one question is, when you're looking for testing of standards, are you looking for testing the technical effectiveness or integrity of these standards or the functional? What I mean, for those of us who do clinical research as well, it's like whether it's efficacy versus effectiveness. In other words, are we testing that what you put into one end of the pipe comes out the other end accurately or are we testing how these things actually affect clinical and pharmacy and so on workflow?

I don't know if you want me to just cluster these or maybe we can do that one at a time. Go ahead and answer that.

J. White: Yes. Let's do them one at a time. There is a basic fundamental level, which is—I need to make sure, we need to make sure, everybody needs to make sure that these standards that we are going to ask everybody to use do what they do and they work well together.

So, regarding the kind of the pipeline question, yes, I do need to make sure that what goes in one end of the pipeline is what comes out the other end of the pipeline that it works the way it does.  This is actually specifically addressed under core evaluation questions up in section I. Are the right data being sent? Is it useful and accurate? Are the data well understood at all points of the transaction, and so on.  There are additional questions that address kind of the logistics around implementation of electronic prescribing; how hard is this to do? The RFA says, "applicants must describe the outcomes that we reported, the examples included but are not limited to medication errors, adverse drug events."

So, beyond just the kind of mechanical do these standards work and do they transmit the information the way they think they do, we'd like applicants to describe other outcomes as they're laid out here in the RFA, which include things like medication errors and adverse drug events.

M. Friedman: This is Maria. I'd like to address another piece of that and that's how the things really work in the environment. The MMA envisioned E-prescribing to do several things. One of them was to save cost and we anticipate, based on the literature, that there will be significant time and administrative savings by reducing call backs and a lot of the to-ing and fro-ing between prescribers and dispensers that occurs today.

Then, secondly, are the significant benefits on accruing from patient safety and reducing adverse clinical events and medication errors.  So to the extent that we get our arms around all of this, that's a good thing.

D. Sands: But let me just ask for a clarification on what you just said.  So that's not testing the individual standard; that's testing an impact of an entire interrelated system. So I understand that those things are important and should be tested and we need to do that, but to me, that doesn't sound like it's testing a specific standard.

M. Friedman: You need to have the standards to make sure that everything works together and works the same way, within and across practice settings and those kinds of things. So the systems that you use must have these specified standards undergirding them.  How fancy the systems that you use or are going to include in the tests are, that's up to you. There may be people out there who have more sophisticated, electronic medical record systems that have a lot of additional functionality build in. As I said previously, that's really great but we're still looking at whatever technology and prescribing environment you're testing this in, they've still got to deal with that basic suite of standards that we specified in the RFA.

J. White: And follow-up to something that you had said, Maria, there is an additional review consideration. This is section 52B that says at the bottom, "Proposed projects which offer and evaluate the impact of financial incentives for electronic prescribing are also encouraged."

So when Maria was talking about some of the financial benefits that we think may be attributed to electronic prescribing, that is the guidance that's contained in the RFA regarding that.

D. Sands: Okay. Then the other question related to standards is, are we comparing the standards to the paper-based systems or are we comparing the standards to any previous iterations of the standards that would be out there?

J. White: This, you're referring to things like medication errors, adverse drug events, things like that.

D. Sands: Yes. If those are the outcomes. Right.

J. White: That's basically what you have to compare them to.

M. Friedman: And in some of these cases, you are comparing to a paper-based system.

J. White: But I would stick specifically to what's in the RFA.  I would just really look at that and go by the guidance that's in the RFA because we can't deviate from that text, that's what's in there.

While we're waiting for another question, we also wanted to mention human subjects requirements in federal grants. And if I could ask Joan Metcalfe to speak to that from our staff.

J. Metcalfe: Many of you may not be familiar with federal regulations involving human subject research because you've not been involved in that before. It's not specifically mentioned in the RFA but it is applicable as appropriate to your project. So, before you submit the application, before you get too far down the pipeline on this, we recommend that you go to the OHRP Web site—that's Office of Human Research Protection, OHRP.gov—and do a little surfing there. They do have a decision tree that will tell you whether those particular regulations are applicable to your situation. If they are applicable, there are some hurdles that they're going to be requiring you to go through. Since we are under a tight time line on this, we want you to be aware of this before the awards go out so that you can minimize the lag time.

J. White: There are points of contact on that Web site that you can talk to people and I found them to be generally very helpful.

Coordinator: Rodney Dobson, you may ask your question.

R. Dobson: Thank you. Two questions if I may. The first is, I guess, relates to our desire or our organization's desire to learn whether or not the participating organizations will be in any way, endeavoring to foster, I guess, in the spirit of fostering a collaborative partnership consortium type initiatives. It would be of assistance to us if we could potentially look to other instant parties as a conduit to contact those parties. I speak as a technology vendor and I guess, as a foreign institution as well.  So, by definition, we're precluded from being a primary applicant but we feel we'd like to participate by a way of, I guess, being a potential technology vendor into the proposal. So I didn't know if there's going to be any sort of channel that would facilitate dialogue between interested parties or not.

J. White: The short answer is not through us. The longer answer is, with contracts, which is a different mechanism, there are things like a vendors list where you could contact interested parties. This is a grants process; this is a scientific review process.  Unfortunately, we do not have any such means of contact available to you through us or through CMS. Deepest apologies, Rodney, but not through us.

R. Dobson: Thank you. My second question, just quick technical question if I may. With regards to, I guess, the stated necessity of testing digital transmission or electronic transmission of prescription information between prescribers and dispensers or transactors, you stated that there is that absolute requirement. I just wondered, is there a preferred definition of what constitutes that electronic transmission, and by that, I mean, is there a preference that the script information be electronically transmitted, for instance, into a pharmacy network, which, for instance, share scripts or would it be a preference that it go directly into, for instance, the pharmacy vendor system, the electronic system being used at prints to retail pharmacy?

Our background is just there'd be a number of satisfying criteria to constitute in electronic receipt of that information at the pharmacy. I just wondered if there was a distinct preference.

M. Friedman: This is Maria. We are technology and vendor neutral, however, to the extent that the technology must use the stated standards that are in the RFA.

J. White: That's correct. Basically, the answer is go by the RFA, go by what we ask you to test in the RFA and that will guide your decision.

Coordinator: Jeffrey McDowell, you may ask your question.

J. McDowell: My name is Jeff McDowell. I just wanted to clarify and again, we've gone over this several times but it was Peter Kaufman's question in response to that and I believe, it was Jon White who made the comment about we will look favorably on contingency plans to deal with the evolving standards.

M. Friedman: That was me, Maria Friedman, and I may have misspoken. Again, I think because some of these standards are not ready for prime time, I think your application needs to address that fact and I—

J. McDowell: My question is if you could clarify "will look favorably on."

M. Friedman: I misspoke on that.

J. White: That comment is retracted.

M. Friedman: I retract that comment.

J. White: The answer is that—

J. McDowell: Go by the RFA.

M. Friedman: Go by the RFA.

J. White: Unfortunately, this is kind of the way it has to be when you're going through this mechanism for moving money through the federal government is that we recognize that that's an issue but you will be judged solely based on what the review criteria are in the RFA. Inasmuch as you address what you recognize to be an existing situation with respect to those review criteria, great. But that's what you're going to be judged by.

J. McDowell: Within the confines of the RFA?

J. White: You got it.

M. Friedman: To expand on that somewhat, if you even get into a contingency situation, you would then, of course, have contingency cost because you don't know what's going on and what's going to happen and we don't consider contingency cost.

Coordinator: Peter Kaufman, you may ask your question.

P. Kaufman: Sometimes with government grants sponsored research and work, the intellectual property belongs to the government. We wanted an assertion that any work that is done to expand the capabilities of a vendor's product would still belong to the vendor and that the intellectual property would not have to be shared?

J. Metcalfe: This is Joan Metcalfe. Intellectual property generally belongs to the grantee, but the federal government will have some sort of walking right, if you will, to use that material for some purposes but the intellectual property rights belong to the grantee.

P. Kaufman: Can you expand on that? You said the federal government have walking rights to use that for their own purposes. If the federal government decided that all Medicare Part D was going to go through federal government implemented E-prescribing program, does that mean that they could write a program with the intellectual property developed by an individual vendor and use that because it was paid for with this grant?

J. Metcalfe: I'm not a lawyer. And I would prefer not to go there. The short answer is that we can use it.

J. White: Thank you for bringing it up. I appreciate it.

Coordinator: Aubrey Jackson?

A. Jackson: Yes. I had a couple of quick questions that I'm sure the rest of the people on the call, these are probably very simple questions. One of the questions I had was concerning the amount of guidance or assistance that we can expect for garnering participants in the study. Do you guys, CMS, AHRQ, will you provide any kind of guidance or assistance to applicants who are going to conduct these studies toward participants?

J. White: Only that which is contained within the RFA.

A. Jackson: Good answer. The other question I guess I had was the—actually, being able to have the standards. I have gone out and I've looked, I'm a new vendor into this space and saw this guide and feel that our technology fits it very nicely. I noticed that the standards that are there are not necessarily publicly available. Is there a place that I can go and examine this information for use?

J. White: If you look in the RFA, I'm assuming you're referring to the link that says—this is under section one to view the prescribing prescription drug program proposed rule, please use the following link, and you…

A. Jackson: But then you refer to NCPDP, SCRIPT and the different standards that you want tested, there is no—I don't know if these guys are publicly available or—

J. White: What I might do, Aubrey, is—that's a good question and I might defer that question.

A. Jackson: Like the Rx Hub protocol and the standards that you referred to, I'm sure the other vendors who have been in this space have them at their disposal if this is going to be—and I guess the other question is, is this actually intended to be an open standard?

J. White: Actually, let me ask Maria. Maria, are these proprietary standards or are they open standards?

M. Friedman: Some of the techie people on the call may want to correct me on this, but the reason that—if you look back to our… proposed rule making and our foundation standards, those were proposed because they're commonly used in the industry.

A. Jackson: I'm not exactly sure that they are open. I guess that's kind of the question.

M. Friedman: I think we can address that for you on an FAQ after the call and put that up on the Web site for you and provide links to where you get the information from.

J. White: Aubrey, that should be available to you within a week after this call. So it should be available to you very shortly.

Coordinator: Lynne Gilbertson.

L. Gilbertson: Hello. This is Lynne Gilbertson from NCPDP. I just thought I pop in case you needed a question answered.

M. Friedman: Can you answer that one, Lynne?

L. Gilbertson: I may have been popping in when the question first got started.  Did we end up with a specific question?

M. Friedman: The question before you was, "Are the standards open standards; NCPDP, SCRIPT and Tel"— Since you're from NCPDP, you can speak to NCPDP, SCRIPT and Telecom.

L. Gilbertson: Right. As in the X12 standards as well, I can't speak for them but I can tell you my experience. The NCPDP and the ASC X12 standards are available for membership fees. With that membership, you receive all the standards and all the guidance that is available. So they are not closed standards but they are available through that membership.

Coordinator: Daniel Sands.

D. Sands: Yes, I have an additional question. You said that the funding for the grants cannot be used to fund any sunk cost but if you're doing testing before and after study, for example, can it be used to support De Novo implementations?

M. Friedman: Yes.

J. White: Yes. And the answer is if that De Novo implementation is required to test the standard, then that can absolutely be part of your application.

D. Sands: Thank you.

Coordinator: Jill Helm.

J. Helm: Yes. Thank you.  I have a couple of questions. One is could someone speak to the state and federal signature requirements and requirements around electronic transmission of prescriptions and how that will affect this pilot? For example, as it relates to controlled substances, is electronic transmission of controlled substances going to be permitted for the purposes of this pilot?

J. White: I will speak to that. As many of you are well aware, there are different state requirements regarding, say, for controlled substances, that is not specifically addressed in the RFA, so I cannot specifically say yes or no to that but the answer is, you need to be very well aware that when you make a proposal that you can do what you proposed to do as based on the requirement in the RFA, in whatever state you're proposing to do it in and that's kind of a roundabout answer to your question but that's the answer.

J. Helm: So then, just to sort of follow on to that, any proposed project plans then should anticipate the regulatory environments?

J. White: I think that would be a wise thing, yes.

M. Friedman: I think you need to deal with today's regulatory environment.  I don't think you can anticipate anything. I think you need to understand or to base the applications on what's legal in the states that you're proposing to work in and also for the kinds of prescriptions that you will be dealing with.

J. White: You got it.

J. Helm: Second question I had, if I may, is under additional characteristics to be considered for the research objective mentioned the use the FDA/NLM structured product label information. Could someone comment on how it's anticipated that that information be used? Must it be used in a transactional format since these standards are transaction-based standards or is it something that can be addressed as more of a static or reference environment?

J. White: That's a great question. I'm jumping down to the part of the RFA that specifies its use.  It is part of the review criteria under additional review criteria but it doesn't really specify how it's to be used.  It's at the bottom of 2B and it says, "Finally, use of the FDA/ NLM structured product labeling for electronic drug information is encouraged."

Again, going back to the earlier comment, this is one of those we'd like to have but not must have thing. The RFA does not specify how you're to use it. So I think that's probably up to the applicant.

M. Friedman: This is Maria. And that's one of the nice things about grants is you tell us how you propose to do these things within the confines or the parameters set forth in the RFA as opposed to us telling you what you must do. You tell us what you can do in your environment given the requirements of the RFA.

J. White: Does that answer your question?

J. Helm: Yes, it does. Thank you.

Coordinator: Krishnan Seshadri, you may ask your question.

K. Seshadri: Yes. I have a question on some—I need some clarification on the eligibility for the applicants. It states, I think, under section one paragraph three that CMS will enter into cooperative agreements of physicians, physician groups and so on and appropriate entities. Does that include technology vendors or does the technology vendor have to ally themselves with one of these groups to participate?

Coordinator: I think the answer to that is appropriate entities are those who meet the criteria outlined above, who are capable of doing the work that we are asking to be done under the RFA. Right after that, there's a sentence that says, "Other appropriate entities include long-term care facilities in rural health clinics" so includes those, but it's not limited to those. Does that help?

K. Seshadri: I guess the question is, as a technology vendor, we can apply as long as we can test the standards with these groups, is that correct?

J. White: Yes. As long as you meet the other criteria, as long as you're a for profit, not for profit, etc.

M. Friedman: Right. You're going to have to test the standards with somebody. And so how you put that together really is up to you. It could be a very small and limited test of the standards between a technology vendor and say a long-term care facility or a rural clinic or something or you could be part of a bigger consortium. How you guys decide that is up to you as long as you pay attention to what's required in the RFA.

Coordinator: Teri Byrne, you may ask your question.

T. Byrne: I have one with Rx Hub. Actually, I wanted to clarify Aubrey's question about the Rx Hub protocol and just offer up that that protocol has been submitted to NCPDP for formulary and benefit and is approved by the membership and should be fully approved in October. So you should be able to get that information from NCPDP if you're a member as well.

Coordinator: Nayman Shah, you may ask your question.

N. Shah: Is there any general guideline as to how many number of patients that would be an appropriate sample size for testing?

J. White: There is no specific guidance for that contained within the RFA.

J. Metcalfe: Also, I would note that there's a range in terms of the size of the grants so part of it could be depending on what you propose to do with your budget. It should just be commensurate with...

M. Friedman: This is Maria. One of the nice things about grants, as I said before, is that the government is not specifying how you put this test together. As I said on response to another caller, you could have a very small test in D or you could be part of a big consortium.

Coordinator: At this time, there are no further questions.

J. White: Thank you very much for participating in this call.  I hope we've referred you back to the RFA enough times for today. Again, look to the NIH Grants Web site at http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-06-001.html. Thank you very much.

M. Friedman: Thank you, everybody.

Current as of October 2005

---

Internet Citation:

Pilot Testing of Electronic Prescribing Standards—Cooperative Agreements. Technical Assistance Conference Call, September 29, 2005. Summary. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/fund/erxtacall.htm

---