Pilot Testing of Electronic Prescribing Standards—Cooperative Agreements

Technical Assistance Conference Call: Summary

---

On September 29, 2005, a technical assistance conference call was held regarding Pilot Testing of Electronic Prescribing Standards [RFA-HS-06-001]. This summary of the conference call gives an overview of the Request for Applications [RFA] and the grants process and answers questions raised about the cooperative agreements RFA.

---

Coordinator: Welcome and thank you for standing by. At this time, all participants are in a listen-only mode. After the presentation, we will conduct a question and answer session. Today's conference is being recorded. If you have any objections, you may disconnect at this time.

Now, I'll turn the meeting over to your host for today's call, Dr. Jon White.

J. White: Hello and thank you very much. I'd like to thank everybody for attending today. This is a technical assistance call for RFA HS06001 pilot testing of electronic prescribing standards cooperative agreements. My name is Jon White. I am on the staff at the Agency for Healthcare Research and Quality [AHRQ].  I have a number of staff here in the room with me who will participate in the call; they'll introduce themselves as it's necessary for them to talk.

The agenda for the call will be a brief overview of the RFA. We will discuss some grant application and management issues. We will answer some of the questions that were submitted ahead of time and then we'll turn the floor open to an open forum.

Some standing rules for the questions; we will likely be pointing mostly to the RFA and try to refer you to the information that's in the RFA for reasons that we'll explain. If we need to discuss an answer before we give it to you, we will put on hold for a minute and discuss it and then get back to you. If we cannot answer a question for now, and I'm anticipating there are probably some that we won't be able to, we will say that we can't give you an answer now, and we will make that answer available on our Web site. So you know this call is being recorded. There will be a transcript of the phone conversation, as well as answers to the questions that we can give answers to available on our Web site within a short period of time after this phone call.

So with that, if we could head into the brief overview of the RFA and this is Request For Applications for cooperative agreements to test standards for electronic prescribing. These projects will be competitively awarded as cooperative agreements. The mechanism is a U18. This is being done by CMS in collaboration with AHRQ. The pilot RFA was announced on September 15, 2005. Copies are available on the NIH Web site at http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-06-001.html.

Key dates associated with this RFA; obviously, today is our technical assistance conference call, September 29; letters of intent receipt date is October 7, 2005; application receipt date is October 25, 2005; and peer review and awards are anticipated to be made by the end of December 2005.

I'm not going to get into a description of the project because I will refer you to the RFA for that. But we will likely get into some of that with the questions that are being asked. So that will end my kind of brief overview of the characteristics of the RFA. I would like to turn it over to some of the other staff that are here with me to discuss some aspects of grant application and management issues.

A. Deal: Hello. This is Al Deal. I work in the grants management office. Some of you may be more familiar with a contract than a grant or cooperative agreement.  So just briefly, I'm going to go over some of the differences that you're going to see between the two.

Starting out, for example, you're going to be submitting applications; you're not going to be submitting proposals such as you would do with request for proposal. You're going to be submitting your applications to the NIH whereas your proposals usually come to AHRQ, and I believe that's referenced in the RFA itself.  Just like a contract, the application receipt date must be adhered to, otherwise the application is not going to be reviewed. The application will be reviewed and scored. We're going to have this one performed by an ad hoc committee, special emphasis panel.

The award is based on technical merit as determined by the peer review and recommendation of program staff and the award is discretionary. There is no competitive range such as what you would find with a contract. In a contract, the competitive range is determined and negotiations occur with those organizations determined to be within that range. That does not occur on a grant or cooperative agreement. And the selection of awardees is not protestable. In other words, it's discretionary. We make the decision based on programmatic decisions and what's in the best interest of the Agency and CMS and that's not protestable as a contract would be.

Negotiation does occur but there is no call for a best and final offer, such as there is with a contract. There's substantial federal programmatic involvement in a cooperative agreement and that's laid out in the terms and conditions of award but is not directive like a contract is.

The award contains a few explicit terms and conditions but makes reference to various authorities, statutes and regulations by which the grantee is bound. A contract itself has clauses that you adhere to.

The award instrument is a notice of grant award. It's only a few pages long rather than a detailed contract, which many of you know, can be several inches thick. And a cooperative agreement has no defined deliverables whereby a contract does. And if you have any questions on that, later on, I'll be happy to address those.

J. White: That is kind of our brief overview of grants and management information. I do want to point out that I did promise everybody on the call a PowerPoint® file and for various reasons, it has not been possible to send that out prior to this call. Instead you'll be able to receive all your guidance from what will be available from this call on the AHRQ Web site, under Funding Opportunities. So, no, you didn't miss anything.

I'm going to jump to some of the questions that were sent to us ahead of time. Perhaps one of the most common questions that I received from those folks were along the lines of, "Can applicants apply to perform a portion of the pilot, for example, testing one or two of the standards or must they perform all aspects of the pilot?" And if you look in the application review information, the reviewers are directed to specifically ask whether or not the study addresses the need to test all the initial standards. So the answer is each applicant must propose to test all the initial standards not a portion of them. So that is one thing that has come up that is important to know.

Another question that came up was, "Will the project continue to be funded as technology enhancements occur after 2006 and if so, to what degree and to what basis?" And the answer is no. This is a one-time award. The work is to be done during calendar year 2006. As specified in the RFA, there's a report that needs to be submitted to us 30 days after the end of the award in January of 2007 and there are various reasons for that. But the answer is no, that these will not continue to be funded beyond 2006.

Another question that I might ask our grants management staff to consider was to clarify the role of the principle investigator and the full question read, "The paragraph that refers to the principle investigator or PI is unclear. One would expect this project proposal response will include one or more principle investigators and can you clarify the role of principle investigator?"

J. Metcalfe: This is Joan Metcalfe at Grants and Administration.  In our world, there is one PI for the record for each application. That named individual is scientifically responsible for the conduct of the project.  That individual is listed on the face or at the beginning of the application. That being said, there can be any number of discretionary assignments of duty, if you will, throughout the application depending on expertise and need.  Again, that's discretionary and we don't formally recognize any other responsibilities. That all falls on the shoulders of the titular PI of the application.

D. Rothstein: This is Debbie Rothstein from the Extramural Research Office. Just a follow-on on Joan's comment. I'm sure most of you, having read this RFA, have noticed that there is a requirement for the PI to be devoting at least 25 percent of his or her time to the project throughout the course of the project.

J. White: I have been asked this, that is a firm and fast rule. Whoever is designated as the principle investigator for this project must devote 25 percent of their time for lots of different reasons. It must be one person and is not dividable among people.  There must be one PI and they must devote 25 percent of their time to this project. From our perspective, the main reason being that this is a one-year project, there is a lot of work to do in this period of time and we need to make sure it happens, that there's somebody who is committed to making sure that that happens. So that is a firm and fast rule.

Another question asked was, "Does every proposal have to use an E-prescribing application/process?" The answer to that is yes. This is to test electronic prescription program standards and it would be difficult to do that without an E-prescribing application or process.

Another question that was asked, which is important for you all to hear, I think was, "Does the pharmacy need to receive these in electronic format or can they receive them in fax?" And while not all of these standards relate to transactions between a prescriber and a pharmacy, the answer is yes. These applications must include processes where the prescriptions or the transactions are received by the pharmacy in electronic format.  Otherwise, you're not really able to test those standards adequately. And it's specified in the RFA that you have to be able to test these standards and be able to test all of them. So the answer to that is yes.

M. Friedman: Jon, Maria. Can you hear me?

J. White: Yes, I can hear you. I'm going to toss one of these questions to you. A lot of these questions follow along this line, "the RFA referenced initial standards and proposed foundation standards and also indicate that the final foundation standards will be included when available for the pilot, can you elaborate more on this?" And I thought you might want to discuss where in the RFA it references the notice of proposal making and the regulation from CMS.

M. Friedman: I'm sure a lot of you on the call have heard this before, but for those who haven't, we are in the process of developing a final rule that specifies what the foundation standards are. It should be out very soon and I know I've said that before, but I really hope it will be out soon enough to help you with your letters of intent and your proposal submission.

Having said that, we cannot talk about what's in a rule before it's published and that's the reason why the RFA is written the way it is. We could not say what are foundation standards and what aren't. But the fact of the matter is that the proposed foundation standards in the NPRM [Notice of Proposed Rule Making] and all of the other standards that aren't incorporated in the pilot must work together. That's really about all I can say about that.  Hopefully, the guidance will be out very soon and it will be helpful to you. Although for purposes of the RFA, you have to adhere to what's in the RFA.

Is there anything else I need to say about that, Jon?

J. White: No, you got that. I'm pulling up a separate answer. Hang on just one second.

M. Friedman: While Jon is doing that, I'd like to introduce myself. I'm Maria Friedman from the Centers for Medicare & Medicaid Services [CMS]. I'm the co-project officer with Jon on this. And I just wanted to take the opportunity on behalf of the Centers for Medicare & Medicaid Services to thank you all for your interest and also to thank AHRQ for all their help in getting this off the ground and doing all this work in collaboration with us. We look forward to getting some really good applications.

J. White: Okay. Thank you.  I would likewise return the laudatory comments. The next question that I'm going to bring out was a question regarding different review criteria. This one specifically said, "The announcement states that at least 25 percent of the patient population must be age 65 or older or Medicare-eligible but also states the priority population includes children. Since they usually do not have Medicare coverage, is it possible to apply some pediatric practice with over 25 percent Medicaid coverage?"

The answer is you're looking at two different types of criteria. The 25 percent Medicare-eligible rate, 65 or older, is a "must have" review criteria. The inclusion of priority populations is a "like to have" review criteria. There are differences between the two, it's important to really carefully review the review criteria in section five, the application review information because that will guide you as to what is absolutely mandatory to have in there and what is that we'd like to have in there is probably the layman's terms way of putting it.

Another thing that I really closely want to draw your attention to is there are seven places in this RFA where it says "this criteria must be met or the application will be returned without review." That is absolutely critical for you to understand what each of those requirements are and to carefully look at the RFA and make sure that you have met those criteria because after your application is received here, it will be reviewed to make sure that you have included that criteria.  And if it is not there, your application will be returned without review and you do not want to go through all the effort of trying to put one of these together only to have it returned because something got forgotten, so I strongly encourage you to look through the RFA for places where it says "or your application will be returned without review" and make sure that those criteria are met.

Another question that came up was regarding the evaluation contractor. There will be a separate evaluation contractor, who is not going to be a grantee, who will help CMS and AHRQ pull together the information that is generated by the work of the grantees and put into a report that is required to be delivered to Congress in April of 2007, and that is going to be the function of that evaluation contractor.

One of the questions asked, "Do we need to include in our budget information relating to doing the evaluation or is that a separate thing or how does the evaluation contractor work in there?"  Your applications, as it says in the RFA, should reflect your cost to thoroughly test these standards and do the things that we ask you to do in the RFA and the budget should not refect anything different than what we ask you to do in the RFA. And if I told you I will be referring back to the RFA a lot, I meant it because really that's the guidance by which we have to go. So I would refer you back to that.

M. Friedman: Jon, this is Maria. I'd just like to make one other point, is that we expect the folks doing the pilot to have to work very closely with whoever the evaluation contractor turns out to be just because of the time frames involved. Normally, you would do your pilot and then you'd do the evaluation sometime after. And because of the way the MMA [Medicare Prescription Drug, Improvement, and Modernization Act of 2003] is structured, we don't have the luxury of doing it in a stepwise fashion. So we just wanted to be very clear that that's going to be one of the expectations.

J. White: There's one more thing that I wanted to bring out.  One participant in the conference call has asked for some reassurance regarding the right to publish information that comes out of this. It is a general AHRQ policy that you retain the right to publish any work that you do under this grant.

As we just mentioned, there is report that goes to Congress in April 2007. As the project officer for this, I might ask you to work with me and perhaps, delay publication until I get the information I need to get to Congress but you absolutely will retain the right to publish any information that comes out of this.  And I don't want anyone to feel otherwise.

We might respectfully ask that you let us know that you're going to publish something ahead of time, maybe let us know what it is. But I won't stop you from publishing it.

M. Friedman: This is Maria. Normally, we do ask for upfront notification of any kind of publicity report, journal, articles, op-eds, press releases, that kind of thing.

D. Rothstein: This is Debbie from AHRQ. I would say that AHRQ has similar types of requirements. And if you look at the RFA, you will see that there is language in there under the section on other submission requirements.  There is a section on publication transmittal, general AHRQ requirements, and it talks about informing the AHRQ Office of Communications and Knowledge Transfer when articles are expected to be submitted.

J. White: There was another question regarding preference for a care provider whether they should be a for profit or not for profit entity.  I will refer you back to the RFA when it talks about eligible applicants for profit and non-for profit organizations can be applicants. You can be subs on the application.

The one issue that I want at least one of our participants to hear is that foreign institutions are not eligible to be the applicant themselves. They may participate in the projects as members of the consortia or as subcontractors only but you are not eligible to be the primary applicant. And also, this is all in Section III, under 1A of the RFA.  It also says that organizations described in Section 501c4, the internal revenue code that engage in lobbying are not eligible to be applicants either.

Another question was, "Can part of the grant funding be allocated to developing extensions to existing systems to meet the requirements of the grant?" This is actually tied to another question, which was the question about, "Do these transactions need to be received at the pharmacy electronically or is fax acceptable?"  The answer is, yes, they do need to be received at the pharmacy electronically. If part of the grant funding is allocated to enable pharmacies to receive prescriptions electronically, that would meet the requirements of the grant also.

M. Friedman: Jon, this is Maria. I'd like to make one point here. We envisioned that we're going to get applications from people—there are a lot of pilots already up and running out there and we envision that we're going to get applications from some of them. We are not going to fund your sunk costs. Obviously, you'll be doing something different. So if you're going to be adding functionality or testing it in different ways or in different places, for example, you want to replicate what you're doing in another city or another state or something like that, that's fine. But we're not going to reimburse your sunk cost.

J. White: The final question that I'm going to bring up from all the questions that were sent in was, "Could I theoretically be PI for more than one pilot?" The answer is, theoretically, yes; practically, you're going to burning the candle at a lot of ends, that would be really challenging. But theoretically, you could be the PI for more than one pilot.

M. Friedman: This is Maria again. We will be looking at that. What we don't want to see is somebody who is going to be PI on ten different applications.

J. White: And that would be impossible since there's a 25 percent requirement on the PI's time.

M. Friedman: So we do look at the applications in total.  And if there are several that look potentially fundable, we do pay attention to the staffing on that.

J. White: I would just remind us to refer back to the RFA and look at what is specified as a word criteria in the RFA. And if you have a final question, go back to the RFA.  That's going to be your ultimate source for this.

With that, we would like to open it up to the rest of the participants in the call.

Coordinator: Our first question comes from Peter Kaufman.

P. Kaufman: Hello. Some of the aspects of the requirements are not quite ready and probably won't be ready.  Some of the people on the call know I've been very involved with Codified SIG and Codified SIG is still a work in progress, although it's pretty close. How are we supposed to address those aspects that are not completed yet and not quite ready to be tested or even developed?

J. White: Good question, Peter. Give us just one minute. Okay, we're back. Maria, if you're still on the line, I would like you to discuss that question.

M. Friedman: Okay. In fact, we recognized that there are a couple of things in there that aren't ready for prime time and we will take that into consideration when we review the application. We would also look favorably on contingency plans for when those things do come online if you plan to address them.

Peter, I know that you guys have been working real hard on the Codified SIG and it's almost ready for prime time.

S. Andres: This is Scott Andres from Division of Scientific Review.  I caution using any other type of review criteria than those explicitly outlined in the RFA.

M. Friedman: No. That is true.  But I think, since some of these are not ready for prime time and they are required that people need to explain what they plan to do about them when they come online.

J. White: The bottom line to know here is that your application will be judged based on the review criteria that is in the RFA. And when you make your application, you should base it on that and that's how decisions will ultimately be made.

Coordinator: Reed Liggin, you may ask your question.

R. Liggin: Yes. I wanted to know about SCRIPT Standard Version 8.1, that's the standard we're going to and your document calls for SCRIPT Standard Version 5, I believe.

J. White: I'll start to answer this, and Maria, you can back me up.  SCRIPT Standard Version 5 is right now, one of the proposed foundation standards. There are aspects of SCRIPT Version 8 that are in the proposed initial standards and there's a distinction between the two. Maria, do you want to comment on that distinction further?

M. Friedman: The other thing is I believe that SCRIPT Version 8 is backwardly compatible with 5.

J. White: Reed, does that answer your question?

R. Liggin: Yes. It sure does.  Thank you.

Coordinator: Thutrang Chang, you may ask your question.

T. Chang: Hello. I just like to go over the requirement for the populations. Is it 25 percent have to be Medicare patients that's including children, anyone with disability that's entitled to Medicare?

J. White: Let me pull it up specifically on the RFA. So this is under the project characteristics. It describes the population as you're asked to describe the number of patients in their demographic characteristics, at least 25 percent of the patient population must be aged 65 or older or Medicare-eligible.  Now, there are folks who are under 65 obviously who are Medicare-eligible. If you think back about why this is in there, the reason this is in there is because our electronic prescribing standards reviews with Medicare Part D. Therefore, Medicare Part D is primarily who we want these standards to work for and that's why at least 25 percent of that patient population has to be that.

Coordinator: Teri Byrne.

T. Byrne: I have actually two questions that I submitted that I didn't get an answer to. The first one is, the new formulary benefits standard that was talked about in the proposed rule but wasn't actually named as a particular standard because there wasn't one yet, has been approved by the NCPDP [National Council for Prescription Drug Programs] membership. But it has not been approved by the board of trustees or ANSI-accredited yet. If that is not named in the final rule, can we just assume that we'll still be testing that particular standard?

J. White: Here's what I would say, your application will be judged solely based on what is in the RFA. So you have to have at least that in there. If you would like to additionally propose that you can test that standard as well, that's great and we encourage that. But your application will be based only on what we ask you to test in the RFA. Does that answer it, Teri? That question?

T. Byrne: It's not actually clear because there really isn't a standard named. It's just a suggestion that a formulary benefit standard be taken to the NCPDP organization.

M. Friedman: I think you're confusing the regulation process with the RFA process. Forget what we proposed as an NPRM or whatever and look at the criteria in the RFA just strictly in and of themselves regardless of their status and accreditation or anything like that.

J. White: Do you have a second question, Teri?

T. Byrne: Yes. There are certain message types within the SCRIPT Standard that are not used today and don't really have a business purpose. One of them was the GetMessage, which is used for mail boxing and the other one is password change, which is really a functionality between two systems and doesn't really have anything to do with electronic prescribing. Can we assume that we wouldn't need to test those messages even though they're included in the SCRIPT Standard?

J. White: Is this under Version 5 of the SCRIPT Standard?

T. Byrne: Yes.

J. White: Inasmuch as the foundation standards are not what's required to be "tested", I think that you're not required to address whether that standard is functional or not. Because it's a foundation standard and we know or at least, we hope that, as a foundation standard, that's functional. You will be asked to, in your proposal, discuss how you plan to test whether or not the initial standards are interoperable both with each other and with the foundation standards. As much as they're part of the foundation standards, we need to know that the initial standards are interoperable with that entire foundation standard.

I guess I should probably leave it at that and otherwise, point you back to the RFA and that's probably not all the guidance you're looking for, but that is much as I can give you.

Coordinator: Anthony Schueth, you may ask your question.

A. Schueth: Hello. This is Tony Schueth from Point-of-Care Partners. My question is about prior authorization. I heard what you said, Jon, about pointing back to the RFA, but the question is similar to what Peter's question that there are some components where, specifically, the RFA say, "support standards development organizations, development of workflow scenarios" and we happen to be working on different transactions that apply to workflow scenarios. The one transaction that's called out in the RFA is the 278, the NTX12278.  I'm wondering if we'll also be required to test the NTX12275 with the PA attachment that we've been working on and some of the other components of the entire workflow that we've been working on.

J. White: Actually, we'll throw that one to Maria. Maria, could you comment on that vis-à-vis the initial standards for …?

M. Friedman: You have to test the suite of standards that are called out in the RFA. However, if you want to test additional functionality on top of all that, that's fine, but you have to do within the RFA.

J. White: Does that answer your question, Tony?

A. Schueth: It does and I appreciate that. Thanks, Maria. It's just that the sentence, "the support standard development" is a little vague and I was hoping for maybe a little more clarity on that. That's all.

J. White: Unfortunately, I can't be a whole lot more clearer than that.

A. Schueth: I appreciate it. Thank you so much.

M. Friedman: In the ideal world, all of us would have been done with all of our different projects at one time.

Coordinator: Shelley Spiro?

C. Spiro: Yes. This is Shelley Spiro. Like what others have said, the long-term care standards are also not completed and we wanted to know how we should address that in [our response to] the RFA?

J. White: That's a great question and thank you for bringing it up. The great equalizer in this is that if it's not finished for you, it's not finished for anybody else either. The short answer is to present it as best you can according to what you're going to be judged by in the RFA under the application review criteria and bring that up.  That's a great point. It's a fact of life, unfortunately. Believe me, like Maria said, we all wish it was otherwise but it's a fact of life and just do your best to address that scenario and try to meet the review criteria as they're laid out.

J. Metcalfe: Maria, I'd just like to jump in on that. If you will look farther down on the RFA, I think we said in there that we had hoped to get applications from long-term care facilities and others. One of the things that's of interest to us is to see if this suite of standards as they currently exist work in long-term care facilities and their interactions with pharmacies and providers. We realized that the workflow is different and that it's a different scenario, but we are hoping that somebody can figure that out in an application.

C. Spiro: And thank you very much for that clarification, and also for allowing us to try to work this through and I think that that was one of the questions that came through is the phased approach. I think within long-term care, if we look at taking the standard and even working through this pilot in a phased type of aspect, would that do enough for what we're trying to do in long-term care?

J. Metcalfe: I don't think this is meant to be a demonstration project where it's the end all and be all of testing. Remember that is a pilot test of the standards that has to be conducted in 2006. So to the extent that you have future research issues and that kind of thing, this is not the place for future research issues; this is a very discrete time bounded testing of specific standards.

J. White: Relating to the phased question, because you actually weren't the only person to ask that particular question, let me see if I can phrase this—the RFA says that you proposed to test all the standards. The RFA does not specify when you proposed to test all the standards, but the RFA does specify that in the work of this particular grant, you test all the standards.

M. Friedman: I think the clarifying question is that not all projects have to run the entire calendar year. It would be desirable to have them run longer so we get better data. But it may well be because of your situation, you want to do something, say, in one half of the year and build on it in the second half of the year. I think how you propose to do that needs to be addressed in the application. Do remember that this is time bounded. They have to start within calendar year 2006 but they sure have to finish at calendar year 2006.

J. White: That is correct. That is absolutely correct.

Proceed to Next Section