References  -  Next

Evaluation Efforts

SCHIP has significant potential to improve access to care and use of health services for children in the target population. However, the long-term success of the program depends on evaluating SCHIP's effectiveness in meeting these goals. According to Newacheck and colleagues, evaluation efforts must consider:

• The short- and long-term impact of individual State programs on expansion of health insurance coverage and improvements in access to care.
• Features of plans that either contribute to success or block enrollment, retention, and access to care.¹

A number of efforts are currently underway to measure the success of SCHIP programs. Some of the most significant efforts are highlighted below.

State Evaluations Required by Statute

In order to measure their progress, the Federal legislation establishing the SCHIP program requires States to report annually on their efforts to reduce the number of uninsured low-income children in their jurisdictions. These annual reports require States to provide a brief description of the operation of their SCHIP plan, including the following elements:

• Progress in reducing the number of uninsured low-income children.
• Progress in meeting the strategic objectives and performance goals identified in the State plan, including monitoring for issues such as crowd-out and outreach.
• Success and barriers associated with program design, planning, and implementation, and the State's proposed approaches to handling these issues.

In addition to the annual reports, each State with an approved plan was required to submit a formal evaluation of their plan to HCFA (now CMS) by March 31, 2000. This evaluation provided States with an opportunity to document program achievements and to assess program effectiveness over the past several years. The evaluation covered:

• The State's success in increasing the number of children with creditable health coverage.
• The effectiveness of other elements of the State's plan, including characteristics of children served, quality, amount and level of assistance, service area, time limits, coverage, and other sources of non-Federal funding.
• The effectiveness of other public and private programs in the State in increasing the availability of affordable health insurance.
• The State's coordination with other public and private programs for children.
• An analysis of the changes and trends in the State that affected the provision of accessible, affordable, quality health insurance and health care to children.
• The State's plan for improving the availability of children's coverage.
• Recommendations for improving the program.
• Any other matters the State or the Secretary of DHHS (the Department of Health and Human Services) considered appropriate.²

Preliminary highlights from the State-submitted evaluations show the following:

• Strong enrollment trends continued up through the time of the evaluation. Between the second quarter of fiscal year 1999 and the second quarter of fiscal year 2000, enrollment increased by more than 80 percent in 43 States.
• States have worked aggressively to eliminate barriers to initial and continued enrollment by simplifying application, enrollment, and re-enrollment processes, and by coordinating eligibility requirements with Medicaid.
• States have implemented a number of school- and community-based outreach strategies to encourage target populations to apply for SCHIP.³

HCFA (now CMS) SCHIP Evaluation

In 1999, HCFA (now CMS) contracted with Mathematica Policy Research, Inc. to conduct a comprehensive 5-year national evaluation that examines the effectiveness of specific SCHIP features in meeting a broad range of SCHIP goals. The study includes a cross-sectional survey of new SCHIP enrollees, established enrollees, and disenrollees, as well as a survey of children enrolled in Medicaid. The evaluation will assess enrollment and disenrollment dynamics and will permit rigorous examination of the links between child and family demographics; program design features; and barriers, access, and satisfaction.⁴

The major findings from the first year of the evaluation were similar to the results from the State-submitted evaluations: steady growth in enrollment, attempts to simplify enrollment procedures, and innovative outreach strategies. The Mathematica first-year report also revealed that:

• States vary in the extent to which they have expanded health insurance coverage for low-income children. As of March 2000, more than half had set the SCHIP eligibility threshold at or above 200 percent of the poverty level, while 23 States chose a threshold below this level.
• While SCHIP enrollment continues to grow steadily, enrollment is heavily concentrated in States that implemented their programs in the first year after passage of the SCHIP legislation.
• States have had varying results with program features designed to boost enrollment. While some States felt that centralization of the application process was beneficial, others felt this was a hindrance. Likewise, mail-in redetermination applications were feasible in some States but impeded communication between SCHIP staff and recipients in others.⁵

Congressionally Mandated Evaluation

As part of the Balanced Budget Refinement Act of 1999, Congress provided $10 million to DHHS to conduct an independent evaluation of the SCHIP program using a sample of 10 States. This evaluation, which is being coordinated through the DHHS Office of the Assistant Secretary for Planning and Evaluation, will be completed by the end of 2001.⁶ According to the authorizing statute, the evaluation must include:

• Surveys of the target population.
• An evaluation of the effectiveness of different outreach strategies.
• An assessment of the effectiveness of coordination between SCHIP and Medicaid programs.
• Information on the effects of cost-sharing requirements.
• An evaluation of retention issues.⁷

This Congressionally mandated evaluation includes a number of components. Several components were combined into a single study that includes surveys of target populations, State case studies, and program data analysis. This study is being conducted by Mathematica Policy Research, Inc., with subcontracts with The Urban Institute and Mayatech.

Another study, being carried out by Health Systems Research, Inc., consists of a series of focus groups conducted with the families of children enrolled in SCHIP or Medicaid, families that are eligible but not enrolled, and those who have disenrolled. It is expected that the focus group findings will be used in the design of the data collection strategies for the case studies and surveys.

Together, the results from these evaluations will inform Congress about:

• Trends in public insurance coverage.
• Barriers in SCHIP program design and policy that affect the number of families enrolled in the program.
• Access and use of services.
• Enrollee satisfaction.⁸

A preliminary draft report of findings is scheduled to be delivered to Congress in December 2001.

Child Health Insurance Research Initiative

One final evaluation effort currently underway is a series of nine projects being conducted under the Child Health Insurance Research Initiative (CHIRI™). These 3-year projects are being funded by the Agency for Healthcare Research and Quality, the David and Lucile Packard Foundation, and the Health Resources and Services Administration. The research will provide Federal and State policymakers with additional information about which health insurance and delivery features work best for low-income children, especially minority children and those with special health care needs.

Researchers anticipate that the results from the CHIRI™ studies will be generalizable enough to be applied across locations, populations, and insurance design and organizational delivery system features. They hope that SCHIP, Medicaid, and other public insurance programs and delivery systems will benefit from the findings.⁹

More information on these and other evaluation and monitoring activities can be found on the American Academy of Pediatrics' SCHIP Web site at: http://www.aap.org/

1. Newacheck P, Halfon N, Inkelas M. Commentary: monitoring expanded health insurance for children: challenges and opportunities. Pediatrics 2000 Apr; 105(4 Pt 2):1004-7.

2. HCFA (now CMS) SCHIP Web site: http://cms.hhs.gov.

3. Health Care Financing Administration. The State Children's Health Insurance Program: preliminary highlights of implementation and expansion.

4. Health insurance for low-income uninsured children: a national evaluation of SCHIP. Washington (DC): Mathematica Policy Research, Inc.

5. Rosenbach M, Ellwood M, Czajka J, et al. Implementation of the State Children's Health Insurance Program: momentum is increasing after a modest start. Cambridge (MA): Mathematica Policy Research, Inc.; Jan 2001.

6. National Health Policy Forum. SCHIP in the formative years: an update. Issue Brief No. 759. Washington (DC): National Policy Forum; Sep 2001.

7. Title VII-State Children's Health Insurance Program. Available at: http://www.cms.hhs.gov/NationalSCHIPPolicy/. (HTML version)

8. Department of Health and Human Services. Request for proposals: State Children's Health Insurance Program focus group study. RFP No. 26-00-HHS-OS. Washington (DC): DHHS; Sep 2000.

9. The Child Health Insurance Research Initiative. Rockville (MD): Agency for Healthcare Research and Quality. Available at: http://www.ahrq.gov/about/cods/chiri.htm.