Evaluation of Child Care Subsidy Strategies

The Evaluation of Child Care Subsidy Strategies conducted four experiments as part of a study launched in 2001 by the Office of Child Care and the Office of Planning, Research and Evaluation (OPRE) in the Administration for Children and Families, US Department of Health and Human Services. The study is being conducted by Abt Associates Inc, with its research partners MDRC and the National Center for Children in Poverty of Columbia University.

The evaluation is a multi-site, multi-year effort to gather research evidence that will inform policy decision-making in states and communities and effective allocation of child care subsidy funds. In particular, the study will determine whether and how different child care subsidy policies and procedures and quality improvement efforts help low-income parents obtain and hold onto jobs and improve outcomes for children. Study staff worked with states and communities across the country to identify significant issues and develop hypotheses about the use of child care subsidy funds that could be rigorously tested in a series of experiments. A guiding principle of the study was that state (or community) interests and preferences would play a large role in the choice of research topics and strategies. Study sites and focus of evaluation include:

• effectiveness of three language and literacy curricula on teaching practices and children's language and literacy outcomes (finished in 2005 in Miami, Dade County, FLA);
• impact of alternative eligibility and re-determination child care subsidy policies on parental employment outcomes (Illinois);
• impact of alternative child care co-payment structures on use of child care subsidies and employment outcomes (Washington State); and,
• effectiveness of training on Learning Games curriculum in changing care-giving practices in family child care homes and children's developmental outcomes (Massachusetts).

Implementation of Three Language and Literacy Interventions in Project Upgrade

Project Upgrade was a two-year experimental test of the effectiveness of three different language and literacy interventions, implemented in child care centers in Miami-Dade County that served children from low-income families. One hundred and sixty-two centers were randomly assigned to one of three research-based curricula or to a control group that continued with its existing program. The curricula, while grounded in a common set of research findings, differed in intensity, pedagogic strategies and use of technology. In each center, one classroom that served four-year-old children was selected for the study. Teachers and aides assigned to the three treatment groups received initial and follow-up training as well as ongoing mentoring over a period of approximately 18 months, from Fall 2003 to Spring 2005.

The study looked at two kinds of outcomes: teacher behavior and interactions with children, and aspects of the classroom environment that support children's language and literacy development, measured through direct observation; and children's language and pre-literacy skills, measured by their performance on a standardized assessment. Study staff conducted classroom observations in Fall 2003, Spring 2004 and Spring 2005. Four-year-old children in the study classrooms were assessed in Spring 2005.

"Effects of Child Care Subsidies for Moderate Income Families in Cook County, Illinois"

Random assignment study conducted in Cook County, Illinois, to answer two child care subsidy policy questions: (1) whether providing subsidies to families whose income exceeds the state's usual income eligibility limit affects their child care and employment outcomes, and (2) whether extending the time before families have to reapply — the re-determination period — affects their receipt of subsidies and related outcomes. Between March 2005 and May 2006, all families who applied for child care subsidies in Cook County, Illinois, whose incomes were between 50 and 65 percent of state median income (SMI) were invited to participate in the study. Half of those who agreed to be in the study were randomly selected to be approved to receive subsidies even though they had too much income under usual state rules. People in this group could receive subsidies for the next two years as long as their incomes remained below 65 percent of SMI and they met other state eligibility requirements. The other half of families in the study formed a control group that could receive subsidies only if their incomes fell below 50 percent of SMI. Among those selected to be approved to receive subsidies, half were randomly assigned a one-year re-determination period (that is, they would not have to reapply for subsidies for a full year), while the other half had to follow the usual state rules and reapply for subsidies every six months. In all, 1,884 families were successfully recruited for the study: 470 of these families were assigned to the six-month re-determination program group, 470 were assigned to the 12-month re-determination program group, and 944 were assigned to the control group.

To examine the effects of the program, data were collected from both administrative records systems and surveys of study participants. Data from administrative records systems include information from an application form completed shortly before random assignment, monthly data from the child care subsidy system, unemployment insurance records on employment and earnings, and records from the food stamp and Temporary Assistance to Needy Families (TANF) programs. One follow-up survey collected a range of information not available through administrative records, including a detailed history of each child care provider the family used, and information on the respondents' levels of satisfaction with the child care situation and on the reliability of the care, job problems related to child care, and out-of-pocket costs of child care.

"Effects of Reducing Child Care Subsidy Co-payments in Washington State"

Two-year random assignment evaluation in Washington State to provide information on one key state policy decision: the amount families should contribute when they receive subsidies to help pay for child care, or co-payments. All families in the state of Washington that were approved to receive child care subsidies between October 18, and November 7, 2005 — a total of 5,106 families — were entered into the study. These families were randomly assigned to two groups: (1) a control group that was assigned the standard co-payment schedule used in the state, and (2) a program group that was assigned an alternative schedule that reduced co-payments for many families. Washington State structures their co-payment schedule by dividing families in three tiers based on their income. For the evaluation, alternative and standard co-payment schedules were the same for families in Tier 1, diverged by $35 each month for families at the bottom end of Tier 2, and differed the most for families in Tier 3. For Tier 3 families, the alternative co-payment schedule was particularly beneficial for families with one and two children in care. Reflecting state rules on parents eligible to receive subsidies, the sample included three types of parents: working parents with family income below 200 percent of the federal poverty level, parents receiving Temporary Assistance for Needy Families (TANF) who were in approved work-related activities, and adult parents under age 22 who were pursuing a high school credential. Washington does not have a waiting list of applicants for subsidies, so all eligible parents who applied for subsidies were approved to receive them.

The evaluation hypothesized that reducing co-payments could have a range of effects on family and child care outcomes. First, making families pay less for care when they receive subsidies could encourage them to receive subsidies for a longer period of time. By helping families pay for their desired form of care, increased subsidy receipt might help families achieve more stable care, either by helping them afford inherently more stable care or by helping them pay for care on a regular basis. More stable care could in turn help parents stay employed. Likewise, reducing the amount that co-payments increase when earnings increase could encourage parents to work and earn more, increasing their income and reducing their reliance on other forms of public assistance such as TANF and food stamps. To investigate the effects of reduced co-payments on these outcomes, the study gathered data from three administrative records sources: the child care subsidy system, which provided information on subsidy use and each form of care for which families used subsidies for each child in the family; the unemployment insurance (UI) system, which provided information on earnings and hours worked for each employer; and the public assistance system, which provided information on TANF and food stamps benefits. UI and public assistance data had the advantage of providing information on outcomes even when families stopped receiving child care subsidies.

"Massachusetts Family Child Care Study"

The Massachusetts Family Child Care study was a two-year evaluation designed to examine the impacts on providers and children of an early childhood education program aimed at improving the development and learning opportunities in the care settings and, as a consequence, the outcomes for children in care. The early childhood education program—LearningGames—focuses on training caregivers to stimulate children's cognitive, language, and social-emotional development through game-like interactions with individual children across the day. The Massachusetts Family Child Care Study tested the value of an effort to improve the quality and outcomes of family child care- an environment that could support the development of these individualized, responsive relationships between the provider and the children in care.

LearningGames was designed to promote children's cognitive and language development through learning opportunities provided by their caregivers. LearningGames focuses in particular on increasing the frequency of rich language interactions between caregivers and individual children. This emphasis grows out of the evidence of the importance of oral language development in children's understanding of words and concepts, in their ability to become competent readers, and in their long-term academic success and of the role that rich language stimulation plays in promoting children's development. This evaluation of LearningGames examines the effectiveness of the program in increasing the frequency of rich, stimulating, individualized interactions as a pathway to improved developmental outcomes for the children who are cared for by providers trained on LearningGames.

The study addressed two major research questions:

• Did LearningGames have significant positive impacts on the developmental support provided by providers to the children in care?
• Did LearningGames homes have significant positive impacts on developmental outcomes for the children in care?

The study was conducted with a sample of family child care providers that are members of family child care networks in Massachusetts. Fifteen networks agreed to participate in the study. Within each network, all eligible family child care providers were recruited, where eligibility was defined as having at least one child less than 36 months of age in care. In each of the family child care networks, half of the providers who agreed to participate in the study were randomly assigned to LearningGames and half were assigned to the control condition. While LearningGames and its precursor had been studied in center-based care, home visiting programs and parenting programs, it had not been studied previously in family child care homes. The study was a randomized cluster design, with family child care providers assigned to treatment (LearningGames) or control (business-as-usual) within networks, and children clustered within provider. The study was intended to be an effectiveness study, in that the impact of LearningGames was studied under typical or real-world conditions. Data obtained for the study came from multiple sources. To address questions on implementation information came from interviews with providers, home visitors, network administrators, and LearningGames trainers; review of tracking documents of technical assistance activities; and ratings of the fidelity of implementation of LearningGames conducted by home visitors as well as study staff. To address the research question on provider impacts, provider outcomes were measured through direct observation by the agency home visitors and by independent study staff. Observations of the homes were conducted at three times over the intervention period: at baseline, after one year of implementation of LearningGames, and after two years of implementation. The impact analyses focused on the observations undertaken at the end of the intervention. At this time point, all treatment group providers remaining in the sample had been in the study for more than two years, and treatment providers had received at least 24 months of systematic training support on LearningGames from the network home visitors.