ACF Research and Evaluation Agenda Introduction

Publication Date: December 30, 2020
2020 Research and Evaluation Agenda Cover

Download Report

Download PDF (800.41 KB)
  • File Size: 800.41 KB
  • Pages: 5
  • Published: 2021

Introduction

The mission of the Administration for Children and Families (ACF) is to foster health and well-being by providing federal leadership, partnership, and resources for the compassionate and effective delivery of human services. Our vision is children, youth, families, individuals, and communities who are resilient, safe, healthy, and economically secure. The importance of these goals demands that we continually innovate and improve, and that we evaluate our activities and those of our partners. Through research and evaluation, ACF and our partners can learn systematically so that we can make our services as effective as possible.

A learning organization requires many types of evidence, including not only research and evaluation but also performance measures, financial and cost data, survey statistics, and program administrative data. Further, continual improvement requires systematic approaches to using information, such as regular data-driven reviews of performance and progress. This agenda focuses on research and evaluation. We hope in future iterations to expand its scope to include other types of learning as well.

ACF’s evaluation policy includes five principles that guide our work: rigor, relevance, transparency, independence, and ethics.

  • Rigor. ACF is committed to using the most rigorous methods that are appropriate to the evaluation questions and feasible within budget and other constraints. Rigor is not restricted to impact evaluations, but is also necessary in implementation or process evaluations, descriptive studies, outcome evaluations, and formative evaluations, and in both qualitative and quantitative approaches. Rigor requires ensuring that inferences about cause and effect are well founded (internal validity); requires clarity about the populations, settings, or circumstances to which results can be generalized (external validity); and requires the use of measures that accurately capture the intended information (measurement reliability and validity). Maintaining rigor requires a skilled workforce, adequate budget, a robust private sector to compete for grants and contracts, and active consultation with leaders in relevant fields.
  • Relevance. Research and evaluation priorities should take into account legislative requirements and Congressional interests and should reflect the interests and needs of ACF, HHS, and Administration leadership; program office staff and leadership; ACF partners such as states, territories, tribes, and local grantees; the populations served; researchers; and other stakeholders. Evaluations should be designed to represent the diverse populations that ACF programs serve, and ACF should encourage diversity among those carrying out the work, through building awareness of opportunities and building evaluation capacity among underrepresented groups. Relevance requires strong partnerships among evaluation staff, program staff, policy-makers and service providers. Policy-makers and practitioners should have the opportunity to influence evaluation priorities to meet their interests and needs. Further, for new initiatives and demonstrations in particular, evaluations will be more feasible and useful when planned in concert with the planning of the initiative or demonstration, rather than as an afterthought. It is important to disseminate findings in ways that are accessible and useful to policy-makers and practitioners. ACF develops dissemination plans tailored to each study’s goals and intended audiences, and pursues broad dissemination channels such as social media, newsletters, blogs, videos, data visualizations, and conferences. It is ACF’s policy to integrate both use of existing evidence and opportunities for further learning into all of our activities. Where an evidence base is lacking, we will build evidence through strong evaluations. Where evidence exists, we will use it. Discretionary funding opportunity announcements will require that successful applicants cooperate with any federal evaluations if selected to participate. As legally allowed, programs with waiver authorities should require rigorous evaluations as a condition of waivers. As appropriate, ACF will encourage, incentivize, or require grantees to use existing evidence of effective strategies in designing or selecting service approaches. The emphasis on evidence is meant to support, not inhibit, innovation, improvement, and learning.
  • Transparency. ACF will make information about planned and ongoing research and evaluation studies easily accessible, typically through posting on the web information about the contractor or grantee conducting the work and descriptions of the questions, methods to be used, and expected timeline for reporting results. ACF will present information about study designs, implementation, and findings at professional conferences. Study plans will be published in advance. ACF will release results regardless of the findings. Evaluation reports will describe the methods used, including strengths and weaknesses, and discuss the generalizability of the findings. Reports will present comprehensive results, including favorable, unfavorable, and null findings. ACF will release results timely — usually within two months of a report’s completion. ACF will archive evaluation data for secondary use by interested researchers.
  • Independence. Independence and objectivity are core principles of evaluation. Agency and program leadership, program staff, service providers, and others should participate actively in setting evaluation priorities, identifying evaluation questions, and assessing the implications of findings. However, it is important to insulate evaluation functions from undue influence and from both the appearance and the reality of bias. To promote objectivity, ACF protects independence in the design, conduct, and analysis of evaluations. To this end: ACF conducts evaluations through the competitive award of grants and contracts to external experts who are free from conflicts of interest; the director of the Office of Planning, Research, and Evaluation reports directly to the Assistant Secretary for Children and Families; has authority to approve the design of evaluation projects and analysis plans; and has authority to approve, release, and disseminate evaluation reports.
  • Ethics. ACF-sponsored studies will be conducted in an ethical manner and safeguard the dignity, rights, safety, and privacy of participants. ACF-sponsored studies will comply with both the spirit and the letter of relevant requirements such as regulations governing research involving human subjects.

ACF has established a common framework for research and evaluation that includes three types of descriptive studies (foundational, exploratory, and design and development studies), and three types of impact studies (efficacy, effectiveness, and scale-up studies).

ACF’s Office of Planning, Research, and Evaluation (OPRE) is responsible for advising the Assistant Secretary for Children and Families on increasing the effectiveness and efficiency of programs to improve the economic and social well-being of children and families. OPRE studies ACF programs and the populations they serve through research and evaluation; collaborates with ACF programs on research and evaluation and other learning activities; provides technical assistance related to research, evaluation, data, and evidence; and supports data-driven decision making by increasing the quality, usefulness, sharing, and analysis of data while protecting privacy and confidentiality. OPRE includes four divisions: the Division of Economic Independence, the Division of Child and Family Development, the Division of Family Strengthening, and the Division of Data and Improvement.

OPRE includes 68 federal staff with training and degrees in fields such as public policy, social work, educational policy, psychology, quantitative methodology, human development, family studies, statistics, evaluation, social policy, sociology, government, public health, and epidemiology. In FY 2020, ACF’s total budget for research and evaluation was $208 million, including over 100 contracts and several dozen grants. This document includes both recently completed projects, as well as projects active as of the end of FY 2020.

ACF is pursuing learning agendas and related activities at multiple levels. At the broadest level, ACF contributes to the HHS multi-year evidence plan and HHS annual evaluation plan required by the Foundations of Evidence-Based Policy Making Act. At a more focused level, OPRE and ACF program offices are developing detailed and comprehensive learning agendas related to specific ACF programs. This research and evaluation agenda draws upon and includes selected examples of research and evaluation activities from those more detailed efforts.

Partnerships between ACF program offices and the Office of Planning, Research, and Evaluation, as well as stakeholder engagements, drive the identification of research and evaluation needs. ACF’s program offices include:

The sections that follow summarize ACF’s research and evaluation activities and plans in the areas of adolescent pregnancy prevention and sexual risk avoidance, child care, child support enforcement, child welfare, Head Start, Health Profession Opportunity Grants, healthy marriage and responsible fatherhood, home visiting, and welfare and family self-sufficiency. Each section includes an overview of the programmatic area, a description of past research and evaluation, a discussion of stakeholder engagement, examples of broad questions and recent and ongoing research and evaluation activities, and a discussion of future directions. Each section also lists relevant measures by which we monitor performance under the Government Performance and Results Act.