National and Tribal Evaluation of the 2nd Generation of Health Profession Opportunity Grants (HPOG 2.0): Descriptive Evaluation Analysis Plan for the National Evaluation

Publication Date: September 24, 2020
Current as of:
This is the cover of National and Tribal Evaluation of the 2nd Generation of Health Profession Opportunity Grants (HPOG 2.0):  Descriptive Evaluation Analysis Plan for the National Evaluation

Download Report

Download PDF (5,050.93 KB)
  • File Size: 5,050.93 KB
  • Pages: N/A
  • Published: 2020

Introduction

Research Questions

  1. Implementation Study: how is HPOG 2.0 designed and implemented, and what innovative and/or promising strategies have programs introduced in the following five focus areas: (i) employer engagement, (ii) basic skills training, (iii) training in the career pathways framework, (iv) work-readiness training, and (v) program sustainability after the grants end? What are the characteristics of HPOG 2.0 participants? At what rates do participants engage in program activities, training courses, and support services?
  2. Outcomes Study: what are participants’ education, employment, and earnings outcomes?
  3. Systems Study: what are the local service delivery systems in which HPOG 2.0 programs operate, and what was the mutual influence of the HPOG programs and the local service delivery systems?

The Health Profession Opportunity Grants (HPOG) Program is designed to provide education and training to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals for occupations in the healthcare field that pay well and are expected to either experience labor shortages or be in high demand. A National Evaluation of 27 grants awarded in 2015 as part of the second round of HPOG grants (HPOG 2.0) is currently underway.

The National Evaluation includes a Descriptive Evaluation of the implementation, outcomes, and local service delivery systems of the grants. The National Evaluation also includes an Impact Evaluation of the grants’ impacts on participants and a study of the HPOG Program’s costs and benefits.

This report presents an analysis plan for the Descriptive Evaluation, which includes three related studies: (i) the Implementation Study, (ii) the Outcomes Study, and (iii) the Systems Study.

Purpose

This Analysis Plan is a supplement to the Descriptive Evaluation Design Report for the National Evaluation and presents detailed plans for analyses to address the research questions in accordance with the specified research design.

Key Findings and Highlights

The National Evaluation’s Descriptive Evaluation will present descriptive findings only. It will not analyze causal relationships or estimate causal impacts. This report describes the analytic approach to each of the Descriptive Evaluation’s component studies including the relevant domains and research questions for each study. This report specifies measures, sample, and data sources, as well as the analytic strategies and presentational approach.

  • The Implementation Study analyses will use data collected through two rounds of telephone interviews with program staff, program administrative data, and other data sources documenting program implementation to describe the distribution of specific program features and implementation strategies across local HPOG programs. It will also document participant characteristics and program engagement.
  • The Outcomes Study analyses will use program administrative data, federal administrative data on employment and earnings, and data on enrollment in and graduation from degree-granting institutions. The Outcomes Study will address research questions in two domains: program outputs and participant outcomes.
  • The Systems Study analyses will use interviews with 15 HPOG program operators, the lead organization directly responsible for the administration of an HPOG program, and their partners to better understand how HPOG programs influenced local service delivery systems and how local systems influenced HPOG implementation.

Methods

The Descriptive Evaluation will use three analytic strategies:

  • Descriptive statistical analysis and presentation of specific measures for program design and implementation strategies; participant characteristics; and participant program experiences, outputs, and outcomes.
  • Qualitative analysis of perspectives of selected program operators and partners on systems activities and how the local service delivery systems for healthcare training (inclusive of training, public workforce, and human services systems and the local policy and economic contexts) influence the HPOG programs, and vice versa.
  • Case studies of strategies that may have contributed to program success in five focus areas:
    • Employer engagement
    • Basic skills training
    • Career pathways training opportunities
    • Work-readiness training
    • Program sustainability after the end of the HPOG 2.0 grant period

Results from the Descriptive Evaluation will be shared through a variety of reports, briefs, and presentations.

Citation

Werner, Alan, Robin Koralek, Pamela Loprest, Lauren Eyster, & Gretchen Locke. (2019). National and Tribal Evaluation of the 2nd Generation of Health Profession Opportunity Grants (HPOG 2.0): Descriptive Evaluation Analysis Plan for the National Evaluation, OPRE Report 2020-92. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Glossary

Career pathways:
a framework for occupational training that combines education, training, and support services that align with the skill demands of local economies and help individuals to enter or advance within a specific occupation or occupational cluster
HPOG or HPOG Program:
the national Health Profession Opportunity Grants initiative, including all grantees and programs
HPOG grantee:
the entity receiving the HPOG grant and responsible for funding and overseeing one or more local programs
HPOG (local) program:
a unique set of services, training courses, and personnel; a single grantee may fund one or more local programs; organizations that run the HPOG programs are referred to as program operators
HPOG partners:
other organizations directly involved in the operations of an HPOG program
HPOG system:
includes the local HPOG program, its network of partners, the broader training opportunities and support services, and the economic and policy context in which the HPOG program operates
Participants:
individuals who meet program eligibility criteria and who participate in an education and training program and/or receive related services supported by HPOG 2.0 grants
Contextual factors, or “system”:
the economic and service delivery environment in which an HPOG program operates
Outcomes:
end goals for HPOG, including participants’ earnings and employment (in general and in healthcare specifically