Introduction
Research Questions
- What is the context in which MI-GPS and AEP/PATH operated? What elements were intended to be part of the two programs? How were the programs implemented? Did they differ from plans or expectations?
- Did the receipt of employment services and staff assistance differ between MI-GPS and AEP/PATH? Specifically, were there differences in the types of services and assistance received and in the delivery mode (group vs. one-on-one) for those services and assistance? What was the effect of MI-GPS on grit, self-efficacy, perceptions of job search skills, motivation to work, and reduced barriers to work relative to AEP/PATH?
- Were there differences between the MI-GPS and AEP/PATH groups in outcomes such as employment, earnings, and public assistance receipt? Are there differences between the MI-GPS and AEP/PATH groups for other outcomes such as job quality (e.g., hourly wage, work-related benefits, regularity of hours)
This report describes the implementation and impact study findings from an evaluation of the relative effectiveness of two approaches to assist cash assistance applicants and recipients in finding and keeping jobs. Conducted from 2016 to 2018 in two counties in Michigan, the study compares: (1) an enhancement to the state’s existing Temporary Assistance for Needy Families (TANF) program that provides goal-oriented coaching to help applicants and recipients set and achieve employment-related goals (called the Michigan Goal Progress Success (MI-GPS) program), and (2) the state’s existing TANF program that primarily focuses on participation in required work activities, starting with a 21-day Application Eligibility Period (AEP) and followed by the “Partnership. Accountability. Training. Hope.” (PATH) program (the two-step program is called AEP/PATH).
Using a rigorous research design, the study did not detect impacts on employment rates (the study’s confirmatory outcome), earnings, or cash assistance receipt over a three to five quarter follow-up period. While participation in employment services was high for both groups, compared to the AEP/PATH program, those assigned to the MI-GPS program received more one-on-one assistance with a number of goal-setting skills and their self-efficacy increased. However, there were no detectable effects on the other intermediate outcomes the MI-GPS was expected to impact.
Purpose
To assess the effectiveness of the goal-oriented coaching program in Michigan, this evaluation uses an experimental design that randomly assigns job ready cash assistance applicants to either the MI-GPS or AEP/PATH program and compares their employment and public assistance outcomes.
The TANF program provides cash assistance to low-income families with children, as well as employment-related services to help them become self-sufficient. Balancing the provision of cash assistance with individual responsibility, TANF requires families with a work-eligible individual receiving cash assistance to participate in a specified set of work activities as a condition of benefit receipt. In addition, some states and localities provide employment-related assistance (and require participation in those activities) during the application process. There is strong evidence that employment services, particularly job search assistance, provided as part of a cash assistance programs are effective in increasing employment, but the impacts are modest. Many low-income individuals struggle to find and keep jobs, and families remain poor despite the assistance provided.
Recently, policymakers and program operators have been exploring the potential of goal-oriented coaching approaches to help cash assistance recipients improve their employment outcomes. These approaches are based on psychology and neuroscience research that suggests chronic stress associated with living in poverty hinders the development of planning, decision-making, and organizational skills. This research has generated interest in designing interventions to strengthen TANF and other employment programs by improving participants’ self-regulation skills and through this helping them identify and attain meaningful goals related to economic independence.
Key Findings and Highlights
Implementation Study Findings
- Both the MI-GPS and AEP/PATH programs operated largely as designed during the study period, differing as intended. MI-GPS coaches consistently reported the goal-oriented coaching tools and the revised orientation developed for the MI-GPS program are valuable in implementing the approach.
- MI-GPS coaches reported difficulties aligning the goal-oriented coaching with work participation rate requirements.
Impacts on Service Receipt and Other Intermediate Outcomes
- No differences in participation in employment-focused activities, including job search assistance, were detected between the MI-GPS and the AEP/PATH groups.
- Compared to the AEP/PATH program, the MI-GPS program increased the receipt of assistance on workplace behaviors and soft skills, which included goal-setting skills, provided through one-on-one meetings.
- MI-GPS increased self-efficacy, relative to the AEP/PATH program. However, the study does not detect an impact on other intermediate outcomes.
Impacts on Employment, Earnings, and Public Assistance Receipt
- There was no detectable impact on employment levels in the second quarter after random assignment for the MI-GPS program, compared to the AEP/PATH program. In addition, no differences in earnings were detected.
- There was no detectable impact on MI-GPS on receipt of TANF benefits or TANF benefit amounts compared to AEP/PATH.
- The MI-GPS reduced the amount of SNAP benefits received compared to the AEP/PATH program. However, no impact on the proportion receiving SNAP benefits was detected.
- There was no detectable difference between the MI-GPS and AEP/PATH groups in the characteristics of their current or most recent jobs, including wages and benefits.
Methods
The JSA evaluation of the Michigan’s programs includes parents receiving support from TANF. The evaluation includes both an implementation study to examine the design and operation of the two programs and an impact study using an experimental design to measure differences in employment, earnings, and public assistance outcomes.
The evaluation randomly assigned 2,081 cash assistance applicants to the MI-GPS and AEP/PATH programs. The evaluation uses several types of data, including data from the National Directory of New Hires, administrative data from Michigan on cash assistance and SNAP benefit receipt, and a survey administered to study participants approximately six months after random assignment. The evaluation also included site visits and a staff survey to document program implementation and operations.
Appendix
Citation
Martinson, Karin, Karen Gardiner, Eleanor Harvill, and Rachel Cook. Implementation and Impact of a Goal-Oriented Coaching Program for Cash Assistance Recipients in Michigan, OPRE Report #2020-73, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.