Using Text Messages to Learn about Participants’ Experiences

Lessons from the B3 Study of Responsible Fatherhood Programs

Three organizations added Just Beginning, a parenting program, and DadTime, a custom-built mobile app, to their existing services. Three different organizations added an employment intervention, Cognitive Behavioral Employment for Justice-Involved Individuals Seeking Employment (CBI-Emp), to their existing services.

Many programs communicate with participants by text message. Text messages can be a helpful tool for communicating with participants in real time and gathering information from them that could help program providers improve their service-delivery approaches. The B3 study used text messages to administer short surveys, collecting information rapidly and directly from participants in the study about their experiences with the study’s fatherhood programs.

 

Why learn about participants’ experiences by text message?

Rapid responses

Programs can hear from participants in real time, while activities are in progress.

Reengagement

Text messages allow programs to reach all participants enrolled in a program. For example, the B3 study asked participants about barriers to attendance when they indicated they did not attend the program.

Limitations

  • People who are not attending the program regularly may be less likely to respond to text messages asking questions about the program.
  • Responses collected from anyone not attending regularly might not reflect the views of all those who do not attend regularly.

Learning from participants

Text messages allow programs to reach all participants enrolled in the program. Responses collected help program providers understand the value of the programs offered and reassess strategies, if needed. For example, the B3 study collected information about fathers’ perceptions of program benefits and their experiences, and shared that information with the B3 programs. The figures below show that most fathers felt they were supported and that they used skills learned from the program.

Example One

Text Message 

B3 Text Message 1

Participant Responses 

B3 Participant Responses

Example Two

Text Message 

B3 Example Two

Participant Responses 

B3 Participant Responses II

These data show programs might consider focusing more on coparenting skills, if that is one of their intended emphases.

Sample size: 154

 

How can programs learn the most about participant experiences using text messages?

Download the Tip Sheet  (PDF)

Get permission.

Ensure that participants have given consent to receive text messages from the program. Consent can be obtained through the program’s enrollment paperwork, for example.

Inform participants.

Make sure participants know to expect text messages from the program, so that they do not mistake messages from the program for spam. Inform participants that their responses will be used to help improve the program and participant engagement, for example, and will not affect their participation in it.

Involve the staff.

Have staff members confirm participants’ contact information when they enroll, and verify that they can receive text messages from the program to ensure technical difficulties are not a barrier to responding. Throughout their participation in services, continue to check in with participants to maintain up-to-date contact information and to encourage them to continue to respond to text messages.

Personalize the introduction.

B3 Personalized Introduction

Use a personal introduction that establishes whom the text is from, such as “Hi [Participant’s Name], this is the [program name]…”

Participants are less likely to ignore messages that are specifically addressed to them, and will want to know with whom they are talking.

Keep the burden low.

Make it as easy as possible for participants to respond and motivate them to respond quickly. In the B3 study, most respondents answered the survey within 30 minutes of receiving a notification on their phones.

Do
  • Keep it short, around 3—10 questions
  • Ask questions that will make the participants feel like they are having a natural back-and-forth conversation
  • Ask questions about participants’ immediate reactions and feelings
  • Ask about experiences in the last day or week
Don’t
  • Ask more than 10 questions
  • Ask questions that require respondents to retrieve additional information, such as remembering an appointment schedule from over a week ago

Don’t give up.

People who do not respond the first time they are invited to share their experiences over text messages may respond to other text messages. In the B3 study, participants who did not share their experiences after receiving a single text message sometimes responded to subsequent messages.

B3 Responses III
Increase in responses after multiple text messages

A total of 1,109 people received the survey. Sending multiple messages increased the response rate from 15 percent to 25 percent.