Five trials to increase student engagement through text messages

By Maija Koponen, King’s College London | 

Universities offer a range of services and activities designed to support students and enrich their university experience, but in many instances widening participation students use these provisions less.

The KCLxBIT project has explored whether behaviourally inspired messages might offer a way of increasing student engagement, with a particular interest in the effect these messages have on widening participation learners.

Our trials have tested both whether receiving a message will increase the likelihood a student will engage with the services, and also whether the type of message received will produce differential outcomes in behaviour for different student groups.

Between September 2016 and February 2017 we carried out five large-scale message trials, each involving around 4,000 first year students. Our aim was to increase engagement with the following services:

  1. Student union Welcome Fair
  2. Study abroad
  3. Compass student advice services
  4. KLaSS online study resources
  5. King’s Connect mentoring platform

All trials included text messages, and two were a combination of texts and e-mails. The trials incorporated a range of behavioural insights and were informed by the Behavioural Insight Team’s EAST framework.

All messages were personalised, meaning we addressed students by their first name – a key EAST framework principle.

The trials were carried out in collaboration with:

  • King’s College London Students Union (KCLSU)
  • King’s Study Abroad
  • The Compass (King’s cross-campus support service)
  • KEATS (King’s primary online learning environment) & IT services
  • King’s Alumni Relations team

The trials were designed as follows.

I. KCLSU Welcome Fair

The student union’s Welcome Fair takes place at King’s in September as part of Welcome Week, and is a key moment for new students to find out about, and sign-up to, student societies.

We wanted to test whether messages focused on the employability benefits of societies, or messages focused on the social belonging aspect of societies, would be more effective at encouraging students to attend the Welcome Fair and sign up for societies.

The belonging messages addressed the fact that many students worry about making friends at university, but that clubs and societies are a great way of meeting new people. The employability message, meanwhile, emphasised the value placed on societies or clubs by future employers.

Each trial arm received a total of 3 text messages, with examples given below.

Control Employability Belonging

[No messages]

Hi Kate. Build your skills & networks by joining a society or club.  Employers value these experiences. Explore Welcome Fair today or tomorrow @ Barbican Centre and see what’s on offer. #link

Hi Kate, lots of students are concerned about making friends in their first few weeks at uni. Don’t worry! There is a society or club for everyone.  Find yours at Welcome Fair @ Barbican Centre today & tomo: #link

II. Study Abroad applications

Students from widening participation backgrounds are less likely to apply to study abroad, which is often associated with more positive labour market outcomes, including a higher employment rate and higher salaries.

A previous trial, in the first year of this project, increased the number of students attending an information session about studying abroad. This time we wanted to test the relative strength of messages around the benefits or perceived barriers of studying abroad on both attendance at the King’s Study Abroad Fair and subsequent applications to study abroad opportunities.

As we had already established the positive impact of text messages on information session attendance, the control group for this trial received three basic messages with information about the Study Abroad Fair. The remaining first year students received four messages either focused on dispelling the potential barriers students might perceive in taking up these opportunities, or emphasising the benefits gained from studying abroad.

One group received a mixture of these messages in case a combination of the two turned out to be most effective. To control for ordering effects, the messages alternated between benefits and barriers-focused messages, with half the group receiving a message on benefits first and the other half receiving a message on barriers first.

Control Benefits Barriers Benefits + Barriers

3 messages

4 messages

4 messages

4 messages

Hi Kate,
King’s offers lots of ways to study abroad.
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

Hi Kate,
Studying abroad is an incredible opportunity to travel the world, experience a different culture, and make lifelong friends.
“What I loved was the atmosphere, and the people were so welcoming.”
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

Hi Kate,
Lots of students worry about the cost of studying abroad, but for King’s students it is often cheaper. For example, for one semester abroad you pay at least £3000 LESS in tuition fees for the year.
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

A combination of messages from the ‘Benefits’ and ‘Barriers’ arms

III. Engagement with the Compass

The Compass is the King’s student advice service, providing information and support on academic, personal and financial issues, and with service desks located at all King’s libraries.

This trial tested whether portraying a need for support as a normal part of the university experience would be effective in encouraging students to seek advice and guidance from the Compass staff, beyond the impact of basic information about the service.

Students in the two trial arms received one text message and one e-mail. Messages in the ‘factual’ trial arm simply provided students with information about the Compass services. The ‘belonging’ arm had the same content, but reassured students that it is normal to struggle in the first term at university and suggested the services provided by the Compass as a way of accessing support.

Control Factual Belonging

[No messages]

Hi Kate. The Compass team provide information and support on everything from academic to personal and financial challenges. Find out more: #link

Hi Kate,
You’ve now been part of the King’s community for a term, and first year students have told us it’s good to have some extra support at this time of year.
The Compass team provide information and support on everything from academic to personal and financial challenges. Find out more: #link

IV. Sign-ups on KLaSS

KLaSS is an online study skills hub, available to all King’s students, from where they can modules to support them in their studies.

For this trial we aimed to encourage students to sign-up to modules over their winter break, in the run-up to January exams.

One trial arm received information about KLaSS alongside an encouragement to make a plan for when they were going to spend some time exploring the resources available. The key behavioural insight here is that you are more likely to complete an action if you plan when you will do it.

The other treatment arm received this same message but it included additional content highlighting that many study skills at university are new to students, and that the KLaSS modules would provide additional support.

Control Planning Planning + Belonging

[No messages]

Hi Kate. Boost your academic performance over the holidays with King’s Learning & Skills Service (KLaSS). It can help with a range of key study skills.
We’ve sent you an email with more info or sign up now: #link

Hi Kate,
Lots of King’s 1st years find adapting to university study takes time.
Boost your academic performance over the holidays with King’s Learning & Skills Service (KLaSS). It can help with a range of key study skills.
We’ve sent you an email with more info or sign up now: #link

V. Sign-ups on King’s Connect

Our final engagement trial focused on increasing sign-ups to King’s Connect, an online platform where students can connect with King’s alumni.

The trial arms received information about the platform via two text messages. For one trial arm these messages just provided information about King’s Connect, whilst another arm also emphasised that King’s Connect is a unique opportunity for King’s students, and added a bit of loss aversion for good measure (“don’t miss out!”).

Control Factual Factual + King’s Opportunity

[No messages]

Hi Kate.
King’s Connect lets you contact 1800 King’s alumni to build mentoring relationships. They can provide support to you through your studies and help you think through questions from module choices to summer plans.
Sign up here: #link

Hi Kate,
King’s has alumni all over the world working in incredible jobs. As our student you have a unique opportunity to speak to them and learn from their experiences.
King’s Connect lets you contact 1800 King’s alumni to build mentoring relationships. They can provide support to you through your studies and help you think through questions from module choices to summer plans.
Don’t miss out, sign up here: #link

The results
What we found fascinating about this project is the valuable insights these nudge trials provide, even when (or perhaps especially when) the trials don’t produce the kind of results we were expecting.

In the spirit of full disclosure, while we have had some important successes, not all trials have produced the outcomes we would have hoped. This proves how important it is to properly test new interventions.

We will be publishing our results on this blog over the coming months, and will be discussing the main takeaway points from all trials – as each of them has definitely given us plenty of food for thought.

Click here to join our mailing list.
Follow us on Twitter: @KCLWhatWorks

1 Trackback / Pingback

  1. What did the students think? Feedback from our trial participants | Behavioural insights

Leave a Reply

Your email address will not be published.


*