By Maija Koponen, King’s College London |
Universities offer a range of services and activities designed to support students and enrich their university experience, but in many instances widening participation students use these provisions less.
The KCLxBIT project has explored whether behaviourally inspired messages might offer a way of increasing student engagement, with a particular interest in the effect these messages have on widening participation learners.
Our trials have tested both whether receiving a message will increase the likelihood a student will engage with the services, and also whether the type of message received will produce differential outcomes in behaviour for different student groups.
Between September 2016 and February 2017 we carried out five large-scale message trials, each involving around 4,000 first year students. Our aim was to increase engagement with the following services:
- Student union Welcome Fair
- Study abroad
- Compass student advice services
- KLaSS online study resources
- King’s Connect mentoring platform
All trials included text messages, and two were a combination of texts and e-mails. The trials incorporated a range of behavioural insights and were informed by the Behavioural Insight Team’s EAST framework.
All messages were personalised, meaning we addressed students by their first name – a key EAST framework principle.
The trials were carried out in collaboration with:
- King’s College London Students Union (KCLSU)
- King’s Study Abroad
- The Compass (King’s cross-campus support service)
- KEATS (King’s primary online learning environment) & IT services
- King’s Alumni Relations team
The trials were designed as follows.
I. KCLSU Welcome Fair
The student union’s Welcome Fair takes place at King’s in September as part of Welcome Week, and is a key moment for new students to find out about, and sign-up to, student societies.
We wanted to test whether messages focused on the employability benefits of societies, or messages focused on the social belonging aspect of societies, would be more effective at encouraging students to attend the Welcome Fair and sign up for societies.
The belonging messages addressed the fact that many students worry about making friends at university, but that clubs and societies are a great way of meeting new people. The employability message, meanwhile, emphasised the value placed on societies or clubs by future employers.
Each trial arm received a total of 3 text messages, with examples given below.
| Control | Employability | Belonging |
|
[No messages] |
Hi Kate. Build your skills & networks by joining a society or club. Employers value these experiences. Explore Welcome Fair today or tomorrow @ Barbican Centre and see what’s on offer. #link |
Hi Kate, lots of students are concerned about making friends in their first few weeks at uni. Don’t worry! There is a society or club for everyone. Find yours at Welcome Fair @ Barbican Centre today & tomo: #link |
II. Study Abroad applications
Students from widening participation backgrounds are less likely to apply to study abroad, which is often associated with more positive labour market outcomes, including a higher employment rate and higher salaries.
A previous trial, in the first year of this project, increased the number of students attending an information session about studying abroad. This time we wanted to test the relative strength of messages around the benefits or perceived barriers of studying abroad on both attendance at the King’s Study Abroad Fair and subsequent applications to study abroad opportunities.
As we had already established the positive impact of text messages on information session attendance, the control group for this trial received three basic messages with information about the Study Abroad Fair. The remaining first year students received four messages either focused on dispelling the potential barriers students might perceive in taking up these opportunities, or emphasising the benefits gained from studying abroad.
One group received a mixture of these messages in case a combination of the two turned out to be most effective. To control for ordering effects, the messages alternated between benefits and barriers-focused messages, with half the group receiving a message on benefits first and the other half receiving a message on barriers first.
| Control | Benefits | Barriers | Benefits + Barriers |
|
3 messages |
4 messages |
4 messages |
4 messages |
|
Hi Kate, |
Hi Kate, |
Hi Kate, |
A combination of messages from the ‘Benefits’ and ‘Barriers’ arms |
III. Engagement with the Compass
The Compass is the King’s student advice service, providing information and support on academic, personal and financial issues, and with service desks located at all King’s libraries.
This trial tested whether portraying a need for support as a normal part of the university experience would be effective in encouraging students to seek advice and guidance from the Compass staff, beyond the impact of basic information about the service.
Students in the two trial arms received one text message and one e-mail. Messages in the ‘factual’ trial arm simply provided students with information about the Compass services. The ‘belonging’ arm had the same content, but reassured students that it is normal to struggle in the first term at university and suggested the services provided by the Compass as a way of accessing support.
| Control | Factual | Belonging |
|
[No messages] |
Hi Kate. The Compass team provide information and support on everything from academic to personal and financial challenges. Find out more: #link |
Hi Kate, |
IV. Sign-ups on KLaSS
KLaSS is an online study skills hub, available to all King’s students, from where they can modules to support them in their studies.
For this trial we aimed to encourage students to sign-up to modules over their winter break, in the run-up to January exams.
One trial arm received information about KLaSS alongside an encouragement to make a plan for when they were going to spend some time exploring the resources available. The key behavioural insight here is that you are more likely to complete an action if you plan when you will do it.
The other treatment arm received this same message but it included additional content highlighting that many study skills at university are new to students, and that the KLaSS modules would provide additional support.
| Control | Planning | Planning + Belonging |
|
[No messages] |
Hi Kate. Boost your academic performance over the holidays with King’s Learning & Skills Service (KLaSS). It can help with a range of key study skills. |
Hi Kate, |
V. Sign-ups on King’s Connect
Our final engagement trial focused on increasing sign-ups to King’s Connect, an online platform where students can connect with King’s alumni.
The trial arms received information about the platform via two text messages. For one trial arm these messages just provided information about King’s Connect, whilst another arm also emphasised that King’s Connect is a unique opportunity for King’s students, and added a bit of loss aversion for good measure (“don’t miss out!”).
| Control | Factual | Factual + King’s Opportunity |
|
[No messages] |
Hi Kate. |
Hi Kate, |
The results
What we found fascinating about this project is the valuable insights these nudge trials provide, even when (or perhaps especially when) the trials don’t produce the kind of results we were expecting.
In the spirit of full disclosure, while we have had some important successes, not all trials have produced the outcomes we would have hoped. This proves how important it is to properly test new interventions.
We will be publishing our results on this blog over the coming months, and will be discussing the main takeaway points from all trials – as each of them has definitely given us plenty of food for thought.
Click here to join our mailing list.
Follow us on Twitter: @KCLWhatWorks
Leave a Reply