By Lucy Makinson, Behavioural Insights Team
I. The benefits of participation in student societies
Participating in extracurricular activities is an important part of being at university. It is an opportunity to meet other students, build social networks, and develop soft skills which are often valued by employers and can improve labour market outcomes after graduation. Participation can also improve outcomes within university by improving academic attitudes.
In addition, being a part of a university club or society may enhance students’ sense of belonging within university which in turn has positive impacts for academic engagement and retention.
There is some evidence from the US that the benefits of campus participation are particularly significant for students from ethnic minority and low-income backgrounds. However, a recent study from the University of Edinburgh found students from Widening Participation backgrounds are half as likely to hold positions in the university’s clubs and societies. There may be several factors driving this – one is that students from low-income backgrounds are less likely to participate in extra-curriculars at school, where they may have also had less extra-curricular provision, with school participation being a good predictor of extra-curricular participation at university. Another is that they may underestimate the benefits of participation, particularly in relation to employment outcomes.
II. The trial
The KCL Welcome Fair runs for three days at the start of the academic year and is an opportunity for students to find out about the clubs and societies available at King’s and sign-up to join them. We therefore wanted to encourage first year students to attend the Welcome Fair, with the final aim of increasing sign-ups to clubs and societies, using a series of messages in the run-up to, and during, the event. Although we considered participation a positive outcome for all students, we were particularly interested in the attendance and sign-up rates of students from WP backgrounds.
Designing the messages
From the existing research on student society participation, as well as the wider behavioural literature, we hypothesised three key reasons for why a student may not attend the welcome fair.
1. Lack of belonging
Students who feel uncertain about whether they belong at university may be more likely to rest on their academic success to justify their being there, and see extra-curriculars as a distraction. They may also be worried about attending the fair alone, possibly through a perception that other students already know each other
2. Lack of information
Students may not perceive either the social or career benefits of being part of a society. They may also be unaware of the range of societies available, and not know where to find societies which match their interests.
3. Lack of planning or priority
Students may not attend the welcome fair due to competing priorities, not feeling like going on the day, or failing to plan appropriately (for example, by not making a clear plan of when to go and making a note of it).
Based on these hypotheses we designed two different sets of messages. One focused on addressing barriers to belonging – reassuring students that it was normal to not know anyone when arriving at university and to feel overwhelmed, but presenting societies as an opportunity to meet other students to support them through the transition. The second focused on addressing informational barriers, in particular emphasising the positive impacts of society participation on employability as well as emphasising their social benefits. Both sets of messages also drew on the extensive literature surrounding planning prompts, and encouraged students to make a specific plan with a link to the Citymapper app to help them plan their route.
Figure 1: Example messages relating to belonging and employability
The Welcome Fair ran from Friday 23rd to Sunday 25th September 2016. Ahead of this we randomly allocated all first year students to one of three groups.
The first (control) group did not receive any text messages about the Welcome Fair, although they would have received information about it through the existing communications with first year students.
The second (belonging) and third (employability) groups each received three text messages: the first message was sent the Monday before the Welcome Fair, 19th September; the second on Friday 23rd, the morning of the first day; and the final message was sent on Saturday 24th, the morning of the second day.
The messages for the ‘belonging’ group all focused on reducing perceived barriers associated with belonging, whilst those in the ‘employability’ group received messages emphasising employment benefits of societies and other informational failures.
The primary outcome measure for this trial was whether a student attended the Welcome Fair. Students entering the Welcome Fair swiped their KCL ID card in order to enter, and this data was recorded by the King’s College London Student Union (KCLSU) and subsequently shared with BIT for analysis.
The secondary outcome measure was whether a student had signed up for at least one society, either at the Welcome Fair or later in the year (data was collected in July 2017). Again, this data was provided by KCLSU who record all sign-ups to student societies throughout the year.
We found that the messages did make students significantly more likely to attend the Welcome Fair, although the most effective message appears to depend on student characteristics.
Looking across all students, the messages centred on belonging had the greatest impact on attendance, resulting in a 6 per cent increase. Employability messages, in contrast, increased attendance by just over 5 per cent, and the increase was not significant at conventional levels.
However, this trend was not true across groups. We conducted separate analyses for students in ACORN groups 1-3 which we used to define non-WP students, students in ACORN groups 4 and 5 which we used to define WP students, and students without an ACORN grouping within KCL’s record (many of these will be international students).
Amongst non-WP students, the employability messages were the most effective and increased attendance by 9 per cent, whilst the belonging messages only increased attendance by 3 per cent which was not significant at conventional levels.
Amongst WP students, the belonging message appears to be more effective indicating an increase in attendance of nearly 5 per cent, whilst attendance amongst WP students in the employability group in fact had lower attendance than those in the control group (who had not received any messages at all). It is worth noting that these changes are not significant at conventional levels, but this will be partly due to the smaller sample of WP students.
Figure 2: Differences in Welcome Fair attendance by message group and ACORN
Looking at sign-up rates to societies, we find that the employability messages increased sign-ups to societies but the belonging messages did not. This is true both at an aggregate level and for non-WP students specifically. This is interesting given that attendance was increased to a greater extent by the belonging messages. Particularly notable is that for WP students, although no sign-up effects were significant, sign-ups in the employability group were almost seven times larger than those in the belonging group relative to the control group, despite belonging messages appearing to be more effective for attendance.
Figure 3: Sign-up rates for student societies by message group
III. What we have learned
This trial has several important lessons for approaches to student engagement. Firstly, it demonstrates that text messages to students can be an effective mechanism for increasing engagement, both for one-off events (such as the Welcome Fair) and for harder to shift outcomes such as participation in clubs and societies.
However, it also demonstrates the importance of good design and rigorous testing when creating engagement initiatives. Not only did responses to the messages differ by group, which means the results of this trial can be used to improve the targeting of messages in the future, but the fact that messages around employability may have decreased attendance for WP students is evidence that texts in and of themselves may not always increase a prompted behaviour.
There are also more speculative conclusions which can be drawn from this work, which would benefit from further research. One is that messages addressing student belonging may be particularly effective for students from WP backgrounds – a hypothesis that we will test in some of our later trials to be shared through this blog. Another is that there may be deeper barriers to student participation in societies which require a wider structural change, but that student motivations for attending the Welcome Fair may make them more or less likely to overcome these barriers. For example, it has been suggested that societies themselves may be intimidating to some students, particularly those from WP backgrounds. It is possible that whilst belonging messages encourage Welcome Fair attendance, those looking to societies for reassurance are intimidated on arrival and subsequently do not sign up, whereas those who are looking to societies for long-term benefits such as employability are less affected by societies which may appear intimidating.
This is just one possible explanation for why the messages do not appear to impact attendance and society sign-up – there are many more possibilities to be considered which may have different implications for deeper interventions.
 Clark, G., Marsden, R., Whyatt, J.D., Thompson, L. and Walker, M., 2015. ‘It’s everything else you do…’: Alumni views on extracurricular activities and employability. Active Learning in Higher Education, 16(2), pp.133-147.
 Andrews, J., & Higson, H. (2008). Graduate employability,‘soft skills’ versus ‘hard’ business knowledge: A European study. Higher education in Europe, 33(4), 411-422.
 Stuart, M., Lido, C., Morgan, J., Solomon, L. and May, S., 2011. The impact of engagement with extracurricular activities on the student experience and graduate outcomes for widening participation populations. Active Learning in Higher Education, 12(3), pp.203-215.
 Fredricks J.A., and Eccles J.S. (2006). Is extracurricular participation associated with beneficial outcomes? Concurrent and longitudinal relations. Developmental Psychology, 42(4): 698–713.
 Darling, N. (2005). Participation in extracurricular activities and adolescent adjustment: cross-sectional and longitudinal findings. Journal of Youth and Adolescence, 34(5): 493– 505.
 Kuh, G.D., Cruce, T.M., Shoup, R., Kinzie, J. and Gonyea, R.M., 2008. Unmasking the effects of student engagement on first-year college grades and persistence. The Journal of Higher Education, 79(5), pp.540-563.
 Sandford-Ward, M. (2016). EUSA study shows need for widening participation in University societies. [online] The Student Newspaper. Available at: http://www.studentnewspaper.org/eusa-study-shows-need-for-widening-participation-in-university-societies/ [Accessed 16 Aug. 2016].
 White, A.M. and Gager, C.T., 2007. Idle hands and empty pockets? Youth involvement in extracurricular activities, social capital, and economic status.Youth & Society.
 Demos. (2015). Learning by Doing. [online] Available at: http://www.demos.co.uk/project/learning-by-doing/ [Accessed 16 Aug. 2016].
 Berk, L. E., & Goebel, B. L. (1987). Patterns of extracurricular participation from high school to college. American Journal of Education, 468-485.
 Randomisation was stratified by gender and ACORN status (coded binary as ACORN 1-3 and ACORN 4-5) to ensure even representation across the three groups.
 At the start of the year all students were sent an introductory message, in which they were told that they would receive messages from Kings Tips and given the opportunity to opt out. Any students who opted out were excluded from all further text trials.
 There were two cases in which a card was not swiped and attendance was not recorded. Signs informed students that their attendance data would be collected if they swiped their card, but that they could show their ID to a security guard if they did not want this data to be collected. In addition, students who forgot their card, or hadn’t yet collected it, could enter the Welcome Fair but their attendance was not recorded.
 At an aggregate level, the increase in sign-ups due to employability messages was significant at the 10% level, but not at the 5% level.