The KCLxBIT Panel Study: our approach to recruitment and retention

By Lucy Makinson, Behavioural Insights Team

There are several aspects of a survey panel which are necessary to ensure that the data drawn from it can be meaningfully interpreted.

1. Sample size
The panel must be sufficiently large to draw reliable conclusions about the broader population. The larger the sample size, the smaller the margin of error.[1] It’s also important to remember that if you want to look at the responses of specific subgroups– in this case, students from a widening participation background- the sample size of those specific subgroups is also important.

2. Representativeness
Representativeness is how well the people on a survey panel reflect the overall population. A panel is unrepresentative if some groups are more or less prevalent in the panel than in the overall population. This is a problem if the views of the groups which are over- or under-represented are different from the population as whole. Representativeness can be tested across observable, but not unobservable, characteristics.[2]

3. Retention
In the case of panel surveys, retention is of key importance. High attrition impacts survey sample size but, often more importantly, representativeness. This is because those leaving the panel are likely to be different from those staying on it – for example, they may be less attached to the university – and therefore the views of those who are less attached to the university will become underrepresented in later waves of the survey.

As well as developing recruitment strategies which deliver on all three criteria it is important to recognise the potential trade-offs between the different criteria in selecting strategies. For example, focusing on a particular type of incentive to increase sample size can impact representativeness, but the scale of these trade-offs is often not apparent from the outset.

Our recruitment and retention strategy

Roughly 4,500 first year students enrol at King’s each year. We set out to recruit a minimum of 700 students to our panel, which would give an initial margin of error of 3.4%,[3] and still imply just a 5% margin of error if we had an attrition rate of 50%. We sought a representative panel in all respects other than WP status.[4] We wanted WP students to make up roughly 30% of the sample so that we could make meaningful comparisons between WP and non-WP responses.

Here we detail some of the core elements of our recruitment and retention strategy. However, it is important to note that the selection of recruitment strategies and the exact form of their implementation is context-dependent.

1. We recruited in waves

Recruiting in waves allowed us to target specific groups with lower response rates to ensure a balanced sample.

On 30th September 2016 1,000 potential participants were selected using stratified random sampling, stratified across gender, ethnicity and department, with widening participation students deliberately oversampled to make up 300 of the 1,000 recipients.

These 1,000 were sent a text offering them a chance to participate in the first survey on 4th October. The message also informed them that they would receive an email with the link should they prefer to complete the survey online, with emails being sent that evening.

A few days later the composition of the panel was reviewed and another round of invitations were sent if the panel sample had not reached 700. The invitations in this round oversampled demographics which had lower panel sign-up rates in the first round to achieve a balanced final panel. There was to be up to three recruitment rounds. The procedure is detailed in full below.

Figure 1: Initial recruitment procedure

Untitled Diagram (1)


2. 
We asked for upfront commitment

Individuals like to be consistent and explicit commitments can have a strong influence on future behaviour,[5] including in the context of mail-response surveys.[6] To support retention through the panel waves we told all potential panel members that there would be six surveys through the year, stated the time commitment required, and emphasised the importance of responses to every survey.

This information was presented at the start of the first survey, and respondents had to make an explicit commitment to answering all six waves in order to continue to the survey. Only on completion of this first survey would they be counted as a member of the panel.

In asking for an upfront commitment we prioritised retention through the survey above the potential negative impacts on initial sign-ups.

3. We emphasised the relevance of the study

Individuals are more likely to respond to surveys when the topic is of interest to them,[7] therefore highlighting the personal relevance and significance of a survey is a useful tool for increasing response rates. However, it could also make the panel less representative if the messages only focus on areas of interest to particular subgroups of students.

In designing messages we focused on a few ways in which the survey might be of interest to students; the focus on understanding their personal experience, the impact the survey would have on the shape of student support at King’s, and the survey’s position as a significant piece of research.

By using a range of possible points of interest, but also by keeping each one broad, we aimed to maximise the recruitment effects whilst minimising associated bias

4. We used incentives

Incentives significantly increase participation in surveys.[8] That might not be ground-breaking news, but there is a lot to the design of good incentives. For example, whilst lottery incentives are effective in many contexts, there is inconclusive evidence in the context of online panel studies[9],[10] and recent work suggests guaranteed incentives are more effective.[11]

The type of incentive is also important. Incentives can bias the panel composition if they offer something which is of greater value to certain groups. In addition, incentives must not significantly change the students’ university experience – for this reason we did not use incentives which were connected to King’s (such as food or drink tokens for student cafes).

For this survey, participants received a voucher for Marks & Spencer following the completion of each wave. This was framed as a ‘thank you’ rather than a payment, to reduce potential crowding-out of intrinsic motivations for survey completion.

5. We sent (lots of) reminders, and personalised them

Students were sent up to four reminders for each wave of the survey, using both SMS messages and email. Reminders drew on a wide range of behavioural approaches, including those used in the initial recruitment. For example, they often reminded panel members of the commitment they had made at the start of the survey, and reiterated the potential impact of the survey.

In later waves the reminders were adapted based on the response pattern of the panel member. Those who had missed a recent wave could receive a message inviting them to return, whilst persistent responders were thanked for their commitment.

Our final sample

We recruited 769 first year students to be part of the panel. The panel was representative of the student population on all observable characteristics other than gender, with females being slightly more prevalent in the panel than in the first year student population at King’s. However, the extent of this divide was minimised through our wave-based recruitment which heavily targeted male students in the later waves.

Neither the slight underrepresentation of the Social Sciences and Public Policy faculty, nor the slight overrepresentation of Life Sciences and Medicine students was statistically significant. However, the final analysis of the panel will weight the data to ensure fair representation in responses.

Figure 2: Panel representativeness

panel retention4

We also achieved a reasonably high retention rate throughout our panel, with over 60% of panel members responding to our final survey. This was particularly notable as the final wave took place a considerable period of time after the conclusion of the academic year, when it is often harder to engage students.

Figure 3: Panel retention across the waves

rentention2

We will be making more details of the survey content available in October, but this will only be available through our mailing list.

If you would like to be added to our mailing list contact lifecycle@kcl.ac.uk.

Remember you can also follow us on Twitter @KCLxBIT to receive alerts of new blog posts.


Footnotes

[1] The margin of error specifies how close to the views of the population you can expect the survey responses to be. A 3% margin of error says that responses to the survey will be within 3% of the responses you would get by asking everyone in the population. This will be true 95% of the time, if the confidence interval is 95%.

[2] Observable characteristics are those we have information on, such as gender or subject studied. There are lots of possible unobserved variables which may affect responses but for which we don’t have data, for example how many other students the respondent knew when they arrived at KCL.

[3] Using a 95% confidence interval.

[4] We defined WP students as those with an ACORN consumer classification of 4 or 5.

[5] Lokhorst, A. M., Werner, C., Staats, H., van Dijk, E., & Gale, J. L. (2011). Commitment and behavior change: A meta-analysis and critical review of commitment-making strategies in environmental research. Environment and Behavior, 0013916511411477.

[6] Hinrichs, J. R. (1975). Effects of sampling, follow-up letters, and commitment to participation on mail attitude survey response. Journal of Applied Psychology, 60(2), 249.

[7] Groves, R. M., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly68(1), 2-31.

[8] Booker, C. L., Harding, S., & Benzeval, M. (2011). A systematic review of the effect of retention methods in population-based cohort studies. BMC Public Health11(1), 249.

[9] Porter, S. R., & Whitcomb, M. E. (2003). The impact of lottery incentives on student survey response rates. Research in Higher Education, 44(4), 389–407.

[10] Heerwegh, D. (2006). An investigation of the effects of lotteries on web survey response rates. Field Methods, 18(2), 205–220.

[11] Stevenson, J., Dykema, J., Cyffka, K., Klein, L. and Goldrick-Rab, S. (2012). What are the Odds? Lotteries versus Cash Incentives. Response Rates, Cost and Data Quality for a Web Survey of Low-Income Former and Current College Students. Presented at The American Association for Public Opinion Research (AAPOR) 67th Annual Conference, May 18th 2012

Introducing the KCLxBIT Panel Survey

By Lucy Makinson, Behavioural Insights Team

Why did we run a panel survey?
Our understanding of the student experience is incomplete. At present the sector relies heavily on annual national-level student surveys, such as The National Student Survey (NSS) and the Student Experience Survey (SES), which are primarily designed to compare universities, and focus on the availability and quality of university provision (such as whether the teaching provided is of a high standard) more than the way students experience that provision and feel about university generally.1

To supplement these we can turn to a handful of one-off surveys, such as the Higher Education Academy (HEA) research into the first-year experience of university. However, as our recall of events can vary from the way we experienced them (see, for example, the Peak-End rule), we can take this research even further by asking people how they are experiencing something as the event takes place.

This is particularly important when considering wellbeing. Questions around overall wellbeing portray a very different picture from the sum of short-term happiness taken at multiple points in time, and even accurate assessments of wellbeing for the whole academic year will not capture the same information as short-term assessments.
For these reasons, tracking the experiences of a fixed cohort of students at regular points throughout the academic year will greatly change the nature, level of detail, and accuracy of the information we are able to collect.

Objectives of the survey
The KCLxBIT panel survey has two key objectives:

  1. To improve our understanding of how student engagement and wellbeing fluctuate over the course of the year through the use of regular measures of wellbeing and student activities;
  2. To identify differences (and similarities) between the university experiences of different groups of students, particularly Widening Participation and ‘traditional’ students. We focused on factors which have been shown to relate to student attrition or attainment.

Content of the panel study
The panel study consisted of six waves. Questions varied across the waves but focused primarily on key areas:

  • Wellbeing (current emotions),
  • What the student was doing (e.g. how long they were spending in the library), and
  • A range of measures found to predict student retention and attainment.

A selection of other questions were also included in specific waves. Some have been drawn from other surveys, such as the NSS, and provide useful comparators. Others have been exploratory questions, often with implications for the design of future support and of particular interest to KCLWP.

Figure 1: An overview of the timing and content of the panel survey waves.
Panel survey overview

The following presents some of the key sources for questions used through the panel.

  • Wellbeing
    In every wave we asked 21 questions to track student wellbeing. These were drawn from a range of sources, representing different aspects of wellbeing, including:

    • The Scale of Positive and Negative Effect (SPANE) which measures the balance of positive and negative emotions experienced in the preceding two weeks. It focuses on frequency of emotions rather than intensity which is found to better correlate with life satisfaction .
    • The HEPI/HEA Student Academic Experience Survey (SAES) which is more short-term focused than the SPANE and allow us to compare responses to those gathered in the national SAES.
    • The Perceived Stress Scale (PSS) which is the most widely used instrument for measuring the psychological perception of stress. This was of general interest from a student wellbeing perspective, and also because stress has been found to predict student persistence.
  • Student activities

In order to understand how students were engaging with the university and their peers, and the role of any external obligations, we adapted eight questions from the US-based National Survey of Student Engagement (NSSE) relating to participation in certain activities, such as co-curricular activities and working on coursework with other students. In the first wave we asked students how often they participate in the eight activities, and in subsequent waves students were asked how regularly they were participating. An additional question relating to the use of the King’s online learning platform, KEATS, was introduced from the second wave.

  • Factors relating to student retention

The literature on student retention focuses on two models to explain why students choose to persist or drop-out of university: Tinto’s theory of student departure (1975, 1987), [otherwise known as the Student Integration Model] and Bean’s model of student attrition (1980, 1983), [the Student Attrition Model]. We drew on two papers which had used surveys to evaluate these models, Pascarella & Terenzini (1980) and Cabrera et al. (1992), and adapted specific subscales which were found to predict student retention, including; peer-group interactions, external support, interactions with faculty, and academic and intellectual development.

The results
Over the next few months we will be releasing select results from the panel survey on this blog, along with more detail on our experience of running the survey.
We will also be making more details of the survey content available in October, but this will only be available through our mailing list.
If you would like to be added to our mailing list, contact lifecycle@kcl.ac.uk.
Remember to follow us on Twitter @KCLxBIT to receive alerts of new blog posts.

__________________________________________________________________________

Footnotes

1. The wellbeing questions in the Student Academic Experience Survey (SAES) are a notable exception.

Five trials to increase student engagement through text messages

By Maija Koponen, King’s College London

Universities offer a range of services and activities designed to support students and enrich their university experience, but in many instances widening participation students use these provisions less.

The KCLxBIT project has explored whether behaviourally inspired messages might offer a way of increasing student engagement, with a particular interest in the effect these messages have on widening participation learners.

Our trials have tested both whether receiving a message will increase the likelihood a student will engage with the services, and also whether the type of message received will produce differential outcomes in behaviour for different student groups.

Between September 2016 and February 2017 we carried out five large-scale message trials, each involving around 4000 first year students. Our aim was to increase engagement with the following services:

  1. Student union Welcome Fair
  2. Study abroad
  3. Compass student advice services
  4. KLaSS online study resources
  5. King’s Connect mentoring platform

All trials included text messages, and two were a combination of texts and e-mails. The trials incorporated a range of behavioural insights and were informed by the Behavioural Insight Team’s EAST framework.

All messages were personalised, meaning we addressed students by their first name – a key EAST framework principle.

The trials were carried out in collaboration with:

  • King’s College London Students Union (KCLSU)
  • King’s Study Abroad
  • The Compass (King’s cross-campus support service)
  • KEATS (King’s primary online learning environment) & IT services
  • King’s Alumni Relations team

The trials were designed as follows.

1) KCLSU Welcome Fair

The student union’s Welcome Fair takes place at King’s in September as part of Welcome Week, and is a key moment for new students to find out about, and sign-up to, student societies.

We wanted to test whether messages focused on the employability benefits of societies, or messages focused on the social belonging aspect of societies, would be more effective at encouraging students to attend the Welcome Fair and sign up for societies.

The belonging messages addressed the fact that many students worry about making friends at university, but that clubs and societies are a great way of meeting new people. The employability message, meanwhile, emphasised the value placed on societies or clubs by future employers.

Each trial arm received a total of 3 text messages, with examples given below.

Control Employability Belonging

[No messages]

Hi Kate. Build your skills & networks by joining a society or club.  Employers value these experiences. Explore Welcome Fair today or tomorrow @ Barbican Centre and see what’s on offer. #link

Hi Kate, lots of students are concerned about making friends in their first few weeks at uni. Don’t worry! There is a society or club for everyone.  Find yours at Welcome Fair @ Barbican Centre today & tomo: #link

2) Study Abroad applications

Students from widening participation backgrounds are less likely to apply to study abroad, which is often associated with more positive labour market outcomes, including a higher employment rate and higher salaries.

A previous trial, in the first year of this project, increased the number of students attending an information session about studying abroad. This time we wanted to test the relative strength of messages around the benefits or perceived barriers of studying abroad on both attendance at the King’s Study Abroad Fair and subsequent applications to study abroad opportunities.

As we had already established the positive impact of text messages on information session attendance, the control group for this trial received three basic messages with information about the Study Abroad Fair. The remaining first year students received four messages either focused on dispelling the potential barriers students might perceive in taking up these opportunities, or emphasising the benefits gained from studying abroad.

One group received a mixture of these messages in case a combination of the two turned out to be most effective. To control for ordering effects, the messages alternated between benefits and barriers-focused messages, with half the group receiving a message on benefits first and the other half receiving a message on barriers first.

Control Benefits Barriers Benefits + Barriers

3 messages

4 messages

4 messages

4 messages

Hi Kate,
King’s offers lots of ways to study abroad.
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

Hi Kate,
Studying abroad is an incredible opportunity to travel the world, experience a different culture, and make lifelong friends.
“What I loved was the atmosphere, and the people were so welcoming.”
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

Hi Kate,
Lots of students worry about the cost of studying abroad, but for King’s students it is often cheaper. For example, for one semester abroad you pay at least £3000 LESS in tuition fees for the year.
More info @ the Study Abroad Fair.
Tues 11-2 @ Great Hall, Strand campus
#link

A combination of messages from the ‘Benefits’ and ‘Barriers’ arms

3) Engagement with the Compass

The Compass is the King’s student advice service, providing information and support on academic, personal and financial issues, and with service desks located at all King’s libraries.

This trial tested whether portraying a need for support as a normal part of the university experience would be effective in encouraging students to seek advice and guidance from the Compass staff, beyond the impact of basic information about the service.

Students in the two trial arms received one text message and one e-mail. Messages in the ‘factual’ trial arm simply provided students with information about the Compass services. The ‘belonging’ arm had the same content, but reassured students that it is normal to struggle in the first term at university and suggested the services provided by the Compass as a way of accessing support.

Control Factual Belonging

[No messages]

Hi Kate. The Compass team provide information and support on everything from academic to personal and financial challenges. Find out more: #link

Hi Kate,
You’ve now been part of the King’s community for a term, and first year students have told us it’s good to have some extra support at this time of year.
The Compass team provide information and support on everything from academic to personal and financial challenges. Find out more: #link

4) Sign-ups on KLaSS

KLaSS is an online study skills hub, available to all King’s students, from where they can modules to support them in their studies.

For this trial we aimed to encourage students to sign-up to modules over their winter break, in the run-up to January exams.

One trial arm received information about KLaSS alongside an encouragement to make a plan for when they were going to spend some time exploring the resources available. The key behavioural insight here is that you are more likely to complete an action if you plan when you will do it.

The other treatment arm received this same message but it included additional content highlighting that many study skills at university are new to students, and that the KLaSS modules would provide additional support.

Control Planning Planning + Belonging

[No messages]

Hi Kate. Boost your academic performance over the holidays with King’s Learning & Skills Service (KLaSS). It can help with a range of key study skills.
We’ve sent you an email with more info or sign up now: #link

Hi Kate,
Lots of King’s 1st years find adapting to university study takes time.
Boost your academic performance over the holidays with King’s Learning & Skills Service (KLaSS). It can help with a range of key study skills.
We’ve sent you an email with more info or sign up now: #link

5) Sign-ups on King’s Connect

Our final engagement trial focused on increasing sign-ups to King’s Connect, an online platform where students can connect with King’s alumni.

The trial arms received information about the platform via two text messages. For one trial arm these messages just provided information about King’s Connect, whilst another arm also emphasised that King’s Connect is a unique opportunity for King’s students, and added a bit of loss aversion for good measure (“don’t miss out!”).

Control Factual Factual + King’s Opportunity

[No messages]

Hi Kate.
King’s Connect lets you contact 1800 King’s alumni to build mentoring relationships. They can provide support to you through your studies and help you think through questions from module choices to summer plans.
Sign up here: #link

Hi Kate,
King’s has alumni all over the world working in incredible jobs. As our student you have a unique opportunity to speak to them and learn from their experiences.
King’s Connect lets you contact 1800 King’s alumni to build mentoring relationships. They can provide support to you through your studies and help you think through questions from module choices to summer plans.
Don’t miss out, sign up here: #link

The results
What we found fascinating about this project is the valuable insights these nudge trials provide, even when (or perhaps especially when) the trials don’t produce the kind of results we were expecting.

In the spirit of full disclosure, while we have had some important successes, not all trials have produced the outcomes we would have hoped. This proves how important it is to properly test new interventions.

We will be publishing our results on this blog over the coming months, and will be discussing the main takeaway points from all trials – as each of them has definitely given us plenty of food for thought.

Don’t forget that you can follow us on Twitter @KCLxBIT to receive alerts of new blog posts.

Or, if you would like to be included in our mailing list, contact lifecycle@kcl.ac.uk

Thinking behaviourally about higher education

By Susannah Hume, Behavioural Insights Team

Small things can have a big impact on our lives. A transport museum visit might inspire you to become a train driver, while watching CSI might spark the idea of a career in forensics. Psychologists have known this for a while–that the way we interact with the world is influenced by many contextual factors, noticed and unnoticed.

However, when we’re designing policies, we often don’t consider these factors–we assume that the people we’re trying to influence are ‘rational’ in the economic sense: they weigh up the pros and cons of all their options before choosing that which, on balance, is best for them. And if the choices made don’t align with what we expect, then we look at the big levers for solutions: regulation, funding and fees, and information.

We tell the young person who wants to be a train driver to apply their skills to an engineering degree instead, because they can get a bursary and they’ll earn more at the other end; or the young person who’s applied to a forensic science course that studying chemistry will give them more flexibility.

And nothing changes.

What we’ve assumed is an information gap is actually something else–not only about the information itself, but the way in which it was given, and often the context in which it was received. This matters particularly for widening participation efforts, because the context will often depend significantly on a student’s background.

Behavioural insights and BIT

Whilst our behaviour may not seem rational to an outside observer, these ‘irrationalities’ can be remarkably consistent across individuals. Often, they result from systematic rules of thumb (or, ‘heuristics’), which we use to simplify complicated decisions we face. The work of Daniel Kahneman and Amos Tversky systematised and quantified some of these systematic deviations from the predictions of standard microeconomic models, while the work of Thaler and Sunstein, which Anne-Marie has already written about, started the work of bridging the gap between the academic insights and their practical applications.

The Behavioural Insights Team, which started life in 2010 inside No. 10 Downing Street, was the world’s first government institution dedicated to systematically applying these insights from the behavioural sciences to improve public policy. We’re now a social purpose company with offices around the world, working in almost every policy area, and with 20 governments worldwide. Our approach has two pillars:

  1. Thinking differently about how people interact with public services; and
  2. Raising the standard of evaluation applied to policy or service changes, be they big or small. Our CEO, David Halpern, is also the government’s chief advisor on the What Works programme.

A case study on university aspirations

The need to both think differently and test interventions was illustrated by a project we ran with the Somerset Challenge in 2014, investigating ways to raise university aspirations among sixth form students in the county. Working with a collection of secondary schools in Somerset we ran a study to test three interventions:

  • providing young people with information about the costs and benefits of attending university;
  • providing the same information to their parents; and
  • giving students a short talk from a former student from their area who went to university.

Perhaps at this point you would be willing to pause and think about which of the three approaches above you expect to be successful, or any that you think might not have been. Once you’ve fixed your prediction in mind, read on!

We ran this study as a Randomised Controlled Trial (RCT – a topic for a forthcoming blog post), which meant that any difference in aspiration we observed following the interventions could be attributed to the intervention they received.

Firstly, the talk significantly increased students’ interest in university and their likelihood of applying (well done if you picked that!). Further analysis revealed that this was driven by the belief that attending university would result in better friends and a more interesting life, but these students also recalled key elements of the financial information.

However, providing parents with information cards had no effect on students’ interest in attending university, while giving the same information to students actually made them less interested in attending.

If you picked that–and you haven’t already read the report–very well done. We certainly didn’t develop these cards with the expectation that they would discourage young people from university; in fact, we thought that perhaps because of the recent tuition fee increase, there might be a genuine information gap about the benefits of university that needed correcting.

However, what the results show is that the how and who of delivery may matter as much–if not more–than the content. The inspirational speaker, who was from the same background as the students he was speaking to, was able to address perceived social and identity-related barriers to university through a medium the students could relate to, and provided the financial information within this context. The cards, which just addressed financial barriers, were not able to do that.

What’s next?

Since that first trial, we’ve also published the results from another RCT, where we increased applications and acceptances by disadvantaged young people to highly selective universities, at a cost of just £45 per additional student who accepted a place. You can read more about that project here.

We’ll be writing more about the behavioural insights approach, the simple ways that we can tweak systems to help people persist and succeed in higher education, and the very exciting results from our collaboration with King’s College London over the coming months–please subscribe to the RSS feed or follow our Twitter account to be the first to hear about updates.

The beginning of KCLxBIT

By Anne-Marie Canning, King’s College London

At King’s College London we’ve always known that helping widening participation students to ‘get in’ is not enough but needs to be coupled with a focus on helping those students to ‘get on’ too. That’s why we have a full lifecycle approach to widening participation:

Full lifecycleIn recent years the Office for Fair Access has become increasingly concerned about student success as well as student access, and rightly so. We know that the student experience at university is highly stratified and the consequences of this have been expertly detailed by my colleague and friend, Dr Anna Mountford-Zimdars, in her 2015 report for HEFCE ‘Causes of differential outcomes.’ It impacts in terms of social belonging, outcomes and labour market progression. If we’re serious about enhancing social mobility trajectories we need to have a joined up approach to student access and success.

An abundance of opportunities

I often think about the student experience as something like a jewellery box. The undergraduate experience is abundant with opportunities. I imagine some learners reaching into that jewellery box and taking out internships, study abroad experiences, research projects, societies and many more ‘high return’ co-curricular activities. We know that these activities are highly valued by employers and have a positive impact on student belonging and satisfaction. ‘Who does what’ at university became a preoccupying issue for me after dipping into the datasets and observing distinct patterns of participation. These ‘game changing’ experiences were concentrated in sections of the student body and I hypothesised that we could shift this by helping students to connect with the right opportunity at the right time.

Always carry books around with you

In June 2015 I had been reading Nudge by Thaler and Sunstein. I’m not sure why I picked up the book but I was having a great time reading it when I met with Professor Jonathan Grant, Director of the Policy Institute, for an introductory meeting. I explained to Jonathan that I had been thinking about how behavioural economics could help to improve the student experience for widening participation learners at King’s. Jonathan kindly offered to introduce me to the Behavioural Insights Team (BIT) at the Cabinet Office. Fast forward a few weeks and I was presenting the challenges of full lifecycle widening participation and explaining how I thought behavioural insights could play a role in improving student outcomes and experiences to David Halpern and Raj Chande, CEO and head of education at BIT respectively. Interesting work on the application of behavioural insights to the university experience was already being led by Professor Ben Castleman (mastermind of the Obama ‘Better Make Room’ campaign and author of ‘The 160 Character Solution’) at the University of Virginia, and Professor Philip Oreopoulous (author of ‘Behavioural Economics of Education: Progress and Possibilities’) at the University of Toronto.  Following my conversation with the team at BIT they agreed that this could be an interesting collaboration and so we commenced the first ever project looking at the application of ‘nudge’ in a UK university context.

A two-year pilot project

The project was considered by the ethics office at King’s College London and we moved into a two year pilot programme overseen by an expert advisory board. Student journey workshops and panel surveys have given us rich insights into the lived student experience. What is so exciting about these models of exploration is the way in which they asked students about them rather than about the institution. This represented a significant departure from the traditional higher education satisfaction survey model and allowed us to work out what matters to students and when we might best offer an intervention. We have delivered a range of complex and simple randomised control trials within the King’s College London ecosystem using institutional datasets to measure efficacy. First year students have benefitted from a range of programmes and pointers to help them make the most of their time at the university. Acrobatic analysis and application of machine learning techniques has helped us to understand the impact of our intervention on different student groups and begin to reshape how we structure the student experience at King’s. Our blog will detail the tribulations and triumphs of our experimental approach. We look forward to sharing our results and lessons learned in the hope that others can take encouragement from our work and adopt behaviourally inspired methods in their own contexts. By bringing new ideas together with old problems I believe we can make faster progress in helping students make the most of their talents and opportunities.

Welcome to Behavioural insights in higher education

Welcome to the Behavioural Insights in Higher Education blog, which has been set up to feature results from KCLxBIT, a collaborative pilot project between King’s College London and the Behavioural Insights Team (BIT).

The project has explored whether behavioural insights can be effective in increasing student engagement and sense of belonging at King’s, with a particular focus on the impact for under-represented student groups. The project has been carried out in two pilot year phases from 2015-2017. In 2016-17, the project has been made up of seven randomised control trials and a 6 point pulse survey.

We will publish results from all trials in this blog over the coming months. Posts will also feature details of trial designs and will provide insights into some of the challenges we have faced during the project.

The posts will lead up to our official launch event, which is taking place on November 16th at King’s College London. 

The project team has been made up of the following staff members from King’s and BIT:

Anne-Marie Canning, Director of Widening Participation, King’s College London
Susannah Hume, Principal Advisor and Head of Skills, Behavioural Insights Team
Lucy Makinson, Associate Advisor, Behavioural Insights Team
Maija Koponen, Widening Participation Officer (Student Lifecycle), King’s College London
Kim Hall, Community Ambassador Project Manager, King’s College London
Clare Delargy, Advisor, Behavioural Insights Team

Follow us on Twitter @KCLxBIT to receive alerts of new blog posts.

Contact e-mail: lifecycle@kcl.ac.uk