By Lucy Makinson, Behavioural Insights Team |
Why did we run a panel study?
Our understanding of the student experience is incomplete. At present the sector relies heavily on annual national-level student surveys, such as The National Student Survey (NSS) and the Student Experience Survey (SES), which are primarily designed to compare universities, and focus on the availability and quality of university provision (such as whether the teaching provided is of a high standard) more than the way students experience that provision and feel about university generally.1
To supplement these we can turn to a handful of one-off surveys, such as the Higher Education Academy (HEA) research into the first-year experience of university. However, as our recall of events can vary from the way we experienced them (see, for example, the Peak-End rule), we can take this research even further by asking people how they are experiencing something as the event takes place.
This is particularly important when considering wellbeing. Questions around overall wellbeing portray a very different picture from the sum of short-term happiness taken at multiple points in time, and even accurate assessments of wellbeing for the whole academic year will not capture the same information as short-term assessments.
For these reasons, tracking the experiences of a fixed cohort of students at regular points throughout the academic year will greatly change the nature, level of detail, and accuracy of the information we are able to collect.
Objectives of the survey
The KCLxBIT Pulse Survey had two key objectives:
- To improve our understanding of how student engagement and wellbeing fluctuate over the course of the year through the use of regular measures of wellbeing and student activities;
- To identify differences (and similarities) between the university experiences of different groups of students, particularly Widening Participation and ‘traditional’ students. We focused on factors which have been shown to relate to student attrition or attainment.
Content of the Pulse Survey
The panel study consisted of six waves. Questions varied across the waves but focused primarily on key areas:
- Wellbeing (current emotions),
- What the student was doing (e.g. how long they were spending in the library), and
- A range of measures found to predict student retention and attainment.
A selection of other questions were also included in specific waves. Some have been drawn from other surveys, such as the NSS, and provide useful comparators. Others have been exploratory questions, often with implications for the design of future support and of particular interest to KCLWP.
The following presents some of the key sources for questions used through the panel.
Wellbeing
- In every wave we asked 21 questions to track student wellbeing. These were drawn from a range of sources, representing different aspects of wellbeing, including:
-
- The Scale of Positive and Negative Effect (SPANE) which measures the balance of positive and negative emotions experienced in the preceding two weeks. It focuses on frequency of emotions rather than intensity which is found to better correlate with life satisfaction .
- The HEPI/HEA Student Academic Experience Survey (SAES) which is more short-term focused than the SPANE and allow us to compare responses to those gathered in the national SAES.
- The Perceived Stress Scale (PSS) which is the most widely used instrument for measuring the psychological perception of stress. This was of general interest from a student wellbeing perspective, and also because stress has been found to predict student persistence.
Student activities
In order to understand how students were engaging with the university and their peers, and the role of any external obligations, we adapted eight questions from the US-based National Survey of Student Engagement (NSSE) relating to participation in certain activities, such as co-curricular activities and working on coursework with other students. In the first wave we asked students how often they participate in the eight activities, and in subsequent waves students were asked how regularly they were participating. An additional question relating to the use of the King’s online learning platform, KEATS, was introduced from the second wave.
Factors relating to student retention
The literature on student retention focuses on two models to explain why students choose to persist or drop-out of university: Tinto’s theory of student departure (1975, 1987), [otherwise known as the Student Integration Model] and Bean’s model of student attrition (1980, 1983), [the Student Attrition Model]. We drew on two papers which had used surveys to evaluate these models, Pascarella & Terenzini (1980) and Cabrera et al. (1992), and adapted specific subscales which were found to predict student retention, including; peer-group interactions, external support, interactions with faculty, and academic and intellectual development.
The results
Over the next few months we will be releasing select results from the Pulse Survey on this blog, along with more detail on our experience of running the survey.
We will also be making more details of the survey content available through our mailing list.
Click here to join our mailing list.
Follow us on Twitter: @KCLWhatWorks
__________________________________________________________________________
Footnotes
1. The wellbeing questions in the Student Academic Experience Survey (SAES) are a notable exception.
Leave a Reply