Category: Technologies

Introduction

In the Law Faculty, we trialed providing recorded audio feedback for formative assessments in Contract Law.  The result?  Of the 91 students who had received either recorded audio feedback or written feedback for their assessments, 66% preferred recorded audio feedback.  3 of the 4 staff members who responded to our survey also preferred providing recorded audio feedback.

Why?

The NSS score for “Assessment and Feedback” at KCL remains stubbornly low at 68%. With the growing body of literature which extols the virtues of recorded audio feedback, we decided to see if it works for Law. We also wanted to investigate the effect it has on students with particular specific learning difficulties.

How?

The Technology

The platform which we used for providing feedback was the Assignment Tool through KEATS (Moodle). Turnitin also has an audio feedback tool but this only allows for three minutes of feedback to be recorded which felt unnecessarily restrictive.

The Law School Professional Services team set up the Assignment Tool submission link on KEATS. They also created a drop-down menu which allowed for grouping of the students by tutorial group. The Law TEL team created bespoke guidance for the teaching team on using the chosen technologies.

The student submission process was almost identical to the standard Turnitin submission process and we doubt whether many of the students even noticed the difference.

We used Kaltura Express Capture to provide the feedback. This was easy to use (although it was reported by tutors that in order to record the feedback it did necessitate rather a lot of “click throughs”).

The Feedback Process

In our instructions to students, we requested that they number the paragraphs in their submission to make it easier for us to refer verbally to specific points in their answers.

In our training for markers, we suggested that the feedback should be between 3 and 5 minutes long. We suggested the following structure to markers:

1. Start with some general positive feedback about the answer.
2. Identify some specific areas for improvement. It is really important that you explain how they can improve rather than just pointing out what has gone wrong.
3. Identify specific things which they have done well. (These could obviously be intertwined with the specific areas for improvement above).
4. Please finish on some really positive things about the answer. Whilst it is important for students to have some specific ways in which they can improve, students should come away from this experience feeling encouraged.
5. In terms of the language which should be used, when you are pointing out some of the negative things, please do not use the second person e.g. “You did not explain this area very clearly.” Instead, use language such as: “It would have been better if the answer explained this area more clearly.” It is preferable not to pin the negative aspects on students personally.

The Student Survey

Students completed three pieces of formative work for Contract Law. Students either received written feedback, recorded audio feedback or a combination of recorded audio feedback with a few comments on the paper. We only surveyed those students who had received recorded audio feedback or recorded audio feedback with some comments written on the paper for at least one formative assessment (i.e. we did not survey those students who had only received written feedback).

We were interested in finding out about the experience of students with a specific learning difficulty. Consequently, in our survey we asked whether students had been diagnosed with or suspected they may have: dyslexia, dyscalculia, dyspraxia, dysgraphia or attention deficit hyperactivity disorder.

Student Survey Results

Recorded Audio Feedback versus Written Feedback:

• 91 students (out of a cohort of approximately 250) responded who had received recorded audio feedback for at least one assessment and who had not experienced the combination of recorded audio feedback and written feedback.
• 60 preferred recorded audio feedback.
• 31 preferred written feedback.

5 of these students had been diagnosed / suspected they have a specific learning difficulty. Of these 5, 4 preferred recorded audio feedback and 1 preferred written feedback.

Recorded Audio Feedback versus a combination of Recorded Audio Feedback and Written Feedback or Written Feedback on its own:

16 students received a combination of recorded audio feedback and written feedback for at least 1 assessment.

• 10 preferred the combination.
• 1 preferred recorded audio feedback.
• 5 preferred written feedback.

5 of these students had been diagnosed / suspected they have a specific learning difficulty. Of these 5, 2 preferred the combination and 3 preferred written feedback.

Staff Survey

4 members of staff (out of a team of 7 colleagues with marking responsibilities on the module who provided recorded audio feedback) responded to the survey. Of those, 3 of the 4 preferred providing recorded audio feedback and 1 preferred providing written feedback. None of the 4 members of staff who responded to the survey had provided a combination of recorded audio feedback with a few comments on the paper.

Technology Gripes

Whilst the tech served the desired purpose, there are a number of ways in which it could be stream-lined. If the following issues with the technology were resolved, this would make the process much easier for staff:

• Leaving audio feedback with Kaltura Express Capture requires an irritating number of clicks.
• You cannot pause Kaltura Express Capture (e.g. if you lose your train of thought.)
• Captions are not switched on automatically. This means that each tutor has to click on each piece of feedback to turn them on.
• The Assignment Tool does not allow the feedback to be set to be released automatically on a certain date in advance. Although it can all be released in one batch, you have manually to release it by clicking a button (not a big issue but you need to put a note in the diary to remember!)
• Obtaining viewing analytics is not as streamlined as Turnitin’s audio feedback feature and requires manually going through each video on Kaltura.

The Outcomes / Lessons Learnt

When given the choice between recorded audio feedback and written feedback, the majority of students prefer recorded audio feedback. The most common 5 reasons they gave were:

1. I felt that it was more personal.
2. I felt that the explanations were easier to understand;
3. It was helpful to hear the tone of my tutor’s voice;
4. The volume (i.e. amount) of feedback was greater.
5. The process of receiving recorded audio feedback is more active than receiving written feedback.

In terms of students with specific learning difficulties, the majority of these students also preferred audio feedback to written feedback.

When given the choice between recorded audio feedback, written feedback or recorded audio feedback with some comments written on the paper, students preferred the combination. (Please note, however, that the sample of students who were given the combination was very small and it was only 1 tutor who provided this method of feedback.)
However, students reporting specific learning difficulties did not favour combined audio and written feedback. Given the option, the preference for them was a return to the familiarity of purely written feedback. It may have been that the combination of both feedback formats made it more challenging for the students with specific learning difficulties to interpret the key take-home message(s) from the feedback. Thus, the combined approach, rather than clarifying or supporting the comments made, actually muddied the waters.

When it came to staff, the most common reasons why the three staff members preferred providing recorded audio feedback to written feedback were as follows:

1. I found it faster to provide recorded audio feedback.
2. I felt that it was more personal.
3. I felt that my explanations were easier to understand.
4. I liked the fact that the students could hear the tone of my voice.
5. I thought that the process of students receiving recorded audio feedback was more active than receiving written feedback.

Conclusion / Recommendations

The commentary from our students and their clear preference for audio feedback supports current pedagogical research regarding the benefits of recording audio feedback above traditional written feedback, such as the perception amongst our students that the feedback experience is richer and more personal. We know from Caruthers et al (2015) that this leads students to be more likely to engage with their feedback. Also, the perception that feedback was both greater in volume yet easier to understand supports Chaing’s (2009) claim that audio feedback provides a greater depth to feedback in comparison to uncontextualized written feedback comments.

Though our overall cohort of students who declared a specific learning difficulty was admittedly small in our data set, it was still interesting to see that these students experienced recorded audio feedback in a more positive way than compared with their past feedback experiences. Though it is clear that for these students particularly, a mixed approach to feedback (i.e. providing a combination of both audio and written feedback) should be avoided.

Overall, colleagues’ experience of providing audio feedback was positive and once familiar with the recording process and given clear support on how to format their audio feedback (including the appropriate length for such feedback to ensure consistency amongst markers), colleagues were able to leave (according to our students) greater volumes of clear, individualized, meaningful feedback. We would, therefore, recommend that other colleagues who have not yet provided audio feedback to their students try the medium as our experience has been, overall, extremely positive from both a student and staff perspective.

Useful Links

Audio Feedback

About The Authors

Caroline van Hensbergen is a Senior Lecturer in Law (Education) at the Dickson Poon School of Law, King’s College London.

Dr Michelle Johnson is a Lecturer in Law (Education) and Faculty Inclusive Education Lead at the Dickson Poon School of Law, King’s College London.