Category: Technologies

Evaluation and Reflection, Technologies

Analysing Student Engagement with KEATS and Academic Performance

Riaz Toorabally

Summary

In this post, we describe the research presented at the King’s Education Conference 2024, which focused on the impact of student engagement with KEATS on academic performance.

The Story

Motivation

Virtual Learning Environments like KEATS offer students convenient and flexible access to course materials. Engaging with these platforms requires a deliberate investment of time and effort from students, with the potential rewards of significantly enhancing their academic performance. Therefore, understanding this relationship is essential for creating effective online learning environments that promote student success. Our study aimed to investigate how the total hours students spent on KEATS correlates with their final exam grades.

Study Context

Our research focused on a one-year mathematics foundation module at King’s Foundation during the 2022-2023 academic year. This module followed a blended learning approach, combining online and face-to-face components. The study involved 238 students, aged 17 to 20, from different nationalities and educational backgrounds. The students were required to spend 2 hours each week on KEATS, engaging with instructional videos, practice problems, and exercises, while also attending weekly seminars and live lectures in person.

Data Collection

We analysed two main datasets: KEATS engagement data and student performance data. The KEATS engagement data included total time spent online, and the performance data consisted of grades from various summative assessments, including the final exam.

The Outcomes

Exploratory Data Analysis

We first used descriptive statistics and visualizations to assess the data. The scatter plot below suggested a linear relationship between time spent on KEATS and final exam grade, indicating that students who spent more time on KEATS generally achieved higher grades. Interestingly, the plot also revealed an empty area in the bottom right corner, showing that no students with high engagement had low final exam grades.

Figure plotting Total Hours Online vs Final Exam Grade

Identifying Patterns

Next, we categorized engagement into quartiles and found that students with higher engagement levels consistently achieved better final exam grades, as shown in the boxplots below. The analysis strongly suggests that lower engagement with KEATS is associated with lower grades and that this pattern is unlikely to be coincidental.

Figure Showing Final Exam Grades by Engagement Level

Correlation and Regression Analysis

Finally, the Pearson correlation coefficient revealed a moderate positive relationship between KEATS engagement and final exam grades (r=0.33). To further investigate this relationship, we used multiple linear regression models. The best model, which included all summative tests and total hours online, explained 39% of the variance in final exam grades. This model suggests that each additional hour spent on KEATS was associated with an increase of approximately 0.2 percentage points in the final exam grade.

Conclusion & Recommendations

Our study indicates that increased interaction with KEATS leads to better academic outcomes. The results of this analysis can inform the development of targeted strategies to support all students, promoting a more inclusive and effective learning environment.

Educational Implications

Based on our findings, we recommend the following actions:

  • Encourage active participation in KEATS to enhance academic performance. Tools like the KEATS Course Participation Report can send targeted messages and reminders to boost engagement.
  • Identify students needing additional support by analysing KEATS data and implementing timely interventions.
  • Monitor KEATS usage and assessment results regularly to track student progress and tailor pedagogical approaches.

We hope this blog provides valuable insights into the importance of student engagement with KEATS. For any questions or further discussion, please feel free to contact me at eleonora.pinto_de_moura@kcl.ac.uk

Technical Note

The data analysis was conducted in Python, using libraries such as pandas for data manipulation, matplotlib and seaborn for data visualization, and statsmodels for statistical modelling.

Useful Links

Intended Audience

Educators, Academic Researchers, Learning Technologists and Administrators.

About the Author

Image of Eleonora Pinto de MouraMy name is Eleonora Pinto de Moura, and I’m a Lecturer in Mathematics Education at King’s Foundations. My research interests include educational technology, data-driven teaching strategies, and enhancing student engagement through innovative learning digital tools.

Evaluation and Reflection, Technologies

Facilitating Groups Assessments on KEATS

Riaz Toorabally

Introduction

Increasingly, modules are diversifying their assessment types and branching out into group assessments instead of essays and other more traditional assignments. As Professional Services, we want to support this in our schools and departments wherever possible, without it adding further administrative burden or creating scope for error. The goal was to facilitate students submitting assessments in groups whilst also enabling module leads and students to get the most out of other functions in KEATS which groups allow, such as group forums, restricting content by group, and creating marking groups for markers. In other words, we wanted a smooth experience for students submitting their assessments, whilst also enabling module leads and contributors to utilise the full extent of KEATS and maximise student engagement.

The Challenge

Previously, students would be a member of only one group within a module page to keep things “simpler”, which unfortunately restricted how students and staff could engage with KEATS. Alternatively, we have kept manual records of student groups and submissions which added additional administrative burden. Neither of these solutions were the ideal we were looking for, so we decided to turn our attention to the underused groupings feature on KEATS. We knew from experience that the settings in groupings must be set up exactly or students may be prevented from submitting or submit in the wrong group. Therefore, the challenge was to learn how to set up the settings on KEATS to interact with different activities successfully. Whilst it was an investment in time and resources at the beginning to work out how groupings operate, I was hopeful that it would streamline things going forward and create a better student and staff experience.

The process commenced with an initial consultation with the SSPP TEL team. I learned that the “groupings” functions as a group of groups and different activities on KEATS could be set up to pull groups from specific groupings. For example, there may be several presentation groups within a module which could all be included in the umbrella group presentation grouping. The Moodle assignment activity associated with this group presentation could then be set up to pull groups directly from the group presentation grouping, hereby avoiding the confusion of potentially pulling the groups from the seminar groups or forum groups instead. The team set me up with a sandbox on KEATS, effectively a practice area, where I could set up the journey that a student would take.

Screenshot of Groupings in KEATS
Screenshot of Groupings in KEATS

The three main things I wished to test were:

  • signing up to a group via the Group Choice Activity
  • being added to other groups in different groupings,
  • submitting as part of a group to see if all ran smoothly.

I replicated the settings on this practice page, following SSPP TEL guidance on groups and groupings, and I enlisted my very patient and enthusiastic Professional Service colleagues from ECS to be enrolled as students on the practice page, asking them to take the journey of the students. When colleagues submitted their mock assessments, the groups were pulled from the correct assessment groups grouping. As I had selected the settings that allowed one student to upload on behalf of all their group members on the Moodle assignment activity, when one group member submitted, the correct group members were also populated with their group’s submission. It appeared the testing showed everything to go to plan and there was no scrambling of submissions. I was in the clear to replicate these settings throughout the module pages where there were group assessments.

The Outcomes/ Lessons Learnt

Outside of the specific details of how to technically set up the various activities, I learned to think holistically about the KEATS pages and always consider how one setting or action might affect and interact with other activities or areas of the page. It’s also underpinned how crucial the initial set up of activities and course pages are, so going forward, I’d always advise plenty of time and, of course, meticulous checking of these settings in advance of busy times such as submission deadlines.

Conclusion/ Recommendations

In conclusion, I’d strongly advise my colleagues, not to be scared of groupings and groups! Yes, they require careful set up, but once up and running, they enable us to utilise a variety of activities and functions on KEATS, increasing interaction and, minimising administration in the long term whilst maximising engagement for students.

Written By Joanne Devenney

Senior Programme Officer
School of Education, Communication and Society

Evaluation and Reflection, Technologies

Giving Recorded Audio Feedback is Easy!

Riaz Toorabally

Introduction

In the Law Faculty, we trialed providing recorded audio feedback for formative assessments in Contract Law.  The result?  Of the 91 students who had received either recorded audio feedback or written feedback for their assessments, 66% preferred recorded audio feedback.  3 of the 4 staff members who responded to our survey also preferred providing recorded audio feedback.

Why?

The NSS score for “Assessment and Feedback” at KCL remains stubbornly low at 68%. With the growing body of literature which extols the virtues of recorded audio feedback, we decided to see if it works for Law. We also wanted to investigate the effect it has on students with particular specific learning difficulties.

How?

The Technology

The platform which we used for providing feedback was the Assignment Tool through KEATS (Moodle). Turnitin also has an audio feedback tool but this only allows for three minutes of feedback to be recorded which felt unnecessarily restrictive.

The Law School Professional Services team set up the Assignment Tool submission link on KEATS. They also created a drop-down menu which allowed for grouping of the students by tutorial group. The Law TEL team created bespoke guidance for the teaching team on using the chosen technologies.

Image of KEATS example assignment
Image of KEATS Assignment Submission.

The student submission process was almost identical to the standard Turnitin submission process and we doubt whether many of the students even noticed the difference.

We used Kaltura Express Capture to provide the feedback. This was easy to use (although it was reported by tutors that in order to record the feedback it did necessitate rather a lot of “click throughs”).

Screenshot of assignment inbox with Kaltura recorded feedback.
Screenshot of assignment inbox with Kaltura recorded feedback.

The Feedback Process

In our instructions to students, we requested that they number the paragraphs in their submission to make it easier for us to refer verbally to specific points in their answers.

In our training for markers, we suggested that the feedback should be between 3 and 5 minutes long. We suggested the following structure to markers:

  1. Start with some general positive feedback about the answer.
  2. Identify some specific areas for improvement. It is really important that you explain how they can improve rather than just pointing out what has gone wrong.
  3. Identify specific things which they have done well. (These could obviously be intertwined with the specific areas for improvement above).
  4. Please finish on some really positive things about the answer. Whilst it is important for students to have some specific ways in which they can improve, students should come away from this experience feeling encouraged.
  5. In terms of the language which should be used, when you are pointing out some of the negative things, please do not use the second person e.g. “You did not explain this area very clearly.” Instead, use language such as: “It would have been better if the answer explained this area more clearly.” It is preferable not to pin the negative aspects on students personally.
Screenshot of recording audio feedback video tutorial.
Screenshot of recording audio feedback video tutorial.

The Student Survey

Students completed three pieces of formative work for Contract Law. Students either received written feedback, recorded audio feedback or a combination of recorded audio feedback with a few comments on the paper. We only surveyed those students who had received recorded audio feedback or recorded audio feedback with some comments written on the paper for at least one formative assessment (i.e. we did not survey those students who had only received written feedback).

We were interested in finding out about the experience of students with a specific learning difficulty. Consequently, in our survey we asked whether students had been diagnosed with or suspected they may have: dyslexia, dyscalculia, dyspraxia, dysgraphia or attention deficit hyperactivity disorder.

Student Survey Results

Recorded Audio Feedback versus Written Feedback:

  • 91 students (out of a cohort of approximately 250) responded who had received recorded audio feedback for at least one assessment and who had not experienced the combination of recorded audio feedback and written feedback.
  • 60 preferred recorded audio feedback.
  • 31 preferred written feedback.

5 of these students had been diagnosed / suspected they have a specific learning difficulty. Of these 5, 4 preferred recorded audio feedback and 1 preferred written feedback.

Recorded Audio Feedback versus a combination of Recorded Audio Feedback and Written Feedback or Written Feedback on its own:

16 students received a combination of recorded audio feedback and written feedback for at least 1 assessment.

  • 10 preferred the combination.
  • 1 preferred recorded audio feedback.
  • 5 preferred written feedback.

5 of these students had been diagnosed / suspected they have a specific learning difficulty. Of these 5, 2 preferred the combination and 3 preferred written feedback.

Staff Survey

4 members of staff (out of a team of 7 colleagues with marking responsibilities on the module who provided recorded audio feedback) responded to the survey. Of those, 3 of the 4 preferred providing recorded audio feedback and 1 preferred providing written feedback. None of the 4 members of staff who responded to the survey had provided a combination of recorded audio feedback with a few comments on the paper.

Technology Gripes

Whilst the tech served the desired purpose, there are a number of ways in which it could be stream-lined. If the following issues with the technology were resolved, this would make the process much easier for staff:

  • Leaving audio feedback with Kaltura Express Capture requires an irritating number of clicks.
  • You cannot pause Kaltura Express Capture (e.g. if you lose your train of thought.)
  • Captions are not switched on automatically. This means that each tutor has to click on each piece of feedback to turn them on.
  • The Assignment Tool does not allow the feedback to be set to be released automatically on a certain date in advance. Although it can all be released in one batch, you have manually to release it by clicking a button (not a big issue but you need to put a note in the diary to remember!)
  • Obtaining viewing analytics is not as streamlined as Turnitin’s audio feedback feature and requires manually going through each video on Kaltura.

The Outcomes / Lessons Learnt

When given the choice between recorded audio feedback and written feedback, the majority of students prefer recorded audio feedback. The most common 5 reasons they gave were:

  1. I felt that it was more personal.
  2. I felt that the explanations were easier to understand;
  3. It was helpful to hear the tone of my tutor’s voice;
  4. The volume (i.e. amount) of feedback was greater.
  5. The process of receiving recorded audio feedback is more active than receiving written feedback.

In terms of students with specific learning difficulties, the majority of these students also preferred audio feedback to written feedback.

When given the choice between recorded audio feedback, written feedback or recorded audio feedback with some comments written on the paper, students preferred the combination. (Please note, however, that the sample of students who were given the combination was very small and it was only 1 tutor who provided this method of feedback.)
However, students reporting specific learning difficulties did not favour combined audio and written feedback. Given the option, the preference for them was a return to the familiarity of purely written feedback. It may have been that the combination of both feedback formats made it more challenging for the students with specific learning difficulties to interpret the key take-home message(s) from the feedback. Thus, the combined approach, rather than clarifying or supporting the comments made, actually muddied the waters.

When it came to staff, the most common reasons why the three staff members preferred providing recorded audio feedback to written feedback were as follows:

  1. I found it faster to provide recorded audio feedback.
  2. I felt that it was more personal.
  3. I felt that my explanations were easier to understand.
  4. I liked the fact that the students could hear the tone of my voice.
  5. I thought that the process of students receiving recorded audio feedback was more active than receiving written feedback.

Conclusion / Recommendations

The commentary from our students and their clear preference for audio feedback supports current pedagogical research regarding the benefits of recording audio feedback above traditional written feedback, such as the perception amongst our students that the feedback experience is richer and more personal. We know from Caruthers et al (2015) that this leads students to be more likely to engage with their feedback. Also, the perception that feedback was both greater in volume yet easier to understand supports Chaing’s (2009) claim that audio feedback provides a greater depth to feedback in comparison to uncontextualized written feedback comments.

Though our overall cohort of students who declared a specific learning difficulty was admittedly small in our data set, it was still interesting to see that these students experienced recorded audio feedback in a more positive way than compared with their past feedback experiences. Though it is clear that for these students particularly, a mixed approach to feedback (i.e. providing a combination of both audio and written feedback) should be avoided.

Overall, colleagues’ experience of providing audio feedback was positive and once familiar with the recording process and given clear support on how to format their audio feedback (including the appropriate length for such feedback to ensure consistency amongst markers), colleagues were able to leave (according to our students) greater volumes of clear, individualized, meaningful feedback. We would, therefore, recommend that other colleagues who have not yet provided audio feedback to their students try the medium as our experience has been, overall, extremely positive from both a student and staff perspective.

Useful Links

Audio Feedback

About The Authors

Caroline van Hensbergen is a Senior Lecturer in Law (Education) at the Dickson Poon School of Law, King’s College London.

Dr Michelle Johnson is a Lecturer in Law (Education) and Faculty Inclusive Education Lead at the Dickson Poon School of Law, King’s College London.

Moodle 4, News and Events, Technologies

KEATS Upgrade: 16 July 2024

Sue Harrison

KEATS will be upgraded to Moodle 4.4.1 on Tuesday 16 July 2024. As part of this upgrade KEATS will be placed into Maintenance Mode and unavailable to staff and students between 06:00-16:00 (UK Time).

This upgrade introduces several enhancements focused on refining the user interface and making it easier to edit your KEATS pages.

Check back on the blog after upgrade for a video highlighting upgraded features.

New look KEATS for desktop and mobile

Enhancements include:

  • Activity icons have been updated with an accessible colour palette.
  • You can now quickly create content by clicking on a plus icon anywhere on your main KEATS page.
  • Text and media areas now include a ‘name’ field, which appear in the course index on the left of your page.
  • When using the grader report, you can now search, collapse columns and view feedback directly on the grader report page.
  • For those using group forums, you can now easily post to all groups with the ‘Post a copy to all groups’ option available directly underneath your forum post.
  • The Tiny MCE editor has been improved and is now the default editor, featuring a full-screen button for an enhanced editing experience.
  • Bulk actions such as deleting, moving, duplicating, hiding, or changing visibility can now be performed directly from the footer.
  • The edit menu now allows you to easily change availability options and group modes.
  • Lastly, you can now copy H5P activities in the content bank to adapt them, making content creation more flexible.

Please note, faculties and departments use KEATS in different ways and may have different templates and approaches for you to be aware of. If you are in the middle of teaching during this upgrade and are experiencing unexpected changes, please do contact your Programme Team in the first instance.

Evaluation and Reflection, News and Events, Pedagogy, Technologies

Exploring the Digital Frontier: Revolutionizing Feedback Delivery with Excel and VBA Macros Part 2

Aaron Sangha

Part 2: My experience and reflections

Part 1 of this blog can be found here: https://blogs.kcl.ac.uk/digitaleducation/exploring-the-digital-frontier-revolutionizing-feedback-delivery-with-excel-and-vba-macros-part-1/

Icons of the Microsoft 365 shown

I embarked on this journey to a few years ago when I led a large course with more than 700 students and a team of 15 markers. It was a challenging for an early career lecturer to manage the administrative tasks and collaborate with the whole team: standardization, moderation, and uploading/creating feedback documents. The demanding nature of the courses allows for no mistakes or human errors. Working through the clunky and often user-unfriendly interface of Moodle/Turnitin is another difficult obstacle: one would require the Internet to do the marking and as some other colleagues would agree that we often work better off the Internet. Traditional methods were often time-consuming and inconsistent, resulting in delayed feedback that left students wanting more. The technology-driven solution aimed to address these challenges head-on. When I joined KCL in September 2022, I faced the same problems and have been made aware of different initiatives at KBS to improve the feedback and assessment process at KBS. I gathered my own systems over the years and implemented the process during the Jan 2023 marking season. Over the Spring Term 2023, I refined the process with feedback from colleagues who shared the excitements and interest. In June 2023, I presented at the festival to share the practices and implementation strategies for an innovative automation system.

The process involved harnessing the power of Microsoft Excel and VBA Macros within Microsoft Word. These technologies allowed us to streamline and automate feedback delivery. Imagine, no more laborious hours spent typing feedback comments and no human errors involved in exporting and uploading the feedback documents to Keats/Turnitin! Instead, we could focus on providing students with valuable insights to help them excel.

Screenshot of Digital Skills Hub

**Challenges Faced:**

Of course, no transformative journey is without its challenges. Some educators were initially resistant to change, finding the prospect of learning VBA Macros daunting. Additionally, ensuring the new system was compatible with various devices and platforms presented a technical hurdle. As I mentioned in the guidance (see from my SharePoint), the set-up and troubleshooting at the beginning can be quite a challenge, particularly for colleagues using the MacOS system (it’s less so for Windows users). Compatibility issues were addressed through rigorous testing and continuous monitoring of system performance. Clear communication with your marking team is also needed to make sure everyone is on the same page with the new system.

But I promise it’s worth the effort and the subsequential usages will be a much smoother sail. And from a marker’s perspective, it is much less work than working through the traditional channels.

The journey from traditional feedback systems to an automated approach using Excel and VBA Macros has been nothing short of transformative. It’s a testament to the power of technology in education, where innovative solutions can overcome challenges and improve the overall learning experience.

As we continue this path of exploration and adaptation, the future of feedback delivery looks brighter than ever to improved student satisfaction and educational outcomes. I hope that a wider adoption of the process could help deliver a more insightful and time effective feedback to our students, thereby addressing the burning issues identified from the student surveys, as well as helping deliver impacts to the quality of feedback giving and student experience, as identified in King’s Vision 2029 and the TEF framework.

Screenshot of TEF award

It takes time and communications with colleagues to identify compatibility issues and resolve them. So far, the method has been used by six Economics courses at KBS, two from the University of Glasgow; and colleagues from Marketing and Adult Nursing, have expressed their interests in using it in their courses.
It is definitely not perfect, and I am very much looking forward to feedback, comments, and of course successful implementations of colleagues.

The blog discusses a transformative journey in education, initiated during The Festival of Technology 2022 at KCL. It explores the adoption of Excel and VBA Macros within Microsoft Word to revolutionize feedback delivery. The main reasons for this change were to enhance feedback quality and efficiency, addressing challenges like resistance to change and compatibility issues. Through workshops, ongoing support, and rigorous testing, the adoption of technology resulted in a more efficient, user-friendly, and collaborative feedback system, empowering educators and improving the overall learning experience.

I would like to thank KBS colleagues, Jack Fosten, Dragos Radu, and Chahna Gonsalves for their encouragement, important suggestions and feedback as well as allowing me to pilot the process in their modules. I also thank various colleagues across other faculties for providing feedback and suggestions as well as identifying compatibility issues (with solutions).

For additional resources, including the workshop slides and a detailed guide with relevant codes and FAQs, please refer to the SharePoint folder linked here.

I am a Lecturer in Economics at the Department of Economics, King’s Business School, King’s College London. I am also an academic supervisor at the Institute of Finance and Technology, University College of London, and a chief examiner for the University of London International Programme (Econometrics). Before joining King’s, I lectured and conducted research at the London School of Economics as an LSE Fellow in Economics, and at the University of Warwick as a postdoctoral fellow (in Economics). I completed my PhD in Economics at the University of Nottingham in 2018.

I have lectured courses in econometrics and macroeconomics at King’s, LSE, and Warwick, and led seminars (tutorials) in various courses at Nottingham. From March 2023, I am the GTA Lead at King’s Business School.

Evaluation and Reflection, News and Events, Pedagogy, Technologies

Exploring the Digital Frontier: Revolutionizing Feedback Delivery with Excel and VBA Macros Part 1

Aaron Sangha

Part 1: The practical guide

In today’s rapidly evolving digital age, the need for efficient and effective systems in education is more pronounced than ever. Traditional platforms like Moodle and Turnitin have served us well, but as educators, we must acknowledge their limitations in providing timely, user-friendly, and collaborative feedback on assignments, exams, and dissertations.

This tutorial aims to be your guiding light towards a better, more streamlined approach to feedback delivery. Drawing upon my workshop presented during The Festival of Technology 2022 at KCL, where I shared practical insights and implementation strategies for this automation system, we’ll delve into the fascinating world of Excel and VBA Macros within Microsoft Word. This comprehensive resource builds upon the principles discussed in that workshop.

By embarking on this journey, you’ll equip yourself with the skills and knowledge to revolutionize your approach to feedback giving. Here’s what you stand to gain:

1. **Efficiency:** Say goodbye to the laborious and time-consuming task of manually providing feedback using KEATs (Moodle/Turnitin). With Excel and VBA Macros, you’ll learn how to automate the process, saving valuable time that can be redirected towards more meaningful feedback and interactions with your students.

 

Screenshot illustrating what the purpose of the document
Picture 1: Our Aims and Objectives

2. **User-friendliness:** Discover how to create a user-friendly feedback documents for both yourself, the marking team, and your students. Your feedback system will become intuitive and accessible, ensuring that learners can easily understand and act upon your comments with a nicely formatted feedback document.

A screenshot showing the step by step summary for collecting marking and feedback
Picture 2: A summary of steps

3. **Collaboration:** Break free from the constraints of limited collaboration within traditional systems. The method will allow a marking team to efficiently collaborate and moderate, making feedback delivery a seamless and cooperative effort.

Screenshot of marking folder contents
Picture 3: What the marking folder looks like? It is sharable with the marking team

4. **Comprehensive Feedback:** Dive into the world of detailed and constructive feedback. You’ll gain the expertise to provide tailored insights that empower students to excel in their academic pursuits.

Screenshot of excel file showing comments
Picture 4: What a short comment looks like? Totally customizable.

This tutorial isn’t just about learning a new tool; it’s about transforming your approach to education. By mastering Excel and VBA Macros for feedback delivery, you’ll become a more effective educator, making a lasting impact on your students. The system will:
– Enhance your teaching methods, creating a more engaging and supportive learning environment.
– Free up your time that was spent on administrative tasks or dealing with Turnitin/Keats for more meaningful activities such as preparing feedback comments and communication with your team and students.
– For repeated courses/assessments, you can prepare a bank of modal comments for lateral uses, as well as a record of common mistakes and suggestions for improvements to communicate with students.

Screenshot of excel sheet
Picture 5: What the end product of a long feedback document looks like? Totally customizable.

Education is a dynamic field, and keeping pace with technological advancements is essential. The automation possibilities offered by Excel and VBA Macros are not just practical but also intriguing. Discovering how to harness these tools to optimize your feedback process can be genuinely exciting.

Screenshot displaying cautions with using excel
Picture 6: A few cautions?

For additional resources, including the workshop slides and a detailed guide with relevant codes and FAQs, please refer to the SharePoint folder linked here. This tutorial serves as a bridge between the insights shared during the workshop and the practical implementation of an automated feedback system. It’s an opportunity to further explore and master these valuable techniques, all while enhancing the overall learning experience for students. Join us as we embark on this transformative journey together.

Part 2 of this blog can be found here: https://blogs.kcl.ac.uk/digitaleducation/exploring-the-digital-frontier-revolutionizing-feedback-delivery-with-excel-and-vba-macros-part-2/

Moodle 4, News and Events, Technologies, Uncategorized

New CPD Sessions

Aaron Sangha

The Centre for Technology Enhanced Learning (CTEL) has a wide range of new development sessions available to all staff. These sessions range from how to record, edit and upload videos; creating interactive learning resources using Kaltura, our video and media service; as well as various workshops on using KEATS, our Virtual Learning Environment.

These sessions will be run by Microsoft Teams Meetings. Click the Skills Forge links below to find timings and book on available sessions. Please contact ctel@kcl.ac.uk for more information.

 

Introductory Session

KEATS

Content Creation and Video Editing

Delivering synchronous online teaching

Assessment

Internationalisation

Evaluation and Reflection, Technologies

Accepting Multiple Assignment Attempts

Riaz Toorabally

KEATS (Moodle) allows assignment submissions in many ways – this is a record of how a simple question became an extended investigation.

Academic Staff Requirements

Can I check what my students have previously uploaded?”

An academic colleague had used Blackboard (another Virtual Learning Environment) before coming to the Faculty of Natural, Mathematical & Engineering Sciences (NMES) at King’s. He asked if KEATS, our Moodle instance, could behave like Blackboard and allow students to submit multiple attempts to a programming assignment any time they want.

After a follow-up call with the academic colleague, it became clear that the aim was to be able to access anything students had uploaded prior to their final submission, as the latter might contain a wrong or broken file, and grade with reference to a previous submission (programme code or essay draft).

He had the following requirements:

  • Notification emails to both staff and students when a file is uploaded successfully
  • Students to be able to submit as often as they want
  • Marker to be able to review all uploaded attempts to:
    • be able to award marks if an earlier submitted programme code worked fine but a later submission introduced bugs breaking the programme, and
    • monitor the programming code development and make comments, compare changes, and to prevent collusion

This investigation looks at practical solutions to administering programming assignments as well as non-programming ones such as essays.

Background: Assessments on Blackboard (VLE)

In a Blackboard  assignment, students are required to click a Submit button for the markers to access their work. If multiple attempts are allowed, students can submit other attempts at any time, which will be stored as Attempt 1, Attempt 2 and so on and are available for the markers to view. This way, staff can review previous submissions, however, they cannot access drafts.

Screenshot of a Blackboard assignment allows multiple attempts
Blackboard assignment allows multiple attempts

KEATS: Moodle Assignment

The first step was to investigate the options and settings in Moodle Assignment, which is the tool that was already used by most colleagues for similar assignments.

With our current default settings, students can make changes to their uploaded files as much as they want, and submission is finalised only at the assignment deadline. Although instructors can see the latest uploaded files (draft)  even before the deadline, files removed/replaced by students will no longer be accessible to staff. This means, only one version is accessible to markers.

Multiple submissions can be enabled with the use of the “Require student to click the Submit button” setting for staff to review previous attempts, as on Blackboard. Feedback can be left on each attempt. However, students cannot freely submit new attempts because staff need to be involved to manually grant additional attempts to each student. Submissions are time-stamped and can be reviewed by students and markers, but students can only get notification emails after grading whereas markers can get notifications for submissions. Our problem was not resolved yet.

Screenshot of a marker accessing unsubmitted drafts and leaving feedback
Marker accessing unsubmitted drafts and leaving feedback
Screenshot of a student reviewing feedback for different attempts
Student reviewing feedback for different attempts.
Screenshot of a student reviewing feedback for different attempts (cropped version with highlight).
Student reviewing feedback for different attempts (cropped version with highlight).

KEATS: Moodle Quiz

We then considered Moodle Quiz, which some departments at King’s already use to collect scanned exam scripts: a Quiz containing an Essay-type question that allows file upload.

Screenshot of quiz attempt.

While exams usually only allow one single attempt, Moodle Quiz can be set to allow multiple attempts (Grade > Attempts allowed). The “Enforced delay between attempts” setting (from seconds to weeks) under “Extra restrictions on attempts” may be used to avoid spamming attempts. Student can submit new attempts as often as needed because no staff intervention is needed. The drawback is that there is no submission notification or emails, but the quiz summary screen should indicate to the student that the file is submitted. The Quiz attempts page for markers allows for easy review of previous attempts and feedback on each attempt. It is also possible to download all submissions as in Moodle Assignment. This was recommended to the academic colleague as an interim solution while we continued the investigation.

Possible Policy Concerns

Regarding unlimited re-submissions, Quality Assurance colleagues reminded us that students may challenge (i) a perceived inequality in opportunities to get feedback, or (ii) subconscious bias based on previous submissions. Good communication with students and a structured schedule or arrangements should improve expectations from both sides.

Turnitin Assignment and Other Assessment Options

Although the Moodle Quiz appeared to be a solution, we also considered other tools, some of which are readily integrated with KEATS at King’s:

Turnitin assignment allows multiple submissions as an option, but re-submissions will overwrite previously uploaded files. Alternatively, if it is set to a multi-part assignment, each part will be considered mandatory. However, the workflow for Turnitin assignment is not optimal for programming assignments.

Turnitin’s Gradescope offers Multi-Version Assignments for certain assignment types. It is available on KEATS for the Faculty of Natural, Mathematical, and Engineering Sciences (NMES). However, its programming assignment does not support assignment versioning yet.

Edit history is available for Moodle Wiki and OU Wiki; whereas Moodle Forum, Open Forum, Padlet and OU Blog allow continuous participation and interaction between students. These tools could be useful for group programming or other social collaborative learning projects, which is not a direct replacement for an individual programming assignment but an alternative mode of assessment.

Portfolios: Mahara has Timeline (version tracking) as an experimental feature. This may be suitable for essays but not for programming assignments.

Tracking Changes

Tracking changes is an important feature to show development in programming assignments or essays, and cloud platforms (OneDrive, Google Drive, GitHub) can host files and track changes. When used for assignments, student can submit a Share link to allow instructors to access and assess their work and how the work evolved over time. The disadvantage for this option is that the grading experience will be less integrated with Moodle. Some cloud platforms offer a File request feature where students can submit their files to a single location.

Programming Assignments

Industries such as software development use Git as a standard and all changes are tracked. GitHub offers GitHub Classroom, and it can be used with different VLEs including Moodle, but it is not readily integrated with KEATS and requires setup. There may be privacy concerns as students need to link their own accounts.

The Outcomes / Lessons learnt

  • This showcases how a simple question from academic colleague can lead to the exploration of various existing options and exploration of new tools and solutions.
  • Different options are available on KEATS with their pros and cons.
  • Existing tools, possible solutions, policies, and other considerations come into play.

Conclusion / Recommendations

KEATS Quiz matches the case requirements and was recommended to the academic colleague. It went smoothly and our colleague mentioned there were no complaints from students and they are happy with the recommended solution. It is relatively easy to setup and straightforward for students to submit. Clear step-by-step instructions  to staff and students should be enough, but trialling this with a formative assignment would also help.

Depending on the task or subject nature, other tools may work better for different kinds of tasks. TEL colleagues are always there to help!

Useful Links

Written by Antonio Cheung

Antonio is a Senior TEL Officer at the Faculty of Natural, Mathematical and Engineering Sciences (NMES).

September 2023

News and Events, Pedagogy, Technologies

Centre for Technology Enhanced Learning (CTEL) CPD Training Sessions

Aaron Sangha

The Centre for Technology Enhanced Learning (CTEL) has a wide range of new development sessions available to all staff. These sessions range from how to record, edit and upload videos; creating interactive learning resources using Kaltura, our video and media service; as well as various workshops on using KEATS, our Virtual Learning Environment.

These sessions will be run by Microsoft Teams Meetings. Click the Skills Forge links below to find timings and book on available sessions. Please contact ctel@kcl.ac.uk for more information.

Introductory Session

KEATS

Content Creation and Video Editing

Delivering synchronous online teaching

Assessment

Internationalisation

Moodle 4, Pedagogy, Technologies

Introducing the New KEATS Education Templates

Aaron Sangha

The KEATS Education Templates (KET) are a set of standardised templates that have been designed to improve the digital learning experience for students at King’s College London. The templates adhere to UXD (user experience design) good practice, and are designed to be performant, scalable, and in line with accessibility standards.

The KET were developed in consultation with King’s students and academic and professional services staff. They were, and continue to be, steered and shaped by the needs of their users.

Three formats for the template are available for use.

Collapsed Topics

King's KEATS Education Template 23/24: Collapsed Topics (for importing) showing the categories of; course, settings, Participants, Grades, Reports and More. The Course highlights different tabs such as; Timetable, Module Overview, Assessment and Week 1/Topic

Grid

Screenshot showing the Module Home page featuring a Timetable, Module Overview, Assessment and Week 1/Topic tabs.

Topics

Screenshot of King's KEATS Education Template 23/24: Topics (for importing) showing the categories of; course, settings, Participants, Grades, Reports and More. The Course highlights different tabs such as; Timetable, Module Overview, Assessment and Week 1/Topic

Why use the KEATS Education Templates?

The benefits of using the KEATS Education Templates, include: 

User Experience: The templates are designed to be easy to use and navigate, making it easier for students to find the information they need. 

Scalability: The templates are designed to be scalable, so they can be used for courses of all sizes. 

Consistency: The templates help to create a consistent look and feel across all KEATS course pages, which can help to improve the overall user experience. 

How to use the KEATS Education Templates

You will find guidance on implementing the templates on the KEATS Education Template (KET) Guidance page.

The guidance includes step-by-step instructions for adding the template to your KEATS page and a checklist of tasks required to edit the page after applying the template.

The first page of the guidance (Overview) includes links to the importable templates, as well as example courses with the templates applied.

Written by Fariha Choi 

Fariha Choi is a Learning Technologist at the Centre for Technology Enhanced Learning and has been with CTEL since June 2021. She has a particular interest in educational multimedia and has worked as a Learning Technologist, eLearning Developer and Learning and Development Manager for the past 11 years.