Evaluation and Reflection, Technologies

Facilitating Groups Assessments on KEATS

Introduction

Increasingly, modules are diversifying their assessment types and branching out into group assessments instead of essays and other more traditional assignments. As Professional Services, we want to support this in our schools and departments wherever possible, without it adding further administrative burden or creating scope for error. The goal was to facilitate students submitting assessments in groups whilst also enabling module leads and students to get the most out of other functions in KEATS which groups allow, such as group forums, restricting content by group, and creating marking groups for markers. In other words, we wanted a smooth experience for students submitting their assessments, whilst also enabling module leads and contributors to utilise the full extent of KEATS and maximise student engagement.

The Challenge

Previously, students would be a member of only one group within a module page to keep things “simpler”, which unfortunately restricted how students and staff could engage with KEATS. Alternatively, we have kept manual records of student groups and submissions which added additional administrative burden. Neither of these solutions were the ideal we were looking for, so we decided to turn our attention to the underused groupings feature on KEATS. We knew from experience that the settings in groupings must be set up exactly or students may be prevented from submitting or submit in the wrong group. Therefore, the challenge was to learn how to set up the settings on KEATS to interact with different activities successfully. Whilst it was an investment in time and resources at the beginning to work out how groupings operate, I was hopeful that it would streamline things going forward and create a better student and staff experience.

The process commenced with an initial consultation with the SSPP TEL team. I learned that the “groupings” functions as a group of groups and different activities on KEATS could be set up to pull groups from specific groupings. For example, there may be several presentation groups within a module which could all be included in the umbrella group presentation grouping. The Moodle assignment activity associated with this group presentation could then be set up to pull groups directly from the group presentation grouping, hereby avoiding the confusion of potentially pulling the groups from the seminar groups or forum groups instead. The team set me up with a sandbox on KEATS, effectively a practice area, where I could set up the journey that a student would take.

Screenshot of Groupings in KEATS
Screenshot of Groupings in KEATS

The three main things I wished to test were:

  • signing up to a group via the Group Choice Activity
  • being added to other groups in different groupings,
  • submitting as part of a group to see if all ran smoothly.

I replicated the settings on this practice page, following SSPP TEL guidance on groups and groupings, and I enlisted my very patient and enthusiastic Professional Service colleagues from ECS to be enrolled as students on the practice page, asking them to take the journey of the students. When colleagues submitted their mock assessments, the groups were pulled from the correct assessment groups grouping. As I had selected the settings that allowed one student to upload on behalf of all their group members on the Moodle assignment activity, when one group member submitted, the correct group members were also populated with their group’s submission. It appeared the testing showed everything to go to plan and there was no scrambling of submissions. I was in the clear to replicate these settings throughout the module pages where there were group assessments.

The Outcomes/ Lessons Learnt

Outside of the specific details of how to technically set up the various activities, I learned to think holistically about the KEATS pages and always consider how one setting or action might affect and interact with other activities or areas of the page. It’s also underpinned how crucial the initial set up of activities and course pages are, so going forward, I’d always advise plenty of time and, of course, meticulous checking of these settings in advance of busy times such as submission deadlines.

Conclusion/ Recommendations

In conclusion, I’d strongly advise my colleagues, not to be scared of groupings and groups! Yes, they require careful set up, but once up and running, they enable us to utilise a variety of activities and functions on KEATS, increasing interaction and, minimising administration in the long term whilst maximising engagement for students.

Written By Joanne Devenney

Senior Programme Officer
School of Education, Communication and Society

Moodle 4, News and Events, Technologies, Uncategorized

New CPD Sessions

The Centre for Technology Enhanced Learning (CTEL) has a wide range of new development sessions available to all staff. These sessions range from how to record, edit and upload videos; creating interactive learning resources using Kaltura, our video and media service; as well as various workshops on using KEATS, our Virtual Learning Environment.

These sessions will be run by Microsoft Teams Meetings. Click the Skills Forge links below to find timings and book on available sessions. Please contact ctel@kcl.ac.uk for more information.

 

Introductory Session

KEATS

Content Creation and Video Editing

Delivering synchronous online teaching

Assessment

Internationalisation

Evaluation and Reflection, Technologies

Accepting Multiple Assignment Attempts

KEATS (Moodle) allows assignment submissions in many ways – this is a record of how a simple question became an extended investigation.

Academic Staff Requirements

Can I check what my students have previously uploaded?”

An academic colleague had used Blackboard (another Virtual Learning Environment) before coming to the Faculty of Natural, Mathematical & Engineering Sciences (NMES) at King’s. He asked if KEATS, our Moodle instance, could behave like Blackboard and allow students to submit multiple attempts to a programming assignment any time they want.

After a follow-up call with the academic colleague, it became clear that the aim was to be able to access anything students had uploaded prior to their final submission, as the latter might contain a wrong or broken file, and grade with reference to a previous submission (programme code or essay draft).

He had the following requirements:

  • Notification emails to both staff and students when a file is uploaded successfully
  • Students to be able to submit as often as they want
  • Marker to be able to review all uploaded attempts to:
    • be able to award marks if an earlier submitted programme code worked fine but a later submission introduced bugs breaking the programme, and
    • monitor the programming code development and make comments, compare changes, and to prevent collusion

This investigation looks at practical solutions to administering programming assignments as well as non-programming ones such as essays.

Background: Assessments on Blackboard (VLE)

In a Blackboard  assignment, students are required to click a Submit button for the markers to access their work. If multiple attempts are allowed, students can submit other attempts at any time, which will be stored as Attempt 1, Attempt 2 and so on and are available for the markers to view. This way, staff can review previous submissions, however, they cannot access drafts.

Screenshot of a Blackboard assignment allows multiple attempts
Blackboard assignment allows multiple attempts

KEATS: Moodle Assignment

The first step was to investigate the options and settings in Moodle Assignment, which is the tool that was already used by most colleagues for similar assignments.

With our current default settings, students can make changes to their uploaded files as much as they want, and submission is finalised only at the assignment deadline. Although instructors can see the latest uploaded files (draft)  even before the deadline, files removed/replaced by students will no longer be accessible to staff. This means, only one version is accessible to markers.

Multiple submissions can be enabled with the use of the “Require student to click the Submit button” setting for staff to review previous attempts, as on Blackboard. Feedback can be left on each attempt. However, students cannot freely submit new attempts because staff need to be involved to manually grant additional attempts to each student. Submissions are time-stamped and can be reviewed by students and markers, but students can only get notification emails after grading whereas markers can get notifications for submissions. Our problem was not resolved yet.

Screenshot of a marker accessing unsubmitted drafts and leaving feedback
Marker accessing unsubmitted drafts and leaving feedback
Screenshot of a student reviewing feedback for different attempts
Student reviewing feedback for different attempts.
Screenshot of a student reviewing feedback for different attempts (cropped version with highlight).
Student reviewing feedback for different attempts (cropped version with highlight).

KEATS: Moodle Quiz

We then considered Moodle Quiz, which some departments at King’s already use to collect scanned exam scripts: a Quiz containing an Essay-type question that allows file upload.

Screenshot of quiz attempt.

While exams usually only allow one single attempt, Moodle Quiz can be set to allow multiple attempts (Grade > Attempts allowed). The “Enforced delay between attempts” setting (from seconds to weeks) under “Extra restrictions on attempts” may be used to avoid spamming attempts. Student can submit new attempts as often as needed because no staff intervention is needed. The drawback is that there is no submission notification or emails, but the quiz summary screen should indicate to the student that the file is submitted. The Quiz attempts page for markers allows for easy review of previous attempts and feedback on each attempt. It is also possible to download all submissions as in Moodle Assignment. This was recommended to the academic colleague as an interim solution while we continued the investigation.

Possible Policy Concerns

Regarding unlimited re-submissions, Quality Assurance colleagues reminded us that students may challenge (i) a perceived inequality in opportunities to get feedback, or (ii) subconscious bias based on previous submissions. Good communication with students and a structured schedule or arrangements should improve expectations from both sides.

Turnitin Assignment and Other Assessment Options

Although the Moodle Quiz appeared to be a solution, we also considered other tools, some of which are readily integrated with KEATS at King’s:

Turnitin assignment allows multiple submissions as an option, but re-submissions will overwrite previously uploaded files. Alternatively, if it is set to a multi-part assignment, each part will be considered mandatory. However, the workflow for Turnitin assignment is not optimal for programming assignments.

Turnitin’s Gradescope offers Multi-Version Assignments for certain assignment types. It is available on KEATS for the Faculty of Natural, Mathematical, and Engineering Sciences (NMES). However, its programming assignment does not support assignment versioning yet.

Edit history is available for Moodle Wiki and OU Wiki; whereas Moodle Forum, Open Forum, Padlet and OU Blog allow continuous participation and interaction between students. These tools could be useful for group programming or other social collaborative learning projects, which is not a direct replacement for an individual programming assignment but an alternative mode of assessment.

Portfolios: Mahara has Timeline (version tracking) as an experimental feature. This may be suitable for essays but not for programming assignments.

Tracking Changes

Tracking changes is an important feature to show development in programming assignments or essays, and cloud platforms (OneDrive, Google Drive, GitHub) can host files and track changes. When used for assignments, student can submit a Share link to allow instructors to access and assess their work and how the work evolved over time. The disadvantage for this option is that the grading experience will be less integrated with Moodle. Some cloud platforms offer a File request feature where students can submit their files to a single location.

Programming Assignments

Industries such as software development use Git as a standard and all changes are tracked. GitHub offers GitHub Classroom, and it can be used with different VLEs including Moodle, but it is not readily integrated with KEATS and requires setup. There may be privacy concerns as students need to link their own accounts.

The Outcomes / Lessons learnt

  • This showcases how a simple question from academic colleague can lead to the exploration of various existing options and exploration of new tools and solutions.
  • Different options are available on KEATS with their pros and cons.
  • Existing tools, possible solutions, policies, and other considerations come into play.

Conclusion / Recommendations

KEATS Quiz matches the case requirements and was recommended to the academic colleague. It went smoothly and our colleague mentioned there were no complaints from students and they are happy with the recommended solution. It is relatively easy to setup and straightforward for students to submit. Clear step-by-step instructions  to staff and students should be enough, but trialling this with a formative assignment would also help.

Depending on the task or subject nature, other tools may work better for different kinds of tasks. TEL colleagues are always there to help!


Useful Links


Written by Antonio Cheung

Antonio is a Senior TEL Officer at the Faculty of Natural, Mathematical and Engineering Sciences (NMES).

September 2023

Evaluation and Reflection

KEATS Similarity Checker Project

Overview of project

Between July 2022 and February 2023, the SSPP TEL team conducted a pilot project to improve the student experience when submitting assignments by creating a special area for students to check the plagiarism/similarity score of their assignments. The goal of the pilot was to make it easier for students with Mitigating Circumstances and the Programme Office staff to manage the process of submitting assignments to KEATS.

Any student who is not subject to Mitigating Circumstances can submit a draft and/or reupload their submission as many times as they wish up to the assessment’s original due date. Many students use this opportunity to submit a draft to check their similarity score before they make their final submission. At the moment, due to technical limitations within KEATS/ Turnitin, students who are granted an extension to an assessment via the Mitigating Circumstances process cannot submit a draft to check their similarity score; they are only allowed to submit once, and after the due date for the assignment passes they no longer have the option to upload their final version.

This is particularly problematic for students who have submitted a draft (sometimes long before the original due date) and then realise they need to apply for Mitigating Circumstances: as they are not able to delete the draft themselves, this draft will be considered their final submission and their MC claim may be rejected on the basis that they have already made a submission. In some departments, PS Staff sometimes agrees to submit and/or delete a draft for a student, but this is time consuming, not consistently applied, and it relies too much on PS Staff being available and inclined to help outside of their normal duties; it is also not sustainable when taking into account the very high number of MC claims we process at the moment.

First Steps

The departments of Geography and Global Health and Social Medicine in the Faculty of SSPP took part in the initial pilot project for their re-sit and dissertation students, and the Similarity Checker (SM) area was created and placed on their Handbook pages on KEATS. Accompanying it was a video and PDF to explain to students how to use the SM, as well as a warning text to reinforce the idea that this did not count as a submission and would not be checked by staff.

Feedback from this small cohort of students led to some revisions and changes to the SM, the most notable of which was around the language used. We had used the words “test area”, meaning to check or trial something, but students for whom English was not their native language found this confusing and equated “test” to mean exam. This was revised and the wording was changed from “test submission area” and “test area” to “Similarity Checker” and “practice area” respectively.

Once we were happy with the revisions, the SM was then rolled out to the rest of the School of Global Affairs, War Studies, and Education, Communication and Society. All Similarity Checker areas have the same layout, same wording and same instructions for parity across all the Schools. Communications for staff and students were also created by Soshana and these were used by Departments to make students and academic staff aware of the existence of the SM.

Layout

The Similarity Checker is made up of several parts. This includes an introductory text explaining what it would be used for, how to use it and a disclaimer that nothing submitted here would ever be moved nor assessed. An explainer video and PDF instructions were added to ensure accessibility and inclusive design were adhered to, so that all students would be able to clearly understand the functionality.

Screenshot of the home screen of the similarity checker.
Screenshot of the geography similarity checker.

The submission areas were divided by level and surname. There is no functional necessity for this, but it aims to prevent Turnitin from getting overloaded by all students in the one department trying to access it at the same time. If students submit in the wrong area there are no effects on their score or submission.

Screenshot of the different Turnitin Submissions.
Screenshot of the different Turnitin Submissions.

Student Feedback

A survey was created by Soshana and shared with all participating Schools, with almost 100 responses. Feedback was generally positive, with students highlighting how the SM improved their experience and confirming that it constitutes an equalising factor for students with extensions. Overall, 90% of respondents have used the SM, 93% found it useful, and 16% used it in the context of an assessment extension (mitigating circumstances).There was also some negative feedback from students who did not find it particularly beneficial, mainly due to the long turnaround time for their score after their third submission, as well as the fact that their score changed repeatedly when uploading a new draft of the same work, depending on how close the assessment due date was. These concerns will be addressed, and elements of response will be provided in future communications.

Overview of survey respondents.
Overview of survey respondents.
Respondents usage by level of study.
Respondents usage by level of study.
Respondents use of the Similarity Checker.
Respondents use of the Similarity Checker.

Conclusion and next steps

The pilot project was a successful start to improving the experience of students and staff using KEATS and Turnitin during their submission period. This was initially to improve the experience of those with Mitigating Circumstances, but we can see that many students without extensions are also using it to check their work.

Next steps will include rolling this out further to other Schools or Departments so that all students in SSPP can access it. Some Departments have their own versions, which we would like to replace with this more modern iteration of the Similarity Checker.

As next steps, the TEL team would like to address some of the points that the students raised as part of the feedback process, and create a communications plan to ensure this is being communicated to students at all relevant points of the academic year.

An all-Faculty stance should also be drawn up if/when a student submits their paper to the Similarity Checker instead of their module page and how this should be dealt with.


Written by Leanne Kelly Leanne Kelly

Leanne is the Digital Education Manager for the Faculty of Social Science and Public Policy (SSPP) at King’s College London. She is responsible for a wide range for digital education processes within the Faculty including instructional design, accessibility, training, innovation and developing new online programmes.

She has a background in publishing and eLearning, and is passionate about using technology to improve the learning experience and make it more accessible to all. She is interested in developing new ways of working, scaling projects and reusing content in new ways, and making online learning an enjoyable process for all.

Written by Soshana Fearn

Soshana Fearn

Soshana is the Senior Postgraduate Programme Officer for the Department of Geography (SSPP) at King’s College London. She delivers the day-to-day administration of taught postgraduate programmes (Masters), offers comprehensive and authoritative advice and support for all staff and students in respect of programme regulations and curriculum choices, services the relevant boards and committees, and oversees the processing of Mitigating Circumstances requests.

She has a background in project coordination and is dedicated to improve the experience of both students and staff through the development and implementation of streamlined innovative solutions, including projects related to institutional processes, policymaking and technology-enhanced learning resources.


 

Moodle 4, News and Events, Pedagogy, Technologies

New look KEATS coming in July 2023

KEATS will be upgraded to Moodle 4 on 18 July 2023 (see KEATS Upgrade to Moodle 4 Blog post for further information).

As part of this upgrade the look-and-feel of KEATS will be updated to bring it in line with emerging King’s Digital Brand Guidelines, improving consistency for staff and students across our digital platforms.

The new theme for KEATS will be primarily available for layouts which display modules and courses in a Topics or Grid formats. All course pages will be presented with a left-hand navigation and a right hand-panel for module/programme specific information as required, both of which can be expanded/collapsed as required.

KEATS Course Grid Format

KEATS new-look grid format
KEATS New Look Grid Format

KEATS Course Topics Format

GIF of a new look KEATS Course Topics Format
New Look KEATS Topics Format

 

Screenshots

Click on the screenshots below to view larger images of the new look-and- feel for KEATS coming in July 2023.

Pedagogy, Technologies

Applying the Digital Education Accessibility Baseline: An Academic’s perspective

This recording by Dr. Manasi Nandi, Reader in Integrative Pharmacology, complements the blog post, Digital Education Accessibility Baseline: Raising our standards for digital accessibility. It focuses on how Manasi approached the implementation of the baseline, provides tips on where to start and how to overcome some of the challenges she encountered. The benefits of accessible KEATS courses and content are explained, and resources to support accessible design are signposted.

Please click on the image below to access the recording:

Applying the Digital Education Accessibility Baseline: An Academic’s perspective recording
Applying the Digital Education Accessibility Baseline: An Academic’s perspective – recording duration 13mins

 

 

 

 

 

 

 

 

Useful links:

Evaluation and Reflection

Part 2: Embedding Digital Accessibility to the heart of everyday TEL work

This article has been divided in two parts. Part 1 discusses how the SSPP TEL team works to boost staff understanding of digital accessibility baseline. Part 2 provides an overview of the TEL training sessions available on the subject.


Part 2: TEL training sessions

The TEL Hub digital accessibility section is aimed at staff to access information in order to meet the basic requirements of digital accessibility baseline from a more administrative perspective. Continue reading “Part 2: Embedding Digital Accessibility to the heart of everyday TEL work”

Evaluation and Reflection

Part 1: Embedding Digital Accessibility to the heart of everyday TEL work

This article has been divided in two parts. Part 1 discusses how the SSPP TEL team works to boost staff understanding of digital accessibility baseline. Part 2 provides an overview of the TEL training sessions available on the subject.


Allyship:

“the state of being an ally (= a person who helps and supports somebody) to a particular group of people that you yourself do not belong to, in order to help ensure their basic rights and ability to be happy and successful in society”- Oxford learner’s dictionary

What does digital accessibility mean to me? It means being able to embrace a platform for its intended use, because the people building it know how to create a great user experience – anticipating, rather than reacting to the needs of our diverse learning community. Being an ally. Continue reading “Part 1: Embedding Digital Accessibility to the heart of everyday TEL work”

Echo Scheduler
Evaluation and Reflection

Part 2: Reflections on collaboration/cross-institutional working in response to and after Covid-19

This article has been divided in two parts. Part 1 explains the online teaching challenges created by Covid-19. Part 2 presents the outcomes and conclusions after 2 years  teaching online.


The volume of shared working, and working through a time of crisis, has helped to build collegiality, camaraderie and significant trust between teams. I believe this is evidenced by the devolution of central services to Faculty TEL teams (and not solely explained by capacity reasoning). For example: the provision of the Echo Scheduler role, Syllabus+ access being extended to TEL; and hopefully – successful pilot permitting – the Kaltura Management Console Super User role.   Continue reading “Part 2: Reflections on collaboration/cross-institutional working in response to and after Covid-19”