Instructors: Dr Andrew Melbourne (andrew.melbourne@kcl.ac.uk), Dr Mel Bovis, Bella Spencer, Alice Taylor-Gee.
Module: MSc/MRes Healthcare Technologies: Critical Skills for Translational Healthcare Technologies: Public Engagement Sub-Module (Biomedical Engineering & Imaging Sciences, Faculty of Life Sciences and Medicine).
Assessment activity: We introduced public engagement (PE) as a compulsory, assessed sub-module in the post-graduate higher education setting. This sub-module, or discrete block of lectures within a wider module, was developed in collaboration with our School Public Engagement Team and assessed via a reflective blog. The sub-module was also externally evaluated.
Why did you introduce the assessment?
Wider programme and module objectives allowed us to introduce a formal written summative assessment. The choice of this assessment was made by considering the wider portfolio of assessment in this module, which already included presentations and both creative and scientific writing. As a team, we also considered that the individual engagement activities were diverse in nature and contained both group and individual activities. We chose to adopt a written, reflective blog format to allow us to assess each individual’s experience of their public engagement activity and allow us to review their use of the training given in the lecture material. This also allowed us to assess the student with less bias linked to the quality or extent of the activity involved which can be influenced by the number of practitioners, the participants and other external factors (this turned out to be especially relevant in 2020).
How did you set it up?
During the academic year 2019/20 we introduced a compulsory PE focused sub-module (a discrete block of themed material within a module) for all 41 registered students into the module ‘Critical Skills in Healthcare Technologies’, including formal assessment devised in collaboration with the Public Engagement team (Alice Taylor-Gee, Melissa Bovis, Bella Spencer) at the Wellcome-EPSRC Centre for Medical Engineering. This was a pilot, both for the taught course (to include public engagement as an assessed critical skill) and for the PE team (to formally teach within a Masters course). The team developed an experiential learning activity of interactive lectures, followed by students embarking on a range of new and existing PE activities, providing opportunities for students to apply taught theory and their knowledge. Working individually or as a group, they were encouraged to develop, deliver and evaluate their project.
How did you design the assessment criteria and weighting?
The overall contribution of this work was 20% of the total module assessment and the module itself is worth 30/180 credits of the programme. Hence, this was a relatively low-stakes assessment for the students at both module and programme level, striking a balance between motivating participation but not overtly penalising students in the case of poor group cohesion or poor reflective writing.
Marks were awarded for the following using weighted criteria for increased transparency and ease of feedback:
- The presentation of the content (readability, neatness, organisation, good balance of conciseness & necessary detail, pictures) [10%]
- The content of the blog (summarising your work, understanding the aims of the exercises, reflecting on your work, [20% for each of these three sections]
- Evidence of integrating skills learnt from the training sessions [20%]
- Evidence of wider reading and any useful links relating to your public engagement project (twitter handles, websites etc.) [10%]
How do you give feedback?
The assessment in this case was summative, but this hides complexity because the key learning outcomes are obtained via the supported PE activity, all of which is formative assessment! Informal feedback was implicit throughout the sub-module because of the nature of the deliberate public-facing experiential activities involved.
Both the students and the PE team were highly motivated to succeed at their activities and it was important for the assessment not to interfere with this process – students were able to focus on the task and the experience rather than the potential grade, and diverse tasks could be assessed equally. Separating the formal assessment from activity related feedback was essential to allowing the PE activities to proceed without the burden of assessment, potential bias of performance, etc.
The external evaluation provided some recommendations for further support and feedback around the assessment, which are detailed below.
How do you explain the assessment to the students?
Students were made aware of the assessment from the start of the module. The PE team produced a short-written handbook detailing the purpose of the training sessions, the method of assessment and the weighting of the component parts of the final blog (see above).
We explained blog conventions, such as informal style and first person writing for reflection. However, we specified expectations of academic standards, e.g. quantitative metrics and qualitative quotes to evidence the impact of their PE activity, accompanied by photos and/or illustrations relevant to the public engagement activity
We also provided exemplars of blog writing on our own King’s Imaging Blog for students to look at.
What benefits did you see?
Confidence and skills building:
We evaluated the course using a survey and 100% of survey respondents agreed with this statement: Students left the sessions feeling confident about delivering a public engagement activity.
Some qualitative comments from students support their learning on the PE module.
“If I got the opportunity, I would definitely take it as I think it would help me tremendously boost my confidence in public speaking and engaging with people that are not within my circle or comfort zone”
“Having completed my PE activity, I feel much more confident to stand in front of any public and teach them and engage with them on scientific matters”
And below, one of our best quotes from a primary school participant indicates the benefits gained by those in the community through the PE:
What challenges did you encounter and how did you address them?
The substantive challenge of the 2019/20 cohort was the disruption due to Covid-19. Several events had been planned for National Science Week in March and had to be cancelled at the last moment without possibility to re-direct the activities. However, this challenge was mitigated by the nature of the reflective blog as an assessment strategy, which gave the students opportunity to focus on what they had learned already and to reflect upon the activity they would have carried out. As a result, it was possible to assess all students acceptably.
The external report provided some future recommendations: around the assessment and feedback
- Consider if there is an opportunity for public groups or partners to have a role in providing formative feedback to students. For instance, include a step in the assessment for the lay summaries to be reviewed by the Kings PPI group.
- Support and encourage the students to think reflectively, consider including reflection points for the students throughout the process to assess what is working and what could be improved.
- Increase opportunities for peer-to-peer support, for instance, invite the students from this year’s cohort to talk about their experiences to next year’s cohort
We look forward to running the module in 2020/21 with a larger class size and a focus on online events. The PE team’s new experiences in the online setting as part of their wider role will enhance our ability to offer diverse and wide-ranging events to the new cohort (online journal clubs, online science demonstrations, podcasts, science artwork) and we are excited to continue this learning journey.
What advice would you give to colleagues who are thinking of trying this type of assessment?
Take opportunities for teaching innovation when they come! – this teaching experience came about from my own growing interest in public engagement. One of my recent research projects had generated a lot of interest amongst my friends and family, and I started to realise that there was a lot of interest in what I was doing, if only I could explain it better. After taking part in some PE workshops and doing some smaller engagement events (with project patient groups and local schools), this culminated in an evening lecture at the Royal Institution, supported by my School PE team, and it was this that made me realise that PE could be adapted for teaching.
Leave a Reply