Instructor: Leslie-Anne Duvic-Paoli.
Module: Global Law of Climate Change (around 15 students, Level 7, optional module, Master of Laws LLM).
Assessment activity: students are asked to complement their 4000-word research essay with a digital artefact which explains the concepts and arguments presented in that essay to a general audience. Students are given the freedom to select from a range of genres and media including short videos or podcasts, posters, blog posts, Wikipedia articles, drawings or songs.
Why did you introduce this approach to assessment?
I was inspired by a King’s Academy workshop on diversifying assessment methods. I was particularly keen to include a more outward-facing component to assessment on my Climate Law module: in recent years, public awareness of climate change and environmental protection has increased dramatically, and I believe that my students have a role to play in trying to solve this crisis by sharing their knowledge with the wider public. An understanding of law is particularly helpful to contribute to this societal debate: legal concepts are very much present in the rhetoric of climate activists, who, for instance, call for the recognition of the crime of ‘ecocide’ or criticise the incompatibility of Free Trade Agreements with climate objectives.
I therefore decided to tweak the format of the main assignment for the course – a research essay – by adding a new component – a digital artefact. I wanted students to be able to translate their arguments and findings into communications which could be understood by a public without specific knowledge of climate law.
I see several advantages to this approach:
- Authenticity: It invites students to engage with different ways of doing, and using, law, that is not restricted to the doctrinal approach to which many have been introduced during their undergraduate legal studies.
- Creativity: By giving students the possibility to choose the format of their artefact, the task invites them to express themselves creatively and to develop their own voice.
- Employability: Through improving students’ digital and communication skills, I also expect to promote students’ employability; I envisage that the artefacts could potentially be shown to prospective employers and discussed during interviews.
Overall, I hope that this assessment increases the significance of the work that students do for the module by giving them the opportunity to engage a public beyond the marker and contribute to important discussions about one of the most pressing issues of our time.
How did you design the assessment criteria and weighting?
My objective was to test a new type of assessment at a small scale without having to make significant changes to the module. I therefore kept the same final assessment – a research essay – but added the public dissemination artefact, weighted at 20%.
The research essay has more weight (80%) because it aligns with more of the learning outcomes for the module. It exercises and assesses students’ skills in identifying and interpreting texts, their ability to synthesise these into accurate knowledge, their criticality and ability to construct a rigorous argument.
The artefact primarily assesses communication; students are evaluated on the clarity of their message and the originality of their piece. We discussed these criteria in class, where I explained to students that they would be assessed on how well they were able to explain technical concepts to a non-specialist audience, how innovative and pertinent their choice of output is, and how powerful or attention-grabbing their message would be to a general public.
How did you give opportunities for formative practice?
I think it is crucial to start explaining the assessment early in the semester so students can start thinking about their artefact and how they are going to produce it. At an early stage, students were asked to submit a draft proposal explaining which directions their paper would take and including preliminary ideas for the artefact alongside any difficulties they could foresee. It gives me a chance to provide guidance and respond with support.
At the end of the semester, students give a presentation of their work-in-progress to the class, in which they explain the content of their essay and share initial ideas about their artefact. I trust that this enables students to become deep learners: they produce quality work that they have elaborated over time, discussed with instructors and peers, and improved by actioning feedback.
How did you introduce the assessment to students?
A description and instructions are posted on KEATS which include the objectives and criteria. In week 4, once students have finalised their course choices, I dedicate time in class to explain the assessment and introduce students to the marking criteria. With regards to their essay, I use exemplars to explicate the characteristics of good quality work. I form discussion groups that bring together students with experiences of different higher education systems and ask them to discuss the strengths and weaknesses of each paper and suggest a grade. With consent, we will have examples of the digital artefacts from this year to share with next year’s students and discuss in class.
Students were reassured that they did not need to buy specific software and that their technical abilities would not be assessed. However, I signposted them to links within KEATS where they could find technical guidance and training. Our faculty learning technologist was available to answer technical questions that might arise and supported students to submit files which were sometimes quite large.
How was the experience of marking this work?
I really enjoyed it! I was impressed by how creative our students were. There were some brilliant pieces. One student reinterpreted a song with new lyrics. It was duet which allowed students to compare the legal design of two climate treaties, the Kyoto Protocol and the Paris Agreement. This is shown below. The chorus captured the essence of the differences in exactly the kind of original and memorable way I had hoped. It was also fascinating to see the diversity of formats that students chose to express themselves.
Common but Differentiated Responsibilities: Who Does it Better?
By Sara Pirri (consent given for the use of these lyrics)
Kyoto Protocol (KP)
I’m the Kyoto Protocol and now I’ll show you what I can do!
I’m the first binding text to introduce differentiation,
North and South differentiation.
Paris Agreement (PA)
Well, I’m the Paris Agreement and I came just after you,
Remedy your mistakes being what I’m trying to do,
Being what I’m trying to do.
KP
I was adopted in a COP,
for a balance to be formed,
for some countries to keep growing
and to stop climate change or at least make it slower,
and in order to do so
the countries who commit
are the ones who developed,
while the other just watch.
I think that I, that I should be the winner, the winner!
Equality is what makes me the winner, the winner!
You say the climate is a common concern,
Yeah, but countries need different treatments.
I think that I, that I should be the winner, the winner!
PA
First thing first
You seem to care just about equality
But don’t forget any agreement also needs efficacy,
Also needs efficacy.
So how can you pretend
To stop the climate change
If the commitments you provide
Leave some countries free to rise
Their emissions, no binding restrictions, without definition
Unrealistic distinction that I do break down,
Break down like…
With a blurred differentiation I’m the winner, the winner!
And, self-differentiation I’m the winner, the winner!
‘cause I balance equity and efficacy,
And that’s the key for making it complete.
I think that I, that I should be the winner, the winner!
KP
You said no to my dichotomy and yes to self-differentiation
As the means for the future
And the agreement to be smoother.
But how you think is going to work?
PA
National contributions are my solution
To respect the differentiation,
While raising ambition.
With a blurred differentiation I’m the winner, the winner!
And, self-differentiation I’m the winner, the winner!
‘cause I balance equity and efficacy,
And that’s the key for making it complete.
I think that I, that I should be the winner, the winner!
Artefacts included videos, poems, posters and twitter threads to explain legal concepts to a general public, as well as legal briefs to law firm clients and victims of climate impacts. I noticed that there was not necessarily a link between the quality of the essay and that of the digital artefact. I believe this shows that this two-part assessment gives students the possibility to work on their different strengths and gives me the opportunity to evaluate their various skills.
How did you give feedback?
Feedback is mostly embedded in formative practice linked to the process of creating the assessment – throughout the semester, I seek to create an on-going dialogue about learning, where feedback is not only received but is discussed and acted upon.
Feedback on the summative assessment is more limited because it comes at the end of the module. Directly after that, students disperse to other optional modules (and in fact some Study Abroad students actually return to their home countries). Students submitted through Moodle Assignment on KEATS; I gave written feedback on there too, to which they are given access during the course of the second semester, with the view that they can learn from the feedback to improve their performance in other modules.
Were there any challenges, and how did you address them?
Technology: students make a dual submission of a Word-processed essay and a file which is sometimes large. As an assessor I need these to be in the same place so I can easily cross-reference and mark them together. Our School Learning Technologist identified Moodle Assignment in KEATS as the best submission tool: it incorporates a Turnitin similarity check on the essay and allows large files to be submitted. He supported students to reduce the file size and made himself available to answer students’ technology-related questions. This of course raises the question of whether such an assessment would be scalable to a larger module, as it might increase the workload of our School learning technologists if too many students seek their help, but so far students have shown to be very tech-savvy and have needed little technological guidance.
Marking: the process has been fairly straightforward because class size is small and I am therefore the only first marker. It could have been challenging to mark outputs that are so varied in style and format; however, the marking criteria are flexible enough to respond to such diversity and having discussed them with the students in class also helped me as an assessor. Marking can be a bit more time consuming compared to a one-part assessment because the addition of the digital artefact essentially means grading, and providing feedback on, two different pieces; but marking such creative pieces is much more enjoyable!
What benefits did you see?
I found that students engaged with the task very well. The module is given in the first semester of a one-year postgraduate programme, which means that most students who take my module are new to UK higher education. They often are not familiar with essay-based assessments, having been evaluated only the basis of in-person exams during their undergraduate studies. As a result, they are often anxious about what is to them a new type of assessment. However, I was pleased to see that they appear confident about producing a digital output, they gave it a lot of thought. Given the freedom to express their creativity, students felt that their ideas and skills were valued. Some told me they were pleased to use their extra-curricular skills (singing, drawing…) in the context of their studies, while others shared that they had presented their digital output to family members and friends. I trust that the authenticity of the task motivated students to produce high quality work.
What advice would you give to colleagues who are considering trying this?
- Encourage students to think about the unique ways their chosen medium allows them to communicate. For example, video is suited to imagery, but for a static recording of a scripted talk has little to offer beyond what a Word-processed document would afford. Next year, I will ask students to write a very short introduction to their artefact to justify their choice of format and style: this year, some decided to explain who the artefact was designed for (eg. a citizen, a law firm client, a victim of human rights violations…) and I would like to make this a consistent requirement as it provides an important clarification about the student’s choice of artefact.
- Giving formative feedback on the different aspects of the assessment is also fundamental as it improves the chance that the information in the artefact is accurate and encourages students to gain in confidence and be more creative about their work.
- Encourage students to start working early on their work. This is particularly important because because students first have to work on their essay and then need to have enough time to distill its core argument into the digital artefact. In the case of this module, it is even more essential because the assessment needs to be submitted on the last day of the first semester, at the very start of January: this means that students are likely to finish working on their assignment during the Christmas break when academic and technology enhanced learning staff are on leave and hence unable to respond to last minute queries. We reminded students not to leave their uploads to the last minute, and everyone who observed this advice was able to successfully submit.
- Remind students to credit any authors whose work (images, music, etc) they use in their own work.
- Consider whether and how to help students publish their work if they want to. This entails helping students identify openly licensed content such as sound, music and images that can be shared outside King’s. Students will probably want to postpone giving consent to publish after they have received their marks and feedback – and they may first want to improve their work in response to the feedback. We are also considering where to host such work. Plenty to consider and comments are welcome!
Leave a Reply