Instructor: Dr Pablo de Orellana
Module: Contemporary Security Issues, Level 4, core War Studies and IR module, SSPP
Assessment : In small groups, students produce a one page written policy brief on a topic or issue of their choice covered during the module and in seminars each week role play being ‘analysts’ presenting their brief to a ‘minister’ played by the seminar leader (GTA or Teaching Fellow).
Why did you introduce the assessment?
I was inspired to design and implement this assessment from my own experience giving policy briefs to Parliament. Being able to present something concisely and in a convincing manner is an important skill for my students.
It also relates directly to the learning outcomes and employability outcomes of the module.
I have a contact at HM Treasury who is in charge of graduate recruitment, who was happy to take part in the assessment. She gives a briefing to students in a lecture at the beginning of the module on how to write policy briefs for policy-makers such as a minister. This inspires the students as they understand the real world nature and specific challenges of the task. In the same lecture, Fiona Richardson from Careers and Employablity gives a short talk on how the module and the assessment relates to core skills that relate to careers, post doc and internships that our students commonly go onto.
Can you tell us how the assessment works?
This assessment takes place on a compulsory first year module for all War Studies and International Relations related degrees. There are about 160 students as a minimum on this course.
Students get into groups of 3 or 4 and sign up to write their brief in the first week if the module. They present their policy brief based on a list of weekly topics throughout the module. Each group has to sign up for one topic but if there are spare topics, a group can sign up for an additional task, from which they receive the higher grade of their two presentations/briefs. I feel this rewards student for taking on more challenging tasks.
The students produce a one-page brief on one side of A4 (simulating the real genre of civil service policy briefs for UK ministers). In the seminars, led by teaching fellows or GTAs, students play the role of civil servants or analysts while the GTA/teaching fellow plays the role of minister. The students are encouraged to pick a very specific security concern (a “dirty bomb” made with atomic material by a terrorist group, for example) that fits within the given topic of the week (every week covers a security sector, nuclear proliferation in the above-mentioned example of a dirty bomb) that they feel is urgent and for which they are required to offer specific policy-feasible solutions. The rest of the discussion in the seminar emerges from the presentation/role play. In particular, GTAs are encouraged to take the proposal of the policy brief and ask the class (who here plays the analysists of the MoD for example) for feedback, opinions and alternatives.
There are only three core readings for each topic; therefore, groups are expected to do much of their own reading and media searches in order to present a convincing case based on data.
How did you design the assessment criteria and weighting?
The assessment is worth 30% of the module. It replaced the original traditional presentation with 10% of the grade. Increasing the weighting of the assessment was more motivating to students because it reflected the amount of work they would put into the assessment. The remaining 70% is an end of module examination.
The criteria and marking scheme is based on the original presentation as the skills are very similar. I added the extra category of approachability, concision or “crunching” i.e. the skill of condensing complex data and policy into readable convincing arguments. In order to gain a first, the recommendations that students make in their briefs must be FEASIBLE within the limitations of existing policy levers (levers are a key consideration in all Public Policy, they are the tools that a government actually has at its disposal at any one time).
Students are given a group grade which is divided between their written brief and presentation, but more highly weighted towards the written brief.
How did you give feedback and provide formative practice?
Groups are given oral feedback in class after their role play. The briefs are turned into pdfs and uploaded to Turnitin where written feedback is given through Feedback Studio.
Feedback is given on the three main criteria points: approachability, research and feasibility. Giving public feedback helps all students learn from each other and motivates the groups as they know they will be getting public feedback.
How did you explain this to students?
I explain the rationale and the value of this type of assessment. By having the speaker from HM Treasury attend the first class to contextualise the assessment gives an added motivation and rationale. It also provides a contact and link to recruitment and highlights employability in a way that traditional exams and essays do not because students are more able to explain the value of their assessment in terms of the skills and knowledge gained.
What benefits did you see?
Authenticity: as above, the requirement that the recommendations must be feasible emphasises the real-world nature of this type of task. Right from level 4, it builds the skills of presenting information coherently, convincingly and concisely. Because students have an audience (even though simulated by their own peers), they have an opportunity to practice a type of speaking and writing in a way that coursework essay or standard academic presentation does not allow.
Personalisation: students are encouraged to be original and focus on aspects of topics which interest them. This can spark interest in the topic which could carry them onto future module and even careers.
Testimonial from Emily Brown, a GTA in this module from 2017:
“The policy brief component of Contemporary Security Issues was initially received with some trepidation by the students because this was unlike anything they had previously been asked to do. However, once we had held the policy brief workshop and they had come to grips with what was required of them the students seemed eager to put what they were learning into practice. They would spend time considering what the significant security issues (and the accompanying societal implications) where in relation to the week’s theme, and then provide thoughtful policy solutions through in-depth presentations during the seminar. The presentations would often set the tone for the rest of the seminar, and the Q and A session afterwards often made for fruitful discussion. The students really enjoyed getting into character, becoming ‘experts’ on one (or two) key issues, and putting the theory into practice in a real and significant way. They could also think more actively and concretely about where their degree might lead them in their future careers.”
What challenges did you encounter and how did you address them?
Marking: the GTA has to be both assessor and interlocutor/actor in the role play, which represents a challenge for them. The module leader should try where possible to choose more experienced GTAs. In order to mitigate against disparity in marking, I hold standardisation sessions in weeks 1 and 2 in order for GTAs to feel more confident in marking and to facilitate but not to take over the discussion in their role as minister but to allow students to have the floor for the majority of the time.
Controversial topics: all views and perspectives are given a voice in this type of platform, which can lead to debate over some controversial opinions. The idea is to allow these debates to flourish, for students to challenge and be challenged. Although usually, one side does not necessarily convince the other, all feel their voices have been heard. Some training or warnings prior about respect for diversity of opinion should be given to students at the beginning of the module. However, the assessments is grade on the ability to convince, the quality and depth of research and its feasibility, rather than the particular ideology expressed.
Inclusivity: some students have social anxiety and many first year students have not yet acquired the skills of presenting confidently in this manner. However, the aims of the assessment are to build this confidence. The weighting of the paper is higher to mitigate against this issue.
What advice would you give to colleagues who are thinking of trying this type of assessment?
Students should be scaffolded and given support for this type of assessment. In particular, training TAs for the expectations and specific needs of this kind of assessment, and sharing with them a number of policy briefs produced by institutions such as POST (UK parliament) and the various Departments.
I feel this type of assessment could also be used in any module for political Economy, Political Science, International Development, and even Medicine and Business.
Click here to see examples of two briefs produces by students:
In this video, Pablo introduces the assessment along with Emily Taylor from HM Treasury, who talks about the key features of a policy brief, and Fiona from C&E who looks at how the KCL KASE framework applies to the module.
A video transcript is available if needed.
Leave a Reply