Two-stage exams for formative practice in Physics and Electrical Engineering

30% group

Jamie:

I run a lab skills module, the first few weeks of which involve the basics and essentials of lab safety and proper use of equipment. I wanted to make this a fundamental part of the assessment so that students would engage with it.

Students have to work in pars in formal labs and one of the learning outcomes of the module was to foster collaboration and group work, to help them see the safety and necessity of working together in the lab.

I tried it for the first time this year.  I got the idea from my experience with the summer school in one of their Community of practice talks. It appealed to me primarily as a means of giving students immediate feedback on what they had learned.

I wanted to make the two-stage exam summative to incentivise, but only 5 %. I also wanted to use the exam for assessing skills rather than knowledge.

I also tried it for the first time this year. Unlike Jamie, my use of the two stage exam was to assess knowledge of content in a conceptually challenging class. Problem classes take place as part of the classical physics module but traditionally a large part of the class don’t get involved, they wait passively for the answers to be given.

Like Jamie, I wanted to emphasise communication and collaboration. Although this isn’t a specific LO on the module, this module is about conceptual understanding and this is facilitated by peer learning.

I wanted to keep the two stage exam as a formative exercise to help students see the value of peer learning.

The exam is a series of MCQs around lab safety with 1 section n safety and 1 on equipment use.  It takes place in the lab itself around a piece of equipment.

First, students are given the individual part; I collect all scripts and then give out second part (which is a direct repeat with all the same questions). I didn’t change any of the questions, because I wanted them to get through the second part quickly.

Students get into groups for the second part but I pre-assign groups and this is randomised. When they enter the exam they go to their work station and their group is already located next to them.

There are usually no more than 30 students in a lab at one time but GTAs monitor to facilitate participation.

I assign less time for group part because I assume more agreement, but this hasn’t always been the case.

Grades are given through an average of individual and group marks – this is to reward the individual and also to incentivise group working, which is the main point of the activity.

I and the GTAs mark both parts and answers are posted with commentary on KEATS the week after.

This year the exam was paper-based.

In the problem classes, there is an average of 160 students in a 2 hour class.

MCQs are projected onto screen for them to work on individually for 25 mins. They discuss solutions in their group and are given 25 minutes. I assume they will not need more time for the discussion as many of the questions will be uncontroversial.

Students get into groups for the second part. Like Jamie, I have randomly pre-assigned beforehand, because I feel they benefit more from working with different people than only choosing their friends.

GTAs monitor to facilitate participation of quieter students. There are only 3 GTAs in a class of 160 however.

Grades are given in a split of 70% for the individual part and 30% for group. I feel this is enough to incentivise group work. However, this is a formative assessment, so the mark is a progress check.

Answers are given in class. Students mark their own work and can ask any questions to clarify in class. No real incentive to cheat as formative and progress testing.

Like Jamie’s model,  the exam was paper based.

During the 1^st week induction, I explain the assessment and explain the value of pair work, which preps them for group exam part.

I explain that it’s a means of getting immediate feedback in a module with a 1:150 staff student ratio. I try to allay any fears about peer feedback, particularly emphasising that the greatest benefits are usually in giving the feedback than in receiving. Because the exercise is formative, students are less anxious about group grades.

1.       As Furqaan says, students get immediate feedback- although their official grade isn’t given until the following week, from talking to their peers, they already know what their strengths and weak areas are.

2.      It reduced my workload because I don’t have to keep repeating issues of lab safety etc throughout the module.

3.      Students spend less time questioning their marks and asking for clarification because they have already learned from their peers.

1.       Students are definitely more engaged. GTAs report and I can see that students are actually doing the work unlike before.

2.      There is no extra workload as all done in class

3.      Last summer, I worked with student interns to create a bank of questions ALTHOUGH there were some issues- see below

4.     In the end of module survey and KEATS mid-term survey, students reported finding the two-stage exam useful. Unfortunately I don’t have any quantitative data for this year but am monitoring how it improves attainment.

There is always some social loafing and those who let group mark come out better so don’t work as hard for the individual part.

Logistics- multiple sessions (90 students in 3 labs of 30) mean that to avoid cheating and swapping answers a big question bank is needed.  This would not happen in larger classes like Furqaan’s where all students can do the exam at the same time.

There is some extra time in marking involved but MCQs are not particularly and some double marking but MCQS so not time consuming. The exam will be done on KEATS not paper next time.   (They will use devices to access KEATS in the lab?)

When students have PAAs, we have to consider how to adjust for extra time in the group section. Usually this can be done by rotating through groups.

The assessment takes place in week three, which is close to the start of term. Not a lot of KIPs are in place yet and students often find it hard to come forward if they have particularly trouble speaking in a group. There is the option of having those types of students take the module online.

Furqaan:

Large class sizes make a two- stage exam more difficult to manage as it can hard to monitor with a limited number of GTAs, and it can be chaotic when they get into groups. But in the second incarnation, as in Jamie’s model, I had them sit in their pre-assigned groups when entering the classroom. This is the way a typical exam is run anyway.

There was still not 100% engagement from all students- some were sceptical of how much time it takes to do questions twice. I think part of the issue of lack of engagement was that some questions were uncontroversial as they were too concrete. This type of activity suits more conceptual and application questions so next summer I will tweak the questions to make them more amenable for discussion.

As above, some students are more reticent and might need more GTA and class support to become accustomed to this sort of culture.

I will transfer the MCQs to KEATS rather than paper based as this will reduce marking time. KEATS then does the automatic randomisation of questions.

Next year, I intend to trying a two-stage exam for knowledge and reflecting on what has been learned. There is a large % of assessment at the end of semester 2, so students need opportunities to reflect on their learning beforehand. I hope that doing a low stakes two stage exam will help to draw attention to what students don’t know and what they need to revise.

I will repeat the two stage exam in the problem classes again with better questions.

I wouldn’t at this stage consider making it high -stakes summative as there might be too much push back on grades. Students would want to pick their own groups, and then they would not gain benefits of working with others, biases would be reinforced and they need to learn to have different roles.

Ensure that you set up the room so the change over from individual to group is not too chaotic.

Consider mixing groups for abilities etc. Don’t tell groups in advance as this causes requests for swaps and more anxiety about who they are working with.

I feel although I could make it summative, for me this defeats my purpose in helping quite reticent students to come out of their shell and see the benefit of peer learning. Motivating engagement through marks could defeat the object of this.