Instructor: Dr Chiamaka Nwosu
Email: chiamaka.2.nwosu@kcl.ac.uk
Module: Research methods for MSc Public Policy & Management, King’s Business School
Why did you introduce this approach to assessment?
The main summative assessment is a 2,500-word research design task which accounts for 75% of total module grade. The module begins with 2 weeks of introductory sessions, followed by 4 weeks of quantitative research methods, and another 4 weeks of qualitative research methods. The quantitative aspect of the module covers a range of statistical tests ranging from t-tests for assessing differences in means, to multiple regression models estimating the relationship between an outcome variable and several explanatory variables. By the end of the module, students should be able to develop a quantitative research design to address a research question relevant to public policy and management. Students are also required to become familiar with STATA, a statistical programming software used to conduct these analyses and should be able to interpret the results from these tests accurately.
To equip students with these tools, this approach introduces a 4-week long authentic formative group assessment. Prior to introducing this assessment, in previous terms, students were introduced to a new dataset each week, which they used to conduct the analyses individually. The lack of continuity with this approach constituted a challenge as students were spending a lot of time trying to familiarise themselves with the new datasets before proceeding with the analysis.
With this new approach, each student joined a group of 5-7 other students, and they were asked to develop a research question which is both of interest to them and relevant to public policy and management. Based on the selected research question, they would then prepare a survey which would be distributed to their peers in the workshop via Microsoft Forms. This would form the unique dataset that each group would analyse using various tests each week.
How did you set it up?
The mini quantitative research project was set up as an ungraded formative assessment carried out over 4 weeks. Using the database tool available in KEATS, each group could create their own entry, input the names of their group members, and specify the research question that they would be addressing. Each week, following the introduction of a new method of analysis, students would then conduct their analysis using their own primary data, and complete the required section of the mini quantitative research project with the results of their analysis and subsequent interpretation of these results. In the final week, each group would deliver a presentation, presenting the findings from their project to the rest of the class.
How do you give feedback?
Students are provided verbal feedback during the workshops each week. Written feedback is provided via the database tool in KEATS which includes an option to provide comments under each submission. After the workshop each week, I would then review all group submissions and provide constructive feedback, highlighting what they did well and what they could improve on, particularly regarding the interpretation of the analysis.
What benefits did you see?
- The low-risk nature of the assessment fostered a positive learning environment during the workshops, motivating students to be creative with their choice of research question without any risk of failure.
- Another benefit is that students were able to collaborate in groups, which encouraged active participation in their learning and allowed them to learn from each other in a low stakes environment.
- Skills-building: In an anonymous evaluation survey administered to students in the final week of the quantitative research methods sessions, close to 70% of students who completed the anonymous evaluation survey agreed that designing their own surveys and analysing their own data helped improve their research and data analysis skills with approximately 15% neither agreeing nor disagreeing. For the question on whether students felt that the mini quantitative research project was useful for developing their quantitative research skills, 60% of students agreed with this while 20% neither agreed nor disagreed.
When asked what they liked most about the module, several students highlighted the mini quantitative research project. A few direct quotes from students below:
“I enjoyed the practical element of making our own projects which allowed me to use what I was learning from the lectures into practical use.”
“Working with primary data that we collected”
“Providing a chance to create our own research project”
“I enjoyed the collaborative mini group project”
Overall, it was great to see students really engaging with this assessment. It was clear that they felt a strong sense of ownership of these projects as they were analysing a topic they were particularly interested in, using data from a survey they designed and administered themselves.
What challenges did you encounter and how did you address them?
- Engagement: There were some students who showed limited engagement with the mini quantitative project because it was ungraded and therefore low risk, preferring instead to focus on the assessment that was worth a large proportion of their final grade. This created some challenges for other members of the group who were left to drive the project on their own. To tackle this challenge, students were required to deliver a presentation in their final week. I emphasised that every member of the group should contribute to this presentation which motivated students to ramp up their participation in the group work.
- Group-work: Students were allowed to self-select into their own groups which was largely beneficial as it meant that those with similar research interests could work together. However, I found that there were some groups formed entirely of friends which meant that there were sometimes issues with concentration during the workshops. To mitigate this, I did my best to redirect students towards the project by frequently stopping by each group to check on their progress.
- Class sizes: The size of the class (more than 60 students) also sometimes hindered my ability to spend a considerable amount of time with each group during the workshop, particularly if one group needed a lot of support. Therefore, I also encouraged students to visit my office hours individually or in groups if they needed additional guidance on any aspect of the curriculum.
- Tech support: Students also needed some guidance in developing their proficiency in STATA, for instance in learning the syntax i.e., what commands to use, and how and when to use them, as well as help with interpreting the results from their analysis. To guide students, I also carried out my own mini quantitative project and created my own entry in the database tool to display the commands I used, the results from my analysis and my interpretation of these results so that students were able to consult this when conducting their own analyses. I also dedicated a considerable amount of time in each workshop going through the STATA syntax using example data, while emphasising the interpretation of the results. I also created a table of useful STATA commands featuring the various commands, their purpose, and examples of usage so that students could refer to it as needed.
What advice would you give to colleagues who are thinking of trying this type of assessment?
My advice to colleagues who are teaching similar modules and are keen to try out this type of formative assessment would be to dedicate some more time to provide guidance to students when selecting their research question and designing their surveys as there were a couple of instances where an error was made in the process of designing the survey e.g., not constructing a question clearly such that it was misunderstood by a large proportion of respondents, or using restricted multiple choice options when open responses would be better.
It might also be beneficial to randomly assign students to groups, rather than allow them to choose their own groups which will improve concentration during group work, and encourage students to take the assessment more seriously.
Leave a Reply