Instructor: Dr Jack Fosten
Module: Intermediate Econometrics, Economics, Kings Business School, Level 5
Assessment activity: This module uses flipped maths questions with video solutions. This is to assist students in developing mathematical arguments individually in their own time. The videos are followed by formative KEATS quizzes which check and reinforce knowledge
Why did you introduce the assessment?
This is a formative type of assessment used specifically on this module. The module topics include the use of statistical (econometric) models for economic forecasting and modelling economic time series.
Maths is often seen as a necessary evil which many students struggle with and should be practiced alone in order to achieve mastery. However, from personal experience, the use of tutorial time to go through maths problems is a) quite boring! b) mistake-prone, both in terms of the student copying down maths AND the teacher making mistakes on the whiteboard, and c) seen as pointless by students because written solutions are always made available which can be looked through at home.
These thoughts have also been expanded upon in the recent paper by Becker and Proud (2018) who note that in this format, “contact is used as delivery mechanism for worked solutions of problems… tutorials fail to deepen the students’ understanding of the material nor do they develop students’ generic skills through active participation.”
It was also felt that using computer labs for the tutorials would be a more effective use of the teaching time on the module. The combination of computer tutorials with flipped maths questions work together to achieve the module learning outcomes which include developing skills in data analysis, problem-solving and a hands-on approach to econometric modelling.
By using flipped maths questions with video solutions, students can develop mathematical arguments by themselves in their own time. Students are encouraged to become independent problem solvers and obtain instant feedback on their performance prior to any summative assessment.
There are other examples of this type of video content being used in economics and econometrics:
How did you set it up?
Students receive a problem set on Friday mornings after the lecture, which includes computer questions (which will be tackled in the tutorial lab session) and maths problems (which will not specifically be tackled in the tutorial, but students will be given video solutions and can then ask questions and obtain verbal feedback in tutorials or office hours).
After receiving the problem set, the students take the weekend and Monday to attempt the maths problems before the video is released on Monday at 5pm. The tutorial takes place a few days later on Thursday. The videos are made using Camtasia to record the screen (Camtasia is available in the KCL Software Centre but may require a user account to be requested from 8888@kcl.ac.uk). For the writing of maths equations in a “blackboard” style, Microsoft Paint is used with a Wacom Intuos Pen Tablet. Here is a screenshot of one of the videos:
Videos are usually between 4 to 7 minutes and one or two are released per week, depending on the questions. It is commonly accepted that video content must be kept relatively short to maximise student engagement in a flipped classroom approach (Zainuddin and Perera, 2019).
Below the video(s) on the KEATS page, there is a KEATS quiz related to the video. Questions are usually an extension or manipulation of something seen in the video and are designed to test higher-order thinking.
How do you give feedback?
The purpose of the formative assessment quizzes are for students to gain immediate feedback independently of the teacher and analyse their own performance. The feedback for incorrect answers is designed to guide students on how they should have approached the problem. Feedback is also given verbally and through peers by discussing the problems in class in the following tutorial. The feedback on an incorrect quiz answer looks like this:
How did you explain this to students?
In the first lecture I explain the structure of the module to the students, including the video and formative quizzes. They are made aware that we will only go through computer questions in tutorial, but that they can ask questions on anything, both in the tutorials and in my office hours.
Occasionally, reminder announcements are sent through KEATS just after the video upload, encouraging students to engage with them.
What benefits did you see?
On student learning:
- Students have expressed that they really like the videos, and that learning through videos is a common way of student learning in the 21st century.
- The videos and quizzes will be a useful revision resource for students which they can pause, rewind and do at their own pace.
- I also feel that this is an inclusive practice, especially with a largely international cohort.
Informal (verbal) feedback in office hours, lectures and tutorials has been very positive. In addition to this, I undertook the Stop Start Continue exercise before reading week. Without prompting, 29/51 responses gave positive comments about the videos:
- “Video solutions are easier to understand than written solutions, so the combination of all of these really makes our understanding easier.”
- “It is also useful so we can go back, re-watch the videos and fully understand the concepts.”
- “Please also continue the videos on keats, they are a big help and very much in line with the way that students tend to learn nowadays (youtube clips etc).”
In terms of teaching and workload:
- Since students practice the maths derivations using the videos, this frees up time to talk about more interesting things and deflects more technical questions from tutorials/lectures
- There is a fair amount of work that goes into making and editing the videos (perhaps between 1-2 hours per video). However, these do become quicker after a few have been made. Also, this is a resource with will last for future years, reducing workload in the longer term.
- Camtasia is very easy to use after some initial training, and support is available through CTEL.
- Uploading the videos to KEATS through the Kaltura Media Resource item can be a bit slow but usually this works without any problems.
It is also possible to use the KEATS analytics to track how many students are engaging with the process. In its early stages, I saw very high engagement.
What challenges did you encounter and how did you address them?
There are a few challenges to making the videos:
- They are a bit time consuming to make (as mentioned above, though this is a one-off cost). Also, if using Camtasia, is not easy to correct the videos if a mistake is spotted, without redoing the whole thing.
- Student engagement is below 50% and there was a significant drop off after the first few weeks. However, students also commented that they were likely to use the videos again for revision and appreciated having them there even if they did not feel they had time to complete them each week.
There is also room to further improve the video-quiz combination. At the moment, the video and quiz appear as separate items on the KEATS page:
However, in collaboration with the TEL Manager at KBS, we have been looking into making this into a one-step process using H5P (HTML5). This allows a quiz to be placed into a video interactively so that the quiz question appears at a certain point of the video. This might enhance the student journey, making it less clunky and making it more likely for them to do the quiz question while watching the video. There have been some initial teething problems with converting maths equations from the quizzes using H5P.
What advice would you give to colleagues who are thinking of trying flipped assessment quizzes?
I think this type of resource is useful to students and has certainly proved to be popular in its early stages. I would advise colleagues to get in touch with their TEL manager or CTEL before starting this as there are many shortcuts and ideas which I did not think of originally. The world of educational videos and software/hardware appears to be developing very quickly, so it is also useful to check what is out there before making an investment of time in making videos.
This kind of approach could potentially be used as a summative assessment. This could either be done by extending the quiz element into a summative form, or by using another type of assessment based on the videos. This would be dependent on the specific context of the module.
Leave a Reply