Summary
In this post, we describe the research presented at the King’s Education Conference 2024, which focused on the impact of student engagement with KEATS on academic performance.
The Story
Motivation
Virtual Learning Environments like KEATS offer students convenient and flexible access to course materials. Engaging with these platforms requires a deliberate investment of time and effort from students, with the potential rewards of significantly enhancing their academic performance. Therefore, understanding this relationship is essential for creating effective online learning environments that promote student success. Our study aimed to investigate how the total hours students spent on KEATS correlates with their final exam grades.
Study Context
Our research focused on a one-year mathematics foundation module at King’s Foundation during the 2022-2023 academic year. This module followed a blended learning approach, combining online and face-to-face components. The study involved 238 students, aged 17 to 20, from different nationalities and educational backgrounds. The students were required to spend 2 hours each week on KEATS, engaging with instructional videos, practice problems, and exercises, while also attending weekly seminars and live lectures in person.
Data Collection
We analysed two main datasets: KEATS engagement data and student performance data. The KEATS engagement data included total time spent online, and the performance data consisted of grades from various summative assessments, including the final exam.
The Outcomes
Exploratory Data Analysis
We first used descriptive statistics and visualizations to assess the data. The scatter plot below suggested a linear relationship between time spent on KEATS and final exam grade, indicating that students who spent more time on KEATS generally achieved higher grades. Interestingly, the plot also revealed an empty area in the bottom right corner, showing that no students with high engagement had low final exam grades.
Identifying Patterns
Next, we categorized engagement into quartiles and found that students with higher engagement levels consistently achieved better final exam grades, as shown in the boxplots below. The analysis strongly suggests that lower engagement with KEATS is associated with lower grades and that this pattern is unlikely to be coincidental.
Correlation and Regression Analysis
Finally, the Pearson correlation coefficient revealed a moderate positive relationship between KEATS engagement and final exam grades (r=0.33). To further investigate this relationship, we used multiple linear regression models. The best model, which included all summative tests and total hours online, explained 39% of the variance in final exam grades. This model suggests that each additional hour spent on KEATS was associated with an increase of approximately 0.2 percentage points in the final exam grade.
Conclusion & Recommendations
Our study indicates that increased interaction with KEATS leads to better academic outcomes. The results of this analysis can inform the development of targeted strategies to support all students, promoting a more inclusive and effective learning environment.
Educational Implications
Based on our findings, we recommend the following actions:
- Encourage active participation in KEATS to enhance academic performance. Tools like the KEATS Course Participation Report can send targeted messages and reminders to boost engagement.
- Identify students needing additional support by analysing KEATS data and implementing timely interventions.
- Monitor KEATS usage and assessment results regularly to track student progress and tailor pedagogical approaches.
We hope this blog provides valuable insights into the importance of student engagement with KEATS. For any questions or further discussion, please feel free to contact me at eleonora.pinto_de_moura@kcl.ac.uk
Technical Note
The data analysis was conducted in Python, using libraries such as pandas for data manipulation, matplotlib and seaborn for data visualization, and statsmodels for statistical modelling.
Useful Links
- KEATS Course Participation Report
- Python Libraries for Data Analysis
- Data Visualization in Python using Matplotlib and Seaborn
Intended Audience
Educators, Academic Researchers, Learning Technologists and Administrators.
About the Author
My name is Eleonora Pinto de Moura, and I’m a Lecturer in Mathematics Education at King’s Foundations. My research interests include educational technology, data-driven teaching strategies, and enhancing student engagement through innovative learning digital tools.
