Understanding learner behaviors online is important in addressing challenges they may be facing, offering appropriate support and ensuring the instructional design meets their needs. This is the object of Learning Analytics (LA), which aims to analyse and optimize learning and the environments in which it occurs.
Research has shown that utilising LA insights can lead to positive impacts on student engagement, and a change in organisational culture towards a more data driven approach across a University’s business [1]. Moreover, examining student activity might help identify effective teaching practices and practitioners and thereby influence the design of future courses [2]. Research suggests that such a proactive approach, in which an institution actively approaches students, is more effective than a reactive approach, in which the institution offers services that students have to initiate themselves. Finally, it has been shown that LA can support students in gaining valuable skills such as: time-management, planning and metacognition [3].
There are many challenges relating to the application and use of LA, such as the potential to misuse the data [1], instructors having to design more opportunities for students to take responsibility of their learning [2] and the requirement for ensuring that any data analysis is overlaid with informed and contextualized interpretations [4]. In this article I’ll focus on challenges relating to the technical limitations of a native Moodle installation.
There are a number of tools inside Moodle which can provide a basic level of learning analytics:
- Logs page – records individual interactions in table format
- Activity reports – presents the popularity of resources and activities
- Course participation -identifies if students have engaged with content within a set timeframe
- Activity completion – shows if students have met the completion criteria that have been set
The challenge with these is the limited meaning in the visualisations (e.g. hundreds of rows of data) and the added complexity of interpreting information that is scattered across multiple pages. In addition, date ranges cannot be filtered, and all logs are mixed up (i.e., admins, tutors, students). This approach significantly reduces the usability and usefulness of the analytics, hindering the ability to develop meaningful insights.
To overcome these challenges, CTEL has developed a PowerBI template which can be shared with colleagues to assist them in the process of analysing Moodle logs data. Our approach takes advantage of the ability to download the logs in Microsoft Excel and provides a simple but effective offline solution (which can be published and shared with colleagues online).
The PowerBI template (known as the KAT – KEATS Analytics Tools) can achieve several things, going much beyond what’s possible in Moodle alone:
- Identify the popularity of resources and activities
- Recognise struggling or uninvolved students (and other student needs)
- Assist in scheduling timely activities, reminders and nudges to extend engagement and motivation
- Set and evaluate measurable goals and record evidence of change in student behaviour online
- Compare student interactions across modules
- Filter by date ranges, users or activities/resources
Moreover, the PowerBI environment is much more dynamic. Clicking on any of the diagrams or graphs instantly filters the other diagrams as well (clicking on specific user, activity or month for example).
Four instances of focus groups were carried out during June 2019; tailored appropriately to the institutional role and time limitations of the participating individuals. Individuals were asked to share their views on the general topic of LA and the prospect of introducing this practice to the King’s College Community.
After analysing the results, we can identify clear eagerness and expectation from both staff and students to begin exploring the application of LA in higher education. Staff generally agreed that LA would allow the creation of more effective KEATS learning spaces (76.4%), enable the application of pedagogic improvements and innovation (88.2%) and lead to more timely information on student’s learning progress (94.1%). From the students’ side there was general consensus that LA would lead to a more personalised learning experience.
LA can be a sensitive subject among the student community. It is therefore important to respect student privacy and request to obtain student consent before engaging with the online data. In addition, it is important to ask the right questions (of the data) and avoid quick conclusions. The data of online interactions cannot provide a complete picture of a student’s learning behavior and habits. It can, however, be used as an important starting point to engage in meaningful conversations.
For more information please take a look at:
- KCL Student Data Collection Notice
- General Data Protection Regulation (GDPR) training
- If you are unsure about your obligations when processing sensitive data seek advice from the Information Compliance team.
For more information please contact ctel@kcl.ac.uk (King’s staff only).
*Note: KEATS is King’s College London’s customised instance of Moodle.
References:
- Case Study: Predictive analytics at Nottingham Trent University.
- Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course Designs” (Research Bulletin). Louisville, CO: EDUCAUSE Center for Applied Research, April 30, 2013, available from http://www.educause.edu/ecar.
- Literature Review: The use of learning analytics to support student retention and success, ABLE Project (2015-1-UK01-KA203-013767).
- Macfadyen, L. P., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Educational Technology & Society, 15 (3), 149–163.
Useful Links:
- [MOOC] edX: Learning Analytics in Higher Education
- [Book] Big Data: Using smart big data analytics and metrics to make better decisions and improve performance – Bernard Marr
- [LinkedIn Learning] PowerBI Essential Training
Written by Andreas Konstantinidis
Andreas Konstantinidis has been working in CTEL as a learning technologist for more than 7 years, advising on, designing, developing and evaluating resources (and associated workshops) to meet the needs of the Centre and its work on formal/informal College-wide Digital Education and related activities.
