Understanding Learner’s Behaviour using Machine Learning in Serious Game

Serious game has been introduced as an interactive educational tool for teaching and learning processes. It incorporates non-entertainment elements into an interactive game environment. Although serious game offers various benefits to support teaching and learning in a variety of contexts, measuring the learner's skills and knowledge improvement is difficult. The results of serious games (i.e. game scores) might be used to determine the effectiveness of using serious games as a teaching medium. However, understanding the learning process of learners that use serious game is difficult. Two main problems are: 1) how to understand the learner’s skill and performance improvement, and 2) how to capture and analyse the data.

Our work focuses on developing a solution to answer these two problems. We develop a scalable and configurable serious game that can gather relevant game data to be used in learning analytics. We incorporate various Machine Learning (ML) techniques to analyze the game data and identify the learner’s learning improvement. The results are then visualized in an interactive dashboard to provide insights for the learners as well as the lecturers to understand the learning processes. It can also be used as a guideline for the lecturers to provide specific scaffolding actions to help the learners understand the topic better. Three ML techniques have been implemented in a serious game for learning IT / programming concepts. These techniques are sentiment analytics, keyword discovery and clustering. The first technique, sentiment analysis, is used to study opinions, sentiments and emotions expressed in learner’s feedback on using a serious game to learn IT / programming concepts. It can be used to identify positive or negative opinions based on a set of positive and negative lexicon. The sentiments (i.e. positive and negative sentiments) are then visualized using graphical tools to provide lecturers with a summary of the sentiments. Using this information, the lecturers can determine whether the learners require additional help based on their emotions and the feedback they provided. The second technique, keyword discovery, is used to find keywords that were commonly mentioned in the feedback. These keywords can help the lecturers to understand the feedback by grouping them together and showing it through a word cloud. A word cloud is a visual representation for word frequencies. The size of the words shown on the word cloud is directly proportional to the frequency. The third technique, clustering, is used to group learners based on their behavior while playing the game. Three groups of learners are formed and displayed to the lecturers. The lecturers can then identify which learners have difficulties in learning the concepts and can interfere by providing specific scaffolding to the learners.

To evaluate the solution, we conducted semi-structured interviews with lecturers. The semi-structured interview sessions were held in December 2020 with seven lecturers. The lecturers commented that the solution would help them to monitor and evaluate the student’s learning. The insights that were captured in the framework are useful for the lecturers to understand the learners’ experience and learning better. Using the insight provided in the dashboard, the lecturers would be able to identify weak leaners to conduct follow-up discussions to improve their learning.

This work proposes an interactive education tool by using serious game to complement teaching and learning processes that is equipped with ML models to understand the learner’s learning behaviour. The serious game would be able to gather game data required to perform learning analysis using sentiment analytics, keyword discovery and clustering algorithms. The insight in then visualized in an interactive dashboard. The lecturers can use this dashboard to understand learner’s improvement and conduct scaffolding.