Automatic Classification of Online Learner Reviews Via Fine-Tuned BERTs

Authors

DOI:

https://doi.org/10.19173/irrodl.v26i1.8068

Keywords:

learner-generated content, automatic classification, fine-tuned, BERTs, course evaluation

Abstract

Massive open online courses (MOOCs) offer rich opportunities to comprehend learners’ learning experiences by examining their self-generated course evaluation content. This study investigated the effectiveness of fine-tuned BERT models for the automated classification of topics in online course reviews and explored the variations of these topics across different disciplines and course rating groups. Based on 364,660 course review sentences across 13 disciplines from Class Central, 10 topic categories were identified automatically by a BERT-BiLSTM-Attention model, highlighting the potential of fine-tuned BERTs in analysing large-scale MOOC reviews. Topic distribution analyses across disciplines showed that learners in technical fields were engaged with assessment-related issues. Significant differences in topic frequencies between high- and low-star rating courses indicated the critical role of course quality and instructor support in shaping learner satisfaction. This study also provided implications for improving learner satisfaction through interventions in course design and implementation to monitor learners’ evolving needs effectively.

Author Biographies

Xieling Chen, School of Education, Guangzhou University, Guangzhou, China

Xieling Chen is an Associate Professor at Guangzhou University, China. Her research interests include artificial intelligence in education and text mining. She has over 60 publications. Stanford University has listed her as one of the World's Top 2% Scientists in 2022, 2023, and 2024.

Di Zou, Department of English and Communication, The Hong Kong Polytechnic University, Hong Kong SAR

Di Zou is an Associate Professor at The Hong Kong Polytechnic University. Her research interests include AI in language education and TELL. She has over 150 publications. Stanford University has listed her as one of the World's Top 2% Scientists in 2021, 2022, 2023, and 2024. She is an Editor of Computers & Education.

Haoran Xie, School of Data Science, Lingnan University, Hong Kong SAR

Haoran Xie is a Professor at Lingnan University, Hong Kong. His research interests include artificial intelligence in education and big data. He has over 320 publications. He is the Editor-in-Chief/Associate Editor of several SCI/SSCI journals. Stanford University has listed him as one of the World's Top 2% Scientists in 2021, 2022, 2023, and 2024.

Gary Cheng, Department of Mathematics and Information Technology, The Education University of Hong Kong, Hong Kong SAR

Gary Cheng is an Associate Professor at The Education University of Hong Kong, Hong Kong. He has been actively involved in organizing events and activities with colleagues to promote STEM among children and adolescents (e.g., STEM Competition in Smart Home Design and STEAM Education: 3-D Chinese Cultural Architectural Design Competition). His interests include data mining, deep learning, and computer programming education.

Zongxi Li, School of Data Science, Lingnan University, Hong Kong SAR

Zongxi Li is an Assistant Professor at Lingnan University, Hong Kong. He has authored over 27 papers including top-tier journal articles, such as Pattern Recognition, Knowledge-based Systems, Information Processing & Management, and IEEE Transactions on Affective Computing, and high-impact conference proceedings, such as AAAI and ACL. He was awarded the Best Paper Award from WI-IAT and the Best Paper Runner-up Award from BESC.

Fu Lee Wang, School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR

Fu Lee Wang is the Dean and Professor at Hong Kong Metropolitan University, Hong Kong. His research interests include e-learning and information retrieval. Professor Wang has over 300 publications and 40 grants with more than 80 million Hong Kong dollars. He was also the Chair of ACM Hong Kong Chapter and IEEE Hong Kong Section Computer Society.

References

Alario-Hoyos, C., Estévez-Ayres, I., Pérez-Sanagustín, M., Delgado Kloos, C., & Fernández-Panadero, C. (2017). Understanding learners’ motivation and learning strategies in MOOCs. International Review of Research in Open and Distributed Learning, 18(3), 119–137. https://doi.org/10.19173/irrodl.v18i3.2996

Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., & Gašević, D. (2020, March). How good is my feedback? A content analysis of written feedback. In Proceedings of the 10th International Conference on Learning Analytics & Knowledge (pp. 428–437). https://doi.org/10.1145/3375462.3375477

Cavalcanti, A. P., Mello, R. F., Gašević, D., & Freitas, F. (2023). Towards explainable prediction feedback messages using BERT. International Journal of Artificial Intelligence in Education, 34, 1046–1071. https://doi.org/10.1007/s40593-023-00375-w

Chen, X., Zou, D., Cheng, G., & Xie, H. (2024). Deep neural networks for the automatic understanding of the semantic content of online course reviews. Education and Information Technologies, 29(4), 3953–3991. https://doi.org/10.1007/s10639-023-11980-6

Conrad, D., & Openo, J. (2018). Assessment strategies for online learning: Engagement and authenticity. Athabasca University Press. https://doi.org/10.15215/aupress/9781771992329.01

El-Rashidy, M. A., Farouk, A., El-Fishawy, N. A., Aslan, H. K., & Khodeir, N. A. (2023). New weighted BERT features and multi-CNN models to enhance the performance of MOOC posts classification. Neural Computing and Applications, 35(24), 18019–18033. https://doi.org/10.1007/s00521-023-08673-z

Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320–341. https://doi.org/10.1111/bjet.12235

Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers & Education, 145, 103724. https://doi.org/10.1016/j.compedu.2019.103724

Li, L., Johnson, J., Aarhus, W., & Shah, D. (2022). Key factors in MOOC pedagogy based on NLP sentiment analysis of learner reviews: What makes a hit. Computers & Education, 176, 104354. https://doi.org/10.1016/j.compedu.2021.104354

Liu, Z., Kong, X., Chen, H., Liu, S., & Yang, Z. (2023). MOOC-BERT: Automatically identifying learner cognitive presence from MOOC discussion data. IEEE Transactions on Learning Technologies, 16(4), 528–542. https://doi.org/10.1109/TLT.2023.3240715

Moore, M. G. (2013). The theory of transactional distance. In Moore, M. G. (Eds.) Handbook of distance education (pp. 66–85). Routledge. https://doi.org/10.4324/9780203803738

Qaddumi, H., Bartram, B., & Qashmar, A. L. (2021). Evaluating the impact of ICT on teaching and learning: A study of Palestinian students’ and teachers’ perceptions. Education and Information Technologies, 26(2), 1865–1876. https://doi.org/10.1007/s10639-020-10339-5

Sebbaq, H., & El Faddouli, N. (2022). Fine-tuned BERT model for large scale and cognitive classification of MOOCs. International Review of Research in Open and Distributed Learning, 23(2), 170–190. https://doi.org/10.19173/irrodl.v23i2.6023

Wulff, P., Mientus, L., Nowak, A., & Borowski, A. (2023). Utilizing a pretrained language model (BERT) to classify preservice physics teachers’ written reflections. International Journal of Artificial Intelligence in Education, 33(3), 439–466. https://doi.org/10.1007/s40593-022-00290-6

Yousef, A. M. F., & Sumner, T. (2021). Reflections on the last decade of MOOC research. Computer Applications in Engineering Education, 29(4), 648–665. https://doi.org/10.1002/cae.22334

Downloads

Published

2025-02-25

How to Cite

Chen, X., Zou, D., Xie, H., Cheng, G., Li, Z., & Wang, F. L. (2025). Automatic Classification of Online Learner Reviews Via Fine-Tuned BERTs. The International Review of Research in Open and Distributed Learning, 26(1), 57–79. https://doi.org/10.19173/irrodl.v26i1.8068

Issue

Vol. 26 No. 1 (2025)

Section

Research Articles

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

This work is licensed under a Creative Commons Attribution 4.0 International License. The copyright for all content published in IRRODL remains with the authors.

This copyright agreement and usage license ensure that the article is distributed as widely as possible and can be included in any scientific or scholarly archive.

You are free to

  • Share — copy and redistribute the material in any medium or format
  • Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms below:

  • No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.