The Impact of a Learning Analytics Based Feedback System on Students’ Academic Achievement and Self-Regulated Learning in a Flipped Classroom
DOI:
https://doi.org/10.19173/irrodl.v26i1.7924Keywords:
learning analytics, flipped learning, academic achievement, experimental design, self-regulationAbstract
Recent advancements in educational technology have enabled teachers to use learning analytics (LA) and flipped classrooms. The present study investigated the impact of a LA-based feedback system on students’ academic achievement and self-regulated learning (SRL) in a flipped learning (FL) environment. The study used a pretest-posttest control group quasi-experimental design with 71 pre-service teachers in the experimental group and 56 pre-service teachers in the control group, both enrolled in an information technology course. The experimental group received LA-based feedback during a 4-week training program in the FL classroom, while the control group did not receive this feedback. Data were collected using an achievement test, an online SRL questionnaire, and a student opinion form. The study found that the students’ SRL and academic achievement were not significantly affected by the LA-based feedback system in FL classrooms. In contrast, according to the qualitative research findings, students claimed the LA-based feedback helped them learn because it allowed them to monitor their learning processes.
References
Aguilar, S. J., Karabenick, S. A., Teasley, S. D., & Baek, C. (2021). Associations between learning analytics dashboard exposure and motivation and self-regulated learning. Computers & Education, 162, Article 104085. https://doi.org/10.1016/j.compedu.2020.104085
Akçayır, G., & Akçayır, M. (2018). The flipped classroom: A review of its advantages and challenges. Computers & Education, 126, 334–345. https://doi.org/10.1016/j.compedu.2018.07.021
Algayres, M. G., & Triantafyllou, E. (2020). Learning analytics in flipped classrooms: A scoping review. Electronic Journal of e-Learning, 18(5), 397–409. https://doi.org/10.34190/JEL.18.5.003
Alvi, M. (2016). A manual for selecting sampling techniques in research (MPRA Paper No. 70218). Munich Personal RePEc Archive. https://mpra.ub.uni-muenchen.de/70218/1/MPRA_paper_70218.pdf
Barnard, L., Paton, V., & Lan, W. (2008). Online self-regulatory learning behaviors as a mediator in the relationship between online course perceptions with achievement. The International Review of Research in Open and Distributed Learning, 9(2). https://doi.org/10.19173/irrodl.v9i2.516
Bayazit, A., Apaydin, N., & Gonullu, I. (2022). Predicting at-risk students in an online flipped anatomy course using learning analytics. Education Sciences, 12(9), Article 581. https://doi.org/10.3390/educsci12090581
Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. International Society for Technology in Education.
Birgili, B., & Demir, Ö. (2022). An explanatory sequential mixed-method research on the full-scale implementation of flipped learning in the first years of the world’s first fully flipped university: Departmental differences. Computers & Education, 176, Article 104352. https://doi.org/10.1016/j.compedu.2021.104352
Büyüköztürk, Ş. (2021). Handbook of data analysis for social sciences (29th ed.), Pegem.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.
Çebi, A., & Güyer, T. (2020). Students’ interaction patterns in different online learning activities and their relationship with motivation, self-regulated learning strategy and learning performance. Education and Information Technologies, 25, 3975–3993. https://doi.org/10.1007/s10639-020-10151-1
Davies, R., Allen, G., Albrecht, C., Bakir, N., & Ball, N. (2021) Using educational data mining to identify and analyze student learning strategies in an online flipped classroom. Education Sciences, 11(11), Article 668. https://doi.org/10.3390/educsci11110668
Doo, M. Y., & Park, Y. (2024). Pre-class learning analytics in flipped classroom: Focusing on resource management strategy, procrastination and repetitive learning. Journal of Computer Assisted Learning, 40(3), 1231–1245. https://doi.org/10.1111/jcal.12946
Dooley, L., & Makasis, N. (2020). Understanding student behavior in a flipped classroom: Interpreting learning analytics data in the veterinary pre-clinical sciences. Education Sciences, 10(10), Article 260. https://doi.org/10.3390/educsci10100260
Fidan, M. (2023). The effects of the microlearning-supported flipped classroom on pre-service teachers’ learning performance, motivation and engagement. Education and Information Technologies, 28, 12687–12714. https://doi.org/10.1007/s10639-023-11639-2
Flipped Learning Network. (2014). The four pillars of F-L-I-P™. https://flippedlearning.org/definition-of-flipped-learning/
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/J.IHEDUC.2015.10.002
George, D., & Mallery, P. (2019). IBM SPSS statistics 26 step by step: A simple guide and reference. Routledge.
Guo, L. (2022). Using metacognitive prompts to enhance self‐regulated learning and learning outcomes: A meta‐analysis of experimental studies in computer‐based learning environments. Journal of Computer Assisted Learning, 38(3), 811–832. https://doi.org/10.1111/jcal.12650
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, Article 100001. https://doi.org/10.1016/j.caeai.2020.100001
Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33(4), 74–85. http://dx.doi.org/10.1016/j.iheduc.2017.02.001
Karaca O., Çınarcık, B. Ş., Aşık, A., Sağlam, C., Yiğit, Y., Hakverdi, G., Yetkiner, A. A., & Ersin, N. (2024). Impact of fully online flipped classroom on academic achievement in undergraduate dental education: An experimental study. European Journal of Dental Education, 28(1), 212–226. https://doi.org/10.1111/eje.12938
Karaoglan Yilmaz, F. G. (2022). Utilizing learning analytics to support students’ academic self-efficacy and problem-solving skills. The Asia-Pacific Education Researcher, 31, 175–191. https://doi.org/10.1007/s40299-020-00548-4
Karaoglan Yilmaz, F. G., & Yilmaz, R. (2020). Student opinions about personalized recommendation and feedback based on learning analytics. Technology, Knowledge and Learning, 25, 753–768. https://doi.org/10.1007/s10758-020-09460-8
Kew, S. N., & Tasir, Z. (2021). Learning analytics in online learning environment: A systematic review on the focuses and the types of student-related analytics data. Technology, Knowledge and Learning, 27, 405–427. https://doi.org/10.1007/s10758-021-09541-2
Kilis, S., & Yıldırım, Z. (2018). Investigation of community of inquiry framework in regard to self-regulation, metacognition and motivation. Computers & Education, 126, 53–64. https://doi.org/10.1016/j.compedu.2018.06.032
Kim, D., Yoon, M., Jo, I.-H., & Branch, R. M. (2018). Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women’s university in South Korea. Computers & Education, 127, 233–251. https://doi.org/10.1016/j.compedu.2018.08.023
Kim, J., Jo, I.-H., & Park, Y. (2016). Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Education Review, 17, 13–24. https://doi.org/10.1007/s12564-015-9403-8
Klašnja-Milićević, A., Ivanović, M., & Stantić, B. (2020). Designing personalized learning environments—The role of learning analytics. Vietnam Journal of Computer Science, 7(3), 231–250. https://dx.doi.org/10.1142/S219688882050013X
Kline, P. (2013). Handbook of psychological testing. Routledge.
Koenka, A. C., & Anderman, E. M. (2019). Personalized feedback as a strategy for improving motivation and performance among middle school students. Middle School Journal, 50(5), 15–22. https://doi.org/10.1080/00940771.2019.1674768
Kovanovic, V., Mazziotti, C., & Lodge, J. (2021). Learning analytics for primary and secondary schools. Journal of Learning Analytics, 8(2), 1–5.
https://doi.org/10.18608/jla.2021.7543
Lan, W. Y., Bremer, R., Stevens, T., & Mullen, G. (2004, April 7–8). Self-regulated learning in the online environment [Paper presentation]. 2004 Annual Meeting of the American Educational Research Association, San Diego, CA, United States.
Li, L.-Y., & Tsai, C.-C. (2017). Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education, 114, 286–297. http://dx.doi.org/10.1016/j.compedu.2017.07.007
Lim, L., Bannert, M., van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., Rakovic, M., Molenaar, I., Moore, J., & Gašević, D. (2023). Effects of real-time analytics-based personalized scaffolds on students’ self-regulated learning. Computers in Human Behavior, 139, Article 107547. https://doi.org/10.1016/j.chb.2022.107547
Lin, C.-J., & Hwang, G.-J. (2018). A learning analytics approach to investigating factors affecting EFL students’ oral performance in a flipped classroom. Journal of Educational Technology & Society, 21(2), 205–219. https://www.jstor.org/stable/26388398
Lu, O. H. T., Huang, J. C. H., Huang, A. Y. Q., & Yang, S. J. H. (2017). Applying learning analytics for improving students engagement and learning outcomes in an MOOCs enabled collaborative programming course. Interactive Learning Environments, 25(2), 220–234. https://doi.org/10.1080/10494820.2016.1278391
Montgomery, A. P., Mousavi, A., Carbonaro, M., Hayward, D. V., & Dunn, W. (2019). Using learning analytics to explore self‐regulated learning in flipped blended learning music teacher education. British Journal of Educational Technology, 50(1), 114–127. https://doi.org/10.1111/bjet.12590
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002
Omarchevska, Y., van Leeuwen, A., & Mainhard, T. (2024, February 22). The flipped classroom: First-time student preparatory activity patterns and their relation to course performance and self-regulation. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-024-09399-0
Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, Article 100020. https://doi.org/10.1016/j.caeai.2021.100020
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592
Park, Y., & Jo, I.-H. (2015). Development of the learning analytics dashboard to support students’ learning performance. Journal of Universal Computer Science, 21(1), 110–133. https://doi.org/10.3217/jucs-021-01-0110
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). Academic Press.
Polat, E., Hopcan, S. & Arslantaş, T. K. (2022). The association between flipped learning readiness, engagement, social anxiety, and achievement in online flipped classrooms: A structural equational modeling. Education and Information Technologies, 27, 11781–11806. https://doi.org/10.1007/s10639-022-11083-8
Polat, H., & Taslibeyaz, E. (2023). Examining interactive videos in an online flipped course context. Education and Information Technologies, 29, 5833–5856. https://doi.org/10.1007/s10639-023-12048-1
Ramaswami, G., Susnjak, T., & Mathrani, A. (2023). Effectiveness of a learning analytics dashboard for increasing student engagement levels. Journal of Learning Analytics, 10(3), 115–134. https://doi.org/10.18608/jla.2023.7935
Roberts, L. D., Howell, J. A., & Seaman, K. (2017). Give me a customizable dashboard: Personalized learning analytics dashboards in higher education. Technology, Knowledge and Learning, 22, 317–333. https://doi.org/10.1007/s10758-017-9316-1
Rubio‐Fernández, A., Muñoz‐Merino, P. J., & Delgado Kloos, C. (2019). A learning analytics tool for the support of the flipped classroom. Computer Applications in Engineering Education, 27(5), 1168–1185. https://doi.org/10.1002/cae.22144
Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling. Lawrence Erlbaum Associates.
Schunk, D. H., & Greene, J. A. (2017). Historical, contemporary, and future perspectives on self-regulated learning and performance. In D. H. Schunk & J. A. Greene (Eds.), Handbook of self-regulation of learning and performance (pp. 1–15). Routledge.
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, Article 105512. https://doi.org/10.1016/j.chb.2018.05.004
Šereš, L., Pavlićević, V., Petrović, G., Horvat, D., & Ivanišević, R. (2022). Learning analytics: Prospects and challenges. Strategic Management, 27(3), 48–55. https://doi.org/10.5937/StraMan2200020S
Shen, D., & Chang, C.-S. (2023). Implementation of the flipped classroom approach for promoting college students’ deeper learning. Educational Technology Research and Development, 71, 1323–1347. https://doi.org/10.1007/s11423-023-10186-4
Siemens, G., & Gašević, D. (2012). Special issue on learning and knowledge analytics. Educational Technology & Society, 15(3), 1–163.
Silva, J. C. S., Zambom, E., Rodrigues, R. L., Ramos, J. L. C., & de Souza, F. D. F. (2018). Effects of learning analytics on students’ self-regulated learning in flipped classroom. International Journal of Information and Communication Technology Education (IJICTE), 14(3), 91–107. https://doi.org/10.4018/IJICTE.2018070108
Sireci, S. G., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26(1), 100–107. https://doi.org/10.7334/psicothema2013.256
Staker, H., & Horn, M. B. (2012). Classifying K–12 blended learning. Innosight Institute. https://files.eric.ed.gov/fulltext/ED535180.pdf
Tempelaar, D., Bátori, A., & Giesbers, B. (2024) Understanding self-regulation strategies in problem-based learning through dispositional learning analytics. Frontiers in Education, 9, Article 1382771.https://www.doi.org/10.3389/feduc.2024.1382771
Ustun, A. B., Karaoglan Yilmaz, F. G., & Yilmaz, R. (2021). Investigating the role of accepting learning management system on students’ engagement and sense of community in blended learning. Education and Information Technologies, 26, 4751–4769. https://doi.org/10.1007/s10639-021-10500-8
Ustun, A. B., Zhang, K., Karaoglan-Yilmaz, F. G., & Yilmaz, R. (2022). Learning analytics-based feedback and recommendations in flipped classrooms: An experimental study in higher education. Journal of Research on Technology in Education, 55(5), 841–857. https://doi.org/10.1080/15391523.2022.2040401
Uzir, N. A., Gašević, D., Matcha, W., Jovanović, J., & Pardo, A. (2020). Analytics of time management strategies in a flipped classroom. Journal of Computer Assisted Learning, 36(1), 70–88. https://doi.org/10.1111/jcal.12392
Wang, D., & Han, H. (2021). Applying learning analytics dashboards based on process‐oriented feedback to improve students’ learning effectiveness. Journal of Computer Assisted Learning, 37(2), 487–499. https://doi.org/10.1111/jcal.12502
Wong, B. T.-M., Li, K. C., & Cheung, S. K. S. (2022). An analysis of learning analytics in personalised learning. Journal of Computing in Higher Education, 35, 371–390. https://doi.org/10.1007/s12528-022-09324-3
Yang, C. C. Y., Chen, I. Y. L., Akçapınar, G., Flanagan, B., & Ogata, H. (2021). Using a summarized lecture material recommendation system to enhance students’ preclass preparation in a flipped classroom. Educational Technology & Society, 24(2), 1–13. https://www.jstor.org/stable/27004927
You, J. W. (2015). Examining the effect of academic procrastination on achievement using LMS data in e-learning. Journal of Educational Technology & Society, 18(3), 64–74. https://www.jstor.org/stable/jeductechsoci.18.3.64
Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License. The copyright for all content published in IRRODL remains with the authors.
This copyright agreement and usage license ensure that the article is distributed as widely as possible and can be included in any scientific or scholarly archive.
You are free to
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
The licensor cannot revoke these freedoms as long as you follow the license terms below:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
