International Review of Research in Open and Distributed Learning

Volume 23, Number 1

February - 2022

AI in Online-Learning Research: Visualizing and Interpreting the Journal Publications from 1997 to 2019

Gwo-Jen Hwang¹, Yun-Fang Tu², and Kai-Yu Tang^3
1National Taiwan University of Science and Technology, ²Fu Jen Catholic University, ³Ming Chuan University

Abstract

This study reviews the journal publications of artificial intelligence-supported online learning (AIoL) in the Web of Science (WOS) database from 1997 to 2019 taking into account the contributing countries/areas, leading journals, highly cited papers, authors, research areas, research topics, roles of AIoL, and adopted artificial intelligence (AI) algorithms. Results indicate that, from 1997 to 2009, AIoL research focused on the combination of intelligent tutoring systems and distance learning. In 2010-2014, AIoL research emphasized learner-oriented learning. In 2015-2019, learner-system interactions to facilitate personalized, adaptive, and collaborative learning became the main focus. “Intelligent tutoring systems” have played the most important role in AIoL, followed by “profiling and prediction,” and “adaptive systems with personalization.” Accordingly, the roles and research trends as well as several suggestions for future research in the field of AIoL are provided as a reference for researchers and policy makers.

Keywords: artificial intelligence, online learning, literature review, trend analysis, visualization

Introduction

In past decades, online learning has been adopted by researchers and educators for delivering courses in various domains (Hwang & Tsai, 2011; Martin et al., 2020). Nevertheless, many researchers have pointed out the problems of this learning mode, including the low assignment completion rate and poor learning outcomes (Lee & Choi, 2011; Yu et al., 2017). Dropout rates are one of the thorny issues in online learning environments. Some scholars have attempted to reduce dropout rates through strategies such as understanding each student’s challenges and potential, providing quality curricular activities with good learning supports, and promoting quality learning experiences with learning guidance (Lee & Choi, 2011; Hussain et al., 2019; Lee et al., 2020). However, it is nearly impossible for teachers to provide personalized learning support or guidance to individual learners when they need to face dozens, hundreds, or even thousands of students in online classes.

The advancement of artificial intelligence (AI) technologies provides an opportunity to address this problem. AI technologies can be used not only to predict students’ learning status, but also to provide required support or guidance by analyzing students’ online learning behaviors, personal characteristics (e.g., preferences or cognitive styles), and learning performances (Hwang et al., 2020). Several scholars have reported that using AI technologies to provide personalized learning supports has good potential to promote learner engagement (Lin et al., 2018) and enhance students’ positive learning experiences (Yu et al., 2017). In addition, AI technology can be used to diagnose learners’ personal learning problems and provide immediate assistance or advice accordingly (Chen et al., 2020; Chen & Lain, 2020).

With the growing interest in AI-supported education, researchers have conducted review studies on particular research domains, including medical education (Han et al., 2019), engineering education (Shukla et al., 2019), higher education (Zawacki-Richter et al., 2019), and e-learning research (Tang et al., 2021). In medical education, Han et al. (2019) proposed some thematic trends and explained the trends of advanced technology and artificial intelligence for future physicians. Using bibliometric analysis, Shukla et al. (2019) compared two reputed databases, the Web of Science and Scopus, and identified some frequently cross-referenced engineering applications of artificial intelligence. Zawacki-Richter et al. (2019) conducted a study to review AI in education research published between 2007 and 2018, and reported that AI technologies can facilitate profiling and prediction as well as the development of intelligent tutoring systems (ITS), adaptive systems, and recommendation systems. However, in their study, the features of online learning and several important characteristics of AI in education (e.g., contributing countries and authors, research areas, and research topics), which are valuable for providing clear direction to novice researchers, were not taken into account. Recently, researchers used co-citation network analysis to identify some highly co-cited research streams and their extensions in the e-learning area (Tang et al., 2021). They found that AI has been mainly used as an adaptive learning environment for learners. In addition to bibliometric analysis, some researchers have also proposed a state-of-the-art overview and positioning review to provide expert opinions to AI development, including machine learning for e-learning (Khanal et al., 2020) and deep learning in medical education (Carin, 2020). The main topics and methods of the previous review studies are summarized in Table 1.

Table 1

A List of Recent AI-Related Review Studies With an Education Focus

The above-mentioned research suggests that a review study would be valuable and could help novice researchers efficiently and effectively acquire knowledge of the research trends and focus in the field. It also implied that it would be important to conduct review studies for AI in online learning (AIoL) since AIoL is becoming an important field of educational technology (Chen & Lain, 2020). Taken together, this present study aimed to contribute to the AIoL literature in two ways. First, a series of bibliometric analyses provided quantitative results to represent the international publication patterns of AIoL research, including the most productive countries, journals, highly cited papers, and authors in the field. The results could complement some conceptual frameworks of previous narrative reviews and positional papers. In addition to the bibliometric results, this study visualized the main characteristics of AIoL research, such as the most frequently referenced keywords and research topics. The visualization results not only present a holistic picture of the field but also provide a structural understanding of the most influential relationships of AIoL research patterns.

To gain an in-depth understanding of the roles and trends of AIoL research, this study aimed to review relevant journal publications from the WoS (Web of Science) database. In this context, with the aims of providing a guide for new research, identifying trends in the field, and comparing existing research on the topic, the following research questions were addressed:

• What were the major countries/areas conducting AIoL research in 1997-2019?
• What/who were the leading journals, papers, and authors of AIoL in 1997-2019?
• What was the distribution of the main research areas of AIoL applications in 1997-2019?
• What were the research topics of AIoL research in 1997-2019?
• What were the roles of AIoL and adopted AI algorithms in AIoL in 1997-2019?

Method

Article Selection Process and Data Coding

The current study was based on the research purpose and referred to the literature reviews on AIoL by Shukla et al. (2019), Wong et al. (2019), and Zawacki-Richter et al. (2019). First, on October 31, 2020, we searched for papers published in the WOS database. Based on the searching list of Social Sciences Citation Index (SSCI) journals, until the end of 2019, there were 315 articles including AI (“artificial intelligence” or “machine intelligence” or “intelligent support” or “intelligent virtual reality” or “chat bot*” or “machine learning” or “automated tutor*” or “personal tutor*” or “intelligent agent*” or “expert system*” or “neural network*” or “natural language processing” or “chatbot*” or “intelligent system” or “intelligent tutor*”) AND online learning (“online learning” or “e-learning” or “Internet learning” or “web based learning” or “web learning” or “online training” or “e-training” or “Internet training” or “web based training” or “web training” or “massive open online course*” or “MOOC*” or “massively open online course” or “distance education” or “personal learning environment”) in the keywords list. In addition, excluding non-article types, 299 articles were retained, and were then reviewed manually according to the content of the articles (including topics and abstracts), excluding duplicates, non-English, literature reviews, and articles not related to AIoL topics. Finally, 224 articles were retained for bibliometric mapping analysis (Figure 1).

Following previous research, we focused on two dimensions of main interest to categorize the 224 articles. The coding scheme was as follows. First, according to the roles of AI mentioned in the collected articles, the four role types of AI were coded as follows: intelligent tutoring systems, profiling and prediction, assessment and evaluation, and adaptive systems and personalization. Next, on the basis of researchers’ frameworks (Chen et al., 2020), adopted AI algorithms were classified into 13 types: (a) Bayesian inferencing and networks, (b) evolutionary algorithms, (c) search and optimization, (d) fuzzy set theory, (e) knowledge elicitation methods via interviewing domain experts, (f) neural networks, (g) case-based reasoning, (h) natural language parsing, (i) ontology, (j) data mining, (k) statistical learning, (l) traditional machine learning approaches (including item response theory, linear regression, polynomial regression, Iterative Dichotomiser 3, support vector machine, classification, and clustering), and (m) mixed. Coding was performed by two researchers who read and classified the papers according to the coding scheme, and the coding results of the two researchers showed high consistency (kappa value = 0.89; Lavrakas, 2008).

Figure 1

Article Selection Process for Bibliometric Mapping Analysis and Content Analysis

Data Analysis

In this study, bibliometric mapping analysis was conducted by employing the VOSviewer software version 1.6.16 to identify the most frequently adopted keywords in AI-supported online learning studies as well as visualizing the citation and co-citation analysis results. The collected data were reviewed by three researchers who examined the descriptive statistics of the data, and discussed and interpreted the findings.

Data Distribution

Figure 2 shows the AIoL studies published from 1997 to 2019. Taking into account the fluctuations in technology, the published AIoL articles were categorized into three time periods, that is, 1997-2009, 2010-2014, and 2015-2019, based on the suggestions of Zawacki-Richter et al. (2019) and Zheng et al. (2016). It was found that there was an increasing trend in AIoL research from the first period (1997-2009, publications = 48) to the second period (2010-2014, publications = 73), and then to the most recent 5 years (2015-2019, publications = 103).

Figure 2

Number of Published Articles on AI in Online Learning Environments from 1997-2019

Note. N = 224.

Results

Analysis of Publication Trends and Country Distribution

Following previous review studies (Hwang & Tsai, 2001), we used the first author’s affiliation(s) as the measure to identify the country of origin at the time the article was published. Researchers have suggested that the first author playing the role of the main contributor in a research collaboration is a well-accepted practice in scientific publications, including AI-supported online learning research. Note that for the few first authors in this study who had two or more affiliations in different countries, the main affiliation was manually checked and counted. As shown in Table 2, the most productive countries (top 18) in the field were Taiwan, China, Spain, and the United States.

Table 2

Number of Articles and Rankings of the Most Productive Countries of the AI-Related Publications in the Context of Online Learning Environments

Note. N = 224.

Main Journals, Papers, and Authors

In this study, the 224 articles retrieved were published in 84 different journals. Figure 3 shows the five journals with the largest number of articles on AIoL published between 1997 and 2019. They are (a) Computers & Education, (b) Computers in Human Behavior, (c) Expert Systems With Applications, (d) Educational Technology & Society and (e) IEEE Transactions on Learning Technologies. In Figure 3, it is shown that IEEE Transactions on Learning Technologies began to publish AIoL articles only in 2010-2014. This is because the journal was founded in 2008, and a total of four articles were published in that time period. Figure 3 also indicates that the top five journals published 38.84% of AIoL articles. The publications in these five journals are generally highly cited, showing the potential of AIoL research.

Figure 3

Top Five Journals by Total Number of Publications from 1997 to 2019

Note. Only journals with 8 or more publications are included.

Furthermore, co-citation analysis and citation sources were also selected for the journal analysis. In this study, co-citation analysis and citation sources were chosen for creating a map of the most cited journals. The minimum number of citations from sources was adjusted to 20, and the number of sources to be selected was automatically displayed as 37. The created map is presented in Figure 4. It shows that the most-cited journals were Computers & Education (citations = 454), Expert Systems with Applications (citations = 244), Lecture Notes in Computer Science (citations = 180), Educational Technology & Society (citations = 103) and Computers in Human Behavior (citations = 94).

Figure 4

Most Cited Journals (Co-Citation Analysis)

Table 3 shows the five most-cited AIoL articles published between 1997 and 2019. Among the most frequently cited articles, many researchers focused on developing online-learning systems with personalized learning mechanisms to assist online learning, and on adaptively providing learning paths to facilitate individual learners’ learning performance. For example, Chen et al. (2005) proposed a personalized e-learning system based on item response theory (PEL-IRT), which considers course material difficulty and learner ability to provide personalized learning paths. García et al. (2007) adopted Bayesian networks to improve the precision of assessing students’ learning styles. At the same time, they pointed out that one of the most desirable features of a Web-based education system is that it is adaptable and personalized and can adjust the curriculum or provide assistance to students according to their needs.

Table 3

Five Most-Cited Papers

This study further analyzed those authors who had published five or more papers. In addition, considering the author’s influence, we selected authors who had been cited more than 50 times. As shown in Table 4, the most productive authors are Chih-Ming Chen, followed by Maria Virvou, and Analia Amandi. They are clearly active authors in the AIoL field.

Furthermore, co-citation analysis and cited authors were also selected. The minimum number of an author’s citations was set to 20 and the number of authors to be selected was automatically set at eight. The created map is shown in Figure 5. It illustrates that Peter Brusilovsky (citations = 74), Chih-Ming Chen (citations = 63), and Cristobal Romero (citations = 41) are the most cited (co-citation) authors in this field.

Table 4

Top Three Authors Ranked by Number of Publications

Note. Only authors with five or more publications are included in the table.

Figure 5

Most Cited Authors (Co-Citation Analysis)

Main Research Areas

Figure 6 presents the 10 most popular research areas, which predominantly fall into the fields of the social sciences and technology. It can also be seen in Figure 6 that most of the AIoL publications from 1997 to 2019 are in the research areas of education and educational research; others are in computer science and interdisciplinary applications, computer science and artificial intelligence, and engineering, electrical and electronic, etc.

Figure 6

Top 10 Research Areas by Total Number of Publications, 1997-2019

Figure 7 shows the distribution of the top 10 research areas of AIoL. The first AIoL research in 1997 came from two research areas, either education and educational research or computer science and interdisciplinary applications. In 1997-2009, AIoL was most frequently applied to education and educational research, followed by computer science and interdisciplinary applications. In 2010-2014, AIoL was applied most in education and educational research, in the order of computer science, interdisciplinary applications, computer science, artificial intelligence, engineering, electrical and electronic and operations research and management science. In 2015-2019, AIoL was used most in education and educational research, and then in computer science, interdisciplinary applications, engineering, electrical and electronic, computer science, artificial intelligence and computer science, information systems. From the above, it can be seen that in the second period, researchers from different fields paid more attention to the research issues related to AIoL.

Figure 7

Published Literature in the Top 10 Research Areas by Total Number of Publications, 1997-2019

In the top 10 research areas related to AIoL research, researchers tended to focus on somewhat different concerns. Figure 8 shows the top keywords and newer keywords for each research area. For example, the studies in education and educational research in 1997-2019 focus on how AI technologies were used in developing an interactive online environment to provide personalized supports to individual learners. This is revealed in the top 10 keywords, such as intelligent tutoring systems, e-learning, and interactive learning environments. Moreover, in recent five years (i.e., 2015-2019), the top 10 keywords in this research area show that researchers focused more on the roles of AI in online learning, such as dropout, personalization, and simulations.

Besides, it is found that those with technological background (e.g., Computer Science) are the major researchers who apply AI technologies in interactive online learning environments. It is also found that the development of AI-based online learning systems was the main focus of these studies. Moreover, the roles and the adoption of AI technologies in online learning are diverse (e.g., information extraction, simulations, e-learning tools, educational data mining, neural networks, decision support tools, and recommendation systems). With advances in technology and society’s development, some researchers have identified the skills needed to solve problems as one of the significant challenges in distance education.

Figure 8

Top Keywords and Newer Keywords Adopted in AIoL Research

The Popular Research Topics

The keywords provide essential information about the literature, and some researchers have pointed out that keyword analysis can help researchers clearly understand the trends in specific fields (Guo et al., 2016). In this study, VOSviewer software version 1.6.16 was used to perform a cluster analysis of keywords in order to understand the research issues related to the AIoL field. A total of 657 keywords were used in the AIoL research, and the network maps with keywords occurring with more than two frequencies are shown in Figure 9.

Figure 9

Network Map of Keyword Co-Occurrence on AIoL Research Over Time

To further examine the dynamic change in research topics, the author keywords of AIoL studies in individual time periods were analyzed, as shown in Figure 9 and Table 5, respectively. In the period 1997-2009, AIoL studies focused on the combination of ITS and distance learning; moreover, researchers started to provide support for learners by taking into account their learning status and needs, such as the development of personalized learning systems or ITS (Baylari & Montazer, 2009; Chen, 2009; Chen et al., 2005), algorithms to assess students’ learning styles based on their online learning behaviors (García et al., 2007), and models for dropout prediction (Lykourentzou et al., 2009).

In comparison with the publications in 1997-2009, the AIoL studies in 2010-2014 focused more on student-centered learning; that is, using AI technologies (e.g., natural language processing and educational data mining) to enable adaptive learning or personalized learning. For example, Lin et al. (2013) developed a personalized creativity learning system based on the data mining technique of decision trees to provide personalized learning paths for maximizing learner motivation and learning effects, optimizing the learner’s creativity performance.

In addition to the provision of online learning supports and the development of ITS, the studies in 2015-2019 focused more on investigating the interactions between learners and learning environments, as well as the provision of interactive learning interfaces in this regard, such as discussion forums, chatbots, educational games, recommendation systems, and decision support tools to facilitate learning outcomes, collaboration, and adaptive learning. For example, several studies aimed to provide personalized learning paths based on individual learners’ preferences and/or learning status (e.g., Kurilovas et al., 2015); some studies used AI technologies, such as artificial neural networks (ANNs), to identify learners’ problems and provide support or feedback by analyzing their conversation data (Hussain et al., 2019).

In addition, it was found that some keywords, although found in earlier time periods, began to be highlighted in this time. These included terms such as “deep learning,” “massive open online courses,” “big data,” “simulations,” “online learning,” “learning paths,” “experiments,” “adaptive e-learning,” “recommendation systems,” “natural language processing,” “machine learning,” “educational data mining and personalization,” “learning objects,” and “neural nets.”

Table 5

Frequency of Top Keywords by Year

Note. Only keywords used ≥ 7 times are included. ^a Initial search terms in this study.

Figure 10 shows that AIoL research themes form three clusters: (a) AI-supported personalized and collaborative online learning, (b) AI-facilitated online-learning management, and (c) development and evaluation of intelligent online-learning systems. These clusters are displayed in three colors: red, green, and blue. There is also a significant correlation between the keywords in each cluster. For example, the red cluster is associated with another keyword, indicating that the research topic is of interest to researchers in the AIoL field and relevant to other areas of AIoL research.

Figure 10

Map of the Structure of Research on AI in Online Learning Environments by Theme

Note. Cluster 1 = AI-supported personalized and collaborative online learning; Cluster 2 = AI-facilitated online-learning management; Cluster 3 = development and evaluation of intelligent online-learning systems.

The main essence of the red cluster (Cluster 1) is AI-supported personalized and collaborative online learning. This is the most important and largest theme cluster in terms of its centrality, overall weight, density, and degree of overlap with the other topics. A large number of its main terms relate to AI, personalization, learners, and/or online learning. The studies in Cluster 1 mainly focused on incorporating AI technologies into online learning to facilitate personalized, adaptive, and collaborative online learning (e.g., Anaya et al., 2013; Chen et al., 2005; Ortigosa et al., 2014; Samarakou et al., 2018). Such a trend has become clearer in recent years, and initiatives include using AI-supported personalized supports to help students complete complex learning tasks (Schiaffino et al., 2008) as well as guide them to make learning plans, examine their own learning status, make reflections, and adjust the plans to promote learning performance (Chen, 2009; Romero et al., 2013; Romero et al., 2019).

Figure 10 also illustrates the strength of the links between the nodes of the keywords in the green cluster (Cluster 2). Thus, these keywords are closely related and form the second theme group in AIoL. The core terms reflect lines of research related to artificial intelligence, online learning, and education. This thematic group focuses on AI-facilitated online-learning management. The outstanding effect of these terms highlights the importance of these concepts in the AIoL framework. The studies in Cluster 2 focused more on using AI technologies to cope with online-learning management problems, such as the provision of automatic evaluation, the prediction of dropout rate or probability, the analysis of learning engagement status using machine learning, natural language processing, educational data mining, support vector machines, and deep learning approaches (e.g., Wise et al., 2017; Xing & Du, 2019; Xing et al., 2019).

The blue cluster (Cluster 3) focuses on the development and evaluation of intelligent online-learning systems. The core terms of this cluster reflect research interests related to ITS, the development and design of human-computer interfaces and interactive learning environments, and teaching/learning strategies. In these terms, intelligent tutoring systems, interactive learning environments, distance education and telelearning, teaching/learning strategies, and human-computer interface stand out above the rest. This demonstrates the importance of this group’s terms throughout the study period. Cluster 3 mainly focuses on the development and evaluation of ITS; for example, Virvou and Alepis (2005) developed an ITS to assist teachers in monitoring students’ learning performances by recording and diagnosing individual students’ learning processes (e.g., learning logs and test scores).

Roles of AI and Adopted AI Algorithms

Figure 11 shows the roles of AI in the AIoL research. It was found that “intelligent tutoring systems” (55.80%) played the most frequent roles, followed by “profiling and prediction” (20.54%) and “adaptive systems and personalization” (18.30%). Except for “intelligent tutoring systems,” the number of AI roles grew from each period.

Figure 11

Roles of AIoL Research in Each Period

Figure 12 shows the adopted AI algorithms. It was found that most studies adopted two or more AI algorithms (22.77%), followed by knowledge elicitation methods via interviewing domain experts (16.96%), ontology (14.29%), neural networks (12.95%), natural language parsing (9.38%), evolutionary algorithms (4.46%), fuzzy set theory (4.46%), data mining (4.46%), statistical learning (4.46%), traditional machine learning approaches (4.46%), and Bayesian inferencing and networks (1.34%). Search and optimization and case-based reasoning were not adopted in any of the studies. The studies which adopted two or more AI algorithms generally aimed to build models to predict students’ learning behaviors or performance (e.g., Hussain et al., 2019; Xing et al., 2019). Knowledge elicitation methods via interviewing domain experts were also related to the development of intelligent learning systems for guiding and assessing learners’ learning status, providing them with feedback (e.g., de la Peña Esteban et al., 2019). Studies have employed ontology to develop prototypes of e-learning systems and recommender systems as a basis for guiding performance-oriented learning, and to help learners understand through the concepts of ontologies of topics (e.g., Capuano et al., 2012; Romero et al., 2019). From the analysis results, the AIoL research not only focuses on developing and evaluating the effectiveness of ITS or recommendation systems in education, but also highlights the usefulness of AI-related technologies for online learning and teaching. For example, Wang et al. (2018) presented a semantic analysis model to track learners’ emotional tendencies, and through emotion quantification and machine learning calculations could predict real-time graduation probability for different learning stages. Xing et al. (2019) adopted an integrated framework of achievement emotions to analyze discussion forum posts to explore the impact of achievement emotions on students’ continued participation in MOOCs. Zhou et al. (2019) employed an end-to-end algorithm of deep learning in machine learning to analyze learners’ confusion in educational games, and obtained 91.04% precision. In online learning settings, Scholten et al. (2019) pointed out that chatbots could significantly impact learners during the learning process compared to mere textual guidance.

Figure 12

Adopted Technologies of AIoL Research in Each Period

Conclusion, Discussion, and Suggestions

Owing to the progress and popularity of computer and network technologies, online learning environments have significantly changed in the past decades. Building on the work of Shukla et al. (2019), Wong et al. (2019), and Zawacki-Richter et al. (2019), this study analyzed the AIoL journal articles published from 1997 to 2019 using the bibliometric mapping analysis approach by categorizing the publications into three time periods. Based on our analysis, we found five findings merited further discussion.

Productive Countries

The finding related to most productive countries differed slightly from what has been reported in several review studies of AI in education (AIEd); for example, Zawacki-Richter et al. (2019) reported that the most productive countries in AIEd research for higher education were the United States, followed by China and Taiwan. This implies that Asian researchers are more focused on applying AI technologies to online learning environments than to other learning contexts. More importantly, the researchers from these countries/regions (e.g., Taiwan and China) are not native speakers of English. The quality and productive research output could be due to government policies promoting online learning. For example, Taiwan’s government conducted a nationwide program from 2003 to 2012 to promote e-learning, which significantly encouraged Taiwan’s researchers to focus on e-learning studies (Hwang & Tsai, 2011; Tsai et al., 2010).

AIoL Studies Accepted in Top-Ranking Journals

The largest number of AIoL studies was published in Computers & Education, followed by Computers in Human Behavior and Expert Systems with Applications. These journals are well recognized as top-ranking journals in the fields of educational technology, experimental psychology, and computer science, respectively. This implies that AIoL studies have gained widespread acceptance among scholars in different fields. This is consistent with the analysis results of AIoL research areas.

Government Programs May Promote Research

Based on the productivity and citation analysis, it was found that the most influential author of AIoL research is Chih-Ming Chen, a researcher from Taiwan, who has published 8 articles with 709 citations. This output could be due to the nationwide e-learning promotion program initiated by Taiwan’s government, which encourages researchers, more than in other countries/regions, to focus on e-learning studies (Tsai et al., 2010).

Three Clusters of Research Topics

In term of research topics, AIol studies can be categorized into three clusters, that is, “AI-supported personalized and collaborative online-learning,” “AI-facilitated online-learning management,” and “development and evaluation of intelligent online-learning systems.” This finding provides a good reference for researchers hoping to design future AIoL studies.

Roles of AI and Adopted AI Algorithms

“Intelligent tutoring systems” have played the most important role in AIoL, followed by “profiling and prediction,” and “adaptive systems with personalization.” This is consistent with the findings of previous studies regarding research issues (Hwang et al., 2020; Zawacki-Richter et al., 2019). Most studies adopted two or more AI algorithms to develop models of learners’ online learning to predict their learning performance, followed by “knowledge elicitation methods via interviewing domain experts,” while search and optimization and case-based reasoning are less commonly employed. From the results of keyword analysis and AI technology adoption surveys, it was found that the application of AI-related algorithms, as well as discussion forums, chatbots, and educational games, showed a significant growth trend (Scholten et al., 2019; Xing et al., 2019; Zhou et al., 2019).

In summary, this study not only reveals the increasing emphasis on AIoL research, but also reports several important trends in this field. Based on the findings of this study, several suggestions for designing AIoL research as well as developing policies to promote AIoL in the future are presented.

1. It is recommended that educational research institutions in various countries follow the practice of some leading countries and regions to promote AIoL through large-scale and long-term programs.
2. In addition to domains in which AIoL research is frequently applied, researchers are encouraged to apply the AIoL approach to less often studied domains such as architecture, economics and management, as well as cross-disciplinary applications.
3. It could also be interesting to extend the AIoL approach to blended learning contexts or ubiquitous learning contexts to investigate the impacts of this approach on integrated real-world and virtual-world learning.
4. One important and often overlooked research topic is the use of AI technologies to optimize learners’ online learning experiences (e.g., information literacy, curiosity, emotions, etc.) to increase their learning engagement and reduce dropout rates.
5. Further research is needed into the role that AIoL could play in providing personalized learning guidance and supports as well as learning paths.
6. It is crucial to investigate students’ higher order thinking and behavioral patterns, and not only their learning achievements in AIoL contexts.

References

Anaya, A. R., Luque, M., & García-Saiz, T. (2013). Recommender system in collaborative learning environment using an influence diagram. Expert Systems with Applications, 40(18), 7193-7202. https://doi.org/10.1016/j.eswa.2013.07.030

Baylari, A., & Montazer, G. A. (2009). Design a personalized e-learning system based on item response theory and artificial neural network approach. Expert Systems with Applications, 36(4), 8013-8021. https://doi.org/10.1016/j.eswa.2008.10.080

Capuano, N., Mangione, G. R., Pierri, A., & Salerno, S. (2012). Learning goals recommendation for self regulated learning. International Journal of Engineering Education, 28(6), 1373-1379. https://www.researchgate.net/profile/Nicola-Capuano/publication/281116341_Learning_Goals_Recommendation_for_Self_Regulated_Learning/links/55d701e308ae9d65948c204e/Learning-Goals-Recommendation-for-Self-Regulated-Learning.pdf

Carin, L. (2020). On artificial intelligence and deep learning within medical education. Academic Medicine, 95(11S), S10-S11. https://doi.org/10.1097/ACM.0000000000003630

Chen, C.-M. (2009). Personalized e-learning system with self-regulated learning assisted mechanisms for promoting learning performance. Expert Systems with Applications, 36(5), 8816-8829. https://doi.org/10.1016/j.eswa.2008.11.026

Chen, C.-M., & Lain, Y.-Y. (2020). Developing a computer-mediated communication competence forecasting model based on learning behavior features. Computers & Education: Artificial Intelligence, 1, Article 100004. https://doi.org/10.1016/j.caeai.2020.100004

Chen, C.-M., Lee, H.-M., & Chen, Y.-H. (2005). Personalized e-learning system using item response theory. Computers & Education, 44(3), 237-255. https://doi.org/10.1016/j.compedu.2004.01.006

Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020). Application and theory gaps during the rise of Artificial Intelligence in Education. Computers & Education: Artificial Intelligence, 1, Article 100002. https://doi.org/10.1016/j.caeai.2020.100002

de la Peña Esteban, F. D., Lara Torralbo, J. A., Lizcano Casas, D., & Martínez Rey, M. A. (2019). Expert system for problem solving in distance university education: The successful case of the subject “operations management”. Expert Systems, 36(5), Article e12444. https://doi.org/10.1111/exsy.12444

García, P., Amandi, A., Schiaffino, S., & Campo, M. (2007). Evaluating Bayesian networks’ precision for detecting students’ learning styles. Computers & Education, 49(3), 794-808. https://doi.org/10.1016/j.compedu.2005.11.017

Guo, L., Xu, F., Feng, Z., & Zhang, G. (2016). A bibliometric analysis of oyster research from 1991 to 2014. Aquaculture International, 24(1), 327-344. https://doi.org/10.1007/s10499-015-9928-1

Han, E.-R., Yeo, S., Kim, M.-J., Lee, Y. H., Park, K.-H., & Roh, H. (2019). Medical education trends for future physicians in the era of advanced technology and artificial intelligence: An integrative review. BMC Medical Education, 19(1), 1-15. https://doi.org/10.1186/s12909-019-1891-5

Hussain, M., Zhu, W., Zhang, W., Abidi, S. M. R., & Ali, S. (2019). Using machine learning to predict student difficulties from learning session data. Artificial Intelligence Review, 52(1), 381-407. https://doi.org/10.1007/s10462-018-9620-8

Hwang, G.-J., & Tsai, C.-C.. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010. British Journal of Educational Technology, 42(4), E65-E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of Artificial Intelligence in Education. Computers & Education: Artificial Intelligence, 1, Article 100001. https://doi.org/10.1016/j.caeai.2020.100001

Khanal, S. S., Prasad, P. W. C., Alsadoon, A., & Maag, A. (2020). A systematic review: Machine learning based recommendation systems for e-learning. Education and Information Technologies, 25, 2635-2664. https://doi.org/10.1007/s10639-019-10063-9

Kurilovas, E., Zilinskiene, I., & Dagiene, V. (2015). Recommending suitable learning paths according to learners’ preferences: Experimental research results. Computers in Human Behavior, 51, 945-951. https://doi.org/10.1016/j.chb.2014.10.027

Lavrakas, P. J. (2008). Encyclopedia of survey research methods. Sage.

Lee, D., Watson, S. L., & Watson, W. R. (2020). The influence of successful MOOC learners’ self-regulated learning strategies, self-efficacy, and task value on their perceived effectiveness of a massive open online course. The International Review of Research in Open and Distributed Learning, 21(3), 81-98. https://doi.org/10.19173/irrodl.v21i3.4642

Lee, Y., & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59(5), 593-618. https://doi.org/10.1007/s11423-010-9177-y

Lin, C. F., Yeh, Y.-C., Hung, Y. H., & Chang, R. I. (2013). Data mining for providing a personalized learning path in creativity: An application of decision trees. Computers & Education, 68, 199-210. https://doi.org/10.1016/j.compedu.2013.05.009

Lin, P.-H., Wooders, A., Wang, J. T.-Y., & Yuan, W. M. (2018). Artificial intelligence, the missing piece of online education? IEEE Engineering Management Review, 46(3), 25-28. https://doi.org/10.1109/EMR.2018.2868068

Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950-965. https://doi.org/10.1016/j.compedu.2009.05.010

Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers & Education, 159, Article 104009. https://doi.org/10.1016/j.compedu.2020.104009

Ortigosa, A., Martín, J. M., & Carro, R. M. (2014). Sentiment analysis in Facebook and its application to e-learning. Computers in Human Behavior, 31, 527-541. https://doi.org/10.1016/j.chb.2013.05.024

Romero, C., Espejo, P. G., Zafra, A., Romero, J. R., & Ventura, S. (2013). Web usage mining for predicting final marks of students that use Moodle courses. Computer Applications in Engineering Education, 21(1), 135-146. https://doi.org/10.1002/cae.20456

Romero, L., Saucedo, C., Caliusco, M. L., & Gutiérrez, M. (2019). Supporting self-regulated learning and personalization using ePortfolios: A semantic approach based on learning paths. International Journal of Educational Technology in Higher Education, 16(1), Article 16. https://doi.org/10.1186/s41239-019-0146-1

Samarakou, M., Tsaganou, G., & Papadakis, A. (2018). An e-learning system for extracting text comprehension and learning style characteristics. Educational Technology & Society, 21(1), 126-136. https://www.jstor.org/stable/26273874

Schiaffino, S., Garcia, P., & Amandi, A. (2008). eTeacher: Providing personalized assistance to e-learning students. Computers & Education, 51(4), 1744-1754. https://doi.org/10.1016/j.compedu.2008.05.008

Scholten, M. R., Kelders, S. M., & Van Gemert-Pijnen, J. E. W. C. (2019). An empirical study of a pedagogical agent as an adjunct to an eHealth self-management intervention: What modalities does it need to successfully support and motivate users?. Frontiers in Psychology, 10, 1063. https://doi.org/10.3389/fpsyg.2019.01063

Shukla, A. K., Janmaijaya, M., Abraham, A., & Muhuri, P. K. (2019). Engineering applications of artificial intelligence: A bibliometric analysis of 30 years (1988-2018). Engineering Applications of Artificial Intelligence, 85, 517-532. https://doi.org/10.1016/j.engappai.2019.06.010

Tang, K.-Y., Chang, C. Y., & Hwang, G.-J. (2021). Trends in artificial intelligence-supported e-learning: A systematic review and co-citation network analysis (1998-2019). Interactive Learning Environments, 1-19. https://doi.org/10.1080/10494820.2021.1875001

Tsai, C.-C., Chen, N.-S., & Chen, G.-D. (2010). The current status and future of e-learning in Taiwan. Innovations in Education and Teaching International, 47(1), 5-7. https://doi.org/10.1080/14703290903525879

Virvou, M., & Alepis, E. (2005). Mobile educational features in authoring tools for personalised tutoring. Computers & Education, 44(1), 53-68. https://doi.org/10.1016/j.compedu.2003.12.020

Wang, L., Hu, G., & Zhou, T. (2018). Semantic analysis of learners’ emotional tendencies on online MOOC education. Sustainability, 10(6), Article 1921. https://doi.org/10.3390/su10061921

Wise, A. F., Cui, Y., Jin, W., & Vytasek, J. (2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32, 11-28. https://doi.org/10.1016/j.iheduc.2016.08.001

Wong, J., Baars, M., Davis, D., Van Der Zee, T., Houben, G.-J., & Paas, F. (2019). Supporting self-regulated learning in online learning environments and MOOCs: A systematic review. International Journal of Human-Computer Interaction, 35(4-5), 356-373. https://doi.org/10.1080/10447318.2018.1543084

Xing, W., & Du, D. (2019). Dropout prediction in MOOCs: Using deep learning for personalized intervention. Journal of Educational Computing Research, 57(3), 547-570. https://doi.org/10.1177/0735633118757015

Xing, W., Tang, H., & Pei, B. (2019). Beyond positive and negative emotions: Looking into the role of achievement emotions in discussion forums of MOOCs. The Internet and Higher Education, 43, Article 100690. https://doi.org/10.1016/j.iheduc.2019.100690

Yu, H., Miao, C., Leung, C., & White, T. J. (2017). Towards AI-powered personalization in MOOC learning. npj Science of Learning, 2(1), Article 15. https://doi.org/10.1038/s41539-017-0016-3

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education-Where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0

Zheng, P., Liang, X., Huang, G., & Liu, X. (2016). Mapping the field of communication technology research in Asia: Content analysis and text mining of SSCI journal articles 1995-2014. Asian Journal of Communication, 26(6), 511-531. https://doi.org/10.1080/01292986.2016.1231210

Zhou, Y., Xu, T., Li, S., & Shi, R. (2019). Beyond engagement: An EEG-based methodology for assessing user’s confusion in an educational game. Universal Access in the Information Society, 18(3), 551-563. https://doi.org/10.1007/s10209-019-00678-7

AI in Online-Learning Research: Visualizing and Interpreting the Journal Publications from 1997 to 2019 by Gwo-Jen Hwang, Yun-Fang Tu, and Kai-Yu Tang is licensed under a Creative Commons Attribution 4.0 International License.