International Review of Research in Open and Distributed Learning MOOCs as a Research Agenda: Changes Over Time

MOOCs (massive open online courses) have attracted considerable attention from researchers. Fueled by constant change and developments in educational technology, the trends of MOOCs have varied greatly over the years. To detect and visualize the developments and changes in MOOC research, 4,652 articles published between 2009 and 2021 were retrieved from Web of Science and Scopus with the aid of CiteSpace. This study sought to explore the number of publications, co-citation network, cluster analysis, timeline analysis, burstness analysis


Article abstract
MOOCs (massive open online courses) have attracted considerable attention from researchers. Fueled by constant change and developments in educational technology, the trends of MOOCs have varied greatly over the years. To detect and visualize the developments and changes in MOOC research, 4,652 articles published between 2009 and 2021 were retrieved from Web of Science and Scopus with the aid of CiteSpace. This study sought to explore the number of publications, co-citation network, cluster analysis, timeline analysis, burstness analysis, and dual-map overlays based on co-citation relationships. The first finding was that the number of publications on MOOCs had increased consistently, and grew especially quickly between 2013 and 2015. Second, the main topic of the top 10 co-cited studies revolved around the problem of learner continuance. Third, blended programs, task-technology fit, and comparative analysis have emerged as popular subjects. Fourth, the development of MOOC research has followed distinct phases, with 2009 to 2012 the starting phase, 2013 to 2015 the high growth phase, 2016 to 2018 the plateau phase, and 2019 to 2021 another peak phase. Lastly, both cluster analysis and dual-map overlays provided empirical evidence of cross-disciplinary research. Our findings provided an in-depth and dynamic understanding of the development and evolution of MOOC research and also proposed novel ideas for future studies.

MOOCs as a Research Agenda: Changes Over Time
Around the world, the education system is changing with the rapid development of information and communication technologies (Hoy, 2014). Massive open online courses (MOOCs) are a product of the development of educational technology. Providers in the US (edX, Coursera, Udacity, Kahn Academy, Udemy), China (Icourse, XuetangX, Zhihuishu), UK (FutureLearn), Europe (Iversity, FUN), Korea (K-MOOC), Middle East (Rwaq, Edraak), and Australia (Open2study) have transformed and upgraded traditional education by providing a more flexible, equitable, and cost-efficient approach (Brahimi & Sarirete, 2015). The growth of MOOCs has attracted many researchers, and the MOOC approach continues to be a popular research topic.
Several studies have looked into the fundamental qualities of MOOCs such as the platforms, courses, students, and teachers. For example, the issue of MOOC design has been raised since curriculum quality, determined by course design, plays an important role in teaching (Jordens & Zepke, 2009). Based on their examination of the teaching environment and characteristics of online platforms, Guàrdia et al. (2013) presented some principles for the design of MOOCs. These principles focused on empowering learners, and encouraged them to develop learning plans, engage in collaborative work, and express diverse opinions, thereby having a profound effect on the design of MOOCs.
An interdisciplinary trend integrating MOOCs into other fields has continued to develop (Wahid et al., 2020). For example, Zhou (2016) raised concerns about students' acceptance in MOOCs, using psychological theories such as the theory of planned behavior (Ajzen, 1985) and self-determination theory (Ryan & Deci, 2000). The combination of public media and MOOCs is another field widely discussed by scholars (Bulfin et al., 2014;Rowan & Hartnett, 2019). For example, Kovanović et al. (2015) focused on the public discourse of MOOCs and retrieved major topics and themes from news reports; they found that the main topics discussed in the media have changed over the years. Interaction is also an important aspect of MOOC research. Past studies have examined interactions between teachers and students, among students, and between humans and computers (Sunar et al., 2016).
Systematic investigations of MOOCs have been fruitful. The first systematic review of MOOCs was by Liyanagunawardena et al. (2013). They retrieved the literature published from 2008 to 2012 and analyzed the development of MOOCs during that period. Similarly, Zheng et al. (2019) explored researchers' interests and the development of MOOCs from 2012 to 2018 by examining the network of citations and overlapping keywords. A more comprehensive review by Wahid et al. (2020) examined 3,118 studies extracted from WOS (Web of Science) and Scopus databases, and analyzed the papers' sources, titles, fields of study, and keywords. They also explored the co-authorship, co-citation, and co-keywords relationships with VOSviewer.
Our study differed from previous studies in several respects. First, with more papers published on MOOCs, our study contained more comprehensive data, showing a bigger picture of MOOCs as a global movement. Furthermore, in contrast to the static analysis of MOOCs research profiles only, our research put effort into tracking its development over time. Moreover, rather than understanding developments and emerging trends through keywords, our research focused on co-citation relationships, an effective way of identifying 195 complex networks and studying their evolution (Callon et al., 1983). Exploring the dynamics of emerging trends, developments, innovation points, and topics of a particular knowledge domain in a given period has become essential to researchers in an ever-changing society (Chen & Liu, 2020). To delve deeply into the developments in MOOCs between 2009 and 2021, this study focused on the number of publications, cocitation network, cluster analysis, timelines (Chen, 2017), burstness (Chen et al., 2012), and dual-map overlays (Chen & Leydesdorff, 2014) based on co-citation relationships and using CiteSpace.

Data Collection
WOS and Scopus provided users with prompt access to information from various literature databases (Mongeon & Paul-Hus, 2016). They were also considered indispensable tools for bibliometric analysis (Meho & Yang, 2006). WOS contained more papers on natural sciences and engineering while Scopus provided more coverage of social sciences and humanities (Mongeon & Paul-Hus, 2016). Combining WOS and Scopus made it possible to obtain complete data. The data used in this study were retrieved on Oct 27, 2021 from WOS and Scopus using the following retrieval formula: • Document types: Articles, review articles, and early access in Science WOS; conference papers and articles in Scopus.
• Language: English. Literature related to MOOCs has been written in many languages, but only papers written in English were collected in this study.
In the end, 4,256 articles were extracted from Scopus and 1,599 articles from WOS. Since there was some overlap between the two databases, CiteSpace was used to filter and reduce the duplication of the literature; 623 articles were discarded. Finally, 4,652 records, including 3,129 Scopus records and 1,523 WOS records, were used in the study.

Research Method
Scientometrics is a powerful method for analyzing bibliometric networks (Van Eck & Waltman, 2014). Among various scientometrics analysis software, CiteSpace can effectively extract citations data; thus, it has been widely used by researchers (Synnestvedt et al., 2005). Compared with other science mapping tools, CiteSpace was designed with strong functionalities to detect and interpret new developments and emerging trends from research disciplines through clear visualized network diagrams (Hou et al., 2018). Therefore, CiteSpace (5.8.R3) was selected for this study.

Co-Citation Network
Co-citation analysis has become the leading tool for empirically studying the structure of scientific communication (Gmür, 2003). Therefore, we first did a co-citation network analysis to find the structure of MOOC research, as shown in Table 2 and Figure 1.
197 Table 2 Top 10 Co-Citation References Network of Top 10 Co-Citation References (2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018)(2019)(2020)(2021)(2022) We obtained 864 nodes and 1,860 links. Figure 2 shows the references whose co-citation count was beyond 70. We chose the 10 references cited most frequently as our research points (Table 2). It was clear that the bibliometrics that included the data from 2008 to 2012 (Liyanagunawardena et al., 2013) and the exploration of the first MOOC in edX (Breslow et al., 2013) tied for first place. The third (Littlejohn et al., 2016) and the fourth (Hew & Cheung, 2014) were related to the motivations of using MOOCs. Hone and El Said (2016), Kizilcec et al. (2013), and Alraimi et al. (2015) ranked fifth, sixth, and seventh respectively; all three explored the factors that influenced low MOOC completion rates. Instructional design quality was explored by Margaryan et al. (2015) and it was ranked eighth in frequency. Both of the last two, Pappano (2012) and Daniel (2012), described MOOCs in general; the former focused more on the attraction of MOOCs, while the latter shed objective light on dropout rates.

Cluster Analysis
The first step in exploring a knowledge domain is to identify highly cited documents using co-citation analysis; the second step is to analyze documents to determine the key research domain (Shi & Liu, 2019). In order to understand the topics of MOOC research, we conducted cluster analysis based on the co-citation network. The results are shown in Table 3 and Figure 2.
199 Table 3 Top  Figure 2 illustrates the network diagram of citation references with labels.

Figure 2
Top Nine Clusters Based on Co-Citation Network (2009-2021 200 In total, 86 clusters, including 840 nodes and 1,146 links, were formed. We analyzed the largest 10 clusters in more detail (Table 3). In Figure 2, the 10 clusters are displayed in the network with corresponding numbers: #0 to #9 indicate different cluster sizes, with 0 representing the largest size and 9 representing the smallest. Accordingly, #0 discussion forum participation was the core topic in the MOOC research field, followed by #1 blended program. Then, #2 educational data science, attracted many scholars and ranked third. The fourth, called #3 task-technology fit, and #4 recent development, indicated that technical advancements made significant contributions to the development of MOOCs. Topics such as #5 anatomy massive open online course, #6 digital culture, and #7 comparative analysis represented emerging frontiers and hotspots in MOOC research, and ranked sixth, seventh, and eighth, respectively. Finally, #8 using deep learning was ranked ninth, followed by #9 massive open online courses.

Timeline Analysis
Timeline analysis displays the labels of clusters year by year, with appearance and disappearance times, thereby vividly showing dynamic changes (Chen, 2017). To reveal the gradual changes of MOOC research topics, we employed the year-by-year clustering function of CiteSpace to draw the timeline map based on co-citation relationships in Figure 3.

Burstness
Citation burstness is defined as an index indicating the frequency with which a particular reference is cited within different periods (Chen et al., 2012). Specifically, the higher the citation rate, the greater scholars' attention to the research topic. Detecting the burstness of literature has been regarded as an important way to explore the research frontier of a particular field in a specific period (Hou et al., 2018). In this study, we also employed the function of burstness in CiteSpace to explore the citation bursts of references (see Table   4).

202
As a result, 243 burst items were found. We analyzed 20 references with the strongest citation bursts. Table  4 presents detailed information-first appearance, beginning year, ending year, strength, and timeline-for the top 20 citations from 2009 to 2021. In terms of the continuity of burst time, Kop (2011) was the citation with the longest burst time, lasting five years in total, followed by Pappano (2012) and Breslow et al. (2013), which each lasted four years. Furthermore, 11 of the 20 burst citations began predominantly in 2013 and 2014. In 2019 there were significant changes in the MOOC research field; five bursts emerged in this period, a development that warrants more attention as we continue to explore the MOOC research frontiers further. As for the periods where significant changes occurred, there were many bursts in 2013 to 2014. On the other hand, there were only slight changes in 2015 and 2018, while 2016 and 2018 were blank years, without any citations. Nevertheless, significant changes and research developments were achieved in 2019 to 2020.

Dual-Map Overlays
Dual-map overlays enabled us to complete several novel visual analytic tasks that were previously impossible to perform intuitively. By tracing the citation arcs with concentrated landing zones from the origin branch, it was simple to determine whether a set of publications integrated prior work from multiple disciplines (Chen & Leydesdorff, 2014). We drew the journal dual-map overlays, shown in Figure 4, to reveal the distribution of the knowledge base of MOOC research.

Figure 4
A Dual-Overlay Network of MOOC Research Based on JCR Journal Map (2009-2021 Note: A higher resolution version is available. Figure 4 maps the subject areas of MOOC research. Each node represents the research subject classified by CiteSpace, based on WOS and Scopus classifications. The left side depicts the base map of citing journals, while the right is the base map of cited journals. The colorful lines between the two maps represent the cited 203 relationships of the target and source. As shown in Figure 4, more citing papers were published in the psychology, education, health area. Moreover, more cited papers were published in the psychology, education, social area. In addition, the subjects of mathematics, systems, mathematical gathered significant literature from citing journals and systems, computing, computer from cited journals.

Number of Publications
Although the overall number of publications for 2021 was incomplete at the time of the study, the increase in yearly publications of MOOC research from 2009 to 2021 was clear. With more students engaged in MOOCs, more attention to them has come from government, educators, and commercial institutions (Yuan & Powell, 2013). The years 2013, 2014, and 2015 were noteworthy breakout years with a clear surge, probably driven by the promotion of MOOC platforms including edX, Coursera, and Udacity in the United States (Schuwer et al., 2015).

Co-Citation Analysis
As for the top 10 papers based on the co-citation analysis, in addition to some fundamental explanation of MOOCs, the focus of MOOC research has been more on students' continuance using MOOCs. Although Daniels (2012) first raised the problem of low completion rates in 2012, the lack of data prevented researchers from doing further studies at that time (Liyanagunawardena et al., 2013). With the publication of Breslow et al. (2013), significant progress was made, as their research was based on the data collected from Circuits and Electronics (6.002x)-the first MOOC developed by edX-and it explored students' learning situations and use of resources (Breslow et al., 2013). Later, studies on the issues of (a) motivation and self-regulated learning (Littlejohn et al., 2016); (b) students and instructors (Hew & Cheung, 2014); (c) retention of students (Hone & El Said, 2016); (d) learner subpopulations (Kizilcec et al., 2013); openness and reputation (Alraimi et al., 2015) tracked the problem of continued use of MOOCs.

Cluster Analysis
The result of cluster analysis showed the interdisciplinary nature of MOOC research. As seen from #3 tasktechnology fit, the task technology fit (TTF) model has been applied in educational studies since 2017 (Wu & Chen, 2017). It provided further evidence of interdisciplinary MOOC research since the TTF model was first introduced in the information science field (Lee et al., 2003). Furthermore, keywords like blended, data science, task-technology-fit, digital, and deep learning were further evidence of the interdisciplinary nature of MOOC research. Our findings were consistent with Veletsianos and Shepherdson (2015), and indicated that the scientific complexity of MOOC research was being addressed by researchers from diverse backgrounds. In particular, the interdisciplinary approach between education and computer science was revealed as the most remarkable.

204
Timeline analysis revealed substantial changes in the topics of MOOC research, which means that researchers in each period had different interests. Despite that, #0 discussion forum participation, attracted scholars' interest early on and has been labeled the hottest research topic in the past. More recently, #1 blended program, #3 task-technology fit, and #7 comparative analysis have emerged as popular subjects.
In terms of #1 blended program, MOOCs have often been viewed as an important supplement to traditional in-class learning (Zhang, 2016). Therefore, as of the 2020s, blended learning has become one of the key topics of interest for researchers.

Burstness Analysis
Burstness also changed year by year. By tracking the beginning year of burstness, it was easy to find some key turning points in the evolution of MOOCs. Before 2012, only one burstness occurred even though five papers had focused primarily on the concept, history, challenges, and trends of MOOCs (Liyanagunawardena et al., 2013). The years 2013 to 2015 were the period with the most rapid development of MOOCs, with 11 bursts appearing. Progress in MOOCs was relatively flat from 2016 to 2018, with only two bursts appearing. In contrast, 2019 was another peak period with five prominent points. Therefore, it was evident that 2009 to 2012 was the initial phase as MOOCs were being proposed, followed by the rapid development phase from 2013 to 2015. The period 2016 to 2018 was a relatively stable phase, with a new peak after 2019.

Dual-Map Overlays Analysis
Dual-map overlays analysis demonstrated that the MOOC research field has encompassed much crossdisciplinary knowledge. This is consistent with the result of cluster analysis, which also illustrated the strong cross-domain nature of MOOCs. The results of the dual-map overlay indicated that papers dealing with MOOCs appeared more often in journals related to education and society, in line with the original purpose of MOOCs, namely to realize UNESCO's goal of open and accessible education (Wahid et al., 2020). However, it is worth noting that both citing and cited journals represented almost all journal types. With the worldwide COVID-19 pandemic, open and massive video lectures from the world's best professors and the most reputable universities (Wu & Chen, 2017) in MOOCs have been regarded as an alternative way to offline classes, thus arousing the interest of many researchers from various perspectives (Bhattacharya et al., 2020).

Conclusion
This study conducted a bibliometric analysis of MOOC publications based on 4,652 items extracted from WOS and Scopus. By mapping the number of publications, co-citation network, clusters, timelines, burstness, and dual-map overlays of MOOC research, we concluded the following. First, the number of MOOC research publications has grown consistently, particularly between 2013 and 2015, with explosive growth. As for the top 10 co-citation articles, their main topics revolved around the problem of MOOC participation continuance. Among all the labels, #1 blended program, #3 task-technology fit, and #7 comparative analysis were the emerging and popular subjects. Regarding analysis of burstness, the development of MOOCs showed clear phases, with 2009 to 2012 the starting phase, 2013 to 2015 the high