What Research Says About MOOCs – An Explorative Content Analysis

Since the first offering of a Massive Open Online Course (MOOC) in 2008, the body of literature on this new phenomenon of open learning has grown tremendously. In this regard, this article intends to identify and map patterns in research on MOOCs by reviewing 362 empirical articles published in peer-reviewed journals from 2008 to 2015. For the purposes of this study, a text-mining tool was used to analyse the content of the published research journal articles and to reveal the major themes and concepts covered in the publications. The findings reveal that the MOOC literature generally focuses on four lines of research: (a) the potential and challenges of MOOCs for universities; (b) MOOC platforms; (c) learners and content in MOOCs; and (d) the quality of MOOCs and instructional design issues. Prospective researchers may use these results to gain an overview of this emerging field, as well as to explore potential research directions.

Other papers have investigated MOOCs in the fields of broadcasting and social media (Bulfin, Pangrazio, & Selwyn, 2014;Deimann, 2015;Kovanovic, Joksimovic, Gasevic, Siemens, & Hatala, 2015;Shen & Kuo, 2015), taking a closer look at the phenomenon by focusing on discourses and sentiments on MOOCs, as well as identifying influencers in broadcasting and social media. Finally, some papers narrowed their scope in analysing MOOC research. For instance, Ossiannilsson, Altinay, and Altinay (2016)  He noted that MOOCs provide many opportunities for learners, faculty members, universities, and MOOC providers. On the other hand, he also identified some challenges that MOOCs need to overcome, such as questionable course quality, high dropout rates, unavailable course credits, ineffective assessments, complex copyright issues, and the lack of necessary hardware required to join

MOOCs.
Whilst previous bibliographic studies, literature reviews, and content analyses looked at theoretical, methodological, and pedagogical approaches, or specific aspects of MOOC research (e.g., quality or learner's perceptions), our study aims to provide an overview of the overall structure of themes and topics of research into MOOCs by means of a computer-assisted content analysis using a text-mining tool.

Method and Sample
This paper is a review study in nature. It uses document analysis to collect and identify relevant articles and content analysis using a text-mining tool to identify themes and concepts covered in the publications ( Figure 1). The articles were selected by searching for the following keywords: MOOC, MOOCs, Massive Open Online Course, and Massive Open Online Courses. In the initial analysis, it was found that four academic databases provide the most comprehensive search results: EBSCO, ERIC, Google Scholar, and Scopus. A total of 888 papers were collected in the screening process and were analysed using the following inclusion criteria: published in a peer-reviewed journal between 2008 and 2015; written in English; online full-text accessibility; and searched keywords appearing in the title. Accordingly, 526 papers that were irrelevant or did not meet the inclusion criteria were excluded from the sample.
Thus, 362 articles that met the criteria formed the corpus for further analysis.  Computer-based content analysis enables us to examine the conceptual structure of text-based information, so it can be used to identify the most important and most commonly occurring themes within large bodies of text (Krippendorf, 2013). For the purposes of this study, the software tool Leximancer™ was used to produce a concept map from the titles and abstracts of the 362 journal articles, as the titles and abstracts of peer-reviewed articles are usually lexically dense and focus on the core concepts, themes, and results of the research.
Leximancer™ has previously been used to analyse the content of academic journals such as Distance Education (Zawacki-Richter, & Naidu, 2016), the Journal of Cross-Cultural Psychology (Cretchley, Rooney, & Gallois, 2010), and the Journal of Communication (Lin & Lee, 2012). Moreover, it has been shown that computer-aided content analysis is an appropriate method to map out a research domain (see Fisk, Cherney, Hornsey, & Smith, 2012). The software tool creates so-called concept maps (see Figure 3) that display the core concepts within the text body (conceptual analysis) and show how these concepts are related to each other (relational analysis) by recording the frequency with which words co-occur in the text. Similar concepts that appear in close proximity are clustered together in the concept map (Smith & Humphreys, 2006): "The map is an indicative visualization that presents concept frequency (brightness), total concept connectedness (hierarchical order of appearance), direct inter-concept relative co-occurrence frequency (ray intensity), and total (direct and indirect) interconcept co-occurrence (proximity)" (p. 264). Depending on the connectedness of concepts, thematic regions are identified, indicated by coloured circles, and named after the most prominent concept in the region.

Limitations
We acknowledge the limitation that the sample selection for the purposes of this content analysis is limited to publications in academic journals in the English language, even though much of the discussion about MOOCs also takes place at conferences and in their proceedings, on blogs, and social media. This choice of methodology was influenced by our aim to explore only fully-fledged research rather than non-evidence-based claims or opinions.
Journal publications are, of course, subject to various influences (Goldenberg & Grigel, 1991): The most important of these is surely the gatekeeping role of editors, editorial boards, and reviewers of submissions to the journal. Quite aside from what one might prefer to do, publication responds to funding possibilities and publishing possibilities, and these in turn respond to connections and selection of a topic, a method, and a choice of potential journal most likely to lead to publication. (p. 436) The text-mining tool Leximancer™ has been shown to produce stable and valid results for this kind of content analysis, as in Zawacki-Richter and Naidu (2016), who used this tool to map out research trends from 35 years of publications in the journal Distance Education. However, Harwood, Gapp, and Stewart (2015) highlighted that: Leximancer is not a panacea, it still requires analytical sensitivity and judgment in its

Findings and Discussion
The concept map in Figure 3 depicts the major topics covered in the selected MOOC articles published between 2008 and 2015. The thematic summary includes a connectivity score to indicate the relative importance of the themes. The results reveal that the thematic region of courses has the most direct mentions within the text (i.e., titles and abstracts) with 599 (100% relative count), followed by MOOC / Massive Open Online Courses (83%), learners (23%), design (10%), analysis (9%), future (7%), and universities (6%). The following table provides an overview of the concepts in terms of their relative relevance in the concept map (see Figure 3). In this section, the results of the text-mining analysis are described along four connected pathways  Table 1) that are linked via the thematic regions in the overall concept map. In the following discussion, representative studies are chosen to illustrate the most prevalent research topics and themes covered in the publications.

The Potential and Challenges of MOOCs for Universities
The The authors acknowledge the potential of MOOCs to deliver education around the world. For instance, it has been reported that MOOCs can create opportunities for accessing quality higher education by building learning communities on a global scale (Mahraj, 2012) and reducing the cost of tuition (Ruth, 2012). There is also the possibility for innovative instructional designs to support self-regulated learning, unlike in traditional online courses (Bartolomé-Pina & Steffens, 2015). MOOCs also have potential in the field of corporate training, where they have been used to promote new recruiting techniques and innovative marketing and branding channels (Dodson, Kitburi, & Berge, 2015).
In addition to the many hopes for MOOCs and the benefits associated with them, the selected articles Other papers address the topic of licensing and intellectual property from the perspective of academic librarians (e.g., Mune, 2015;Gore, 2014). The business models on which MOOCs are based are important for their sustainability; Porter (2015) describes various models that are used by MOOC platforms and providers ("MOOConomics") and finds that most MOOCs are currently based on a freemium model, in which "a certain amount of a product is available to all, freely, whilst other parts of the product are charged for" (p. 57).

MOOC Platforms
The concepts MOOC and platforms are directly connected in the concept map (see concept path:

MOOC-Massive Open Online Courses-platforms).
This pattern is related to the popularity of xMOOCs, which are provided through learning platforms, as opposed to cMOOCs, which are provided in online, distributed, networked learning spaces.
Whilst Coursera, edX, and Udacity are the most established MOOC platforms, supporting very large numbers of learners, Ahn, Butler, Alam, and Webster (2013) explore alternative platforms "that promote more participatory modes of education production and delivery" (p. 160). They describe the platform of the Peer 2 Peer University, which invites any user to design and develop their own courses that can be taken by any other member of the community. The study explores how learners participated and engaged with online learning and course development using log data from the platform.
Other authors discuss MOOC platforms within specific content domains or national and cultural contexts; for instance, the Hasso Plattner Institute in Germany created the OpenHPI platform for special courses in information technology with a web tool for interactive software experiments (Neuhaus, Feinbube, & Polze, 2014). Adham and Lundqvist (2015) give an overview of Arab initiatives in the Middle East to launch their own country-specific MOOC platforms, such as Edraak in Jordan,

Learners and Content in MOOCs
In order to produce effective learning experiences with quality learning materials, the analysis of learner characteristics and profiles is the starting point in the instructional design process (Morrison, Ross, Kalman, & Kemp, 2011;Stöter, Bullen, Zawacki-Richter, & von Prümmer, 2014). It is therefore not surprising that the concept path students-MOOC-learners-content forms a central backbone in the concept map. improving their skills" (p. 10). In a more technical paper in the field of computer science, Piedra, Chicaiza, López, and Tovar (2015) propose an architecture and model for searching for OER for use in MOOCs. On the other hand, content creation has to be funded somehow, and the use and reuse of learning materials is part of the protected business model of the largest MOOC providers. Coursera, Udacity, EdX, and Future Learn have strict regulations in their terms and conditions that prohibit the reproduction, duplication, or redesign of any of their content. This is a major problem from the point of view of the Open Education Movement (Atenas, 2015).

Quality of MOOCs and Instructional Design Issues
The discussion about the quality of MOOCs is directly linked to research related to instructional design (see concept path: MOOC-study-quality-pedagogical-design) as evaluation and quality assurance is an integral part of the instructional design process (see Morrison et al., 2011). Around 2014, the first systematic MOOC quality assurance initiatives began to emerge; for example, Read and Rodrigo (2014) presented a quality model for MOOCs at UNED, the Spanish distance teaching university. In the European Excellence E-Learning Quality Project, Rosewell and Jansen (2014) developed a quality label based on benchmarks for MOOCs derived from the E-xellence label; an instrument for assessing the quality of e-learning in higher education. The European Foundation for Quality in E-Learning (EFQUEL) has also developed a special framework for the quality assurance of MOOCs (Creelman, Ehlers, & Ossiannilsson, 2014).
In contrast to these general quality frameworks, other authors elaborate in more detail on indicators of pedagogical or instructional quality. For example, in the context of teacher training, Aleman de la Garza, Sancho Vinuesa, and Gomez Zermeño (2015) administered a questionnaire with a set of indicators related to pedagogical, functional, technological, and time factors, in order to assess the quality of a MOOC on educational leadership with over 10,000 participants. Margaryan, Bianco, and Littlejohn (2015) compared and assessed the instructional design quality of xMOOCs and cMOOCs, concluding that "most MOOCs are well-packaged; [but] their instructional design quality is low" (p. 77). Admiraal, Huisman, and Van de Ven (2014) expressed particular concerns about the quality of self-and peer assessment in MOOCs. In a comparison of three MOOCs with 98,071 participants, they conclude that the quality of self-and peer assessment was only low to moderate, and that "both selfassessment and peer assessment should be used as assessment for learning instead of assessment of learning" (Admiraal, Huisman, & Van de Ven, 2014, p. 119).

Conclusion and Future Directions
This study provides an overview of the current state of research on MOOCs by analysing the titles and abstracts of publications in academic journals with a text-mining tool, in order to determine the prevailing themes and concepts in the MOOC studies. The research areas covered in these articles can Of course there are aspects that are unique to MOOCS, for example the obvious challenge to support and help very large numbers of students to succeed and to avoid dramatic drop-out rates. As Admiraal et al. (2014) discussed, carefully designed opportunities for peer support as well as self and peer assessment for learning (rather than assessment of learning) might be part of the solution, however more research is needed in this area.
In contrast, many MOOCs follow an instructional approach that leads to expository teaching and passive learning with poor student support. Research in the field of distance education has shown that student support and personal interaction, independent of time and space, is a critical factor in providing high quality learning opportunities. Furthermore, the evaluation of MOOCs and quality assurance is a very prominent and relevant topic in the publications. Rather than developing new quality frameworks for MOOCs from scratch, it is recommended to build upon quality models and instruments that were developed to measure the quality of multimedia applications, learning objects, and open educational resources (see Yuan & Recker, 2015).
After this first wave of MOOC hysteria, research and practice should focus on how best to harness the enormous opportunities that MOOCs might afford for providing access to knowledge and education, whilst equally addressing problematic issues like high dropout rates and the development of sustainable cost models. Major lessons learnt from the field of open, distance, and flexible learning (see Zawacki-Richter & Anderson, 2014), especially in the area of student support, instructional design, and quality assurance, should be kept in mind whilst moving forward.