International Review of Research in Open and Distributed Learning

Volume 20, Number 3

July - 2019

Case Study as a Research Method for Analyzing MOOCs: Presence and Characteristics of those Case Studies in the Main Scientific Databases

Ramón Montes-Rodríguez, Juan Bautista Martínez-Rodríguez, and Almudena Ocaña-Fernández
Universidad de Granada (Spain)

Abstract

Educational research is one of the many fields of knowledge that frequently use case studies as a research method, particularly when applying an interpretive approach. Based on literature reviews and a systematic analysis of current scientific literature, this paper examines the prevalence and characteristics of the case study as a methodology for research on MOOCs. Ninety-two documents were selected from the search results returned by two of the most prestigious scientific databases: Web of Science (WOS) and SCOPUS. Findings showed that (a) even when searching solely for case studies, quantitative research paradigms were more prevalent than interpretive approaches; (b) geographical distribution of these studies was partially biased; (c) case studies were less prevalent in these databases than other empirical investigations on MOOCs; (d) the data collection and data analysis methods most frequently used in the case studies were more aligned with a quantitative approach; and (e) there is still very little instructor-focused research using this methodology. In the light of these findings and their discussion, future directions for research using case study methodology are proposed, given the potential of this method to illustrate certain issues for which other approaches have proved inadequate or insufficient.

Keywords: MOOC, case study, literature review, research methods, literature analysis

Introduction and Literature Review

The study of MOOCs is very diverse and encompasses many disciplines, fields of study, and ways of understanding research, both epistemologically and methodologically (Bates, 2014). Case study is an adequate, necessary method for certain research tasks in the field of social sciences (Flyvbjerg, 2006), and provided there is a large number of thoroughly executed case studies available, this approach contributes to better and more effective disciplines (Kuhn, 1987). The prevalence of this method in research on MOOCs (Kennedy, 2014; Raffaghelli, Cucchiara, & Persico, 2015) has varied over the past few years. Despite the amount of literature on case study research designs over the past 40 years (e.g., Bennett & Elman, 2008; Gomm, Hammersley, & Foster, 2000; Merriam, 2007; Mitchell, 1983; Simons, 2009; Stake, 1995; Yin, 2009), and attempts at defining a typology to assist researchers in structuring and analyzing such studies (Thomas, 2011), there is more than one way of understanding case studies. Historically, case studies in the literature were more associated with the interpretive paradigm. However, there are now some very diverse ways of understanding case study as a research methodology. Case studies can be conducted using quantitative/qualitative paradigms or mixed methods. Indeed, the vast amount of techniques and methods for data collection and analysis that can be used for such studies has led some authors to explicitly state that “the case study survives in a curious methodological limbo” (Gerring, 2004, p. 341).

A large number of reviews of the literature on MOOC research have been published, some of which focused on analyzing thematic aspects, while others dealt more with the methodological aspects. This study presents a brief review of these papers, with special emphasis on research using the case study as a methodology, and describes the data collection and data analysis methods used.

In the first review to be conducted in a systematic manner (Liyanagunawardena, Adams, & Williams, 2013), the selected literature was categorized into different areas of interest, and the authors proposed directions to guide future research. They also identified a number of thematic and methodological gaps in the scientific literature at that time, describing four challenges facing MOOC researchers and designers: (a) the need for all perspectives of MOOCs (e.g., learners, creators, teachers, institutions) to be explored; (b) cultural tensions among pedagogies, resources, and learning environments; (c) ethical aspects of using data generated by MOOCs; and (d) the implementation of effective learning strategies in order to achieve a successful balance between information overload and self-regulated learning within MOOCs. The authors classified 21 documents that included case study, most of which had used multiple methods, with surveys being the most common data collection method.

Ebben and Murphy (2014) postulated that MOOC research at the time of their writing could be divided into two phases, the first being more related to cMOOCs, engagement, and creativity, and the second focusing on xMOOCs, learning analytics, evaluation, and critical discourse on MOOCs. These authors presented the dominant theories, the directions followed by research up to that point, and the most prevalent topics dealt with in the literature. Methodological aspects and case studies were mentioned in their study but were not the main focus of their work. In the same year, Hew and Cheung (2014) conducted another review of the literature, focusing on the motivations and challenges relating to MOOC courses, namely diversity of topics, the perspective from which they were addressed (students or instructors), and main findings to date. Although the authors spoke briefly about the techniques used (p. 47), this was not either the main objective of their analysis.

Jacoby (2014) presented a review of the literature on the theory of disruptive innovation, reporting that prior research had been predominantly qualitative, particularly comprising case studies and narrative research (p. 74). This author also postulated the need for a broader methodological range to enhance data triangulation. In a similar review, Kennedy (2014) analyzed the limitations and gaps identified in previous research on MOOCs and put forward a number of recommendations for future research. Her review included only a short paragraph on methodological aspects (p. 7), noting the wide range of methodologies such as mixed methods, case studies, narrative inquiry, and comparative studies.

In a review of research proposals submitted to the MOOC Research Initiative (MRI), Gasevic, Kovanovix, Joksimoviv, and Siemens (2014) identified a number of topics that might be used as a framework for future research. They also analyzed the methodologies used in these proposals, reporting that 42.3% had used mixed methods, 33.3% quantitative methodology, and 24.4% a qualitative approach. However, subsequent analysis showed that this research does not distinguish between paradigm, and data collection and analysis methods (Veletsianos & Shepherdson, 2016).

Raffaghelli et al. (2015) was the first published review of MOOC literature with the sole objective of analyzing methodological approaches. The authors identified trends, gaps, and criticalities derived from methodological decisions taken by MOOC researchers in the period 2008 to 2014. They noted that this field of research was still in its infancy at the time of writing, and that much of the research they reviewed relied on theoretical-conceptual research and case studies, which they considered a preliminary step toward identifying methods to deal with large cohorts or large amounts of data. These authors postulated that research on MOOCs was still in the early stages of the full cycle of educational research (Gorard & Cook, 2007).

Veletsianos and Shepherdson (2015) carried out a further review on the concept of interdisciplinarity, as well as the ways in which research published in the years 2013 to 2015, which was more aligned with the concept of xMOOC, was more interdisciplinary in nature than research conducted during the first phase (Ebben & Murphy, 2014). Most of the work was carried out by researchers from the field of education or from computer science disciplines, whose contributions to the research on MOOCs are increasingly frequent. A subsequent review published the following year provided an overview of geographic distribution, publication outlets, methodologies used, and research strands followed in studies on MOOCs (Veletsianos & Shepherdson, 2016). The findings of this review showed that researchers had used quantitative rather than qualitative insight in their works, particularly surveys and automated methods. Further, the authors reported that qualitative research on MOOCs in the period analyzed was often basic, and very few studies had actually used methods traditionally associated with qualitative research.

Two reviews have been published in the past two years that focused wholly on methodological issues (Deng & Benckendorff, 2017; Zhu, Sari, & Lee, 2018). The first reported surveys as being the most widely used method for data collection, followed by interviews and log files, and that most of the articles examined had focused on the learner-student perspective (90.6%). The second review was based on 146 articles, of which 45.9% were quantitative, 35.6% had used mixed methods, and 18.5% were qualitative in nature; this review also reported surveys as being the most common method of data collection. Descriptive statistics, inferential analysis, and content analysis were the most usual data analysis methods found in this review.

Figure 1. Reviews analyzed and their research objectives. Black indicates that a theme is the article’s main focus and gray indicates a secondary focus.

The above-mentioned review of the literature (summarized in Figure 1) provided an overview of previous MOOC research. It showed that this field of study is expanding and constantly changing (Veletsianos & Shepherdson, 2016), and although in the early years it was mainly comprised of conceptual studies and case studies (Jacoby, 2014), its evolving nature brought forth a large number of macro-type empirical studies (e.g., big data, learning analytics) that were facilitated by the current availability of large datasets. However, no studies were found on the actual concept of case studies, or on the characteristics of case studies that are used to gain a deeper understanding of MOOC platforms. Generally, case studies in the field of social sciences have been understood as a methodology associated with qualitative, interpretive, or hermeneutic paradigms. Therefore, it is essential—and would enhance this field of study, and advance our understanding of the methodological aspects—that we examine how the scientific community is using case studies to explore online environments, and ascertain what paradigms/approaches and methods are used in these case studies, what is their research focus, and their prevalence in the main scientific databases.

Statement of Research Problem and Purpose

The main aim of this review was to gain a better understanding of the characteristics of case studies relating to MOOCs in the WOS and Scopus databases (i.e., the methodological approaches used, how they were implemented, and their focus). An in-depth review of extant literature in the major academic databases can provide information on the characteristics of the publications available in these peer-reviewed outlets, as well as what has already been done, what remains to be done, and what might be the possible direction for future research based on this methodology. The authors of this paper wished to contribute to one of the ideas for future research proposed by Veletsianos & Shepherdson (2016), namely that “future research endeavors in this area may focus on examining how particular methodologies have shaped the field” (p. 215). Their research identified potential research gaps in the study of MOOCs and this paper confirms and extends that research to provide possible directions for future research using case studies.

In order to explore the presence of case study as a methodology for research on MOOCs, we posed the following research questions:

1. What methodological approaches did case studies on MOOCs follow?
    
2. What were the usual publication outlets for case studies on MOOCs: journals or conference proceedings? Which journals published the highest number of case studies on MOOCs?
    
3. Which case studies had the highest citation count, and how were the publications distributed over the years?
    
4. What data collection and data analysis methods were used in these case studies?
    
5. What was the main focus of these case studies?

Research Methodology

A systematic literature review was conducted based on an analysis of 92 documents of peer-reviewed literature indexed in the WOS and Scopus databases during the period January 2012 to June 2018. In order to answer the research questions, several methods were systematically followed to collect the extant literature in WOS and Scopus, and analyze the corpus of selected papers.

Data Collection

Searches of WOS and Scopus were performed in July 2018 using the keywords MOOC, MOOCs, massive open online courses, xMOOC, cMOOC, and these were interrelated with the keyword case study in the fields article, keywords, and abstract in the two databases (in WOS, the field is shown as topic). In order to analyze the returned results in greater detail, the following selection criteria were applied in order to selectively eliminate: (a) duplicated search results returned from separate databases; (b) documents that appeared to be case studies but were not, or had not used a case study as a part of their methodology; (c) documents in languages other than English or Spanish; (d) documents that were not proceedings or journal articles (in SCOPUS only journal articles were selected); and (e) any documents with a zero citation count. Application of these selection criteria brought forth a corpus of 92 papers.

The first step was to define the structure of a database to store the most relevant information from the documents analyzed. Records in the database were structured as shown in Table 1. Each of the records included objective information (e.g., title, author(s), keywords, year of publication, DOI/URL) as well as several other fields that were completed after reading and analyzing the full text of each document, including methodological approach, data collection methods, data analysis methods, and focus of the study.

Table 1

Database Structure

Data Classification and Analysis

The corpus was analyzed quantitatively and qualitatively. Quantitative analysis was used to classify the documents by year of publication, type of publication, country of author affiliation, citation count, and mean citation count per year. The documents were also analyzed qualitatively using open-coded content analysis, a technique that has been used previously in other literature review studies on MOOCs (Liyanagunawardena et al., 2013). First, one of the authors read each of the documents in order to identify the methodological approach, data collection methods, data analysis methods, and focus of the paper. Subsequently, the other authors carried out a similar review, sharing their results with the first researcher. In case of discrepancies, researchers would re-read and examine the pertinent text together, and reach a joint decision.

The basic unit of analysis was a single selected paper; a constant comparison method was used for its classification and analysis. The first text was analyzed and coded by a researcher using emergent coding, and the topic category and methodological approaches were defined. The second text was analyzed in the same way and checked to determine whether it could be classified into the same category as the previous document; otherwise, a new category was created. The process was subsequently repeated until all documents had been read and analyzed. To eliminate the possibility of a document being classifiable in more than one category, the categories were thoroughly examined and verified by all authors to ensure that each category was exclusive and was not repeated in any way. The categories were shared and agreed to by all authors throughout the analysis process.

Results

The following tables show the results of the review of the corpus of collected papers for each of the research questions posed at the outset of this study.

Question One: What Methodological Approaches did Case Studies on MOOCs Follow?

On analyzing the individual documents, based on the previously described method, it was found that 30 documents had used a quantitative approach, and 26 a qualitative approach; 25 had used mixed methods, and 11 an unclear/not explicit approach.

Table 2

Methodological Approaches

Question Two: What Were the Usual Publication Outlets for Case Studies on MOOCs: Journals or Conference Proceedings? Which Journals Published the Highest Number of Case Studies on MOOCs?

Figure 2. Type of document collected.

Of the 92 documents collected, 70 articles were from peer-reviewed journals and 22 were from conference proceedings. With regard to journal publications, 44 journals had each published one item; the remainder had published 2 or more documents, as shown in Table 3. The journal International Review of Research in Open and Distributed Learning (IRRODL) published 9 articles, the highest number of documents.

Table 3

Main Publication Outlets

Figure 3 illustrates the geographic distribution of those who authored the 92 articles. Overall, 30 documents had one or more authors from the United States of America (32.6% of the corpus), there were 17 documents with at least one author from Spain (18.5% of the corpus), 11 with at least one author from the United Kingdom (11.9%), six with at least one author from China, five with at least one author from Germany, and so forth. There are other 21 different author affiliations present in just one document, each accounting for 1.1% of the corpus. The most highly represented region in the corpus was Europe (42.4%), then North America (38%), followed by Asia (19.6%).

Figure 3. Authors’ affiliation and number of documents.

Question Three: Which Case Studies Had the Highest Citation Count, and How Were the Publications Distributed Over the Years?

As Figure 4 shows, 37 of the documents analyzed were published in 2016 (40.2%), with 19 of the documents from the remaining years published in 2015 (20.6%), 15 in 2017 (16.3%), 12 in 2014 (13%), 5 in the first semester of 2018 (5.4%), 3 in 2013 (3.3%), and 1 in 2012 (1.1%).

Figure 4. Documents by publication year.

Table 4 shows the highest citation counts in the Web of Science database (more than 10 citations) for the publications included in the corpus of this review; three of these were published in the International Review of Research in Open and Distributed Learning, three in Computers & Education, and three in other journals.

Table 4

Documents Most Frequently Cited

Note. Documents located as a result of search for MOOC + case study query.

A search was also performed of the WOS database using just the keyword MOOC, without the words case study. The search results list returned the three most-cited case studies in the corpus of our review (De Freitas, Morgan, & Gibson, 2015; Mackness, Waite, Roberts, & Lovegrove, 2013; Muñoz-Merino, Ruipérez-Valiente, Alario-Hoyos, Pérez-Sanagustín, & Delgado Kloos, 2015) in positions 26, 29, and 38 when sorted by citation count, which means that none of these case studies were among the 25 most-cited documents on MOOC literature.

Question Four: What Data Collection and Analysis Methods Were Used in These Case Studies?

An analysis of the main data collection methods showed that most of the studies reviewed had used one (41.4%) or two (35.7%) methods for collecting data. Three or more data collection methods were used in 22.9% of the documents.

Table 5

Data Collection Methods and Their Prevalence

The most frequently used methods and techniques as shown in Table 5 are, in this order: surveys and questionnaires, user datasets or logs, and forum participation. Other collection methods not mentioned in the table were found in solely one document of the corpus (e.g., rubrics or email messages).

Table 6

Data Analysis Methods and Their Prevalence

Table 6 shows the data analysis methods used in the corpus documents, statistical analysis (various types) being the most frequent method used (47), followed by qualitative content analysis (17), grounded theory (11), and experimental or quasi-experimental analysis. Other methods used are shown in the table above: thematic analysis (7), software-assisted automated analysis (6), crosschecking between different analyses (5), discourse analysis (5), big data analysis (4), and other methods mentioned in only one of the corpus documents.

Question Five: What Was the Main Focus of These Case Studies?

Six categories were established to describe the research reported in this corpus (see Table 7), using a categorization similar to that of Veletsianos and Shepherdson (2016). Most of the 92 MOOC case studies included in the sample were focused on learners/students (39) or the actual platform (28). Just 12 documents focused on pedagogical design, eight on instructors/teachers, three on the community, two on methodological design, and four on other elements. A small number of papers were included in two categories (4).

Table 7

Main Focus of the Case Studies

Learner-focused. All documents in which the key element was the figure of the learner were included in this category, regardless of the topic covered in the research. Other topics included learner expectations and perceptions (e.g., Cross & Whitelock, 2017), participation level (e.g., Veletsianos, 2017), roles and identities (e.g., Baxter & Haycock, 2014), user behaviors (e.g., Zhang & Yuan, 2016), or engagement/retention level (e.g., De Freitas et al., 2015).

Platform-focused. This category included papers focusing on the platform as the main research element. Papers covered topics such as platform usability (e.g., Yousef, Chatti, Schroeder, & Wosnitza, 2015), innovative tools (e.g., Fu, Zhao, Cui, & Qu, 2017), intelligent and adaptive systems (e.g., García-Peñalvo, Fidalgo-Blanco, & Sein-Echaluce, 2018), mediation and control (e.g., Nyoni, 2013), or the relationship between the pedagogical design and the platform design (e.g., Drake, O’Hara, & Seeman, 2015).

Pedagogical design-focused. Papers in this category focused on social and collaborative learning (e.g., Fidalgo-Blanco, Sein-Echaluce, & García-Peñalvo, 2016; Harp Ziegenfuss & Furse, 2016), connectivism and other theories (e.g., Anders, 2015), assessment and learning environments (e.g., Hills & Hughes, 2016), or social interactions between learners and instructors.

Instructor-focused. Papers in this category focused on the role of MOOC instructors (e.g., Haavind & Sistek-Chandler, 2015), instructor perspectives and experiences (e.g., Haavind & Sistek-Chandler, 2015), or the relationship between teaching and learning environments (e.g., Ramírez, Rivera, & García, 2015).

Community-focused. This category included a number of studies that focused on understanding the set of stakeholders in a MOOC, as a community (e.g., Grünewald et al., 2013, Jones, Stephens, Branch-Mueller, & De Groot, 2016).

Research methodology-focused. Papers in this category discussed improving data analysis and visualization (e.g., Pardos, Whyte, & Kao, 2016), or new data mining techniques (e.g., Maté, De Gregorio, Cámara, Trujillo, & Luján-Mora, 2016).

Other. This category included papers that could not be classified in any of the other categories because they concerned topics such as ethics and privacy (Jones & Regner, 2016), or plagiarism (Tsoni & Lionarakis, 2014).

Discussion

MOOC literature reviews published prior to this paper were examined. For the purpose of the present review, 92 publications that had used case study methodology for MOOC research were collected and analyzed. A dataset of MOOC-related case studies was created to facilitate identification of internal methodological approaches, publication outlet, prevalence in the databases, methods used for data collection and analysis, and the papers’ research focus. Findings showed that despite being a methodology generally linked to more interpretive paradigms (Simons, 2009; Stake, 1995), almost 60% of the studies analyzed had used a quantitative approach (32.6%) or a mixed method (27.2%). The papers in the corpus analyzed in this study had been published in a wide range of journals and conference proceedings; some journals had published a greater number of studies than others. Findings also showed that the source of more than 80% of research using the case study method was Europe and North America. Further, it was observed that unless the keyword case study was included in the search query, the search results returned for the 25 most cited papers in the literature on MOOCs did not include any case studies. With regard to the data collection methods that appeared in these documents, the most notable methods were linked to quantitative paradigms such as surveys or the platform dataset; statistical analysis was the main data analysis method used. The focus of the case studies analyzed was essentially the students or the actual platform. Accordingly, these findings have a number of implications for future research on MOOCs using case study methodology, and for the state of the field as a whole.

Our Choices May be Limiting a Deeper Understanding of MOOCs

Coinciding with previous reviews of MOOC literature, this analysis suggests that researchers tended to choose a quantitative rather than a qualitative approach, even for case studies. This might suggest that more interpretive, hermeneutical, or qualitative research is needed. For example, as has also been noted in previous reviews (Veletsianos & Shepherdson, 2016), an in-depth analysis of the role of instructors in MOOC courses could serve to illustrate certain topics that do not seem to be much in evidence in the current literature. Qualitative case study would be a very useful methodological approach for such a purpose, thus providing some excellent possibilities for future research.

Certain Regions are Setting the Pace of Case Study Research on MOOCs

Author affiliation in more than 80% of the documents analyzed was either Europe or North America. This is in keeping with the findings from previous reviews researching geographical distribution, except for a higher number of author affiliations from Spain, given that Spanish was included as a language in this review. Nonetheless, the results show that these databases analyzed may be favoring literature from certain regions, or otherwise limiting and failing to give visibility to the literature from other countries, which would possibly indicate a direction for future research, to analyze this geographical bias. Countries such as India or China, with a large presence in the MOOC community by the number of people enrolled in these courses, have little or no presence in the selected corpus.

Case Studies Tend to be Less Prevalent in the WOS and SCOPUS Databases Than Are Other Types of Empirical Studies

It was noted from the findings that the citation count for the case studies analyzed was somewhat lower than for other types of empirical studies. More than half of the documents returned in the search results had a zero-citation count. Further, the most frequently cited case study in the corpus ranked 26th in the listing. On analysis, only two of the ten most cited case studies had used a qualitative methodology, which raises the question of whether the research reported in the most highly ranked papers in these databases is perhaps quantitative rather than qualitative, even for case studies. Although a number of previous reviews in MOOC literature referred to the existence of case studies and their abundance (Raffaghelli et al., 2015), they were not always listed in the search results returned by the main databases. This highlights the higher presence of case studies in other databases (e.g., Google Scholar, ERIC, EBSCOhost) and a secondary role in WOS and SCOPUS.

Higher Prevalence of Quantitative Methods and Greater Diversity of Qualitative Methods

Surveys, questionnaires, and datasets were the most predominantly used methods, even in a large number of papers reporting so-called qualitative research. However, qualitative data collection methods, which were generally less prevalent, were found to be more diverse. Similar findings were obtained in regard to data analysis methods—a large number of the documents analyzed had used different types of statistical analysis (usually descriptive), and papers that had used methods more closely linked to the qualitative paradigm frequently used multiple techniques to analyze the data (e.g., content analysis, thematic analysis, discourse analysis, grounded theory). The database used for this study also showed a sharp increase in recent years in the use of automated analysis methods, and of research using big data or data mining techniques for data analysis in case studies, most of which were based on a quantitative approach.

Scarcity of Instructor-Focused Research

As noted in other literature reviews, the number of studies highlighting the role of instructors is very limited. Case study would be an appropriate methodology for such research, given that it focuses on the characteristics of unique, specific cases. Case study could help illustrate how instructors experience their involvement in MOOC courses, how they perceive their relationships with colleagues or students, how these differ from relationships in an offline environment, or what motivates them to become instructors. These are research strands that are currently little explored, and case study is a highly flexible methodology that could help to clarify certain aspects regarding the people who perform teaching tasks in these courses.

Limitations

This study has several limitations. First, only the Scopus and WOS databases were used to find case studies, and although these two databases are the most recognized in the academic field, there might be other case studies, in other journals, that were not indexed in these databases. Second, there could be different documents indexed in WOS or Scopus that used case study as a methodology but had failed to explicitly indicate so, and therefore, were not included in this review. Third, by not incorporating articles that have not yet been cited, the study may not reflect the very latest trends in the field (e.g., monetization and business models, big data and learning flexibility, sustainability). Fourth, English, as the language for publications, is over-represented in the WOS and Scopus databases, in the same way that certain scientific fields are more prevalent than others; social sciences, as well as arts and humanities, for example, may be underrepresented (Mongeon & Paul-Hus, 2016). The results of this study are also affected by the rapid evolution of scientific literature on MOOCs in recent years. In this case, it was decided to review solely journal articles and proceedings in order to define the characteristics of these papers in a peer-reviewed process. The authors note that this review did not include a large number of case studies that are available in other formats (e.g., books, blogs, reports, non-indexed journals), or in languages other than English or Spanish, which undoubtedly also form part of the debate in this field of knowledge.

Conclusion

A large number of research papers have been published on MOOCs since their inception, some of which used the case study as a methodology. The data from this review reveals that their prevalence and citation count in databases was limited compared to other empirical works. According to the findings from this analysis of 92 case studies published between 2012 and 2018, (a) more than half of these papers used quantitative methods for data collection and analysis; (b) much of the research focused essentially on learners; and (c) author affiliation was predominantly North American or European, showing a clear geographic bias. Although this study focused on a specific methodology, its findings nonetheless replicated the results of previous studies in which more than one method was examined (e.g., Veletsianos & Shepherdson, 2016; Zhu et al., 2018), and which reported a scarcity of research focusing on instructors, or other studies that found a higher prevalence of quantitative methods (Deng & Benckendorff, 2017; Gasevic et al., 2014). The authors of this paper hope that these findings will encourage future studies on those aspects of MOOCs that have not yet been explored in depth. There are still many possible topics in this area for further research using case study methodology, and the authors recommend that similar reviews of MOOC literature be conducted with regard to other methodological approaches or to different databases containing more research from other regions written in other languages.

Acknowledgments

This research was partially funded by Ministerio de Economía y Competitividad of the Spanish Government through the Research & Development project ECOEC (EDU2014-51961-P), and under the terms of FPU2014-02298 research contract agreed between Ministerio de Educación, Cultura y Deporte of the Spanish Government and researcher Ramón Montes-Rodríguez.

References

Anders, A. (2015). Theories and applications of massive online open course [MOOCs]: The case for hybrid design. The International Review of Research in Open and Distributed Learning, 16(6), 39-61. doi: 10.19173/irrodl.v16i6.2185

Bates, T. (2014). MOOCs: Getting to know you better. Distance Education, 35(2), 145-148. doi: 10.1080/01587919.2014.926803

Baxter, J. A., & Haycock, J. (2014). Roles and student identities in online large course forums: Implications for practice. The International Review of Research in Open and Distributed Learning, 15(1), 20-40. doi: 10.19173/irrodl.v15i1.1593

Bennett, A., & Elman, C. (2008). Case study methods. In C. Reus-Smit, & D. Snidal (Eds.), The Oxford handbook of international relations (pp. 498-517). Oxford University Press. doi: 10.1093/oxfordhb/9780199219322.003.0029

Cross, S., & Whitelock, D. (2017). Similarity and difference in fee-paying and no-fee learner expectations, interaction and reaction to learning in a massive open online course. Interactive Learning Environments, 25(4), 439-451. doi: 10.1080/10494820.2016.1138312

De Freitas, S. I., Morgan, J., & Gibson, D. (2015). Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. British Journal of Educational Technology, 46(3), 455-471. doi: 10.1111/bjet.12268

Deng, R., & Benckendorff, P. (2017). A contemporary review of research methods adopted to understand students’ and instructors’ use of massive open online courses (MOOCs). International Journal of Information and Education Technology, 7(8), 601-607. doi: 10.18178/ijiet.2017.7.8.939

Drake, J. R., O’Hara, M., & Seeman, E. (2015). Five principles for MOOC design: With a case study. Journal of Information Technology Education: Innovations in Practice, 14(1), 125-143. Retrieved from http://www.jite.org/documents/Vol14/JITEv14IIPp125-143Drake0888.pdf

Ebben, M., & Murphy, J. S. (2014). Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328-345. doi: 10.1080/17439884.2013.878352

Fidalgo-Blanco, Á., Sein-Echaluce, M. L., & García-Peñalvo, F. J. (2016). From massive access to cooperation: Lessons learned and proven results of a hybrid xMOOC/cMOOC pedagogical approach to MOOCs. International Journal of Educational Technology in Higher Education, 13(1). doi: 10.1186/s41239-016-0024-z

Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219-245. doi: 10.1177/1077800405284363

Fu, S., Zhao, J., Cui, W., & Qu, H. (2017). Visual analysis of MOOC forums with iForum. IEEE Transactions on Visualization and Computer Graphics, 23(1), 201-210. doi: 10.1109/TVCG.2016.2598444

García-Peñalvo, F. J., Fidalgo-Blanco, Á., & Sein-Echaluce, M. L. (2018). An adaptive hybrid MOOC model: Disrupting the MOOC concept in higher education. Telematics and Informatics, 35(4), 1018-1030. doi: 10.1016/j.tele.2017.09.012

Gasevic, D., Kovanovic, V., Joksimovic, S., & Siemens, G. (2014). Where is research on massive open online courses headed? A data analysis of the MOOC Research Initiative. The International Review of Research in Open and Distributed Learning, 15(5). doi: 10.19173/irrodl.v15i5.1954

Gerring, J. (2004). What is a case study and what is it good for? The American Political Science Review, 98(2), 341-354. Retrieved from https://www.jstor.org/stable/4145316

Gomm, R., Hammersley, M., & Foster, P. (2000). Case study method: Key issues, key texts. London: SAGE.

Gorard, S., & Cook, T. (2007). Where does good evidence come from? International Journal of Research & Method in Education, 30(3), 307-323. doi: 10.1080/17437270701614790

Grünewald, F., Mazandarani, E., Meinel, C., Teusner, R., Totschnig, M., & Willems, C. (2013, March). openHPI: A case-study on the emergence of two learning communities. Proceedings of the 2013 IEEE Global Engineering Education Conference (pp. 1323-1331). doi: 10.1109/EduCon.2013.6530277

Haavind, S., & Sistek-Chandler, C. (2015). The emergent role of the MOOC instructor: A qualitative study of trends toward improving future practice. International Journal on E-Learning: Corporate, Government, Healthcare, and Higher Education, 14(3), 331-350. Retrieved from https://www.learntechlib.org/primary/p/150663/

Harp Ziegenfuss, D., & Furse, C. (2016). Opening up collaboration and partnership possibilities: Re-valuing library resources, skill sets, and expertise. Digital Library Perspectives, 32(2), 103-116. doi: 10.1108/DLP-09-2015-0014

Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58. doi: 10.1016/j.edurev.2014.05.001

Hills, L., & Hughes, J. (2016). Assessment worlds colliding? Negotiating between discourses of assessment on an online open course. Open Learning, 31(2), 108-115. doi: 10.1080/02680513.2016.1194747

Jacoby, J. (2014). The disruptive potential of the massive open online course: A literature review. Journal of Open Flexible and Distance Learning, 18(1), 73-85. Retrieved from http://www.jofdl.nz/index.php/JOFDL/article/view/214

Jones, K. M. L., Stephens, M., Branch-Mueller, J., & De Groot, J. (2016). Community of practice or affinity space: A case study of a professional development MOOC. Education for Information, 32(1), 101-119. doi: 10.3233/EFI-150965

Jones, M. L., & Regner, L. (2016). Users or students? Privacy in university MOOCS. Science and Engineering Ethics, 22(5), 1473-1496. doi: 10.1007/s11948-015-9692-7

Kennedy, J. (2014). Characteristics of massive open online courses (MOOCs): A research review, 2009-2012 - Journal of Interactive Online Learning, 13(1). Retrieved from http://www.ncolr.org/issues/jiol/v13/n1/1

Kuhn, T. S. (1987). What are scientific revolutions? In L. Kruger, L. J. Daston, & M. Heidelberger (Eds.), The probabilistic revolution (Vol. 1): Ideas in history (pp. 6-27). Cambridge, MA: MIT Press.

Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14(3), 202-227. doi: 10.19173/irrodl.v14i3.1455

Mackness, J., Waite, M., Roberts, G., & Lovegrove, E. (2013). Learning in a small, task-oriented, connectivist MOOC: Pedagogical issues and implications for higher education. The International Review of Research in Open and Distributed Learning, 14(4), 140-159. doi: 10.19173/irrodl.v14i4.1548

Maté, A., De Gregorio, E., Cámara, J., Trujillo, J., & Luján-Mora, S. (2016). The improvement of analytics in massive open online courses by applying data mining techniques. Expert Systems, 33(4), 374-382. doi: 10.1111/exsy.12119

Merriam, S. B. (2007). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.

Mitchell, J. C. (1983). Case and situation analysis. The Sociological Review, 31(2), 187-211. doi: 10.1111/j.1467-954X.1983.tb00387.x

Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of web of science and scopus: A comparative analysis. Scientometrics, 106(1), 213-228. doi: 10.1007/s11192-015-1765-5

Muñoz-Merino, P. J., Ruipérez-Valiente, J. A., Alario-Hoyos, C., Pérez-Sanagustín, M., & Delgado Kloos, C. (2015). Precise effectiveness strategy for analyzing the effectiveness of students with educational resources and activities in MOOCs. Computers in Human Behavior, 47, 108-118. doi: 10.1016/j.chb.2014.10.003

Nyoni, J. (2013). The viral nature of massive open online courses (MOOCs) in open and distance learning: Discourses of quality, mediation and control. Mediterranean Journal of Social Sciences, 4(3), 665-672. doi: 10.5901/mjss.2013.v4n3p665

Pardos, Z. A., Whyte, A., & Kao, K. (2016). moocRP: Enabling open learning analytics with an open source platform for data distribution, analysis, and visualization. Technology, Knowledge and Learning, 21(1), 75-98. doi: 10.1007/s10758-015-9268-2

Raffaghelli, J. E., Cucchiara, S., & Persico, D. (2015). Methodological approaches in MOOC research: Retracing the myth of Proteus. British Journal of Educational Technology, 46(3), 488-509. doi: 10.1111/bjet.12279

Ramírez, M. S., Rivera, N., & García, A. (2015, November). MOOC learning: Challenges and opportunities of using team teaching. Proceedings of ICERI2014 7th International Conference of Education, Research and Innovation (pp. 5751-5756). Retrieved from https://pdfs.semanticscholar.org/e87d/750a4b95189ca49d006abdf471bc3eb8b6ed.pdf

Simons, H. (2009). Case study research in practice. Los Angeles, CA: SAGE.

Stake, R. E. (1995). The art of case study research. Los Angeles, CA: SAGE.

Thomas, G. (2011). A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry, 17(6), 511-521. doi: 10.1177/1077800411409884

Tsoni, R., & Lionarakis, A. (2014, November). Plagiarism in higher education: The academics’ perceptions. In IMCL2014 International Conference on Interactive Mobile Communication Technologies and Learning (pp. 296-300). doi: 10.1109/IMCTL.2014.7011151

Veletsianos, G. (2017). Toward a generalizable understanding of Twitter and social media use across MOOCs: Who participates on MOOC hashtags and in what ways? Journal of Computing in Higher Education, 29(1), 65-80. doi: 10.1007/s12528-017-9131-7

Veletsianos, G., & Shepherdson, P. (2015). Who studies MOOCs? Interdisciplinarity in MOOC research and its changes over time. The International Review of Research in Open and Distributed Learning, 16(3). doi: 10.19173/irrodl.v16i3.2202

Veletsianos, G., & Shepherdson, P. (2016). A systematic analysis and synthesis of the empirical MOOC literature published in 2013-2015. The International Review of Research in Open and Distributed Learning, 17(2). doi: 10.19173/irrodl.v17i2.2448

Yin, R. K. (2009). Case study research: Design and methods. Los Angeles, CA: SAGE.

Yousef, A. M. F., Chatti, M. A., Schroeder, U., & Wosnitza, M. (2015). A usability evaluation of a blended MOOC environment: An experimental case study. The International Review of Research in Open and Distributed Learning, 16(2), 69-93. doi: 10.19173/irrodl.v16i2.2032

Zhang, T., & Yuan, B. (2016). Visualizing MOOC user behaviors: A case study on XuetangX. In H. Yin, Y. Gao, B. Li, D. Zhang, M. Yang, Y. Li, ... A. J. Tallón-Ballesteros (Eds.), Intelligent data engineering and automated learning - IDEAL 2016 (pp. 89-98). Cham (Switzerland): Springer International Publishing.

Zhu, M., Sari, A., & Lee, M. M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014-2016). The Internet and Higher Education, 37, 31-39. 10.1016/j.iheduc.2018.01.002

Case Study as a Research Method for Analyzing MOOCs: Presence and Characteristics of those Case Studies in the Main Scientific Databases by Ramón Montes-Rodríguez, Juan Bautista Martínez-Rodríguez, and Almudena Ocaña-Fernández is licensed under a Creative Commons Attribution 4.0 International License.