Systematic Mapping Study of Academic Engagement in MOOC

MOOCs are presented as an affordable and easily accessible modality that offers the opportunity to democratize education in our time; however, this convenience training favors a low completion rate of the participants. Faced with this situation, scholars have suggested that it is necessary to deepen the construct of academic engagement, a concept that has been addressed in the study of face-to-face training, to better understand how students participate in this educational modality. This article systematically explores the existing literature, in the period of 2015-2018, about the construct of academic engagement in online, massive and open learning courses, through a Systematic Mapping of Literature, a method which aims to identify the characteristics of production in a given subject. The results show that there is a considerable increase in published articles that associate academic engagement and MOOCs, mainly from the United States, Australia, and the United Kingdom. Most of the mapped publications employ qualitative methods, with an exploratory approach, although there are several correlational studies. The study of participation patterns and instructional design appear as the main topics of interest in the field. In addition to providing a general overview of production on the subject, the research provides accurate information that will identify works for more in-depth reviews. Thus, it also offers a replicable and flexible literature search method for different research interests.


Introduction
Talking about massive, open and online courses (MOOCs) means referring to a low-cost educational offering, with the possibility of connection at any time and place. Although the idea of education at convenience is accessible to the user, it also encourages participants to postpone, forget, or disengage from carrying out the academic activities (Kizilcec, Piech, & Schneider, 2013;Milligan, Littlejohn, & Margaryan, 2013). Academic research records show that although a large number of students enroll to start MOOCs, only a small fraction manages to complete them (Halawa, Greene, & Mitchell, 2014;Jordan, 2014).
Therefore, there is consistent criticism among MOOCs researchers, one of which is that this method does not offer students the necessary structures to learn significantly and autonomously, which causes lack of persistence, lack of motivation and, finally, course desertion (Conole, 2015;Jordan, 2014;Milligan et al., 2013).
The possibility offered by MOOCs to democratize education and the limitation of its low completion rate have led to an area of interest for educational research. Although some authors consider that research in MOOCs is an incipient and challenging area (de Barba, Kennedy, & Ainley, 2016;Gašević, Dawson, & Siemens, 2015;Greene, Oswald, & Pomerantz, 2015), since their emergence in 2006 research has focused on (1) studying aspects to motivate participants to complete the courses (e.g., Kizilcec, Pérez-Sanagustín, & Maldonado, 2016;Kizilcec & Schneider, 2015); (2) identifying aspects related to self-regulation of learning to reduce dropout or predict performance and/or retention (e.g., Kizilcec et al., 2016); and, (3) analyzing course design elements for the same purpose (e.g., Conole, 2015). These three topics are linked to what other researchers have called academic engagement in the classroom modality.
Researchers have studied the academic engagement construct as a way to improve discontent, avoid boredom, improve motivation and student participation in academic activities, increase success levels, and understand the positive development of students (Appleton, Christenson, & Furlong, 2008;Carter, Reschly, Lovelace, Appleton, & Thompson, 2012;de Barba et al., 2016;Valdivia, Ramírez-Montoya, & Valenzuela, 2018). Academic engagement is also studied as being a valuable construction to capture the gradual process by which students abandon academic activities (Appleton et al., 2008;Kizilcec et al., 2013). In MOOCs, researchers and educators consider academic engagement as the main theoretical foundation to intervene and understand possible dropouts, to improve positive performance, and encourage the completion of an educational goal (Joksimovic et al., 2018).
As every cognitive construct, there is no single definition or form of measurement for engagement. Newmann, Wehlage, and Lamborn (1992) define it as the psychological inversion in which the student invests energy and effort to understand something. Meanwhile, York, Gibson, and Rankin (2015) indicate that engagement is a term generally used to refer to the student's psychological investment, his or her willingness to invest time in educational behaviors, or to a general reference of student involvement in educational activities. In MOOCs, engagement can be conceptualized in a similar way as in face-to-face education; however, its operationalization, in terms of the forms and processes of data collection, is totally different. According to Joksimović et al. (2018), in MOOCs, engagement consists of time spent on course activities, participation in tests and exams, time spent in videos, and participation in exercises and assignments.
Given the emerging condition of academic engagement as a construct associated with MOOCs as a response to the problem of low success found in these educational environments, this research aims to map the scientific production on academic engagement in MOOCs published in the years 2015, 2016, 2017 and in the beginning of 2018, to identify the specific lines of study within this topic. The research answers the question: What has been the production in the three-year period between 2015 and 2018 on academic engagement in MOOCs?
Studies on the academic engagement of participants in MOOCs are recent; however, the subject is in consolidation as a line of study, and several literature reviews associated with the construct have been done.
Different authors have identified academic engagement as a research trend in MOOCs. Ebben and Murphy (2014), for example, analyzed 25 articles published between 2011 and 2013 with the objective of identifying research topics on MOOCs. The following trends stand out in their results: academic engagement, creativity, learning analytics, evaluation, and critical discourses. Subsequently, Sa'Don, Alias, and Ohshima (2014), examined 164 articles published between 2008 and 2014 with the same objective as Ebben and Murphy, specifically in institutions of Higher Education. Their results highlight research trends such as evaluation and engagement/motivation, social interaction, retention, politics, instructional design, and cultural diversity. Authors like Anderson, Huttenlocher, Kleinberg, and Leskovec (2014) and Kizilcec, Pérez-Sanagustín, and Maldonado (2016) argue that there is still little understanding of how students participate and become involved in MOOCs, and that this construct is still under construction.
Bozkurt, Akgün-Özbek, and Zawacki-Richter (2017) conducted a systematic literature review that identified trends and research patterns in massive environments. The authors reviewed 362 empirical articles from 2008 to 2015 and conducted content and discourse analyses. Among their results they found that: (1) research on MOOCs would increase in subsequent years; (2) conceptual/descriptive studies are the most used methodology in MOOCs, constituting the majority of articles (53.3%) in almost all years studied; (3) the three main areas of research in MOOCs are: theories and models, characteristics of the students, and instructional design; and (4) the second most used methodology is quantitative research (19.6%) with few surveys, correlational, or experimental studies. Raffaghelli, Cucchiara, and Persico (2015)  (4) qualitative methods, which are a minority in their study, use interviews, observations, and focus groups; and (5) little research is done about the instructor or expert (Veletsianos & Shepherdson, 2016). Joksimovic et al. (2018) conducted a literature review on learning approaches in MOOCs. In their study, they analyzed the constructs related to the learning used in the prediction and measurement of the engagement and the learning outcome (Joksimovic et al., 2018). One of the results reported by the authors was the lack of solid frameworks to explain learning in an open online environment, thus they proposed an appropriate framework for open online contexts based on the model as developed by Reschly and Christenson (2012), which defines engagement as a process and as a result (Joksimovic et al., 2018).
The analysis of the reviews allows us to deduce that, although the conclusions of the research have suggested that academic engagement is a relevant construct to understand the participation of students in MOOCs, researchers on this environment have not incorporated the concept in their literature reviews. Thus, as pointed out by Anderson et al. (2014) and Kizilcec et al. (2016), the understanding of how students participate is still scarce.

Method
The research was developed through a Systematic Mapping Study (SMS). SMS is a type of literature review used to identify, select, and synthesize production in a specific field or associated with a concept, with the purpose of identifying what evidence is available on the subject (Cooper, 2016;Kitchenham & Charters, 2007). As outlined by Kitchenham and Charters (2007), a SMS focuses on classification, thematic analysis, and identification of publication without evaluating quality. This type of study differs from systematic reviews, which focus on quality, in order to identify the best practices based on empirical evidence (Kitchenham & Charters, 2007).

First Moment: Search Approach
The approach consisted in the formulation of the following questions to guide the inquiry:

Second Moment: Development of the Search Protocol
The search protocol was designed based on the steps performed by Petersen, Vakkalanka, and Kuzniarz (2015) both for the selection of scientific production and for its analysis. Figure 1 graphically represents this process. As recommended by Dybå, Dingsøyr, and Hanssen (2007), indexes were chosen that met different quality criteria: (1) containing intelligent tools to track, analyze and visualize the research; (2) integrating global critical and scientific research; and (3) having peer evaluation. The two databases chosen that met the quality criteria mentioned were: (1) Scopus and (2) Web of Science (WoS). This allowed a global search of production in fields such as science, technology, medicine, social sciences, and arts and humanities.
Searches in Scopus and WoS were carried out on March 1st, 2018. To have control over the search result for analysis after this work, several individual search expressions were created. Once the individual searches were carried out, a combination of searches was done. Table 1 shows the individual and combined searches, and the results obtained in each step. Four inclusion criteria were defined: (1) per year (2015-2018), (2) per area (the one with the highest frequency), (3) per type of document and source (articles, articles in press and conferences), and (4) per language (English). One exclusion criterion was set: without emerging indexes. Regarding the fourth inclusion criterion (per language), it is important to mention that no documents were found in a language other than English, so it was not considered as part of the flow diagram of the search protocol. The results for each index are shown in Table 2.

Third Moment: Analysis and Classification
As a part of this step, data extraction was performed. Of the 176 documents chosen (see Figure 1), the following information was extracted from each database: (1) authors, (2) names, (3) abstracts, (4) year of publications, (5) type of sources, (6) number of citations, and (7) type of documents.
In some cases, journals are indexed in both databases, Scopus and WoS. For this reason, the next step was to identify duplicated documents that were found in both databases; eight duplicated documents were deleted and 168 documents were eligible for consideration. Finally, an analysis was made to detect and exclude documents with face-to-face components (blended, flipped classroom). Thus, 78 documents were discarded, and a total of 90 documents were considered in the investigation (see Figure 1 to consult the analysis and classification procedure).
The 90 documents selected were then grouped into the three main research perspectives: quantitative, qualitative, or mixed methods. To address other questions of the present investigation, the classification scheme of educational research designs provided by Creswell (2007Creswell ( , 2012, and Creswell and Poth (2018) was used, in addition to Hurtado's (2010) research approach classification. Figure 2 shows the different alternatives for perspectives, designs, and approaches sought in the review. On the other hand, the specific thematic lines were identified in an emergent way from reading, coding, and classifying the thematic contents of the abstracts of the selected works. In the following section, the results obtained are presented.   Table   3 shows the sets of articles according to their type and year of publication.

In Which Country Were the Works Published in the Period Indicated?
To know in which countries the research on the academic engagement in MOOCs was published, the places of affiliation of the first authors in the selected documents were identified; the geographical distribution is presented in Figure 4. The results show that the research of the construct is present in the five continents, with the largest number of articles published in the United States of America (21), the United Kingdom (16), and Australia (11).

Who Are the Authors of the Most Cited Documents?
A total of 9 documents have 11 or more citations reported in the databases, 5 of them from the Scopus database and 4 from WoS. As shown in Table 4

What Documents Are Referenced Most Frequently?
To answer this research question, the references of the 90 selected documents were obtained. Once the database was created, the references that were not in APA format were eliminated, and a total of 2,131 references formed the database to be analyzed. Table 5 shows the references with the highest frequency used for the documents.  As shown in Table 6, the journals in which research on the construct of academic engagement is published most frequently are in the Quartile 1 with the highest level of impact, also, their h-indexes are greater than 24. This indicates the quality and quantity with which the scientific works of the researchers in the area are being published. Of the studies in the area, 51% correspond to qualitative research, 38% to quantitative research, and 10% to mixed method studies. Figure 5 summarizes the educational research perspectives identified in the documents.

Figure 5. Perspectives of educational research used in documents.
Following the classification scheme of educational research designs provided by Creswell (2007Creswell ( , 2012 and Creswell and Poth (2018), it was found that most of the documents correspond to correlation research (31%), followed by phenomenological research (23%), case studies (18%), mixed method studies (9%), surveys (6%), and grounded theory (4%). The least recurrent were the narrative studies (3%) and the experimental investigations (2%). In addition to the previous classification, and as the objective of this research, the Systematic Review category is reported with 3% of production. Figure 6 shows the designs in the mapped production.  documents (those that were available in their full-text version), were also read. An exploratory approach, which seeks to know more about some unknown phenomenon, was found in 32% of the documents; 23% of the documents look for explanations of relationships between factors to determine what will be the future behavior or the trend of that event, that is, they follow a predictive approach; 20% of the documents follow a descriptive approach; in 11% of the documents, two or more groups are studied and their behavior compared, situating them in the comparative approach; 4% of the documents follow an evaluative approach; 3% seek to understand the situations in terms of the relationships of their components following an analytical approach; and 3% corresponds to confirmatory approaches in which hypotheses are tested. With 2% and 1% are the interactive and projective approaches.   Table 8 presents the sets of documents that belong to each approach and in each research path.

What Type of Instruments Are Most Used in the Study of Academic Engagement in MOOCs Participants?
To answer this research question, the method section of the documents found in full text (36) was read, and 4 documents that explicitly mentioned the data collection instruments in the abstract were added. In total, the instrumentation used in 40 documents is reported in this section.
Learning analytics, questionnaires, interviews, and surveys were the most commonly used instruments (see Table 9). It is also worth noting that social networks (Facebook and Twitter) and new technologies in education such as eye-tracking were also used to measure the construct of academic engagement. Eye-tracking A89 Note. More than 40 instruments were found since each document used between one and three instruments.
Some of the less frequent thematic lines in the MOOCs construct research were: eye-tracking, perseverance, multitasking, gender, evaluation, curriculum design, credits, coaching, and access to Higher Education. The categorization of documents by thematic line type can be found in Table 10.

Conclusions
Although in a SMS the decisions about the search nucleus and the limits are chosen by the authors, an arbitrary choice, the mappings are at the upper end of the spectrum of reliability in an investigation to gain a vision of the state of the art of a research topic (Cooper, 2016;Perryman, 2016). The results of this research provide useful information about the state of the art of research on the construct of academic engagement in massive and open online environments.
The results confirm that the production of the construct is increasing and, as in the research of Raffaghelli et al. (2015) and Veletsianos and Shepherdson (2016), in the study of academic engagement this same trend is forecast for 2018. Until now, research on the construct is reported more frequently in journals than in conference proceedings/or reading books. The main journals in which it is published are in the quartile 1 (Q1) with the highest position and highest impact factor according to the Journal Citation Report, with hindexes greater than 24. This speaks not only of the current impact of the issue but also of the quality in which the results are being disseminated.
This research also sheds light on which documents have the greatest incidence in the area, in addition to providing information about the authors and documents with the highest number of citations. The three categories in which the most referenced documents are grouped are: (1) instructional design, (2) how to achieve learning, and (3) motivation and persistence. This tells us that research in this area is only recently emerging, a finding consistent with the research of Joksimovic et al. (2018). This can also be reflected in the documents identified in this research study with the highest number of citations (see Table 6). In these documents, and according to Joksimovic et al. (2018), researchers and professors interested in online education are searching for a framework for the academic engagement construct, one that may provide infrastructure as well as allow for comparison and contrast of the different dimensions of the engagement and pedagogical practices in MOOCs, and thus lead to a greater scientific understanding of how learning happens at scale.
Most of the documents selected in this study correspond to qualitative research that seeks to describe a phenomenon to understand it in depth. This result could be due to the fact that the theoretical and conceptual frameworks for online learning environments are still in development (de Barba, et al., 2016;Gašević et al., 2015;Greene et al., 2015). However, following the methodological design classification scheme of Creswell (2007Creswell ( , 2012 and Creswell and Poth (2018), 31% of the documents have correlational designs. The objective of predicting the success of students in MOOCs, not only to construct predictive models but also to explain the variance in diverse dependent variables of interest, is a very relevant objective to incorporate interventions for the improvement of these learning environments. The previous results contrast with the research carried out by Veletsianos and Shepherdson (2016) in which they report that research in MOOCs follows a quantitative approach with automated methods; however, the results support the research of Raffaghelli et al. (2015) and Bozkurt et al. (2017) which argue that the majority of MOOCs research consists of conceptual/descriptive studies. One limitation of this result is that, of the 90 documents selected for this SMS, only 36 were available in their full-text version. Therefore, it was only possible to read the method section of 36 out of 90 documents. The reading of only the abstract and title of the rest of the documents, on some occasions, may not have accurately reflected the methodological designs used.
Learning analytics emerged as the main source of data collection in the area, followed by questionnaires, interviews, surveys, and forums. However, other more easily accessible methods of data collection emerged, such as the use of the social networks Twitter and Facebook. Methods that might be considered more technological also emerged, such as eye-tracking. The systematic study of the dimensions and variables used to measure the construct is an interesting topic to study in future research.
Although the SMS was carried out in a short time (2015)(2016)(2017)(2018), the results showed that the research of the academic engagement construct seems to be distributed and have a presence in the five continents, with the largest number of articles published in the United States of America (23%); this result was expected since the most popular platforms were developed in that country (e.g., Edx, Coursera, Udacity). These statements differ from the research of Veletsianos and Shepherdson (2016) in which they report that 80% of MOOCs studies were published in North America and Europe. In this case, in the research of academic engagement, only 56% were published in these two areas, Asia and Oceania participated with 16% and 20% respectively. An interesting question for future research would be to investigate which topics are addressed in the different geographical regions.
Some of the research trends that were identified within the study at hand were also identified within previous research. For example, Ebben and Murphy (2014) (2) participation patterns, which tells us about the interest of researchers to find empirical evidence about the major challenges of MOOCs with respect to retention (Greene et al., 2015), desertion (Halawa et al., 2014), motivation (Kizilcec & Schneider, 2015), and a design that enhances all of the above (Conole, 2015).
Although less frequent, themes that arose within the research reviewed that could be of interest for future research include the study of perseverance, the production of digital media, policies of access to Higher Education, eye-tracking, the competence of multitasking, and mentoring or coaching. The latter was also identified by Raffaghelli et al. (2015) as a deep area in need of research on ways to improve learning outcomes in these environments.
In summary, the results of this study contribute to the investigation of online, massive and open learning environments in two ways. First, the research that has been carried out on academic engagement in said environments was identified; and, second, it provides the academic community with a better understanding of the opportunities for future research, identifying relevant issues and challenges in the area.
Given that research in these environments is a topic with growing academic activity, this work recognizes the importance of new frames of reference that strengthen the knowledge we have about mass learning.
Undoubtedly, the gaps and research challenges in MOOCs cannot be achieved without ambition for a better understanding of the academic engagement construct.