International Review of Research in Open and Distributed Learning

Volume 22, Number 4

November - 2021

What We Talk About When We Talk About Virtual Learning Environments

Michael Flavin¹ and Aditi Bhandari^2
1School of Global Affairs, King’s College, London; ²Independent Researcher

Abstract

This is a literature review analysing articles published on virtual learning environments (VLEs), otherwise known as learning management systems, in higher education in the years 2014-2018. The Web of Science database was used to identity relevant articles over this five-year period. The sample comprises 99 peer-reviewed, academic journal articles. A coding sheet is used to analyse each article, identifying the research method, the classification of research (macro, meso, or micro), the focus of research (students, instructors, or both), and, where applicable, the specific brand of VLE. Most output on VLEs is found to be quantitative, to focus on students and on the micro level of learning and teaching, to not have a clear theoretical focus, to not specify which brand of VLE is used, and to be produced in affluent countries. This article adds to the understanding of VLE research by identifying the most frequent foci of research on VLEs, as well as identifying areas that have been under-researched.

Keywords: virtual learning environment, learning management system, higher education, educational technology, literature review, Web of Science

Introduction

Virtual learning environments (VLEs), otherwise known as learning management systems, are established technologies that have been supporting learning and teaching in higher education for over 20 years. Newman et al. (2018), in a large-scale survey of 37,720 students in the United Kingdom, showed 75% of students relying on their VLE to support coursework. In a further large-scale survey in the UK, 72% of higher education students stated they relied on their VLE to do their coursework (Langer-Crame et al., 2019). Because the VLE is embedded in higher education, and because distance learning has increasingly become the norm in the age of the COVID-19 pandemic, now is a timely point at which to reflect on studies on VLEs with a view to informing possible research hereafter, in part by identifying areas that have been under-researched to date.

In their early days, VLEs offered the possibility of innovation, even transformation. According to Dutton et al. (2004), “VLE systems began to diffuse widely in the late-1990s and quickly became a status symbol of innovation” (p. 135). By allowing anytime, anywhere access, VLEs had the potential to transform learning and teaching, enabling synchronous and asynchronous collaboration irrespective of spatial and temporal boundaries. However, VLEs are often used in practice in limited ways to store content (Gordon, 2014; Rienties et al., 2016). Newman et al. (2018) note that 55% of students in their sample said VLEs were well designed, but only 26% enjoyed using the collaborative features. The limited usage of VLEs (relative to their capabilities) is not necessarily a problem: Rodriguez-Ardura and Meseguer-Artola (2016) argue that VLEs help users achieve learning goals (p. 1046). That said, and while VLEs get jobs done in higher education, the gap between their potential and their actual use is considerable.

This article, a literature review with analysis, undertakes a survey of articles published in academic journals on VLEs in higher education in 2014-2018, identified through the Web of Science database. The specific questions considered are the following:

• What is the pattern of academic journal articles on VLEs over the period 2014-2018?
• What can be deduced about VLE scholarship in higher education from published work over the 2014-2018 period?
• What areas of inquiry are under-explored in research on VLEs?

The questions enable exploration of what the studies have to say about the status of the VLE in higher education; the distribution of scholarship on VLEs by country and by academic journal; and what aspects of VLEs are under-researched. Other issues explored in this paper include geopolitical aspects of the pattern of scholarship on VLEs and the specific brand of VLEs used. This study also examines whether VLE research focuses on the micro level of teaching and learning, the meso level of managerial and organisational aspects of VLEs, or the macro level of VLEs viewed through specific theoretical lenses or as part of education systems.

Literature Review

This section summarizes research relevant to the current study, encompassing both articles on VLEs and on technology-enhanced learning more broadly. It identifies articles with research questions similar to the present study, and with similarities in research interests, too, in order to compare findings.

In an early article, Paulsen (2003) surveyed VLE provision at 113 European higher education institutions using 52 different VLEs, including in-house systems. The research was undertaken in late 2001 and early 2002, before Moodle was available. The most popular system was WebCT (20 institutions), followed by a Norwegian system, ClassFronter (16 institutions), followed by Blackboard (14 institutions). Blackboard and WebCT merged in 2005 and now trade as Blackboard.

VLEs relocate didactic modes of teaching to the online environment, as reflected in scholarship. Fry and Love (2011) interviewed lecturers in business, whose metaphors for VLEs included “security blanket,” “crutch,” and “electronic filing cabinet” (p. 54), underlining the extent to which VLEs were used to store content. Students’ and lecturers’ use of VLEs as storage facilities may be a choice. However, it may also be a consequence of VLEs’ design and how they are deployed by universities: McGill and Hobbs (2008) argue, “Student use of a VLE is primarily as an end-user. ... Instructors design the interface for their course and implement different levels of functionality” (p. 192). In addition, Naveh et al. (2010) argue that “instructors can maintain their conservative teaching habits. ... From an organizational perspective, this can be done at low cost, yielding relatively high student satisfaction” (p. 132).

VLEs are customarily produced in affluent countries but exported worldwide. Jones (2014) argues that learning technologies “are largely the outcome of design and development carried out by multi-national US based corporations” (p. 170). Moodle’s head office is in Australia, but the other main VLE provider, Blackboard, is based in Washington, DC. Sakai was developed by a group of universities in the United States. WebCT, a major player in the VLE market until it was taken over by Blackboard, also began in the United States. Zeide and Nissenbaum (2018) argue, of VLE vendors, “By adopting commercial marketplace norms, these providers undermine core functions and values of education” (p. 280). Moreover, the production of VLEs in affluent countries can lead to colonial pedagogy whereby the design features of VLEs prompt culturally specific forms of learning.

Moccia (2016) argues, “Technologies are always expensive so leaders in higher education have to bet on where to invest their sometimes-rare resources” (p. 32). A large-scale survey in the UK showed the VLE receiving most institutional investment (Jenkins et al., 2018). Castaneda and Selwyn (2018) argue, “The use of digital technology in higher education is now a multi-billion dollar business ... the commercial design of educational systems and software increasingly shapes the forms of teaching and learning that take place in universities” (p. 6). VLEs are a lucrative market because of their ubiquity. In addition, VLEs’ design influences the kind of learning and teaching that takes place in higher education.

Latchem (2006) undertook a survey of 374 items (265 articles, 86 colloquia, and 23 editorials) published in the British Journal of Educational Technology, 2000-2005, arguing that educational technology needed more theory-driven research. Latchem (2006) also showed that “the vast majority of the articles reflected work and thought in western countries or overseas universities whose curricula and methodologies are essentially western-influenced” (p. 509), suggesting the use of VLEs is political as well as technical, reproducing geopolitical structures and imbalances of wealth and power.

Baydas et al. (2015) used content analysis to study 1,255 articles published in the British Journal of Educational Technology and Educational Technology, Research and Development, from 2002 to 2014, finding that the most common research method was quantitative. The articles included research on VLEs but were not limited to articles on VLEs. Risquez et al. (2015) reported on a survey of VLE use in Ireland, which has been taking place on a rolling basis since 2008. Students wanted VLEs that were easy to navigate and identified lecturers’ lack of use as a barrier to engagement. A later study of survey responses from 521 staff in Irish higher education institutions found the most common use of VLEs was as content repositories and as a channel for communication (Farrelly et al., 2018). Cobo et al. (2014), in a study in Spain, and Oproiu (2015), in a study in Romania, found similar VLE usage patterns. Ngai et al. (2007), in research conducted in Hong Kong higher education from a Technology Acceptance Model perspective (Davis, 1989), found ease of use and usefulness to be the dominant factors affecting the use of VLEs. Van Raaij and Schepers (2008), also using the Technology Acceptance Model, studied 45 students in an executive MBA programme in China, finding that perceived usefulness had a direct effect on VLE usage.

Shih et al. (2008) undertook a content analysis of journal articles concerned with cognition in e-learning published in five journals (British Journal of Educational Technology, Innovations in Education and Teaching International, Computers and Education, Educational Technology Research and Development, and Journal of Computer Assisted Learning), recording year of publication, journal, and research topic; they found most of the articles took a quantitative approach. Hung (2012) analysed the abstracts of 689 articles on e-learning published between 2000 and 2008, with a research question investigating trends and patterns of e-learning research. Hung (2012) used the Web of Science database and limited the research to journal articles and proceedings papers. The five most prolific countries for e-learning research were England (conflated with the UK), the US, Taiwan, China, and Germany. The five most prolific journals were Lecture Notes in Computer Science, Computers and Education, British Journal of Educational Technology, Educational Technology and Society, and Lecture Notes in Artificial Intelligence.

Sosa et al. (2019) surveyed research on emerging technologies, 2006-2016, though their survey was not limited to higher education. From a sample of 288 studies, they found that students were the most frequently studied population in individual papers. Very few of their studies focused on instructors. Other features they categorized included year of publication and research methodology, both of which are categories in the present study. Hsu et al. (2013) examined the abstracts of 2,997 articles on education technology published between 2000 and 2010. The categories for their analysis included productivity by journal and by country, and they also analysed data by year of publication.

Zawacki-Richter et al. (2009) surveyed 695 articles on distance education published between 2000 and 2008. They classified research into three categories: macro (systems and theories), meso (management and organisation), and micro (teaching and learning), finding a shortage of articles at the meso level. Their categories have been adapted for the coding sheet supporting the present study (see Appendix B) because the categories enable a useful subdivision of research. Zawacki-Richter et al. (2009) found most of their sample adopted a qualitative approach, identified gaps in distance learning literature around innovation and intercultural aspects of distance learning, and noted 80% of their sample came from five countries: the US, Canada, the UK, Australia, and China.

The authors of the present study have been unable to find a survey of articles on VLEs later than 2016, and that article covers the period 2005-2011 (de Oliveira et al., 2016), identifying 14, mainly quantitative, articles on VLEs. Existing research is weighted towards particular and frequently affluent contexts—contexts that are also dominant in the design and production of VLEs. The present study addresses a research gap by providing a more recent survey of research published on VLEs in higher education. The present study also contributes to scholarship on VLEs by identifying popular areas for research, under-researched aspects of VLEs in higher education, and weighting of scholarship on VLEs towards affluent countries, reflecting the dominance of those countries in VLE design and implementation. The breadth of time frame for the present study, while limited, is similar to that in Latchem (2006) and de Oliveira et al. (2016).

Research Method

The Web of Science database, also used by Hung (2012), is used in this study. The following search terms were used: virtual learning environment and higher education; learning management system and higher education; virtual learning and higher education; virtual learning, higher education, and innovation; virtual learning environment, e-learning, and higher education; and virtual learning environment, ICT (information and communications technology), and innovation. The research was limited to articles published in English. A coding sheet was devised (see Appendix B) to identify core features of the articles. It had originally been hoped to pose a further research question relating to the extent to which scholarship advocated innovative usage of VLEs, but institutional funding given to support the project was reduced, and the research questions were scaled down accordingly. It was noted that the term learning management system was used as an alternative to virtual learning environment to describe the same type of learning platform (though Pinner [2014] distinguishes between the two in an opinion piece), but virtual learning environment is used in this paper for consistency.

The initial search, undertaken by the second author, yielded 2,211 articles. Following the initial identification, the titles, abstracts, and keywords were screened for eligibility to the study, identifying articles in which VLEs were a significant element, using inter-rater discussion (comparative verification on the part of the authors), a method also used in a survey of articles on educational technology by Baydas et al. (2015). The keyword virtual learning proved unhelpful as it identified articles concerned with computer simulations or virtual worlds. The screening process resulted in a total of 99 articles selected for coding. A similar approach was adopted by Sung et al. (2016), who performed a meta-analysis of articles published on mobile learning over the period 1993-2013. This study has drawn selectively from the methodology of previous studies in order to better address the specific research questions: disaggregating the data into macro, meso, and micro categories highlights patterns in publications on VLEs and enables conclusions to be drawn, including the identification of research areas that have been relatively unaddressed to date.

For this paper, the analysis categorized the sample in terms of years published, country of origin, areas of focus, brand of VLE, journal, research method, and theoretical perspective. All the articles making up the sample are cited in Appendix A, subdivided by year of publication (2014-2018).

Results and Discussion

The year-by-year distribution of the sample is shown in Table 1.

Table 1

Distribution of the Sample by Year

With the exception of 2014, the number of articles published each year is consistent, varying between 19 and 25. There was no obvious explanation for the shortage of articles in 2014. Data on years prior to 2014 were not collected, and therefore it is not possible to state if 2014 is an outlier or in line with years outside the sample.

Aggregating the data over five years and taking the home higher education institution of the first author as means of identification, the most articles were published in the UK (14), Spain (12), and Australia (11). Countries with two or more journal articles over the period are summarized in Figure 1.

Figure 1

Countries of First Author (Two Articles or More)

There were 26 countries with one article each, including Colombia, Germany, Kenya, the Kyrgyz Republic, and Pakistan. The geographical spread of articles on VLEs over the period from 2014 to 2018 was extensive, but with a few hot spots (notably, the UK, Spain, and Australia). The output was largely though not wholly weighted towards affluent countries. The relatively low count from the US was noticeable. Latchem (2006), in a survey of articles in the British Journal of Educational Technology covering the period 2000-2005, found 40% of the articles were from the UK, 12% from the US, and 8% from Australia (p. 505), though Latchem’s survey was of all articles published in the British Journal of Educational Technology (articles, editorials, and colloquia) and not solely those pertaining to VLEs. Hsu et al. (2013) also identified the top 10 countries for publication in the field of educational technology, 2000-2010. Five of their top ten countries also appear in Figure 1: USA, UK (though Hsu et al. [2013] conflate England with the UK), Australia, Turkey, and Germany. Five of their top ten countries do not feature in Figure 1: Taiwan, the Netherlands, Canada, Greece, and Singapore. The present study identifies Spain as the second most productive country for articles on VLEs in 2014-2018. This is at odds with other surveys of productivity in research, though the other surveys were of technology-enhanced learning articles more widely rather than being focused on VLEs. No obvious reason was discernible for Spain’s relative prominence in the field.

The division between macro, meso, and micro, as per the categories defined by Zawacki-Richter et al. (2009), and as recorded on the coding sheets, features in Table 2.

Table 2

The Sample Divided by Categories

Note. Based on categories defined by Zawacki-Richter et al. (2009).

Most articles focused on learning and teaching with VLEs. However, the second category, numerically, was the meso level, which contrasts with Zawacki-Richter et al.’s (2009) study, which found a shortage of articles at the meso level. The fewest number of articles looked at VLEs as part of national higher education systems or as validations or refutations of particular theoretical perspectives. Latchem (2006) divided his sample into empirical enquiry (78%) and theoretical enquiry (22%), indicating a similar tendency to prioritize studies of practice over explorations of theory.

Each article was evaluated in terms of whether its research focus was on students, instructors, or both (recorded on the coding sheets). A small number of articles did not fall into any of these categories in instances where the focus was on administrators or education leaders. The distribution of the sample in terms of the research focus is shown in Table 3.

Table 3

The Sample Divided by Research Focus

A sizeable number of the articles focus on students alone, and a further 18 focus on students in conjunction with instructors, totalling 77 articles in the sample. Sosa et al. (2019), in their study of literature on emerging technologies in 2006-2016, also identified a research focus on students; 152 articles from their overall sample of 288 focused on students, of which 117 focused on higher education students (p. 136). In the present article, a shortage of articles was found examining VLEs from the perspective of instructors only (lecturers or tutors), identifying a potentially under-researched area of practice with VLEs, especially as instructors can often be responsible for VLEs’ content (McGill & Hobbs 2008).

Over half the articles in the sample did not specify which brand of VLE was used. The identification of the specific VLE may not have been essential for the research, but the omission is surprising in view of the fact VLEs comprise significant expenditure for universities (Castenada & Selwyn, 2018; Jenkins et al., 2018; Moccia, 2016), and information on which brand of VLE is used could have value. The distribution of the sample, in terms of the particular VLE, is listed in Table 4.

Table 4

Specific Virtual Learning Environments (VLEs) Used in the Sample

The number of unspecified VLEs could potentially limit research by neglecting the specific brand. That said, commercial VLEs are more conspicuous by their similarities than their differences, offering comparable capabilities regarding information storage and communication channels, and hence, results from published studies may have use value irrespective of which brand of VLE is used. Benner and Tripsas (2012) argue that competing alternatives often converge on a dominant design in a process of mimetic isomorphism, and hence the brand of VLE may make no significant difference. The standardization of VLEs as products imposes, in practice, a digital hegemony and a standardized, reductive, and culturally exclusive mode of learning and teaching, in view of the dominance of affluent countries in the design and sale of VLEs.

The coding sheet recorded the journal title for each article. The numbers of articles from individual journals are featured in Table 5. Only journals with two or more articles are included. Table 5 also positions the number of articles published in each journal alongside that journal’s position in the Scimago Journal and Country Rank (SJCR) 2018, where applicable.

Table 5

Journals in the Sample Featuring Two or More Relevant Articles, and the Journals’ Scimago Journal and Country Rank (SJCR) 2018

Most of the journals in Table 5 are slanted towards an interest in technology-enhanced learning. Only six have a more generic interest in education. Research on VLEs gets focused in journals relating closely to technology-enhanced learning rather than being diffused more widely through education journals, suggesting that VLE research is of more interest to a niche community than to educators as a whole. That said, we recognize that journals are not unchanging entities but, instead, change direction and editorial personnel.

Half of the journals in Table 5 featured in the top 50 journals identified under the heading “E-Learning” (SJCR, 2018). Only three of the journals in Table 5 featured in the SJCR top 10, suggesting research on VLEs is not concentrated in high-impact journals and is distributed more widely through technology-enhanced learning publications. Identifying the impact factor of journals in which educational technology research features was also part of Hsu et al.’s (2013) methodology.

Most of the articles adopted a quantitative approach to data gathering and analysis, as shown in Table 6.

Table 6

Research Methods for the Sample

The eight articles that did not belong to any of the categories were either position papers or literature reviews. This contrasts with Sosa et al.’s (2019) findings that most of the research surveyed was qualitative, though the research was on emerging technologies rather than focused on VLEs and was not limited to studies of higher education.

The paucity of theorized accounts of VLEs was noticeable. Very few of the articles declared an explicit theoretical lens in their title, abstract, or keywords. Instances of two or more articles adopting a specific theoretical approach are summarized in Figure 2.

Figure 2

Theories Used to Analyse Articles From the Sample

The most popular theoretical lens was the Technology Adoption Model (Davis, 1989). This is not surprising, as this model analyses how people engage with technologies and the criteria they apply to evaluate technologies’ usefulness, rooted in Fishbein and Ajzen’s (1977) theory of reasoned action. The Technology Adoption Model is well suited to analysing engagement with VLEs.

The Community of Practice Theory (Lave & Wenger, 1991; Wenger, 1998) is suitable for analysing VLEs from a social perspective and monitoring learners’ progress in learning communities, as the theory focuses on how learning communities evolve and on how people establish effective identities in communities of practice. Data collected by VLEs can illuminate patterns of engagement within communities. Activity Theory (Leontiev, 1978, 1981) is more suitable for analysing technical and social aspects of technologies and how they interact, by anatomizing systems into interactive nodes. Grounded Theory (Strauss & Corbin, 1998) takes a different approach, gestating a theory from the data gathered.

The data suggest that certain, generally affluent countries tend to dominate scholarship on VLEs, as they dominate production of VLEs (Jones, 2014). Furthermore, articles on VLEs tend to get published in technology-enhanced learning journals, such as Computers & Education and the British Journal of Educational Technology, both of which also featured as prolific journals in the study by Hung (2012), rather than get published in education journals more widely, despite the ubiquity of VLEs.

Conclusion

This article, a literature review with analysis, has surveyed research on VLEs in higher education over the period 2014-2018, identifying patterns in the sample as well as areas that have been under-explored, with a view to informing future research.

Research on VLEs over the period from 2014 to 2018 tends to be quantitative. It also tends to be focused on students and on the micro level of learning and teaching. It is not likely to have an explicit theoretical focus, an issue identified by Latchem (2006) in a review of 374 published items in the British Journal of Educational Technology, as well as by Oliver (2011) in relation to research in technology-enhanced learning published from 2001 to 2011.

Some countries published a higher-than-average volume of research on VLEs (UK, Spain, and Australia) over the period studied, and two journals averaged more than one article per year on VLEs (International Journal of Emerging Technologies in Learning and Computers & Education). VLEs are embedded technologies in higher education, but they are not a frequent subject of interest. Furthermore, articles on VLEs leave numerous areas under-explored, including comparison of active and relatively inactive geographical concentrations of research, which may, in turn, reflect unequal relationships in the design and implementation of VLEs and technology-enhanced learning more widely. Ferreira et al. (2020) argue that technological solutionism has supported marketization of the education system in Brazil, comprising a form of digital colonialism, which this paper largely reinforces through a survey of articles on VLEs that indicates dominance by affluent countries in systems and research.

The data from the present study indicate opportunities exist for more theorized approaches to VLEs because different theoretical lenses can illuminate different aspects of technologies and practice. There are also opportunities for more qualitative approaches to data gathering and analysis, as well as for research focusing on instructors, who play a notable role in designing interfaces and implementing levels of VLE functionality (McGill & Hobbs, 2008). In addition, opportunities exist to explore in-depth interactions on VLEs, using content analysis as a methodological approach, as well as to extend research on VLEs beyond the end of 2018.

A limitation of the present study is that despite the use of specific search terms, some relevant articles are likely to have been missed. One database (Web of Science, originating in the US) was used, but additional databases would probably have produced additional articles. However, the articles used are cited in Appendix A, helping to identify notable omissions for the benefit of future researchers, and the sample comprises a range of relevant articles from across the world, albeit all the articles are published in English. A further limitation of the present study is the relatively short period of time surveyed, but similar time periods were studied by Latchem (2006) and de Oliveira et al. (2016).

This article contributes to research on VLEs by identifying prevalent practices regarding research in 2014-2018 and identifying gaps. VLEs are embedded technologies in higher education, but research on their use is weighted towards particular methodological approaches (quantitative) and particular stakeholders (students). Moreover, VLE scholarship (and VLE design) is weighted towards affluent countries. Segooa and Kalema (2015) argue that “many VLEs are not contextualized to meets the needs of educational institutions in developing countries ... elearning implementation should focus more on the social contexts rather than the technological solution” (pp. 353-354). Surveys of publications on VLEs show considerable continuity of findings in terms of research methodologies and areas of focus. That said, this paper also notes the dominance of Western companies in the provision of VLEs and, consequently, the application of Western pedagogic models globally, comprising digital colonialism. This research thus argues that geopolitical aspects of implementation and usage of VLEs are under-explored. Moreover, the VLE is an object of increased interest because it has become a dominant mode of learning and teaching during the COVID-19 pandemic, with more students learning at a distance. There is more to talk about when we talk about VLEs.

References

Baydas, O., Kucuk, S., Yilmaz, R. M., Aydemir, M., & Goktas, Y. (2015). Educational technology research trends from 2002 to 2014. Scientometrics, 105(1), 709-725. https://doi.org/10.1007/s11192-015-1693-4

Benner, M. J., & Tripsas, M. (2012). The influence of prior industry affiliation on framing in nascent industries: The evolution of digital cameras. Strategic Management Journal, 33(3), 277-302. https://doi.org/10.1002/smj.950

Castaneda, L. & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15(22). https://doi.org/10.1186/s41239-018-0109-y

Cobo, A., Rocha, R., & Rodriguez-Hoyos, C. (2014). Evaluation of the interactivity of students in virtual learning environments using a multicriteria approach and data mining. Behaviour and Information Technology, 33(10), 1000-1012. https://doi.org/10.1080/0144929X.2013.853838

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008

de Oliveira, P. C., Cunha, C. J. C., & Nakayama, M. K. (2016). Learning management systems (LMS) and e-learning management: An integrative review and research agenda. Journal of Information Systems and Technology Management, 13(2), 157-180. https://doi.org/10.4301/S1807-17752016000200001

Dutton, W. H., Cheong, P. H., & Park, N. (2004). An ecology of constraints on e-learning in higher education: The case of a virtual learning environment. Prometheus, 22(2), 131-149. https://doi.org/10.1080/0810902042000218337

Farrelly, T., Raftery, D., & Harding, N. (2018). Exploring lecturer engagement with the VLE: Findings from a multi-college staff survey. Irish Journal of Technology Enhanced Learning. https://research.thea.ie/handle/20.500.12065/2669

Ferreira, G. M. D. S., Rosado, L. A. D. S., Lemgruber, M. S., & Carvalho, J. D. S. (2020). Metaphors we’re colonised by? The case of data-driven educational technologies in Brazil. Learning, Media and Technology, 45(1), 46-60. https://doi.org/10.1080/17439884.2019.1666872

Fishbein, M., & Ajzen, I. (1977). Belief, attitude, intention, and behavior: An introduction to theory and research. Addison-Wesley. https://people.umass.edu/aizen/f&a1975.html

Fry, N., & Love, N. (2011). Business lecturers’ perceptions and interactions with the virtual learning environment. International Journal of Management Education, 9(4), 51-56. https://doi.org/10.3794/ijme.94.325

Gordon, N. (2014). Flexible pedagogies: Technology-enhanced learning. Higher Education Academy. https://www.hv.se/globalassets/dokument/stodja/paper-theme-3.pdf

Hsu, Y. C., Hung, J. L., & Ching, Y. H. (2013). Trends of educational technology research: More than a decade of international research in six SSCI-indexed refereed journals. Educational Technology Research and Development, 61(4), 685-705. https://doi.org/10.1007/s11423-013-9290-9

Hung, J. L. (2012). Trends of e‐learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology, 43(1), 5-16. https://doi.org/10.1111/j.1467-8535.2010.01144.x

Jenkins, M., Walker, R., & Voce, J. (2018). The rhetoric and reality of technology-enhanced learning developments in UK higher education: Reflections on recent UCISA research findings (2012-2016). Interactive Learning Environments, 26, 858-868.

Jones, C. R. (2014). The politics of networked learning in an age of austerity. In S. Bayne, C. Jones, M. de Laat, T. Ryberg, & C Sinclair (Eds.), Proceedings of the 9th International Conference on Networked Learning (pp. 170-177). http://ljmu-test.eprints-hosting.org/id/eprint/150/

Langer-Crame, M., Newman, T., Beetham, H., Killen, C., & Knight, S. (2019, January). Digital experience insights survey 2019: Findings from students in UK further and higher education. JISC. http://repository.jisc.ac.uk/7202/1/digital-experience-insights-survey-anz-2018.pdf

Latchem, C. (2006). Editorial: A content analysis of the British Journal of Educational Technology. British Journal of Educational Technology, 37(4), 503-511. https://doi.org/10.1111/j.1467-8535.2006.00635.x

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press. https://www.cambridge.org/ca/academic/subjects/psychology/developmental-psychology/situated-learning-legitimate-peripheral-participation?format=PB&isbn=9780521423748

Leontiev, A. N. (1978). Activity, consciousness and personality (Trans. M. J. Hall). Prentice Hall. https://www.marxists.org/archive/leontev/works/1978/activity-consciousness-personality.pdf

Leontiev, A. N. (1981). Problems of the development of the mind. Progress Publications.

McGill, T. J., & Hobbs, V. J. (2008). How students and instructors using a virtual learning environment perceive the fit between technology and task. Journal of Computer Assisted Learning, 24, 191-202. https://doi.org/10.1111/j.1365-2729.2007.00253.x

Moccia, S. (2016). Managing educational reforms during times of transition: The role of leadership. Higher Education for the Future, 3(1), 26-37. https://doi.org/10.1177/2347631115611278

Naveh, G., Tubin, D., & Pliskin, N. (2010). Student LMS use and satisfaction in academic institutions: The organizational perspective. Internet and Higher Education, 13, 127-133. https://doi.org/10.1016/j.iheduc.2010.02.004

Newman, T., Beetham, H., & Knight, S. (2018). Digital experience insights survey 2018: Findings from students in UK further and higher education. JISC. https://repository.jisc.ac.uk/6967/1/Digital_experience_insights_survey_2018.pdf

Ngai, E. W., Poon, J. K. L., & Chan, Y. H. (2007). Empirical examination of the adoption of WebCT using TAM. Computers & Education, 48(2), 250-267. https://doi.org/10.1016/j.compedu.2004.11.007

Oliver, M. (2011). Learning technology: Theorising the tools we study. British Journal of Educational Technology, 44(1), 31-43. https://doi.org/10.1111/j.1467-8535.2011.01283.x

Oproiu, G. C. (2015). A study about using e-learning platform (Moodle) in university teaching process. Procedia-Social and Behavioral Sciences, 180, 426-432. https://doi.org/10.1016/j.sbspro.2015.02.140

Paulsen, M. F. (2003). Experiences with learning management systems in 113 European institutions. Journal of Educational Technology & Society, 6(4), 134-148. https://www.jstor.org/stable/pdf/jeductechsoci.6.4.134.pdf

Pinner, R. (2014). What is the difference between an LMS and a VLE? eLearning Industry. https://elearningindustry.com/difference-between-lms-and-vle

Rienties, B., Giesbers, B., Lygo-Baker, S., & Ma, H. W. S. (2016). Why some teachers easily learn to use a new virtual learning environment: A technology acceptance perspective. Interactive Learning Environments, 24(3), 539-552. https://doi.org/10.1080/10494820.2014.881394

Risquez, A., Raftery, D., & Costello, E. (2015). The open dataset on students’ perceptions of virtual learning environments in Ireland: Collaborating to listen to the student voice. British Journal of Educational Technology, 46(5), 1070-1074. https://doi.org/10.1111/bjet.12323

Rodriguez-Ardura, I., & Meseguer-Artola, A. (2016). What leads people to keep on e-learning? An empirical analysis of users’ experiences and their effects on continuance intention. Interactive Learning Environments, 24(6), 1030-1053.

Scimago Journal and Country Rank (SJCR). (2018). Subject category: E-learning. https://www.scimagojr.com/journalrank.php?category=3399&wos=true

Segooa, M. A., & Kalema, B. M. (2015). Virtual learning environments for higher learning institutions in developing countries. In 2015 Fifth International Conference on e-Learning (pp. 353-359). https://doi.org/10.1109/ECONF.2015.67

Shih, M., Feng, J., & Tsai, C. C. (2008). Research and trends in the field of e-learning from 2001 to 2005: A content analysis of cognitive studies in selected journals. Computers & Education, 51(2), 955-967. https://doi.org/10.1016/j.compedu.2007.10.004

Sosa, E., Salinas, J., & De Benito, B. (2019). Emerging technologies (ETs) in education: A systematic review of the literature published between 2006 and 2016. International Journal of Emerging Technologies in Learning, 12(5), 128-149. https://doi.org/10.3991/ijet.v12i05.6939

Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd ed.). Sage Publications. https://psycnet.apa.org/record/1999-02001-000

Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252-275. https://doi.org/10.1016/j.compedu.2015.11.008

Van Raaij, E. M., & Schepers, J. J. L. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50, 838-852. https://doi.org/10.1016/j.compedu.2006.09.001

Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge University Press. https://doi.org/10.1017/CBO9780511803932

Zawacki-Richter, O., Bäcker, E. M., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. The International Review of Research in Open and Distributed Learning, 10(6), 21-50. https://doi.org/10.19173/irrodl.v10i6.741

Zeide, E., & Nissenbaum, H. (2018). Learner privacy in MOOCs and virtual education. Theory and Research in Education, 16(3), 280-307.

Appendix A

Articles Used

2014

Agudo-Peregrina, Á. F., Hernández-García, Á., & Pascual-Miguel, F. J. (2014). Behavioral intention, use behavior and the acceptance of electronic learning systems: Differences between higher education and lifelong learning. Computers in Human Behavior, 34, 301-314. https://doi.org/10.1016/j.chb.2013.10.035

Cobo, A., Rocha, R., & Rodríguez-Hoyos, C. (2014). Evaluation of the interactivity of students in virtual learning environments using a multicriteria approach and data mining. Behaviour & Information Technology, 33(10), 1000-1012. https://doi.org/10.1080/0144929X.2013.853838

Dias, S. B., & Diniz, J. A. (2014). Towards an enhanced learning management system for blended learning in higher education incorporating distinct learners’ profiles. Journal of Educational Technology & Society, 17(1), 307-319. https://www.proquest.com/docview/1502989120

Emelyanova, N., & Voronina, E. (2014). Introducing a learning management system at a Russian university: Students’ and teachers’ perceptions. The International Review of Research in Open and Distributed Learning, 15(1), 272-289. https://doi.org/10.19173/irrodl.v15i1.1701

Habib, L., Johannesen, M., & Øgrim, L. (2014). Experiences and challenges of international students in technology-rich learning environments. Journal of Educational Technology & Society, 17(2), 196-206. https://eric.ed.gov/?id=EJ1031020

Han, I., & Han, S. (2014). Adoption of the mobile campus in a cyber university. The International Review of Research in Open and Distributed Learning, 15(6). https://doi.org/10.19173/irrodl.v15i6.1950

Jackson, S., & Fearon, C. (2014). Exploring the role and influence of expectations in achieving VLE benefit success. British Journal of Educational Technology, 45(2), 245-259. https://doi.org/10.1111/bjet.12029

Lara, J. A., Lizcano, D., Martínez, M. A., Pazos, J., & Riera, T. (2014). A system for knowledge discovery in e-learning environments within the European higher education area: Application to student data from Open University of Madrid, UDIMA. Computers & Education, 72, 23-36. https://doi.org/10.1016/j.compedu.2013.10.009

Palmer, S., & Holt, D. (2014). Development of student and academic staff perceptions of the elements of an online learning environment over time. Australasian Journal of Educational Technology, 30(4). https://doi.org/10.14742/ajet.581

Schoonenboom, J. (2014). Using an adapted, task-level technology acceptance model to explain why instructors in higher education intend to use some learning management system tools more than others. Computers & Education, 71, 247-256. https://doi.org/10.1016/j.compedu.2013.09.016

Stantchev, V., Colomo-Palacios, R., Soto-Acosta, P., & Misra, S. (2014). Learning management systems and cloud file hosting services: A study on students’ acceptance. Computers in Human Behavior, 31, 612-619. https://doi.org/10.1016/j.chb.2013.07.002

2015

Carceller, C., Dawson, S., & Lockyer, L. (2015). Social capital from online discussion forums: Differences between online and blended modes of delivery. Australasian Journal of Educational Technology, 31(2), 150-163. https://doi.org/10.14742/ajet.1448

Carruthers, C., McCarron, B., Bolan, P., Devine, A., McMahon-Beattie, U., & Burns, A. (2015). “I like the sound of that”: An evaluation of providing audio feedback via the virtual learning environment for summative assessment. Assessment & Evaluation in Higher Education, 40(3), 352-370. https://doi.org/10.1080/02602938.2014.917145

Colomina, R., & Remesal, A. (2015). Social presence and virtual collaborative learning processes in higher education/Presencia social y procesos de aprendizaje colaborativo virtual en educación superior. Infancia y Aprendizaje, 38(3), 647-680. https://doi.org/10.1080/02103702.2015.1054664

Đurđević Babić, I. (2015). Predicting student satisfaction with courses based on log data from a virtual learning environment: a neural network and classification tree model. Croatian Operational Research Review, 6(1), 105-120. https://doi.org/10.17535/crorr.2015.0009

Hegazy, A. F., Khedr, A. E., & Al Geddawy, Y. (2015). An adaptive framework for applying cloud computing in virtual learning environment at education: A case study of “AASTMT.” Procedia Computer Science, 65, 450-458. https://doi.org/10.1016/j.procs.2015.09.121

Humanante-Ramos, P. R., García-Peñalvo, F. J., & Conde-González, M Á. (2015). Personal learning environments and online classrooms: An experience with university students. IEEE Revista Iberoamericana de tecnologías del aprendizaje, 10(1), 26-32. https://doi.org/10.1109/RITA.2015.2391411

Lewis, E., & Wang, C. (2015). Using an online curriculum design and a cooperative instructional approach to orientate adjunct faculty to the online learning environment. The Journal of Continuing Higher Education, 63(2), 109-118. https://doi.org/10.1080/07377363.2015.1042995

Li, Y., Wang, Q., & Campbell, J. (2015). Investigating gender and racial/ethnic invariance in use of a course management system in higher education. Education Sciences, 5(2), 179-198. https://doi.org/10.3390/educsci5020179

Limniou, M., Downes, J. J., & Maskell, S. (2015). Datasets reflecting students’ and teachers’ views on the use of learning technology in a UK university. British Journal of Educational Technology, 46(5), 1081-1091. https://doi.org/10.1111/bjet.12332

Oproiu, G. C. (2015). A study about using e-learning platform (Moodle) in university teaching process. Procedia—Social and Behavioral Sciences, 180, 426-432. https://doi.org/10.1016/j.sbspro.2015.02.140

O’Shea, S., Stone, C., & Delahunty, J. (2015). “I ‘feel’ like I am at university even though I am online.” Exploring how students narrate their engagement with higher education institutions in an online learning environment. Distance Education, 36(1), 41-58. https://doi.org/10.1080/01587919.2015.1019970

Park, J. Y. (2015). Student interactivity and teacher participation: An application of legitimate peripheral participation in higher education online learning environments. Technology, Pedagogy and Education, 24(3), 389-406. https://doi.org/10.1080/1475939X.2014.935743

Porta Simó, L., Beneito Montagut, R., Melenchón Maldonado, J., & Marín Ametller, A. (2015). Effects of applying the site map principle in an online learning environment in higher education. International Journal of Emerging Technologies in Learning, 10(7), 31-38. https://online-journals.org/index.php/i-jet/article/view/4620

Risquez, A., Raftery, D., & Costello, E. (2015). The open dataset on students’ perceptions of virtual learning environments in Ireland: Collaborating to listen to the student voice. British Journal of Educational Technology, 46(5), 1070-1074. https://doi.org/10.1111/bjet.12323

Sevilla-Pavón, A. (2015). Computer assisted language learning and the internationalisation of the Portuguese language in higher education contexts. Digital Education Review, 28, 37-44. https://eric.ed.gov/?id=EJ1088259

Songkram, N. (2015). E-learning system in virtual learning environment to develop creative thinking for learners in higher education. Procedia—Social and Behavioral Sciences, 174, 674-679. https://doi.org/10.1016/j.sbspro.2015.01.600

Songkram, N., Khlaisang, J., Puthaseranee, B., & Likhitdamrongkiat, M. (2015). E-learning system to enhance cognitive skills for learners in higher education. Procedia—Social and Behavioral Sciences, 174, 667-673. https://doi.org/10.1016/j.sbspro.2015.01.599

Swart, A. J. (2015). Student usage of a learning management system at an open distance learning institute: A case study in electrical engineering. International Journal of Electrical Engineering Education, 52(2), 142-154. https://doi.org/10.1177/0020720915575925

Umek, L., Aristovnik, A., Tomaževič, N., & Keržič, D. (2015). Analysis of selected aspects of students’ performance and satisfaction in a Moodle-based e-learning system environment. EURASIA Journal of Mathematics, Science & Technology Education, 11(6), 1495-1505. https://doi.org/10.12973/eurasia.2015.1408a

2016

Barreto, C. R., & Haydar, J. M. (2016). Pedagogical intercultural practice of teachers in virtual environments. Turkish Online Journal of Distance Education, 17(4), 190-202. https://eric.ed.gov/?id=EJ1116812

Cann, A. J. (2016). Increasing student engagement with practical classes through online pre-lab quizzes. Journal of Biological Education, 50(1), 101-112. https://doi.org/10.1080/00219266.2014.986182

Casquero, O., Ovelar, R., Romo, J., Benito, M., & Alberdi, M. (2016). Students’ personal networks in virtual and personal learning environments: A case study in higher education using learning analytics approach. Interactive Learning Environments, 24(1), 49-67. https://doi.org/10.1080/10494820.2013.817441

Cassidy, S. (2016). Virtual learning environments as mediating factors in student satisfaction with teaching and learning in higher education. Journal of Curriculum and Teaching, 5(1), 113-123. https://doi.org/10.5430/jct.v5n1p113

Cavanaugh, C., Mayberry, J., & Hargis, J. (2016). Participation in the virtual environment of blended college courses: An activity study of student performance. International Journal on E-Learning, 15(4), 423-432. https://www.learntechlib.org/primary/p/147380/

Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96, 42-54. https://doi.org/10.1016/j.compedu.2016.02.006

da Costa, F. R., & Pelissari, A. S. (2016). Factors affecting corporate image from the perspective of distance learning students in public higher education institutions. Tertiary Education and Management, 22(4), 287-299. https://doi.org/10.1080/13583883.2016.1225227

ElShaer, A., Calabrese, G., Casanova, D., & Huet, I. (2016). Building a community of practice for engaging pharmacy students to learn in a collaborative research environment. Currents in Pharmacy Teaching and Learning, 8(5), 698-707. https://doi.org/10.1016/j.cptl.2016.05.001

Guitart, I., & Conesa, J. (2016). Adoption of business strategies to provide analytical systems for teachers in the context of universities. International Journal of Emerging Technologies in Learning, 11(7), 34-40. https://online-journals.org/index.php/i-jet/article/view/5887

Kent, M. (2016). Adding to the mix: Students’ use of Facebook groups and Blackboard discussion forums in higher education. Knowledge Management and eLearning, 8(3), 444-463. https://doi.org/10.34105/j.kmel.2016.08.028

Khlaisang, J., & Mingsiritham, K. (2016). Engaging virtual learning environment system to enhance communication and collaboration skills among ASEAN higher education learners. International Journal of Emerging Technologies in Learning, 11(4), 103-113. https://online-journals.org/index.php/i-jet/article/view/5503

Marks, A., Al-Ali, M., & Rietsema, K. (2016). Learning management systems: A shift toward learning and academic analytics. International Journal of Emerging Technologies in Learning, 11(4), 77-82. https://online-journals.org/index.php/i-jet/article/view/5419

Martins, J., & Nunes, M. B. (2016). The temporal properties of e-learning: An exploratory study of academics’ conceptions. International Journal of Educational Management, 30(1), 2-19. https://doi.org/10.1108/IJEM-04-2014-0048

Meadows, C., Soper, K., Cullen, R., Wasiuk, C., McAllister-Gibson, C., & Danby, P. (2016). Shaping the future of learning using the student voice: We’re listening but are we hearing clearly? Research in Learning Technology, 24. https://doi.org/10.3402/rlt.v24.30146

Mkhize, P., Mtsweni, E., & Buthelezi, P. (2016). Diffusion of innovations approach to the evaluation of learning management system usage in an open distance learning institution. The International Review of Research in Open and Distributed Learning 17(3), 295-312. https://doi.org/10.19173/irrodl.v17i3.2191

Nagy, J. T. (2016). Using learning management systems in business and economics studies in Hungarian higher education. Education and Information Technologies, 21(4), 897-917. https://eric.ed.gov/?id=EJ1100497

Nkonki, V., & Ntlabathi, S. (2016). The forms and functions of teaching and learning innovations on Blackboard: Substantial or superficial? Electronic Journal of e-Learning, 14(4), 257-265. https://eric.ed.gov/?id=EJ1120627

Pedersen, M. K., Skyum, B., Heck, R., Müller, R., Bason, M., Lieberoth, A., & Sherson, J. F. (2016). Virtual learning environment for interactive engagement with advanced quantum mechanics. Physical Review Physics Education Research, 12(1). https://doi.org/10.1103/PhysRevPhysEducRes.12.013102

Power, J., & Kannara, V. (2016). Best-practice model for technology enhanced learning in the creative arts. Research in Learning Technology, 24. https://doi.org/10.3402/rlt.v24.30231

Rienties, B., Giesbers, B., Lygo-Baker, S., Ma, H. W. S., & Rees, R. (2016). Why some teachers easily learn to use a new virtual learning environment: A technology acceptance perspective. Interactive Learning Environments, 24(3), 539-552. https://doi.org/10.1080/10494820.2014.881394

Varga-Atkins, T. (2016). A study of the role of a technology-enhanced learning implementation group in mediating an institutional VLE minimum standards policy. Research in Learning Technology, 24. https://doi.org/10.3402/rlt.v24.32815

Vuopala, E., Hyvönen, P., & Järvelä, S. (2016). Interaction forms in successful collaborative learning in virtual learning environments. Active Learning in Higher Education, 17(1), 25-38. https://doi.org/10.1177/1469787415616730

Wihlborg, M., & Friberg, E. (2016). Framework for a virtual nursing faculty and student learning collaboration between universities in Sweden and the United States: A theoretical paper. Nurse Education Today, 41, 50-53. https://doi.org/10.1016/j.nedt.2016.03.012

Zanjani, N., Edwards, S. L., Nykvist, S., & Geva, S. (2016). LMS acceptance: The instructor role. The Asia-Pacific Education Researcher, 25(4), 519-526. https://doi.org/10.1007/s40299-016-0277-2

2017

Altunoglu, A. (2017). Initial perceptions of open higher education students with learner management systems. Turkish Online Journal of Distance Education, 18(3), 96-104. https://doi.org/10.17718/tojde.328939

Asamoah, M. K., & Oheneba-Sakyi, Y. (2017). Constructivist tenets applied in ICT-mediated teaching and learning: Higher education perspectives. Africa Education Review, 14(3-4), 196-211. https://doi.org/10.1080/18146627.2017.1279956

Baleghi-Zadeh, S., Ayub, A. F. M., Mahmud, R., & Daud, S. M. (2017). The influence of system interactivity and technical support on learning management system utilization. Knowledge Management & E-Learning: An International Journal, 9(1), 50-68. https://doi.org/10.34105/j.kmel.2017.09.004

Bissonette, D. (2017). The promise and perils of asynchronous learning: How faculty, students, and administrators can collaboratively increase retention and satisfaction in the online classroom. Journal of Pedagogic Development, 7(3), 13-23. http://hdl.handle.net/10547/622376

Derboven, J., Geerts, D., & De Grooff, D. (2017). Appropriating virtual learning environments: A study of teacher tactics. Journal of Visual Languages & Computing, 40, 20-35. https://doi.org/10.1016/j.jvlc.2017.01.002

Dreamson, N., Thomas, G., Lee Hong, A., & Kim, S. (2017). Policies on and practices of cultural inclusivity in learning management systems: Perspectives of Indigenous holistic pedagogies. Higher Education Research & Development, 36(5), 947-961. https://doi.org/10.1080/07294360.2016.1263830

Essel, D. D., & Wilson, O. A. (2017). Factors affecting university students’ use of Moodle: An empirical study based on TAM. International Journal of Information and Communication Technology Education, 13(1), 14-26. https://doi.org/10.4018/IJICTE.2017010102

Hadullo, K., Oboko, R., & Omwenga, E. (2017). A model for evaluating e-learning systems quality in higher education in developing countries. International Journal of Education and Development Using ICT, 13(2), 185-204. https://www.learntechlib.org/p/180643/

Halperin, R. (2017). Learning practice and technology: Extending the structurational practice lens to educational technology research. Learning, Media and Technology, 42(3), 279-294. https://doi.org/10.1080/17439884.2016.1182925

Kerimbayev, N., Kultan, J., Abdykarimova, S., & Akramova, A. (2017). LMS Moodle: Distance international education in cooperation of higher education institutions of different countries. Education and Information Technologies, 22(5), 2125-2139. https://doi.org/10.1007/s10639-016-9534-5

Kurt, Ö. E., & Tingöy, Ö. (2017). The acceptance and use of a virtual learning environment in higher education: An empirical study in Turkey, and the UK. International Journal of Educational Technology in Higher Education, 14(1). https://doi.org/10.1186/s41239-017-0064-z

Moreira, J., Henriques, S., Goulo, M., & Barros, D. (2017). Digital learning in higher education: A training course for teaching online—Universidade Aberta, Portugal. Open Praxis, 9(2), 253-263. https://doi.org/10.5944/openpraxis.9.2.539

Nunes, F. B., Herpich, F., Do Amaral, É. M., Voss, G. B., Zunguze, M. C., Medina, R. D., & Tarouco, L. M. (2017). A dynamic approach for teaching algorithms: Integrating immersive environments and virtual learning environments. Computer Applications in Engineering Education, 25(5), 732-751. https://doi.org/10.1002/cae.21833

Park, Y., & Jo, I. H. (2017). Using log variables in a learning management system to evaluate learning activity using the lens of activity theory. Assessment & Evaluation in Higher Education, 42(4), 531-547. https://doi.org/10.1080/02602938.2016.1158236

Rhode, J., Richter, S., Gowen, P., Miller, T., & Wills, C. (2017). Understanding faculty use of the learning management system. Online Learning, 21(3), 68-86. https://eric.ed.gov/?id=EJ1154161

Salvador, P. T. C. D. O., Bezerril, M. D. S., Mariz, C. M. S., Fernandes, M. I. D., Martins, J. C. A., & Santos, V. E. P. (2017). Virtual learning object and environment: A concept analysis. Revista brasileira de enfermagem, 70(3), 572-579. https://doi.org/10.1590/0034-7167-2016-0123

Songkram, N. (2017). Online course design for creativity and innovative skills in virtual cultural ASEAN community. International Journal of Emerging Technologies in Learning, 12(1), 4-20. https://online-journals.org/index.php/i-jet/article/view/6032

Virtanen, M. A., Kääriäinen, M., Liikanen, E., & Haavisto, E. (2017). The comparison of students’ satisfaction between ubiquitous and Web-based learning environments. Education and Information Technologies, 22(5), 2565-2581. https://doi.org/10.1007/s10639-016-9561-2

Wang, F. H. (2017). An exploration of online behaviour engagement and achievement in flipped classroom supported by learning management system. Computers & Education, 114, 79-91. https://doi.org/10.1016/j.compedu.2017.06.012

Zanjani, N. (2017). The important elements of LMS design that affect user engagement with e-learning tools within LMSs in the higher education sector. Australasian Journal of Educational Technology, 33(1), 19-31. https://doi.org/10.14742/ajet.2938

2018

Algozzini, L. C., Gabay, V. L., Voyles, S. D., Bessolo, K., & Batchelor, G. (2018). Breaking the code: Barriers affecting inclusive thinking—HETL Scotland 2017. Journal of Applied Research in Higher Education, 10(2), 118-129. https://doi.org/10.1108/JARHE-04-2017-0031

Arenas-Gaitán, J., Rondán-Cataluña, F. J., & Ramírez-Correa, P. E. (2018). Modelling the success of learning management systems: Application of latent class segmentation using FIMIX-PLS. Interactive Learning Environments, 26(1), 135-147. https://doi.org/10.1080/10494820.2017.1283335

Baragash, R. S., & Al-Samarraie, H. (2018). Blended learning: Investigating the influence of engagement in multiple learning delivery modes on students’ performance. Telematics and Informatics, 35(7), 2082-2098. https://doi.org/10.1016/j.tele.2018.07.010

Bond, M., Marín, V. I., Dolch, C., Bedenlier, S., & Zawacki-Richter, O. (2018). Digital transformation in German higher education: Student and teacher perceptions and usage of digital media. International Journal of Educational Technology in Higher Education, 15(1), Article 48. https://doi.org/10.1186/s41239-018-0130-1

Boulton, C. A., Kent, C., & Williams, H. T. (2018). Virtual learning environment engagement and learning outcomes at a “bricks-and-mortar” university. Computers & Education, 126, 129-142. https://doi.org/10.1016/j.compedu.2018.06.031

Cantabella, M., López, B., Caballero, A., & Muñoz, A. (2018). Analysis and evaluation of lecturers’ activity in learning management systems: Subjective and objective perceptions. Interactive Learning Environments, 26(7), 911-923. https://doi.org/10.1080/10494820.2017.1421561

Chootongchai, S., & Songkram, N. (2018). Design and development of SECI and Moodle online learning systems to enhance thinking and innovation skills for higher education learners. International Journal of Emerging Technologies in Learning, 13(3), 154-172. https://online-journals.org/index.php/i-jet/article/view/7991

Chow, J., Tse, A., & Armatas, C. (2018). Comparing trained and untrained teachers on their use of LMS tools using the Rasch analysis. Computers & Education, 123, 124-137. https://doi.org/10.1016/j.compedu.2018.04.009

Dreamson, N., Thomas, G., Lee Hong, A., & Kim, S. (2018). The perceptual gaps in using a learning management system: Indigenous cultural perspectives. Technology, Pedagogy and Education, 27(4), 431-444. https://doi.org/10.1080/1475939X.2018.1490665

Findik-Coşkunçay, D., Alkiş, N., & Özkan-Yildirim, S. (2018). A structural model for students’ adoption of learning management systems: An empirical investigation in the higher education context. Journal of Educational Technology & Society, 21(2), 13-27. http://www.jstor.org/stable/26388376

García-Álvarez, M. T., Novo-Corti, I., & Varela-Candamio, L. (2018). The effects of social networks on the assessment of virtual learning environments: A study for social sciences degrees. Telematics and Informatics, 35(4), 1005-1017. https://doi.org/10.1016/j.tele.2017.09.013

Ghazal, S., Al-Samarraie, H., & Aldowah, H. (2018). “I am still learning”: Modeling LMS critical success factors for promoting students’ experience and satisfaction in a blended learning environment. IEEE Access, 6, 77179-77201. https://doi.org/10.1109/ACCESS.2018.2879677

Grech, A., & Cassar, M. (2018). Can virtual learning environments be used as research platforms? Nurse Researcher, 26(3), 32-35. https://doi.org/10.7748/nr.2018.e1568

Gunathilaka, T. M. A. U., Fernando, M. S. D., & Pasqual, H. (2018). Individual learning path personalization approach in a virtual learning environment according to the dynamically changing learning styles and knowledge levels of the learner. International Journal of Advanced and Applied Sciences, 5(5), 10-19. https://doi.org/10.21833/ijaas.2018.05.002

Holmes, K. A., & Prieto-Rodriguez, E. (2018). Student and staff perceptions of a learning management system for blended learning in teacher education. Australian Journal of Teacher Education, 43(3), 21-34. https://doi.org/10.14221/ajte.2018v43n3.2

Holmes, N. (2018). Engaging with assessment: Increasing student engagement through continuous assessment. Active Learning in Higher Education, 19(1), 23-34. https://doi.org/10.1177/1469787417723230

Hussain, M., Zhu, W., Zhang, W., & Abidi, S. M. R. (2018). Student engagement predictions in an e-learning system and their impact on student course assessment scores. Computational Intelligence and Neuroscience, 2018. https://doi.org/10.1155/2018/6347186

Laeeq, K., & Memon, Z. A. (2018). An integrated model to enhance virtual learning environments with current social networking perspective. International Journal of Emerging Technologies in Learning, 13(9), 252-268. https://online-journals.org/index.php/i-jet/article/view/8000

Louw, W., & Michau, A. (2018). E-learning: Effective strategy, or “just another brick in the wall”? Africa Education Review, 15(3), 38-48. https://doi.org/10.1080/18146627.2016.1268062

McKenna, B. A., & Kopittke, P. M. (2018). Engagement and performance in a first year natural resource science course. Journal of Computer Assisted Learning, 34(3), 233-242. https://doi.org/10.1111/jcal.12236

Nurakun Kyzy, Z., Ismailova, R., & Dündar, H. (2018). Learning management system implementation: A case study in the Kyrgyz Republic. Interactive Learning Environments, 26(8), 1010-1022. https://doi.org/10.1080/10494820.2018.1427115

Orey, D. C., & Rosa, M. (2018). Developing a mathematical modelling course in a virtual learning environment. ZDM, 50(1-2), 173-185. https://doi.org/10.1007/s11858-018-0930-8

Pecori, R. (2018). A virtual learning architecture enhanced by fog computing and big data streams. Future Internet, 10(1), Article 4. https://doi.org/10.3390/fi10010004

Ranjbarfard, M., & Heidari Sureshjani, M. (2018). Offering a framework for value co-creation in virtual academic learning environments. Interactive Technology and Smart Education, 15(1), 2-27. https://doi.org/10.1108/itse-08-2017-0040

Straková, Z., & Cimermanová, I. (2018). Developing reflective skills of student teachers in the virtual learning environment. Electronic Journal of e-Learning, 16(2), 107-121. https://eric.ed.gov/?id=EJ1199905

Appendix B

Coding Sheet

Author:

Title & Journal:

Abstract:

Research Method

Classification of Research

Area of Interest

VLE Type

Theoretical Lens (if applicable):

What We Talk About When We Talk About Virtual Learning Environments by Michael Flavin and Aditi Bhandari is licensed under a Creative Commons Attribution 4.0 International License.