What We Talk About When We Talk About Virtual Learning Environments

This is a literature review analysing articles published on virtual learning environments (VLEs), otherwise known as learning management systems, in higher education in the years 2014–2018. The Web of Science database was used to identity relevant articles over this five-year period. The sample comprises 99 peer-reviewed, academic journal articles. A coding sheet is used to analyse each article, identifying the research method, the classification of research (macro, meso, or micro), the focus of research (students, instructors, or both), and, where applicable, the specific brand of VLE. Most output on VLEs is found to be quantitative, to focus on students and on the micro level of learning and teaching, to not have a clear theoretical focus, to not specify which brand of VLE is used, and to be produced in affluent countries. This article adds to the understanding of VLE research by identifying the most frequent foci of research on VLEs, as well as identifying areas that have been under-researched.


Introduction
Virtual learning environments (VLEs), otherwise known as learning management systems, are established technologies that have been supporting learning and teaching in higher education for over 20 years.Newman et al. (2018), in a large-scale survey of 37,720 students in the United Kingdom, showed 75% of students relying on their VLE to support coursework.In a further large-scale survey in the UK, 72% of higher education students stated they relied on their VLE to do their coursework (Langer-Crame et al., 2019).Because the VLE is embedded in higher education, and because distance learning has increasingly become the norm in the age of the COVID-19 pandemic, now is a timely point at which to reflect on studies on VLEs with a view to informing possible research hereafter, in part by identifying areas that have been under-researched to date.
In their early days, VLEs offered the possibility of innovation, even transformation.According to Dutton et al. (2004), "VLE systems began to diffuse widely in the late-1990s and quickly became a status symbol of innovation" (p.135).By allowing anytime, anywhere access, VLEs had the potential to transform learning and teaching, enabling synchronous and asynchronous collaboration irrespective of spatial and temporal boundaries.However, VLEs are often used in practice in limited ways to store content (Gordon, 2014;Rienties et al., 2016).Newman et al. (2018) note that 55% of students in their sample said VLEs were well designed, but only 26% enjoyed using the collaborative features.The limited usage of VLEs (relative to their capabilities) is not necessarily a problem: Rodriguez-Ardura and Meseguer-Artola (2016) argue that VLEs help users achieve learning goals (p.1046).That said, and while VLEs get jobs done in higher education, the gap between their potential and their actual use is considerable.This article, a literature review with analysis, undertakes a survey of articles published in academic journals on VLEs in higher education in 2014-2018, identified through the Web of Science database.The specific questions considered are the following: • What is the pattern of academic journal articles on VLEs over the period 2014-2018?
• What can be deduced about VLE scholarship in higher education from published work over the 2014-2018 period?
• What areas of inquiry are under-explored in research on VLEs?
The questions enable exploration of what the studies have to say about the status of the VLE in higher education; the distribution of scholarship on VLEs by country and by academic journal; and what aspects of VLEs are under-researched.Other issues explored in this paper include geopolitical aspects of the pattern of scholarship on VLEs and the specific brand of VLEs used.This study also examines whether VLE research focuses on the micro level of teaching and learning, the meso level of managerial and organisational aspects of VLEs, or the macro level of VLEs viewed through specific theoretical lenses or as part of education systems.

Literature Review
This section summarizes research relevant to the current study, encompassing both articles on VLEs and on technology-enhanced learning more broadly.It identifies articles with research questions similar to the present study, and with similarities in research interests, too, in order to compare findings.
In an early article, Paulsen (2003) surveyed VLE provision at 113 European higher education institutions using 52 different VLEs, including in-house systems.The research was undertaken in late 2001 and early 2002, before Moodle was available.The most popular system was WebCT (20 institutions), followed by a Norwegian system, ClassFronter (16 institutions), followed by Blackboard (14 institutions).Blackboard and WebCT merged in 2005 and now trade as Blackboard.
VLEs relocate didactic modes of teaching to the online environment, as reflected in scholarship.Fry and Love (2011) interviewed lecturers in business, whose metaphors for VLEs included "security blanket," "crutch," and "electronic filing cabinet" (p.54), underlining the extent to which VLEs were used to store content.Students' and lecturers' use of VLEs as storage facilities may be a choice.However, it may also be a consequence of VLEs' design and how they are deployed by universities: McGill and Hobbs (2008) argue, "Student use of a VLE is primarily as an end-user.… Instructors design the interface for their course and implement different levels of functionality" (p.192).In addition, Naveh et al. (2010) argue that "instructors can maintain their conservative teaching habits.… From an organizational perspective, this can be done at low cost, yielding relatively high student satisfaction" (p.132).
VLEs are customarily produced in affluent countries but exported worldwide.Jones (2014) argues that learning technologies "are largely the outcome of design and development carried out by multi-national US based corporations" (p.170).Moodle's head office is in Australia, but the other main VLE provider, Blackboard, is based in Washington, DC.Sakai was developed by a group of universities in the United States.WebCT, a major player in the VLE market until it was taken over by Blackboard, also began in the United States.Zeide and Nissenbaum (2018) argue, of VLE vendors, "By adopting commercial marketplace norms, these providers undermine core functions and values of education" (p.280).Moreover, the production of VLEs in affluent countries can lead to colonial pedagogy whereby the design features of VLEs prompt culturally specific forms of learning.
Moccia (2016) argues, "Technologies are always expensive so leaders in higher education have to bet on where to invest their sometimes-rare resources" (p.32).A large-scale survey in the UK showed the VLE receiving most institutional investment (Jenkins et al., 2018).Castaneda and Selwyn (2018) argue, "The use of digital technology in higher education is now a multi-billion dollar business … the commercial design of educational systems and software increasingly shapes the forms of teaching and learning that take place in universities" (p. 6).VLEs are a lucrative market because of their ubiquity.In addition, VLEs' design influences the kind of learning and teaching that takes place in higher education.Latchem (2006) undertook a survey of 374 items (265 articles, 86 colloquia, and 23 editorials) published in the British Journal of Educational Technology, 2000Technology, -2005, , arguing that educational technology needed more theory-driven research.Latchem (2006) also showed that "the vast majority of the articles reflected work and thought in western countries or overseas universities whose curricula and methodologies are essentially western-influenced" (p.509), suggesting the use of VLEs is political as well as technical, reproducing geopolitical structures and imbalances of wealth and power.Baydas et al. (2015) used content analysis to study 1,255 articles published in the British Journal of Educational Technology andEducational Technology, Research andDevelopment, from 2002 to 2014, finding that the most common research method was quantitative.The articles included research on VLEs but were not limited to articles on VLEs.Risquez et al. (2015) reported on a survey of VLE use in Ireland, which has been taking place on a rolling basis since 2008.Students wanted VLEs that were easy to navigate and identified lecturers' lack of use as a barrier to engagement.A later study of survey responses from 521 staff in Irish higher education institutions found the most common use of VLEs was as content repositories and as a channel for communication (Farrelly et al., 2018).Cobo et al. (2014), in a study in Spain, and Oproiu (2015), in a study in Romania, found similar VLE usage patterns.Ngai et al. (2007), in research conducted in Hong Kong higher education from a Technology Acceptance Model perspective (Davis, 1989), found ease of use and usefulness to be the dominant factors affecting the use of VLEs.Van Raaij and Schepers (2008), also using the Technology Acceptance Model, studied 45 students in an executive MBA programme in China, finding that perceived usefulness had a direct effect on VLE usage.It had originally been hoped to pose a further research question relating to the extent to which scholarship advocated innovative usage of VLEs, but institutional funding given to support the project was reduced, and the research questions were scaled down accordingly.It was noted that the term learning management system was used as an alternative to virtual learning environment to describe the same type of learning platform (though Pinner [2014] distinguishes between the two in an opinion piece), but virtual learning environment is used in this paper for consistency.
The initial search, undertaken by the second author, yielded 2,211 articles.Following the initial identification, the titles, abstracts, and keywords were screened for eligibility to the study, identifying articles in which VLEs were a significant element, using inter-rater discussion (comparative verification on the part of the authors), a method also used in a survey of articles on educational technology by Baydas et al. (2015).The keyword virtual learning proved unhelpful as it identified articles concerned with computer simulations or virtual worlds.The screening process resulted in a total of 99 articles selected for coding.A similar approach was adopted by Sung et al. (2016), who performed a meta-analysis of articles published on mobile learning over the period 1993-2013.This study has drawn selectively from the methodology of previous studies in order to better address the specific research questions: disaggregating the data into macro, meso, and micro categories highlights patterns in publications on VLEs and enables conclusions to be drawn, including the identification of research areas that have been relatively unaddressed to date.

What We Talk About When We Talk About Virtual Learning Environments
Flavin and Bhandari For this paper, the analysis categorized the sample in terms of years published, country of origin, areas of focus, brand of VLE, journal, research method, and theoretical perspective.All the articles making up the sample are cited in Appendix A, subdivided by year of publication (2014)(2015)(2016)(2017)(2018).

Results and Discussion
The year-by-year distribution of the sample is shown in Table 1.With the exception of 2014, the number of articles published each year is consistent, varying between and 25.There was no obvious explanation for the shortage of articles in 2014.Data on years prior to were not collected, and therefore it is not possible to state if 2014 is an outlier or in line with years outside the sample.
Aggregating the data over five years and taking the home higher education institution of the first author as means of identification, the most articles were published in the UK ( 14), Spain (12), and Australia (11).
Countries with two or more journal articles over the period are summarized in Figure 1.

Countries of First Author (Two Articles or More)
There were 26 countries with one article each, including Colombia, Germany, Kenya, the Kyrgyz Republic, and Pakistan.The geographical spread of articles on VLEs over the period from 2014 to 2018 was extensive, but with a few hot spots (notably, the UK, Spain, and Australia).The output was largely though not wholly weighted towards affluent countries.The relatively low count from the US was noticeable.Latchem (2006), The division between macro, meso, and micro, as per the categories defined by Zawacki-Richter et al.
(2009), and as recorded on the coding sheets, features in Table 2.  prioritize studies of practice over explorations of theory.
Each article was evaluated in terms of whether its research focus was on students, instructors, or both (recorded on the coding sheets).A small number of articles did not fall into any of these categories in instances where the focus was on administrators or education leaders.The distribution of the sample in terms of the research focus is shown in Table 3.

Table 3
The Over half the articles in the sample did not specify which brand of VLE was used.The identification of the specific VLE may not have been essential for the research, but the omission is surprising in view of the fact VLEs comprise significant expenditure for universities (Castenada & Selwyn, 2018;Jenkins et al., 2018;Moccia, 2016), and information on which brand of VLE is used could have value.The distribution of the sample, in terms of the particular VLE, is listed in Table 4. Unspecified 40 The number of unspecified VLEs could potentially limit research by neglecting the specific brand.That said, commercial VLEs are more conspicuous by their similarities than their differences, offering comparable capabilities regarding information storage and communication channels, and hence, results from published studies may have use value irrespective of which brand of VLE is used.Benner and Tripsas (2012) argue that competing alternatives often converge on a dominant design in a process of mimetic isomorphism, and hence the brand of VLE may make no significant difference.The standardization of VLEs as products imposes, in practice, a digital hegemony and a standardized, reductive, and culturally exclusive mode of learning and teaching, in view of the dominance of affluent countries in the design and sale of VLEs.
The coding sheet recorded the journal title for each article.The numbers of articles from individual journals are featured in Table 5.Only journals with two or more articles are included.suggesting that VLE research is of more interest to a niche community than to educators as a whole.That said, we recognize that journals are not unchanging entities but, instead, change direction and editorial personnel.
Half of the journals in Table 5 featured in the top 50 journals identified under the heading "E-Learning" (SJCR, 2018).Only three of the journals in Table 5 featured in the SJCR top 10, suggesting research on VLEs is not concentrated in high-impact journals and is distributed more widely through technologyenhanced learning publications.Identifying the impact factor of journals in which educational technology research features was also part of Hsu et al.'s (2013) methodology.
Most of the articles adopted a quantitative approach to data gathering and analysis, as shown in Table 6.

Table 6
Research Methods for the Sample

Research method Number of articles
Quantitative 48 Qualitative 24

Mixed-methods 19
No original data 8 The eight articles that did not belong to any of the categories were either position papers or literature reviews.This contrasts with Sosa et al.'s (2019) findings that most of the research surveyed was qualitative, though the research was on emerging technologies rather than focused on VLEs and was not limited to studies of higher education.
The paucity of theorized accounts of VLEs was noticeable.Very few of the articles declared an explicit theoretical lens in their title, abstract, or keywords.Instances of two or more articles adopting a specific theoretical approach are summarized in Figure 2.

Figure 2
Theories Used to Analyse Articles From the Sample The most popular theoretical lens was the Technology Adoption Model (Davis, 1989).This is not surprising, as this model analyses how people engage with technologies and the criteria they apply to evaluate technologies' usefulness, rooted in Fishbein and Ajzen's (1977) theory of reasoned action.The Technology Adoption Model is well suited to analysing engagement with VLEs.
The Community of Practice Theory (Lave & Wenger, 1991;Wenger, 1998) is suitable for analysing VLEs from a social perspective and monitoring learners' progress in learning communities, as the theory focuses on how learning communities evolve and on how people establish effective identities in communities of practice.Data collected by VLEs can illuminate patterns of engagement within communities.Activity Theory (Leontiev, 1978(Leontiev, , 1981) is more suitable for analysing technical and social aspects of technologies and how they interact, by anatomizing systems into interactive nodes.Grounded Theory (Strauss & Corbin, 1998) takes a different approach, gestating a theory from the data gathered.
The data suggest that certain, generally affluent countries tend to dominate scholarship on VLEs, as they dominate production of VLEs (Jones, 2014).Furthermore, articles on VLEs tend to get published in technology-enhanced learning journals, such as Computers & Education and the British Journal of Educational Technology, both of which also featured as prolific journals in the study by Hung (2012), rather than get published in education journals more widely, despite the ubiquity of VLEs.

Conclusion
This article, a literature review with analysis, has surveyed research on VLEs in higher education over the period 2014-2018, identifying patterns in the sample as well as areas that have been under-explored, with a view to informing future research.
Research on VLEs over the period from 2014 to 2018 tends to be quantitative.It also tends to be focused on students and on the micro level of learning and teaching.It is not likely to have an explicit theoretical focus, an issue identified by Latchem (2006) McGill & Hobbs, 2008).In addition, opportunities exist to explore in-depth interactions on VLEs, using content analysis as a methodological approach, as well as to extend research on VLEs beyond the end of 2018.
A limitation of the present study is that despite the use of specific search terms, some relevant articles are likely to have been missed.One database (Web of Science, originating in the US) was used, but additional databases would probably have produced additional articles.However, the articles used are cited in Appendix A, helping to identify notable omissions for the benefit of future researchers, and the sample comprises a range of relevant articles from across the world, albeit all the articles are published in English.
A further limitation of the present study is the relatively short period of time surveyed, but similar time periods were studied by Latchem (2006) andde Oliveira et al. (2016).
This article contributes to research on VLEs by identifying prevalent practices regarding research in 2014-2018 and identifying gaps.VLEs are embedded technologies in higher education, but research on their use is weighted towards particular methodological approaches (quantitative) and particular stakeholders (students).Moreover, VLE scholarship (and VLE design) is weighted towards affluent countries.Segooa and Kalema (2015) argue that "many VLEs are not contextualized to meets the needs of educational institutions in developing countries … elearning implementation should focus more on the social contexts rather than the technological solution" (pp. 353-354).Surveys of publications on VLEs show considerable continuity of findings in terms of research methodologies and areas of focus.That said, this paper also notes the dominance of Western companies in the provision of VLEs and, consequently, the application of Western pedagogic models globally, comprising digital colonialism.This research thus argues that geopolitical aspects of implementation and usage of VLEs are under-explored.Moreover, the VLE is an object of increased interest because it has become a dominant mode of learning and teaching during the COVID-19 pandemic, with more students learning at a distance.There is more to talk about when we talk about VLEs.
Shih et al. (2008) undertook a content analysis of journal articles concerned with cognition in e-learning published in five journals(British Journal of Educational Technology, Innovations in Education and   Teaching International, Computers and Education, Educational Technology Research and Development,   and Journal of Computer Assisted Learning), recording year of publication, journal, and research topic; they found most of the articles took a quantitative approach.Hung (2012) analysed the abstracts of 689 articles on e-learning published between 2000 and 2008, with a research question investigating trends and patterns of e-learning research.Hung (2012) used the Web of Science database and limited the research to journal articles and proceedings papers.The five most prolific countries for e-learning research were England (conflated with the UK), the US, Taiwan, China, and Germany.The five most prolific journals were Lecture Notes in Computer Science, Computers and Education, British Journal of Educational Technology, Educational Technology and Society, and Lecture Notes in Artificial Intelligence.Sosa et al. (2019) surveyed research on emerging technologies, 2006-2016, though their survey was not limited to higher education.From a sample of 288 studies, they found that students were the most frequently studied population in individual papers.Very few of their studies focused on instructors.Other features they categorized included year of publication and research methodology, both of which are categories in the present study.Hsu et al. (2013) examined the abstracts of 2,997 articles on education technology published between 2000 and 2010.The categories for their analysis included productivity by journal and by country, and they also analysed data by year of publication.Zawacki-Richter et al. (2009) surveyed 695 articles on distance education published between 2000 and 2008.They classified research into three categories: macro (systems and theories), meso (management and organisation), and micro (teaching and learning), finding a shortage of articles at the meso level.Their categories have been adapted for the coding sheet supporting the present study (see Appendix B) because the categories enable a useful subdivision of research.Zawacki-Richter et al. (2009) found most of their sample adopted a qualitative approach, identified gaps in distance learning literature around innovation (information and communications technology), and innovation.The research was limited to articles published in English.A coding sheet was devised (see Appendix B) to identify core features of the articles.
a survey of articles in the British Journal of Educational Technology covering the period 2000-2005, found 40% of the articles were from the UK, 12% from the US, and 8% from Australia (p. 505), though Latchem's survey was of all articles published in the British Journal of Educational Technology (articles, editorials, and colloquia) and not solely those pertaining to VLEs.Hsu et al. (2013) also identified the top 10 countries for publication in the field of educational technology, 2000-2010.Five of their top ten countries also appear in Figure 1: USA, UK (though Hsu et al. [2013] conflate England with the UK), Australia, Turkey, and Germany.Five of their top ten countries do not feature in Figure 1: Taiwan, the Netherlands, Canada, Greece, and Singapore.The present study identifies Spain as the second most productive country for articles on VLEs in 2014-2018.This is at odds with other surveys of productivity in research, though the other surveys were of technology-enhanced learning articles more widely rather than being focused on VLEs.No obvious reason was discernible for Spain's relative prominence in the field.
of the articles focus on students alone, and a further 18 focus on students in conjunction with instructors, totalling 77 articles in the sample.Sosa et al. (2019), in their study of literature on emerging technologies in 2006-2016, also identified a research focus on students; 152 articles from their overall sample of 288 focused on students, of which 117 focused on higher education students (p.136).In the present article, a shortage of articles was found examining VLEs from the perspective of instructors only (lecturers or tutors), identifying a potentially under-researched area of practice with VLEs, especially as instructors can often be responsible for VLEs' content(McGill & Hobbs 2008).

Table 1
Distribution of the Sample by Year

Table 2
The Sample Divided by Categories Latchem (2006)r et al.'s (2009) withZawacki-Richter et al.'s (2009)study, which found a shortage of articles at the meso level.The fewest number of articles looked at VLEs as part of national higher education systems or as validations or refutations of particular theoretical perspectives.Latchem (2006)divided his sample into empirical enquiry (78%) and theoretical enquiry (22%), indicating a similar tendency to

Table 4
Specific Virtual Learning Environments (VLEs) Used in the Sample Table 5 also positions the number of articles published in each journal alongside that journal's position in the Scimago Journal and Country Rank (SJCR) 2018, where applicable.

Table 5
Journals in the Sample Featuring Two or More Relevant Articles, and the Journals' Scimago Journal and Most of the journals inTable 5 are slanted towards an interest in technology-enhanced learning.Only six have a more generic interest in education.Research on VLEs gets focused in journals relating closely to technology-enhanced learning rather than being diffused more widely through education journals, Oliver (2011)f 374 published items in the British Journal of Educational Technology, as well as byOliver (2011)in relation to research in technology-enhanced learning published from 2001 to 2011.Some countries published a higher-than-average volume of research on VLEs (UK, Spain, and Australia) over the period studied, and two journals averaged more than one article per year on VLEs (International Journal of Emerging Technologies in Learning and Computers & Education).VLEs are embedded technologies in higher education, but they are not a frequent subject of interest.Furthermore, articles on VLEs leave numerous areas under-explored, including comparison of active and relatively inactive geographical concentrations of research, which may, in turn, reflect unequal relationships in the design and implementation of VLEs and technology-enhanced learning more widely.Ferreira et al. (2020) argue that technological solutionism has supported marketization of the education system in Brazil, comprising a form of digital colonialism, which this paper largely reinforces through a survey of articles on VLEs that indicates dominance by affluent countries in systems and research.The data from the present study indicate opportunities exist for more theorized approaches to VLEs because different theoretical lenses can illuminate different aspects of technologies and practice.There are also opportunities for more qualitative approaches to data gathering and analysis, as well as for research focusing on instructors, who play a notable role in designing interfaces and implementing levels of VLE functionality (