|
|
|
Tharindu Rekha Liyanagunawardena1, Andrew Alexandar Adams2, and Shirley Ann Williams1
1University of Reading, UK, 2Meiji University, Japan
Massive open online courses (MOOCs) are a recent addition to the range of online learning options. Since 2008, MOOCs have been run by a variety of public and elite universities, especially in North America. Many academics have taken interest in MOOCs recognising the potential to deliver education around the globe on an unprecedented scale; some of these academics are taking a research-oriented perspective and academic papers describing their research are starting to appear in the traditional media of peer reviewed publications. This paper presents a systematic review of the published MOOC literature (2008-2012): Forty-five peer reviewed papers are identified through journals, database searches, searching the Web, and chaining from known sources to form the base for this review. We believe this is the first effort to systematically review literature relating to MOOCs, a fairly recent but massively popular phenomenon with a global reach. The review categorises the literature into eight different areas of interest, introductory, concept, case studies, educational theory, technology, participant focussed, provider focussed, and other, while also providing quantitative analysis of publications according to publication type, year of publication, and contributors. Future research directions guided by gaps in the literature are explored.
Keywords: MOOC; massive open online course; massively open online course; systematic review; connectivism
Distance education has a long history, with correspondence courses making use of reasonable cost universal postal services for the delivery of study material to learners and for submission/return of assignments by/to students (Casey, 2008). Further developments of distance education have appeared with each new communication technology: radio, television, video recorders, home computing. The latest development, that of the Internet (including very recently the mobile Internet), has similarly been adopted by many existing higher education providers but has also supported the emergence of a new model dubbed a massive open online course (MOOC(s)), the term coined in 2008 to describe an open online course to be offered by the University of Manitoba in Canada. A range of both topics and platforms have since emerged and the term was described as “the educational buzzword of 2012” by Daniel (2012) reflecting widespread interest in the concept. MOOCs are widely discussed across a range of media, including blogs and the specialist and popular press, however this includes “thinly disguised promotional material by commercial interests … and articles by practitioners whose perspective is their own MOOC courses” according to Daniel (2012).
This paper seeks to classify academic research relating to MOOCs, based on a systematic review of the existing peer reviewed MOOC literature. Search techniques for papers related to MOOCs are considered and a corpus of papers identified, then a grounded research approach is presented from which a classification of the works emerges.
Since the early days of computing, academics have shared digital content (Lane & McAndrew, 2010) and recently there has been much interest in the sharing of open educational resources (OER), particularly relating to higher education, which has also become an important resource base for teachers and learners (Adams, Liyanagunawardena, Rassool, & Williams, 2013). In 2001 the Massachusetts Institute of Technology (MIT) launched its pioneering OpenCourseWare (OCW), with the aim of publishing materials from all its courses permanently on the open Web, with licenses allowing its use, modification, and redistribution. Since then many other established universities have joined the movement such as the Open University of UK through the OpenLearn project and the Open Learning Initiative by Carnegie Mellon University. OER were made available for two purposes: Learners could access the material directly and, hopefully, learn from it; educators could use the material as part of their own teaching (as produced or by amending it themselves). A significant proportion of these OER, however, were of limited use since they were usually produced in order to be a specific part of a larger educational experience within a specific educational framework. This limitation was particularly frustrating for many aspiring learners attempting to use them directly, but could also cause problems when used naively by educators (Liyanagunawardena, 2012; Weller, 2007). The concept of open access to learning was taken in a different direction with the introduction of the massive open online courses or MOOCs (Fini, 2009).
A MOOC brings together people interested in learning (or “students”) and an expert or experts who seek to facilitate the learning. Connectivity is usually provided through social networking, and a set of freely accessible online resources provides the content or the study material. Furthermore, they generally have no prerequisites, fees, formal accreditation, or predefined required level of participation (McAuley, Stewart, Siemens, & Cormier, 2010). Participation in a MOOC is completely voluntary and is dependent on the interested individual. The collaborative space of a MOOC can span across many different platforms and technologies. For example, MOOC participants may create their own blog posts discussing aspects of the MOOC in different spaces and/or may use microblogs such as Twitter to express themselves. “Connectivism and Connective Knowledge”, an online course offered through the Learning Technologies Centre and Extended Education at the University of Manitoba and facilitated by George Siemens and Stephen Downes (Downes, 2008), is considered the first MOOC. This online course had 25 paid enrolments (for credit) with around 2,200 non-credit, non-fee paying students. It used the principles of connectivism (Siemens, 2005) and unlike the traditional form of online learning was not primarily relying on resources posted through a learning management system (LMS)1.
More recently MOOCs have developed within international co-operative partnerships such as Coursera (www.coursera.org), a partnership of 62 world class universities (as of April 11, 2013) led by Stanford University, and edX (www.edx.org) which includes the Massachusetts Institute of Technology, École Polytechnique Fédérale de Lausanne, The Hong Kong University of Science and Technology. Udacity (www.udacity.com), P2P University, and Futurelearn (the UK Open University’s MOOC platform) are other related platforms. The numbers registering for MOOCs have reached 160,000 in the case of a 2011 Artificial Intelligence online course offered by Stanford University (Rodriguez, 2012).
With the increasing uptake and interest in MOOCs, it has become a popular topic in the educational press such as the Time Higher Education Magazine article (Corbyn, 2012), which presents an account of running a MOOC. Likewise there are many blog posts relating to MOOCs, posted from a variety of viewpoints including course leaders, participants, and outsiders (for example http://mooctalk.org chronicles the experience of a mathematician leading a MOOC in late 2012).
Academic papers on MOOCs began to appear in the peer reviewed literature (such as journals, conference proceedings, and professional magazines) in 2008 (Downes, 2008), with an increasing number of papers appearing each year since. This phenomenon is similar to many other technologies that created interest in academic communities. For example, Twitter appeared as a micro-blogging tool in 2006 and the literature on Twitter has grown exponentially from three papers in 2007 to hundreds of papers in 2011 (Williams, Terras, & Warwick, 2013).
Researchers use different methods to identify papers to consider for a literature review (Ellis 1989; Ellis & Haugan, 1997): Methods include searching in databases or search engines and chaining from known research papers. For a systematic review it is important that the methods of identifying papers are described and justified, and that the approach can be re-applied by others (Fink, 2010). The aim of this study was to locate and analyse MOOC related academic literature to provide an understanding of developing research areas, methods applied in research, and topics lacking published research.
Relevant papers were identified through a series of search efforts, using an approach based on the methods used in other systematic reviews including two studies of literature related to the microblogging system Twitter (Gao, Luo, & Zhang, 2012; Williams, Terras, & Warwick, 2013). Papers were classed as relevant if their primary focus was to explore the concept of a MOOC or the implications for higher education, report on experiments with MOOCs, or compare MOOCs with other educational approaches. There was insufficient statistical data across the papers found to undertake a meta-analysis (Fink 2010).
Firstly, the search terms and boundaries to be used were established. Initially two search terms (and their plurals) were selected:
However it was identified that some authors used “Massive” instead of “Massively” (for example Kop, Fournier, & Mak 2011) and so the third term (and its plural) was added:
The search period was limited to the period from the year in which the first MOOC was run (2008) to the year this study started (2012). Where possible the search was limited to titles and abstracts to reduce the workload of manual filtering of irrelevant papers.
Secondly, following the approach of Gao, et al. ( 2012), a number of academic journals in the disciplines of educational technology and distance education were selected for a preliminary search (Table 1). These journals were The British Journal of Educational Technology, Distance Education, American Journal of Distance Education, and Journal of Online Learning and Teaching. The search returned a small number of papers in one of these journals, but on detailed examination none of these were relevant for this systematic review. For example Roderick (2008) was predominantly about connected learning in digital spaces, and massively multiplayer online role-playing games were discussed as an example of connectivity. We believe that even though the search terms were used within inverted commas as phrases, the search algorithm may have returned articles such as this by comparing single search terms.
Thirdly, the same search terms were then used to search various academic databases: ISI Web of Knowledge, ProQuest (ERIC, British Education Index and Australian Education Index), JSTOR (education titles), IEEEXplorer, and Scopus (Table 2). A number of the articles returned were unrelated to this study, for example papers returned included some on “Multiple Optical Orthogonal Code Sequences” and some on “Management of Organizational Change”2, both of which were abbreviated to MOOC.
Fourthly, the same search was conducted using the Google Scholar search engine with 94 results (November 21, 2012). From these results only 33 were found to be relevant. However, the same search repeated on Google Scholar later (on January 08, 2013) only resulted in 25 results suggesting that some articles previously found were not online and/or Google had changed its search algorithms. For example, the first result returned by the search done on the 21st November 2012 was the article by Fini (2009); but the recent search that returned only 25 results did not contain this article. Presentations and papers that did not discuss MOOCs as the primary concern of the paper were discounted after a careful reading of the abstract.
In order to improve the coverage of relevant publications to be included in this review, the chaining technique of Gao, et al. (2012) was used: consulting the reference lists of papers that were already in the corpus to locate other relevant work. This resulted in the addition of one more article to the corpus. Google Scholar searches were also performed for each of the articles in the corpus to identify other articles that had referenced them (forward referencing). This resulted in the addition of one more conference paper. During December 2012 two additional papers were identified: one in the Communications of the ACM and another in the Journal of Interactive Media in Education.
The data collection process resulted in the identification of 45 distinct articles – 17 from journals, 13 conference publications (including one poster conference presentation), 10 academic magazine articles, 3 reports, and 2 workshop presentations.
Articles were classified both quantitatively and qualitatively. The quantitative analysis was used to classify the papers according to the publication year and the type of publication in which the article appeared. Papers were qualitatively classified using open coded content analysis, a technique used by the two studies of literature relating to Twitter (Gao, et al., 2012; Williams, Terras, & Warwick 2013). Initially the first author read each of the papers to identify themes, types of inquiry employed, and the future research directions indicated. These classifications were refined in an iterative manner by all authors reading and re-reading articles, and considering alternative classifications and stratifications.
The first MOOC related paper was published in 2008, with again just one paper identified in 2009, seven papers in 2010, 10 in 2011, and 26 in 2012. Clearly an increasing pattern in the number of articles published on MOOCs can be seen from Figure 1.
The majority of identified articles published to date are in journals (17 papers), with a smaller number of articles appearing in conference proceedings and magazines (13 and 10 respectively). Figure 2 diagrammatically presents the percentage of papers in each type of publication.
Figure 3 illustrates the separation of articles by publication type and year. From 2009 to 2012 there is a gradual increase of the number of journal articles and conference papers. It is noticeable that the first magazine article about MOOCs appeared in 2011 and the following year there is a fourfold increase, reflecting the recent general interest in MOOCs.
It is noticeable that while many of the publications identified only had one article on MOOCs, exceptionally among the journals the International Review of Research in Open and Distance Learning published six articles on MOOCs, while the European Journal of Open Distance and E-Learning published three. Communications of the ACM published the highest number of magazine articles (three) while eLearn Magazine and Learning Solutions magazine published two articles each. The International Conference on Networked Learning published three articles on MOOCs, two in 2010 and another in 2011.
A list of authors with more than one article being reviewed here are presented in Table 3. Kop has the highest number of publications (five) in the review while deWaard, Hogue, Koutropoulos, and Rodriguez have four each. It can be seen that the authors with more than three publications have worked in teams and several such teams can be identified from co-authorship, for example Kop and Fournier; de Waard, Abajian, Gallagher, Hogue, Keskin, Koutropoulos and Rodriguez; Mackness, Mak, and Williams.
The articles embraced a wide-range of themes relating to MOOCs; an initial list included3:
Most papers included some introduction to the term MOOC and a history of the concept. The majority of articles were primarily concerned with the concept of MOOCs, discussing challenges and trends, while other themes generally appeared within only one paper except for the concept of connectivism and its implications. The themes were re-stratified into the following categories.
The articles were then individually re-considered and assigned to the categories which reflected the contents; many articles were assigned to more than one category. Table 4 shows the spread of these categories.
Across the articles classified, 21 had a case study element, and in total 13 different MOOCs were studied.
The MOOCs CCK08, CCK11, CritLit, and PLENK2010 have designers and deliverers in common. Of the case study papers, one (Rodriguez, 2012) compared different MOOCs (including CCK08, PLENK2010, MobiMOOC, EduMOOC, Stanford CS221, and CS101 from Udacity); two (Kop 2011; Kop, Fournier, & Mak, 2011) considered PLENK2010 and one other MOOC (CritLit and CCK11, respectively); and another (Bell 2010a) considered CCK08 and CCK09; the remainder focussed on single MOOCs. Table 5 shows the number of articles studying each MOOC and the number of articles studying each as the single central case study. The related MOOCs (PLENK2010 and CCK08) were the most studied. MobiMOOC is the only other course to be considered in more than one paper.
The majority of case studies have used multiple methods for data collection in line with the general practice of case study research (Yin, 2003). In most case studies online surveys were used to collect data from participants in the MOOC(s). SurveyMonkey (www.surveymonkey.com) and LimeSurvey (www.limesurvey.org) were popular tools used in survey data collection. Four studies considering PLENK2010 as a case used multiple surveys (active participant survey, lurker survey, and end course survey). Researchers also reported collecting data via email interviews, focus groups, Moodle log data, discussion forum data, blogs, and observations. Three articles reported the use of techniques based on virtual ethnography and another used narrative inquiry as their research approach.
Seven of the case study articles defined the researcher’s role in relation to the study. In two of these instances the researcher was described as a learner in the MOOC, in another two as a participant, and in the remaining three as an observer. The demarcation (if one really exists) between a learner and a participant in a MOOC is not clear, and may well simply be a matter of differing nomenclature.
The majority of the participants in the case study research were people who had participated in the MOOC under consideration. Two articles also included data collected from instructors.
Rodriguez (2012) classifies MOOCs into two categories: connectivist MOOCs (c-MOOCs) and AI-Stanford like courses. He associates courses similar to AI-Stanford predominantly with cognitive-behaviourist approaches and c-MOOCs with connectivist approaches. By comparing multiple case studies Rodriguez (2012) claims that courses similar to AI-Stanford have a more individualist learning approach while c-MOOCs have a more social approach to learning. He also shows the different roles played by facilitators in each type of MOOC. Daniel (2012) on the other hand discusses cMOOCs and xMOOCs, which he claims are a bifurcation of MOOCs. He refers to “xMOOCs now being developed by elite US institutions that follow a more behaviourist approach” (Daniel 2012) but does not provide a definition. It is probable that both Rodriguez (2012) and Daniel (2012) are similarly classifying MOOCs but using the two different labels for the same thing: “AI-Stanford like courses” and “xMOOCs”. However, our systematic review has shown that there is a gap in the scholarly literature in defining these different types of MOOCs.
The aim of this study is to develop a classification of academic research relating to MOOCs, based on a systematic consideration of the existing peer reviewed MOOC literature. Other materials such as blog posts and unpublished reports are not included in this review. Blog posts always present difficulties for authors of reviews in determining the credibility of the posts: There are few studies on how researchers use blogs (Shema, Bar-Ilan, & Thelwall, 2012); the transient nature of blogs means that they are difficult to suitably include in a systematic consideration. It should be noted that the same search terms applied above used in Google Web search returned over 50,000 items and used in a Google Blog search resulted in 570 results (December 04, 2012), however these items are not analysed here. Daniel (2012) includes references to such resources. Inclusion of such resources in a systematic review is contentious as they are often highly subjective and have rarely been critically reviewed by peers. The question of whether such resources should be included in a systematic review needs further methodological consideration in the literature, but at present it is unusual due to issues of credibility.
Articles published in languages other than English were not considered for this review. When searching forward references using Google Scholar, the authors came across publications that referenced articles in the review but were published in other languages such as Chinese. Because of this, the authors believe that there could be articles that would have been included in the review were they published in English.
This analysis seems to reflect the growing interest in MOOCs. However, most research has investigated the learner perspective, with a significant minor focus on the institutional threats and opportunities. The lack of published research on MOOC facilitators’ experience and practices leaves a significant gap in the literature. Mak, et al. (2010) suggest that there has been unacceptable behaviour (for example, forceful intellectual debates, feelings of participation being demanded, and rude behaviour) from some MOOC participants, which has led other participants to cease posting on forums. The possible cultural differences of participants in MOOCs and their MOOC experience would be an interesting avenue of research in relation to cultural tension in MOOCs. Many studies that used data generated from MOOCs have limited their qualitative analysis to posting on a formal LMS and a sample of other sources (such as blog posts) due to the volume of data (Kop, 2011; Kop, & Fournier, 2010). Inevitably this leaves out potentially useful data on Twitter, blogs, and other social media spaces. For example, Liyanagunawardena (2012) shows that there is a significant use of external communications (other than the official LMS discussions) to support learner groups in traditional, blended, and online distance education settings.
Despite many research studies using publicly available data from a MOOC for research purposes, only a few papers have considered the ethical aspects of such use. This may well be due to a lack of good guidance and best practice examples on the general question of the use of data from online social media and similar sources for such research (Zimmer, 2010). This opens up a new avenue of research leading to MOOC participants’ (both students’ and facilitators’) views on the ethical issues of using the data generated by them on a MOOC for research and a researcher’s role in the MOOC. However this topic is discussed by MOOC participants and observers in blogs such as Robbins (2013).
Many studies have presented participant demographics and it was observed that a large majority of participants were from North America and Europe. There were very few participants, if any at all, from Asia and Africa with a few from South East Asia (deWaard, Abajian, Gallagher, et al., 2011; Kop, 2011; Koutropoulos, et al., 2012). This is hardly surprising due to the possible barriers of access to online learning that can be presumed in those parts of the world, both technological and linguistic (see Liyanagunawardena, 2012 for a discussion of the similar problems in the introduction of online learning in Sri Lanka). However, a number of Asian and African learners participated in MobiMOOC (Koutropoulos, et al. 2012). There are two plausible reasons: It could be that the course content of the MobiMOOC (mLearning) is of more interest to learners from those regions or it could be that mLearning is significantly more accessible in those places. deWaard, Koutropoulos et al. (2011) report that 77.5% of the participants in their survey had used mobile devices to access the course even though it was not required, indicating that the flexibility offered by mLearning is attractive for those MOOC participants. This shows the potential for exploring the possibility of mLearning MOOC provision in the developing world, both for increasing and widening participation.
As discussed already, some of the researchers who used data generated through MOOCs have limited their qualitative analysis due to the large volumes. Regarding MOOC participant approaches, Fini (2009) similarly argues that
people seem to be torn between the time-saving advantage offered by the ‘Daily’ solution [an email message sent daily by the facilitators with a summary of the topics of the existing conversation] and the multi-faceted, time-consuming alternative represented by direct access to unfiltered information.
Due to the large volumes of data generated by the ‘massive’ number of students engaged in a MOOC, being up-to-date with the ongoing discussions can be challenging or even overwhelming. Exploring the strategies used by students who continue to be active participants in a MOOC could provide some insight into possible solutions to the information overload in a MOOC environment for both other MOOC learners and for MOOC researchers. In fact, Milligan, Margaryan, and Littlejohn (2013) show that all but one active participants of Change11 MOOC have participated in previous MOOCs and question whether a learner has to learn how to learn in a MOOC. While many MOOC participants struggle to keep going “there’s some people who are everywhere you turn in the Change11 MOOC: there’s this group of people who are inspirational, just phenomenal the way they just keep going and they know their way around it” (Milligan, Margaryan, & Littlejohn 2013). Confidence gained after successful participation in one MOOC together with network effects where people start building communities and networks in one MOOC then moving together onto another MOOC may result in more previous MOOC participants taking up other MOOCs.
It is acknowledged that MOOCs have high withdraw/dropout rates (Koutropoulos, et al., 2012); however, data on completion rates of MOOCs are not readily available. According to Jordan’s (2013) collated completion rates for 24 MOOCs (as of March 11th, 2013), the highest completion rate achieved was 19.2% on Functional Programming Principles in Scala, a MOOC offered by Coursera in 2012. The majority of MOOCs had completion rates of less than 10%. There is very little known about the experiences of non-completing MOOC participants (Koutropoulos, et al., 2012). This would be an interesting and useful avenue to explore further in future research, though engaging those who have started but not finished an educational course in research into the reasons for their non-completion is likely to be difficult.
Motivation is identified as an important contributor to student engagement in a MOOC by Milligan, Margaryan, and Littlejohn (2013). One can speculate about an individual’s motivation to participate in a MOOC: the desire to achieve an academic credential at a reduced cost, personal enrichment, and/or self satisfaction. However, why individuals participate in MOOCs has yet to be explored. It would be valuable to learn about the actual motivations in place, the percentage of participants taking up MOOCs for those reasons, and to know how those motivations might vary from one course or discipline or even provider to another.
Recognition for MOOCs and their accreditation is another area of debate. MOOCs run by educational technology companies such as Coursera and EdX provide the option to pay for certification. For example, Coursera offers proctored exams for a fee, which will earn (if successful) certification. On the other hand, most MOOCs offer badges for completion of either the full course or each unit (or week of work, for example OLDSMOOC by the Open University). Some MOOC-offering institutions and/or instructors provide a Statement of Accomplishment for successful students. However, these generally do not carry college credits. Recently, the American Council on Education recommended five Coursera courses for college credit (Coursera, 2013) while a California bill is seeking credit for students taking faculty approved courses online (Levin, 2013). However, given that existing in-person “closed book” examinations are subject to cheating (Shimbun, 2011) and that at least one enterprising IT worker in the US successfully outsourced his own job to China (Kim, 2013), the challenges of validating the assessments of MOOC participants are clear.
MOOCs have created wide interest as a change agent in higher education, and the peer-reviewed research literature on them is growing but still limited. Many articles published to date have discussed empirical evidence from case studies, the influence on higher education structure, or educational theory relating to MOOCs. While there is research into the learner perspective, neither the creator/facilitator perspective nor the technological aspects are being widely researched (or at least such research has not yet been published). MOOCs generate a plethora of data in digital form for interested researchers. However, this volume has so far limited researchers to analysing only a tiny portion of the available data, restricting our understanding of MOOCs. There are further interesting research avenues such as cultural tensions within courses and the ethical aspects of using data generated by MOOC participants still to be explored.
Adams, A. A., Liyanagunawardena, T. R., Rassool, N., & Williams, S. (in press). Use of open educational resources in higher education. British Journal of Educational Technology.
Anderson, T., & McGreal, R. (2012). Disruptive pedagogies and technologies in universities. Education, Technology and Society, 15(4), 380-389. Retrieved from http://www.ifets.info/journals/15_4/32.pdf.
Ardis, M. A., & Henderson, P. B. (2012). Software engineering education (SEEd): Is software engineering ready for MOOCs? ACM SIGSOFT Software Engineering Notes, 37(5), 14-14.
Bell, F. (2010a). Connectivism: Its place in theory-informed research and innovation in technology-enabled learning. The International Review of Research in Open and Distance Learning, 12(3), 98-118. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/902/1827
Bell, F. (2010b). Network theories for technology-enabled learning and social change: Connectivism and actor network theory. In L. Dirckinck-Holmfeld, V. Hodgson, C. Jones, M. de Laat, D. McConnell & T. Ryberg (Eds.), Proceedings of the 7th International Conference on Networked Learning 2010 (pp. 526-533).
Bonino, R (2012). International student mobility: Status, trends and possible impact of e-learning and social media, media applications for business, learning and society. VCOBAM Conference Proceedings, 3(1), 81-91.
Bremer, C. (2012).New format for online courses: The open course future of learning. Proceedings of eLearning Baltics eLBa 2012. Retrieved from http://www.bremer.cx/vortrag67/Artikel_elba2012_opco_bremer.pdf
Bull, D. (2012). From ripple to tsunami: The possible impact of MOOCs on higher education. DE Quarterly, 2012 Spring, 10-11.
Butin, D. W. (2012). What MIT should have done. eLearn Magazine, June 2012. Retrieved from http://elearnmag.acm.org/featured.cfm?aid=2263018
Cabiria, J. (2012). Connectivist learning environments: Massive open online courses.The 2012 World Congress in Computer Science Computer Engineering and Applied Computing, Las Vagas, July 16-19, 2012. Retrieved from http://elrond.informatik.tu-freiberg.de/papers/WorldComp2012/EEE6065.pdf
Casey, D. M. (2008). A journey to legitimacy: The historical development of distance education through technology. TechTrends, 52(2), 45–51.
Chamberlin, L., & Parish, T. (2011). MOOCs: Massive open online courses or massive and often obtuse courses? eLearn Magazine, August 2011. Retrieved from http://elearnmag.acm.org/featured.cfm?aid=2016017
Corbyn, Z. (2012, December 6). This could be huge... Times Higher Education. Retrieved from http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=422034&c=1
Coursera. (February 07, 2013). Five courses receive college credit recommendations. Retrieved from http://blog.coursera.org/post/42486198362/five-courses-receive-college-credit-recommendations
Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility. Journal of Interactive Media in Education, 3. Retrieved from http://www-jime.open.ac.uk/jime/article/viewArticle/2012-18/html
deWaard, I. (2011). Explore a new learning frontier: MOOCs. Learning Solutions Magazine. Retrieved from http://www.learningsolutionsmag.com/articles/721/explore-a-new-learning-frontier-moocs/print
deWaard, I., Abajian, S., Gallagher, M. S., Hogue, R., Keskin, N., Koutropoulos, A., & Rodriguez, O. C. (2011). Using mLearning and MOOCs to understand chaos, emergence, and complexity in education. The International Review of Research in Open and Distance Learning, 12(7), 94-115. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1046/2043
deWaard, I., Koutropoulos, A., Keskin, N., Abajian, S. C., Hogue, R., Rodriguez, C. O., & Gallagher, M. S. (2011). Exploring the MOOC format as a pedagogical approach for mLearning. In Proceedings of 10th World Conference on Mobile and Contextual Learning (pp. 138-145).
Downes, S. (2008). Places to go: Connectivism & connective knowledge. Innovate, 5(1). Retrieved from http://www.innovateonline.info/index.php?view=article&id=668
Ellis, D. (1989). A behavioural approach to information retrieval system design. Journal of Documentation, 45, 171–212.
Ellis, D., & Haugan, M. (1997). Modelling the information seeking patterns of engineers and research scientists in an industrial environment. Journal of Documentation, 53, 384–403.
Esposito, A. (2012). Research ethics in emerging forms of online learning: issues arising from a hypothetical study on a MOOC. Electronic Journal of e-Learning, 10(3). Retrieved from http://www.ejel.org/issue/download.html?idArticle=210
Fini, A. (2009). The technological dimension of a massive open online course: The Case of the CCK08 course tools. The International Review of Research in Open and Distance Learning, 10(5). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/643/1410
Fink, A. (2010). Conducting research literature reviews: From Internet to paper (3rd ed.). Thousand Oaks, California: Sage.
Fournier, H., Kop, R., & Sitlia, H. (2011). The value of learning analytics to networked learning on a personal learning environment. Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 104-109). ACM, New York.
Frank, S. J. (2012). Review: MITx’s online circuit and analysis course. Spectrum, 49(9), IEEE, 27-28.
Gao, F., Luo, T., & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008-2011. British Journal of Educational Technology, 43(5), 783-801.
Goldberg, C. (2001, April 4). Auditing classes at M.I.T., on the Web and free. The New York Times. Retrieved from http://www.nytimes.com/2001/04/04/us/auditing-classes-at-mit-on-the-web-and-free.html
Hyman, P. (2012). In the year of disruptive education. Communications of the ACM, 55(12), 20-22.
Jordan, K. (2013, March 11, 2013). MOOC completion rates: The data. Retrieved from http://www.katyjordan.com/MOOCproject.html
Kim, S. (2013, January 17). US software developer caught outsourcing his job to China. ABC News online.Retrieved from http://abcnews.go.com/story?id=18230346
Kirkwood, K. (2010). The wisdom of the clouds: Distributed learning, MOOCs, edupunks, and the challenge to formal education. In Proceedings of the Second International Workshop on Open Source and Open Content (WOSOC) 2010 (pp. 96-103). Gunadarma University, Jakarta, Indonesia.
Kohil, S., & Kumar, E. (2011). Evolution of user dependent model to predict future usability of a search engine. In Information and Communication Technologies (WICT), 2011 World Congress on Information and Communication Technologies (pp. 285-290).
Kop, R., & Fournier, H. (2010). New dimensions to self-directed learning in an open networked learning environment. International Journal of Self-Directed Learning, 7(2).
Kop, R. (2011). The challenges to connectivist learning on open online networks: Learning experiences during a massive open online course. The International Review of Research in Open and Distance Learning, Special Issue-Connectivism: Design and Delivery of Social Networked Learning, 12(3). Retrieved from http://nparc.cisti-icist.nrc-cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=18150443
Kop, R., Fournier, H., & Mak, J. S.F. (2011). A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses. International Review of Research in Open and Distance Learning, Special Issue - Emergent Learning, Connections, Design for Learning, 12(7), 74-93. Retrieved from http://nparc.cisti-icist.nrc-cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=19040607
Kop, R., & Carroll, F. (2012). Cloud computing and creativity: Learning on a massive open online course. European Journal of Open, Distance and E-Learning. Retrieved from http://www.eurodl.org/index.php?article=457
Koutropoulos, A., & Hogue, R. J. (2012). How to succeed in a MOOC-Massive online open course. eLearning Guild, October 08, 2012. Retrieved from http://www.cedma-europe.org/newsletter%20articles/eLearning%20Guild/How%20to%20Succeed%20in%20a%20MOOC%20-%20Massive%20Online%20Open%20Course%20(Oct%2012).pdf
Koutropoulos, A., Gallagher, M. S., Abajian, S. C., deWaard, I., Hogue, R. J., Keskin, N. Ö., & Rodriguez, C. O. (2012). Emotive vocabulary in MOOCs: Context & participant retention. European Journal of Open, Distance and E-Learning. Retrieved from http://www.eurodl.org/?p=Special&sp=init2&article=507
Lane, A., & McAndrew, P. (2010). Are open educational resources systematic or systemic change agents for teaching practice? British Journal of Educational Technology, 41(6), 952-962.
Levin, T. (March 12, 2013). California bill seeks campus credit for online study. The New York Times. Retrieved from http://www.nytimes.com/2013/03/13/education/california-bill-would-force-colleges-to-honor-online-classes.html?_r=1&
Levy, D. (2011). Lessons learned from participating in a connectivist massive online open course (MOOC). Proceedings of the Chais conference on instructional technologies research 2011: Learning in the technological era. Eshet-Alkalai, Y., Caspi, A., Eden, S., Geri, N. & Yair, Y. (Eds.), The Open University of Israel, Raanana, 31-36. Retrieved from http://www.openu.ac.il/research_center/chais2011/download/f-levyd-94_eng.pdf
Liyanagunawardena, T. R. (2012). Information communication technologies and distance education in Sri Lanka: A case study of two universities (PhD Thesis). University of Reading, Reading.
MacIsaac, P. L. (2012). What a librarian can bring to your open online course development team. EDUCAUSE 2012. Retrieved from http://auspace.athabascau.ca:8080/bitstream/2149/3218/2/MacIsaac_MOOCs_and_Information_literacy_draft_paper.docx
Mackness, J., Mak, S. & Williams, R. (2010). The ideals and reality of participating in a MOOC. In Proceedings of the 7th International Conference on Networked Learning 2010, University of Lancaster, Lancaster, 266-275.
Mahraj, K. (2012). Using information expertise to enhance massive open online courses. Public Services Quarterly, 8(4), 359-368.
Mak, S., Williams, R., & Mackness, J. (2010). Blogs and forums as communication and learning tools in a MOOC. In Networked Learning Conference, University of Lancaster, Lancaster, 275-285.
Martin, F. G. (2012). Will massive open online courses change how we teach? Communications of the ACM, 55(8), 26-28. Retrieved from http://dl.acm.org/citation.cfm?id=2240246
Masters, K. (2011). A brief guide to understanding MOOCs. The Internet Journal of Medical Education, 1(2).
McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital practice, SSHRC Knowledge Synthesis Grant on the Digital Economy. Retrieved from http://www.edukwest.com/wp-content/uploads/2011/07/MOOC_Final.pdf
Mehaffy, G.L. (2012). Challenge and change. EDUCAUSE Review, September/October 2012, 25-41. Retrieved from http://net.educause.edu/ir/library/pdf/ERM1252.pdf
Mehlenbacher, B. (2012). Massive open online courses (MOOCs): Educational innovation or threat to higher education? Proceedings of the Workshop on Open Source and Design of Communication, ACM, New York (p. 99).
Milligan, C., Margaryan, A., & Littlejohn, A. (2013). Patterns of engagement in massive open online courses. Journal of Online Learning with Technology (Special Issue on MOOCs) – Under Review.
Roberts, G. (2012). Open Line Project (Final Report). Oxford Brookes University. Retrieved from http://www.heacademy.ac.uk/assets/documents/oer/brookes_final_report_101012.pdf
Robbins, J. (March 25, 2013). The ethics of MOOCs. Retrieved from http://www.insidehighered.com/blogs/sounding-board/ethics-moocs
Roderick, S. (2008). Rethinking (e)learning: A manifesto for connected generations. Distance Education, 29(2), 153-164.
Rodriguez, C. O. (2012). MOOCs and the AI-Stanford like courses: Two successful and distinct course formats for massive open online courses. European Journal of Open, Distance and E-Learning. Retrieved from http://www.eurodl.org/?p=Special&sp=init2&article=516
Sadigh, D., Seshia, S. A., & Gupta, M. (2012). Automating exercise generation: A step towards meeting the MOOC challenge for embedded systems. In Proceedings of the Workshop on Embedded Systems Education (WESE). Retrieved from http://www.eecs.berkeley.edu/~dsadigh/WESE12.pdf
Schrire, S., & Levy, D. (2012). Troubleshooting MOOCs: The case of a massive open online course at a college of education. In World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2012(1), 761-766.
Shema, H., Bar-Ilan, J., & Thelwall, M. (2012). Research blogs and the discussion of scholarly information. PLoS ONE, 7(5).
Shimbun, Y. (2011). Web exam leak rocks elite school / Poster asked for answers in mathematics, English on Yahoo! Japan site. February 28, 2011. Retrieved from www.yomiuri.co.jp/dy/national/T110227003929.htm
Siemens, G. (2005). Connectivism - A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2(1), 3–10.
Stewart, B. (2010). Social media literacies and perceptions of value in open online courses. Retrieved from http://portfolio.cribchronicles.com/wp-content/uploads/2012/11/612701_Social_Media_Literacies_MOOCs.pdf
Tschofen, C., & Mackness, J. (2012). Connectivism and dimensions of individual experience. The International Review of Research in Open and Distance Learning, 13(1), 124-143. Retrieved, from http://www.irrodl.org/index.php/irrodl/article/download/1143/2117
Vardi, M. Y. (2012). Will MOOCs destroy academia? Communications of the ACM, 55(11). Retrieved from http://dl.acm.org/citation.cfm?id=2366317
Vihavainen, A., Luukkainen, M., & Kurhila, J. (2012). Multi-faceted support for MOOC in programming. SIGITE’12, October 11–13, 2012, Calgary, Alberta, Canada, ACM. Retrieved http://sigite2012.sigite.org/wp-content/uploads/2012/08/session13-paper02.pdf
Weller, M. (2007). Learning objects, learning design, and adoption through succession. Journal of Computing in Higher Education, 19(1), 26–47.
Williams, S., Terras, M., & Warwick, C. (in press). What people study when they study Twitter: Classifying Twitter related academic papers. Journal of Documentation, 69(3), 384-410.
Yin, R. K. (2003). Case study research design and methods (3rd ed.). Thousand Oaks: Sage.
Zimmer, M. (2010). «But the data is already public»: On the ethics of research in Facebook. Ethics and Information Technology, 12(4), 313-325.
1 1. However, it is worth mentioning that prior to the wide acceptance of a LMS as the medium to publish course content, institutions used television broadcasts. For example, many people who were not engaged in the Open University courses watched various Open University programs telecasted, particularly when video recorders became cheaply available. This unregistered informal accessing of educational materials reduced when the Open University main broadcasts stopped and course materials were delivered primarily through the post (on CD and DVD) and later via online delivery. Similar closure happened when early adopters of online provision of course content through the open web closed off access by placing it behind authentication restrictions in LMS (such as Moodle and Blackboard) (Goldberg, 2001).
2 2. Automatic fuzziness in search engines may also have reported “Management of Change” (MOC) results, particularly given the overlap of “Management of (Organizational) Change” MOOC/MOC terminology (Kohil & Kumar, 2011).
3 2. 3. The initial full list included: Agency, Apprenticeship, Autonomy, Case study, Chaos theory, Cloud computing, Comparative, Connectivism, Actor Network Theory, Dangers, Educational theory, Ethics, Framework, Futures, Higher education, Information science, Introductory, Learner experience, Learning analytics, Opportunities, Pedagogies, Retention, Support, Technology, Trends, Opinions, and Reviews.