Pushing Toward a More Personalized MOOC: Exploring Instructor Selected Activities, Resources, and Technologies for MOOC Design and Implementation

This study explores the activities, tools, and resources that instructors of massive open online courses (MOOCs) use to improve the personalization of their MOOCs. Following email interviews with 25 MOOC and open education leaders regarding MOOC personalization, a questionnaire was developed. This questionnaire was then completed by 152 MOOC instructors from around the world. While more than 8 in 10 respondents claimed heavy involvement in designing their MOOCs, only one-third placed extensive effort on meeting unique learner needs during course design, and even fewer respondents were concerned with personalization during course delivery. An array of instructional practices, technology tools, and content resources were leveraged by instructors to personalize MOOC-based learning environments. Aligning with previous research, the chief resources and tools employed in their MOOCs were discussion forums, video lectures, supplemental readings, and practice quizzes. In addition, self-monitoring and peerbased methods of learner feedback were more common than instructor monitoring and feedback. Some respondents mentioned the use of flexible deadlines, proposed alternatives to course assignments, and introduced multimedia elements, mobile applications, and guest speakers among the ways in which they attempted to personalize their massive courses. A majority of the respondents reported modest or high interest in learning new techniques to personalize their next MOOC offering. Keyword: massive open online courses (MOOCs), personalization, instructional design, MOOC instructors Pushing Toward a More Personalized MOOC: Exploring Instructor Selected Activities, Resources, and Technologies for MOOC Bonk, Zhu, Kim, Xu, Sabir, and Sari


Résumé de l'article
This study explores the activities, tools, and resources that instructors of massive open online courses (MOOCs) use to improve the personalization of their MOOCs. Following email interviews with 25 MOOC and open education leaders regarding MOOC personalization, a questionnaire was developed. This questionnaire was then completed by 152 MOOC instructors from around the world. While more than 8 in 10 respondents claimed heavy involvement in designing their MOOCs, only one-third placed extensive effort on meeting unique learner needs during course design, and even fewer respondents were concerned with personalization during course delivery. An array of instructional practices, technology tools, and content resources were leveraged by instructors to personalize MOOC-based learning environments. Aligning with previous research, the chief resources and tools employed in their MOOCs were discussion forums, video lectures, supplemental readings, and practice quizzes. In addition, self-monitoring and peer-based methods of learner feedback were more common than instructor monitoring and feedback. Some respondents mentioned the use of flexible deadlines, proposed alternatives to course assignments, and introduced multimedia elements, mobile applications, and guest speakers among the ways in which they attempted to personalize their massive courses. A majority of the respondents reported modest or high interest in learning new techniques to personalize their next MOOC offering.

Introduction
Massive open online courses (MOOCs) and their many derivatives allow for thousands of learners to simultaneously engage in a learning experience (Bonk, Lee, Reeves, & Reynolds, 2018;Pappano, 2012;Siemens, 2012b). While a relatively recent phenomenon, MOOCs have the potential for large scale usage and impact by helping learners in developing parts of the world obtain access to education (Bowman, 2012;Jagannathan, 2015). While promising in terms of access, many studies point to retention issues in MOOCs (e.g., Hew & Chueng, 2014;MOOC @ Edinburgh 2013-Report #1, 2013Yuan, Powell, & Olivier, 2014).
Despite MOOCs being promoted and leveraged by universities and international organizations for several years, there are scant empirical studies evaluating how MOOCs and similar types of open educational courses address diverse learner needs through the personalization of the course content and experiences.
The purpose of this study was to better understand how MOOC instructors adapt their courses to enhance or personalize MOOC design and delivery. Personalization, however, is a complex construct (Bethke, 2016) and hard to succinctly define or agree upon. In a meta-review of the literature on personalization, Fan and Poole (2006) caution that: At the conceptual level, personalization means different things to different people in different fields. For architects, personalization means creating functional, pleasant personal spaces; for social scientists it is a way of enhancing social relationships and building social networks; for some computer scientists, personalization is a toolbox of technologies to enhance the Web experience through graphic user interface design. Different conceptualizations in turn dictate different research methodologies and implementations. Cognitive scientists resort to explicit mental modeling to differentiate users, whereas e-commerce marketers rely on user profiles and purchase records to segment customers. (p. 181) Seeking to provide a common theoretical framework from which to study personalization and aid in the design of more personalized systems, Fan and Poole (2006)  While personalization is a difficult concept to pin down, the goal of this study was to determine the types of activities, resources, and technology tools that can enhance the quality, and ultimately the retention rates, of MOOCS. Unlike most MOOC research (Zhu et al., 2018), the MOOC instructor perspective is the primarily focus of this study.
According to Kop (2011), instructors are one of five key elements to a successful MOOC; the other four are learners, topic, material, and context. Of the five elements that Kop (2011) delineates, instructors are one of the least researched (Veletsianos & Shepherson, 2016;Zhu et al., 2018). To address this gap, in the present study, MOOC personalization was explored from an instructor perspective. More specifically, this study focuses on the four research questions listed below.
1. How much self-identified effort do instructors place on addressing unique learner needs in the design and development of their MOOCs? 2. What are the personalization practices of MOOC instructors in terms of the pedagogical activities and task structures employed?
3. What are the personalization practices of MOOC instructors in terms of content resources and associated technology tools employed? 4. How would these instructors structure their next MOOC differently in terms of personalization?
To answer these questions, this study explores the practices of experienced MOOC instructors. By interviewing experts to develop a questionnaire, and then surveying MOOC instructors from a wide range of disciplines and locales, it was hoped that this research would help reveal instructional design and delivery practices toward personalization that could enhance the quality and long-term impact of MOOCs.
There is some early history to build upon in terms of MOOC personalization. In 2010, for instance, a MOOC titled "Personal Learning Environments Networks and Knowledge" (aka PLENK2010) was taught with personalized learning as an objective (Kop, Fournier, & Mak, 2011). Levy (2011) asserts that this particular MOOC used connectivistic theory and ideas throughout. Such a course later became categorized as a "cMOOC" (Reeves & Hedberg, 2014;Siemens, 2012a). A cMOOC is more focused on knowledge generation and sharing than on knowledge consumption and passive forms of learning . It is in the loosely organized learning networks or spaces of a cMOOC that the facilitator (or instructor) helps foster connections between the participants and the open sharing of knowledge and resources . PLENK2010 required participants to use social media, including tools such as Second Life and Facebook, to share and co-create knowledge, thereby enhancing learner motivation through the creation of personal networks (Kop, 2011;Kop et al., 2011). In effect, there was enhanced learner choice in how participants would engage with and reflect upon the content and ideas related to the course .
In 2011, another type of MOOC emerged: the xMOOC (Sneddon, 2015). xMOOCs were based on interactive media such as videos, texts, and lectures that leveraged structured learning pathways on central platforms (Sneddon, 2015). Despite the sudden popularity of xMOOCs within Ivy league universities and abundant media attention (Pappano, 2012;Rodriguez, 2012), there was extensive concern related to how instructors could be responsive to learners in such large-scale courses. Unlike cMOOCs, xMOOCs focused more on content delivery and individual learning. As such, they were criticized for adopting instructional approaches more akin to behavioral theories and models, rather than learning through peers and social networks as with cMOOCs (Bates, 2012;Bonk et al., 2018;Daniel, 2012).

MOOC Personalization
While several researchers have evaluated MOOC elements for personalization, such as course design, assessments, and means of content delivery (de Oliveira Fassbinder, Fassbinder, & Barbosa, 2015), there is a dearth of empirical studies that specifically investigate MOOC personalization from instructor perspectives (Veletsianos & Shepherson, 2016;Zhu et al., 2018). Instead, much of the focus of the literature on MOOCs examines learner completion trends and participant-based data (Balch, 2013;Heutte, Kaplan, Fenouillet, Caron, & Rosselle, 2014;Jordan, 2014). MOOC research also trends towards descriptive case studies based on an individual MOOC (Fini, 2009), rather than analyzing a spectrum of MOOCs through meta-analyses. However, in one meta-analysis of MOOC-related studies from 2008, Nkuyubwatsi (2013 determined that MOOCs provided adult learners opportunities to engage with materials while personalizing their learning environment through content manipulation.
Recently, Hayworth (2016) suggested that a range of technologies can help personalize learning environments, such as social bookmarking, wikis, blogs, image sharing, and collaborative tools. He also notes that such personalized learning environments (PLEs) have significant implications for distance educators, instructional designers, life-long learners, and administrators in terms of the mixing and sharing of content and resources, monitoring and managing the learning process, making learning-related suggestions and recommendations, content creation, and so on (Hayworth, 2016). Hayworth cautioned, however, against placing too much emphasis on technology-based solutions. As he and others (e.g., McLoughlin& Lee, 2010) point out, adult learners often exhibit a preference for learning which is social, participatory, and media supported rather than technocentric (Hayworth, 2016;McLoughlin & Lee, 2010).
Too often researchers exploring personalization in online environments focus on technology infrastructures rather than the pedagogical scaffolds provided by instructors and instructional designers to support learners (Aleven, Stahl, Schworm, Fischer, & Wallace, 2003). According to researchers like McLoughlin and Lee (2010), online instructor roles, instructional practices, and design decisions must be evaluated holistically to better understand how online personalized learning environments can be crafted.
Personalized forms of learning are grounded in learner-centered and constructivist learning perspectives (Reigeluth, Myers, & Lee, 2017;Watson & Watson, 2017). Such theoretical viewpoints attempt to address specific learner needs based on their learning interests and preferences, prior knowledge and experiences, and overall backgrounds (Levy, 2011;Xu, Huang, Wang, & Heales, 2014). In effect, personalization is the means used to tailor a particular learning environment's resources, tools, activities, and content to better address individual learner needs, skills, and issues (Kelly, 2016). From a learning theory standpoint, the personalization of instructional spaces lends itself to a more learner-centered paradigm that can address diverse learner requirements, competencies, and backgrounds (Green, Facer, Rudd, Dillon, & Humphreys, 2005). Also vital from this point of view is learner-learner interaction and dialogue (Brown, Collins, & Duguid, 1989;Reigeluth et al., 2015). Peers can often offer guidance that is more relevant to true learner needs and experiences (Rogoff, 1990;Vygotsky, 1978). Siemens (2007) offered a simplified definition of personalized learning that includes two key elements: (1) the tools, and (2) the ideals that guide the design. His colleague, Downes (2016), argued that the phrase "personalized learning" has appeared so much in the educational literature during the past decade that it has begun to "lose its meaning" (para. 1). According to Downes, some refer to personalized learning as the pedagogical differentiation of instruction according to different participant variables such as learning styles and preferences of learning, whereas others refer to decisions made related to the order or pathways in which the curriculum can be offered. Instead of externally provided personalized environments, Downes claims that personalized learning must empower learners by allowing them to customize and organize their own learning directives. From this viewpoint, greater emphasis is placed on the learner deciding what to learn, how to learn, and where to learn (Downes, 2016).
While empathizing with Downes' (2016) perspective, this study focuses on how MOOC instructors adapt their instruction and set of course resources and tasks to personalize the learning process in a MOOC. As a result, for the purpose of our study, we chose to define personalization as: the process by which MOOC instructors adapt their courses and instructional practices to meet diverse learner needs, skills, prior experiences, and situations.
In an example of MOOC personalization with extensive peer reliance, Kim and Chung (2015) mapped out how they attempted to create an ecology of learning in their MOOC "Designing a New Learning Environment," which was hosted on the Stanford Venture-Lab/NovoEd MOOC platform. The participants in this MOOC supported one another through social media like Twitter and discussion forum solicitations when there was missing or incomplete information (Kim & Chung, 2015). For instance, some participants responded to peer requests by creating low bandwidth versions of instructor videos for those who lived in developing regions of the world, and others translated these videos into other languages and added words and nuances that were specific to the local language to make them understandable to target groups (Kim & Chung, 2015). Instructors facilitated a space which allowed learners to personalize content for their peers so that issues of access and linguistic barriers would not hinder learning (Kim & Chung, 2015).
Similarly, Severance (2015), who has taught three different and highly successful MOOCs (i.e., Python Programming, Programming for Everybody, and Internet History, Security, and Society) has attempted to personalize his MOOC offerings by taking a learner point of view. For instance, he has designed unique "Office Hours" in cafes, hotel lobbies, and other locations wherein he locally meets his global participants in cities around the world to discuss the course with them and get their suggestions for improvement (Severance, 2015). He also has a YouTube channel specific for his MOOCs that features personal stories and contributions from participants that appear as "Voices of the Students in MOOCs" (Severance, 2015). The creation of the YouTube channel and "Office Hours" allows participants to integrate their life experiences with MOOC experiences facilitating a unique type of blended personalized learning experience (Severance, 2015).
The primary intent of the present study is to explore the extent to which MOOC instructors use such forms and types of personalization practices in their MOOCs. Just how is personalization operationalized in the design and delivery of MOOCs?

Method
To understand how MOOC instructors personalize their courses to best meet individual learner needs, both quantitative and qualitative data were employed. The study is comprised of two distinct datasets: (1) email interviews of 25 international MOOCs experts related to how to personalize the MOOC experience; these 97 experts were selected since they all had recently contributed to an edited book on MOOCs and open education, and (2) an online survey questionnaire which was sent via SurveyMonkey to more than 1,026 MOOC instructors, of which 152 qualified and completed the instrument.

Expert Email Interviews
It is important to mention that the email interviews provided the thematic and categorical foundations from which the survey instrument was created. The experts had useful and insightful advice and pedagogical ideas that helped in the design of the survey instrument. To create the database of MOOC instructors to whom the questionnaire would be distributed, the names and affiliations of the MOOC instructors, course title, subject area, course URL, institution, course start and end date, and course duration for over 1,000 MOOCs were mined from Class Central and the MOOC List (which included courses from Open2study, Canvas, NovoEd, Blackboard, iversity, and Kadenze).

Web-Based Survey
Additionally, the researchers directly searched individual vendors and organizational sites (e.g., specific vendor lists from Coursera, edX, FutureLearn, and Open2study) to ensure the maximum scope within the MOOC listings database. The researchers further compiled a list of approximately 50 Korean MOOCs (i.e., K-MOOCs) (http://www.kmooc.kr/). Next, the researchers cross checked the database for redundancy and errors. The final list included MOOC instructors from universities, organizations, and institutions in more than two dozen countries, including Australia, Belgium, Canada, China, Denmark, Germany, Grenada, India, Ireland, Italy, Japan, Korea, Macau, Mexico, the Netherlands, New Zealand, Norway, Russia, Singapore, South Africa, Spain, Sweden, Switzerland, the United Kingdom, and the United States. The largest percentage of participants were from institutions in the United States.

Expert Email Interviews
In terms of the email interviews, some of the experts in the field of MOOCs and open education argued for greater use of collaborative projects, whereas others mentioned the need for MOOC toolkits or platforms that are designed for access in low bandwidth conditions as a means to personalize the experience. Among these experts, a senior education specialist from the Open Learning Campus of the World Bank indicated that they attempted to incorporate badging and customized discussion forums as a means to personalize the experience. Another MOOC expert in the Philippines stated that "one feature that we have integrated into our MOOCs... to personalize learning is to allow the learner to choose whether to learn through the video lessons, text lessons, or podcasts." These expert interviews were thematically coded to develop the Web survey of MOOC instructors, mentioned earlier. Survey questions were drafted related to how MOOC instructors fostered feedback, interaction, and engagement in the learning process. Questions were also drafted related to the types of course resources that were embedded to help personalize the MOOC.

Online Survey Results
Some of the online survey findings are recapped below starting with key demographic data related to the instructor experience with MOOCs. Of the 978 valid survey requests, 152 individuals completed the survey.
This 15.5% response rate is considered more than acceptable for opt-in, online surveys (Cho & LaRose, 1999). These 152 instructors taught MOOCs in fields such as science, social sciences, the humanities, engineering, medicine, business, language, mathematics, art, and law. Nearly one-third of the MOOC respondents were from medical and health sciences, or from the field of education. Another 9% came from the field of business, and 9% from computer science (see Figure 1). The prior MOOC teaching experience among the survey participants was quite varied. Of these 152 respondents, roughly 55.3% had taught just one MOOC, 19.7% had taught two MOOCs, and nearly 25% had taught three or more MOOCs in the past. In contrast, more than half of these instructors (n=84; 55.2%) had never completed a MOOC as a learner, while 25 (16.5%) had completed one MOOC in the past. It is also important to note that 43 (28.3%) of the respondents had completed two or more MOOCs.
The instructors were requested to reflect on their instructional practices for their most recent MOOC.

Research Question #1
How much self-reported effort do instructors place on addressing the unique participant or learner needs in the design and development of their MOOCs?
As course personalization can depend on an instructor's involvement in the course design, participants (n=152) were asked to rank on a scale of 1 (low) to 10 (high) their involvement in designing the course.
When collapsed to three categories (i.e., 1-3 Low; 4-7 Medium; and 8-10 High), only five instructors (3.3%) indicated low involvement in designing the course, and 17 (11.2%) exerted modest involvement (see Figure   2). The remaining 130 MOOC instructors (85.5%) indicated a high level of involvement, of which 94 (72.3%) instructors marked "10" out of 10 on the scale. The average rating was 8.92 (SD=1.88); indicating heavy involvement from instructors in the design of their MOOCs. Given that the vast majority of the respondents were extensively involved in the design of their most recent MOOC, they had some influence over the degree to which that course was adapted to learner needs and

Research Question #2
What are the personalization practices of MOOC instructors in terms of the pedagogical activities and task structures employed?
Participant interaction is another means to address learner needs as exemplified in the differences between cMOOCs and xMOOCs. In this study, MOOC instructors indicated that they attempted to foster learner-tolearner connections and interactions to some degree, with an average of 6.24 on a scale of 1 (low) to 10 (high) (n=137). However, when asked about the ways in which peer interaction was encouraged in their most recent MOOC, the methods selected were limited. When presented with a list of nine options (including "not applicable"), more than 80% of MOOC instructors indicated that they relied on system-built discussion forums for learner-learner forms of interaction. No other resource or activity was employed by more than half of the respondents. For instance, only one in four instructors checked that they used pairbased assignments or tasks (e.g., critical friend activities). Furthermore, synchronous forms of meetings or conferencing were used by less than one in 10 of the respondents. Break-out discussion forums or groups were employed by 31 (22.6%) of the respondents, whereas local meet ups were being used by 22 (16.1%) of the MOOC instructors.
As mentioned earlier, Downes (2016) argues that learner empowerment and choice is a key part of personalization. When asked about the structures that they provided in their most recent MOOC from a list of ten items, the survey participants (n=126) primarily relied on optional readings (74.6%) and learner selected incentives such as certificates, badges, or course credit (64.3%). The respondents also indicated that they employed course tasks and assignments (38.1%), learner discussion and negotiation of content  Personalization also requires monitoring learner progress and awareness of learning accomplishments (Reigeluth et al., 2015). In terms of monitoring or tracking learner progress in a MOOC, 42.3% of MOOC instructors (n=137) relied on learner self-monitoring and evaluation. Approximately, one in three (34.3%) employed modular or unit-based forms of assessment. About one in four (24.8%) used weekly or daily reports from learning analytics. A similar percentage (23.4%) used moderator, tutor, or teaching assistant feedback to monitor or track progress. While 13.9% used a hybrid system of tracking learner progress and participation, another 13.1% relied on peer-based reports. Just 7.3% employed personal tracking from the instructor; in contrast 14.6% noted that learner progress was not tracked.
Human and system forms of feedback are another mechanism to address learner needs in a MOOC. Given the typically large number of MOOC participants, it was not too surprising that peer feedback was used by 87 (64.4%) of the 135 instructors who responded to this "check all that apply" question. In addition, 78 (57.8%) of the respondents relied on computer or system-based forms of feedback (57%) (see Figure 5).

Research Question #3
What are the personalization practices of MOOC instructors in terms of content resources and associated technology tools employed?
There are many resources, activities, and technology tools from which to make attempts to personalize Enhancing the intelligence of the system has the potential to result in greater personalization of the course.
As more than half of the MOOC instructors were utilizing computer-based forms of feedback to enhance their courses, the role of automation and artificial intelligence (AI) for personalization warranted further probing. As evident in Figure 6, the use of an automated grading system was the only feature leveraged by more than half of the MOOC instructors (n=67 of 127 respondents; 52.8%). Automated or system generated feedback was employed by 28 (22.1%) of the 127 respondents. Similarly, automated alerts for missed assignments were used by 24 (18.9%) of the respondents and automated alerts to participants who do not log in regularly were used in 21 (16.5%) of the MOOCs. Almost nonexistent were tools for automated group allocation (n=7; 5.5%), automated forms of plagiarism checking and detection (n=5; 3.9%), and embedded agents for learner advice (n=3; 2.4%). System adaptation to user performance was found in a single course (n=1; 0.8%). Another line of inquiry on personalization and tools was centered on MOOC participant communication to instructors, especially if barriers exist. Over half of the 135 respondents indicated that learners could email the course or system (n=78; 57.8%) or send direct emails to instructors (n=75; 55.6%). Less common was emailing teaching assistants (n=42; 31.1%) or relying on social media for support (n=35; 25.9%). Even fewer used synchronous conferencing (n=18; 13.3%), synchronous chat tools (n=11; 8.2%), or face-to-face meet ups (n=4; 3.0%). Nearly nonexistent was the use of personal visits (n=1), virtual world types of environments (n=1), and mobile phones (including text messaging) (n=0).

Research Question #4
How would these instructors structure their next MOOC differently in terms of personalization?  Figure 7). Combining the modest and high interest groups shows that three-fourths of MOOC instructors were interested in MOOC personalization in the future. Advocating for MOOC instructor professional development and training, therefore, seems highly warranted. Several interesting comments were proposed in the open-ended question regarding how respondents might redesign their courses to enhance course personalization and overall effectiveness. For instance, one MOOC instructor would "hire some of our students and alumni to get involved -the students really loved the additional points-of-view and the interaction." Another instructor stated that she would, "introduce Google Hangouts. Develop alternative pathways for content. Allow students more space to share own competencies and knowledge levels (perhaps wikis etc.)." Another example is an instructor planning to "offer more examples on different topics and offer different tracks (e.g., just video, video and quizzes, video, quizzes and peer review assignments, etc.)."

Additional Open-Ended Comments
Across the open-ended questions, other personalization practices of the respondents included greater instructor participation in discussion forums, increasing opportunities for learner reflection, designing online learning communities, creating shorter and less formal videos, fostering more peer interaction, subtitling content in different languages, and utilizing formative assessments in the form of participant surveys at the end of each week. The most frequent comment from these MOOC instructors was that they attempted to incorporate "flexible deadlines," including allowing students to post discussion comments and complete tasks at their own pace. In addition, many also leveraged social media, multimedia, mobile Level of interest in learning new ways to personalize your next MOOC applications, and readings to supplement course materials. Among the other personalization methods employed, several instructors mentioned relying on guest speakers, whereas others employed case-based learning. A few instructors attempted to empower the participants by allowing them to choose their own assignments, make multiple attempts to complete assignments, or create their own student groups.

Discussion
The purpose of this study was to investigate how MOOC instructors adapt their courses to the individualized learning needs of students who enroll in a MOOC. In effect, the goal was to better understand the instructional design and personalization approaches of instructors related to MOOCs. The researchers realize that the personalization of MOOCs is a highly idealized and contested concept. We also acknowledge that the massive size of MOOCs makes personalization extremely difficult, if not impossible. However, the goal was to push toward a more personalized MOOC experience through the exploration of MOOC instructor activities, resources, and technologies involved in MOOC design and implementation.

107
One issue noted in this study was the lack of learner monitoring and feedback. Learner progress was left to self-monitoring or was ignored all together. Similarly, peer feedback and system feedback, while important to learner success, were more pervasive than that coming from the instructor or instructional assistants.
Finding ways to build expert feedback (including soliciting alumni of these courses for feedback), which was rare in this study, might be one way to foster greater learner personalized attention and overall success.
There were a variety of ways in which to have participant questions answered (i.e., contacting the instructor, teaching assistants, social media, synchronous chat, meet ups, synchronous conferencing, etc.). This issue is consistent with challenges faced by MOOC instructors and designers as they struggle to develop a feedback mechanism which "reinforces learning and identifies inconsistencies in the learner process" (Davis et al., 2014, p. 8). Perhaps social media interactions and local meet ups with peers and instructors within MOOCs will increase in the coming decade (see Severance, 2015 for ideas).
Among the more interesting findings from the survey of 152 MOOC instructors polled was that automated alerts, adaptive forms of instruction, and AI do not seem to be playing much of a role in MOOCs. While only addressed in a single questionnaire item, the findings lend doubt to claims that such technologies and systems will soon be taking on a prominent role in MOOCs and other forms of open education. In fact, automated checking of participant progress and the flagging of potential issues were not widely implemented in these MOOCs, nor was the sending of reminders or feedback on accomplishments.
While several prominent technology pioneers have been promoting adaptive digital courseware and AI technology to help reform education, including Mark Zuckerberg (Singer, 2017) and Bill Gates (Schaffhauser, 2014;Straumsheim, 2016), these findings seem to indicate that the impact of AI thus far in the field of MOOCs and open education is quite limited. Even if AI technology was more prominent in MOOCs, automated alerts, reminders, and feedback do not offer MOOC participants "a sense of being treated as an individual, and, therefore," such forms of course automation fall "short in providing personalized learning" (Fournier & Kopp, 2015, p. 298). As Bates (2012) laments, at present, technologies embedded in MOOCs do not yet offer the timely and pointed comments and questions that can nurture rich and interactive online discussions, a sense of caring and encouragement, and a robust understanding of individual student needs. Nevertheless, much investment is being made today in AI technology around the world that should eventually lead to inroads toward more customized and personalized MOOC experiences (Metz & Satariano, 2018).

Limitations
As with any educational research project, there are several important limitations to mention. First of all, as indicated in the methods section, we assembled a database of more than 1,000 MOOC instructor names, courses, and associated contact information from selected lists and vendor websites. However, the researchers did not collect information from all MOOC vendors, nor were MOOCs taught in languages other than English or Korean included, unless the course was cross listed in a researcher-mined MOOC vendor list. Secondly, participants self-selected into this study on "how massive open online course (MOOC) instructors personalize learning." Therefore, survey respondents may have devoted more time to their instructional and pedagogical approaches than those who did not respond. Thirdly, no actual teaching was 108 directly observed nor was any instructional content analyzed. Additionally, the researchers did not conduct follow-up interviews or focus groups with survey participants on their specific personalization and other instructional design practices. Another limitation is that while survey participants were provided with a loose definition of personalization, the 25 experts were not. By not operationally defining personalization for all participants, any in vivo thematic coding schemes created by the researchers have potential constraints and flaws. At the same time, however, it is important to recognize that the term "personalization" has many different connotations and interpretations; one definition may not work for all stakeholders. Another term not explicitly defined was "effort." Once again, each respondent may have a vastly different understanding of what incredibly high or low effort might entail.

Future Directions
Findings from these data sets are merely the first steps in the process. There is a clear and present need to perform in-depth, follow-up inquires with MOOC instructors about their actual instructional design practices; specifically, the means by which personalized learning is attempted, and any instructional modifications and adaptations implemented over time. Interviews with instructors, via email or Web conferencing, would help uncover effective instructional practices undertaken for MOOC personalization as well as course redesign efforts pending or in progress. In addition to interviews, follow ups can take place via focus groups, content analysis, active participation in MOOCs, reviews of historical records, additional surveys, or a combination of these methods. MOOC participants and instructional designers could be solicited to verify and extend the findings of the knowledge base related to MOOC personalization during design and development. Additional research on MOOC personalization is necessary to create effective instructional design and delivery guidelines, frameworks, and models. A better understanding of instructors and participants will help foster more engaging, personalized, and culturally sensitive MOOCbased learning environments.

Implications and Final Comments
Research undertaken in this vein has the possibility of enhancing the planning, development, and delivery of courses that impact millions of learners. Even if minor or modest enhancements are made, the potential impact is immense. Recent data from Class Central indicate that in 2017, over 78 million students signed up for more than 9,400 MOOCs offered by more than 800 different universities (Lederman, 2018;Shah, 2018). Such data is a huge increase from the prior year which documented over 700 universities worldwide offering nearly 7,000 MOOCs to more than 58 million participants in 2016. In comparison, just 35 million learners enrolled in MOOCs at 500+ universities in 2015 (Shah, 2015). Coursera accounted for 30 million of the MOOC enrollments in 2017 as compared to 23 million in 2016 (Shah, 2016(Shah, , 2017. Another 14 million enrollments were in edX in 2017; 4 million more than in 2016 (Shah, 2016(Shah, , 2017. Equally impressive, 23 million participants in 2016 registered for a MOOC for the first time (Shah, 2016), and another 20 million new MOOC participants enrolled in 2017 (Shah, 2017). What is clear from this data is that MOOCs are a phenomenon that is receiving accelerating attention, investment, and overall societal importance. They are 109 no longer a cultural anomaly or learning novelty that has limited value due to low completion rates. Instead, tens of millions of individuals are apparently finding some value from enrolling in MOOCs offered by thousands of universities worldwide.
Even when considering the highly advertised course retention and completion problems and issues, MOOCs are impacting scores of lives around the planet each day. Clearly, better understanding of how MOOCs are designed and participant progress is monitored should eventually result in higher quality course design and delivery, and improved completion and retention rates. Continued research in this area can assist countless MOOC instructors to enhance their massively open online courses with techniques, activities, and resources that engage and inspire learners from around the world into their respective disciplines. The study also informs MOOC vendors about MOOC platforms and associated tool design. In addition, it can apprise government funding agencies about the types of MOOC tools and resources that can foster improvements in the evolution of the field of open and distance learning for capacity building.
Given the number of participants that MOOCs attract, this study has the potential to provide marked insight into an emerging phenomenon that has immense global, local, and societal ramifications. With such wide impact potential, our research team continues to expand the database of MOOC instructors and courses that we have collected. The goal as we move forward is to determine more about the psychological, instructional, and technological issues, challenges, and opportunities of MOOCs and other emerging types of open online courses and educational experiences.