International Review of Research in Open and Distributed Learning Is My MOOC Learner-Centric? A Framework for Formative Evaluation of MOOC Pedagogy

MOOCs popularly support the diverse learning needs of participants across the globe. However, literature suggests well-known scepticism regarding MOOC pedagogy which questions the effectiveness of the educational experience offered by it. One way to ensure the quality of MOOCs is through systematic evaluation of its pedagogy with the goal to improve over time. Most existing MOOCs’ quality evaluation methods do not account for the increasing significance of learner-centric pedagogy towards providing a richer learning experience. This paper presents a MOOC evaluation framework (MEF), designed with a strong pedagogical basis underpinned by theory and MOOC design practices, which evaluates the integration of learner-centric pedagogy in MOOCs. Using mixed-methods research, the internal validation was conducted through expert reviews (N = 2), and external validation (N = 13) was conducted in the field to test model usability and usefulness. The framework was classified as “good” (SUS: 78.46) in terms of usability. A high perception of usefulness (84%–92%) was observed for the framework as a formative evaluation tool for assessing the integration of learner-centric pedagogy and bringing a positive change in MOOC design. Different participants acknowledged new learning from varied dimensions of the framework. Participants also recognized that the scores obtained using the MEF truly reflected the efforts taken to incorporate learner-centric design strategies in the evaluated dimensions. The framework focuses on learner-centric evaluation of MOOC design with a goal to facilitate improved pedagogy.

MOOCs popularly support the diverse learning needs of participants across the globe. However, literature suggests well-known scepticism regarding MOOC pedagogy which questions the effectiveness of the educational experience offered by it. One way to ensure the quality of MOOCs is through systematic evaluation of its pedagogy with the goal to improve over time. Most existing MOOCs' quality evaluation methods do not account for the increasing significance of learner-centric pedagogy towards providing a richer learning experience. This paper presents a MOOC evaluation framework (MEF), designed with a strong pedagogical basis underpinned by theory and MOOC design practices, which evaluates the integration of learner-centric pedagogy in MOOCs. Using mixed-methods research, the internal validation was conducted through expert reviews (N = 2), and external validation (N = 13) was conducted in the field to test model usability and usefulness. The framework was classified as "good" (SUS: 78.46) in terms of usability. A high perception of usefulness (84%-92%) was observed for the framework as a formative evaluation tool for assessing the integration of learner-centric pedagogy and bringing a positive change in MOOC design. Different participants acknowledged new learning from varied dimensions of the framework. Participants also recognized that the scores obtained using the MEF truly reflected the efforts taken to incorporate learner-centric design strategies in the evaluated dimensions. The framework focuses on learner-centric evaluation of MOOC design with a goal to facilitate improved pedagogy.

Introduction
MOOCs have been reported to benefit varied stakeholders, including students, academicians, and corporate professionals (Egloffstein & Ifenthaler, 2017;Konrad, 2017). Additionally, there has been an increase in the acceptance of MOOCs by higher education institutions using varied models (Burd et al., 2015). However, such MOOC initiatives also raise concerns as the quality of learning experience in MOOCs still remains debatable (Lowenthal & Hodges, 2015;Margaryan et al., 2015;Toven-Lindsey et al., 2015). The research points towards known limitations in the design of MOOCs and its insufficiency to cater to the diversity of learners with varied motivations and learning requirements (Hew, 2018;Yousef et al., 2014). This questions the meaningfulness and effectiveness of the educational experience that is offered by MOOCs. Therefore, MOOCs are expected to meet some quality standards in their pedagogical considerations.
Numerous MOOC design guidelines and frameworks in literature enumerate important learning support elements to address the pedagogical quality of MOOCs (Conole, 2013;Hew, 2018;Lee et al., 2016;Pilli & Admiraal, 2017). However, the overall integration of these principles in MOOCs is found to be low in literature (Lowenthal & Hodges, 2015;Margaryan et al., 2015;Watson et al., 2017). This indicates that though there are some guiding pedagogical frameworks and principles, the potential direction towards improving the instructional quality of MOOCs seems to be lacking. A recent literature review on MOOCs' quality also suggested deepening of research in the subject and designing new guidelines that ensure quality (Stracke & Trisolini, 2021).
Since quality is the output of the systematic process of design and evaluation, one way to ensure the quality of MOOCs is through the evaluation of these courses with the goal to improve over time (Alturkistani et al., 2020;Jansen et al., 2017). There are a few existing e-learning quality approaches, some specific to MOOCs, intended to provide a useful overview and guide to certain quality issues (Ossiannilsson et al., 2015).
However, most of these methods need to be enriched to effectively evaluate the instructional design in MOOCs implementing active learning pedagogies (Aloizou et al., 2019). This study presents an enhanced learner-centric instructional framework, termed MOOC Evaluation Framework (MEF), for MOOC creators or instructors for formative evaluation of their MOOCs in order to improve upon their pedagogical design.
In addition, the study evaluates the usability and usefulness of the MEF from the perspective of MOOC creators in the field, their experience with the MEF in terms of new learning, and their perception on its usefulness in evaluating the integration of a learner-centric approach in MOOCs.

Significance of Learner-Centric Approach in MOOCs
Over the years, studies have been presenting strategies such as collaboration, peer interaction, feedback, learner-instructor connection, and so forth to enhance students' engagement, improve academic achievement and lower the attrition rates in MOOCs (Hew, 2018;Pilli & Admiraal, 2017;Yousef et al., 2014). Though the proposed strategies for improvements differ in details, a remarkably consistent theme is the call to emphasise learner-centric instructional strategies such as active learning, problem-centric learning, and instructor accessibility in MOOCs. The learner-centric instructional design encourages interactions with peers and instructors, and focuses on recurrent learning activities and feedback mechanisms. The approach engages learners to create their own learning experience and become independent and critical thinkers (Bransford et al., 2000).
There exist several MOOC design models and frameworks guiding the development of MOOCs (Conole, 2013;Fidalgo-Blanco et al., 2015;Lee et al., 2016) for enhanced learner experience and quality of MOOCs.
A recently presented learner-centric MOOC model for MOOC design also established the role of learnercentric pedagogy in attaining active learner participation and higher completion rates in MOOCs (Shah et al., 2022). However, there is an identified research gap in the quality assurance methods of MOOCs implementing active learning strategies (Aloizou et al., 2019). Most of the existing quality evaluation methods for MOOCs do not account for the increasing significance of learner-centric pedagogy that aims at providing richer learning experience for participants.

Existing Evaluation Measures for MOOCs
Most of the studies which evaluate MOOCs' pedagogy use instructional design principles or some standardized frameworks. This section provides a brief overview on some of these evaluation approaches and a few observations associated with respect to their quality criteria and applications.

Instructional Design and E-Learning Principles
Merrill's first principles of instruction, abstracted from key instructional design theories and models, were supplemented by five additional principles, abstracted from literature (Maragaryan et al., 2015;Merrill, 2002). A MOOC evaluation study presented an analysis of design quality determined from first principles of instruction for 76 MOOCs (Maragaryan et al., 2015). The majority of MOOCs scored poorly on most instructional design principles but highly on organisation and presentation of course material. This indicated that although most MOOCs were well-packaged, their instructional design quality was low. A similar evaluation study conducted on 27 open courses, using the first principles of instruction, showed parallel results with poor instructional design of courses (Chukwuemeka et al., 2015). Another study applied Merrill's first principles of instruction to review nine MOOCs and found that the principles were generally well incorporated into the course design (Watson et al., 2017). However, here the evaluation included MOOCs that specifically targeted attitudinal change.
A recent evaluation study conducted on six courses, using Chickering and Gamson's principles (Chickering & Gamson, 1987) and part of the quality online course initiative rubric, showed the need for further enhancement to support active learning in these courses (Yilmaz et al., 2017). Another recent study used Clark and Mayer's e-learning guidelines (Clark & Mayer, 2016) to evaluate the pedagogical design of 40 MOOCs (Oh et al., 2020). The findings of the study indicated a relatively low application of these principles in general, with the exception of those related to the organization and presentation of content. The principles which scored particularly low included practice, worked examples, and feedback.

Standardized Frameworks and Quality Models
Different organisations have come up with a number of standard models or frameworks for quality check of e-learning, with some specifically developed for MOOCs. One of these includes OpenupEd (Rosewell & Jansen, 2014), initiated by the European Association of Distance Teaching Universities (EADTU). This comprises 11 course-level and 21 institutional-level benchmarks that cover six areas, including strategic management, curriculum design, course design, course delivery, staff support, and student support. Though OpenupEd promotes features that put the learner in the centre, the quick scan should be further fleshed out using a more detailed self-assessment process (Jansen et al., 2017). Another quality assurance model with a similar approach is the European Foundation for Quality in e-Learning (EFQUEL) which operates the UNIQUe certification (Creelman et al., 2014). These models are mainly intended for certification, accreditation, benchmarking, or labelling as a frame of reference (Ossiannilsson et al., 2015). Read and Rodrigo (2014) also reported a quality model for Spain's National Distance Education University MOOCs; however, it presented high-level guidelines on course design aspects such as topic, reuse of existing content, overall duration, course structure, and so forth. This work was based largely on MOOCs which were adaptations of existing courses to a MOOC format. Quality Matters (QM), though not specifically addressing the context of MOOCs, is another assurance system for evaluating online courses such as MOOCs (Shattuck, 2015). The framework consists of 47 specific criteria with eight general standards. According to a study, none of the six courses reviewed using QM achieved a passing score of 85% (Lowenthal & Hodges, 2015).
The study also mentioned the tendency of the framework to heavily focus on the aspects of course design and not enough on instructional approaches for active engagement, communication, and collaboration.
Another framework, the Quality Reference Framework (QRF), was developed by the European Alliance for the Quality of MOOCs called MOOQ (Stracke et al., 2018). The framework consists of two quality instruments with action items for potential activities and leading questions to assist in MOOC design and development. It is a generic framework that can be adapted to specific contexts for improving MOOC design, development, and evaluation of created MOOCs (Stracke, 2019). However, no evidence from testing the framework in the field has yet been reported.
Hence, the application of existing instructional design principles often remains limited, and most of the quality evaluation models and frameworks tend towards certification and accreditation with high-level guidelines. A recent study which evaluated three mature quality analysis tools, including the 10-principle framework, the OpenupED, and Quality Matters proposed the need for clear and simple questions, assessing specific elements of the active learning pedagogies to make accurate conclusions about MOOC quality (Aloizou et al., 2019). According to another recent systematic review, one of the least studied aspects of MOOC evaluation of effectiveness is pedagogical practices (Alturkistani et al., 2020). Hence, the goal of our research was to create an evaluation framework which is focused on the pedagogical perspective of MOOC design, with a learner-centric approach at its core.

MOOC Evaluation Framework
The MEF distinguishes itself from other MOOC evaluation measures as it primarily focuses on evaluation of learner-centric pedagogy in MOOC design. Though a few existing frameworks evaluate certain learnercentric components in online courses (Rosewell & Jansen, 2014;Shattuck, 2015;Stracke, 2019), greater emphasis has been observed in constructs such as learning objectives, learning activities, assessment, and so forth with broad guidelines. While broad course guidelines can help evaluate and improve MOOC quality to some extent, they do not address specific pedagogical challenges of poor learner engagement, learner interaction, collaboration, feedback, and so forth. (Lowenthal & Hodges, 2015;Maragaryan et al., 2015;Oh et al., 2020;Yilmaz et al., 2017). The MEF offers evaluation and guidance on incorporation of learnercentric practices, which have been shown to address some of these pedagogical challenges (Shah et al., 2022). The framework goes beyond a crisp checklist, in the form of questions or high-level recommendations, to provide comprehensive indicators for MOOC creators or reviewers. It also provides an opportunity for formative evaluation of MOOC design in a structured and comprehensive manner.
Formative evaluation, a term first coined by Scriven (Scriven, 1967), is a process of reviewing pilot stage courses to determine their strengths and weaknesses before the programme of instruction is finalized (Tessmer, 2013). In this setting, formative evaluation through the MEF will allow the instructor to continuously monitor the integration of learner-centric activities during the development phase of the MOOC. It will provide constant feedback and suggest ways to improve through reflective and easy to comprehend design indicators, organised in different dimensions.

Theoretical Basis of the Framework
The MEF is grounded in a number of theoretical approaches. Following the cognitive load theory (Paas & Sweller, 2014;Sweller et al., 2011) and the theory of multimedia learning (Mayer, 2019), the framework evaluates design elements to ensure ease in processing of learning content and reduction in extraneous processing. Cognitivists believe in making the learning process meaningful by organizing the information into structured and smaller chunks. In the context of MOOCs, chunking of concepts into small-length video content with in-video activities makes knowledge meaningful and connects new information with prior knowledge (Shah et al., 2022). Such design interventions related to video content are used by the framework.
Based on constructivist approaches of learning (Mayer, 2019), the MEF evaluates MOOC content for building learner knowledge rather than passive consumption of information. The learning activities are evaluated for promotion of active participation from students where they construct new knowledge based on their prior knowledge. Attention is given to learner diversity, individual differences, and presence of multiple visual representations. The theory of social constructivism (Vygotsky & Cole, 1978) emphasises the importance of social interactions in constructing one's own learning. The framework ensures evaluation of aspects such as peer interactions, collaborative learning, building of learner community, and so forth.
Through evaluation of immediate and constructive feedback in learning activities, assignments, and forum tasks, the framework incorporates reflection (Bransford et al., 2000).
The framework incorporates the principle of constructive alignment where curriculum objectives, teachinglearning activities, and assessment tasks are aligned with each other (Biggs, 1999). Moreover, the evaluation criteria are also drawn from the knowledge of first principles of instruction and other research-based practices (Hew, 2018;Margaryan et al., 2015;Merrill, 2002), keeping a focus on learner-centric design.

Scope and Dimensions of the Framework
The MEF evaluates the pedagogy design of MOOCs with a learner-centric lens to achieve active learner participation, stronger learner connection with the course content and team, and effective collaboration.
The framework focuses on xMOOCs, characterised by structured learning components such as videos, learning activities, assessment, discussion forums, and additional learning resources (Conole, 2013).

143
Aspects that are out of the scope of this framework include institutional policies and evaluation of technological platforms. Also, it is not designed to evaluate the effectiveness of MOOCs in terms of learning outcomes, learner retention, or learner experience.
The framework organises the integral learning components and pedagogical features of a MOOC into eight dimensions. As shown in Figure 1, the MEF includes five structural dimensions (D2, D3, D4, D5, and D6) and three operational dimensions (D1, D7, and D8). The structural dimensions of the framework are integral learning components for most xMOOCs, while the operational dimensions have evolved in view of the need to keep the course learner-centric in its content, practices and offering.

Types of Dimensions in the MEF
Each dimension further consists of quality criteria related to various aspects of that dimension (Table 1).
The goodness of a criterion is described by an array of benchmark indicators, which define the set of actions that need to be fulfilled in order to achieve a high-quality pedagogical design. It is by means of these indicators that the framework aids in formative evaluation of pedagogical features to bring a positive change in MOOC design. inadequate, adequate, or proficient) ranging from 0 to 3, demonstrating the level of performance. Figure 2 illustrates a part of dimension 3, listing two criteria: "Offering of supplementary learning resources" and "Addressing diverse learner needs and interests," with multiple indicators. Some indicators entail numerous items (each denoted by a hyphen). In such cases, the greater the number of items fulfilled, the closer an indicator will be to the proficient level of performance.

Figure 2
Screenshot of the MEF Toolkit Displaying a Part of Dimension 3

147
The MEF calculates an average score for every criterion as well as dimension, depending on the user's selection for the corresponding indicators. Based on average dimension score, the user receives feedback which is distributed over 6 score bands: 0-0.9 (missing or minimal); 1.0-1.4 (inadequate); 1.5-1.9 (towards adequate); 2.0-2.4 (adequate); 2.5-2.9 (towards proficient); 3 (proficient). These score bands act as a standard to measure the strength of learner-centric design in a particular dimension. However, the choice of such benchmarks is inevitably arbitrary, and the effects of prevalence and bias on the score must be considered when judging its implication (Sim & Wright, 2005). Hence, even though these scoring bands help in visualizing the interpretation of the obtained score, some amount of individual judgement and circumstances should also be taken into account.

Methodology
A schematic outline of the steps involved in development of the framework and validation studies is shown in Figure 3. A total of 15 MOOC creators, who are responsible for the overall vision, content creation, design, and orchestration of the course, participated in this study. Out of these, two experts conducted the internal validation of the framework. These reviewers had expertise not only in instructional design, but also in model development and learner-centric pedagogy. Thirteen MOOC creators participated in the external validation of the framework. All participants were provided with the MEF toolkit along with detailed guidelines to perform their respective MOOC evaluations. The usability and usefulness of the framework was evaluated through mixed-methods research using quantitative and qualitative analyses, drawing on the strengths of both approaches. The convergent design approach was used for the quantitative and qualitative data collection performed at similar times, followed by an integrated analysis.

Research Questions
Internal validation studies were performed to evaluate the components and processes of MEF creation, whereas the following three research questions were investigated through the external validation study.
1. How did the MEF perform in terms of usability and usefulness from the perspective of MOOC creators?
2. How was participants' experience during their first encounter with the MEF in terms of new learning and their perception on its potential use in the field?
3. What were participants' reflections on their scores obtained using the MEF in relation to their course pedagogy design?

Internal Validation
An internal validation study was conducted through expert review using a questionnaire followed by an

149
to the character of internal validation (Richey, 2006). In addition, there were questions derived from an instrument created for validating the model theorization process (Lee et al., 2016). These questions attempted to seek answers for 6 items: synthesis of literature for creation of the framework, use of appropriate terminology, comprehensibility, comprehensiveness, validity, and its usefulness. Interviews were conducted to gain thorough understanding of experts' suggestions on certain aspects of the framework.

External Validation
Based on literature recommendation (Richey, 2006), this evaluation study examined the ease of usability and usefulness of the framework through questions which aimed at answering aspects such as: Do MOOC creators find the MEF useful in meeting their MOOC evaluation needs? Why should the MEF be made available/unavailable to MOOC creators for formative evaluation? The questionnaire entailed 13 multiple- choice and eight open-ended questions. Ten multiple-choice questions, derived with slight modification from the original system usability scale or SUS (Bangor et al., 2008), majorly focused on usability of the framework. Remaining questions focused on the strong and weak points of the MEF, with suggestions for improvement, for detailed evaluation of its effectiveness.

Quantitative Analysis
From the internal validation study, quantitative data of experts' ratings were analysed for content validity index (CVI) and inter-rater agreement (IRA) using earlier methodology (Rubio et al., 2003). In the external validation study, SUS was calculated to determine usability of the framework (Brooke, 1996). The quantitative data received from questionnaire responses on the usefulness of the course was examined by performing frequency analysis from the Likert scale to yield percentages.

Qualitative Analysis
The qualitative data received during interviews with internal validation experts was used as feedback to revise the framework. For the external validation study, the qualitative data provided a detailed description of factors evaluating the ease and complexity in usability and degree of usefulness of the framework. After the questionnaire was completed, member checks were conducted, in the form of interviews, to discuss certain remarks more deeply. Inductive thematic analysis (Braun & Clarke, 2006) of responses was performed to understand and classify participants' perceptions on usability and usefulness of the MEF in evaluating the integration of learner-centric pedagogy in MOOC design.

Internal Validation
Two learner-centric pedagogy experts reviewed the MEF to evaluate six items in order to validate model components and assess its usefulness. The mean for different component scores ranged from 4.5 to 5, with CVI and IRA as 1 in 6 items. Since the CVI and IRA were above 0.8, it suggested strong content validity and reliability (Rubio et al., 2003). In addition, data from the open-ended questions and interviews with experts was used to make subsequent revisions in the framework, which strengthened it further to Version 1.1.

External Validation
Of the whole group (N = 13), 10 participants were faculty, whereas three were final year PhD students who have been active participants in MOOC creation and offering (Table 2). Participants chosen for this study have created one or more MOOCs, and belonged to eight different recognised educational institutes in India. Thirteen MOOCs from six different disciplines, evaluated in this research using the MEF, were created by study participants in their respective institutions. All participants evaluated their MOOCs in all eight dimensions of the framework.

Usability and Usefulness of the MEF
We focused on our first research question through quantitative analysis of usability and usefulness of the MEF for MOOC creators. The average equivalent score for the scale of usability for all participants was 78.46. Hence, based on the mean system SUS scores rating (Bangor et al., 2008), MOOC instructors classified the MEF as "good" in terms of its usability. There was a high level of agreement observed in the perception of usefulness for each dimension, ranging from 84% to 100% (average 97%). Additionally, open-ended questions on MEF components that were missing or difficult to comprehend provided insightful feedback. The responses from the participants were analysed and categorised into two groups: (a) as useful suggestions which further helped in improving clarity and resulted in Version 1.2 of the MEF, and (b) as identified limitations of the framework which have been acknowledged in the discussion section.

Participants' Experience in Terms of New Learning and the Potential Use of the MEF
To answer our second research question, the participants were asked to provide (a) examples of new learning from the MEF and (b) their perception on making this framework available to MOOC creators for formative evaluation. One interesting point to note was that different participants (n = 12) acknowledged different aspects from varied dimensions as new learning. The content analysis of these responses showed that new learning branched from all dimensions, except dimension 1, which may seem fundamental to many ( Figure 5). Some of the learning aspects were emphasised multiple times by different learners. The learning aspect which was cited most often was the pedagogical design of the video content and associated in-video activities.

Visual Distribution of New Learning Aspects, as Identified by the Study Participants, From Different
Dimensions of the MEF Participants (n = 12) expressed their rationale to provide the MEF to MOOC creators for formative evaluation of their courses. Inductive thematic analysis led to the generation of two themes of perception for potential use of the MEF in the field. These included the use of the MEF (a) as a comprehensive guide for MOOC creators, and (b) in bringing reflections and improving the MOOC experience (Table 3). The framework was recognised as a comprehensive guide, which was perceived to provide effective pedagogical direction towards planning, creation, and evaluation of MOOCs. Additionally, the learners also perceived the framework as bringing reflections on various aspects of course design and learner connection, thus improving the MOOC experience for both instructors and students.

Participants' Reflections on Their MOOC Scores Obtained Using the MEF
Employing the MEF, participants obtained dimension-based scores for their respective courses. To answer our third research question, participants were asked a focused question: "Can you provide one example from your MOOC scores to reflect on your experience with the MEF in evaluating integration of a learnercentric approach?" Participants (n = 12) provided examples, expressing ways in which they could reflect on the differences in scores obtained for different dimensions and how that related to their corresponding design efforts (Table 4). As observed in Table 4, MOOC creators agreed that the scores obtained for their MOOCs using the MEF aligned with the efforts with respect to their pedagogy design. The dimensions that included more learnercentric interventions in their MOOC scored higher as compared to the dimensions which were not learnercentric or less learner-centric. This implied the effectiveness of the framework in evaluating the integration of learner-centric pedagogy in MOOCs.

Discussion
Systematic evaluation of pedagogical practices is one way of bringing a positive change to MOOC design (Jansen et al., 2017). There are some existing quality evaluation approaches which provide a useful overview and guide for e-learning design (Ossiannilsson et al., 2015;Rosewell & Jansen, 2014;Shattuck, 2015). The In terms of limitations, the framework does not particularly focus on specially-abled learners in its design.
However, some efforts have been made toward including diverse learner needs in dimension 3 and dimension 8. Secondly, the large-scale MOOC enrolment may interfere with straightforward evaluation of a few indicators related to collaboration-based activities. Thirdly, two MOOC creators pointed out that evaluating all dimensions at once, using the MEF toolkit, involves a time-consuming process. Though it may take a little longer for reflection during the first encounter with the MEF, it is a comprehensive tool which can be used for structured and straightforward formative evaluation of all future MOOC offerings.
In terms of generalizability, the adoption of the MEF is not only restricted to MOOCs but may also be used for pedagogy evaluation of other online or blended courses following a similar course structure. However, as a limitation, the current study was restricted to a small sample size in the local context and did not evaluate the use of the MEF with a large and diverse population of MOOC creators or MOOC providers. To address the same, subsequent to this primary implementation of the MEF towards its validation, the framework will be re-examined by diverse users during its large-scale field implementation in order to establish its generalizability. In view of a recent field study (Kizilcec et al., 2020) which emphasised on the context-based effects of interventions in MOOCs, it will be intriguing to examine the influence of the MEF in different contexts. The impact of the MEF on the effectiveness of MOOCs and its learning experience will also be investigated in future studies.

Conclusion
This study demonstrates the MOOC Evaluation Framework which evaluates the integration of learnercentric pedagogy in MOOC design. The framework provides an opportunity to MOOC creators for formative evaluation of their pedagogy to promote active learner participation and enhance engagement of learners with content, course team, and peers. Thirteen MOOC creators from eight different educational institutes evaluated their courses, in six different disciplines, using the MEF. The framework was found to be useful as a formative evaluation tool for evaluating integration of learner-centric approaches and bringing a positive change in pedagogy design of MOOCs. Benefits of the framework, expressed by MOOC creators, aligned with literature recommendations for new MOOC pedagogy evaluation measures, that is, to assess specific elements of active learning pedagogy; detect weaknesses in course elements; and acquire important learning for designing or redesigning a MOOC. The MEF seems to be a critical step forward for MOOC creators and MOOC providers to ensure learner-centric approach in pedagogy design with bigger goals to facilitate active learner participation, enhance learner engagement, and lower the attrition rates in MOOCs.