Does Formal Credit Work for MOOC-Like Learning Environments?

Although a number of claims have been made describing massive open online courses (MOOCs) as a disruptive innovation in education, these claims have not yet been proven through research. Instead, MOOCs should perhaps be considered as an integrative model for higher education systems, but to do so will require recognition of credentials. Initial experiments of MOOCs were not offered academic credit, but recently there has been some attempts to offer course credit for MOOCs or MOOC-like courses. However, whether earning a credit affects students’ performance and behavior in MOOCs has not been explored closely. Therefore, the aim of this study is to assess the effect of crediting on students’ achievement, perceived intrinsic and extrinsic goal orientations, and perceived course value. A causal comparative research design was applied. Data were collected via 516 responses to an online survey and achievement tests. Three credit conditions were compared: credit bearing, non-credit bearing, and credit careless. ANOVA results showed a significant difference between the credit bearing group and non-credit bearing group for all dependent variables. The credit bearing group also scored significantly higher achievement scores than the credit careless group. Credit clearly and significantly affected all dependent variables investigated in this study. Therefore, various possible models can be adopted by higher education institutions to integrate MOOCs as a credit. Further studies can explore the effects of credit on students’ online behaviors, such as engagement with online activities and user events on MOOC platforms.


Résumé de l'article
Although a number of claims have been made describing massive open online courses (MOOCs) as a disruptive innovation in education, these claims have not yet been proven through research. Instead, MOOCs should perhaps be considered as an integrative model for higher education systems, but to do so will require recognition of credentials. Initial experiments of MOOCs were not offer academic credit, but recently there have been some attempts to offer course credit for MOOCs or MOOC-like courses. However, does earning a credit will affect students' performance and behavior in MOOCs has not been explored closely. Therefore, the aim of this study is to assess the effect of crediting on students' achievement, perceived intrinsic and extrinsic goal orientations, and perceived course value. A causal comparative research design was applied. Data was collected via 516 responses to an online survey and achievement tests. Three credit conditions were compared: credit bearing, non-credit bearing, and credit careless. ANOVA results showed a significant difference between the credit bearing groups and non-credit bearing groups for all dependent variables. The credit bearing group also scored significantly higher achievement scores than the credit careless group. Credit clearly and significantly affected all dependent variables investigated in this study. Therefore, various possible models can be adopted by higher education institutions to integrate MOOCs as a credit. Further studies can explore the effects of credit on students' online behaviors, such as engagement with online activities and user events on MOOC platforms.

Introduction
With advances in information and communication technologies (ICT), opportunities to improve global knowledge and information access, transfer, and sharing have skyrocketed. Embedded within this growth is the rapidly expanding idea of open access. The concept emerged in software development in the 1960s, when a lack of commercial software compelled researchers to share their code (Moon & Sproull, 2002).
Over time, the openness philosophy spread to academic publishing, textbooks, and educational materials (Humbert, Rebillard, & Rennard, 2008;Wiley, 2006). It was in the late 1990s that the openness 76 philosophy was started to be seen in sharing course materials. Afterwards, this movement was called open educational resources (OER). Criticisms about OER referred to its isolation from real settings (Liyanagunawardena, Adams, & Williams, 2013), lack of online learning experience (Sclater, 2011), unavailable assessment and accreditation (Stacey, 2007), and insufficient instructional design (Ferreira, 2014), all reasons that triggered a new movement, the massive online open courses (MOOCs). MOOCs have partly eliminated the above limitations of OER. The majority of MOOCs provide planned and programmed online learning experiences with assessment models such as peer evaluation or online quizzes (Raposo-Rivas, Martinez-Figueira, & Campos, 2015). They mostly feature a course syllabus with fixed start and end dates (Pundak, Sabag, & Trotskovsky, 2014), as well as recognition in the form of a certificate of completion or participation (OpenupEd, 2015).
Although it has been claimed that MOOCs will change the business model for higher education (Horn & Christensen, 2013;Lentell, 2014;Simm & Pinto, 2012), this change is unlikely to occur in the near future (Kalman, 2014). Whenever a new technology or movement emerges, exaggerated claims are made about how it will revolutionize education. Just as such claims were raised about radio, television, and computers, they are being made now for MOOCs; history clearly shows how those expectations have failed (Cuban, Kirkpatrick, & Peck, 2001). In this sense, instead of looking at MOOCs as a disruptive movement that significantly changes higher education, it is more meaningful to consider them as a complementary or integrative model (Kiley, 2013). That is, MOOCs should be counted as formal higher education courses.
The way to do this is crediting of MOOCs.
The literature reveals initial experiments with MOOCs did not offer academic credit (McAuley, Stewart, Siemens, & Cormier, 2010;Jaschik, 2013). Currently, most MOOCs only offer non-credit alternatives such as completion, attendance, or participation certificates (Yousef et al., 2014). Furthermore, a successful MOOC completion is not recognized as a formal credit by most universities (Billington & Fronmueller, 2013). Despite an uncertain start, some attempts have recently been made to offer course credit for MOOCs or MOOC-like courses, including legislative proposals in Florida and California (Negrea, 2014;Sandeen, 2013b). Colorado State University may be the first American university to offer college credit for a MOOC (Mangan, 2012). Schools in other countries, such as the University of Helsinki in Finland, have also started to accept MOOCs for credit (Kurhila, 2012), and the American Council on Education (ACE) has approved some courses offered by Coursera and Udacity for credit (Hollands & Tirthali, 2014;Kolowich, 2013a;Lederman, 2013). In addition, an algebra MOOC at the University of California, Irvine, has been recommended by ACE for developmental math vocational credit (Hollands & Tirthali, 2014), and San Jose State University is collaborating with Udacity to offer yet another course worth credit (Rai & Chunrao, 2016). Hew and Cheung (2014) have suggested that if higher education institutions would offer formal course credit, this incentive will work for students to redeem their learning for a traditional degree. The Chronicle of Higher Education conducted a survey with 103 professors who have taught a MOOC, many of whom stated MOOCs should be integrated into traditional systems of higher education (Kolowich, 2013b). In another study, students reported that participating in a credited MOOC made them more committed to completing the course and enhanced their understanding of the topic (Chamberlin & Parish, 2011). Baylor Teaching, Learning & Technology Committee (2013) indicated that online courses requiring a financial commitment and bearing a credit are more likely to have better retention rates and offer greater learning opportunities. They also reported that since MOOCs do not offer formal credit, students exhibit decreased motivation to complete the course. Similarly, in her dissertation, Schulze (2014) pointed out the potential value of MOOCs for adult learners, if only higher education institutions continue to invest in them and begin offering them for credit. Considering high dropout rates as the most problematic issue for MOOCs (Daniel, 2012;Fischer, 2014;Sandeen, 2013a;Yousef et al., 2014), providing formal course credit might be the solution (De Waard, 2011;El-Hmoudova, 2014;Hollands & Tirthali, 2014). Yuan, Powell, and Olivier (2014) have proposed that with increased numbers of credit bearing options, the overall number and potential of MOOCs will continue to grow.
Although offering credit for MOOCs has a number of obvious advantages, this topic is an area of debate in the literature (Liyanagunawardena et al., 2013). In their study, Hollands and Tirthali (2014) interviewed 83 stakeholders from 62 institutions. A number of interviewees expressed that "credits should not be offered for MOOCs unless they provide the same rigor in learning, assessment, and identifiability of participants as face-to-face courses" (p. 45). Similarly, Sandeen (2013a) argued that online identity management must be resolved before awarding credit for MOOCs. That is, it should be ensured that the person who enrolls in the course is the same person who takes the exam (Pundak et al., 2014;Sandeen, 2013a). In a survey conducted with professors who taught MOOCs, only a quarter of them stated that those who passed the MOOC deserved credit (Kolowich, 2013b). Further, in connectivist MOOCs, since everybody completes the same assignment, credit can again be a problematic issue (Rodriguez, 2012). The majority of these limitations can be eliminated by executing applicable models. For example, identity management can be resolved with a proctored exam, either on campus or online. Coursera's Signature Track is one example of successful proctored exams online (Coursera, 2013). Another model can be useful for persuading students to enroll in an online degree program. MOOC2Degree is an example of such a model (Desire2learn, 2013). Professional development programs that do not require a diploma, such as in-service training programs offered within the context of lifelong learning, are yet another model for crediting MOOCs.
Rather than simply addressing which models solve which problems, one of the most important questions to explore is what are the effects of credit bearing MOOCs on student behavior? In spite of news about failures of initial steps taken for crediting MOOCs (Haber, 2013;Kolowich, 2013c), no research study which supports this claim have been encountered. In this point, information is particularly lacking on the 78 impact of credit bearing and non-credit-bearing courses on student behavior. Crediting MOOCs is likely to affect the MOOC processing model as well as student behavior in MOOC platforms. Success of students in these platforms or their motivations to pursue their education are some of the issues in which crediting is likely to make a difference. Growing interest in MOOCs from higher education institutions (Martin, 2012;Raposo-Rivas et al., 2015), promotion of MOOCs to students by colleges and universities (Negrea, 2014), high dropout rates in MOOCs (Daniel, 2012;Fischer, 2014) are some of the other reasons that make it necessary to conduct the present study.
The research to date has especially tended to focus on how to conduct valid and reliable assessment in MOOCs, identity management of students, and providing quality education (Hollands & Tirthali, 2014;Sandeen, 2013a;Yuan et al., 2014). In spite of the increasing number of papers on MOOCs appearing recently in the peer-reviewed literature (Liyanagunawardena et al., 2013;Yousef et al., 2014), the effects of credit in MOOCs or MOOC-like learning environments have still not been closely examined. In this sense, the purpose of this study is to understand the effect of credit-bearing courses on students' achievement, perceived intrinsic goal orientation, perceived extrinsic goal orientation, and perceived course value. The following research questions guided this study: 1. Is there a difference in achievement between the credit bearing, non-credit bearing, and credit careless groups for the course, Ataturk's Principles and History of Turkish Revolution I? 2. Is there a difference in perceived intrinsic goal orientation between the credit bearing, non-credit bearing, and credit careless groups for the course Ataturk's Principles and History of Turkish Revolution I?
3. Is there a difference in perceived extrinsic goal orientation between the credit bearing, non-credit bearing, and credit careless groups for the course Ataturk's Principles and History of Turkish Revolution I? 4. Is there a difference in perceived course value between the credit bearing, non-credit bearing and credit careless groups for the course, Ataturk's Principles and History of Turkish Revolution I?

Method Research Design
This study applied causal comparative research design, which aims to determine the cause for or consequences of differences between groups (Fraenkel & Wallen, 2011). To understand the effect of a credited MOOC, a Web-based survey was designed and administered to campus-based students enrolled in the online course, Ataturk's Principles and History of Turkish Revolution I. Achievement scores for midterm and final exams were also gathered.

Context
Most universities in Turkey offer the common entry-level courses included in every university curriculum through distance education for their campus-based students. The university where the data were collected has four such courses: Ataturk's Principles and History of Turkish Revolution I, Foreign Language (English), Turkish Language, and Basic Law. Of these courses, Ataturk's Principles and History of Turkish Revolution I was selected because it has both credit and non-credit forms at the faculty of education, making the group a good source for exploring the effects of credit. Asynchronous course materials (selflearning units and video recordings) comprised the bulk of the distance education system. Self-learning units consisted of 15-20 pages with learning objectives at the beginning and a summary and review questions at the end. In the video recordings, course tutors provide summary and main points of the unit.
Students accessed the materials through a learning management system via their personal university accounts. Most of the students studied the course based on these self-learning materials. They can study the course whenever and wherever they want. Students can select the pace and order of their learning.
The course was fully online and students also had opportunities to meet face-to-face with their tutors to ask questions in office hour sessions, but these sessions were not compulsory. Midterm, final, and makeup examinations were centrally conducted; each student answered the same questions at the same time in proctored exams.

Population and Sample
This course is taken by 12,956 campus-based students each semester. Distributions of the sample and population with respect to gender and semester are shown in Table 1. As seen in the table, sample and population ratio are close to each other. The sample (n = 516) consisted of faculty of education students who responded to the online survey. The population, on the other hand, consisted of 1564 students; 66.8% were female, 33.2% were male and most were in their first semester (67.8%). As seen in Table 1, more females (70.7%) participated the study than males (29.3%), and most participants were in their first semester of university (60.7%). Note. f = sample frequency, F=population frequency  Note. n=sample size, N=population size

Dependent and Independent Variables
Four dependent variables were examined in this study: course achievement score, perceived intrinsic goal orientation, perceived extrinsic goal orientation, and perceived course value. Achievement scores were calculated according to midterm and final exams, while the other three dependent variables were obtained from an online survey (see Instrumentation). Credit condition, with three sub-groups, was the independent variable of this study. The credit bearing group consisted of participants taking Ataturk's Principles and History of Turkish Revolution I for credit. For the credit bearing course, the midterm exam was worth 40% of the score, 60% for the final exam. For the non-credit bearing group, the midterm and final exams equally affected course achievement. While only a credited course affects a student's grade point average (GPA), students must pass both credit and non-credit bearing courses in order to graduate.
The third group, credit careless, consisted of students from both groups. These students selected not sure when asked for the credit condition of the course in the survey, demonstrating their indifference towards earning credit. Table 3 shows the distribution of participants across study groups and their type of enrolled program. In total, 516 students responded to the survey. Out of those, 441 (85.5%) students were enrolled in programs where Ataturk's Principles and History of Turkish Revolution I received credit and 75 (14.5%) were from non-credit bearing programs. There were 159 students in the credit careless group, but 141 (88.7%) were from credit bearing programs.

Instrumentation
The main data collection instrument consisted of two parts: demographic questions and questions related to intrinsic goal orientation, extrinsic goal orientation, and course value. The second part of the questionnaire was extracted from the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich, Smith, Garcia, and McKeachie (1991). It had a 7-point Likert scale (not at all true of me to very true of me) with 81 items across fifteen subscales. Completion of the questionnaire took about 25 minutes, but subscales could also be administered individually (Pintrich et al., 1991). This questionnaire has been widely used (Credé & Phillips, 2011) in different countries (e.g., from Turkey, Sungur, 2004; from Thailand, Suksamram, 2003), in various settings (e.g., in online classes, Gaythwaite, 2006), and with several target groups (e.g., with nursing students, Tutor, 2006;with elementary students, Kiran, 2010). This study applied three subscales from the MSLQ: the intrinsic goal orientation scale, the extrinsic goal orientation scale, and the value scale. The intrinsic goal orientation scale consisted of four items that assessed whether a student's participation in an academic task was an end in itself and whether the student was focused on mastery of the topic. The extrinsic goal orientation scale consisted of four items that assessed the degree to which a student participated in a task for good grades, rewards, or approval. The value scale included six items that assessed the degree to which the student believed that the subject matter was interesting and worth learning (Credé & Phillips, 2011). Sungur (2004) translated and adopted the MSLQ into Turkish, performing confirmatory factor analysis for each section and determining fit statistics similar to the original instrument. Cronbach's alpha scores were calculated with the data of this study and it was found to be .81 for the extrinsic goal orientation scale, .86 for the intrinsic goal orientation and .92 for the value scale. These values demonstrated appropriate internal consistency for the all three sub scales.

82
The other data collection instruments were the midterm and final exam achievement tests, which consisted of 25 multiple choice questions prepared by a group of experienced tutors based on the selflearning units published on the learning management system. Each question was worth four points, and wrong answers did not affect the final score.

Data Collection and Analysis
Data were collected at the end of the semester in January of 2015, giving students maximum exposure to the course. An online survey was administered to faculty of education students and they saw the survey in an online system where their exam score was announced and optical response sheet was published.
Achievement scores for midterm and final exams were gathered from the student information system.
Before conducting analysis, data cleanup was performed, and assumptions and outliers were checked for each group. Two outliers were encountered with regards to achievement for the credit group and one for the non-credit group. Two outliers were also discovered with regards to external goal orientation for the non-credit group. These outliers were dropped, and analyses were conducted. Histograms of distributions and quantile-quantile plots (q-q plots) for each credit group were relatively normally distributed, with skewness and kurtosis values falling between 0.8 and -0.8 for all groups. The acceptable range for skewness or kurtosis is below +1.5 and above -1.5 (Tabachnick & Fidell, 2013). To test homogeneity of variance, Levene's test was also conducted; except for achievement, equality of variances was found to be non-significant. Since Levene's test was significant for achievement, Games-Howell post-hoc comparisons were used (Field, 2005). An Analysis Variances (ANOVA) was warranted since the interval data met the assumptions of no significant outliers, normality, and homogeneity of variance.

Results
Results of the study are presented in line with the research questions in this section. Table 4 shows overall results of the study. Note. M = sample mean, SD= standard deviation, n=sample size, F=F-ratio, η 2 =effect size

Achievement
There was a statistically significant difference between groups as determined by the one-way ANOVA, with a medium effect size, F (2, 504) = 38.354, p = .000, η2 = 0.13. As stated above, the results of Levene's test were significant, meaning the variance was unequal and also sample sizes were very different (see Table 4), so Games-Howell comparisons were used (Field, 2005). Post hoc comparisons using Games-Howell indicated that the mean for the credit bearing group (M = 76.64, SD = 10.32) was significantly different from the non-credit bearing group (M = 66.64, SD = 10.04) and credit careless group (M = 68.04, SD = 14.76). However, there was no significant difference between the credit careless group and non-credit bearing group.

Intrinsic Goal Orientation
There was a significant effect of credit on student intrinsic goal orientation with a small effect size for  Table 4).

Extrinsic Goal Orientation
There was a significant effect of credit on student extrinsic goal orientation with a small effect size for Ataturk's Principles and History of Turkish Revolution I, F(2, 502) = 16.24, p = 0.000, η 2 = 0.06. Because of the difference in sample sizes, post hoc comparisons were made using Hochberg, which indicated that the credit bearing group (M = 4.73, SD = 1.28) was significantly different from the non-credit bearing group (M = 3.68, SD = 1.07). There was also a significant mean difference between the credit careless group (M = 4.65, SD = 1.21) and the non-credit bearing group. However, there was no significant difference between the credit bearing group and credit careless group.

Course Value
There was a significant effect of credit on student course value with a small effect size for Ataturk's

Discussion and Conclusion
The study set out with the aim of assessing the effect of credit in MOOC-like learning environments on student achievement, perceived intrinsic and extrinsic goal orientation, and course value. When examining findings, the mean score hierarchy was generally the same for all three groups across each dependent variable. That is, mean scores of the credit bearing group were the highest, followed by the credit careless group and then the non-credit bearing group. The most important finding of the study 84 might be the significant differences between the credit bearing group and non-credit bearing group for achievement score, perceived intrinsic goal orientation, perceived extrinsic goal orientation, and perceived course value. The likelihood of this finding has been pointed out in the literature (Chamberlin & Parish, 2011;Hew & Cheung, 2014).
Regarding achievement, the credit bearing groups had significantly higher mean scores than both other groups. Most importantly, 13% of the variance is likely to result from the credit earned from the MOOC.
This result is parallel to the predictions of the Baylor University Teaching, Learning & Technology Committee (2013), who reported that online courses requiring a financial commitment and providing credit are likely to lead to better retention rates and enhance learning opportunities. Though the difference was not significant, achievement scores were slightly higher for the credit careless group than for the non-credit bearing group. What is interesting in these data is a large standard deviation, about 50% higher, for achievement scores by the credit careless group compared to the other two groups (Table   4). While the state of being unsure may have prompted some students to work harder, other students appeared to have been negatively affected by the uncertainty, possibly causing the higher variance.
Lack of credit decreased students' perceived intrinsic and extrinsic goal orientation, and perceived course value: the credit bearing group scores were not significantly different from the credit careless group but were significantly different compared to the non-credit bearing group. This result may imply that even the possibility of earning credit affected students' perceptions in the credit careless group. Another finding of the present study was that the scores of the credit careless group were significantly higher than those of the non-credit group for extrinsic goal orientation. This result, as well as high mean scores for all other dependent variables for the credit careless group, may be explained by the fact that the majority of students (88.7%) in the credit careless group came from programs where the course earns credit (Table   3). While perceived scores were high in the credit careless group, this result did not affect achievement scores; increased achievement scores require dedication and hard work, while perceived scores are simply students reporting their intentions. Another question that this study sought to answer was whether awarding a course for credit influences the value of the course. Compared to its non-credit alternative, the perceived value of this course was definitely affected from credit.
In conclusion, credit did make a difference for all dependent variables investigated in this study when compared to a non-credit-bearing alternative. In other words, the credit-bearing group performed better than the non-credit-bearing group in terms of achievement and had significantly higher perceived extrinsic and intrinsic goal orientation and course value scores. Although the possibility of earning credit had an effect on students' perception scores, especially for extrinsic goal orientation, this possibility did not affect achievement scores. After resolving problematic crediting issues such as identity verification (Sandeen, 2013a) and quality assurance (Hollands & Tirthali, 2014), policymakers should create mechanisms that allow students to take MOOCs for credit. Various models of credit recognition can be employed along the MOOC-to-credit pathway, such as recognition of prior learning, articulation, content licensing, and reciprocal arrangements (Universities UK, 2013). Pundak et al. (2013), on the other side, noted four models: background model, integrated model, replacement course, and enrichment. Whatever credit recognition model is employed, it is important to note that MOOCs for credit will likely increase the overall number of MOOCs (Yuan et al., 2014) and their sustainability (Universities UK, 2013), as well as decrease dropout rates (Schulze, 2014), which are their most severe problem (Agarwala, 2013).

Limitations and Further Studies
This study employed a causal comparative research design; therefore, result interpretation bears the limitations of related research design. For instance, not eliminating differences that might result from student programs may be a limitation. Since the study was conducted in the context of the course Ataturk's Principles and History of Turkish Revolution I, results might also be affected by the subject matter. Therefore, similar studies should be conducted on other subjects, and results can be compared.
Sample sizes between groups were different due to population structure, which may also be a limitation.
No precautions were able to be taken other than selecting appropriate statistical analysis methods.
Learning level of the students was calculated as a result of midterm and final exams with multiple-choices tests. This might not reflect learning outcomes appropriately so it is recommended that in future research, alternative assessment methods can be administrated.
In addition, this study was conducted in a MOOC-like learning environment rather than a real MOOC setting since regulations in Turkey do not currently allow students to earn credits with MOOCs. In spite of their similarities, these environments have different dynamics, so the study should be replicated in a real MOOC setting; however, it is worth noting that MOOCs are not a totally new phenomenon, so MOOC research should not ignore existing distance education research (Bates, 2014). Finally, networked learning, creating effective interaction and collaboration ways in massive learning environments are other critical topics to be investigated in future studies. Besides these, further studies could explore the effect of crediting on student behaviors in MOOC platforms or on their engagement in online activities.