Influence of Incentives on Performance in a Pre-College Biology MOOC

There is concern that online education may widen the achievement gap between students from different socioeconomic classes. The recent discussion of integrating massive open online courses (MOOCs) into formal higher education has added fuel to this debate. In this study, factors influencing enrollment and completion in a pre-college preparatory MOOC were explored. University of California at Irvine (UCI) students of all preparation levels, defined by math Scholastic Aptitude Test (SAT) score, were invited to take a Bio Prep MOOC to help them prepare for introductory biology. Students with math SAT below 550 were offered the explicit incentive of an early change to the biology major upon successful completion of the MOOC and two additional onsite courses. Our results demonstrate that, among course registrants, a higher percentage of UCI students (>60%) completed the course than non-UCI registrants from the general population (<9%). Female UCI students had a greater likelihood of enrolling in the MOOC, but were not different from male students in terms of performance. University students entering with low preparation outperformed students entering who already had the credentials to become biology majors. These findings suggest that MOOCs can reach students, even those entering college with less preparation, before they enter university and have the potential to prepare them for challenging science, technology, engineering, and mathematics (STEM) courses.


Résumé de l'article
There is concern that online education may widen the achievement gap between students from different socioeconomic classes. The recent discussion of integrating massive open online courses (MOOCs) into formal higher education has added fuel to this debate. In this study, factors influencing enrollment and completion in a pre-college preparatory MOOC were explored. University of California at Irvine (UCI) students of all preparation levels, defined by math Scholastic Aptitude Test (SAT) score, were invited to take a Bio Prep MOOC to help them prepare for introductory biology. Students with math SAT below 550 were offered the explicit incentive of an early change to the biology major upon successful completion of the MOOC and two additional onsite courses. Our results demonstrate that, among course registrants, a higher percentage of UCI students (>60%) completed the course than non-UCI registrants from the general population (<9%). Female UCI students had a greater likelihood of enrolling in the MOOC, but were not different from male students in terms of performance. University students entering with low preparation outperformed students entering who already had the credentials to become biology majors. These findings suggest that MOOCs can reach students, even those entering college with less preparation, before they enter university and have the potential to prepare them for challenging science, technology, engineering, and mathematics (STEM) courses.

Introduction
The demand for a diverse workforce of individuals with degrees in science, technology, engineering, and mathematics (STEM) has focused attention on how to increase the enrollment and retention of undergraduate STEM majors (National Science Foundation, 2004;National Science Board, 2006;Wilson et al., 2012;National Science Board, 2012). Attrition in STEM majors is particularly high during the first two years of undergraduate education (Seymour & Hewitt, 1997 ;Shuman et al., 1999;Chang, Cerna, Han, & Sàenz, 2008). At many universities, large introductory STEM courses, often referred to as "gate-keepers," represent major barriers that discourage students from entering or persisting in STEM disciplines. Moreover, females (Felder, Felder, Mauney, Hamrin, & Dietz, 1995;Brainard & Carlin, 1997;Blickenstaff, 2005), first-generation college students (Engle & Tinto, 2008), students from low-income families (Lam, Srivatsan, Doverspike, Vesalo, & Mawasha, 2006), and under-represented minority students (Cole & Espinoza, 2008) are more likely to drop out from STEM majors.
In recent years, with the number of college applications increasing (Clinedinst & Hawkins, 2014), there are growing numbers of low-income, under-represented minority, and first generation college students in entering classes. These students represent a tremendous resource for developing a diverse community of STEM graduates but they are also at high risk for dropping out since they are often academically under-prepared (Perna & Titus, 2005;Gumport, 2007;Breneman, Jr., & Hoxby, 1998). Providing these incoming students with access to low cost, high quality courses that help them transition to college and be successful in STEM majors is therefore of great importance. However, financial constraints have made it increasingly difficult for the brick-and-mortar campuses to provide transition face-to-face instruction to support these students (Clinedinst & Hawkins, 2014), especially in the summer before initial entry.
The emergence of massive open online courses (MOOCs) potentially affords universities the opportunity to provide students with preparatory courses before they enter university, at relatively low cost. MOOCs are typically developed by university faculty and are free for anyone who has access to a computer and the internet. In general, the completion rate of stand-alone MOOCs is very low (about 5%) (Ho et al., 2014).
Enrollment surveys indicate that the students in the new massive courses are largely male and well-educated learners, rather than the underserved (Christensen et al., 2013).
A pilot study conducted in San Jose State University, which introduced MOOCs into the curriculum, reported that matriculated students performed better than nonmatriculated students. However, the completion rate of the specifically targeted at-risk students was disappointingly low. It was reported that students' effort was the strongest indicator of their success in the course (Firmin et al., 2014). In addition, students claiming a high intention to complete a MOOC in a pre-survey are much more likely to finish the course than others (Koller et al., 2013). This raises the question of whether it is possible to increase learners' motivation, engagement, and success in MOOCs by providing external incentives. The goal of this study was to develop a course that would help under-prepared students who had been accepted to the UCI gain skills and knowledge that would increase their probability of success in a large freshman STEM course, Bio 93. We chose to develop this as a MOOC to determine if this could be done in the context of a course that would also potentially benefit a broad group of individuals in the general population. We first introduce the rationale for providing this course, explain how it was organized, and describe the incentive offered to increase motivation for under-prepared students, and then analyze the extent to which the course achieved our goals.

Research Questions
The study addresses the following questions.

Results
There were differences in performance in the MOOC between the three groups examined: strong math UCI students, weak math UCI students and non-UCI MOOC participants. Typical of many MOOCs, the completion percentage of non-UCI participants was low, with 92% not completing. Of those that completed, approximately 7% earned a Normal certificate and <2% earned Distinction (Figure 1). In marked contrast, the two groups of UCI students had a much higher percentage of completion and Distinction. For the strong math UCI students only 36% did not complete the course, while 37% earned a Normal certificate and 27% earned a Distinction certificate ( Figure 1). The percentage of weak math UCI students (n=156) that did not complete the course (31%) was similar to the strong math group, but more earned a Distinction certification (39%) than a Normal certificate (30%) (Figure 1).  MOOC Enrollment Table 1 shows the odds ratios from the nested logistic regression models predicting UCI Bio 93 students enrolled in the Bio Prep MOOC, using data from the entire cohort of Bio 93 students. As our hypothesis predicted, we found weak math students are more likely to enroll in the course. However, when we control for gender the effect of math is not significant. This indicates that women are more likely to enroll in the MOOC than men.
There was no effect of ethnicity on enrollment.

MOOC Completion
We hypothesized that students with weak math SAT scores would be more likely to complete the MOOC with distinction, given the incentive provided by the university. As predicted, the probability of obtaining the Distinction certificate for students with low SAT math was about 1.66 times greater than the high SAT math students (

Discussion
Our results demonstrate that the percentage of UCI students completing the MOOC and earning Distinction was much higher than the non-UCI MOOC students. Female students were more likely to enroll than male students, and low math SAT students were more likely to earn Distinction.
There would be little support for investing in development of college preparatory classes if the completion rates are similar to those reported for most MOOCs (~5%) (Ho et al., 2014). A study at San Jose State University indicated that students in a MOOC who are also matriculated and the course counts for university credit have a higher completion The exposure to knowledge and/or skills relevant to a first quarter class was sufficient to result in this increase since incoming 1st year students with the credentials to become biology majors completed at approximately the same rate as underprepared students who were given an explicit incentive.
There is considerable concern that online education programs, while effective for some, may amplify rather than narrow educational and social divides (Bolt & Crawford, 2000).
Some quasi-experimental studies show that online education reflects the same divide commonly observed in the brick-and-mortar settings. For example, Black and Hispanic students performed more poorly than White students in online courses (Newell, 2007).
In an experimental study involving multiple sections of an economics course no significant difference was found between online and lecture courses among students with higher prior GPA; however, among students with lower GPA, online students scored significantly lower than face-to-face lecture students (Figlio, Rush, & Yin, 2010).
In the San Jose State University study, the low course pass rates may be due to the target group being at-risk students (Firmin et al., 2014).
In the present study we found that UCI incoming students with weak SAT math skills had a higher probability of completing the MOOC with Distinction than students with strong math skills. It is likely that this difference in completing the MOOC with Distinction is associated with the incentive policy enacted for incoming freshmen who did not meet the SAT math requirement for a biology major. This provided freshmen who obtained a Distinction certificate the opportunity to enter the biology major two quarters earlier than those who did not complete or earned only a Normal certificate.
Importantly this incentive did not result in any costs for the students, the instructors, or the university.
Our results also indicate that among UCI students, females were more likely to enroll in the MOOC than male students. This is interesting given previous research showing that the majority (51%~87%) of MOOC population are males (Ho et al., 2014) and it is potentially important given the underrepresentation of women in many STEM fields (Beede et. al, 2011) and women's greater likelihood to transfer out of STEM majors (Chen & Soldner, 2013). One possible interpretation, consistent with a recent study, is that female students are more motivated and adaptive to online education than male students in an educational setting (Xu & Jaggars, 2013). In contrast, unlike the Xu and Jaggars study, we did not find negative associations between ethnicity and enrollment or performance in the MOOC.
Although previous literature shows that low performance students can be further disadvantaged by online education, our results suggest that a MOOC with no-cost incentives provides an additional learning opportunity for low-performance students.