Impact of OpenCourseWare Publication on Higher Education Participation and Student Recruitment

The free and open publication of course materials (OpenCourseWare or OCW) was initially undertaken by Massachusetts Institute of Technology (MIT) and other universities primarily to share educational resources among educators (Abelson, 2007). OCW, however, and more in general open educational resources (OER),1 have also provided well-documented opportunities for all learners, including the so-called “informal learners” and “independent learners” (Carson, 2005; Mulder, 2006, p. 35). Universities have also increasingly documented clear benefits for specific target groups such as secondary education students and lifelong learners seeking to enter formal postsecondary education programs. In addition to benefitting learners, OCW publication has benefitted the publishing institutions themselves by providing recruiting advantages. Finally enrollment figures from some institutions indicate that even in the case of the free and open publication of materials from online programs, OCW does not negatively affect enrollment. This paper reviews evaluation conducted at Massachusetts Institute of Technology, Johns Hopkins Bloomberg School of Public Health (JHSPH), and Open Universiteit Nederland (OUNL) concerning OCW effects on higher education participation and student recruitment.


Résumé de l'article
The free and open publication of course materials (OpenCourseWare or OCW) was initially undertaken by Massachusetts Institute of Technology (MIT) and other universities primarily to share educational resources among educators (Abelson, 2007). OCW, however, and more in general open educational resources (OER), have also provided well-documented opportunities for all learners, including the so-called "informal learners" and "independent learners" (Carson, 2005;Mulder, 2006, p. 35). Universities have also increasingly documented clear benefits for specific target groups such as secondary education students and lifelong learners seeking to enter formal postsecondary education programs. In addition to benefitting learners, OCW publication has benefitted the publishing institutions themselves by providing recruiting advantages. Finally enrollment figures from some institutions indicate that even in the case of the free and open publication of materials from online programs, OCW does not negatively affect enrollment. This paper reviews evaluation conducted at Massachusetts Institute of Technology, Johns Hopkins Bloomberg School of Public Health (JHSPH), and Open Universiteit Nederland (OUNL) concerning OCW effects on higher education participation and student recruitment.
lifelong learners seeking to enter formal postsecondary education programs.
In addition to benefitting learners, OCW publication has benefitted the publishing institu- In this paper we will use one reference term only (OCW), just for convenience and being fully aware of the definition differences between OCW and OER. Only in the case of possible misunderstanding we refer specifically to OER.

Stephen Carson 1 , Sukon Kanchanaraksa 2 (not shown), Ira Gooding 2 , Fred Mulder 3 , and Robert
Schuwer 3 (Mulder, 2006). Their learning materials are specifically designed for independent learning, putting the learner in the centre rather than the teacher. Clearly the combination of this OU model with OER offers great potential to increase and widen participation in higher education, which in Europe is an explicit ambition.
The OpenER project (Schuwer & Mulder, 2009) was set up as a 'portal of enticement' to higher education in The Netherlands. Existing barriers should be removed and the willingness of individuals to invest in learning be stimulated. The assumption was that free access to high-quality and attractive learning materials for self-study in an informal learning mode would facilitate this. The project has run from 2006 until 2008, has published 25 courses (at bachelor level, 25 hours study load each), and has attracted more than 1 million unique visitors. Spin-offs were the use of OpenER courses in secondary schools and the introduction of the so-called Spinoza Series in collaboration with the National Organization for Scientific Research. In this series Spinoza laureates 3 share their knowledge with a broad audience in OpenER type courses. Currently the OpenER course base is the free access part of the regular OUNL offering and is extended gradually.

Data Sources
Data presented in this paper were gathered independently by researchers at the included institutions without prior coordination, largely as part of ongoing program self-evaluation.
For that reason, data are not directly comparable from one institution to another. Researchers comparing findings noted complementary outcomes, which resulted in the de- Surveys are conducted using a pop-up window or banner invitation on the site and are collected with an online survey tool. Completion rates for the surveys conducted 2004-2010 were typically 3-5% of invited visitors. As compared to web metrics, the sample over-represents returning as opposed to first-time visitors (survey samples are 60% returning whereas metrics indicate 30-50% of traffic is returning visitors).
• 2010 undergraduate survey: The entire undergraduate student population of 4,232 was invited by e-mail to complete a survey through an online survey tool, and 1,891 students completed the survey, yielding a 44.7% response rate. Skip logic was employed to direct respondents to appropriate questions based on prior responses. Respondents were entered in a random drawing to receive movie tickets as an incentive.
• 2010 graduate survey: The same type of survey was conducted among 6,152 graduate students, and 1,644 students completed the survey, yielding a 26.7% response rate.
This survey data is supplemented by interviews conducted with selected survey respondents and MIT community members. While OCW was originally conceived at MIT as a tool for sharing course materials with educators, and educators were expected to be primary users of the site, the MIT OpenCourse-Ware staff documented from the start of the program the unexpectedly large number of so-called "informal learners" coming to the site. These were site visitors who were not currently enrolled in or teaching at an institution of higher education. In the early stages of MIT OpenCourseWare, informal learners were measured as just under 52% of site visitors, with 31% being students and 13% educators (Carson, 2005). More recent measures from the 2010 survey indicate that 43% of MIT OpenCourseWare visitors are informal learners, 42% are students, and 9% are educators. As early as 2003, MIT OpenCourseWare was able to document that 8% of informal learners were using the site to "plan a future course of study as a student." By 2005, nearly 18% of informal learners indicated using the site to "plan a future course of study" (Carson, 2005).
Findings from OUNL surveys further the case that OER / OCW supports informal learners in planning a return to formal education. Sources A and B resulted in the following: Findings from OUNL surveys further the case that OER / OCW supports informal learners in planning a return to formal education. Sources A and B resulted in the following data.
In a slightly different way the question about influence of OpenER on study plans was asked in sources C and C': 25 Thirty percent of respondents were already enrolled in a program of formal education, both at OUNL and other institutions. OUNL also surveyed 1,684 students who purchased a course for credit, asking whether OpenER had influenced their decision to enroll in the paid course. Nine percent indicated this was the case (Schuwer & Mulder, 2009).
While both the MIT and OUNL data indicate that OCW can be an aid to or influence on informal learners considering a return to formal education, the figures do not reveal how OCW provides such support. On the MIT 2010 site visitor survey, 21% of independent learners indicated they were using the site to "plan a return to formal study." These respondents were asked follow-up questions asking for more detail on how they were using the site for this purpose and how successful they were at achieving these goals (see Table 3).
When presented with three more specific scenarios of use related to planning a return to formal study, the majority (53%) indicated using the site to select an area of study; a large percentage (46%) indicated using the site to review basic knowledge from previous formal study; and just under a third of respondents were using the site to prepare for entrance exams. For all of these subscenarios, more than 70% of respondents indicated they were "completely successful" or "mostly successful" at meeting their goals for visiting the site.
Ten percent of respondents chose "Other" as a response to the above question and provided a free-text description of their site usage. While many of these might be reasonably correlated to the above subcenarios ("preparation for exams," "learning more about the field I would like to study eventually"), others provide interesting nuance regarding how OCW supports a return to formal study. One response demonstrates how OCW resources can support a transition from one field to another: "advance beyond prior study, as my B.A. was not math-related, but I wish to pursue M.S. in Mathematics." Another response indicates how OCW can provide a "leg up" on learning, so that students are more comfortable with the materials they will encounter in formal study: "learning the material on my own so the concepts will be easier to grasp when I enter a formal course." Follow-up questions were also asked about the benefits generated through using the site to plan a return to formal education. Respondents were also asked to indicate the level of each type of benefit, providing insight into the relative amount of each benefit provided. These responses are summarized in Table 4. The above responses highlight the importance of access to higher education materials in providing inspiration for respectively developing confidence in considering a return to formal education. For these two options 69% and more than 71% of respondents rate the level of benefit as "very significant" or "significant." A clear majority also assigns "very significant" or "significant" benefit to review prior learning (almost 69%) and to selecting programs and courses, both by improving decision-making (over 63%) and by saving time (57%). Finally, more than half of the respondents report "very significant" or "significant" benefit in improving performance on entrance exams (55%) and in saving time in its preparations (over 52%).

Student Recruitment
The survey results presented above indicate that OCW can be an effective tool for helping learners make the transition between informal and formal study. This, however, does not necessarily indicate a direct benefit for the publishing universities. And they, of course, are keenly interested in documenting any ways in which OCW publication might improve their student recruitment, thus providing direct benefits.

MIT.
The MIT OpenCourseWare staff has surveyed both undergraduate and graduate MIT populations to explore effects of OCW on student recruitment. The most recent survey results indicate that the site does help MIT to be more competitive in attracting students at both levels. A 2010 undergraduate survey indicates 57% of undergraduates were aware of the OCW site before they chose to attend MIT, and among them 31% indicate the site had a "very significant" or "significant" influence on their choice of school; a similar 2010 survey of MIT graduate students indicates that 45% of graduate students were aware of the MIT OpenCourseWare site prior to choosing the school they would attend, and 30% of those aware describe the site's influence on their choice as "very significant" or "significant." Even early on at MIT, surveys of incoming freshmen indicated some effect on student re-  Table 5.
As the number of courses present on the site increases, the influence of the site in generally presenting unique aspects of the MIT community/culture diminishes, from 36% for seniors to 30% for freshmen; whereas, types of influence more directly linked to specific content ("demonstrated quality of teaching materials" and "provided insight into teaching approaches") increase, from 33% to 48% and from 21% to 38% respectively. This would suggest that while some of the recruitment benefits of OCW can be accomplished with a relatively limited set of courses, additional benefits accrue as coverage of curriculum increases.

JHSPH.
Data collected by Johns Hopkins Bloomberg School of Public Health provide some support to the overall conclusion of influence, albeit slight, of OCW publication on student enrollment at the publishing institution. Surveys sent to new JHSPH students in [2006][2007][2008] and to all students in 2010-2011 indicate that JHSPH OCW influenced 6.9% of respond- 28 ing students in their decision to attend JHSPH. When students who were unaware of JH-SPH OCW prior to receiving the survey are excluded from the analysis, the percentage of responding students who report being influenced by OCW climbs to at least 11.8%. The percentage could be higher depending on how many students became aware of OCW in the interval between making their enrollment decision and receiving the survey.
After students make their initial institutional enrollment decision, they also face an ongoing series of course enrollment decisions. JHSPH data indicate that OCW publication has a greater influence on these course-level decisions, with some students reporting that their academic planning has been influenced by OCW. Survey results collected in 2010 and 2011 indicate that 31.2% of students who reported being aware of OCW also reported that they used it during their course selection process. When these students were asked if the use of OCW during course selection influenced their decision to enroll in a particular course, 69.9% responded in the affirmative. Furthermore, 35.1% of students who used OCW during course selection also reported that it influenced their decision to not enroll in a particular course.

OUNL.
As mentioned before, at OUNL the OpenER site's influence on the enrollment decisions of students is about 9%, with a total of 24 courses available. This outcome is more or less supported by the results of the survey among the voluntarily registered visitors of the OpenER site. Forty-two percent of the people returning the survey reported they applied for a formal study program or they had bought a course. It is not clear, however, whether this was already the case before visiting the OpenER site.
Note that because most of these surveys investigating influence on school choice were only sent to enrolled students, little information was gathered on site visitors who might have been influenced to not attend the publishing university, a possible outcome of viewing the materials as well. In Table 2, 4% of respondents to OUNL surveys indicate that visiting the OpenER site did influence them to decide not to study more formally, so some evidence of this outcome exists.

Online Program Enrollment
Despite indications that OCW publication can provide significant advantages in student recruitment, universities offering online programs have been understandably concerned that the open publication of materials similar to those they are using in for-credit and for-fee distance learning would undercut enrollment. MIT does not offer distance learning programs, but Johns Hopkins Bloomberg School of Public Heath does, and data collected by this program indicates that OCW publication has no apparent negative effect on distance learning enrollment.
Between 2005  Finally, open universities, where the (online) learning materials are generally designed for self-study by "independent learners," are at risk of a possible decline in enrollment because potential students could indeed stick to the "informal learning" mode when the courses are offered as OER. This explains their hesitation to convert their full course base to OER.
OUNL has just finished an explorative study into the effects on enrollment of different OER scenarios. We can report here that the results for a representative sample of the Dutch population not being OUNL students indicate that the ultimate (100%) OER scenario would generate a little more (although not statistically significant) expected enrollment than the less radical OER scenarios (Janssen, Schuwer, & Mulder, 2012). This outcome may be considered a surprise, but at the same time it is relevant and encouraging because it gives ground to abandon the so far well-understood reserve among open universities with respect to a full OER approach.

Return on Institutional Investment
Open educational resources efforts are undertaken at institutions for a wide variety of reasons, generally most directly related to fulfilling the core mission of higher education to create and disseminate knowledge, but also to build awareness of and goodwill toward publishing institutions, as well as to foster improvements in teaching and learning effectiveness. Improving student recruitment is a documented benefit but not the only measure of return on institutional investment. That said, even a rough analysis of the return on investment at both JHSPH and OUNL indicates that the benefits likely far outweigh the costs, even when considering only the narrow question of student recruitment. However, MIT, due to its student recruitment approach, does not have a clear method of estimating return on investment from student recruitment.

MIT.
MIT accepts a limited number of students each year, and so MIT OpenCourseWare does not act to increase the number of students attending and paying tuition. Based on the data presented above, it appears that MIT OpenCourseWare helps to increase the quality of incoming students by making the Institute more competitive against peer institutions, but there is no clear way to measure the monetary impact of such an influence.

JHSPH.
The total cost of operating JHSPH OCW is low (managed by 0.5 FTE or half time of a staff member). Very modest recruitment impacts would offset the operating expenses. In 2011-2012 academic year, 993 new students enrolled at JHSPH. If only a couple of students were attracted to JHSPH because of its OCW, JHSPH would have recovered its annual investment.

OUNL.
As noted above, 9% of OUNL students reported being influenced by OpenER in their decision to buy a regular course. In US dollars, this represents $1.36 M in tuition revenues attributable at least in part to the site. This would significantly offset the annual cost of operating OpenER, which on the average over the two years of operation plus additional two years of availability of the courses has been around $201,000. It is, however, not possible to be firm on this because it is not known how many of these students would have enrolled also in the absence of OpenER and how many potential students decided not to buy a regular course because of the free offer through OpenER.

Conclusions
OCW can be an important tool to increase or widen participation in formal higher education, especially in supporting a return to formal education by lifelong learners.
Learners access openly available educational resources for informal study in very large numbers, indicating a widespread and pervasive need for continued learning opportunities. A subset of this use is in the context of learners contemplating a return to formal education, and, in this case, OCW appears to serve a number of important purposes, both inspirational and functional. OCW appears to both inspire interest in continued education and help learners develop confidence in their ability to succeed by providing in-depth access to the learning materials they will encounter during formal study. OCW also appears to be a significant aid in selecting programs and courses, preparing for entrance exams, and getting a "head start" on studies.
These effects are significant at the systemic level and should be considered by governments, NGO's, and foundations seeking to increase higher education participation among broad populations. Especially among populations with many first-generation college students, informal exposure to higher education materials published as OCW may be a useful way for potential students to understand what it means to study at the college level. For situations where increased levels of participation in higher education are explicit goals, such as is the case in the Netherlands (and in many countries), OCW may serve as an important and costeffective entry to formal higher education.
OCW can be a significant tool for improving student recruitment.
OCW projects demonstrate fairly consistent influence on student school choice of between 10-30% of students who have viewed the OCW sites. These figures indicate a clear opportunity for schools to improve recruitment through the open publication of learning materials.
Because an aspect of OCW's influence appears to be communication of the culture of the publishing school, it's arguable that even if OCW influences potential students to not attend a given school, such an outcome may be in the best interests of both the student and the school by contributing to a better match between the two.
It may be the case that MIT, JHSPH, and OUNL enjoy first mover advantages in publishing educational resources openly and student recruitment advantages may become less pronounced as more schools publish open educational resources; conversely, if OER publication is a widely embraced practice, schools choosing not to openly publish their materials may generate suspicion among prospective students. These are issues for further study.
Issues of student retention and success were not addressed in the data collection and are also an area for further study.
OCW does not appear to negatively affect enrollment of online programs from which materials have been openly shared.
While this is an area for future study, preliminary indications from the JHSPH experience indicate no discernable impact of OCW publication on online course enrollment when the same materials are used in both cases. This may be an outcome specific to the licensure requirements for practice in a field such as public health and may not generalize into other domains where certification is relatively less critical. However, the recently published OUNL study similarly gives support to this third conclusion. As more online programs experiment with OER and OCW approaches, a clearer picture of the relationships should emerge.