|
|
|
|
Stephen Carson (1), Sukon Kanchanaraksa (2) (not shown), Ira Gooding (2), Fred Mulder (3), and Robert Schuwer (3)
(1) Massachusetts Institute of Technology, US (2) Johns Hopkins Bloomberg School of Public Health, US
(3) Open Universiteit, The Netherlands
The free and open publication of course materials (OpenCourseWare or OCW) was initially undertaken by Massachusetts Institute of Technology (MIT) and other universities primarily to share educational resources among educators (Abelson, 2007). OCW, however, and more in general open educational resources (OER)1, have also provided well-documented opportunities for all learners, including the so-called “informal learners” and “independent learners” (Carson, 2005; Mulder, 2006, p. 35). Universities have also increasingly documented clear benefits for specific target groups such as secondary education students and lifelong learners seeking to enter formal postsecondary education programs.
In addition to benefitting learners, OCW publication has benefitted the publishing institutions themselves by providing recruiting advantages. Finally enrollment figures from some institutions indicate that even in the case of the free and open publication of materials from online programs, OCW does not negatively affect enrollment. This paper reviews evaluation conducted at Massachusetts Institute of Technology, Johns Hopkins Bloomberg School of Public Health (JHSPH), and Open Universiteit Nederland (OUNL) concerning OCW effects on higher education participation and student recruitment.
Keywords: Distance education; open learning; open universities; distance universities; higher education; e-learning; online learning
On April 4th, 2001, MIT announced its intention to publish the core educational materials, including syllabi, lecture notes, assignments and exams, from all of its courses freely and openly on the Web for use by educators and learners worldwide through a project dubbed “MIT OpenCourseWare.” MIT was soon joined in this effort by universities around the world for which the mission of openly sharing educational content resonated strongly with long-established and deeply held institutional commitments. Two such institutions were Johns Hopkins, through its Bloomberg School of Public Health and its longstanding commitment to improving global public health, and Open Universiteit Nederland, with its mission to make higher education accessible to anyone with the necessary aptitudes and interests, regardless of formal qualifications and with independent learning as its characteristic model.
OpenCourseWare materials as originally envisioned by MIT encompass the core documents provided to students in the MIT classroom setting, plus other digital resources, such as simulations, animations, and sample code, created in the course of normal instruction of MIT students; subsequent projects at MIT and elsewhere have extended the OCW concept to include materials specifically designed for free and open use on the Web.2
Officially launched on October 1, 2003, MIT OpenCourseWare contains materials from more than 2,080 MIT courses. These courses widely cover MIT’s course offerings at both the undergraduate and graduate level, providing a comprehensive view into the Institute’s curriculum. MIT OpenCourseWare educational materials have reached an estimated 100 million individuals through the main site (see http://ocw.mit.edu); translation sites sharing 1,000 versions of MIT courses in languages including Chinese, Spanish, Portuguese, Farsi, and Thai; more than 290 copies of the site distributed to universities in bandwidth-constrained regions; and through secondary uses of the materials in the classrooms of educators worldwide. In 2007, MIT OpenCourseWare launched Highlights for High School, which identifies more than 2,600 resources within the MIT OpenCourseWare publication appropriate for US Advanced Placement study of biology, calculus, and physics. In 2010, MIT published the first 5 of a planned 20 “OCW Scholar” courses, designed specifically for use by independent learners with no access to supplemental resources.
Launched in May of 2005, JHSPH OpenCourseWare includes teaching resources from more than 100 academic courses, symposia, and trainings offered by JHSPH. Its goals are to provide a spectrum of educational opportunity that includes learners who cannot formally enroll and to support public health educators who require high-quality adaptable resources.
Each JHSPH OCW offering features lecture materials and a course syllabus with learning objectives, and many include additional materials such as assignments, lab projects, and readings. Twenty-five offerings also feature audio or video resources. JHSPH OCW covers a wide variety of public health topics, such as infectious diseases, epidemiology, nutrition, reproductive health, and biostatistics. In 2008, the OCW Image Library was launched to improve direct access to the many illustrations and charts created for JHSPH OCW. In 2012, JHSPH OCW will launch a new Web site with the goal of making it easier for visitors to find, share, and use OCW materials.
In 2006 two open universities, in the UK and in The Netherlands, initiated a new OCW wave by making available a small part of their course base as OER. They combined their classical openness (open entry, freedom of time, pace, and place, open programming, and open to people) with the new openness of the digital era, in particular with OER (Mulder, 2006). Their learning materials are specifically designed for independent learning, putting the learner in the centre rather than the teacher. Clearly the combination of this OU model with OER offers great potential to increase and widen participation in higher education, which in Europe is an explicit ambition.
The OpenER project (Schuwer & Mulder, 2009) was set up as a ‘portal of enticement’ to higher education in The Netherlands. Existing barriers should be removed and the willingness of individuals to invest in learning be stimulated. The assumption was that free access to high-quality and attractive learning materials for self-study in an informal learning mode would facilitate this. The project has run from 2006 until 2008, has published 25 courses (at bachelor level, 25 hours study load each), and has attracted more than 1 million unique visitors. Spin-offs were the use of OpenER courses in secondary schools and the introduction of the so-called Spinoza Series in collaboration with the National Organization for Scientific Research. In this series Spinoza laureates3 share their knowledge with a broad audience in OpenER type courses. Currently the OpenER course base is the free access part of the regular OUNL offering and is extended gradually.
Data presented in this paper were gathered independently by researchers at the included institutions without prior coordination, largely as part of ongoing program self-evaluation. For that reason, data are not directly comparable from one institution to another. Researchers comparing findings noted complementary outcomes, which resulted in the development of this after-the-fact analysis. Coordination of data collection in future studies could enhance the comparability of data across institutions.
Gathering data on the use and impact of open resources such as OpenCourseWare has inherent challenges. Because audiences are broad and come to the sites freely, often with no registration or prior identification, most data gathered comes from web metrics (which reveal behavior but not attitudes or intent) or from surveys and interviews of a self-selecting group of users assumed to be among the most engaged and supportive visitors to the sites. For this reason, the results presented should generally be understood as being representative of a relatively engaged subset of users. Notable exceptions are surveys of defined audiences who may or may not be familiar with the OpenCourseWare resources being investigated, such as the MIT and JHSPH surveys of new students.
Other studies regarding benefits and impact of OpenCourseWare or more generally OER are in most cases performed in the context of business models and questions about sustainability of offering OER. A recent overview is provided by Butcher and Hoosen (2012). Attracting around 6,000 enrollments and a forefront role in the area of open education are among the benefits of the OpenLearn project from the Open University UK (Lane, 2012). In the area of secondary education, Wiley et al. (2012) report significant cost savings when using open textbooks without loss of quality of learning materials. In a TED Talk early 20124, Peter Norvig mentioned as institutional benefit from the so-called massive open online courses (MOOCs) the gathering of enormous amounts of students data that can be mined to improve education.
Data regarding MIT OpenCourseWare for this analysis has been gathered through a series of surveys conducted by the MIT OpenCourseWare staff. These surveys include the following.
This survey data is supplemented by interviews conducted with selected survey respondents and MIT community members.
Data regarding Johns Hopkins Bloomberg School of Public Health OpenCourseWare for this analysis were gathered through two surveys, one conducted in 2006-2008 and the other in 2010 and 2011.
Data regarding OUNL OpenER for this analysis were gathered through two surveys (both conducted in 2008), a feedback form on the Web site (from 2006 to 2008) and an order form on the web shop of OUNL for ordering regular courses (from February to June 2008):
While OCW was originally conceived at MIT as a tool for sharing course materials with educators, and educators were expected to be primary users of the site, the MIT OpenCourseWare staff documented from the start of the program the unexpectedly large number of so-called “informal learners” coming to the site. These were site visitors who were not currently enrolled in or teaching at an institution of higher education. In the early stages of MIT OpenCourseWare, informal learners were measured as just under 52% of site visitors, with 31% being students and 13% educators (Carson, 2005). More recent measures from the 2010 survey indicate that 43% of MIT OpenCourseWare visitors are informal learners, 42% are students, and 9% are educators. As early as 2003, MIT OpenCourseWare was able to document that 8% of informal learners were using the site to “plan a future course of study as a student.” By 2005, nearly 18% of informal learners indicated using the site to “plan a future course of study” (Carson, 2005).
Findings from OUNL surveys further the case that OER / OCW supports informal learners in planning a return to formal education. Sources A and B resulted in the following:
In a slightly different way the question about influence of OpenER on study plans was asked in sources C and C’:
Thirty percent of respondents were already enrolled in a program of formal education, both at OUNL and other institutions. OUNL also surveyed 1,684 students who purchased a course for credit, asking whether OpenER had influenced their decision to enroll in the paid course. Nine percent indicated this was the case (Schuwer & Mulder, 2009).
While both the MIT and OUNL data indicate that OCW can be an aid to or influence on informal learners considering a return to formal education, the figures do not reveal how OCW provides such support. On the MIT 2010 site visitor survey, 21% of independent learners indicated they were using the site to “plan a return to formal study.” These respondents were asked follow-up questions asking for more detail on how they were using the site for this purpose and how successful they were at achieving these goals (see Table 3).
When presented with three more specific scenarios of use related to planning a return to formal study, the majority (53%) indicated using the site to select an area of study; a large percentage (46%) indicated using the site to review basic knowledge from previous formal study; and just under a third of respondents were using the site to prepare for entrance exams. For all of these subscenarios, more than 70% of respondents indicated they were “completely successful” or “mostly successful” at meeting their goals for visiting the site.
Ten percent of respondents chose “Other” as a response to the above question and provided a free-text description of their site usage. While many of these might be reasonably correlated to the above subcenarios (“preparation for exams,” “learning more about the field I would like to study eventually”), others provide interesting nuance regarding how OCW supports a return to formal study. One response demonstrates how OCW resources can support a transition from one field to another: “advance beyond prior study, as my B.A. was not math-related, but I wish to pursue M.S. in Mathematics.” Another response indicates how OCW can provide a “leg up” on learning, so that students are more comfortable with the materials they will encounter in formal study: “learning the material on my own so the concepts will be easier to grasp when I enter a formal course.”
Follow-up questions were also asked about the benefits generated through using the site to plan a return to formal education. Respondents were also asked to indicate the level of each type of benefit, providing insight into the relative amount of each benefit provided. These responses are summarized in Table 4.
The above responses highlight the importance of access to higher education materials in providing inspiration for respectively developing confidence in considering a return to formal education. For these two options 69% and more than 71% of respondents rate the level of benefit as “very significant” or “significant.” A clear majority also assigns “very significant” or “significant” benefit to review prior learning (almost 69%) and to selecting programs and courses, both by improving decision-making (over 63%) and by saving time (57%). Finally, more than half of the respondents report “very significant” or “significant” benefit in improving performance on entrance exams (55%) and in saving time in its preparations (over 52%).
The survey results presented above indicate that OCW can be an effective tool for helping learners make the transition between informal and formal study. This, however, does not necessarily indicate a direct benefit for the publishing universities. And they, of course, are keenly interested in documenting any ways in which OCW publication might improve their student recruitment, thus providing direct benefits.
The MIT OpenCourseWare staff has surveyed both undergraduate and graduate MIT populations to explore effects of OCW on student recruitment. The most recent survey results indicate that the site does help MIT to be more competitive in attracting students at both levels. A 2010 undergraduate survey indicates 57% of undergraduates were aware of the OCW site before they chose to attend MIT, and among them 31% indicate the site had a “very significant” or “significant” influence on their choice of school; a similar 2010 survey of MIT graduate students indicates that 45% of graduate students were aware of the MIT OpenCourseWare site prior to choosing the school they would attend, and 30% of those aware describe the site’s influence on their choice as “very significant” or “significant.”
Even early on at MIT, surveys of incoming freshmen indicated some effect on student recruitment; on a 2004 freshman survey, 52% of incoming freshmen were aware of the OCW site, and nearly 11% of all freshmen indicated that OCW influenced their decision to attend MIT. At this time, MIT OpenCourseWare’s coverage was far from comprehensive, with only about 500 of a planned 1,800 courses available. Students indicated two ways of influence: first, by “illustrating unique aspects of the MIT community/ culture” (30.5%), and because they found the mission of OCW inspiring and wanted to attend a school that would undertake such a project (not a preselected answer, but the overwhelming majority of “other” free-text responses, 33.5%).
Subsequent surveys continue to demonstrate a 25-35% rate of influence among freshmen aware of the site prior to choosing the school they would attend. In 2009, all undergraduates were asked if the site influenced their decision to come to MIT, and, if so, to indicate how. More than 26% overall indicated the site influenced their choice to attend the Institute, and their responses, which track the changing influence of the site over time, provide interesting insight. The senior class in 2009 entered MIT in 2005, when the site contained only roughly half of the 1,800 planned courses; by 2006, when 2009 juniors entered, the site contained about 1,200 courses; by 2007, when 2009 sophomores entered, the site contained 1,550; and by 2008, when 2009 freshmen would have been looking at the site, the site contained the full 1,800 courses planned. Ways the site influenced attendance, broken out by class, are presented in Table 5.
As the number of courses present on the site increases, the influence of the site in generally presenting unique aspects of the MIT community/culture diminishes, from 36% for seniors to 30% for freshmen; whereas, types of influence more directly linked to specific content (“demonstrated quality of teaching materials” and “provided insight into teaching approaches”) increase, from 33% to 48% and from 21% to 38% respectively. This would suggest that while some of the recruitment benefits of OCW can be accomplished with a relatively limited set of courses, additional benefits accrue as coverage of curriculum increases.
Data collected by Johns Hopkins Bloomberg School of Public Health provide some support to the overall conclusion of influence, albeit slight, of OCW publication on student enrollment at the publishing institution. Surveys sent to new JHSPH students in 2006-2008 and to all students in 2010-2011 indicate that JHSPH OCW influenced 6.9% of responding students in their decision to attend JHSPH. When students who were unaware of JHSPH OCW prior to receiving the survey are excluded from the analysis, the percentage of responding students who report being influenced by OCW climbs to at least 11.8%. The percentage could be higher depending on how many students became aware of OCW in the interval between making their enrollment decision and receiving the survey.
After students make their initial institutional enrollment decision, they also face an ongoing series of course enrollment decisions. JHSPH data indicate that OCW publication has a greater influence on these course-level decisions, with some students reporting that their academic planning has been influenced by OCW. Survey results collected in 2010 and 2011 indicate that 31.2% of students who reported being aware of OCW also reported that they used it during their course selection process. When these students were asked if the use of OCW during course selection influenced their decision to enroll in a particular course, 69.9% responded in the affirmative. Furthermore, 35.1% of students who used OCW during course selection also reported that it influenced their decision to not enroll in a particular course.
As mentioned before, at OUNL the OpenER site’s influence on the enrollment decisions of students is about 9%, with a total of 24 courses available. This outcome is more or less supported by the results of the survey among the voluntarily registered visitors of the OpenER site. Forty-two percent of the people returning the survey reported they applied for a formal study program or they had bought a course. It is not clear, however, whether this was already the case before visiting the OpenER site.
Note that because most of these surveys investigating influence on school choice were only sent to enrolled students, little information was gathered on site visitors who might have been influenced to not attend the publishing university, a possible outcome of viewing the materials as well. In Table 2, 4% of respondents to OUNL surveys indicate that visiting the OpenER site did influence them to decide not to study more formally, so some evidence of this outcome exists.
Despite indications that OCW publication can provide significant advantages in student recruitment, universities offering online programs have been understandably concerned that the open publication of materials similar to those they are using in for-credit and for-fee distance learning would undercut enrollment. MIT does not offer distance learning programs, but Johns Hopkins Bloomberg School of Public Heath does, and data collected by this program indicates that OCW publication has no apparent negative effect on distance learning enrollment.
Between 2005 and 2011, JHSPH OCW published materials from 24 of its for-credit online distance learning courses. To judge whether addition of materials to OCW influenced for-credit enrollment in the subset of online courses with materials on OCW, JHSPH staff compared enrollment figures from immediately before and immediately after the addition of each of these 24 courses. The average percentage change in enrollment across all 24 courses was +10.68%. There was no enrollment change at all in 3 courses, 11 courses experienced enrollment growth, and 10 courses experienced decline. Overall the change in course enrollment was not statistically significant (Wilcoxon signed-rank test, P > 0.05).
To account for annual fluctuations in course enrollment, JHSPH staff adjusted the post-OCW enrollment for each course by the average enrollment change experienced across all online JHSPH courses during the corresponding year. After making these adjustments, the overall change in course enrollment was still not statistically significant (P > 0.05). Admittedly, this analysis does not account for all of the factors that potentially influence course enrollment in a given academic year. For example, internal factors, such as a special cohort of students entering the school’s Master of Public Health (MPH) program, can affect enrollment dramatically. Likewise, external factors, such as the general economic climate, can also influence enrollment decisions. Despite the inability to control for all of these factors, JHSPH staff conclude on the basis of these data that the publication of OCW materials from JHSPH online courses does not influence enrollment in those courses.
Finally, open universities, where the (online) learning materials are generally designed for self-study by “independent learners,” are at risk of a possible decline in enrollment because potential students could indeed stick to the “informal learning” mode when the courses are offered as OER. This explains their hesitation to convert their full course base to OER. OUNL has just finished an explorative study into the effects on enrollment of different OER scenarios. We can report here that the results for a representative sample of the Dutch population not being OUNL students indicate that the ultimate (100%) OER scenario would generate a little more (although not statistically significant) expected enrollment than the less radical OER scenarios (Janssen, Schuwer, & Mulder, 2012). This outcome may be considered a surprise, but at the same time it is relevant and encouraging because it gives ground to abandon the so far well-understood reserve among open universities with respect to a full OER approach.
Open educational resources efforts are undertaken at institutions for a wide variety of reasons, generally most directly related to fulfilling the core mission of higher education to create and disseminate knowledge, but also to build awareness of and goodwill toward publishing institutions, as well as to foster improvements in teaching and learning effectiveness. Improving student recruitment is a documented benefit but not the only measure of return on institutional investment. That said, even a rough analysis of the return on investment at both JHSPH and OUNL indicates that the benefits likely far outweigh the costs, even when considering only the narrow question of student recruitment. However, MIT, due to its student recruitment approach, does not have a clear method of estimating return on investment from student recruitment.
MIT accepts a limited number of students each year, and so MIT OpenCourseWare does not act to increase the number of students attending and paying tuition. Based on the data presented above, it appears that MIT OpenCourseWare helps to increase the quality of incoming students by making the Institute more competitive against peer institutions, but there is no clear way to measure the monetary impact of such an influence.
The total cost of operating JHSPH OCW is low (managed by 0.5 FTE or half time of a staff member). Very modest recruitment impacts would offset the operating expenses. In 2011-2012 academic year, 993 new students enrolled at JHSPH. If only a couple of students were attracted to JHSPH because of its OCW, JHSPH would have recovered its annual investment.
As noted above, 9% of OUNL students reported being influenced by OpenER in their decision to buy a regular course. In US dollars, this represents $1.36 M in tuition revenues attributable at least in part to the site. This would significantly offset the annual cost of operating OpenER, which on the average over the two years of operation plus additional two years of availability of the courses has been around $201,000. It is, however, not possible to be firm on this because it is not known how many of these students would have enrolled also in the absence of OpenER and how many potential students decided not to buy a regular course because of the free offer through OpenER.
OCW can be an important tool to increase or widen participation in formal higher education, especially in supporting a return to formal education by lifelong learners.
Learners access openly available educational resources for informal study in very large numbers, indicating a widespread and pervasive need for continued learning opportunities. A subset of this use is in the context of learners contemplating a return to formal education, and, in this case, OCW appears to serve a number of important purposes, both inspirational and functional. OCW appears to both inspire interest in continued education and help learners develop confidence in their ability to succeed by providing in-depth access to the learning materials they will encounter during formal study. OCW also appears to be a significant aid in selecting programs and courses, preparing for entrance exams, and getting a “head start” on studies.
These effects are significant at the systemic level and should be considered by governments, NGO’s, and foundations seeking to increase higher education participation among broad populations. Especially among populations with many first-generation college students, informal exposure to higher education materials published as OCW may be a useful way for potential students to understand what it means to study at the college level. For situations where increased levels of participation in higher education are explicit goals, such as is the case in the Netherlands (and in many countries), OCW may serve as an important and cost-effective entry to formal higher education.
OCW can be a significant tool for improving student recruitment.
OCW projects demonstrate fairly consistent influence on student school choice of between 10-30% of students who have viewed the OCW sites. These figures indicate a clear opportunity for schools to improve recruitment through the open publication of learning materials. Because an aspect of OCW’s influence appears to be communication of the culture of the publishing school, it’s arguable that even if OCW influences potential students to not attend a given school, such an outcome may be in the best interests of both the student and the school by contributing to a better match between the two.
It may be the case that MIT, JHSPH, and OUNL enjoy first mover advantages in publishing educational resources openly and student recruitment advantages may become less pronounced as more schools publish open educational resources; conversely, if OER publication is a widely embraced practice, schools choosing not to openly publish their materials may generate suspicion among prospective students. These are issues for further study. Issues of student retention and success were not addressed in the data collection and are also an area for further study.
OCW does not appear to negatively affect enrollment of online programs from which materials have been openly shared.
While this is an area for future study, preliminary indications from the JHSPH experience indicate no discernable impact of OCW publication on online course enrollment when the same materials are used in both cases. This may be an outcome specific to the licensure requirements for practice in a field such as public health and may not generalize into other domains where certification is relatively less critical. However, the recently published OUNL study similarly gives support to this third conclusion. As more online programs experiment with OER and OCW approaches, a clearer picture of the relationships should emerge.
Abelson, H. (2007, May) The creation of OpenCourseWare at MIT. Journal of Science Education and Technology.
Butcher, N., & Hoosen, S. (2012). Exploring the business case for OER. Commonwealth of Learning
Lane, A. (2012). Case studies on institutional open approaches: The Open University. JISC. Retrieved from http://oro.open.ac.uk/id/eprint/33245.
Carson, S. (2005). Program evaluation findings. MIT OpenCourseWare. Retrieved from http://ocw.mit.edu/about/site-statistics/
Carson, S. (2003). Program evaluation. MIT OpenCourseWare (Unpublished manuscript).
Janssen, B., Schuwer, R., & Mulder, F. (2012). A business model approach for OER in open universities. In Conference Proceedings Cambridge 2012: Innovation and Impact - Openly Collaborating to Enhance Education. Cambridge, UK.
Mulder, F. (2006). The advancement of lifelong learning through open educational resources in an open and flexible (self)learning context. In Presentations Dies Natalis 21 september 2006. Open Universiteit Nederland, Heerlen, The Netherlands. Retrieved from http://www.ou.nl/Docs/dies%20natalis/Redes_Dies_2006.pdf
Schuwer, R., & Mulder, F. (2009). OpenER, a Dutch initiative in open educational resources. Open Learning, 24(1), 67–76.
Wiley, D., Hilton III, J.L., Ellington, S., & Hall, T. (2012). A preliminary examination of the cost savings and learning impacts of using open textbooks in middle and high school science classes. The International Review of Research in Open and Distance Learning, 13(3).
1 In this paper we will use one reference term only (OCW), just for convenience and being fully aware of the definition differences between OCW and OER. Only in the case of possible misunderstanding we refer specifically to OER.
2 One could say that this extension moves OCW into the OER domain.
3 The annual Spinoza Premium is the most prestigious research award in The Netherlands: 2.5 M€.