Assessing the Savings from Open Educational Resources on Student Academic Goals

Our study found that most students considered OER to be as good or better in terms of quality and engagement as traditional texts, while also allowing them to put saved funds toward their educational pursuits. As rising costs in higher education affect current and potential students, faculty and students are looking for ways to cut costs where possible. Open educational resources (OER) are a viable option to replace expensive traditional textbooks without sacrificing quality. This article presents the results of a study conducted with students at a Virginia community college who took courses that used OER. At the end of the semester, students were asked to rate their perceptions of the OER quality and their level of engagement with OER as compared to traditional textbooks. Results indicate that a majority of students found the OER to be as good as or better than traditional textbooks in both quality and engagement. While similar studies have been conducted, this study also asked students to briefly describe how they used the money saved by not having to purchase a textbook. Many students indicated they used the money to reinvest in their education by paying tuition, purchasing materials for other courses, or taking additional courses; day-to-day expenses and savings were the next most common responses. Further research needs to be conducted to understand the effect these savings and reinvestment have on students’ completion of academic goals.


Résumé de l'article
Our study found that most students considered OER to be as good or better in terms of quality and engagement as traditional texts, while also allowing them to put saved funds toward their educational pursuits. As rising costs in higher education affect current and potential students, faculty and students are looking for ways to cut costs where possible. Open educational resources (OER) are a viable option to replace expensive traditional textbooks without sacrificing quality. This article presents the results of a study conducted with students at a Virginia community college who took courses that used OER. At the end of the semester, students were asked to rate their perceptions of the OER quality and their level of engagement with OER as compared to traditional textbooks. Results indicate that a majority of students found the OER to be as good as or better than traditional textbooks in both quality and engagement. While similar studies have been conducted, this study also asked students to briefly describe how they used the money saved by not having to purchase a textbook. Many students indicated they used the money to reinvest in their education by paying tuition, purchasing materials for other courses, or taking additional courses; day-to-day expenses and savings were the next most common responses. Further research needs to be conducted to understand the effect these savings and reinvestment have on students' completion of academic goals.

Introduction
Costs for students in higher education are rising and moving college out of reach for more and more prospective students (Mitchell & Leachman, 2015). Chen (2016) also found that freshman enrollment rates are inversely proportional to the costs of university tuition and fees. Costs of higher education include items such as tuition and fees, room and board, books and supplies, and transportation. While tuition and living costs constitute a sizable portion of the expenses, faculty and students have limited ability to influence these expenditures. One area where faculty and students can affect higher education prices is textbook costs.
Many professors assign at least one required textbook while some assign two or more textbooks with the intent to only use portions of each (Wiley & Green, 2012). Due to copyright laws, which often prevent professors from selecting and remixing relevant content in traditional textbooks, instructors may require students to purchase multiple resources. A recent report estimates that students should budget between $1200 to $1350 in an academic year for textbooks and supplies (The College Board, 2016). For some lowertuition institutions, this amount can be nearly 40% of the price of tuition (Senack, 2015). Open educational resources (OER) are one alternative to traditional, high-cost textbooks. The William and Flora Hewlett Foundation define OER as: teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge.
(Hewlett Foundation, n.d.) These types of materials are low-cost resources that can be adapted to meet instructor and student needs.
When using OER, professors have legal permission to revise, adapt, and continually improve materials specific to their course (Wiley & Green, 2012). Other researchers have found OER to be at least as effective as traditional materials when measured by student performance and by soliciting student feedback (Allen, Guzman-Alvarez, Molinaro, & Larsen, 2015;Bowen, Chingos, Lack, & Nygren, 2014). For open materials to have any effect on students, positive or negative, a professor must first choose to adopt OER.
Faculty and student perception of such materials often influences their decision to adopt and continue using OER. As more open educational resources are created and adopted, the perception of OER has the potential to change significantly. In order to further understand, and therefore improve student perceptions, more research must be conducted. The purpose of this study is to better understand student perceptions of the quality of OER, the effect of OER on student engagement, and how students repurpose money saved by using OER.

Literature Review
Although the term open educational resources was coined by UNESCO in 2002, and is gaining popularity and becoming more widespread, awareness of OER is still relatively low (Allen & Seaman, 2014). In a survey involving over 2,000 faculty respondents, Allen and Seaman (2014) found that that only 20% of all participants considered themselves very aware or aware of OER while 65% considered themselves 128 unaware. Similarly, a 2012 Florida study found that over 72% (10,007 students) of participating higher education students had never heard of open textbooks (Florida Virtual Campus, 2012). Programs in the United States such as Tidewater Community College, Mercy College, Salt Lake Community College, among others, have helped increase awareness and provided support for faculty members to adopt OER in their courses (Fischer, Robinson, Hilton, & Wiley, 2015;Pawlyshyn, Braddlee, Casper, & Miller, 2013). Studies have been conducted to gauge faculty and student perceptions of the open resources, the pedagogical benefits afforded by OER, and student savings due to foregoing the purchase of a traditional textbook. We have referenced and separated some of these studies into the following categories: perception, effects on pedagogy, and cost.

Perception
While cost can greatly influence faculty and student attitudes towards OER, their perception of the quality of open resources also plays a role in their willingness to use or adopt OER. Jhangiani, Pitt, Hendricks, Key, and Lalonde (2016) explored faculty perception of OER at British Columbia post-secondary institutions through an online survey disseminated using email and social media. More than half of respondents rated OER as good as or better than proprietary materials; those who had adopted OER rated it significantly higher than those who had not (Jhangiani et al., 2016).
Faculty who are more prepared tend to have a better experience with OER. In a survey of 128 educators with awareness of OER, Kelly (2014) found that faculty who were more comfortable with technology in general considered themselves more confident in using OER. However, when respondents found OER difficult to use, they perceived it as less useful (Kelly, 2014). The author suggested that in order to be widely adopted, OER must be both easy to locate and easy to use. Even if the content is adequate, these concerns may hinder it from adoption.
While faculty make most decisions about textbooks, student perceptions also play an important role. In a previous study, Bliss, Hilton, Wiley, and Thanos (2013a) explored the perceptions of open textbooks from community college students and faculty members. Of the 132 students who responded to the survey, 56% (67 students) thought the open textbook was same as the quality of texts in other courses, 41% (49 students) considered it better than the quality of texts in other courses, and only 3% (4 students) felt the open textbook was worse than the quality of texts in other courses. Measuring perception of quality based on student preparedness, faculty members found a majority of students were equally prepared (60%) or more prepared (30%) than they were with traditional materials. Roughly 75% of students surveyed in that same study responded with completely positive comments about the open text, 14% were a mixture of positive and negative comments, and 10% were completely negative (Bliss et al., 2013a). Lindshield and Adhikari (2013) found similar results in their study, which took place over the course of three semesters at a large Midwestern, public institution of higher education. A survey was administered to 198 students, comprised of online and campus students, asking them to rate their experience with an open textbook, which they referred to as a flexbook (Lindshield & Adhikari, 2013). Both online and campus students preferred the flexbook to a traditional text and rated the quality of the flexbook as high (Lindshield & Adhikari, 2013).

Effects on Pedagogy
Improved pedagogy and student outcomes can be an important issue for faculty decision making. Beyond the effect of perceived quality of the text, Petrides, Jimes, Middleton-Detzner, Walling, and Weiss (2011) found that OER had a positive impact on both teaching and learning. Using the open textbook increased student interactivity with the material; students reported this increased interactivity positively impacted their study habits. Rowell (2015) surveyed 80 students in various community college courses that used an open textbook and discovered that students rated their motivation to learn, which may be correlated with academic achievement, highest among the varying metrics. Faculty members also reported being inspired by the advantages afforded by OER, such as customization and the ability to tailor course content to their own students (Jhangiani et al., 2016;Pawlyshyn et al., 2013).

Cost
The potential for reducing the costs associated with education through the adoption and use of OER has been a focus for research conducted regarding open resources (Bowen et al., 2014;Hilton, Robinson, Wiley, & Ackerman, 2014;Florida Virtual Campus, 2012;Murphy, 2013;Wiley, Hilton, Ellington, & Hall, 2012). Petrides et al. (2011) surveyed 31 faculty members and 45 students regarding their perception of the open statistics textbook, Collaborative Statistics. Both faculty members and students cited reduced cost as the most important factor for their interest in continuing to use OER or adopt OER in the future. One faculty member cited expensive textbooks as "an obstacle to college access" (Petrides et al., 2011, p. 43). Hilton et al. (2014), in a study with seven community colleges across the nation, found the average textbook cost to be $90.61, which would result in a full-time student spending over $900 on textbooks per year. To cope with these high costs, some students forego purchasing the required text, drop or withdraw from a course, or take fewer courses (Florida Virtual Campus, 2012).
Costs on the institutional level can also be lowered by adopting OER. Bowen et al. (2014) created a rough cost simulation at three university campuses for adopting hybrid courses using OER as a way to decrease costs. They estimated the costs of personnel and facilities to implement a hybrid course and compared them to the current costs of a traditional course. These universities can expect a 19-57% savings on these courses depending on the sort of hybrid model adopted. While the authors stipulate that there are many further variables to consider in terms of costs, they feel that that these findings suggest that adopting OER and hybrid learning can cut costs while maintaining, or even improving, student outcomes, which is a concern for many educators looking to adopt OER.
Although it is beyond the scope of this study, the efficacy of open materials is of importance to many faculty members and students. For those interested in this topic, Hilton (2016) provides a comprehensive review of research studies on about efficacy and perceptions as they pertain to OER and college textbook choices.
The most cited reason that students mention for preferring OER over traditional resources is cost savings.
However, what they do with those savings is unknown. While previous studies have addressed valuable aspects of OER, this study aims to add an understanding of how students use the money they save from courses that opt to use OER instead of traditional textbooks. Currently, there is limited literature regarding this issue. Understanding how students repurpose saved funds from using OER materials can help us identify long-term implications of OER usage beyond the scope of a semester-long course. Most courses used open textbooks from OpenStax, which can be accessed in a digital format or, if students are willing to pay the cost of printing, requested in a professionally printed format. Literature faculty members provided students with links to online resources and PDF files of works available in the public domain while English Composition faculty used resources from free sites in addition to other materials.
Faculty members in the biology department also developed their own materials that were used with the OpenStax textbooks. A repository of OER materials was created for use among participating faculty.
In late April 2015, students (N=425) were asked to respond to a survey regarding their perception of the quality of the OER materials in their course and the level of engagement they experienced with the OER materials compared to a traditional textbook. Students gave a rating from 1 (poor quality/not engaging) to 5 (excellent quality/very engaging). After each question, respondents had the opportunity to elaborate on their response with short answers. The last item on the survey asked students to explain what they did with the money they saved by not having to purchase a textbook.
Prior to emailing students with a link to the survey, faculty members in face-to-face sections conducted informal discussions about the use of OER. The survey was sent to a total of 425 students (each of the 17 sections contained 25 students), and 206 completed the survey. Responses were recorded through SurveyMonkey. We analyzed the numerical data using descriptive statistics. All 206 students who completed the survey also included free responses for each of the three questions. Free responses from students were analyzed and coded for patterns and themes exploring why students may have rated OER materials as high or low.
Each student comment received at least one code but, depending on statements made within the comment, could be assigned multiple codes. For example, in response to the question of how money saved from using OER materials was repurposed, one student replied, "I [was] able to use that money for gas, groceries, and other things I need in my day-to-day life." This response received codes of 1 (Day-to-day Expenses), 6 131 (Groceries/Food), and 7 (Gas/Transportation). Comments could receive multiple codes but could not receive any specific code more than once. For instance, a single student comment that mentioned purchasing food or groceries twice would only receive code 6 once.

Data Analysis
Based on student responses, our analysis will focus on three specific areas for which data was collected: student perception of quality, level of engagement, and how students spent money saved from OER usage.

Perception of Quality
Students were asked the following question: "On a scale of 1 (poor) to 5 (excellent), how would you rate the quality of the OER material versus a textbook?" A majority (54.9%, 113 students) of the participants rated the open material as excellent (Figure 1). Roughly 39% (81 students) considered the quality as good as a traditional text or slightly favored the quality of the OER material. Less than 6% (12 students) considered the quality of the OER material to be less than that of a traditional textbook. These data are consistent with previous studies (Bliss, Robinson, Hilton, & Wiley., 2013b;Lindshield & Adhikari, 2013;Petrides et al., 2011), which found students overwhelmingly considered OER materials to be as good as or better than traditional materials.  Of the 48 responses that indicated improved content, 11 students (23%) considered the OER materials more engaging and interactive. Multiple responses mentioned how the OER materials were easier to use and that students preferred them to a traditional text, however, these 19 responses did not specifically state why. The ability to adapt resources to a specific course played a role in students' positive perception of open materials content as shown by the 17 students (35.4%) who included this in their comments. One student said that "OER material seemed to provide for a class in which the professor was allowed flexibility to teach something they could be enthusiastic about, creating a more engaging environment." Students appreciated content that was tailored specifically to the course. Traditional textbooks, which are protected by copyright laws and unable to be revised, may include, in one student's view, "a lot of information that may or may not be necessary to the exact line of learning that we are doing." This student continued that when using OER materials "the information that we have been given to use for our studies is more directly related to the assignment we are working on." Another student expressed a similar sentiment that "not only does [OER material] save money and increase accessibility, but it ensures that the readings are both precisely what we need, and current to the day." The flexibility to remix and revise content means educators can cull from a more abundant source of material. Students recognized the benefit this provides and believed "there is no way a single text book could have covered all that we covered using all the sources that [the professor] used." Fifteen students specifically mentioned the variety of open resources their professor incorporated as a beneficial aspect of using OER materials.
In addition to the improved content, reduced costs influenced student perceptions of the quality of OER materials used in their courses. One of the 47 students who included the impact of price in their comments stated, "I learned just as much with the OER material as I would with a textbook and the textbook cost way too much." For one student being able to save money from OER materials "impacted [the] decision on taking the course" and "assisted with financial difficulties." The "best part" according to another student was "[the textbook was] an expense I no longer need to worry about, and that alone relieves much unnecessary stress." Easy access to the course information also affected student perceptions of OER materials. Because the sources were available online, 16 (42.1%) of the 38 students who mentioned accessibility included this as a positive aspect of using open materials in their course. One respondent thought "OER materials [were] easier to carry around" and "easily accessible." The issue of portability was included in 15 (39.5%) comments regarding accessibility. Another respondent said, "OER did not require me to lug a heavy textbook around campus and the readings were just as informative and much cheaper." Even with the added benefit of reduced costs and greater accessibility, some students felt the quality of the material was more important than saving money. Multiple students expressed that it was "easier to understand OER information as opposed to a regular textbook" because the open material was more direct, better organized, and more specific to their courses. Table 1 illustrates the frequency of each of the six overarching themes mentioned in student free responses regarding the perception of quality. The third column does not total 100% because many comments received multiple codes. comments that also included positive feedback, such as "I enjoyed using the OER material, however sometimes it was difficult to read portions of it and I had to print a lot of it out" and "It clearly explains the material, but requiring internet access can be challenging." Issues concerning the digital nature of OER conflate "OER" and "digital," a topic that we address further in the "Discussion and Limitations" section below. Needing internet to access the material was a downside cited by 14 students (31.8% of the 44 students). The next most common theme (29.5%, 13 students) was students having a preference for paper to more easily flip between pages, highlight material, and mark pages. This did not inherently mean these students perceived the quality of OER to be lesser than that of traditional texts. As one student shared "the OER material is absolutely beneficial and helpful, actually having a text book makes it easier to go back and reference rather than having to scroll through previous weeks assignments or through paper we have to print with the information." Another group of 8 students (18.2%) shared that they do not like reading on a computer screen, which affected their view of OER materials.

Level of Engagement
Students were asked to rate their level engagement with the OER materials as compared to a traditional textbook on a scale from 1 (not engaging) to 5 (very engaging) (Figure 2). Most students (74.2%, 153 students) found the OER textbook more engaging than a traditional textbook. Only a handful of students (10.2%, 21 students) found it not as engaging in comparison.

Figure 2. Student engagement with OER materials (n=206).
In addition to providing a numerical rating, students were asked to comment on their engagement in a free response question. All 206 students who completed the survey provided feedback in the free response. Of these comments, 130 (63.1%) included positive statements, 18 (8.7%) were neutral, 32 (15.5%) included negative statements, and 35 (17.0%) did not provide enough information to respond to the question, such as "I use it a lot" or "I was able to complete assignments." Students who found traditional textbooks more engaging than OER materials shared a number of ways that OER can improve. Quality was the most common culprit in terms of visual cleanliness, organization, and materials despite generally positive reflections on quality in the above question.
Those who found OER more engaging than traditional textbooks reported a variety of reasons including time savings, flexibility, and interaction. Almost half (43.8%, 57 students) of the 130 students who provided positive comments mentioned being more engaged. Respondents discussed a perceived improvement in available content, especially in terms of resources and applicability. Students appreciated how OER allowed their instructors to provide "a lot of resources...in order to understand the material." Another elaborated, "I think you are more likely to get into the material more when you are looking at different things (websites, articles, etc.) than just looking at a text book." While the ability to point to different materials is not unique to OER, when materials are openly licensed and able to be freely used by others, it increases the pool of resources professors can incorporate in their courses and adapt accordingly.
Twenty-eight students (21.5% of students who gave positive comments) included that they valued content tailored to their course and the variety of sources they were able to choose from. As one student said, "All the OER material was specifically aimed at the assignments we had that week. It also allowed the instructor to find material relating to our assignment, instead of finding assignments to relate to material." Another felt more control over the course because of OER.

135
It is very engaging because my professor has an ability to pick readings due to taste and not the limitations of a textbook. This leaves more options for me, as a student, to give my input and hopefully study something of my own interest.
Students also enjoyed the flexibility of organizing the material that OER provided. "OER material is VERY engaging because it provides the ability for students to learn material that interests them. It is material that stimulates one's mind, not useless linear banter." A theme amongst positive comments was that of OER material being easier to follow due to the organization of information. One literature student remarked: I really enjoy being able to study literature based on the timeless issues being addressed, instead of just chronologically. Every literature textbook I've previously read goes by decades, lumping authors together according to when they were alive, not the issues they wrote about. This way is much more engaging, in that there is more of a purpose for reading the literature presented.
While this student focuses on an instructional design issue, it does highlight a benefit of OER. Because teachers had the legal and technical ability to make revisions to the text they were able to rework it in a manner that was appreciated by many of their students. Approximately 17% (22) of students included the influence of the OER materials' digital capabilities on their perceived engagement. Being able to make digital annotations, zoom in and out with text, and search within the text for keywords, helped students interact more with the material.
Of the 18 students who indicated they were just as engaged with OER materials as with traditional textbooks, many comments echoed the same sentiment, such as "It was about the same level as a textbook" or "I think that it is just as engaging as a textbook because we still learn and do the same things as someone would with the textbook. We just get to do it for free." Among the negative comments, 14 students (43.8%) said they preferred printed materials to online materials; this was by far the most common response. Again, this conflates "digital" with OER, as some commercial textbooks are available in digital format, and open textbooks can be printed. The second most common response, given by 8 students (25%), pertained to the difficulty they experience reading on a computer. Six students believed the organization and format of the OER material made it less engaging than a traditional textbook. Other responses from respondents included getting distracted from being online while accessing the open materials and being impeded by the need for internet to access OER. Overall, however, we found that students perceived OER materials to be more engaging than traditional materials.  Sources)," and "Leisure," were used, but these categories were relatively small (6.3%, 2.9%, and 0.97%, respectively).
For students who responded that they invested the money back into their education, 57 of them (65.5%) used the funds to purchase materials for other classes. The majority (54.4%) of these students reported that they applied the money towards purchasing textbooks for other classes. Some of the other school materials mentioned were calculators, laptops, and flash cards, but these categories contained less than three students each. One student was "able to afford the textbooks I needed for other classes, as well as was able to take the money I would have spent on a textbook for this class and save it to pay for another class next semester." The remaining 30 students (34.5%) who reinvested in their education did so in the form of paying current tuition (11 students), taking an extra class that semester (9 students), and using it in their other classes (7 students). Students did not specify how they used it in their other class, whether they purchased materials or if they meant to imply it was additional course they would not have been able to take otherwise. One student stated, "I was able to take another class and take a step closer to achieving my dreams and my degree." A student who self-identified as a single mother said, "I am trying to better myself...I was able to put the funds to another class...to take three classes instead of two." Day-to-day expenses was the next highest cited area. Many of the 63 responses in this category included various elements, resulting in these comments receiving multiple codes. More than half (51%) of these responses clustered around the necessities of buying groceries and paying bills. One student reported, "It gave me more money for bills and food, my basic needs." Nearly one-third of the 63 comments mentioned applying the money to transportation needs through purchasing gas (20.6%, 13 students) and car Having saved money "helped relieve some financial stress" for a respondent and allowed them to "live more comfortabl[y], thus focusing more on school instead of working hard to make the money for the textbooks." Various students expressed the stress caused by the burden of textbooks costs was removed when using OER materials. "The money not spent on textbook books actually benefited me and my overall stress...The money helped me fix my [flat] tire, and also allowed me to buy sets of ink cartridges for all of my assignments for the semester." A total of 42 free responses said that they saved the money instead of spending it during the semester. Of that number, 18 students (42.9%) were saving the money to pay for future classes and tuition, 6 students (14.3%) kept it in their savings account, and 3 students (7.1%) planned to use it when they transfer to a fouryear university. Approximately 29% of the comments stated the money was saved but did not provide any additional information. The remaining three indicated "savings" by being able to take out a smaller loan.

Discussion and Limitations
Throughout this study, a number of students commented on issues that may not be directly connected to OER, but that they perceived as relevant to their perception of quality and experience of engagement with OER materials. These other factors include the preference for printed resources, the accessibility and portability of an online textbook, interactive online features of the text, multimedia materials such as videos, and the role of the course professor. While we recognize that all of these items can influence student perceptions, they are not specific to using OER materials. For example, a traditional textbook, which does not permit instructors to adapt the resource, can be purchased in an online format providing the same digital features and making it just as accessible and portable as online OER materials.
A limitation of the study is the dependence upon student self-reports, which may not be standardized across all students. For example, a value of 4 on a scale from 1 to 5 can be interpreted differently among participants. Another possible reason is that students may have written responses according to what they believed was the "correct" answer, even though all data was collected anonymously. Self-report data about how students spent the money they saved from OER usage may also be subject to inaccuracies due to the amount of time that passed during the course of the semester. Students had a general idea of how they used the funds, but were not completely sure in some cases.Another limitation is the sample size used in the study. We analyzed the OER usage at one community college across a limited set of OER materials. Due to this, these findings cannot necessarily be extrapolated to other populations.

Conclusion
As the costs of higher education continue to rise, open educational resources are a possible solution to ease the financial burden students face. These resources have been found, in previous studies, to be just as effective as traditional materials used in the classroom. In our research we found that most students who 138 participated in the study believed the quality of OER was better than that of traditional textbooks. They also reported feeling more engaged with open materials because of the interactivity these materials facilitated. Although increased engagement is not a defining attribute of OER, and may be due to the instructional design behind the open materials and course layout, OER provide more flexibility and adaptability for professors to influence the materials and restructure their courses. Further research is needed to more carefully tease apart the potentially conflating variables. While many previous studies have been conducted about student perceptions of quality, this study aimed to understand the impact of financial savings from OER on students. Our main finding is that a majority of students reinvested that money in their education or had plans to do so in the future. Students reported taking more classes in a given semester with the money they did not have to put towards a traditional textbook. This means OER could help students achieve their educational goals in less time by freeing up funds to pay for additional classes.
Further research needs to be done in this area, but the potential of OER to aid students in the quality of their learning and the quantity of time it takes to reach their educational goals is promising.