A Preliminary Examination of the Cost Savings and Learning Impacts of Using Open Textbooks in Middle and High School Science Classes

Proponents of open educational resources claim that significant cost savings are possible when open textbooks displace traditional textbooks in the classroom. Over a period of two years, we worked with 20 middle and high school science teachers (collectively teaching approximately 3,900 students) who adopted open textbooks to understand the process and determine the overall cost of such an adoption. The teachers deployed open textbooks in multiple ways. Some of these methods cost more than traditional textbooks; however, we did identify and implement a successful model of open textbook adoption that reduces costs by over 50% compared to the cost of adopting traditional textbooks. In addition, we examined the standardized test scores of students using the open textbooks and found no apparent differences in the results of students who used open textbooks compared with previous years when the same teachers’ students used traditional textbooks. However, given the limited sample of participating teachers, further investigation is needed.

Proponents of open educational resources claim that significant cost savings are possible when open textbooks displace traditional textbooks in the classroom. Over a period of two years, we worked with 20 middle and high school science teachers (collectively teaching approximately 3,900 students) who adopted open textbooks to understand the process and determine the overall cost of such an adoption. The teachers deployed open textbooks in multiple ways. Some of these methods cost more than traditional textbooks; however, we did identify and implement a successful model of open textbook adoption that reduces costs by over 50% compared to the cost of adopting traditional textbooks. In addition, we examined the standardized test scores of students using the open textbooks and found no apparent differences in the results of students who used open textbooks compared with previous years when the same teachers' students used traditional textbooks. However, given the limited sample of participating teachers, further investigation is needed.

Introduction
Public education budgets continue to shrink while the public's expectations for the performance of its educational institutes continue to increase. This tension places many school districts in a difficult position as they attempt to find ways to do more with less (Odden et al., 2007). Over the last two decades, textbooks and other educational resources have repeatedly undergone scrutiny in an effort to determine whether the amount of learning they facilitate justifies their costs (Card & Krueger, 1996;Chaudhary, 2009;Hanushek, 2002).
Open education resources (OER), educational materials that are available at no cost and 263 under open copyright licenses or in the public domain, offer an alternative to traditional textbooks and resources. According to the Organization for Economic Co-operation and Development (OECD), "the definition of OER currently most often used is 'digitized materials offered freely and openly for educators, students, and self-learners to use and reuse for teaching, learning, and research'" (OECD, 2007, p.10). In addition to potentially saving school and district resources, OER can also be adapted to individual circumstances, printed on demand or used in digital formats, and leveraged to enable new pedagogical practices.

Background and Literature
Despite over a decade of research, development, foundation funding, and other efforts, open educational resources (OER) have yet to show a discernable impact on public education in the United States. Open education resources are often used in distance education programs in a supplementary fashion alongside traditionally copyrighted materials (Butcher & Wilson-Strydom, 2008). However, open textbooks can also be used in classrooms to replace expensive, proprietary textbooks. Several teachers in higher-education settings have examined the possibility of substituting open textbooks for proprietary textbooks (e.g., Baker, 2008;Baker, Thierstein, Fletcher, Kaur, & Emmons, 2009), but K-12 education has been slower to respond to the open textbook opportunity. This delay is partly attributable to textbook selection processes that are typically slow and bureaucratic (Armstrong & Bray, 1986;Watt, 2009;Frydenberg, Matkin, & Center, 2007).
As public education budgets have tightened, open educational resources have become part of the educational conversation as a potential source of cost savings (Odden et al., 2007).
However, in addition to bureaucratic adoption hurdles, educators lack appropriate research data to support a decision to use or reject open textbooks. But a small and growing body of research about OER effectiveness does exist. For example, OER allow teachers and students to remix content in locally meaningful ways (D'Antoni, 2009), to share a variety of types of learning resources (Downes, 2007), and to enable the best resources for teaching a specific topic to be more easily found (Gurrell & Wiley, 2008). OER have received considerable attention in higher education (Baker, 2005;Koch, 2006), and researchers are examining the question of how students are receiving open textbooks and how these textbooks affect student learning (Frith, 2009 Curriculum materials are an important part of student learning and represent a significant, recurring cost to public schools (Ansari, 2004). In the United States, core high-school science textbooks (without supplemental materials) from commercial publishers available on Amazon.com cost $80-$120 per copy, and teacher editions typically cost over $100 per copy. More problematically, the economic difficulties presented by the rising cost of textbooks can translate directly into pedagogical challenges. In the best cases, where schools/ districts can afford to provide students with up-to-date textbooks, these materials must be preserved and reused for several years. Consequently, this preservation mindset translates into prohibitions on student highlighting or note taking in textbooks, which makes studying cumbersome and difficult. This is unfortunate because annotating textbooks has been shown to be an effective learning strategy (Simpson & Nist, 1990;Lebow, Lick, & Hartman, 2004;Wolfe & Neuwirth, 2001;Annis & Davis, 1978;Fowler & Baker, 1974). In other cases, students are forced to share books or go without them because their school or district cannot afford to purchase textbooks in a difficult budget year (Orfield & Lee, 2005). Clearly, textbook sharing arrangements prevent many students from being able to take books home for after-school study.

Context of the Present Study
As stated, this study examines issues of both the cost effectiveness and the educational effectiveness of open textbooks compared to traditional textbooks. Figure 1 shows possible outcomes of this study. In the present study, seven middle-or high-school science teachers in the state of Utah replaced their commercial textbooks with open textbooks for one academic year. The teachers were instructed to continue to supplement the text with online and additional materials and activities in ways consistent with their previous classroom practices.
The textbooks used in the study are published by the CK-12 Foundation, the largest publisher of K-12 open textbooks in the United States. The CK-12 authoring model uses classroom teachers to do the initial writing, with subsequent review and refinement by subject- Research Articles June 2012 265 matter experts (e.g., university faculty with PhDs in the content areas).
In this article, we report three key findings from the study.  Table 1). Once teachers had modified and adapted the textbooks according to their needs, CK-12 personnel reviewed the textbooks for clarity and accuracy. (Because CK-12 provides these and other services freely to everyone, we do not factor these costs into our comparison.) Each teacher then chose the way he/she wanted the textbook to be bound and distributed to his/her students. Of the seven teachers, three (Teachers B, C, and D) chose a loose-leaf option (printed on three-hole-punched paper and assembled in a three-ring binder), two (Teachers E and F) chose to print with a perfect-bind option (a print-on-demand, paperback format), and two chose to go completely digital with no printing. Cost Results: 2010-2011 The cost data described above are summarized and juxtaposed with relevant student data in Table 2. As demonstrated in Table 2, the average cost of using open textbooks-across a range of levels of teacher adaptation effort, book lengths, students served, and final format-was higher than the cost of simply adopting a traditional textbook. However, it is also clear from Table   2 that some of the specific models of using open textbooks were less expensive than simply adopting a traditional textbook. As we analyzed these differences, we began to understand the forces driving costs down on some of the textbooks.
Few of the teachers in the study invested significant adaptation efforts to prepare their textbooks before the school year. Several teachers decided that they would just begin the school year with the complete CK-12 textbook and mark sections for deletion throughout the school year. This approach resulted in books with large page counts and relatively high amounts of irrelevant content. This contributed to the higher costs of the most expensive open textbooks in the study.
Once everything was printed, we were surprised to find that having the textbooks printed in a perfect-bound paperback format was cheaper than printing loose-leaf, three-holepunched pages and putting them in three-ring binders. Many of the teachers believed that the loose-leaf approach would be less expensive when it was, in fact, much more expensive.
Some teachers who used the less expensive print-on-demand approach wanted to print their books in several parts. Instead of a single 500-page book, for example, they printed five separate 100-page books. This tactic proved to be extremely expensive as the "setup" cost of running the print job was incurred five times rather than once.
Finally, we did not print large numbers of any of the books (relative to publisher standards for large orders). Because significant printing discounts are tied to making very large orders (over 1,200 books for the best discounts from Lulu, a typical print-on-demand vendor), our smaller orders contributed to higher per-book costs.
To summarize, the easiest way to spend more money on open textbooks than on traditional textbooks is to simultaneously • fail to exercise any of the adaptation/revision rights provided by open textbooks, adopting longer books that contain unnecessary information; • print these longer books on loose-leaf paper and put them in three-ring binders; • disaggregate these longer books into multiple smaller books; and • print small numbers of the books (100-200 copies).

Methodology: 2011-2012
Based on the principles just described, we did things differently for the 2011-2012 school year. Teachers met together early and agreed to make careful revisions based on material they would actually need. In addition, we learned from Teacher E that when one teacher adopts another teacher's book, the modification costs per adopting student significantly decrease. Thus, we brought teachers together from just one district and invited teachers in that district to make one textbook that all would use. This cut down on the number of ver- . While more teachers participated in the pilot, fewer teachers modified the books, thus amortizing the modification costs across many adopting students. Table 3 shows the costs of modifying textbooks this year. drastically decreased the cost of the books. Table 4 presents the new year's data in the same format as Table 2 above. As per Table 4, the average annual cost of a traditional textbook was $11.43. This figure represents the cost of the textbook amortized over the seven-year replacement cycle. The average cost of an open textbook was $5.14. This represents a savings of $6.29 per student per course per year. If a district of 10,000 students adopted open textbooks for its four science courses (earth science, biology, chemistry, and physics) over a seven-year adoption period, the savings would amount to $62,900 × 4 courses × 7 years, or $1,761,200. • print these shorter books as black-and-white paperback books through a print-on-demand vendor such as Lulu.com; • print each book as a single book rather than disaggregating it into smaller pieces; and • print relatively large numbers of the books (ideally 1,000 copies or more).

Methodology: Examining Student Test Scores
Having demonstrated that open textbooks can decrease costs, a critically important question remains: What is the impact of these inexpensive open textbooks on student learning?
We examined this question by using data from Utah's annual standardized tests, known as the Criterion-Referenced Tests (CRT). Specifically, we compared the CRT scores of students whose teachers used the open textbooks to the CRT scores of those same teachers' students in previous years.
While our original design called for CRT scores for 2011 and the three previous years for every teacher, these data were not available. Some of the teachers in the study were new (no data beyond 2010) and some had changed schools, making it difficult to get data beyond 2010. While these limitations were real, we did receive the 2011 and 2010 CRT scores for each teacher, as well as the 2009 scores for four of the participating teachers. These CRT scores, listed in Table 5, represent the percentage of students in each class who demonstrated proficiency on the exam. Given so little data, we can only present a descriptive analysis. However, given the lack of research and data in the space overall, we feel that even a simple analysis is worthwhile. 271 central tendency of this small data set. Table 6 shows the change scores.   272 of these are identified above), models do exist that provide significant cost savings. The model described above reduced textbook costs by just under 40% in the first year (when the majority of the adaptation was performed) and by over 50% in subsequent years, when compared to the cost of using traditional textbooks. No change in educational outcomes was detected. Using the concept of Figure 1, Figure 2 shows the outcomes of this study. This study does have substantial limitations. First, it was carried out in the United States, where easy access to affordable print-on-demand services exists together with other factors that may confound the generalizability of the findings to other countries. In addition, the cost savings we realized only happened when we worked with a single district. We acknowledge these limitations and hope this initial study inspires others in both additional and larger contexts.
Simply substituting open textbooks for traditional textbooks did not appear to have an effect on student test scores. However, we stress the limited nature of the data presented above. Future studies need to expand both the number of teacher and student participants and the sophistication of the consequent analysis.
One area of particular interest is the teachers whose classes saw relatively large (23% and 14%) increases in their CRT scores after adopting open textbooks. One of these teachers said, "The better students write in their textbooks more." If this comment turns out to be representative of a broader phenomenon, we hypothesize that student test scores will improve when professional development is provided to teachers to help them understand the new activities and pedagogies made possible by the open textbooks (e.g., students highlighting and taking notes directly in their books). Even without significant improvements in student learning outcomes, reducing the cost of textbooks by half with no net loss in