Volume 18, Number 4
Lane Fischer1, David Ernst2, and Stacie Mason3
1,3Brigham Young University, 2University of Minnesota
Using data collected from peer reviews for Open Textbook Library titles, this paper explores questions about rating the quality of open textbooks. The five research questions addressed the relationship between textbook and reviewer characteristics and ratings. Although reviewers gave textbooks high ratings generally, reviewers identified differences in quality according to criteria and discipline. Physics and chemistry textbooks earned significantly lower ratings than other textbook types. Ratings were not significantly associated with reviewers' status and experience, but were associated with reviewers' place of residence. We discuss the implications for OER efficacy studies and other research.
Keywords: open educational resources (OER), open textbooks
For several years, the cost of higher education has been increasing at an alarming rate. The U.S. Government Accountability Office (2013) reported that from 2002 to 2012, tuition and fees increased by 89% and new textbook prices increased 82%, while overall consumer prices increased 28%. Each year over 1 million college students graduate with student loan debt averaging about $30,000 per student, which has very serious implications (Cochrane & Reed, 2015). Part of that debt can be attributed to textbook costs. Faced with prohibitive costs, some students are enrolling in fewer courses than they otherwise would, taking on more debt, or choosing not to buy required texts (Hilton, Gaudet, Clark, Robinson, and Wiley, 2013; Senack, 2014).
One promising development helping to address burgeoning college costs is the growing availability of open textbooks. Open textbooks are a type of Open Educational Resource (OER), with an open license that may allow copying, distribution, modification, and in some cases, sale (Bissell, 2009). Most open textbooks are available free online, often with a low-cost print version. Potential benefits of using open textbooks include significant cost savings to students and increased flexibility for instructors (Hilton et al., 2013; Senack, 2015).
Although open textbooks offer some substantial cost-advantages, they represent only a small fraction of the textbooks adopted by faculty members. One reason for this low adoption is that most professors do not know that open textbooks exist, much less where to find them. In a nationally representative survey of college faculty, Allen and Seaman (2014) found that two-thirds of the 2,144 respondents were not aware of OER. Even instructors who are aware may avoid open textbooks because they question the quality of freely available resources, especially when such sources aren't connected to a reputable publishing house, institution, or author (Pitt, 2015). College instructors who have their students' best interest in mind are not willing to replace a high-quality, high-cost textbook with a low-quality, low-cost alternative, and determining the quality of a non-traditional text may take more time and effort than instructors prefer to spend.
To help instructors determine the quality of open textbooks, the Open Textbook Library, housed in the Center for Open Education at the University of Minnesota, has been aggregating peer reviews of these textbooks. Reviews are for faculty, by faculty; "The project pays faculty members to review open textbooks, as does a similar effort at the University of British Columbia (Canada), and the two initiatives share those peer reviews" (Center for Education Attainment and Innovation, 2015, p.2). To be included in the library, textbooks "[m]ust be currently in use at multiple higher education institutions, or affiliated with a higher education institution, scholarly society, or professional organization" (Center for Open Education, n.d., para. 1). In this paper, we examine data collected from peer reviews for Open Textbook Library titles to discuss questions about rating the quality of OER.
Several recent studies provide insight into faculty perceptions of open textbooks. As of this writing, all published perception studies report positive perceptions of open textbooks. Petrides, Jimes, Middleton-Detzner, Walling, and Weiss (2011) reported perceptions of OER by faculty and students who had used an open statistics textbook. The 31 instructors interviewed cited cost, content quality, and ease of use as the main factors that influenced them to adopt open textbooks. For the 45 students, advantages of OER included cost, portability, and ease of use. Both instructors and students said that the open textbook had positive impacts on teaching and learning. In a similar study focusing on a single open textbook, Lindshield and Adhikari (2013) examined online and campus students' perceptions of an open textbook used in Human Nutrition courses. While online students used the open textbook more frequently and rated it more highly than campus students, both campus and online students reported high levels of satisfaction and preferred using the online text to buying a traditional textbook.
Unlike the previous two studies, most research articles on OER involve multiple open resources. In one such study, Bliss, Hilton, Wiley, and Thanos (2013) reported results from a survey of 11 instructors and 132 students using OER at eight community colleges. A large majority of respondents reported positive perceptions of OER. Ten of the eleven instructors stated their students were as or more prepared than students using traditional texts, and 97% of students said that the open textbooks were as good as or better than traditional textbooks. Building on the previous study, Bliss, Robinson, Hilton, and Wiley (2013), surveyed an additional 58 teachers and 490 students about the cost, outcomes, usage and quality of OER used in their college courses. As with the previous study, a large majority of respondents rated OER as equal or higher in quality compared to traditional textbooks. The respondents who rated OER quality lower than that of traditional textbooks (11% of instructors and 6% of students) cited poor text quality and technology problems as the reasons for their ratings.
Hilton, Gaudet, Clark, Robinson, and Wiley (2013) examined the impact, in terms of cost savings, student retention and success, and user perceptions of adopting open resources in mathematics classes at Scottsdale Community College (SCC). Using data from 2,043 students taking five different courses, researchers determined that by using OER, students saved a significant amount of money without suffering adverse effects in terms of educational outcomes. In surveys, students and faculty reported mostly favorable perceptions of the OER materials used in the courses.
Feldstein et al. (2012) reported findings from a year-long pilot study in which 991 students taking courses in the Virginia State University School of Business used free online textbooks and other resources provided by Flat World Knowledge. Of the 315 students who completed the semester-end survey, 95% agreed that the OER course materials were easy to use, and most respondents preferred the digital content to traditional textbooks. Students in the courses that used open textbooks earned higher grades than students in similar courses that used traditional textbooks.
All of the above studies include student perceptions; several studies focus only on instructor perceptions. In their 2014 article, Allen and Seaman (2014) reported that among respondents aware of OER, 74% rated the quality of OER materials as similar or superior to that of traditional resources. About 85% rated the efficacy of OER materials as similar or superior to the efficacy of traditional materials. Kelly's 2014 study examined educator perceptions of OER and the impact of perception on adoption and use. Survey participants included 128 educators from higher education, K-12 schools, and workplace training. Kelly found that individuals with high self-efficacy were more likely to perceive OER as easy to use, and that ease of use correlated with perceived usefulness.
As part of a collaborative project between the BCcampus-led Open Textbook Project and OER Hub, researchers surveyed post-secondary educators interested in OER to examine awareness, usage, barriers, outcomes, and perceptions (Jhangiani, Pitt, Hendricks, Key, & Lalonde, 2016). Of the 78 respondents, 77% had used OER. Most respondents rated OER quality as comparable or superior to that of traditional materials; educators who had adopted OER rated the quality significantly higher than educators who had not. Respondents reported that the top two barriers to using OER were finding relevant and high quality OER. Faculty at research-intensive universities reported significantly lower barriers to finding high-quality OER than did faculty at teaching-intensive universities or colleges/institutes.
Pitt (2015) reported results of two surveys of 126 educators who had used open textbooks that were published by OpenStax. In relation to the issue of quality, almost 75% of the 2013 respondents and 95.7% of 2014/2015 respondents reported that they would be more likely to choose OER produced by a trusted author than OER produced by a lesser known or trusted entity. Pitt concludes that "the role of educators in promoting and giving credibility to OER, such as open textbooks, cannot be underestimated" (2015, p.141).
To date, concerning OER quality have typically been surveys of students and/or faculty members that have used OER in their classroom. One weakness with this approach is that students may be more likely to report based on convenience, rather than true efficacy (Kvavik, 2005). In addition, the faculty members may have been biased in terms of reviewing materials that in many instances they helped to create or select. In the present study, we will address this limitation by looking at a different aspects of reviews of open textbook quality. Rather than use survey results, we will us data from the peer reviews facilitated by the Open Textbook Library. These faculty reviewers had not adopted the resource at the time of their review. Therefore, their reviews may be somewhat less biased. Moreover, beyond reporting descriptive values of quality, we considered evaluations in terms of reviewers' characteristics (tenure status, job track, location), and textbook content areas. Our specific research questions will be discussed below.
The Open Text Network housed in the College of Education and Human Development at the University of Minnesota (open.umn.edu) acquires and disseminates OER written for higher education. By January 2016 it had catalogued 210 textbooks in the Open Textbook Library. Faculty from the US and Canada were invited to review and evaluate the quality of the textbooks.
Faculty reviewers evaluated 121 of the textbooks across 10 qualities on a 5-point Likert scale anchored with "1" as low and "5" as high. The 10 criteria were Comprehensiveness, Accuracy, Relevance/Longevity, Clarity, Consistency, Modularity, Organization, Interface, Grammar, and Cultural Relevance. Researchers summed the evaluations across all 10 qualities and divided by 10 to estimate an Overall Quality value. Included in this study were 416 reviews of the 121 textbooks. Of the 416 reviews, 377 were by faculty at the University of Minnesota; 44 were by faculty of British Columbia College in Vancouver, Canada. Some textbooks received multiple reviews with the number of reviews per textbook ranging from 1 to 21.
Reviewers were sorted into types for analysis purposes. Reviewers could be considered as Tenure Track or Non-tenure Track. They were further sorted into four groups: Adjunct, Tenure Track Assistant Professors, Tenured Associate or Full Professors and Other (such as student life advisors that teach courses). Reviewers were also sorted by US and Canada origin.
Textbooks were sorted into nine discipline types: Math/Statistics, Biology/Physiology, Physics/Chemistry, Social Science, English/Humanities/Communications, Computer Science/Information Systems, Business/Economics, Law, and Other.
Researchers sought answers to five research questions:
Question 1: What are the average ratings of each quality criterion and overall quality?
Question 2: Do Tenure Track and Non-tenure Track reviewers rate the textbooks differently?
Question 3: Do the four subgroups of reviewers rate the textbooks differently?
Question 4: Do the US and Canadian reviewers rate the textbooks differently?
Question 5: Do the nine textbook types differ across the quality criteria or overall quality?
Because the data were ordinal Likert scales and the distributions were significantly skewed (as seen in Table 1), all inferential tests were completed using non-parametric Kruskal-Wallis tests of differences. Although the evaluations were nested with repeated measures, the data were not robust enough to use multi-level modeling. The results should be interpreted cautiously, with an eye toward general trends more than solid inferential tests of null hypotheses.
The distributions of the rating across the 10 quality criteria and overall quality are displayed in Table 1. In general, the median ratings were high (4 & 5). The means were high with reasonable standard deviations and all the distributions were significantly negatively skewed.
Table 1
Descriptive Statistics of Criteria Variables
| Criteria | Mean | Median | Standard deviation | Skew | Standard error of skew | t-test of skew |
| Comprehensiveness | 4.23 | 4.00 | 0.816 | -0.95 | 0.12 | -7.90 |
| Accuracy | 4.41 | 5.00 | 0.740 | -1.15 | 0.12 | -9.54 |
| Relevance/longevity | 4.28 | 4.00 | 0.803 | -1.07 | 0.12 | -8.88 |
| Clarity | 4.43 | 5.00 | 0.800 | -1.48 | 0.12 | -12.37 |
| Consistency | 4.57 | 5.00 | 0.723 | -1.97 | 0.12 | -16.40 |
| Modularity | 4.49 | 5.00 | 0.759 | -1.38 | 0.12 | -11.49 |
| Organization/flow | 4.28 | 4.00 | 0.849 | -1.16 | 0.12 | -9.70 |
| Interface | 4.04 | 4.00 | 0.975 | -0.82 | 0.12 | -6.84 |
| Grammar | 4.71 | 5.00 | 0.605 | -2.41 | 0.12 | -20.11 |
| Cultural relevance | 4.20 | 4.00 | 0.968 | -1.33 | 0.12 | -9.45 |
| Overall rating | 4.36 | 4.50 | 0.474 | -1.28 | 0.12 | -10.65 |
Table 2 illustrates the results associated with all inferential tests. Other tables will present the descriptive statistics across all criteria. As seen in Table 3, all reviewers generally rated the textbooks favorably across the 10 criteria. There were no significant differences between the Tenure Track and Non-tenure Track reviewers on any of the 10 quality criteria or overall quality score (Table 2). The only significant difference among the four subgroups of reviewers on quality criteria or overall quality score was observed between the Other reviewers and the Adjunct Faculty reviewers in terms of their assessment of modularity (Table 2). The Other reviewers judged the modularity of the books to be lower than Adjunct Faculty reviewers.
Table 2
Summary of Kruskal-Wallis Tests of Differences Across Groups in Terms of 10 Criteria
| Criteria | Reviewer status | p-value | Adjunct v tenure track | p-value | Associate v assistant | p-value | BCC v UMN | p-value | Text type | p-value |
| Comprehensiveness | 2.105 | 0.551 | 0.106 | 0.745 | 0.026 | 0.872 | 12.03 | 0.001* | 7.022 | 0.534 |
| Accuracy | 6.769 | 0.08 | 0.003 | 0.955 | 5.058 | 0.025 | 24.66 | p < 0.001* | 23.796 | 0.002* |
| Relevance | 2.71 | 0.439 | 0.517 | 0.472 | 0.216 | 0.642 | 2.73 | 0.099 | 5.679 | 0.683 |
| Clarity | 5.651 | 0.13 | 0.001 | 0.978 | 3.977 | 0.046 | 9.26 | 0.002* | 45.702 | p < 0.001* |
| Consistency | 5.564 | 0.135 | 0.942 | 0.322 | 4.04 | 0.044 | 5.39 | 0.02 | 21.154 | 0.007 |
| Modularity | 8.662 | 0.034 | 2.047 | 0.153 | 0.378 | 0.539 | 0.895 | 0.344 | 17.942 | 0.022 |
| Organization | 0.596 | 0.897 | 0.006 | 0.937 | 0.164 | 0.685 | 7.19 | 0.007 | 22.14 | 0.005* |
| Interface | 0.545 | 0.909 | 0 | 0.986 | 0.397 | 0.528 | 2.68 | 0.102 | 25.584 | 0.001* |
| Grammar | 7.595 | 0.055 | 2.285 | 0.131 | 4.393 | 0.036 | 2.19 | 0.138 | 7.487 | 0.485 |
| Cultural relevance | 5.562 | 0.135 | 0.155 | 0.1694 | 0.278 | 0.598 | 0.675 | 0.411 | 15.934 | 0.043 |
| Overall quality | 3.809 | 0.283 | 0.275 | 0.6 | 1.582 | 0.208 | 13.746 | p < 0.001* | 19.771 | 0.011 |
*Bonferroni correction for eleven tests required p-values to be less than 0.0045.
Table 3
Ratings by Reviewer Status
| Reviewer title | Assistant Professor (Non-tenured, tenure track) | Associate or Full Professor (Tenured) | Adjunct (Non-tenure track) | Other (Non-tenure track) | Total | |||||
| Mean | Median | Mean | Median | Mean | Median | Mean | Median | Mean | Median | |
| Comprehensiveness | 4.18 | 4.00 | 4.18 | 4.00 | 4.28 | 4.00 | 4.12 | 4.00 | 4.23 | 4.00 |
| Accuracy | 4.54 | 5.00 | 4.29 | 4.00 | 4.45 | 5.00 | 4.27 | 4.00 | 4.41 | 5.00 |
| Relevance & longevity | 4.25 | 5.00 | 4.21 | 4.00 | 4.35 | 4.00 | 4.12 | 4.00 | 4.28 | 4.00 |
| Clarity | 4.55 | 5.00 | 4.33 | 5.00 | 4.46 | 5.00 | 4.36 | 4.00 | 4.43 | 5.00 |
| Consistency | 4.73 | 5.00 | 4.54 | 5.00 | 4.55 | 5.00 | 4.48 | 5.00 | 4.57 | 5.00 |
| Modularity | 4.45 | 5.00 | 4.38 | 5.00 | 4.59 | 5.00 | 4.24 | 4.00 | 4.49 | 5.00 |
| Organization & flow | 4.25 | 4.00 | 4.30 | 4.00 | 4.26 | 4.00 | 4.36 | 5.00 | 4.28 | 4.00 |
| Interface | 3.95 | 4.00 | 4.04 | 4.00 | 4.05 | 4.00 | 4.12 | 4.00 | 4.04 | 4.00 |
| Grammar | 4.77 | 5.00 | 4.55 | 5.00 | 4.77 | 5.00 | 4.76 | 5.00 | 4.71 | 5.00 |
| Cultural relevance | 4.20 | 5.00 | 4.11 | 4.00 | 4.29 | 5.00 | 3.91 | 4.00 | 4.20 | 4.00 |
| Overall quality | 4.38 | 4.50 | 4.29 | 4.40 | 4.40 | 4.50 | 4.27 | 4.30 | 4.36 | 4.50 |
As seen in Table 4, US and Canadian reviewers generally rated the textbooks favorably. There were some significant differences between US and Canadian reviewers (Table 2). These occurred on Comprehensiveness, Accuracy, Clarity, and Overall Quality. In all these cases, the Canadian reviewers tended to rate the textbooks lower than the US reviewers.
Table 4
Ratings by Reviewer University
| Criteria | UMN (U.S.) | BCC (Canada) | Total | |||
| Mean | Median | Mean | Median | Mean | Median | |
| Comprehensive-ness* | 4.27 | 4.00 | 3.82 | 4.00 | 4.23 | 4.00 |
| Accuracy* | 4.47 | 5.00 | 3.89 | 4.00 | 4.41 | 5.00 |
| Relevance & longevity | 4.31 | 4.00 | 4.05 | 4.00 | 4.28 | 4.00 |
| Clarity* | 4.48 | 5.00 | 4.05 | 4.00 | 4.43 | 5.00 |
| Consistency | 4.59 | 5.00 | 4.36 | 5.00 | 4.57 | 5.00 |
| Modularity | 4.50 | 5.00 | 4.39 | 5.00 | 4.49 | 5.00 |
| Organization & flow | 4.31 | 5.00 | 4.00 | 4.00 | 4.28 | 4.00 |
| Interface | 4.07 | 4.00 | 3.82 | 4.00 | 4.04 | 4.00 |
| Grammar | 4.72 | 5.00 | 4.61 | 5.00 | 4.71 | 5.00 |
| Cultural relevance | 4.22 | 4.00 | 4.05 | 4.00 | 4.20 | 4.00 |
| Overall quality* | 4.39 | 4.50 | 4.10 | 4.00 | 4.36 | 4.50 |
*Denotes significant differences, with p < 0.0045
There were some significant differences in ratings among the textbook types (Table 5). These occurred on Accuracy, Clarity, Organization, and Interface. In the post-hoc pairwise comparisons the 12 Physics/Chemistry textbooks tended to be the consistently outlying (lower-rated) textbooks.
Table 5
Mean Ratings by Text Type
| Criterion | Math & Stats | Biology & Physiology | Physics & Chemistry | Social Science | English/Humanities/Communication | Computer Sci/IS | Business & Econ | Law | Other | Total |
| Comprehensiveness | 4.28 | 4.36 | 3.87 | 4.3 | 4.14 | 4.2 | 4.24 | 4.43 | 4.17 | 4.23 |
| Accuracy* | 4.59 | 4.2 | 3.97 | 4.42 | 4.44 | 4.73 | 4.43 | 4.71 | 4.38 | 4.41 |
| Relevance & longevity | 4.41 | 4.38 | 4.1 | 4.19 | 4.29 | 4.2 | 4.24 | 4.57 | 4.28 | 4.28 |
| Clarity* | 4.31 | 4.07 | 4 | 4.72 | 4.63 | 4.4 | 4.32 | 5 | 4.48 | 4.43 |
| Consistency | 4.66 | 4.35 | 4.29 | 4.66 | 4.59 | 4.67 | 4.62 | 5 | 4.52 | 4.57 |
| Modularity | 4.42 | 4.53 | 3.97 | 4.66 | 4.54 | 4.67 | 4.43 | 4.29 | 4.52 | 4.49 |
| Organization* & flow | 4.45 | 4.29 | 3.68 | 4.39 | 4.11 | 4.27 | 4.27 | 4.86 | 4.48 | 4.28 |
| Interface* | 4.3 | 4.02 | 3.52 | 4.27 | 3.86 | 4.27 | 3.68 | 3.71 | 4.21 | 4.04 |
| Grammar | 4.7 | 4.75 | 4.42 | 4.73 | 4.77 | 4.67 | 4.65 | 5 | 4.79 | 4.71 |
| Cultural relevance | 4.28 | 4.25 | 4.13 | 4.26 | 4.01 | 4.6 | 4.24 | 4.57 | 3.93 | 4.2 |
| Overall quality | 4.44 | 4.32 | 3.99 | 4.46 | 4.34 | 4.47 | 4.31 | 4.61 | 4.38 | 4.36 |
*Denotes significant differences, with p < 0.0045
Our first research question was, "What are the average ratings of each quality criterion and overall quality?" In answer to this question, we found that textbook quality was generally rated high for each criterion and for overall quality. The highest-rated criteria were grammar (4.67) and consistency (4.52). The lowest-rated criteria were interface (3.95) and comprehensiveness (4.13). This finding indicates that reviewers are identifying strengths and weaknesses of the textbooks they review across the different categories. The fact that the reviews were favorable across all nine criteria indicates that the reviewers judged the textbooks housed on the Open Textbook Library to be of relatively high quality.
Our second and third research questions asked whether reviewers' professional track and status affect their ratings. We found no significant difference in the quality of ratings by tenure-track and non-tenure track reviewers. Dividing reviewers into four groups - Adjunct, Tenure Track Assistant Professors, Tenured Associate or Full Professors, and Other (such as student life advisors that teach courses) - again yielded no significant difference in the quality of reviews. This finding has important implications for research studies of OER use at community colleges, where many instructors are adjuncts. The data suggests that quality can be identified without particular bias according to status; more experienced teachers (such as full professors) were not more discriminating than less-experienced teachers in terms of the quality of textbooks. In other words, in studies involving faculty ratings, it would not be accurate to argue that reviews should be discounted because they are by faculty of different ranks, because faculty in this study tended to agree about the textbook quality, regardless of rank.
Fourth, we asked whether U.S. and Canadian reviewers rate the textbooks differently. Across all categories, typical ratings by Canadian reviewers were lower than typical ratings by U.S. raters. Our data indicates that raters' country of residence predicts some differences in ratings; however, with the data we have we cannot determine why this occurred. We might speculate that the reviewers from Canada seem to have higher standards than U.S. reviewers. There are a few important implications of this finding. First, although OER are typically available online facilitating international use, they may not be perceived as being equally useful in different educational systems. In some contexts, instructors may need to significantly revise open textbooks produced elsewhere, which revising is allowed under most open licenses. Second, when consulting online reviews of open textbooks, readers should be aware of reviewers' contexts. Reviews may be most useful when the reviewer and reader teach similar courses in the same country. Third, in studies such as this one involving raters from multiple countries, average scores may hide differences among various groups of raters.
Finally, we asked whether quality varies among textbook types. According to the data, perceived quality varies by discipline or textbook type. Among the nine disciplinary categories, physics and chemistry books tended to be rated significantly lower in some criteria than textbooks in the other disciplines. In our data, there is a correlation between low scores for science textbooks and low ratings by Canadian reviewers; Canadian reviewers rated a higher proportion of physics and chemistry books than any other disciplinary category.
Taken together, the data indicates that overall quality of textbooks in the Open Text Library is high, but textbooks do vary in quality, and those variances can be identified by reviewers. Therefore, online ratings can be used to help consumers discriminate among various open textbooks. Furthermore, quality is an important covariate in OER research. Up to this point, most efficacy studies involving open textbooks have not considered the quality of the textbooks as a covariate. In comparing the effectiveness of open textbooks and proprietary materials, researchers need to acknowledge and account for differences in quality among available texts, whether open or proprietary. It is not just the presence or absence of OER at play, but the quality of each resource used that affects perceptions and efficacy.
As noted previously, this study was limited by the available data. This study included 416 reviews of 121 textbooks, but some textbooks received many more reviews than others. Reviewers were given very little guidance in the rating process. Of the 416 reviewers, only 44 were Canadian, and the Canadians reviewed only 5 of the 10 text types. Based on these limitations, the results suggest patterns, but may not be generalizable. Further studies are needed to understand how textbook type and reviewer characteristics affect textbook ratings.
Another limitation stems from the nature of the reviews. In the Open Textbook Library, reviewers attach their names to their ratings. The rationale for publishing reviewers' names was to increase rater accountability and consumer trust. However, it is possible that named reviewers inflate scores compared with anonymous reviewers. Based on this limitation, comparisons between Open Textbook Library ratings and anonymous ratings may not be valid. For this study, all comparisons are among Open Textbook Library ratings.
In the brief history of the open textbooks movement, proponents of open textbooks have necessarily focused much effort on development and awareness. There are now thousands of open textbooks online and thousands of instructors using open textbooks (MERLOT, 2015), but as yet, relatively few studies about perceived quality and measured effectiveness of open textbooks. This study is among the first to examine online textbook rating systems. We hope that others will pursue similar research to help explain OER ratings and quality.
In this study, we examined faculty ratings of textbooks in the Open Textbook Library, and found that the ratings tend to be high, with some variance. Based on the ratings, instructors wishing to find high quality open textbooks should be able to trust the quality of texts in the Open Textbook Library. As Pitt (2015) suggested, recommendations by faculty are highly influential. However, the ratings reflect only perceptions, not outcomes for students who use the textbooks, nor comparisons of those outcomes with outcomes for students using proprietary textbooks. Perhaps the Open Textbook Network and other open textbook repositories could add measures of effectiveness to their online rating systems. Combining ratings with efficacy measures would provide a more complete picture than ratings alone, greatly benefiting potential users of open textbooks.
Allen, I. E., & Seaman, J. (2014). Opening the curriculum: Open educational resources in US Higher Education, 2014. Babson Survey Research Group.
Bissell, A. N. (2009). Permission granted: open licensing for educational resources. Open Learning, 24(1), 97-106. doi: 10.1080/02680510802627886
Bliss, T. J., Hilton III, J., Wiley, D., & Thanos, K. (2013). The cost and quality of online open textbooks: Perceptions of community college faculty and students. First Monday, 18(1). doi: 10.5210/fm.v18i1.3972
Bliss, T. J., Robinson, T., Hilton, J., & Wiley, D. (2013). An OER COUP: College teacher and student perceptions of open educational resources. Journal of Interactive Media in Education, 2013(1). Retrieved from http://jime.open.ac.uk/articles/10.5334/2013-04/
Center for Education Attainment and Innovation. (2015). Open textbooks: The current state of play. Washington, D.C.: American Council on Education. Retrieved from http://www.acenet.edu/news-room/Documents/Quick-Hits-Open-Textbooks.pdf
Center for Open Education. (n.d.). Open textbook library: About our textbooks. Retrieved from http://open.umn.edu/opentextbooks/Ourbooks.aspx
Cochrane, D., & Reed, M. (2015). Student debt and the class of 2014. Project on Student Debt. Retrieved from: http://ticas.org/sites/default/files/pub_files/classof2014.pdf
Feldstein, A., Martin, M., Hudson, A., Warren, K., Hilton III, J., & Wiley, D. (2012). Open textbooks and increased student access and outcomes. European Journal of Open, Distance and E-learning, 15(2). Retrieved from http://www.eurodl.org/materials/contrib/2012/Feldsteint_et_al.pdf
Hilton, J., Gaudet, D., Clark, P., Robinson, J., & Wiley, D. (2013). The adoption of open educational resources by one community college math department. The International Review of Research in Open and Distance Learning, 14(4), 37–50. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1523
Jhangiani, R. S., Pitt, R., Hendricks, C., Key, J., & Lalonde, C. (2016). Exploring faculty use of open educational resources at British Columbia post ‐ secondary institutions. BCcampus Research Report. Victoria, BC: BCcampus. Retrieved from http://viuspace.viu.ca/handle/10613/2705
Kelly, H. (2014). A path analysis of educator perceptions of open educational resources using the technology acceptance model. The International Review of Research in Open and Distance Learning, 15, 26–42. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1715/2881
Kvavik, R. B. (2005). Convenience, communications, and control: How students use technology. In D. G. Oblinger, & J.L. Oblinger (Eds.), Educating the net generation. EDUCAUSE Center for Applied Research, 7.1-7.21. Retrieved from https://net.educause.edu/ir/library/pdf/pub7101.pdf
Lindshield, B., & Adhikari, K. (2013). Online and campus college students like using an open educational resource instead of a traditional textbook. Journal of Online Learning & Teaching, 9(1), 1–7. Retrieved from http://search.proquest.com.erl.lib.byu.edu/docview/1500389471?accountid=4488
MERLOT. (2015). Programs and projects (2015). Retrieved from http://info.merlot.org/merlothelp/index.htm#merlot_collection.htm
Petrides, L., Jimes, C., Middleton‐Detzner, C., Walling, J., & Weiss, S. (2011). Open textbook adoption and use: implications for teachers and learners. Open learning, 26(1), 39-49. doi: 10.1080/02680513.2011.538563
Pitt, R. (2015). Mainstreaming open textbooks: Educator perspectives on the impact of OpenStax College open textbooks. The International Review of Research in Open and Distributed Learning, 16(4). Retrieved from http://files.eric.ed.gov/fulltext/EJ1082205.pdf
Senack, E. (2014). Fixing the broken textbook market. US Public Interest Research Group, Student PIRG. Retrieved from http://www.uspirg.org/reports/usp/fixing-broken-textbook-market
Senack, E. (2015). Open textbooks: The billion dollar solution. The Student PIRGs. Retrieved from http://www.studentpirgs.org/sites/student/files/reports/The%20Billion%20Dollar%20Solution.pdf
U.S. Government Accountability Office. (2013). College textbooks: Students have greater access to textbook information. (Report No. GAO-13-368). Washington, D.C.: Author. Retrieved from https://www.gao.gov/assets/660/655066.pdf
Rating the Quality of Open Textbooks: How Reviewer and Text Characteristics Predict Ratings by Lane Fischer, David Ernst, and Stacie Mason is licensed under a Creative Commons Attribution 4.0 International License.