Khan Academy Videos in Chinese: A Case Study in OER Revision

Over the past decade, great progress has been made in improving the quality and availability of Open Educational Resources (OER). OER proponents often discuss the ability for users to revise and remix OER to make them more suitable for local contexts; however, much OER goes unmodified. This note from the field examines the efforts of NetEase Online Open Courses, a Chinese organization, to take more than two thousand Khan Academy videos and translate them into Chinese. We provide background on this initiative and also analyze site metrics to determine what type of use these derivative OER have received.

Over the past decade, great progress has been made in improving the quality and availability of Open Educational Resources (OER). OER proponents often discuss the ability for users to revise and remix OER to make them more suitable for local contexts; however, much OER goes unmodified. This note from the field examines the efforts of NetEase Online Open Courses, a Chinese organization, to take more than two thousand Khan Academy videos and translate them into Chinese. We provide background on this initiative and also analyze site metrics to determine what type of use these derivative OER have received.

Introduction
An important topic in education concerns the use of Open Educational Resources (OER). D' Antoni (2009) cited the William and Flora Hewlett Foundation, a key donor for OER, as stating that OER are "resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others" (p. 4).
A key aspect of this definition is that OER permits free re-purposing; in other words, individuals can legally revise or remix OER. Hilton, Wiley, Stein, and Johnson (2010)

Review of Literature
Revision of OER Duncan (2009) found that very few OER adopters revise or remix the work. Revision is defined as a modification to the OER, such as a translation, or an addition or deletion of material. When additional resources are combined with an OER, the new work is considered to be a remix. In Duncan's study of an OER collection containing 3,519 utilized components, less than 3% were translated, and less than 3% were modified in some other way. Thus, in this study, less than 6% of the OER in the reviewed set were revised or remixed.
Additionally, Petrides, Nguyen, Jimes, and Karaglani (2008) noted that in the same OER collection observed in Duncan's analysis, 88% of those who altered the open resources were the original authors themselves. If a fundamental foundation of the OER model is the ability for others to modify the materials, then the findings of Petrides et al. (2008) convey a limited extension of a principle objective. Consequently, this report challenges the opinion that people are anxious to refashion resources for their own purposes.
To further explore the customization of open resources, Hilton, Lutz, and Wiley (2012) investigated the degree to which instructors reused, revised, and remixed Flat World Knowledge (FWK) open textbooks. By comparing custom texts to the original material, the examination catalogued additions, deletions, reorders (changing the sequence of the material), and remixes (introducing the content of another FWK resource into the custom text). This study likewise observed low revision and remix rates, with only 7.5% of the texts customized over the two-year period. More significantly, a noteworthy pattern became apparent; the simpler to make the change, the more of those changes were observed. Deletions, the easiest alteration, were found in 60.32% of customized materials, followed by re-orderings in 40.49%. Additions were made in 16.60% of customized materials, and finally, remixes, the most challenging modification, occurred in only 1.62% of the remixes.

Amiel (2013) examined decisions and challenges accompanying the creation of a work by translating and
remixing open education resources. Amiel produced an instructional booklet on OER for Brazilian K-12 teachers and chronicled the efforts that it took to create this resource. Amiel identified six different routes a utilized original resource could take. A contribution could be "in complete form attempting to maintain the original meaning and context (revision); adapting language to reflect local meaning and needs (revision/remix); modifying substantially to the point of departing from the original source (remix); only as a source of inspiration, demonstrating a need to create content based on the principles and ideas but not borrowing directly from the source itself (reference); identifying instead other, more compatible resources (search); developing entirely new content when none was available (create)" (Amiel, 2013, pp. 133-134).
When creating a new OER, Amiel selected resources with more open licenses over superior material with narrowed licensing. Amiel found several concerns with this English-based westernized OER that invoked multiple concerns, such as cultural barriers. Amiel concluded that although OER are available to be altered, in practice, performing the actual remix is not an easy task. While revising OER at the level of discrete learning resources appears to happen relatively irregularly, research indicates that some important books, or large-scale OER projects, do receive extensive revisions.
For example, Hilton (2009) found that many people were willing to do large scale revisions (e.g., translation, creating an audio version, etc.) of key works such as Larry Lessig's Free Culture. Carson, Kanchanaraksa, Gooding, Mulder, and Schuwer (2012)  The focus of the present study is a translation of Khan Academy videos into Mandarin; we next discuss additional context for this study by first providing an overview of Khan Academy, as well as what the literature says about its impact on student learning.

Khan Academy
Initially founded as a hobby of a hedge fund analyst who was a long distance tutor for his cousins, Khan Academy now has impacted millions of lives across the globe ("Stories," 2016). Salman Khan transitioned from tutoring using Yahoo's Doodle notepad to creating YouTube videos, and after unexpected popularity, established Khan Academy in October 2006. Three years later, Khan abandoned his finance work to focus on his nonprofit educational organization full time-in a walk-in closet (Noer, 2012).
Khan Academy has released more than 5,500 videos on YouTube and its website, khanacademy.org, in addition to hundreds of thousands of practice problems that students can work on at their own pace.
Subjects covered in the micro-lectures include math from the elementary to college level, science and engineering, computing, arts and humanities, economics and finance, and test prep for exams such as the SAT, MCAT, and GMAT ("About," 2016). Murphy, Gallagher, Krumm, Mislevy, and Hafter (2014) examined the efficacy of Khan Academy as a supplemental resource in schools between 2011 and 2013. Two thousand students in grades 5-10 participated in the study; nine sites, 20 schools, and 70 teachers were involved. Findings from the study include the following: student perceptions of their time spent on Khan Academy was highly positive, student engagement was generally high, and a majority of teachers stated that they planned to continue using the Academy after the conclusion of study. Teachers also reported that the program increased their capacity to support students across multiple areas, and that they found Khan Academy reports useful to review student 308 progress. Murphy et al. (2015) found a positive correlation between increased Khan Academy use and " (1) improvements in student test scores, and (2) improvements in three of the four self-reported nonachievement outcomes -math anxiety, math self-concept, and academic efficacy (i.e., belief in one's ability to succeed in academic endeavors)" (p. 12).
In a separate research study, Cohen and Phillips (2015) examined an implementation of Khan Academy across Idaho 47 different schools in 33 that focused on personalized learning. Data from 5,309 students between 3 rd and 8 th grade showed positive correlations between the usage of Khan Academy and academic progress (Cohen & Phillips, 2015). In a comparison between Khan Academy mission (a personalized set of grade-level content) completion and target growth, "students who completed 0-10% of their mission on average grew as expected," "students who completed 40% or more of their mission on average grew 1.5 times more than their expected growth in one year," and "students who completed 60% or more of their mission on average grew 1.8 times their expected growth in one year" (Cohen & Phillips, 2015, p. 22). Other studies have found learning gains when Khan Academy videos were implemented in classroom settings (Castillo & McIntosh, 2012;Park, 2016).

Khan Academy licenses its videos with a Creative Commons BY-NC-SA license. This open license allows
people to retain, reuse, revise, remix, and redistribute the videos. Thus, Khan Academy legally grants permission for people to edit their videos and share them in different formats, as long as it is done for noncommercial purposes.

Khan Academy Translations
Because of the value of Khan Academy videos, significant efforts have been made to translate them into other languages. Khan Academy has a volunteer translation portal, by which much of its content is available, following a volunteer translation model that has been used by others (Beaven, Comas-Quinn, Hauck, de los Arcos, & Lewis, 2012). However, in the case of the Chinese translation of Khan Academy videos, the translation was done by NetEase Open Courses, a subsidiary of called NetEase (网易). NetEase is an Internet company based in Beijing, China that focuses a variety of Internet services including email and online gaming. As part of a non-profit initiative within the company, NetEase Open Courses hosts multiple OER and has been involved in a variety of initiatives relating to making educational content freely available, such as working with Coursera to deliver free online courses (Penna, 2013).
In 2012, Khan Academy announced, At present, 927 videos from Khan Academy have been translated by NetEase Open Courses, covering mathematics, chemistry, biology, geometry, physics, and other disciplines…NetEase is the first portal site in China to partner with Khan Academy to localize its content and this greatly enhances NetEase Open Courses' content and user base. Meanwhile, the cooperation with NetEase will promote the localization of Khan Academy courses in China, to achieve the purpose of providing a world class education for anyone anywhere ("NetEase subtitles," 2012).
As of September, 2016, NetEase has translated 2,931 Khan Academy videos into Chinese, across 88 different courses (e.g., U.S. History, see appendix for complete course list).

309
In the present study we examine the impact of NetEase's revision of OER, specifically the Khan Academy videos, by asking the following questions: 1. How frequently have the Chinese Khan Academy videos been viewed? 2. Which Chinese Khan Academy videos are most popular?

Method
We used analytics data provided by NetEase Open Courses to determine which videos had been viewed most frequently. The two principal ways that people can access the Khan Academy videos in Chinese are through NetEase's website (http://open.163.com/khan/) and their apps. Unfortunately, accurate analytics data were not available prior to 2016; thus, we used the analytics data available between January 1 -September 30, 2016 to determine the number of page views. These data were provided by NetEase Open Courses and capture visits that took place through the NetEase Open Courses app, as well as their web portal. Access to videos is much more likely to come from the NetEase app than the website. Most importantly, it is clear that there is a large amount of interest in these videos, with the top 10 courses garnering more than 60 million views.

Discussion
These data prompt several observations about the ability to revise OER and its attendant usefulness.