A Bibliometric Mapping of Open Educational Resources

Open educational resources (OER) is a topic that has aroused increasing interest by researchers as a powerful contribution to improve the educational system quality and openness, both in face to face and distance education. The goal of this research is to map publications related to OER, dating from 2002 to 2013, and available through the Web of Science and Scopus scientific databases as well as in the OER Knowledge Cloud open repository. Data were used to explore relevant aspects related to the scientific production in OER, such as: (i) number of publications per year; (ii) most cited publications; (iii) authors with higher number of publications; (iv) institutions and countries with more publications and (v) most referenced bibliography by the authors. The analysis has included 544 papers, written by 843 authors, from 338 institutions, from 61 different countries. Moreover, the analysis has included the publications referenced and the author’s keywords, considering 6,355 different publications and 929 different keywords. Besides presenting a bibliographic mapping of the research on OER, this paper also intends to contribute to consolidate the idea that OER is a promising field for researchers, in line with the spreading of the Open movement.


Résumé de l'article
Open educational resources (OER) is a topic that has aroused increasing interest by researchers as a powerful contribution to improve the educational system quality and openness, both in face to face and distance education. The goal of this research is to map publications related to OER, dating from 2002 to 2013, and available through the Web of Science and Scopus scientific databases as well as in the OER Knowledge Cloud open repository. Data were used to explore relevant aspects related to the scientific production in OER, such as: (i) number of publications per year; (ii) most cited publications; (iii) authors with higher number of publications; (iv) institutions and countries with more publications and (v) most referenced bibliography by the authors. The analysis has included 544 papers, written by 843 authors, from 338 institutions, from 61 different countries. Moreover, the analysis has included the publications referenced and the author's keywords, considering 6,355 different publications and 929 different keywords. Besides presenting a bibliographic mapping of the research on OER, this paper also intends to contribute to consolidate the idea that OER is a promising field for researchers, in line with the spreading of the Open movement.

Introduction
The Open movement is based on the idea that knowledge is a public good (Mulder, 2013) that can be disseminated and shared freely on the Web for the benefit of all who wish to use it, enabling an equitable development (Broad, 2011). For this purpose, technology in general, and the Web in particular, can offer opportunities for the use, reuse, revise, remix and redistribution of knowledge.
Centered in the "openness" paradigm (Hylén, 2006), the term open educational resources (OER) was coined in the Forum on the Impact of OpenCourseWare for "Higher Education in Developing Countries", sponsored by UNESCO and held in Paris in 2002 as a way to uniform other existing terms like "open courseware", "open learning resources" and "open teaching/learning resources" (UNESCO, 2002, p. 24).
In general, the OER movement is inspired by the open source software paradigm, denoting that the resources can be accessed openly for educational purposes (Pawlowski & Bick, 2012), and setting freedom to reuse, revise, remix and redistribute educational materials with the least possible restrictions, although respecting copyright issues (Molloy, 2011). For the Organisation for Economic Co-operation and Development (OECD) the most accepted definition of OER is "[…] digitised materials offered freely and openly for educators, students and self-learners to use and reuse for teaching, learning and research" (OECD, 2007, p.30).
About OER some thematic literature reviews were prepared for specific purposes, such as the work of ), Hemingway, Angell, Hartwell and Heller (2011. These publications provide a literature review and systematic online search to identify studies that associate public health with OER. They were also part of a project called Public Health Open Resources held in the University Sector (PHORUS), which had the intention of identifying OER focused on public health training in the United Kingdom.
Other studies, such as Santos-Hermosa, Ferran, Ferrer and Abadal (2012), address the theme OER from a set of relevant perspectives: definitions, characteristics, evolution, interoperability, copyright, quality and repositories. Arimoto and Barbosa (2012) provide a systematic literature review about development, delivery and reuse of OER and Cobo (2013) offers a comparative analysis regarding the linguistic diversity of OER studies.
Through the application of bibliometric techniques, this paper provides a mapping of publications related to OER, aiming to offer researchers interested about the subject a systematic view of the main publications, sources of scientific information, authors and geographical origin of those studies. For this bibliometric study three different databases, Web of Science (WoS) (https://webofknowledge.com/), Scopus (http://scopus.com) and OER Knowledge Cloud (https://oerknowledgecloud.org/), were used. For the bibliometric analysis were considered books, book chapters, articles in scientific journals, publications in conference proceedings,

Methodological Procedures for the Conduct of Bibliometric Research
Bibliometrics are characterized by the "[...] application of statistical analysis to the bibliographic production [...]" (Bufrem & Prates, 2005, p. 11), and literature review deals with the collecting of publications on a specific subject (Sampieri Collado & Lucio, 2006), for example in scientific databases. In this study the literature search was done based on a step by step bibliometric analysis, as suggested in Zancanaro et al (2013). The main steps are: 1) collecting, filtering, and data standardization; and 2) analysis and synthesis of the collected data. These phases, along with its steps, are shown in Figure 1.

4
The following paragraphs describe how these steps were executed. Step

-Definition of the Search Terms
In this step, the term "Open Educational Resources" was set up focusing on the databases search in English, Spanish and Portuguese languages. The option for English derives from its relevance as an international knowledge communication tool. Portuguese and Spanish were also considered due to the nationality and geographical context of the authors. These databases were chosen based on their multidisciplinary nature. They are recognized as main sources of relevant information by the international scientific community and they allow to export the data to a bibliographic management software in a standardized format.

Step 2 -Scientific Databases and Open Repository Search
Step 3 -Export of the Records to a Bibliographic Management Software The records were generated by applying the queries to the databases containing information such as title, authors, year, place of publication, author keywords, and were imported to a bibliographic management software, thus forming a single set of records. Step

-Implementation of Criteria for the Selection of Papers
In the bibliographic management software the following criteria for the selection of papers were applied: a) the removal of the articles without authorship; b) exclusion of those who were doubled; c) removal of the works in which the text was not available for free download or could not be found; and d) exclusion of those that either were not focused on the context of this study, or were not books, book chapters, articles in scientific journals, publications in conference proceedings, reports and academic theses. Step

-Classification of Works in Thematic Research
In this step the analysis and classification in macro-themes of the final set of works resulting from step 4 was developed.
This work is licensed under a Creative Commons Attribution 4.0 International License.

-Standardization of Data from Different Databases
The records resulting from different databases (WoS, Scopus and OER Knowledge Cloud) had different criteria for submission of fields, such as the way the name of the authors was displayed, which generated the need for standardization. Moreover, information concerning the institutional affiliation of authors, author keywords and references were often not available in scientific databases, so it was necessary to collect the information directly from the articles. Thus a new database was created, in which information of each item was standardized and completed manually, including: the authors and co-authors (surname, name and institutional affiliation), year of publication, language, document type and periodic, the references (author, year, title of work) and keywords. Step

-Inclusion of Other Works Related to the Topic
In this step, other publications that were not indexed in the searched databases were also incorporated into the set of materials deemed as relevant to the study. To achieve this purpose, the following procedures were executed: a) inclusion of the most used references in the selected publications; b) inclusion of selected papers published in conferences, technical reports published by relevant international institutions such as UNESCO, OECD and Commonwealth of Learning, as well as books and book chapters. These materials also passed through the data standardization procedure described in step 6. Step

-Analysis of Data
Having the final set of selected and standardized content, we could finally perform queries and generate images that could best interpret the data, in line with the goals of the research.

Step 9 -Analysis and Preparation of the Final Report
Based on the references used from the selected publications, it was possible to identify the works to be considered thematic reference in the OER topic. These works were identified, analyzed and synthesized. Later, in possession of all the information, we could draw up the final report, presenting the analyses of the material and the results obtained.

Results of the Research
In this section, we will present the results obtained from the analysis and synthesis of information performed, using the procedures described above, over the set of selected publications.
This work is licensed under a Creative Commons Attribution 4.0 International License. 6

General Bibliometric Research Data
The bibliometric research was carried on September 2014, and database records from January 2002 to December 2013 were selected. The 2002 boundary was elected because the OER term was first coined in 2002 by UNESCO. Table 1 shows a summary of the tasks performed for the analysis process as previously described in steps 4 and 7. Note. In the divided cells, tasks 2 to 5, the quantity on the top represents the amount of publications that have been removed and the quantity below represents the total number of remaining articles. In task 6, the quantity on the top represents the number of items that have been added and the quantity below the total final number of articles analyzed.
Among the 544 papers selected, 246 are indexed articles in scientific journals, 189 are publications in conference proceedings, 52 are reports, 39 are book chapters, 15 are books and 3 are academic theses; 522 were published in English, 17 in Spanish and 5 in Portuguese. Table 2 summarizes some relevant details about the publications considered on this study. In the next sections an analysis focused on each of the bibliometric items from Table 2 is presented.  8 international community. It was also in 2013 that the greatest number of book chapters were published, 16 in total. On the other hand, the number of publications in conference proceedings tends to oscillate, as the periods of greater productivity were in 2010 with 59 papers, and in 2012 with 51 papers. This pattern is also found in the reports, with 11 publications in 2010 and 10 in 2012.

Publications by Year
The first work to coin the term OER was the final report of the Forum on Impact of Open Courseware for Higher Education in Developing Countries, held in 2002(UNESCO, 2002. 2005 saw the first published article about OER, titled "OpenSeminar: A web-based collaboration tool for open educational resources" (Rappa, Smith, Yacoub, & Williams, 2005). This work made the first approaches related to tools for sharing open resources.
It was then in 2006, with the publication of the work of Hylén (2006) addressing the opportunities and challenges of OER, that the theme has attracted the attention of researchers. In 2007, the evidence indicates that, as a result of investments in research by institutions such as the William and Flora Hewlett Foundation (Atkins, Brown & Hammond, 2007) and OECD (2007), the OER movement took on a greater theoretical foundation, boosting research in the following years.
Other organizations also made available publications disseminating the OER concepts and culture. For instance: UNESCO, in partnership with the Commonwealth of Learning, published a basic guide of OER (Butcher, 2011) and a Guide to OER in Higher Education (UNESCO/COL, 2011), both in the year 2011.  Table 3 shows the major sources of publications.

Main Sources of Publications
This work is licensed under a Creative Commons Attribution 4.0 International License.
9 As for conferences, the "Open Ed Conference" is worth mentioning. It took place in Barcelona on November 2010, organized by the Open University in the Netherlands and the Universidad Oberta de Catalunya (UOC), with 25 published papers on OER. Sixteen publications were also identified in another symposium, the "Regional Symposium on Open Educational Resources: An Asian perspective on policy and practices", that took place on September 2012 in Penang, Malaysia.
On book chapters, we must highlight the book "Open Educational Resources: An Asian Perspective", edited by Ajaraj Dhanarajan and David Porter and published by the Commonwealth of Learning and OER Asia, in which 8 of the 20 chapters were indexed in the queried databases.

Main Authors, Institutions and Countries
By analyzing the selected works' authorship, we identified the main authors and corresponding institutional affiliation (Table 4). Table 5 and Table 6 present, respectively, the institutions with the greatest number of authors and the corresponding countries.  From the data of the institutions to which the authors are affiliated, it was possible to generate a map ( Figure 3) showing a geographical fingerprint of OER research. The red points on the map show that the largest concentration of authors publishing about OER are from North America (specifically from the United States of America and Canada) and Europe (specifically, from the United Kingdom, Spain, Netherlands and Germany). Table 6 shows the number of authors by country.

Main Authors Keywords and Topics Addressed by the Publications
In the 544 publications considered in this study, 929 different keywords were used. The most used was "OER", used in 310 publications, "Open Course Ware" was used in 43, "e-learning" in 32, "Open Access" in 22, "Higher Education" in 20, "Open Education" in 19 and "Sustainability" was used in 17. Focusing on keywords related to distance education, 12.9% of the publications use the terms: e-learning, distance education, massive open online courses (MOOCs), distance learning, distance teachers, distance teaching or distance universities. The cloud in Figure 4 shows the most used terms as keywords in the analyzed publications. By analyzing the publications, it was also possible to identify the main themes (macro-themes) treated ( Figure 5). Those publications classified as "theoretical discussions", the authors cover different subjects, mainly the conceptualization of the word Open, as in Vaart (2013); the development of the OER as in Kernohan and Thomas (2012); the sharing culture as in D'Antoni (2008); the OER potentialities as in Wilson-Strydom (2009); and also the OER movement trends as in Jacobi et al. (2013).
In publications that address "quality", the authors discuss issues related to how to ensure the quality of the available materials, as in Clements and Pawlowski (2012). Considering the theme "barriers of use, difficulty and challenges", authors such as Chen (2010), Espinosa (2010) and 13 Banzato (2012) seek to point out the existing problems in the adoption of OER in a more meaningful way. In the "open education" theme, Pisutova (2012) and Smith (2009), for example, provide an overview about this topic. Regarding the subject "incentive policies", it is worth considering the works from Mora et al. (2008),  and Stacey (2013). On the theme "survey", the publications provide results of the research carried out mainly on the "use" of OER as in the work of Wright and Reju (2012), about "reuse" as discussed in papers by Hanna and Wood (2011) and Petrides et al (2008) and as "shared" in the work of Rolfe (2012).
This work is licensed under a Creative Commons Attribution 4.0 International License. 14 As far as the different "types" of OER are concerned, the authors approach the theme "textbooks", as reported in the work of Hilton III and Wiley (2010) and Hilton III, Wiley, and Lutz (2012); "OpenCourseWare", as the work of Johansen andWiley (2011) andVlǎdoiu (2011); and "MOOCs", as discussed in the work of Pisutova (2012).
The macro-theme "Sustainability" characterizes works which had the goal of studying different business models focused on financing OER initiatives, such as the works of Downes (2007), De Langen (2011 and Butcher and Hoosen (2012).
Finally, on the theme Open Licenses, emphasis is given to the type of "Creative Commons" licenses, dealing with the work of González and Olite (2010), Aliprandi (2011) andHagedorn et al. (2011); "copyrights", as mentioned in the OECD report (2007) and Downes (2011).

Main References Cited
From the 544 analyzed publications, 6,355 different bibliographic references have been used. Table 7 presents the main references used by the authors.
Sustainability is the central theme of the Stephen Downes article "Models for sustainable open educational resources" (Downes, 2007). It presents models of sustainability for the movement such as: Endowment, Membership, Donations, Conversion, Contributor-Pay, and Sponsorship.
The report "Open Educational Practices and Resources: OLCOS Roadmap 2012", (Geser, 2007), presents the Observatory of e-Learning Open Content. This observatory has the mission of mapping OER initiatives worldwide.
A special mention for the final report of the "Forum on the Impact of Open Courseware for Higher Education in Developing Countries", promoted in Paris in July 2002 by UNESCO in collaboration with the William and Flora Hewlett Foundation. It was where and when the term OER was adopted, in order to substitute other terms in use to express the very same concept.
The set of publications considered in this study aim to contribute to theoretical basis about the OER topic, addressing questions like: What are OER? Which business models can be used to provide sustainability for OER projects? How is it possible to reuse materials so that copyrights are respected? What are the barriers preventing educational resources to be made available in an open format? Which stimuli are necessary for greater engagement in the OER cause? How can the quality of openly available learning resources be assured? How is it possible to locate and deploy OER?

Conclusions
Through the application of bibliometric techniques this paper identifies the profile and characteristics of scientific publications that address the topic open educational resources, as well as the main conceptual aspects underlying the OER movement.
Data used to this bibliometric analysis was retrieved in September 2014 from the Scopus and Web of Science scientific databases, as well as from the OER Knowledge Cloud open repository. The results reinforced the idea that bibliometric research can help to identify the theoretical basis of an area of study.
From the 1,048 results obtained with the first search on the databases we excluded a total of 275 (26.2%), 40 were excluded due to the lack of author indication and 235 due to the fact that they were repeated. Additionally, from this total corpus of 773 publications, 26.3% were removed from the analysis since the authors or publishers did not make them available for full text download or could not be found in the Web. This restriction itself contradicts the philosophy of the openness movement that treats knowledge as a public good. This fact reinforces the idea that many authors on OER still consider to publish their research on journals that are not open access. There is, therefore, a lack of concern with the openness philosophy on how and where to publish.
Considering the author keywords analysis it is important to stress that 36.2% of the publications analyzed do not present any author keywords, which limited the full analysis of this topic. The lack of keywords reference by authors lowers and restricts the access to their publications, when using search engines and, afterwards, the access and sharing of knowledge.
It can be noted that 73.1% of the published reports on OER are concentrated in between the years of 2010 and 2013, which conveys the idea of an increase on research trends and main research sponsors and contributors in the last 4 years. However, we can notice that research is mainly held by European and North American countries. As result, it is necessary to invest on OER studies and culture throughout the continents, considering the singularities of each country.
2010 is the year when the largest number of publications were made available by the 13 main authors identified in this research. Nevertheless, when comparing the results from 2010 with the results from 2013, there is a drop of 56.4% in the number of released publications by the same authors. This fact can be understood as a turning point in the scope of their study, or as the closing down of some existing projects, or even as the absence of new ones.
When considering the scientific production of other researchers on OER, from other institutions, we note that these have kept on publishing. This tendency can be identified when comparing data from 2010 with data from 2013, noticing a drop from 50 to 21 in the number of publications made available by the affiliated authors of the Open University of United Kingdom. This tendency may not be interpreted as general since there are other examples that state an increase in the number of publications, as Athabasca University, which registers an increase of 2 results in 2010 to 15 results in 2013.
The bibliometric mapping on OER presented in this paper promotes the understanding of the research current state in international levels, providing: a) a brief historical overview of the evolution of this subject as a scientific field; b) the main sources of publication; c) the identification of authors, institutions and countries that lead the research on this topic since 2002; d) the main author keywords and topics addressed in the publications; and e) the main concepts that underlay the OER theoretical basis. This paper hopes to contribute for the highlight of main scientific references so that researchers interested in the OER subject, specially newcomers to the field, can find in it a basis to their studies, and a recommendation of reference publications that provide a sound theoretical basis, as well as inputs to the discussion about the current trends of the OER movement.