|
|
Andres Chiappe and Vivian Arias
Universidad de La Sabana, Colombia
This article describes both the process and the results of an extensive literature review on reusability as one of the key factors for the creation of open content in the context of open and distance learning. Qualitative analysis of texts was made from 1992 to early 2014 in order to identify meanings associated with reusability and its evolution over time. Results show that reusability is not a univocal concept and its meaning has evolved from its early years, especially focusing over the last decade on adaptation processes. Furthermore, the review indicates that reusability has traveled through a complex path from a technological to an educational context and the ideas related to it, in each case, projected different practical implications. A more profound understanding of such ideas is critical to face institutional approaches to create and reuse open educational resources as well as the design and implementation of open educational practices.
Keywords: Reusability; open educational resources; learning objects; information and communication technology; content analysis; educational technology
Since the mid 1990s, along with the massification of the Internet in which education began to seriously consider the inclusion of information and communication technologies (ICT) as a change factor in educational processes globally (Cabero Almenara, 2002; Hawkins, 2002).
In this sense, the global educational scenario and especially Latin America opened its eyes to the incorporation of ICT as an alternative to expanding educational coverage and strengthen its quality, particularly through the implementation of higher education programs in e-learning mode (McAnally Salas, 2006; Rama Vitale, 2010). These programs, because of its flexible and remote nature, are ideal for people who have difficulties in accessing campus, due to geographical location or other reasons related to time availability, family or labor issues, among others (Lupiáñez & Montoliu, 2005).
However this idea is simple in concept, its practical implications are quite complex due to the fact that the implementation of these internet-based programs require elaborate processes for teacher training, content creation, technological readiness, and strong administrative or academic adjustments within the universities. Furthermore, the development of digital educational content to be used in such programs is necessary, thereby increasing the cost of creating and deploying such programs.
It is within this context where reusability is emerging as a key concept. In fact, it is very tempting for working teams or teachers to avoid initiating the process of producing educational content from scratch. From this perspective, a productive scheme based on content reuse seems to be an alternative that opens up possibilities to the sustainability of the whole operation of creating educational programs based on the use of the Internet (Harasim, 1996; Kearsley, 2000).
Now, based on the aforementioned, since the beginning of the twenty-first century a lot of academic iniciatives have started worldwide, particularly associated with learning objects (LO), which from some of its most recognized conceptualizations have been considered as reusable educational materials (Boyle, 2003; Wiley, 2000). It is interesting to note that more than a decade after, most of the LO accessible via Internet is not the kind of material that someone can adjust to a new educational environment changing or replacing some of their components. So, if they are considered reusable in the first place, why can’t they be adapted? A clear misconception of reusability is causing confusion both in teachers and content creators about what open content should be.
In this regard, there are lots of reusability-related definitions in the literature about LO that mention a lot of different things, but, in fact, there is no consistent idea in the literature about what reusability is (Downes, 2001; Parrish, 2004). This is happening to the point that its meaning has become confusing and perhaps this is one of the main reasons why this kind of research review becomes a key issue for the academic community that studies the role of reusing content in the context of growing open education worlwide.
Moreover, it is clear that the role of learning objects has lost importance in recent years against a concept that includes them and is presented and conceptualized more clearly as open educational resources (OER). Atkins, Brown and Hammond (2007, p. 4) defined OER as “materials for teaching, learning and research, which reside in the public domain or have been released under an intellectual property license that permits their free use or repurposing for others.” This definition has been taken up by other authors, including UNESCO, who have added elements such as adaptation, redistribution and use without restrictions or limited restrictions, for purposes of licensing (Commonwealth of Learning, 2011).
At present, the use and production of open educational resources is a growing and emerging international educational trend (Scanlon, 2014; Whyte, Schmid, van Hazebrouck Thompson, & Oberhofer, 2014), which is part of what is known as “Open Educational Movement” which, in turn, meant to contribute to the strengthening of educational innovation through the promotion and exaltation of free access to science and knowledge.
This movement is committed to the free distribution and access to educational resources (Celaya Ramírez, Lozano Martínez, & Ramírez Montoya, 2010; Montoya & Aguilar, 2012), and is supported by international organizations such as UNESCO, OECD, the European Union and institutions of higher education across the globe.
With regard to the above, it is necessary to start removing personal and institutional barriers to enhance free access to knowledge and make visible the relevance of producing and sharing reusable digital educational resources in the form of content, tools, and learning activities.
Therefore, if an educational institution wants to advance steadily in open education, a more profound understanding of reusability must take place. This is why a literature review such as this was conducted, oriented from an educational perspective that allows, through a systematic analysis based on document tracking, a wider comprehension of the concept as a theoretical background (Guimarães, Moraes, & Guarido, 2007).
The literature review was developed as a process of content analysis, which was not based on quantitative identification of frequencies or co-occurrences but on observation of evidence of the meaning of reusability in the texts analyzed.
However, qualitative data emerged from the review, which enriched the overall qualitative analysis of texts selected for this study.
The review process began with setting preliminary search keywords in both English and Spanish: “reusability”, “reuse”, “reusable content”.
The results of the first search were subjected to a revision process of title, abstract and keywords to identify relevance and pertinence for a second and more in-depth review.
According to Dennis et al. (1995), to ensure the reliability and validity of the review, it is necessary to consider at least two major issues. First, the purpose of the review must be explicitly stated. This purpose was formulated as the need to identify the various meanings related to reusability and identify their evolution over time. Second, a coherent strategy was drawn up based on the previous issue. It was related to text inclusion and exclusion in the review. Within this strategy, inclusion of texts in both English and Spanish were considered relevant, and bibliographic databases, search terms and descriptors in both languages were included.
Based on the above, a documentary corpus was set using a text mining computer program called VantagePoint, which was applied to the most recognized academic databases related to reusability knowledge domain: ISI web of knowledge, Scopus, Scielo, EBSCOhost, ScienceDirect, Dialnet and Google Scholar, as meta-searcher.
This software allowed for the unification of duplicate records (the same item may be listed in more than one database) as well as the verification of the completeness and relevance of bibliographic metadata for each item.
As a result of this step, the universe of documents for this review was composed of 834 peer reviewed papers, from which a set of 150 items were randomly selected that equally covered a period of time ranging from 1992 to early 2014.
In that sense, when referring to the reuse of educational content, 1994 is considered a landmark because it was the year that academia started talking about learning objects (Polsani, 2003). Two years prior to 1994 they were included in selected papers with the purpose of identifying evidence of reuse of digital educational content before that special year.
A second key issue was the selection of information sources. In this regard, the literature review sought to reflect on a state of the art that would track the European, North American and Latin American thinking about reusability in which the choice of databases ensured this.
Each text selected underwent a process of re-reading from which segments of text with evidence of any meaning related to reusability were identified. After finishing this process, a chart-graph of the results was generated, which allowed for the identification of patterns in the data (common sets of meaning converted later in categories of analysis) from which the corresponding in-depth analysis was conducted.
In order to analyze the items so that they would be free of bias, two readings were done each by two different readers for the same text. These readers identified evidence presented in each analyzed text in relation to the categories of analysis formulated before.
Those readings produced two different records that were compared using Cohen’s kappa coefficient, whereby the consistency of the results were established.
The coincidence of the records relative to the observer A was 95% and observer B was 83%. Comparison of the two logs obtained a coefficient of 0.714 which represents a reliable observation (Barnhart & Williamson, 2002; Sim & Wright, 2005; Vieira, Kaymak, & Sousa, 2010).
Finally, after reading the selected papers, five categories of analysis were identified as the principal meanings related to reusability: “use again”, “granularity”, “adaptation”, “interoperability” and “aggregation or combine”.
The results of this in-depth review allow for the identification of some key elements of discussion about the reuse of digital educational content, especially related to their genesis and meaning transformation over time.
Under this category, reusability involves using content more than once without modification, in different contexts, so that the more generic and less contextualized content is, the higher the chances of reuse are. (Examples of this conception of reusability can be found in Johnson & Rees, 1992; Morris, Weisenford, & Boland, 1999; Organista-Sandoval, 2010; Wiley, 2003.)
This concept of reusability is the most common that was found in the literature. Of the 150 papers analyzed, 95 of them (63%) showed evidence of this conceptualization.
The first major benefit provided by the LO model is the one imported from OOP: reuse. A learning object designed by one person is made available to other instructors who can use them for different educational purposes. For example, a learning object that discusses how autos behave differently with and without anti-lock brakes might be used in several different educational domains: the physics of friction, automotive design, or insurance liability (Quinn & Hobbs, 2000, p. 14).
However, “use again” has been the most common conceptualization in the literature, it has gradually arousing the interest in those who research on this topic. This in turn can be seen as follows: the initial period (1992 to 2001) 60% of the items were published; in the following period (2002-2012) up to 40% of the publishing decreased. While in the earlier years, almost 90% of the published articles pertained to reusability in terms of “use again”, in recent years, this percentage has dropped to 44%.
Unlike the above, reuse in terms of granularity has nothing to do with the repeated use of a single content but as part of a larger content. In that sense, content is reused when added to other reusable content to create a larger content structure. (Examples of this conceptualization can be found in Dharinya & Jayanthi, 2012; Nelson & Poulis, 1995; Wiley, 1999; Wiley, Gibboons, & Recker, 2000.) Furthermore, for instance, the learning objects that are articulated to create topics, these are used to create lessons, and these can also be used to create modules, and so on. In this context, it is understood that the smaller the content, the greater the chance of reuse.
When several of the smallest elements are combined (e.g., into a web page) they become a “Level 1 resource,” and when several Level 1 resources are combined (e.g., into a web site) they become a “Level 2 resource.” Thus these organizations view the level of granularity of a learning object as the degree to which small media elements have been combined to comprise the larger learning object. This is a media-centric definition of granularity (Wiley et al., 2000, p. 2).
Of the five categories of analysis from which reusability was studied, the granularity has very little presence in the items analyzed. In fact only 25% of them evidenced the presence of granularity as an essential feature of reusing digital educational content. Moreover, it has been manifested as a much more interesting topic in recent years than in its origins. During the first decade of publications on reusability, only 32% of them mentioned granularity, while a decade later, this number rose to 68%.
This is the latest accepted meaning about reuse. To reuse content in different educational contexts it is necessary to modify the content to suit the new requirements of a new context. (Examples of this conceptualization can be seen in Agaba & Lubega, 2014; Bernardin, 2014; Bradley & Boyle, 2003; Cervone, 2012; Chiarani & Pianucci, 2006; Hodgins, 2006; Lieberherr, 1992; Šaloun, Velart, & Nekula, 2013; Wiley, 2010.) From this perspective, a reusable content should not only be thought about and designed so it can be edited by a third party, but must be produced with a technology that allows for this edition. This feature makes learning objects a clear example of an open educational resource.
The historically prevailing learning objects metaphors have limited our vision of reuse to “inclusion of a learning object in a novel aggregation” and prevented us from seeing reuse as “the possibility of changing the learning object itself in substantive ways. (Wiley, 2010, p. 1)
After “reuse”, “adaptation” is the second and most abundant category found in the review of literature on reusability. In the analyzed texts, 45% of them showed evidence for adaptation as a central or fundamental feature in the process of reuse of digital educational content. Their growing presence in literature (26% to 74%) indicates that over the years this began to be regarded not as a secondary aspect but as a representative feature of reusability as a concept.
This meaning is essentially technical and indicates that content reuse occurs when it can be displayed on different platforms or systems. (Examples of this are found in Chaabouni & Laroussi, 2014; Griffiths, Blat, García, & Sayago, 2004; Hodgins, 2002; Kim & Moon, 2013; Polo Poveda, 2011; Wiley, 2000; Wiley, 1999.)
“The learning content is the heart of the education, however it is locked into certain tools and platforms which make it impossible to reuse within new platforms” (Raju & Ahmed, 2012, p. 247).
This is the third most common category in the analyzed texts. It should be noted that the interest in this subject has grown significantly over the years, in parallel to the interest on adaptation. A growth in publications mentioning this feature as a key component of reusability jumps from 17% to 83%.
Following the Lego metaphor (Wiley, 2000), reusability involves the use of content to join with another thus forming a third product. (Examples of this are evidenced in Boyle, 2002; Boyle & Jones, 2009; Hoffman & Grossman, 1999; Parra Castrillón, 2011.)
The main difference with the approach from the granularity is that the reused content maintains its integrity and joins others to form larger units, while in the aggregation or combination each reused content integrates with others, setting a mixing process.
“Reusable learning objects are now being seen as the fundamental components and building blocks of on-line learning courses” (Oliver, 2001, p. 3).
Although it is one of the categories with the least amount of presence in selected publications, it is interesting to note that, as with the previous two categories, its presence in the literature has grown significantly (28% to 72%).
The literature review has revealed that reusability was not a univocal term over time and its different changes in its meaning significantly represents challenges and leads to interesting practical implications. For example, depending on how reuse is conceived, this will be the way in which work teams will trace their production tasks.
As mentioned previously, five key concepts were identified from which reusabilty has been understood. Those concepts became the five major categories of analysis that lead this study: use again, granularity, adaptation, interoperability, and aggregation or combined.
Of the 150 articles analyzed, 63% of them include the category “use again” within the concept of reuse; 45% of them include the category “adaptation”; 43% include the category “interoperability”; 33% include the category “aggregation”; and finally only 25% of the items analyzed included the category “granularity” as an essential element for understanding reuse.
The meanings associated with reuse have grown in complexity over time. Between 1992 and 2001, the vast majority of published papers (75%) identified reuse only with one or at most two of the categories of analysis. For the next period (2002 - 2012), a contrary result is shown where the number of articles published refers to three, four or five of the categories of analysis, more than double the others which only mention one or two.
From a historical perspective, it is possible to identify four main stages over the two decades included in the literature review in which the approach to reusability was clearly different.
A first stage corresponds to the period between 1992 and 1996 in which there was a clear tendency to understand reusability in terms of “use again”. In addition to this, a slight inclusion of adaptation as part of the meaning of reuse is perceived, which is a reflection of its relation with object oriented programming (OOP) (Lieberherr, 1992). This topic, for the moment, was in vogue and it was assumed as one of the key inputs for the formulation of learning objects in 1994.
In order to evaluate the usefulness of thinking about learning objects in terms of Computing Science (CS) programming techniques, we start by looking at past attempts to define learning objects, and then continue by looking at the ostensible link between learning objects and Object Oriented Programming theory. (Sosteric & Hesemeier, 2002, p. 2)
In accordance with the above and reviewing the timeline of the literature review, it appears that “adaptation” has been a key element in the concept of reusability along with “use again “. However, the first, unlike the second, lost prominence for a few years and reawakened in the early 2000s, associated with the revival of some of the principles of the free software movement (Adell & Bernabé, 2007; Stallman, 2004) and the emergence of high-profile initiatives such as OpenCourseWare (Abelson, 2008; Huijser, Bedford, & Bull, 2008; Long, 2002).
A second stage corresponds to the period between 1997 and 2000 in which more articles are published in association with the reuse of learning objects.
It is interesting to note that this relationship began in 1994 with the formulation of the term “learning objects” by Wayne Hodgins (Polsani, 2003), but it wasn’t until the end of 1996 and early 1997 that researchers started to publish numerous papers where it became clear that reuse began to break away from OOP and a new road was opened to introduce reuse in the realm of education, through learning objects (Hoffman & Grossman, 1999; Liu, 1997).
The third stage covers the period between 2001 and 2005, which marked a highly prolific and geographically diverse period in intellectual production in the field. At this stage about 60% of the recovered and analyzed papers were published in this study and also a variety of conceptual proposals for learning objects as reusable educational content were generated. In this third stage the great leading element was reusability as “interoperability” (65%), followed by “aggregation or combination” (35%).
In relation to this, it is interesting to note that although interoperability appears consistently over time as an element associated with reuse and that in this period appears to be particularly relevant, in its early years only original theoretical elaborations were produced on this subject; from there, the vast majority of texts cite these early elaborations, but no new theoretical developments in this matter arise (Castillo Cortés, 2009; Leyva Leyva & Tamayo Avila, 2008).
This is interesting because it allows inference of the high difficulty of reusability, which has to exceed the boundaries of the technological field and to advance to more practical and applicable elements in the educational context.
Finally, a fourth period between 2006 and early 2014 is configured in which it shows a holistic understanding of reusability to the extent that it is considered that there are multiple coexistent factors that characterized it (Moreiro González, Cuadrado, & Lara, 2012; Sanz-Rodriguez, Dodero, & Sanchez-Alonso, 2011). Reusing educational content then is no longer associated with only a particular factor, but in most cases it is considered as an activity that must serve different and not mutually exclusive synergistic dimensions, within which “adaptation” stands as the most appropriate.
In this case, the relevance of adaptation can be associated with the rise of open educational resources (Downes, 2007).
For an educational community immersed in incorporating ICT in education and more specifically in the context of open education, digital content production is clearly a matter of great importance and in this process, reuse of content plays a key role. As such, understanding reusability can lead and direct these processes to very different and innovative directions.
Available literature on this subject is quite extensive and diverse, heterogeneous in their concepts and often contradictory in their findings. The fact that reusability means so many different things and that there is no universal consensus about it dramatically increases the complexity of its implementation into educational institutions.
The analysis of the historical evolution of reusability indicates that its life cycle associated with learning objects is nearing the end of a downward curve, which would infer that it is an issue that is running out or is ceasing to be interesting for the academic community. Nevertheless, we should also recognize that reusabilty is still an interesting topic but now related to open educational resources.
Despite high production of theory regarding reuse of open content derived from English speakers, when a detailed analysis is made, some interesting issues emerge to the surface. For example, at the period of greatest production of papers, Latin American lines of thought converge with American and European regarding the relevance of adaptation as the contemporary engine of reusability.
Moreover, this convergence is then presented after a long period of time in which technological issues prevailed against educational. This is considered as one of the most likely causes why reusability, having a great application potential in education, has not generated major changes in academic practices within educational institutions.
Also, the results show that in terms of reuse, interoperability begins to lose importance and adaptation starts to gain this field from the year 2006, a period in which technological developments, mainly from web 2.0 tools, starts to provide opportunities for non-experts, not only to create potentially reusable educational content but to modify such content.
A milestone on reusability that arises in this same period of time and is inferred as another important factor to this situation is the overcrowding of the Creative Commons licenses (Leal Fonseca, 2008), which was made possible clarifying the legal scenario for adaptation of digital content through computer networks.
As a complement to this, it is possible to conclude that the limited availability of technological and legal tools supporting adaptive processes during the late 90s and early 2000s meant that adaptation was temporarily out of scene as one of reusability’s key factors.
In that sense, an important relation between OER, reusability and open licensing is rising especially for higher education context. Universities need to enhance their processes of digital content creation due to the increased amount of Internet-based programs offered. If OER starts to be a relevant issue for content creation, a wide understanding of reusability should lead to proper and sustainable processes that achieve fair indicators on this matter.
On the other hand, reusable digital educational content must be delivered with a clear set of rules regarding author-user relationship, for which Creative Commons licenses are good indeed. In the absence of clear rules about what a user can or cannot do with another author’s educational content, processes of use and reuse of it will be slow and unproductive.
Finally, regarding the prospects for the future developments of reuse it is interesting to note that the road in sight for development is linked to the growth and appropriation of open educational resources. In that sense, a new element not previously addressed or associated with reusability emerges as relevant: free access to educational content. Future studies should find the scope and limitations of free access to promote the enrichment of knowledge by way of their adaptation, customization and adjustment to new educational settings.
Abelson, H. (2008). The creation of OpenCourseWare at MIT. Journal of Science Education and Technology, 17(2), 164–174.
Adell, J., & Bernabé, Y. (2007). Software libre en educación. Retrieved from http://elbonia.cent.uji.es/jordi/wp-content/uploads/docs/Software_libre_en_educacion_v2.pdf
Agaba, J. E., & Lubega, J. T. (2014). On addressing design issues that hinder reusability of learning objects in instruction design. International Journal of Information & Education Technology, 4(1), 25–28.
Atkins, D. E., Brown, J. S., & Hammond, A. L. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities. Creative common. Retrieved from http://www.hewlett.org/uploads/files/ReviewoftheOERMovement.pdf
Barnhart, H. X., & Williamson, J. M. (2002). Weighted least-squares approach for comparing correlated kappa. Biometrics, 58(4), 1012–1019.
Bernardin, E. (2014). Business model for MOOCS: A perspective of reusability. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2014, pp. 164–170). Retrieved from http://www.editlib.org/p/148737/
Boyle, T. (2002). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, United Kingdom(19), 46–58.
Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19(1), 46–58.
Boyle, T., & Jones, R. (2009). Patrones de Objetos de Aprendizaje para la Programación. RED. Revista de Educación a Distancia, United Kingdom(10), 1–15.
Bradley, C., & Boyle, T. (2003). The development and deployment of multimedia learning objects. In Learning Technology Research Institute, London Metropolitan University, Learning Objects Symposium 2003 (Vol. United Kingdom, pp. 13–19). Honolulu, USA. Retrieved from http://www.aace.org/conf/edmedia/lo2003symposium.pdf
Cabero Almenara, J. (2002). La aplicación de las TIC: ¿esnobismo o necesidad educativa?. Red Digital: Revista de Tecnologías de La Información Y Comunicación Educativas, 1, 1–15.
Castillo Cortés, J. (2009). Los tres escenarios de un objeto de aprendizaje. Revista Iberoamericana de Educación, 50(1), 1–8.
Celaya Ramírez, R., Lozano Martínez, F., & Ramírez Montoya, M. S. (2010). Apropiación tecnológica en profesores que incorporan recursos educativos abiertos en educación media superior. Revista Mexicana de Investigación Educativa, 15(45), 487–513.
Cervone, H. F. (2012). Digital learning object repositories. OCLC Systems & Services, 28(1), 14–16.
Chaabouni, M., & Laroussi, M. (2014). A pedagogical cloud for reusability, interoperability and portability of pedagogical indicators. In The 2nd International Workshop on Learning Technology for Education in Cloud (pp. 161–174). Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-94-007-7308-0_17
Chiarani, M. C., & Pianucci, I. G. (2006). Objetos de Aprendizaje para Carreras Informáticas. In EduQ@ 2008 (pp. 1–7). Argentina.
Commonwealth of Learning. (2011). Guidelines for open educational resources in higher education. UNESCO, Commonwealth of Learning. Retrieved from http://unesdoc.unesco.org/images/0021/002136/213605e.pdf
Dennis, R., Ruiz, J. G., Ruiz, A., Rodríguez, N., & Lozano, J. M. (1995). Estándares metodológicos para revisiones de la literatura biomédica. Acta Med Colomb, 20(6), 263.
Dharinya, V. S., & Jayanthi, M. K. (2012). An approach towards redefining granularity of learning objects for effective and adaptive personalization. Journal of Theoretical and Applied Information Technology, 41(1). Retrieved from http://www.jatit.org/volumes/Vol41No1/13Vol41No1.pdf
Downes, S. (2001). Learning objects: Resources for distance education worldwide. The International Review of Research in Open and Distance Learning, 2(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/32
Downes, S. (2007). Models for sustainable open educational resources. Retrieved from http://nparc.cisti-icist.nrc-cnrc.gc.ca/npsi/ctrl?action=rtdoc&an=5764249
Griffiths, D., Blat, J., García, R., & Sayago, S. (2004). La aportación de IMS Learning Design a la creación de recursos pedagógicos reutilizables. In I Simposio Pluridisciplinar sobre Diseño, Evaluación y Descripción de Contenidos Educativos Reutilizables. México. Retrieved from http://spdece.uah.es/papers/Griffiths_Final.pdf
Guimarães, J. A. C., Moraes, J., & Guarido, M. D. M. (2007). Análisis documental de contenido de textos narrativos: bases epistemológicas y perspectivas metodológicas. Avances Y Perspectivas En Sistemas de Información Y Documentación En Entorno Digital. Zaragoza: Prensas Universitarias de Zaragoza, 93–100.
Harasim, L. (1996). Online education. Computer Networking and Scholarly Communication in the Twenty-First-Century University, 203–214.
Hawkins, R. (2002). Ten lessons for ICT and education in the developing world. The Global Information Technology Report 2001–2002: Readiness for the Networked World. Retrieved from http://www.azouk.com/193468/IT-Frontmatter.qxd/open#page=54
Hodgins, H. W. (2002). The future of learning objects. In e-Technologies in Engineering Education: Learning Outcomes Providing Future Possibilities (Vol. United States, pp. 76–82). Davos, Switzerland. Retrieved from http://dc.engconfintl.org/cgi/viewcontent.cgi?article=1012&context=etechnologies
Hodgins, W. (2006). Interview with Wayne Hodgins - Profetic. Retrieved from http://www.profetic.org/spip.php?article7949
Hoffman, B., & Grossman, S. (1999). Object learning on the World Wide Web. In WebNet World Conference on the WWW and Internet (pp. 511–516). Hawaii: Association for the Advancement of Computing in Education (AACE). Retrieved from http://www.editlib.org/p/7471/
Huijser, H. J., Bedford, T., & Bull, D. (2008). OpenCourseWare, global access and the right to education: Real access or marketing ploy? The International Review of Research in Open and Distance Learning, 9(1). Retrieved from http://www.doaj.org/doaj?func=fulltext&aId=250165
Kearsley, G. (2000). Online education: Learning and teaching in cyberspace. Wadsworth Thomson Learning Belmont, CA. Retrieved from http://www.utb.edu/vpaa/coe/Documents/Syllabi%202011-2012/Spring%202012/TLI/EDTC%203321%20Spring2012.pdf
Kim, K. R., & Moon, N. M. (2013). Designing a social learning content management system based on learning objects. Multimedia Tools and Applications, 64(2), 423–437.
Leal Fonseca, D. E. (2008). Iniciativa colombiana de objetos de aprendizaje: situación actual y potencial para el futuro. Apertura: Revista de Innovación Educativa, 8, 76–85.
Leyva Leyva, D., & Tamayo Avila, D. (2008). Repositorio semántico de objetos de aprendizaje para la reutilización de contenidos educativos. Presented at the VI Congreso Internacional Universidad 2008, Spain.
Lieberherr, K. J. (1992). Component enhancement: An adaptive reusability mechanism for groups of collaborating classes. In IFIP Congress (1) (pp. 179–185). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.37.8260&rep=rep1&type=pdf
Liu, W. (1997). Metadata applications in developing countries: The example of China. Retrieved from http://www.kevenlw.name/downloads/uploads/papers/LW4IYLIM(eng)revised.doc
Long, P. D. (2002). OpenCourseWare: Simple idea, profound implications. Syllabus, 15(6), 12–14.
Lupiáñez, F., & Montoliu, J. M. D. (2005). E-strategias en al introducción y uso de las TIC en la universidad. Revista de Universidad Y Sociedad Del Conocimiento, 2(1), 5–31.
McAnally Salas, L. (2006). La integración de la tecnología educativa como alternativa para ampliar la cobertura en la educación superior. Revista Mexicana de Investigación Educativa, 11(28), 11–30.
Montoya, M. S. R., & Aguilar, J. V. B. (Eds.). (2012). Movimiento Educativo Abierto. México: Crown Quarto. Retrieved from http://www.uv.mx/personal/albramirez/files/2012/05/REA_libro.pdf
Moreiro González, J. A., Cuadrado, S. S., & Lara, J. M. (2012). Mejora de la interoperabilidad semántica para la reutilización de contenidos mediante sistemas de organización del conocimiento. Encontros Bibli: Revista Eletrônica de Biblioteconomia E Ciência Da Informa\ccão, 17(33), 46–58.
Nelson, M. L., & Poulis, T. (1995). The class storage and retrieval system: Enhancing reusability in object-oriented systems. ACM SIGPLAN OOPS Messenger, 6(2), 28–36.
Oliver, R. (2001). Learning objects: Supporting flexible delivery of flexible learning. Retrieved from http://ro.ecu.edu.au/ecuworks/4781/
Parra Castrillón, E. (2011). Propuesta de metodología de desarrollo de software para objetos virtuales de aprendizaje -MESOVA. Revista Virtual Universidad Católica Del Norte, 34(1), 113–137.
Parrish, P. E. (2004). The trouble with learning objects. Educational Technology Research & Development, 52(1), 49–67.
Polo Poveda, A. (2011). Los objetos de aprendizaje: aprender y enseñar de forma interactiva en biociencias. (Spanish). ACIMED, 22(2), 155–166.
Polsani, P. R. (2003). Use and abuse of reusable learning objects. Journal of Digital Information, 3(4). Retrieved from http://journals.tdl.org/jodi/article/viewArticle/89/88
Quinn, C., & Hobbs, S. (2000). Learning objects and instruction components. Educational Technology & Society, 3(2), 13–20.
Raju, P., & Ahmed, V. (2012). Enabling technologies for developing next-generation learning object repository for construction. Automation in Construction, 22, 247–257. doi:10.1016/j.autcon.2011.07.008
Rama Vitale, C. (2010). La despresencialización de la educación superior en América Latina:?` tema de calidad, de cobertura, de internacionalización o de financiamiento? Apertura Nueva Época Digital, (6). Retrieved from http://colab.udgvirtual.udg.mx/ojs23/index.php/apertura4/article/view/82/94
Šaloun, P., Velart, Z., & Nekula, J. (2013). Towards automated navigation over multilingual content. Studies in Computational Intelligence, 418, 203–229.
Sanz-Rodriguez, J., Dodero, J. M., & Sanchez-Alonso, S. (2011). Metrics-based evaluation of learning object reusability. Software Quality Journal, 19(1), 121–140.
Scanlon, E. (2014). Scholarship in the digital age: Open educational resources, publication and public engagement. British Journal of Educational Technology, 45(1), 12–23.
Sim, J., & Wright, C. C. (2005). The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Physical Therapy, 85(3), 257–268.
Sosteric, M., & Hesemeier, S. (2002). When is a learning object not an object: A first step towards a theory of learning objects. The International Review of Research in Open and Distance Learning, 3(2). Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/106
Stallman, R. (2004). Software libre para una sociedad libre. Madrid: Traficantes de Sueños, 2004. Retrieved from http://bibliotecalibre.org/handle/001/144
Vieira, S. M., Kaymak, U., & Sousa, J. M. (2010). Cohen’s Kappa coefficient as a performance measure for feature selection. In Fuzzy Systems (FUZZ), 2010 IEEE International Conference on (pp. 1–8). IEEE. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5584447
Whyte, S., Schmid, E. C., van Hazebrouck Thompson, S., & Oberhofer, M. (2014). Open educational resources for CALL teacher education: The iTILT interactive whiteboard project. Computer Assisted Language Learning, 27(2), 122–148.
Wiley, D. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. The Instructional Use of Learning Objects: Online Version. United States. Retrieved from http://reusability.org/read/chapters/wiley.doc
Wiley, D. (2010). Impediments to learning object reuse and openness as a potential solution. Revista Brasileira de Informática Na Educacão, 17(3), 8–10.
Wiley, D. A. (1999). Learning objects and the new CAI: So what do I do with a learning object? (United States). Retrieved from http://opencontent.org/docs/instruct-arch.pdf
Wiley, D. A., Gibboons, A., & Recker, M. (2000). A reformulation of the issue of learning object granularity and its implications for the design of learning objects. United States. Retrieved from http://reusability.org/granularity.pdf
© Chiappe and Arias