A wide range of projects and organizations is currently making digital learning resources (learning objects) available to instructors, students, and designers via systematic, standards-based infrastructures. One standard that is central to many of these efforts and infrastructures is known as Learning Object Metadata (IEEE 1484.12.1-2002, or LOM). This report builds on Report #11 in this series, and discusses the findings of the author's recent study of ways in which the LOM standard is being used internationally.

Characteristics of Learning Object Metadata Surveyed

“Metadata” refers to systematically created and formatted descriptions of resources, intended for learning, informational, or other purposes. The LOM standard has become the most widely used solution for classifying and describing digital resources intended specifically for learning and education. It is only one way of describing digital and online resources, however. Other metadata standards and methods have been developed for the same purpose, including Dublin Core and the Rich Site Summary (RSS): see Report #11 in this series. A common feature of these standards and methods is the fact that each defines the function and structure of a number of data elements. Examples of these include the title, author, and location of the resource. RSS, for example, focuses on three of these data elements - title, link, and description; while Dublin Core specifies only 16 metadata elements. The LOM standard, on the other hand, includes 76 data elements, covering wide-ranging characteristics attributable to LOs, including their size, level and type of interactivity, and the educational context to which they are best suited.

The LOM defines all of its data elements in interrelationships that are both hierarchical and iterative. At the top of the hierarchy of LOM elements are nine broad category elements: General, Lifecycle, Meta-metadata, Technical, Educational, Rights, Relation, Annotation and Classification. The category elements each contain sub-elements, which, in turn, often contain further sub-elements. Many of the category elements, sub-elements, and subordinate elements can be repeated. This results in complex hierarchical and iterative structures, allowing for a total of over 16,000 possible, concatenated element repetitions. Some of the sub-elements in the LOM (e.g., the title element) can be assigned an alphanumeric value. Other elements are associated with a limited set of pre-defined values (e.g., describing educational context such as school, higher education or training). In this last case, the set of values is often referred to as a “vocabulary” or “controlled vocabulary.” Still other elements in the LOM contain descriptions of persons (authors, editors, etc.) that are specially formulated and formatted using a specification known as vCard.

Given its relative size and complexity, as well as the fact that it is the first technical e-learning standard to be widely adopted, the implementation of the LOM presents an excellent opportunity for study and research. By looking at how it has been implemented in projects and in specific metadata records, it is possible to learn valuable lessons about e-learning standards implementation, and about how to develop and refine further standards to meet implementers' and educators' needs.

The current report presents the basic findings of an international survey of the implementation of the LOM standard. This survey was undertaken as a part of ongoing Canadian work in an international e-learning standardization forum: the ISO/IEC (International Standards Organization/International Electrotechnical Commission) subcommittee on Information Technology for Learning, Education and Training. The survey was conducted in two phases. The first involved the manual analysis of very small sets of randomly selected metadata records from a variety of collections and projects. The second stage involved the statistical, aggregate analysis of much larger sets of sample records, taken from five large collections from widely varying regions, including the European Union, Canada, and China. The findings of both stages of the survey were consistent and mutually reinforcing (see below). Only general findings and conclusions are reported in this paper. More detailed survey data and analysis are available in the original survey reports, submitted to the ISO/IEC committee (Friesen and Nirhamo, 2003; Friesen, 2004).

Survey Questions

The survey of LOM implementation was guided by three specific questions. Each question relates to the data elements of the LOM, and to the way in which each element is understood and used (or alternatively, not used). These questions, and their contextualizing explanations, are provided here:

1. Which elements are being designated for use in LOM implementations? As a first step in implementing the LOM, organizations, projects, consortia, and national entities will frequently designate a particular set of LOM elements for use in their respective d'mains. Such localized sets of elements are called application profiles, and are often created in a process separate from technical implementation, as a matter of policy. Elements are explicitly recommended, required, or excluded from use. These policies are often applied in both e-learning content development and the creation of infrastructures to support the exchange of this content. Such an element set can include custom elements (element extensions), adding new elements to the 76 already in the LOM. More often, however, a subset of LOM elements is chosen, reducing the number of LOM elements, often by as much as a half.
   
   
2. Which elements are actually used in metadata records? Regardless of the elements required, recommended or excluded in application profiles and policy documents, the elements that are actually used provide additional information about element utility and metadata requirement. Of those elements actually populated, some may be utilized consistently, and repeated in order to have a range of appropriate values assigned to them. Others may appear only once, with an apparently arbitrary value assigned to them.
   
   
3. What values are assigned to these elements? Finally, when elements are used, it is important to determine how they are actually applied to the needs of individual projects and resources. Quantifying the kinds of values assigned to elements can be difficult in some cases; but those elements with controlled vocabularies and value sets that are otherwise constrained (e.g., through the use of vCard) can be analyzed quite readily.

Findings

The findings of the current survey are presented as responses to each of the three questions raised above.

1. Which elements are being designated for use in LOM implementations? The survey has shown that, in many cases, the elements designated for use in application profiles overlap with those already designated in the smaller, simpler metadata element sets represented by Dublin Core and RSS. In addition, educational elements in the LOM (those aspects of the data model that add obvious special value for educational applications) are frequently not designated for mandatory use in application profiles. Given some of the findings discussed below, this raises the question of whether the challenges and costs presented by LOM implementation are readily offset by its benefits – especially in comparison with alternative metadata solutions such as Dublin Core.
   
   
2. Which elements are actually used in metadata records? The answer is essentially the same as the first one (above), with some qualifying details identified in the survey data. The elements actually populated in the metadata records studied can be characterized as focusing on the intellectual content of the resource. Many of these elements have rough or exact equivalents in the Dublin Core Metadata element set. The same can be said for those elements which describe the resource in terms of its characteristics as a media and Internet file: they are well-utilized and also correspond to elements in the Dublin Core element set. Those elements which attempt to describe the resource as a software object, or to associate with it an educational context or level, are much less frequently utilized. This is reinforced by vocabulary values which are used to identify contributions to the creation of the resource - i.e., the roles of author and publisher were well-utilized (together constituting over 95 percent of the roles or values chosen), but roles associated with software, instructional design, and media development (e.g., initiator, terminator, graphical designer, instructional designer) were ignored.
   
   
3. What values are assigned to these elements? Again, the finding is in keeping with the answers to the first two questions. In many cases, elements with controlled vocabularies were assigned values that reflected traditional, even print-oriented understandings of the resource as a 'ublished asset, rather than as a modular software object. These elements include not only the roles of contributors to the object (as above), but also the many values which can be assigned to indicate the resource's technical format (45 percent of which were indicated as “text/html”).

A number of other findings pointed to issues additional to those raised in the questions above.

1. A surprising result was observed regarding the process whereby metadata records are combined from a wide variety of collections into a single collection for aggregate analysis. It was found that it is very difficult or, given limited resources, actually impossible to import the various records into a single database, and then to perform database queries to discover divergent and common characteristics. This seems to have been the case in other, more limited survey efforts (e.g., Najjar, Ternier, and Duval, 2003). LOM structures, with their hierarchical and iterative interrelationships, make data portability difficult to realize using conventional, low-cost technologies. Data portability and reuse is presumably the raison d'tre of the LOM. The difficulties the LOM presents to educational implementations in this regard are not positive indicators of the prospect of increased sharing and reuse between implementations and across jurisdictions.
   
   
2. Very little of the complexity and detail that vCard information can supply about contributors is actually exploited (almost 90 percent of the vCard fields were unused in all instances studied). Any advantage that the inclusion of vCard might present in LOM records, is far outweighed by the difficulties of its implementation, and the under-utilization of vCard fields.
   
   
3. Only a small number of the potential element iterations and vocabulary values were used overall. This is unfortunate. Given the difficulties that these nested iterations and vocabulary choices can present to systems developers and record creators, the fact that few are used is cause for concern.

Conclusions

What do these findings mean for learning object implementation, and for the many projects and initiatives where learning object metadata are being used? On a positive note, the survey has revealed considerable convergence among implementations in element choice and utilization. Implementers have consistently opted to use roughly the same subset of elements, focusing on the description of the intellectual content of the resource. The fact that these same elements are also included in other, simpler metadata solutions, however, raises an important question: “What is the value added by the multiplicity and complexity of elements and element structures in the LOM?” The fact that a range of elements, and many of the possible element iterations in the LOM, remain unused means that their value is not being realized. At the same time, the price paid for this complexity and multiplicity, in terms of implementation work and data portability issues, is appreciable. These conclusions suggest that a very considerable return on learning object investment will be required for profit ultimately to accrue to learners and end-users.

———————————————————————

The next report in the series examines recent developments in the WebCT course management system.

N.B. Owing to the speed with which Web addresses are changed, the online references cited in this report may be outdated. They can be checked at the Athabasca University software evaluation website: http://cde.athabascau.ca/softeval/. Italicised product names in this report can be assumed to be registered trademarks.

JPB Series Editor, Technical Evaluation Reports

References

Friesen, N. (2004). Final Report on the International LOM Survey. Retrieved 20 October, 2004, from: http://dlist.sir.arizona.edu/archive/00000403/

Friesen, N., Nirhamo, L., and Knoppers, J. (2003). International Survey of Learning Object Metadata Implementation for ISO/IEC. Retrieved 20 October, 2004, from: http://mdlet.jtc1sc36.org/doc/SC36_WG4_N0029.pdf

Najjar, J., Ternier, S., and Duval, E. (2003). The Actual Use of Metadata in ARIADNE: an empirical analysis. Proceedings of the 3rd Annual Ariadne Conference. Retrieved 20 October, 2004, from: http://www.cs.kuleuven.ac.be/~najjar/papers/EmpiricalAnalysis_ARIADNE2003.pdf