A product evaluation was conducted of Desire2Learn, an online learning management system gaining popularity in educational institutions. Since an online trial version of the software was not available for evaluation, an inspection meeting with the vendor was arranged. This provided the evaluation team with an opportunity to examine the precautions necessary for ensuring the objectivity of a product evaluation based on vendor-supplied information. The report outlines the team’s use of evaluator-driven enquiry and a triangulated approach to information checking. The Desire2Learn product is assessed as a flexible and useful addition to the range of learning management system options.
A previous report in this series (#25) adopted the convention of grouping course and content management systems under the heading Learning Management System (LMS) to avoid confusion between CMS systems. Proliferation of such acronyms is just one example of the types of confusion currently reigning in this rapidly advancing field. Report #10 in the series described another source of confusion, relating to the lack of objectivity that can arise when software is evaluated with the assistance of its developer/vendor. Often there may be no choice in the matter, when a trial version of the product is not available for inspection. The previous report listed the precautions required when the evaluator is obliged to seek the vendor’s assistance in order to gain access to the product. The current study provides a direct example of this situation, in describing an LMS evaluation conducted via a face-to-face demonstration by the vendor in a Vancouver educational institution. The report discusses the precautions taken by the evaluation team in maintaining its objectivity throughout this process, and the outcome of the study.
Desire2Learn (D2L) is an integrated online learning management system (LMS), which provides synchronous and asynchronous interaction between students, teachers, and learning content. Such platforms are usually integrated with administrative databases (e.g., commercial products such as SCT Banner, Peoplesoft, Datatel, and Campus Pipeline).
Information for possible use in evaluating the product was collected in September and October 2003, from varied sources, including:
Prior to the visit to the D2L representative by two of the three-member evaluation team, steps were taken to ensure the depth and objectivity of the information that would be obtained. The caveat emptor recommendations of Report #10, were followed in this connection. In addition to recognizing the need for independent corroboration of the data collected, the evaluators knew they should request the evidence for unsubstantiated claims and should be prepared, if doubtful of any claims, to challenge the vendor to prove them. Objective technical information was sought, where necessary, to replace merely qualitative product descriptions.
Highlights of the information-gathering process were as follows:
The ‘forewarned-is-forearmed’ approach was considered beneficial, and it is gratifying to report the company responded obligingly to the evaluation team’s approach at all times. (In fact, the D2L company’s reputation for a high level of customer service had already reached us by word-of-mouth before the evaluation was conducted.) The D2L representative was open and forthcoming, receptive to feedback and committed to our discussion of educational criteria.
The D2L software has a startup cost based on the number of features, the level of integration with existing systems, and the support needs of the institution. The product has the capability to be used as a course-by-course delivery tool, or scaled up to a larger entity such as a province-wide license. Annual licensing costs are based on a flat fee or on a per-student basis. Users are not required to download any software. The company’s servers contain all the software to support their LMS platform with the exception of Java, which is used under the license. This minimal use of third-party software enables the company to give flexible responses to clients’ requirements. The vendor provides access to a support person per 5000 students covered by the license.
The product is essentially asynchronous in its approach to online collaboration; at the time of the evaluation, it does not contain synchronous tools such as audio, whiteboard, instant polling, and co-browsing. Its asynchronous conferencing tools are flexible for both administrators and users. The licensing approach allows administrators to assign roles across multiple organizations to facilitate a consortium approach. Administrators can create an unlimited number of customized user roles, with specific privileges for instructors and students. D2L is XML standard-based, and uses a learning-object repository approach that allows instructors to store content (i.e., pages, graphics and test-item banks, etc.). These objects can be shared across any number of courses or programs. (By contrast, other prominent LMS systems use hard-coded HTML pages uploaded to a specific version of a course.) Administrators can assign students the ability to search for specific course content and discussion threads in the learning-object repository. A context-sensitive help feature provides help on each of the major tools for both developers and students. The software is compliant with leading international course-sharing standards including SCORM and IMS, and meets international standards for access by disabled users. D2L databases can be configured to communicate directly with other institutional databases.
The customizable features of D2L provide administrators with straightforward means of creating conferences for the members of a class or its sub-groups. Students can be assigned access privileges for system-wide text-chat rooms. Once conferences are created for an individual course, new conferences can easily be created for subsequent offerings of the course. Within a specific course offering, administrators can edit or delete threads, and can control the access privileges of users at different levels, including the right to edit and/ or delete one’s own messages. Threaded messages are displayed clearly, and users have control over keyword searches and message sortings by date, topic, and sender. Other options include saving, printing, editing and deleting of postings.
A primary strength of D2L is the breadth and adaptability of its features. It is a fully integrated learning management system that has evidently evolved through close relationships between the software developer and educational institutions. Because D2L is XML standard-based, it is relatively easy to integrate it with other database systems. The product has a high degree of usability. Its course structure and appearance are easy to manipulate, and administrative control over the usage process is broad and easily specified. Owing to its high degree of flexibility and customizability, the product can take many forms. For this reason, the evaluation team recognized the vendor’s preference for demonstrating its product in person, on an individualized basis rather than via a more generalized online demonstration approach
The authors thank Ben Moore of the Desire2Learn company for his assistance in providing information about this software product; also Bryan Fair, evaluator of Desire2Learn at the British Columbia Institute of Technology.
The next report in the series describes the use of synchronous audio methods in a cross-cultural setting.
N.B. Owing to the speed with which web addresses become outdated, online references are not cited in this report. They are available, together with updates to the current report, at the Athabasca University software evaluation site: http://cde.athabascau.ca/softeval/. Italicised product names in this report are assumed to be registered trademarks.
JPB, Series Editor, Technical Evaluation Reports