April - 2004

Technical Evaluation Report

30. Vendor-Assisted Evaluation of a Learning Management System

Patricia Fahrni, Jim Rudolph, and Adrienne De Schutter
Master's of Distance Education Program
Athabasca University - Canada's Open University


A product evaluation was conducted of Desire2Learn, an online learning management system gaining popularity in educational institutions. Since an online trial version of the software was not available for evaluation, an inspection meeting with the vendor was arranged. This provided the evaluation team with an opportunity to examine the precautions necessary for ensuring the objectivity of a product evaluation based on vendor-supplied information. The report outlines the team’s use of evaluator-driven enquiry and a triangulated approach to information checking. The Desire2Learn product is assessed as a flexible and useful addition to the range of learning management system options.


A previous report in this series (#25) adopted the convention of grouping course and content management systems under the heading Learning Management System (LMS) to avoid confusion between CMS systems. Proliferation of such acronyms is just one example of the types of confusion currently reigning in this rapidly advancing field. Report #10 in the series described another source of confusion, relating to the lack of objectivity that can arise when software is evaluated with the assistance of its developer/vendor. Often there may be no choice in the matter, when a trial version of the product is not available for inspection. The previous report listed the precautions required when the evaluator is obliged to seek the vendor’s assistance in order to gain access to the product. The current study provides a direct example of this situation, in describing an LMS evaluation conducted via a face-to-face demonstration by the vendor in a Vancouver educational institution. The report discusses the precautions taken by the evaluation team in maintaining its objectivity throughout this process, and the outcome of the study.


Desire2Learn (D2L) is an integrated online learning management system (LMS), which provides synchronous and asynchronous interaction between students, teachers, and learning content. Such platforms are usually integrated with administrative databases (e.g., commercial products such as SCT Banner, Peoplesoft, Datatel, and Campus Pipeline).

Information for possible use in evaluating the product was collected in September and October 2003, from varied sources, including: