The ILIAS learning management system (LMS) was evaluated, following its favourable rating in an independent evaluation study of open source software (OSS) products. The current review found ILIAS to have numerous features of value to distance education (DE) students and teachers, as well as problems for consideration in the system’s ongoing development. The current findings were compared with DE students’ reactions to a similar LMS product, ATutor, also rated highly in the independent OSS evaluation. In comparing an ATutor course website with a simple HTML-based version of the same site, the ten students voted unanimously to retain the simpler site. This result is consistent with previous evaluation findings in the current series of reports, and indicates that increasing integration of product features does not necessarily improve a product’s ease of use or educational effectiveness.
As the current series of reports has indicated, the number of online learning management systems (LMS) has proliferated, and at least a hundred are currently available. Leslie (2004) has identified over 50 such systems based on open source software (OSS) alone. In evaluating 35 OSS systems, Farrell (2003) has rated ATutor and ILIAS as the best, with Moodle close behind. The current evaluation studies have so far evaluated Moodle (Report #25) and ATutor (Report #37), rating both products favourably with respect to their potential value in distance education (DE). The current report completes this trilogy of reviews by examining the DE merits of Ilias. In deciding the criteria for its review, the evaluation team reviewed the LMS selection standards used by (1) the American Society for Training & Development (ASTD: see previous reports in this series); (2) the Commonwealth of Learning evaluation study by Farrell (2003); and (3) the software comparison service, Edutools.info (2003). The criteria considered the most appropriate were combined and some new ones added. The final set of criteria is considered to represent the judgments that administrators, facilitators, designers and end users need to make in determining the value and usefulness of an LMS in DE. In addition, previous online opinions of ILIAS were considered in the light of the version currently reviewed (v. 3.2.0.). Following the ILIAS review, the report also examines the trends that have evolved in the review series so far, with respect to DE students’ reactions to complex, fully-featured LMS platforms, in comparison with those described in Report #8 as “lean and mean”. In keeping with the convention adopted earlier in this series (Report #25), the LMS acronym is adopted in order to avoid the confusion of terms currently arising in the educational software literature.
ILIAS is a free, open source (OS) Web-based learning management system (LMS) developed at the University of Cologne (2004). Originally released in July 2001, with early versions dating from mid-2000, ILIAS stands for Integrated Learning, Information and Co-Operative Work System. It has been designed as a collaborative, hypermedia working/ learning environment, flexible for use by teachers with different pedagogical approaches, and allowing the reuse and exchange of materials based on course-sharing standards (e.g., IMS, The Aviation Industry CBT [Computer-Based Training] Committee (AICC) and ARIADNE Foundation for the European Knowledge Pool). As the previous report in the series (Report # 42) has indicated, all OSS is by definition a work in progress, constantly enhanced and modified as a result of user input and collaboration. ILIAS is no exception. Its website currently reports its use on over 100 servers, mainly at Universities in Europe and Asia. In the OSS sharing spirit, therefore, the current evaluation team contacted the four listed North American users with inquiries about their ILIAS experience (though none responded).
Particular aspects of ILIAS that support the moderate constructivism philosophy are:
ILIAS is a cross-browser system, requiring Internet Explorer 6.0, Mozilla 1.4, Safari 1.0, and/ or Lynx 2.8.4 (or higher), and runs with JavaScript disabled. The system is frames-based, cannot support text-only navigation, and is not usable with large fonts. It requires MySQL 4.0.16 or higher. A digest of the product’s features is as follows:
ILIAS currently suffers from limitations common to many OSS products. These include: lack of interoperability and integration; lack of support for audio/ video-conferencing, time-zone or date localization; poor accessibility standards; and an unclear estimate of the true cost of ownership. Currently, many OS LMS platforms lack integration with the applications typically used for registration and employee management in universities, colleges, and industry (Bailetti, 2004). The same writer also notes that OSS systems tend to lack good user, developer and administration documentation; and this is true of ILIAS. Although the software contains a 13-language translation facility, up-to-date documentation for it is available only in German; non-German users have little chance of influencing the product’s development via participation in the various help forums. (On the other hand, Farrell (2003) notes a growing following for the product in China.) While ILIAS is classified as freeware, its use is not actually cost-free. The time spent in installing and maintaining the system must be added to the costs of the hardware, technical support and training; and the current evaluators (both experienced server administrators) found the logistical problems of ILIAS installation to be considerable. They spent several days trying to install and configure it on Linux, in conjunction with its dozen or more required third party applications. Poor English documentation contained conflicting instructions. As a last resort, the messages in the German-language ILIAS user forums were scoured, until an English description of a Windows-based installation alternative was found. With this, ILIAS was working on a Windows/2000 server within 30 minutes. Despite its many current shortcomings, it has an admirable range of features, pedagogical integrity, and good potential for DE users. OSS applications such as this have come a long way within the three years of the current evaluation series, and they are continuing to advance quickly.
Previous reports in this series have indicated, however, that fully-featured online tools do not necessarily find favour with DE users. Students can find them difficult to master, and DE teachers do not necessarily share a need for the many features they offer. In addition, their integration with existing institutional systems can be expensive, and the overall costs of proprietary systems especially, may be excessive in the relation to the actual uses to which they are subjected. In Report #8 in this series, a direct comparison was made between DE students reactions to a fully-featured conferencing product and a simple ‘bare-bones’ alternative. Unexpectedly, a majority of students preferred the simple application, and regarded a transfer to the more complex option as not worth the effort. In the two years since that study, online collaborative tools have converged within ambitious integrated products such as WebCT Vista, and in OSS applications (e.g., ATutor, ILIAS, and Moodle). Having now established that each of these products has a wide range of features of use to DE teachers and students, the next section of this report replicates the earlier study, by asking DE students to compare the comprehensive open-source ATutor product with the more limited HTML-based site of their current online course.
Ten students in a graduate Masters in Distance Education (MDE) course at Athabasca University – Canada’s Open University, were asked to compare the simple, home-grown, frames-based format of their current course website (see Figure 1) with that of the OSS LMS ATutor. The content of the course site was replicated in ATutor (see Figure 2), using the free trial and hosting facilities of the product’s website. ATutor was selected for this purpose in view of its top rating by Farrell (2003), and its favourable evaluation in Report #37 of this series. The current study fulfilled a promise stated in that report, that ATutor would be adopted for the MDE course depending on feedback from its current students as a whole.
Figure 1. The frames-based format of an online Master’s in Distance Education course website
Figure 2. A sample configuration of ATutor (same course as Figure 1)
A representative selection of the students’ responses to the two course platforms is as follows.
Despite ATutor’s clear technical superiority in comparison with the existing HTML-based course site, the majority of students voted to retain the existing frames-based site shown in Figure 1. None voted to move to the ATutor platform. This result was disappointing to the course instructor, who had hoped for student feedback justifying a move to the superior support that would be available for ATutor in the University. A common theme in the students’ responses is their reaction to the ATutor display as too “cluttered/ busy” with too many “obstacles/ bells and whistles.” These responses are essentially the same as those reported in the similar study reported earlier in the series (Report #8). The clearest explanation for this preference for a simple course delivery format was provided in the present study by a student who stated: “I see advantages to both, but nothing compelling enough to make a change.”
The recurring finding of these studies brings into question the wisdom of providing DE students with ever more complex online learning systems; and it suggests the need to balance comprehensiveness of product features against clarity of usage. In the absence of unequivocal justification for moving to a new learning platform, both teachers and students are apt to select “the devil they know”. If new systems have definite advantages, these should be carefully pointed out to them. It is inevitable that the developers of online software (OSS as well as proprietary) will continually offer “bigger and better mousetraps” on the assumption that the more features their products contain, the more competitive they will be. OSS as well as proprietary software developers are moving in this direction; and the increasing comprehensiveness of LMS software such as ILIAS and ATutor is evident from the current product reviews and those provided by Farrell (2003) and edutools.info (2003). Reviews based solely on the volume of a product’s features, however, do not necessarily predict users’ reactions to the product or its educational effectiveness; and it is clear that simple preferences for proprietary or OSS approaches do not predict the educational merits of a product either. DE students’ reactions to the highly rated ILIAS and ATutor products, as expressed in the present report, suggest that too much product integration can be counter-productive in the online learning process, and that teachers and students may both benefit from a simple, modular-based approach that allows them to expand their online tools and skills as and when they need to do so.
The authors thank the members of Athabasca University’s MDDE662 course (Fall 2004), for their comments quoted in this study.
The next report in the series will continue the current discussion of attitudes to LMS methods.
N.B. Owing to the speed with which Web addresses are changed, the online references cited in this report may be outdated. They are available, together with updates to the current report, at the Athabasca University software evaluation site: http://cde.athabascau.ca/softeval/. Italicized product names in this report can be assumed to be registered industrial or trademarks.
JPB. Series Editor, Technical Evaluation Reports
Bailetti, T. (2004). Improving Open Source Learning Content Management Systems. Article found at Eduforge.org website. Retrieved November 2, 2004 from: http://eduforge.org/wiki/wiki/eduforge/wiki?pagename=Improving%20Open%20Source
Edutools.info (2003). ILIAS Review. Retrieved November 2, 2004 from: http://www.edutools.info/course/productinfo/detail.jsp?id=203
Farrell, G. (2003). COL LMS Open Source. Paper found at the Commonwealth of Learning website. Retrieved November 2, 2004 from: http://www.col.org/Consultancies/03LMSOpenSource.pdf
Leslie, S. (2004). Open Source Course Management Systems. EdTechPost
Blog. Retrieved November 2, 2004 from: http://www.edtechpost.ca/pmwiki/pmwiki.php/EdTechPost/Open
SourceCourse
ManagementSystems
University of Cologne (2004). Didactical concept of ILIAS. Retrieved November 2, 2004 from: http://www.homer.ilias.uni-koeln.de/iliasdoc/doc/html/6_1.html