August - 2004

Technical Evaluation Report

33: Evaluating Digital Authoring Tools

Russ Wilde
Masters of Distance Education Program
Athabasca University – Canada’s Open University

Abstract

As the quality of authoring software increases, online course developers become less reliant on proprietary learning management systems, and develop skills in the design of original, in-house materials and the delivery platforms for them. This report examines the capabilities of digital authoring software tools for the development of learning materials. Seven software tools are evaluated, representing the following categories: single purposes; activity creation; course development and presentation; general presentation; testing and assessment.

Evaluating Authoring Tools

Despite debate over the efficacy of multimedia, computer and Web-based learning materials (Clark, 1983; Greenagel, 2002), there is compelling evidence that they can be an effective and efficient means of delivering education at a distance in the right situations and conditions (Kirkwood, 1998; Bates, 2000; Mayer, 2001). The development of high-quality digital learning materials, however, involves a complex combination of subject matter expertise, instructional design knowledge, and technical skills (graphic design, photography, audio, video, computer programming, etc.). These result in long and expensive curriculum development cycles (Bates, 2000). Numerous “learning management system” (LMS) products have become available for the purpose of streamlining the process of online courseware creation and delivery. These tend to be expensive, however, and do not always give the students and teachers the flexibility they require. By contrast, a wide range of inexpensive courseware authoring tools has emerged, enabling courseware designers to develop effective in-house materials and delivery platforms.

For the purpose of this evaluation, seven authoring products were selected, representing the following uses:

1) Single Purpose Tools: designed for basic purposes rather than for creating a suite of varied tools. Most tools in this category are not specifically designed for production of instructional materials.
2) Activity Creation Tools: designed to produce small, stand-alone, interactive activities that may be incorporated into courses.
3) Course Development and Presentation Tools: specifically designed for developing and presenting online courses and training programs. These tools are typically organized around specific concepts, lessons, and modules.
4) General Presentation Tools: designed for the presentation of multimedia content, and with specific uses in online education, though not intended for this purpose exclusively.
5) Testing and Assessment Tools: designed to produce tests, quizzes, and other types of assessment for print, computer, and/ or Web-based delivery.

1. Single Purpose Authoring Tools

[Note: All evaluations were conducted on a 600 MHz PC with 192 MB RAM and Windows 98 SE]

Screen Hunter 4.0 is a freeware edition of a popular screen capture utility. It allows users to capture any image displayed on a computer screen and then paste the image into another application (Word, PowerPoint, FrontPage, etc.) or save the capture as a graphic file. The program worked well on the test system. It was possible to capture the entire screen, active windows, and rectangles of defined sizes quickly and easily. The “zoom tool” makes capturing rectangles very precise by showing a magnified view of the capture area. The help provided is sparse, but the program is so easy to use that this is not a problem. This tool would be especially useful for an instructor creating software-training materials that require students to see exactly what particular screens and operations look like.

Summary

Screen Hunter 4. 0 is a free and reliable tool for the specific purpose of screen-capturing.

Audacity 1.2 is a free, open-source audio editor that allows users to: record and play sounds; import and export .wav, .aiff, Ogg Vorbis, and MP3 files (others available through plug-ins); edit sounds using cut, copy and paste (with unlimited undo) functions; mix multiple tracks together; apply effects to recordings, including echo, change tempo, noise removal, VST and LADSPA plug-in effects. The program installed, uninstalled, and ran smoothly on the test system. The interface seemed different at first, but soon became comfortable to use. It was easy to record and compress sounds from the computer microphone (speech, etc.) to MP3 files, and fairly easy to cut and paste from the visual waveform view so as to remove or reorder different parts of a recording. The noise removal effect worked well in “cleaning up” a low-quality recording. The tool has the impressive ability to record and import from multiple sound sources, and to apply effects, then to mix them into an audio file (MP3 or .wav). For an instructor who needs to include extensive audio in their instructional materials, Audacity provides a powerful and usable option. The built-in help file is adequate, enough to get a new user started. Further help and examples are available on the product’s website.

Summary

Audacity 1.2 is an excellent, multi-featured freeware audio editing tool.

2. Activity Creation Tools

Hot Potatoes suite is a set of six authoring tools created by the Research and Development team at the University of Victoria Humanities Computing and Media Centre. The tools permit the development of several types of interactive Web-based exercise. No prior Web programming knowledge is required. The user enters the data (text, questions, answers etc.) into a template, and the software creates the webpages for posting on the site. Almost every aspect of the pages can be customized. The following six tools are included: JCloze (creates fill-in-the-blank exercises, and is useful for reviewing vocabulary and for word pattern recognition); JMatch (creates matching and exercises and flash-card reviews; useful for reviewing vocabulary and definitions and for review drills on any subject); JQuiz (creates multiple-choice, short-answer, and hybrid questions; useful for reviewing and subject and self-monitoring of progress); JCross (creates online crossword puzzles, useful for reviewing vocabulary and definitions); JMix (creates mixed-up-sentence exercises; useful for verbal pattern recognition); the Masher (compiles activities created with individual Hot Potatoes tools into a single linked, indexed set.

In the tests, all the tools proved easy to understand and use. They support the inclusion of links, reading texts, images, and media files in the learning materials, though these materials must be obtained from other sources. The activities created have obvious value for teaching basic language education skills. Higher-order functions of analysis and synthesis would be more difficult to support through these activities, though well written questions combined with images and other media could make the quiz tool effective in many disciplines. While Hot Potatoes can easily be used by those with little or even no Web development skills, it is useful to be able to modify the HTML files directly in some cases. For example, the instructions were not always sufficient for some of the activities, and the evaluator chose to modify the pages produced so that students could understand better how to work through the activity. Some knowledge of HTML is also helpful in integrating the finished activities into a complete website. Hot Potatoes is not freeware, but is free for personal and educational uses. Free registration is required to access all the product’s functions, and the full use of some features – e.g., the Masher, requires a purchased license.

Summary

Hot Potatoes is an easy way to create interactive activities, but more useful for some learning objectives than others.

3. Course Development and Presentation Tools

Course Creation Toolkit 4.0 is designed to enable non-technical subject-matter experts and course developers to create quality online training materials as easily and efficiently as possible. The users enters information manually, cutting-and-pasting from other sources, and building a “lesson” by linking to a variety of media files. Interactive questions can be placed throughout any lesson, allowing learners to check their understanding and to receive feedback. The same tools can be used to produce tests and surveys. Learners work through the material in a linear fashion using “next” and “back” links, and can also access pages directly through the menu. All lessons and tests can be configured to work with a learning management system (LMS), to track learner progress and performance.

In the test, a short lesson was produced from existing materials including Word documents and PowerPoint presentations. Text pasted from Word retained most of its original formatting, though adjustments were necessary to fix spacing and to delete various code characters. It is easier to paste unformatted text and to use the product’s own tools to adjust its appearance. It was also not possible to paste graphics from a Word document. They had to be saved first as graphic files and then inserted into the material. An inserted link to a PowerPoint presentation worked smoothly, opening and playing it in a new browser window. Creating test questions was simple, with both multiple-choice and short answer types. No problem was encountered in installing and uninstalling the demo version of the program. Some problems were experienced in moving the finished lesson to a Web server, as some links remained absolute, continuing to point to files on the hard drive rather than in the Web directory. The link to the PowerPoint presentation ceased to work once moved to the Web, and at the time of writing, this problem remains unsolved.

While Course Creation Toolkit is fairly simple to learn and offers a good range of features, it is also somewhat rigid in its course design strategy, and does not allow the creation of interactive materials beyond test questions. Dynamic content must be created using another appropriate program (PowerPoint, Flash, etc.) and then inserted into the lesson. For this reason, novice users may tend to create electronic “page turners” with little advantage over the printed text. The user interface and online help are adequate. Despite these limitations, Course Creation Toolkit is a simple, non-threatening way for educators to transfer a variety of content online, without the need for technical training.

Summary

Course Creation Toolkit 4.0 is an effective, low-cost solution for those who do not require advanced features and flexibility.

KnowledgePresenter is a comprehensive tool for creating dynamic, interactive, online content such as: multimedia lessons, both self-running and interactive; software simulations and demonstrations; and multiple-choice quizzes. All content plays in the browser and can be viewed on many platforms. The version reviewed here was KnowledgePresenter Pro, including the full feature set, separate programs for screen movie capture (ScreenTeacher), still-screen capture (KookaCap 2004) and a learning management system. It is a professional-level tool and carries a high price tag. With the product’s power and flexibility comes a certain degree of complexity. At first, its huge range of features and possibilities can be quite overwhelming. Fortunately, the software contains a user-friendly interface, a set of wizards and templates, and a complete set of tutorials and exercises to get new users up-to-speed. Once past the substantial initial learning curve, the program proved fairly easy to use for simple tasks. The basic design of KnowledgePresenter is similar to that of MS PowerPoint, with separate screens for each stage of the presentation, each containing a variety of text, graphics, video, or other media objects. The many export options (Web, pocket PC, animated .gif, Windows video, etc.) provide great flexibility of delivery. The demo version installed and uninstalled without incident, though it made heavy demands on the test system, causing it to “hang” occasionally. While the product does not require users have a technical background, they must be willing to invest some effort in becoming proficient with it in order to realize its potential. Previous experience with multimedia and Web development concepts would be helpful. The product is likely to appeal to institutional users in need of a solution that can efficiently and consistently produce quality online materials, but is simple enough for motivated instructors and course developers to use.

Summary

KnowledgePresenter is a powerful tool for a quality product – with a price to match.

4. General Presentation Tools

Powerbullet is a layout tool for creating Web-based electronic brochures and slideshows. It generates Shockwave Flash (SWF) files that can played back with the Flash Player or in a webpage. The program is similar in concept to MS PowerPoint, though is designed specifically to create animated, self-running, Web-delivered presentations. The rather cluttered interface allows the user to import a variety of objects (text, graphics, mp3 files, etc.) and to apply animation and timing to each object in synchronizing the presentation. In the current test, it was possible to create a simple presentation almost immediately, as the basic operations are much like those found in similar programs. This was fortunate, as online help for the product is limited. The exported Shockwave presentation played back well in a browser, and the stand-alone executable version was much smoother in the Flash Player. A series of technical problems was encountered with the program on the test system. When a sound clip (mp3) was added to the presentation, it played back poorly in a browser (jerky, stuttering sound and video), though it still worked well in the Flash Player. In addition, the program crashed a few times while exporting the presentation, and occasionally text was omitted from the exported Shockwave file. It was noted that the Powerbullet support forum lists similar problems, most posted by Windows 98 users, so the product may run more smoothly on more recent systems. It has received excellent reviews elsewhere (e.g., http://www.snapfiles.com/).

Summary

Powerbullett is a great concept, easy to use, and FREE, but possibly “buggy” on certain systems.

5. Testing and Assessment Tools

ExamView is a test development and publishing tool designed to improve the efficiency of the assessment process. It allows users to develop test item banks that can then be organized into different individual tests. The specialized interface allows the development and formatting of tests more quickly than is possible with a word processor. In addition, tests and study guides may be published on the Internet or on an organization’s local network, for uses including automated marking and reporting of student results. Publisher-prepared ExamView test banks are available for over 2,000 high school and post-secondary textbooks. The program installed and uninstalled easily on the test system. Using the wizard interface, it was to produce a multiple-choice and true/ false test in a few minutes. Exporting the exam for use on the Web was also simple, requiring only a few mouse clicks. Writing custom test items required more work, but was simpler than using a standard word processor to write and format a test. Complex math equations were easily formatted after only a few minutes of practice with the equation editor. Inserting media files into questions was also simple, and allowed the production of online tests that ask students to answer questions about a short video clip. Once new questions are created, they can be tagged with metadata (descriptive keywords, etc.), and stored as a part of a new test bank. In this way, questions can be searched, sorted, and reused for different tests and study guides over time, to make efficient use of resources. The product includes effective help files. Overall, the evaluator concluded that ExamView’s potential for improved efficiency and delivery of assessments more than justifies its cost. Summary: A time saving, professional quality assessment tool at a reasonable price.

Conclusion

The current range of high-quality digital authoring tools offers great potential for improved efficiency in education, by allowing instructors, subject-matter experts, and others to undertake online learning projects with only limited support from information technology support staff.

References

Bates, A. W. (2000). Managing Technological Change: Strategies for college and university leaders. San Francisco: Jossey-Bass.

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445 – 459.

Greenagel, F. L. (2002). The Illusion of e-Learning: Why we are missing out on the promise of technology. League for Innovation in the Community College White Papers.

Kirkwood, A. (1998). New Media Mania: can information and communication technologies enhance the quality of open and distance learning? Distance Education, 19(2), 228 – 241.

Mayer, R. E. (2001). Multimedia Learning. Cambridge, UK.: Cambridge University Press.

The next report in the series discusses the development of online learning communities.

N.B. Owing to the speed with which Web addresses become outdated, online references are not cited in these summary reports. They are available, together with updates to the current report, at the Athabasca University software evaluation site: cde.athabascau.ca/softeval/. Italicised product names in this report can be assumed to be registered trademarks.

JPB. Series Editor, Technical Notes