This paper examines student performance, satisfaction, and retention of information in online classes as a function of student choice as to the format of instruction. Student outcomes are studied for two groups enrolled in online classes: those who were allowed to choose between an online and a ground-based format and who chose the online format voluntarily; and those who were obliged to take classes in the online format without being afforded the opportunity to choose.
Keywords: attitudes; business schools; comparative studies; distance learning; education evaluation; Internet; online instruction; student satisfaction
The market for courses provided over the Internet has experienced the fastest growth among all segments of the education industry. It has been estimated that college revenues from Internet and Intranet courses will reach US $15 billion in 2002, up from only US $4 billion four years ago (Jones, 2000), and that Web-based instruction will account for 46 percent of the education and training market by 2003 (Mottl, 2000). Nevertheless, many colleges and universities that provide courses over the Internet continue to offer the same courses in the traditional, ground-based format, as well. This practice permits students to select between the two formats according to their perception of the advantages or drawbacks that each provides in their own particular situation. For example, a student who is a “self starter” and who values flexibility and convenience may choose to take a course in the online format, while students who are less “self-directed” and value guidance and the supportive environment of a traditional classroom, may select the ground-based format.
On the other hand, some institutions offering online courses do not afford students the opportunity to choose. Several, such as Jones International University and Cardean University, offer all their courses in the online format only. Others, such as the University of Phoenix enroll students in separate ground-based and online tracks and compel them to take all their courses in the specific track, which they have chosen. Still others, such as the authors’ school, the Wayne Huizenga Graduate School of Business and Entrepreneurship at Nova Southeastern University, provide most courses in both formats, but also offer others only online.
The purpose of the authors’ research was to determine whether there exists a significant difference in online students’ learning outcomes, depending on whether they enrolled in online courses by choice, or because no other alternative was available. In this article the term “online class” refers to classes in which students never physically enter a traditional classroom, with all interaction being via an online medium. The term “ground-based course” refers traditional courses where interaction between students and instructors occurs predominantly face-to-face in classrooms.
There is a wealth of literature on student outcomes in online courses, but the main theme is comparisons between online and ground-based formats. Most researchers have found no significant difference in outcomes between the two formats (Dominguez and Ridley 1999; Ryan, 2000; Gagne and Shepherd, 2001; Cooper 2001; Shea et. al., 2001). Some, however, have cited student concerns about instructional quality in online courses (Bloom, 1998; Terry, 2000), while others consider virtual courses an “inferior technology” particularly in the teaching of complex material (Farrington, 1999; Brown and Liedholm, 2002).
Other researchers have focused on approaches and methods, which may improve outcomes in the online environment. For example, Haythornthwaite et al. (2000), found that a ground-based “boot camp” preceding online courses can help build a sense of community among distance learning students and enhance their satisfaction and learning outcomes. Doran (2001) found that small group collaborative activity improved outcomes in online courses. A similar finding was made by one of the authors of the present study, who compared the grades and course evaluations of two student groups that had taken an online MBA-level Managerial Economics class at two different times. He found that the second group, who were given assignments on which they had to collaborate (via chat rooms, bulletin boards, conference calls, etc.) showed significantly higher levels of satisfaction and information retention than the first group, that was allowed to work individually (Yatrakis, 2001).
To date, however, there have been no studies of the consequences of allowing students to choose between taking courses online or in the traditional, ground based format, despite numerous indications that such consequences may, in fact, exist. For example, it is widely recognized that online courses appeal to students who value flexibility (Cooper, 2001), but that they also require much more work and discipline than is expected in ground-based courses (Brown and Liedholm, 2002). The learning styles of online MBA students and their preferred methods of course delivery have also been identified (Barnes, Preziosi and Gooden, 2002). It would stand to reason that, ceteris paribus, students who feel most comfortable with online course delivery would be the most likely to enroll in courses taught in the online format. This process of self-selection may indeed contribute to the findings of no significant difference in outcomes between students taking courses in either format. What happens, however, when students are not given that choice, but are compelled to take courses in the online format whether or not they fit the profile of potentially successful online students? Students themselves seem to recognize that the availability of alternatives can be advantageous. In Cooper’s (2001) study, 131 students (94 from traditional and 37 from online classes) were asked whether a computer applications class should be offered only online, only on the ground, or in both formats; 100 percent responded that it should be offered in both formats. It is the purpose of the present study to examine this issue in the context of the authors’ own institution, and specifically in its Master of Business Administration (MBA) program, where some courses are offered only in an online format, while others are conducted both online and in traditional, ground-based classrooms.
The Huizenga Graduate School of Business and Entrepreneurship is part of Nova Southeastern University, the 12th largest private university in the United States, with campuses in Ft. Lauderdale, Florida and programs in 22 states and eight countries outside the US. Enrollment at the Huizenga School totals about 2,500 students, of whom approximately 1,600 are MBA candidates, with the rest enrolled in other Master’s degree programs and a Doctoral program. The MBA program offers 46courses, including prerequisites, core courses, specialty courses, and a capstone course. Of these, all fiveprerequisite courses are only available online, as is the information technology course mentioned previouslyAll other courses are taught in both ground-based and online formats, except for the capstone course, which is ground-based only. Registration for the online classes is capped at 15students, which is about two-thirds the size of ground-based MBA classes. The online classes make use of the usual support facilities, including bulletin boards, chat rooms, assignment managers, grade managers and the like. At the instructor’s discretion, streaming video and/or voice chat sessions may be used. Information technology personnel are available on weekdaysfrom 8:30 a.m. to 10 p.m., on Saturdays from 8:30 a.m. to 8 p.m. and on Sundays from noon to 6 p.m. to solve problems and provide real-time assistance to instructors and students.
At the conclusion of the course, students were required to complete a uniform evaluation in which they were asked to rate the course, the instructor, and the school’s infrastructure and support facilities. Students were asked to assign a value of one through five in answer to statements in the evaluation, according to the following scale: 1 – strongly disagree, 2 – disagree, 3 – neutral, 4 – agree, and 5 – strongly agree.
Two of these statements were particularly relevant to the issues examined in the present study: (1) “I was very satisfied with the amount of time spent interacting with my classmates”; and (2) “The course work and discussions resulted in high retention of information.”
Although the first statement does not rate overall satisfaction with the online course, it is the only statement in the evaluation that measures student satisfaction of any kind; hence, it was used as a proxy for overall student satisfaction. It should also be noted that the second question measured student “perceptions” of information retention. However, as Ryan (2000) observed, comparisons of student perceptions constitute good indicators of the relative effectiveness of different delivery formats. They may also be useful in comparing the effects of student choice or lack thereof. In any event, the present study also uses a third metric, the actual grades achieved by students enrolled in online classes for which there was no ground-based alternative, as well as those achieved by students in classes where ground-based alternatives were available.
Student responses to the two course evaluation statements cited above were collected and tabulated from a sample of 397 students who chose to enroll in courses that were available in either online or ground-based formats (the “choice” group); as well as a sample of 433 students who were enrolled in courses that were available only online (the “no choice” group). Not all students answered both questions: the response rate to the first question (satisfaction) was 409 out of a possible 433 (94.5 percent) for the “No Choice” group, and 305 out of 397 (76.8 percent) for the “Choice” group.
On the second question (retention of information), 429 of the “No Choice” students responded (99.1 percent), compared to 315 (79.3 percent) of the “Choice” group.
Student grades for these courses were also tabulated for the two student groups. “Two-Factor Analysis of Variance (ANOVA) without Replication” was used to determine whether the two groups differed significantly in their responses to the two questions, and in their course grades.
Tables 1, 2 and 3 present the results of the ANOVAs. Figures 1 through 3 are graphic representations of the student grade and evaluation data.
Figure 1. Academic performance of "Choice" and "No Choice" groups
Table 1. Academic performance of "Choice" and "No Choice" groups
Figure 2. Student satisfaction* of "Choice" and "No Choice" groups
Table 2. Student satisfaction of "Choice" and "No Choice" groups
Figure 3. Student retention* of "Choice" and "No Choice" groups
Table 3. Student retention of "Choice" and "No Choice" groups
As may be seen from the data contained in tables and figures 1 through 3, students who enrolled in online courses after being given a choice between the online and ground-based formats (the “Choice” group) rated their courses higher on satisfaction and retention of information that did students in the “No Choice” group. The P-values for the two ANOVAs indicate that the null hypothesis of no difference in responses between the two groups is rejected at the nine percent confidence level for perceived retention of information, and at the 11 percent confidence level for satisfaction. Though significant at levels somewhat higher than the conventional five percent, the results nevertheless suggest that considerable differences exist between the two student groups in their responses to these two questions.
However, the ANOVA of student grades conclusively supports the null hypothesis of no difference between the two groups. Indeed, the median grades for both groups are the same (A- or 3.7), as is their mean grade average (3.3). The only discernible difference appears to be at both tails of the grade distribution, which are fatter for the “No Choice” group.
The results of this study suggest that students who choose to enroll in courses in an online format achieve higher rates of satisfaction and perceived retention of information than do students who enroll in online courses when no such choice is provided. This finding is consistent with those of earlier studies that reported different types of learning styles and characteristics of successful online learners, and suggests that students enrolled in such courses by choice probably possess the attributes likely to make online learning a satisfactory and constructive experience. The results seem to indicate that voluntary online courses apparently attract a higher percentage of such students than do the mandatory ones, and therefore produce higher levels of satisfaction and perceived information retention as measured by the students’ course evaluations.
The findings of this study with respect to student grades do not seem to correlate with students’ perceptions. It might reasonably be assumed that students reporting higher levels of satisfaction and perceived retention of information should also perform better as measured by course grades. In fact, our research shows that this was not the case. These seemingly conflicting results suggest that students feel a greater degree of satisfaction when allowed to self-select for online courses, and that this “self selection” may carry over into their perception of retained information. But in terms of actual learning, to the degree that this is measured by grades, being afforded a choice appears to make little difference in outcomes.
The results with respect to grades do suggest that the two groups of students are similar in their learning abilities, and that the instructors’ grading is reasonably uniform and consistent across the online courses of the Huizenga School’s MBA program. Moreover, this appears to be true regardless of whether or not students are afforded the opportunity to select the format of course delivery.
The implications of this study’s findings for college administrators are two-pronged. On the one hand, student choice increases student satisfaction and perception of information retention. On the other hand, choice apparently makes no difference to grade outcomes. Accordingly, the results can be used to support choice as satisfying the preferences of the learning institutions’ customers, or to deflect requests to implement such choice on the grounds that it makes no difference in learning outcomes. In practical terms, the provision of choice may come down to financial considerations, and tradeoffs between these and student satisfaction. It may be more costly to provide courses on both the online and ground-based tracks, but the additional costs should be weighed against the generally positive learning experience which choice provides.
It should be noted that this study is far from definitive, and that further work is needed to corroborate these findings in a broader context. There is surely some random noise in the data: by definition, the courses taken by the two student groups are not the same; the instructors are not the same; and it is possible that the characteristics of the students themselves may vary across the two groups. These potential differences are mitigated by the application of uniform admission standards, but they may not be entirely absent. For example, it is possible that students enrolling voluntarily in online courses may have higher GMAT scores or may be more motivated or more committed than their peers. Still, any substantial difference between the two student groups could be expected to show up in their relative performance, and no such difference is apparent with respect to their grades.
A related area which should be investigated is the performance of students in conventional ground-based classes when allowed to self-select and when afforded no such choice: a study, in other words, analogous to the present analysis except in its focus on ground-based rather than online students. Such research could corroborate, and would certainly complement, the findings of the present study with respect to student choice.
Finally, the aforementioned differences between the two groups in the “tails” of the grade distribution is curious and warrants further investigation. A comparison indicates that the “Choice” group displays a tighter distribution of grades about the mean, while the “No Choice” group shows comparatively fatter tails at both extremes. The proportion of “No Choice” students receiving high grades (A and A-) exceeds that of “Choice” students, as does the proportion receiving low grades (C+, C and F). Conversely, the proportion of “Choice” students with grades of B-, B and B+ is higher than that of the “No Choice” students, while median and mean grades for the two groups are basically the same. Further research on these issues might expand the analysis to other institutions of higher education, utilize broader measures of student learning, and perhaps focus on specific characteristics of the two groups, which might contribute to the observed differences in their behavior.
Barnes, F. B., Preziosi, R. C., and Gooden, D. (2001). An Examination of the Learning Styles of Online MBA Students and Their Preferred Delivery Methods. Paper presented at The Institute of Behavior and Applied Management, San Diego, CA.
Bloom, D. F. (1998). Digital Diploma Mills, Part III. The Bloom Is Off the Rose, Monograph. Retrieved May 15, 2002 from: http://www.vpaa.uillinois.edu/tid/resources/noble.html
Brown, B. W., and Liedholm, C. (2002). Can Web Courses Replicate the Classroom in Principles of Microeconomics? American Economic Review 92(2), 444 – 448.
Cooper, L. W. (2001). A Comparison of Online and Traditional Computer Applications Classes. THE, 28(8), 52 – 58.
Dominguez, P. S., and Ridley R. (1999). Reassessing the Assessment of Distance Education Courses. THE Journal Technological Horizons in Education 27(2). Retrieved May 15 2002 from: http://www.thejournal.com/magazine/vault/A2223.cfm
Doran, C. L. (2001). The Effective Use of Learning Groups in Online Education. New Horizons in Adult Education 15(2). Retrieved May 13, 2002 from: www.salzburgseminar.org/reports/ASC28bib.pdf
Farrington, G. C. (1999). The New Technologies and the Future of Residential Undergraduate Education 34(4), 38 – 44.
Gagne, M., and Shepherd, M. (2001). Distance Learning in Accounting. THE 28(9). Retrieved June 17, 2002 from: http://www.thejournal.com/magazine/vault/A3433.cfm
Haythornthwaite, C., Kazmer, M. M., and Robins, J. (2000). Community Development Among Distance Learners: Temporal and Technological Dimensions. Journal of Computer-Mediated Communication, 6(1). http://jcmc.indiana.edu/vol6/issue1/haythornthwaite.html
Jones, D. (2000). Will Business Schools Go Out of Business? E-Learning, Corporate Academies Change the Rules. USA Today, May 23.
Mottl, J. N. (2000). Learn At A Distance. Information Week, p.767.
Ryan, R. C., (2000). Student Assessment Comparison of Lecture and Online Construction Equipment and Methods Classes. THE Journal, 27(5). Retrieved May 13, 2002 from: http://www.thejournal.com/magazine/vault/A2596.cfm
Shea, P., Fredericksen, E., Pickett, A., Pelz, W., and Swan, K. (2001). Measures of Learning Effectiveness in the SUNY Learning Network. In J. Bourne, and J. C. Moore (Eds.) Online Education, Volume 2, Learning Effectiveness, Faculty Satisfaction, and Cost Effectiveness. SLOAN-C Series 2001.
Terry, N. (2000). MBA Student Perceptions About the Effectiveness of Internet Instruction. Business Education Forum 54(4), 42 – 44.
Yatrakis, P. G. (2001). The Effect of Self-Selection on Student Performance in Online Courses. Paper presented at International Council for Open and Distance Education (ICDE) Conference. Dusseldorf, Germany, April.