A Review to Weigh the Pros and Cons of Online, Remote, and Distance Science Laboratory Experiences

The effectiveness of traditional face to face labs versus non-traditional online, remote, or distance labs is difficult to assess due to the lack of continuity in the literature between terminology, standard evaluation metrics, and the use of a wide variety non-traditional laboratory experience for online courses. This narrative review presents a representative view of the existing literature in order to identify the strengths and weaknesses of non-traditional laboratories and to highlight the areas of opportunity for research. Non-traditional labs are increasingly utilized in higher education. The research indicates that these nontraditional approaches to a science laboratory experience are as effective at achieving the learning outcomes as traditional labs. While this is an important parameter, this review outlines further important considerations such as operating and maintenance cost, growth potential, and safety. This comparison identifies several weaknesses in the existing literature. While it is clear that traditional labs aid in the development of practical and procedural skills, there is a lack of research exploring if non-traditional laboratory experiments hinder student success in subsequent traditional labs. Additionally, remote lab kits blur the lines between modality by bringing experiences that are more tactile to students outside of the traditional laboratory environment. Though novel work on non-traditional labs continues to be published, investigations are still needed regarding cost differences, acquisition of procedural skills, preparation for advanced work, and instructor contact time between traditional and non-traditional laboratories.


Article abstract
The effectiveness of traditional face to face labs versus non-traditional online, remote, or distance labs is difficult to assess due to the lack of continuity in the literature between terminology, standard evaluation metrics, and the use of a wide variety non-traditional laboratory experience for online courses. This narrative review presents a representative view of the existing literature in order to identify the strengths and weaknesses of non-traditional laboratories and to highlight the areas of opportunity for research.Non-traditional labs are increasingly utilized in higher education. The research indicates that these non-traditional approaches to a science laboratory experience are as effective at achieving the learning outcomes as traditional labs. While this is an important parameter, this review outlines further important considerations such as operating and maintenance cost, growth potential, and safety. This comparison identifies several weaknesses in the existing literature. While it is clear that traditional labs aid in the development of practical and procedural skills, there is a lack of research exploring if non-traditional laboratory experiments hinder student success in subsequent traditional labs. Additionally, remote lab kits blur the lines between modality by bringing experiences that are more tactile to students outside of the traditional laboratory environment. Though novel work on non-traditional labs continues to be published, investigations are still needed regarding cost differences, acquisition of procedural skills, preparation for advanced work, and instructor contact time between traditional and non-traditional laboratories.

Introduction
Despite a wealth of research on the topic of online versus traditional higher education courses, less focus has been aimed at comparing laboratory experiences. A longstanding question within the science education community is: "What is the effectiveness of lab experiences (traditional vs. non-traditional) relative to each other?" Some studies compared a traditional laboratory course to a non-traditional laboratory course as a whole while others compared the outcomes of an individual lab experiment or activity by modality (Table   1). Some modality comparison studies of science courses that traditionally have a lab neglected to describe if the course analyzed included a laboratory component (Colorado Department of Higher Education, 2012;Rosenzweig, 2012).  (Domingues, Rocha, Dourado, Alves, & Ferreira, 2010) An important consideration before initiating a comparison of lab modalities is to establish the value of the laboratory component in the science course. In introductory science courses designed for non-majors, the laboratory environment may be a tool to reinforce the lecture content (Feig, 2010). However, in many cases, the main goals of the laboratory experience include developing learner skills in making and recording observations as well as deductive reasoning and hypothesis construction (Adlong et al., 2003).
Furthermore, the purpose of the lab course often includes the development of practical instrumentation skills and safety awareness or transferable skills such as teamwork, time management, communication, and conflict resolution (Boyer, 2003;Woods, Felder, Rugarcia, & Stice, 2000). The science lecture courses are not necessarily expected to cultivate these skills and instead tend to deliver general concepts and information. Thus, the laboratory section, both traditional and non-traditional, are often expected to put the ideas into practice and provide students with a practical skills experience (Waldrop, 2013).
The goal of this review was to organize and synthesize the existing literature in order to outline the benefits and drawbacks of the traditional face-to-face approach for laboratory experiences compared to nontraditional laboratory experiences, which can take many forms. This novel work systematically compared various types of non-traditional lab experiences to evaluate the strengths and weaknesses of the experiences, which no other work currently does in the literature. Furthermore, this investigation identified multiple gaps in the literature and future research in targeted areas was recommended.

Research Method
An initial survey indicated that the literature was neither robust enough nor was it homogeneous enough to justify a systematic review or meta-analysis. For this reason, a narrative review was executed. Studies focused explicitly on engineering or computer laboratory experiments were largely excluded, except when discussing terminology. Conference papers and unpublished materials were not explored. This review is representative, not exhaustive, and some relevant educational studies may have been excluded.
The collected literature was then analyzed based on the following: terminology used, learning outcomes, multiple benefits, practical skill development, cost, growth potential, accessibility, student-instructor communication, safety, and instructional design. These specific topics were investigated because they had the most inconsistencies between articles and/or there was a dearth of information regarding these themes.

Non-Standard Terminology
Even with a clear research question in mind, the first stumbling block was encountered immediately. Nontraditional labs can include simulated labs, remote labs, lab kits, or some combination. Furthermore, some traditional face-to-face laboratory courses have adopted non-traditional experiences to varying degrees.
Confounding this even further is the fact that the literature does not present standard terminology for nontraditional experiences. In order to code the terms used in the literature for various non-traditional laboratory experiences, the following definitions were used. An online laboratory was defined as a laboratory experience where the learner accessed simulated experiments, instruments, or equipment through a computer. A remote laboratory was defined as a laboratory experience where the learner accessed real experiments, instruments, or equipment virtually through a computer. A distance laboratory 158 was defined as a laboratory experience where the learner performed hands-on labs outside of a traditional laboratory space through portable laboratory kits, often delivered through the mail.
The inconsistency of the terminology was highlighted by Ma and Nickerson (2006) and no resolution has emerged in the literature since. This lack of standard terminology means that the same online laboratory experience can be labeled "simulated labs," "virtual labs," or "distributed learning labs." Engineering tends to account for a large percentage of the literature on non-traditional labs and these studies often use confusing labeling like referring to remote labs as "online labs" and "remote labs" in the same discussion (Ma & Nickerson, 2006;Tuttas & Wagner, 2001). Table 2 provides a sample of variable terms present in the literature.

Comparing Learner Outcomes
Beyond the lack of standard language to discuss lab modalities, there is no standard evaluation criteria to compare their effectiveness. The literature disagrees on the appropriate measures to use to answer the question of modality equivalence. For example, studies supporting non-traditional labs lean towards outcomes in content knowledge (using quizzes and exams as assessment tools) while studies supporting traditional labs rely on qualitative measures (surveys) (Brinson, 2015). A recent large-scale review of this question concluded that laboratory learning outcomes can be achieved at equal or even greater frequency in non-traditional labs than traditional labs, regardless of the outcome category being measured (Brinson, 2015). In a biology-specific review, these findings are supported (Biel & Brame, 2016).

Comparing Other Variables
While the effectiveness of a lab experience at achieving the learning outcomes is critical to both educators and administrators, it is not the only variable to consider. There are pedagogical, economic, and safety benefits and drawbacks for all permutations of a laboratory experience. Some variables are straightforward (Table 3) while others fall into a gray zone. there is no evidence in the literature to suggest that students who took an introductory lab through nontraditional methods perform worse in more advanced labs than those who participated in a traditional introductory lab. With that said, it is fair to say that there are procedural skills that involve sensory feedback where a simulation would simply not be equivalent (Brinson, 2015). A study on the necessity of touch sensory feedback in the K-16 science classrooms (study includes kindergarten through undergraduate level), applying both embodied cognition and additional (touch) sensory channel theories, found that the touch sensory feedback is not necessarily a critical component for learning through science experimentation (Zacharia, 2015). Due to inconsistencies in the literature, Zacharia (2015) was unable to arrive at a framework to describe when touch sensory feedback is ideal for learning through experimentation.
Compounding this issue, some laboratories are blurring the lines between modalities. In some cases, robotics bring in a more tactile experience by allowing students to remotely control an experiment and monitor the progress in real-time using video (Rivera, 2014). In other scenarios, laboratory settings involve both physical and virtual manipulatives (Zacharia & Olympiou, 2008).
Cost. The cost to students in various modalities varies by institution. In some cases, students must pay lab fees and purchase a lab manual for traditional labs while other institutions do not assess lab fees.

160
The cost of lab kits vary significantly based on the extent of utilization in the course. Online simulations often have a lower cost than lab kits.
Growth potential. One clear benefit of non-traditional labs is the growth potential. With traditional labs, space is limited by facilities and bottlenecks can occur. Cal-Tech addresses this very issue (Rivera, 2014)

Accessibility. A clear benefit of non-traditional labs is the expanded accessibility. With online
simulations, remote access, and lab kits, learners have the opportunity to engage with the material on their own schedules. This is particularly ideal for non-traditional students who have career and family responsibilities or military deployments. These non-traditional laboratory formats also offer multiple access opportunities that are typically not available in a traditional hands-on laboratory. In many settings using non-traditional labs, the learners have extended time to work with the material, compared to a typical 3 hour weekly lab session. Surveys have indicated that students recognize this as a benefit (Turner & Parisi, 2008). This format also allows increased access to those with physical or psychological disabilities that prevent them from attending traditional laboratories.
One drawback of the lab kits compared to online or traditional labs is the inability to replicate experiments, particularly if an error was made. The lab kits typically do not provide excess reagents for the microscale experiments. A spill or an oversight in the procedure could prevent the student from being able to complete the experiment. Furthermore, students would not have the opportunity to replicate experiments either for error calculations or to confirm unexpected results. Even in a face-to-face laboratory setting with instructor oversight and guidance, experiments sometimes do not go as planned and students have to start from scratch. This is simply not an option for lab kits.
Another aspect of accessibility is the technological hurdles in getting computer simulations or remote control software to work on the various computers used by students. This poses a unique challenge for students, instructors, and the institutions' IT support staff.

Student-instructor contact time.
Another consideration is that non-traditional labs are often asynchronous in nature. This means that the instructor or teaching assistant is not directly in front of the student. This can also limit peer collaboration, depending upon pedagogical choices in the course design.
Additionally, the unsupervised nature of asynchronous laboratory experiences can provide a barrier to asking timely questions. For non-traditional labs using lab kits, this can diminish safety awareness and increase risks associated with the laboratory work.

Safety considerations.
Another factor to consider is the experiences that each modality can support. Hands-on labs (traditional and lab kits) not only reinforce subject area content but also procedural skills. Safety should be an integral component of the course in order to control risks. Due to the unsupervised nature of working with lab kits, the types of experiments are inherently limited. Lab kits need to be able to operate without generating hazardous waste, without advanced instrumentation, and without easily mitigated chemical and physical risks (Boschmann, 2003). The challenge is to not only develop activities that are safe for transportation/delivery and unsupervised experimentation but activities that are also engaging and do not have obvious results that would detract from motivation to complete an experiment. For these reasons, lab kits are microscale and use low-risk chemicals (Gould, 2014).
On the other hand, labs taught through online simulations are strong in reinforcing content but often gloss over safety and often do not approximate actual procedural skills. Remote labs are likely better at approximating procedural skills but safety may not inherently be addressed.
In online labs, there are often safety oversights and over-simplifications. For example, Late Nite Labs does not address hazardous waste. Students dispose of chemical waste in a bin labeled "chemical recycling" with a biohazard symbol. This does not meet waste management standards established by the Resource Conservation and Recovery Act (RCRA; Environmental Protection Agency, 2011). The oversimplification and recognition that it is "not really happening" can affect student motivation (Rivera, 2014). A benefit of online labs, however, is the ability to explore reactions and procedures that are too expensive or simply too dangerous to perform in a hands-on setting. Safety in remote lab experiences is likely to be variable based on the procedure being remotely operated and the presence of personnel at the physical location of the equipment or experiment being remotely operated.

Non-Traditional Lab Course Design Suggestions
The literature has an abundance of advice regarding creating an effective learning environment in online and non-traditional lecture courses. A common theme, which is easily applicable to the non-traditional laboratory, is active, visible, intentional engagement with students. Deep engagement has been shown to correlate with increases in student performance (Jaggars, Edgecombe, & Stacey, 2013). Instructional design focused on developing students' skills in self-regulated learning is critical for their success in online courses.
Student strengths in time, study environment, and effort regulation have been shown to have a significant positive influence on student performance in online courses while rehearsal, metacognitive self-regulation, time, and study environment correlated with student satisfaction with the course (Puzziferro, 2008).
Literature providing course design and execution guidance for online courses is abundant, with less focus placed on providing this guidance specifically for non-traditional labs. Much of the literature on nontraditional labs focuses on infrastructure, student outcomes, or student satisfaction, with little attention to pedagogical design. However, data-supported guidance is present. Inquiry is often considered a best practice for laboratory courses and non-traditional lab courses are no exception. An inquiry cycle presented for online laboratory courses proceeds with Orientation, Conceptualization, Investigation, Conclusion, and Discussion . An analytical taxonomy of guidance for inquiry in online courses presents the following categories: performance dashboard, prompts, process constraints, heuristics, scaffolds, and direct presentation of information (de Jong & Lazonder, 2014). Zacharia (2015) presents this taxonomy as ideal for consideration in design and execution of non-traditional lab courses, identifying one of the strengths of this taxonomy being that the guidance is classified in a way that is context independent (e.g., inquiry phase or discipline The literature also reveals additional course features that promote success in an online laboratory environment. As with any online course, student success has been shown to be improved by the use of an online laboratory course orientation (Garman, 2012). The development of an online learning community that allows for peer collaboration has also been demonstrated as a best practice (Garman, 2012;Lowe, Berry, Murray, & Euan, 2009;Palloff & Pratt, 2013). Student surveys from online laboratory courses have highlighted the importance of a well-organized calendar for the course that includes hyperlinks to the laboratory activities, assessments, and deliverables (Reeves & Kimbrough, 2004). For remote laboratory exercises, following industry standards regarding technology platforms ensures students develop skills that are easily transferable and not outdated (Esquembre, 2015).

Conclusion
In conclusion, a well-designed, non-traditional lab can be as effective as a traditional face-to-face laboratory experience when measuring either content knowledge or student opinions as the metric for equivalence.
While there is a limited generalizability of the findings, this mirrors results for meta-analyses comparing traditional and non-traditional modalities for lecture courses and non-science courses (Allen et al., 2004;Means, Toyama, Murphy, Bakia, & Jones, 2009;Shachar & Neumann, 2003). However, there are other considerations that institutions must weigh when deciding to take a traditional or non-traditional approach to a laboratory course. The ideal choice of format (traditional, online, remote, or distance) will vary based on the needs and goals of both the institution and the learner.
There are still some rather important questions that have yet to be properly addressed. First, there should be a large-scale cost comparison for various modalities of laboratory courses, for both institutional costs as well as student costs. Second, a long-term study should explore whether non-traditional introductory laboratory experiences properly prepare students for more advanced laboratory experiences, particularly in comparison to those who participated in traditional introductory lab courses. And third, it would be interesting to learn if the instructor-student and peer-to-peer contact time significantly vary in traditional laboratories compared to non-traditional laboratories. With more attention to resolving these questions, the literature may finally arrive on standard terminology and metrics for evaluating equivalency.