Promoting Distance Learners’ Cognitive Engagement and Learning Outcomes: Design-Based Research in the Costa Rican National University of Distance Education

To explore effective learning design for students’ cognitive engagement, a design-based case study was conducted in a quality control course in the Costa Rican National University of Distance Education between the 2011 and 2012 academic years. The course was revised for the 2012 provision in terms of the assignment structure, the number of face-to-face sessions, and facilitation strategies. This study documents how the course redesign impacted the distance learners’ cognitive engagement and learning outcomes. Theories of cognitive engagement and transactional distance informed the design-based investigation. Research findings indicate that the design revisions positively influenced both students’ cognitive engagement and learning outcomes within this distance higher education context; however, the student performance represented by their assessment grades might not always reflect this improvement.


Introduction
Design-based research integrates empirical investigation with theory-based learning design (Swan, Mattews, Bogle, Boles, & Day, 2012). The ultimate goal of design-based research is to improve students' learning in real-world educational environments through systematic innovations (Wang & Hannafin, 2005). Since research and development simultaneously occur through continuous cycles of design, enactment, analysis, and redesign in a design-based research project (Cobb, 2001), this approach enables educators to elucidate "how, when, and why educational innovations work in practice" (The Design-Based Research Collective, 2003, p. 5). Particularly, design-based research entails the study of learning in context through systematic design and instructional strategies (Brown, 1992). It thus leads to "contextually-sensitive design principles and theories" (Wang & Hannafin, 2005, p. 7), which also can create a number of meaningful implications for distance education.
The purpose of this study is to report on the first cycle of a design-based research process for designing, implementing, and revising a quality control course in the Costa Rican National University of Distance (Universidad Nacional Estatal a Distancia in Spanish). Considering the iterative nature of design-based research and the dual purpose to redesign the course and to create design principles (Swan et al., 2012;Wang & Hannafin, 2005), the study documents the design changes made for the 2012 course provision and their effect on students' cognitive engagement and learning outcomes.
The target course has been provided at an undergraduate level in a hybrid format; it features field trips related to students' individual research projects as well as a series of supplementary face-to-face classes. Lauzon (1992) pointed out that one of the fundamental challenges for distance educators is to "search out means of reducing structure and increasing dialogue so that learners may move from being simply recipients of knowledge to actively embracing and working with objective knowledge to make it their own" (p. 34). In an attempt to improve the course quality through ongoing design experiment, we attempted to determine how the redesign influenced the pedagogical processes and outcomes in this specific context of open and distance higher education. The quality control course in the Costa Rican National University of Distance Education was chosen for this design-based research because the course involved multiple pedagogical elements which might illuminate how the changes intended to enhance the course quality influenced students' learning experiences.
The conceptual framework of the research is described in Figure 1. After reviewing the student survey results compiled after the 2011 course, we redesigned the course structure and content with two guiding theories of cognitive engagement and transactional distance in order to facilitate engagement with learning. More specifically, the assignment structure, facilitation strategies, formats of supplementary face-to-face sessions, and examination contents were redesigned in the line with the two theories. Subsequently, we examined the impact of the design changes in terms of the students' cognitive engagement and performance as well as knowledge development. Theoretical Frameworks

Cognitive Engagement
Given complex online learning environments, how distance learners engage in a specific educational circumstance and attendant instructional materials has been posited as a significant research inquiry because learners' cognitive engagement is pivotal for any distance educational pedagogy. Distance learners' engagement in and/or commitment to learning has been examined along with diverse pedagogical issues, such as learning effectiveness (Swan, 2003), student satisfaction (Sun, et al., 2008;Shea, Pickett, & Pelz, 2003), learning motivation (Hoskins & Van Hoof, 2005), and learning strategies (Brown, Meyers, & Roy, 2003;Stoney & Oliver, 1999). Cognitive engagement, as a theoretical construct, bridges among those heterogeneous conceptions in explicating how distance learners experience a learning context while accounting for their individual experiences and characteristics (Biggs, 1987). Since "the integration and utilization of students' motivations and strategies in the course of their learning" are the key to a successful distance education pedagogy (Richardson & Newby, 2006, p. 23), a conceptualization of cognitive engagement in relation to online course design supports effective teaching strategies, high learner motivation, and productive distance education pedagogy. The term cognitive engagement was first coined by Corno and Mandinach (1983) to investigate students' learning in relation to the pedagogical process as well as individual characteristics. As cognitive engagement affects the amount and quality of effort that students exert in classroom activities, it indicates the level and/or kind of their motivations (Corno & Mandinach, 1983). Corno and Mandinach (1983) further argued that self-regulated learning is a representative form of cognitive engagement that leads students to a higher level of thinking. The conception of cognitive engagement has been used in various areas, such as literacy (Guthrie, 1996), multimedia (Bangert-Drowns & Pyke, 2001;McLoughlin & Luca, 2000;Stoney & Oliver, 1999), and mathematics (Henningsen & Stein, 2002;Marks, 2000). These primary inquiries encompass cognitive abilities, affective motivations, and learning experiences as defining aspects of students' cognitive engagement.
In the current study, cognitive engagement was regarded as one significant indicator of students' learning motivation in the context of open and distance higher education. The students revealed varying amounts and kinds of motivation and strategy in their learning tasks. This phenomenon can be further explained along with the distinction between deep and surface engagement with their learning (Biggs, 1987;Craik & Lockhart, 1972). Deep engagement is associated with intrinsic motive to create a more complex knowledge structure by means of one's existing knowledge and pedagogical materials (Biggs, 1987;Kardash & Amlund, 1991). On the other hand, surface engagement involves mere memorization, simple reproduction, and other kinds of superficial engagement with learning materials, such as just re-reading textbooks or class notes (Walker, Greene, & Mansell, 2006). Whereas surface engagement frequently results in unmet learner needs or underachievement in learning tasks, deep engagement with learning can be embodied in students' thoughtful cognitive processing and selfregulatory strategies (Corno & Mandinach, 1983;Greene & Miller, 1996;Wolters & Benzon, 2013). Furthermore, the students' experience of deep cognitive engagement is more likely to influence their future use of meaningful strategies that they develop through the learning process (Schunk, 1991).
In addition to individual motivation, interaction also plays a crucial role in the students' engagement with learning. In most educational environments, a classroom culture has a significant impact on the conditions that either restrict or improve certain pedagogical strategies and particular types of interaction among the pedagogical subjects (Edwards & Mercer, 1987). That is, cognitive engagement of students can be influenced by certain teaching strategies and interactions in the educational context (Blumenfeld, Puro, & Mergendoller, 1992). Moore (1989) identified three major categories of interaction in any pedagogical context: learner-teacher, learner-content, and learner-learner. The learning (re)design in this research thus included various strategies to promote interaction among the students (e.g., by changing discussion assignments) as well as between the students and the instructor (e.g., by increasing the number of face-to-face sessions). Interaction between the students and learning content was accounted for through variations in lesson units and reading assignment. To this end, the changes made to improve students' cognitive engagement was assessed through course content reviews, such as discussion forums, assignments, and the mid and post self-assessments as well as the Cognitive Engagement/Transactional Distance (CE/TD) survey implemented at the end of the course. Survey questions particularly focused on the themes of e-learning, metacognitive ability, and self-regulation, which were specifically intended to capture transactional distance perceived by the students and to gauge their cognitive engagement in learning.

Theory of Transactional Distance
To further promote various interactions through our course (re)design, the theory of transactional distance (or transactional distance theory) was considered. Moore (1997), the pioneer of this theory, defined transactional distance as "a concept describing the universe of teacher-learner relationships that exist when learners and instructors are separated by space and/or by time" (p. 22). Most importantly, the concept of transactional distance denotes the psychological, rather than physical, distance among the pedagogical subjects. This observation is premised with physical separation between individuals that creates "a psychological and communications space to be crossed, a space of potential misunderstanding between the inputs of instructor and those of the learner" (Moore, 1997, p. 22). One of the fundamental theoretical implications of transactional distance is that an educational exchange among the pedagogical subjects, which is facilitated by educational mediations, can reduce miscommunication or psychological disconnection in order to lead to an effective educational transaction (Shearer, 2009).
In a nutshell, the theory of transactional distance concerns the pedagogical phenomenon of interaction between teachers and learners, or among learners themselves, in the distance educational context primarily influenced by diverse relations between dialogue and structure. More specifically, the structure consists of course design elements, such as learning objectives, activities, assignments, and assessments, whereas dialogue refers to the meaningful communication between the pedagogical subjects. Moreover, the theory accounts for the importance of autonomy, which indicates a learner characteristic in line with the degree of self-control or selfmanagement in learning (Moore, 1997;Shearer, 2009). The theory thus allows us to elucidate how relations among the three fundamental variables in distance educational settings can "describe the extent to which course components can accommodate or be responsive to each learner's individual need" (Moore & Kearsley, 1996, p. 200).
At a macro level, transactional distance theory helps us to understand how the three variables interact in the context of distance education (Shearer, 2009). As discussed by Moore (1980;1997) and supported by Saba and Shearer (1994) and Shearer (2009), transactional distance or psychological separation is diminished when dialogue is high and structure is low. However, in the occasion that learners are highly autonomous, low dialogue does not necessarily exasperate the transactional distance. These relationships imply that a high level of dialogue may not always be required by autonomous learners for their effective learning. The relationships among the three variables are visualized in  In the process of designing a distance education course, Moore (1997)

Course Revisions
The Quality Control course consists of four units (see Table 1). The course generally involves four types of evaluation: two exams, case study, a research project (plant tour), and participation (i.e., discussion forums and communications). In the case study, students are asked to present a problem case that a food factory may confront in its production lines, usually related to statistics and control graphs. Students should identify specific non-conformities that might cause the problems and provide adequate resolutions, using either their own experiences or readings. The case study covered 20%  After the 2011 course provision, the department decided to redesign the quality control course according to two reviewers' comments 1 and the results of the CE/TD survey. The pedagogical issues identified through these review processes are threefold. First, the course structure needed a revision as the majority of students pointed out problems that stemmed from giving them flexibility to explore broad topics to be covered in the course. This problem led us to rethink the amount of knowledge and information that students must focus on in their learning. Particularly, for those who had less statistical skills and experiences, this problem appeared even more salient. Second, the students highlighted the necessity to promote communications between the instructor and themselves, especially when they had confronted difficulties with the assignments and examinations. Some students at a distance requested a video conference with the instructor to grapple with this issue. Third, more clearly stated guidelines were requested for the assessment activities. In particular, many students reported that the instructions for the research project and exams were so ambiguous that the instructor should have provided more precise and concrete information.
Those issues mainly highlighted the needs for improvement in assignments,  Table 2. Students were guided to read the entire textbook.
Students read specific topics in the textbook.

Number of face-toface sessions
There was one face-to-face class during the course.
There were two face-to-face classes during the course.

Facilitation strategy
The instructor used no facilitation strategies other than answering students' questions.
The instructor employed multiple facilitation strategies through diverse communication channels (e.g., phone calls, emails, video conferences, etc.).

Examination
Exams included questions developed from a wide variety of topics in the course.
Exams were designed to measure students' expertise and deep learning in a limited number of topics.
Taken together, the design changes for the 2012 course provision were grounded in the two guiding theories of cognitive engagement and transactional distance. The improvements were intended to promote the students' deep engagement in their learning and to reduce the transactional distance between the instructor and the students.  Students gained higher scores in the assessments of the forum and the research project, whereas they accomplished relatively low scores in the first and second exams. This analysis highlights the gap between their motivation or cognitive engagement, which is represented by the discussion forum and the research project scores as well as the learning outcomes measured by the written exams. Despite the gap, a positive correlation between cognitive engagement and learning outcomes was observed; that is, the more a student was engaged in the course, the higher scores he/she attained in the exams (Spearman's rho = 0.78). In total, a third of the students obtained final grades between 71% and 90% in 2011.
In 2012, the students were also required to complete an online self-checklist (Appendix II) where they were asked to reflect upon the improvement of their knowledge and skills in regards to the subject. This self-assessment encompassed three topics (i.e., descriptive and inferential statistics, control graphics and other techniques, and normalization and quality administration) and 13 yes/no question items. In sum, the students marked "yes" on 54% of the question items.

Data Collection and Analysis
This design-based research was adapted to explore the effects of course revisions on students' cognitive engagement and learning outcomes. Even though the research team used some qualitative data such as reviewers' comments and course materials in the process of this design-based research, the students' grades, surveys, and self-assessment conducted in 2012 were three major methods for data collection and analysis. Learning outcome measures included scores on each assessment method as well as the final course grades. On the one hand, the first exam measured students' learning of quality control systems, elementary statistics, and process control graphs, whereas the second exam tested their knowledge about standards, norms, quality engineering, and quality administration. The case study aimed to assess the students' abilities to apply relevant statistics and process control graphics, and the research project represented the course goal of analyzing the quality management process in a real situation.
In order to evaluate students' cognitive engagement and experiences of transactional distance, all students were asked to complete the CE/TD survey that was composed of 5- cognitive engagement and transactional distance in the context of distance higher education. Additionally, every student in both 2011 and 2012 was asked to submit the self-assessment of their knowledge and skills developed through this course.
A quasi-experimental approach was employed to compare students' grades before and after the course revisions. That is, the revisions were posed as the independent variables while the dependent variables were students' scores in the series of course assessment methods. Additionally, a two-sample t test was applied to the CE/TD survey results

Results
Given the small sample size, the students' cognitive engagement measured by the survey and learning outcomes measured by their performance in each assessment could not be statistically generalized. Nevertheless, descriptive statistics show that students' cognitive engagement and performance were notably improved after the course redesign (t (30) =22.09, p < 0.0001). In particular, the students perceived that course content and their engagement in the units were significantly improved along with the course revisions ( Figure 4). In terms of students' performance, the most contiguous difference was observed in Exam 1 and the case study, which supported the revision of the course assignments and the increased number of face-to-face sessions.    this end, the result shows how the revisions oriented toward reducing the transactional distance between the instructor and students as well as promoting students' cognitive engagement for the 2012 course enhanced the quality of the course.
The result of the self-checklist completed by the 2012 students also reflected this positive impact of the course redesign. Even though the self-checklist is subjective and constructivist by nature, the fact that a majority of the students in 2012 checked off most key subject areas of the course as well understood attests to effective facilitation of students' development as intended by the design changes. Over 80% of the students answered "yes" to more than 12 out of 16 checklist items, and only 2 student marked "yes" to less than 8 questions. Since the self-checklist was not provided to the students in 2011, this result is not comparable between the two student cohort groups.
In summary, the improvement of the learning processes and outcomes measured by the CE/TD survey and the checklist assessment suggest that interventions aimed to enhance students' cognitive engagement as well as reduce the transactional distance led to positive learning processes and outcomes in this open and distance higher education context. Therefore, these design-based research findings seem to correspond to the theoretical assumptions of cognitive engagement and transactional distance.

Discussion and Conclusion
Linking online learning design to students' motivation and learning outcomes is a persistently significant quest for distance education researchers and scholars. To further develop our knowledge regarding these constructs, this study attempted to measure the impact of the course redesign in terms of cognitive engagement and transactional distance as experienced by distance learners in an institution of open and distance higher education. Results of the study indicate that, taken together, the revisions based upon the theories of cognitive engagement and transactional distance could be linked to the improvement of students' motivation and learning outcomes.
This finding suggests a need to consider the specific implications of the range of educational contexts in which learning in distance higher education takes place. The theory of transactional distance (Moore, 1980) was employed to examine the educational context, especially by accounting for the multiple relationships between learners or between teacher and learners. More specifically, the study exemplifies that the changes in the facilitation strategies and the increased number of face-to-face sessions could lead to improved dialogue, which results in the reduction of psychological Furthermore, the 2012 revisions oriented toward clarifying learning tasks by changing the structure enhanced learning outcomes. This result might contradict the theoretical assumption of transactional distance, which is grounded in the inverse relation between dialogue and structure in a distance education course (Dron, Seidel, & Litten, 2004).
However, as Moore (1977) previously noted the possibility of high dialogue and high structure (as in correspondence programs) or low dialogue and low structure (as in selfdirected independent programs), the desired balance between dialogue and structure is only reliable when it is based on the educational sophistication of the learner and the subject content (Moore, 1997). Given the learner characteristics (i.e., mostly part-time distance adult learners and transitioning or returning college students) and the subject content that requires high-level statistical skills and hands-on exercises in the current research, the heightened level of the structure in lessons, assignments, and assessments of the 2012 course had a positive impact on dialogue, which made students feel less transactional distance.
More significantly, "[a] delicate balance between course structure and dialogue of the instructor and learners is critical for online learner success" (Murphy & Cifuentes, 2001, p. 298). Previous studies that investigated the role of course structure in student satisfaction and perceived learning in online learning environments, such as Shea, Pickett, and Pelz (2003) and Stein, Wanstreet, Calvin, Overtoom, and Wheaton (2005), also support reduced transactional distance with high structure and high dialogue. By tightening the structure, the course redesign consequently promoted the level of student motivation and adaptability of content, which resulted in deeper cognitive engagement and richer learning outcomes among the student group.
To sum up, this design-based research reaffirms the strong correlation of less transactional distance with productive learning outcomes recently attested by Benson andSamarawickrema (2009) andFlowers, White, andRaynor (2012). Specifically, this study implies that the learning context for distance higher education is highly dependent on the learning design delicately prepared to support learners' characteristics as well as dialogue. Since an effective instructional systems design model considers various aspects of the learning context, such as process, systems, outcomes, and delivery (Morrison, Ross, & Kemp, 2010), transactional distance in relation to those heterogeneous design elements can inform future design-based research studies that are similar in context to the one examined in the current research. To this end, drawing upon the lessons that we have obtained through the first cycle of this design-based research, we will investigate how the combination of those multiple pedagogical components can be optimized to reduce the transactional distance in this specific context of open and distance education in the next cycle of course redesign.