International Review of Research in Open and Distributed Learning Doctoral Students’ Learning Success in Online-Based Leadership Programs: Intersection With Technological and Relational Factors

This study examines how technological and relational factors independently and interactively predict the perceived learning success of doctoral students enrolled in online-based leadership programs offered in the United States. The 73-item Online Learning Success Scale (OLSS) was constructed, based on existing instruments, and administered online to collect self-reported data on three primary variables: student learning success (SLS), relational factors (RF), and technological factors (TF). The SLS variable focuses on the gain of knowledge and skills, persistence, and self-efficacy; the RF on the student-student relationship, the student-faculty relationship, and the student-non-teaching staff relationship; and the TF on the ease of use, flexibility, and usefulness. In total, 210 student responses from 26 online-based leadership doctoral programs in the United States were used in the final analysis. The results demonstrate that RF and TF separately and together predict SLS. A multiple regression analysis indicates that, while all dimensions of TF and RF are significant predictors of SLS, the strongest predictor of SLS is the student-faculty relationship. This study suggests that building relationships with faculty and peers is critical to leadership doctoral students’ learning success, even in online-based programs that offer effective technological support.


Article abstract
This study examines how technological and relational factors independently and interactively predict the perceived learning success of doctoral students enrolled in online-based leadership programs offered in the United States. The 73-item Online Learning Success Scale (OLSS) was constructed, based on existing instruments, and administered online to collect self-reported data on three primary variables: student learning success (SLS), relational factors (RF), and technological factors (TF). The SLS variable focuses on the gain of knowledge and skills, persistence, and self-efficacy; the RF on the student-student relationship, the student-faculty relationship, and the student-non-teaching staff relationship; and the TF on the ease of use, flexibility, and usefulness. In total, 210 student responses from 26 online-based leadership doctoral programs in the United States were used in the final analysis. The results demonstrate that RF and TF separately and together predict SLS. A multiple regression analysis indicates that, while all dimensions of TF and RF are significant predictors of SLS, the strongest predictor of SLS is the student-faculty relationship. This study suggests that building relationships with faculty and peers is critical to leadership doctoral students' learning success, even in online-based programs that offer effective technological support.

Introduction
Student learning success (SLS) is everyone's business in higher education. Learning success among doctoral students in growing online programs is a particular concern for three reasons. First, doctoral student completion, an indicator of learning success, is known to be at a lower rate than other educational endeavors.
The PhD Completion Project evaluated doctoral completion rates and attrition patterns across major universities in the United States and Canada and found that only 56.6% of students completed their programs with the lowest completion rates occurring in the social sciences and humanities (Sowell, Zhang, Redd, & King, 2008). Considering that each individual and institution embarking on the PhD journey is investing significant time, money, and intellectual resources, unsuccessful doctoral learning means a substantial waste of resources to the students themselves, their families, the faculty and staff of the institutions, and the intellectual community as a whole.
Second, online degree-granting programs, particularly at the graduate level, are growing significantly in the United States. According to the National Center for Education Statistics (2018), 31.7% of students enrolled in degree-granting postsecondary institutions in 2016 were engaged in distance or online education, either partially or fully. For graduate students, this percentage increased to 36.8%. In 2017, 239 online leadership doctoral programs were offered in the United States, according to our website search of all 50 state departments of education. Online programs provide convenience to graduate students who, while maintaining their work responsibilities, learn anywhere at any time through technology-facilitated tools such as discussion boards, web conferencing, blogging, and social networks (Alammary, Sheard, & Carbone, 2014;Hill, 2012). Online-based education is regarded as the future of higher education, and an increasing number of institutions include online programs in their long-term strategic planning (Allen, Seaman, Poulin, & Straut, 2016;Bayne, Gallagher, & Lamb, 2014). Despite the fact that online-based learning creates different challenges to the learning success of students than face-to-face learning (Kennedy, Terrell, & Lohle, 2015;Lambie, Hayes, Griffith, Limberg, & Mullen, 2014;Rockinson-Szapkiw, Wendt, Whighting, & Nisbet, 2016), the impact of technology on doctoral SLS has not been fully explored.
Third, although the modality of instruction changes, student learning needs based on relationships do not disappear even in online environments. For example, social support from family, friends, and peers has a positive impact on academic self-regulation (Akyol & Garrison, 2011;Williams, Wall, & Fish, 2019) and student learning even in technology-facilitated environments (Gardner, 2009;Garrison, 2007;Lee, 2014).
Students still seek timely feedback, encouragement, and openness as they explore new concepts through productive online dialogue with peers and instructors (Bolliger & Halupa, 2012;Kumar, 2014). In addition, interactions with staff are indicators of service quality and have a direct impact on student loyalty and satisfaction (Martínez-Argüelles & Batalla-Busquets, 2016;Ravindran & Kalpana, 2012).
Considering these problems, this study intends to explore how technological factors (TF) and relational factors (RF) predict doctoral SLS in U.S. online-based leadership programs. The purpose of this study is explored with the following research questions: 63 1. How do technological factors and relational factors separately and interactively predict doctoral student learning success in online-based leadership programs? 2. Which subfactors of the technological and relational factors are the best predictors of doctoral student learning success in online-based leadership programs?

Theoretical Framework
Three constructs-technological factors (TF), relational factors (RF), and student learning success (SLS)make up the theoretical framework of this study. The relationship among these constructs is represented as follows:

Technological Factors
Colleges and universities use technology at various degrees to create online learning environments. Some instruction is delivered fully online, heavily relying on embedded technological features, while others use technology to complement face-to-face instruction. Despite some variations, the common thread is a focus on technology as an integral means of providing instruction. A review of the literature highlights three aspects of technology-facilitated instruction: flexibility, usefulness, and ease of use (Arbaugh, 2000;Bures, Abrami, & Amundsen, 2000;Hart, 2012).
Flexibility, the first technological subfactor, allows students to pursue degrees across geographical, cultural, professional, and generational borders (Bolliger & Halupa, 2012;Sampson, Leonard, Ballenger, & Coleman, 2010). Although doctoral students in online-based programs require discipline and independence to be academically successful, these potential challenges are outweighed by the convenience of utilizing technology to access quality conversations with professors and peers from a distance, while balancing work obligations and family responsibilities with a flexible schedule of academics (Erichsen, Bolliger, & Halupa, 2014;Garrison, 2007). Arbaugh (2000) argued that online learning transcending time and location 64 restriction would enable participants to reach levels of relational intimacy comparable to face-to-face groups, albeit over a longer time period.
Usefulness, the second technological subfactor, refers to the degree to which the technology can enrich and enhance the learning experience (Davis, 1989). Both usefulness and accessibility contribute to the effectiveness of technology (Edmunds, Thorpe, & Conole, 2012;Joo, Lim, & Kim, 2011). A study by Edmunds, Thorpe, and Conole (2012) of 421 university students in the United Kingdom found that the perceived usefulness of technology predicted the actual use of technology for work, school, and social reasons. Arbaugh's (2000) student satisfaction study discovered that graduate management education students who believed technology was valuable and perceived it to be easy to use were more likely to engage in technology for their degree work.
Ease of use, the third technological subfactor, refers to the degree to which technology can be used without undue effort or distraction (Davis, 1989). Ease of use was determined as a critical element affecting student acceptance of technology. A study of technology as a method of course delivery in a study of 136 students in a full-time online-based college program found that student attitude was the most important determinant of the acceptance of technology as a learning tool (Cheung & Vogel, 2013). A positive mindset about technology as a flexible, valuable, and easy-to-use resource motivates toward intentional use of as a means of developing relationships (Davis, 1989;Edmunds et al., 2012;Joo et al., 2011).

Relational Factors
Educational theorists have historically pointed to the integration of academics with social involvement and engagement as critical to student retention up to and including graduation (Tinto, 1999). The community of inquiry framework emphasizes the importance of social presence even when technology is used for learning. It is argued that the social, cognitive, and teaching presence interactively create deep meaning in an academic environment that is mediated by technology (Akyol & Garrison, 2011;Arbaugh et al., 2008;Garrison, 2007;Lai, 2015;Shea & Bidjerano, 2009). The online delivery of instruction does not negate the need for building a sense of school community to increase student satisfaction and retention, but simply changes the methods used to interact (Roach & Lemasters, 2006). As RF, three types of relationships were examined for this study: student-student, student-faculty, and student-non-teaching staff.
Student-student interaction, the first relational subfactor, is considered critical to the individual cognitive development of students in an online higher education environment according to Shea and Bidjerano (2009). A study of graduates' reflections on an online-based doctorate in educational technology determined that well-selected readings, open-ended questions, and guided conversations were influential in promoting interaction between students and critical thinking about the subject matter (Fuller, Risner, Lowder, Hart, & Bachenheimer, 2014). A quantitative content analysis of discussion board messages from two groups of college students found that the online discussion board was an effective means of developing community, which enabled individual members to reason through the topics and construct thought (Lee, 2014).
Student-faculty interaction, the second relational subfactor, has been determined to be the most critical aspect of student satisfaction. The qualities being sought after by the students included timely feedback and responsiveness to questions, attentiveness, encouragement, and sincerity (Bolliger & Halupa, 2012). A study of second-year doctoral students found that 90% of the students credited the instructors for facilitating productive dialogue and providing timely feedback that encouraged the exploration of new concepts (Kumar, 2014). In addition, the student-faculty interaction also influences the future enrollment of the program because their satisfaction is translated into their willingness to recommend the program to others (Martínez-Argüelles & Batalla-Busquets, 2016).
The last relational subfactor, student to non-teaching staff, was also found to be as important to overall satisfaction within online-based higher education programs. Contact personnel in departments such as registration and records are an influencing consideration in student evaluation of the service quality of the university. This satisfaction in service quality leads to the retention and success of students (Ravindran & Kalpana, 2012;Sohail & Shaikh, 2004).

Student Learning Success
The success of the doctoral student is typically culminated by the completion of the dissertation and the attainment of the doctoral degree. However, a deeper exploration of student success addresses academic achievement; engagement in educationally purposeful activities; satisfaction; acquisition of desired knowledge, skills, and competencies; persistence; attainment of educational objectives; and post-college performance (Im & Kang, 2019;Kuh, Kinzie, Schuh, & Whitt, 2010). While educational "success" has been broadly and often studied, York, Gibson, and Rankin (2015) acknowledged a lack of comprehensive instrumentation for measuring success outside of academic achievements such as grades, GPA, and degree attainment. This study created a tool to focus on three specific indicators to predict SLS, or perceived success, in doctoral endeavors by focusing on the gain of knowledge and skills, self-efficacy, and persistence.
All of these are shown to lead to degree completion, which is the ultimate measure of student success (Gardner, 2009;Ivankova & Stick, 2007;Lambie et al., 2014).
Beyond the earned degree, success for doctoral programs is defined as the gain of knowledge and skills which will allow the student to think critically and creatively (Gardner, 2009). A survey of 131 graduate students found that students who actively engaged in the online learning community both socially and cognitively had a greater sense of perceived scholarship that contributed to their course success (Rockinson-Szapkiw et al., 2016). For doctoral students, active engagement in learning, resulting in the perceived gain of knowledge and skills, is considered critical to developing self-efficacy.
Successful completers of doctoral programs are likely to be students who believe in their own ability to conduct empirical research and successfully write research findings. An exploratory investigation of PhD education students found the self-efficacy of students increased with the completion of classes and involvement in research opportunities (Lambie et al., 2014). Bandura (1997) equates self-efficacy with a person's choices, goals, expended effort, and willingness to persist in the face of adversity. Self-efficacy can cause students to either obstruct their own progress through self-destructive stress or raise a student above Persistence, leading to degree completion, is considered a measure of institutional and programmatic success. The rate of doctoral students who fail to earn their PhDs is approximately 50% in the social science, humanities, and educational arenas. This number goes 10% to 15% higher for students enrolled in technology-based programs (Kennedy et al., 2015). In an online-based learning environment, mentoring and faculty support allow the doctoral student to persist in independently conducting, analyzing, and presenting research in completion of the doctoral program (Ampaw & Jaeger, 2012;Erichsen et al., 2014).
In summary, the theoretical framework of this study connects three constructs: TF, RF, and SLS. The first construct, TF, which serves as an independent variable, consists of three subfactors: flexibility, usefulness, and ease of use. The second construct, RF, also serves as an independent variable and focuses on studentstudent, student-faculty, and student-non-teaching staff relationship. Finally, the construct of SLS, the dependent variable, consists of three subfactors: gain of knowledge and skills, successful completion, and persistence. The relationship between this dependent variable of SLS and two independent variables-TF and RF-was established based on the studies discussed in this section.

Methods
This correlational study engaged 210 doctoral students from 26 online-based leadership doctoral programs in the United States. This section describes the context, participants, instruments, and data collection and analysis in detail.

Context and Participants
This study involved doctoral students from programs that offer a PhD, EdD, or PsyD with "leadership" in their degree titles and that deliver instruction in fully or partially online environments. All leadership doctoral programs in U.S. higher education institutions were identified, drawing upon doctoral program directories, compiled and shared by individual leadership scholars or organizations, and websites of all 50 state higher education agencies. Website information on each program was examined to determine if learning was delivered online. If not readily identified, further investigation was done, including an examination of course catalogs or schedules. It must be recognized that, while extensive, the Web search was only as accurate as the information provided on the website of each university. The demographics of the respondents are summarized in Table 1.

Instruments
The "Online Learning Success Scale (OLSS)" was constructed, drawing upon nine existing scales listed in the "References" column in Table 2. OLSS measures three major variables: technological factors, relational factors, and student learning success with three subfactors for each variable (see the "Factors" and "Subfactors" columns in Table 2). Some conceptual categories and questions were modified to measure the constructs intended for the study and doctoral leadership contexts. Cronbach's alpha was used to measure the reliability of each variable of the final OLSS (see the "Reliability Coefficients" column in Table 2). The reliability values of the factors ranged from .936 to .949, and those of the subfactors ranged from .857 to .967.
The first independent variable, TF, consists of three subfactors: (a) usefulness, (b) flexibility, and (c) ease of use. "Usefulness" refers to the positive impact of the online delivery system on students' learning and doctoral experience; "flexibility" to the advantages of using a technological tool to overcome time and geographic limitations; and "ease of use" to the minimal effort involved in engaging within an online platform. The second independent variable, RF, consists of three subfactors: (a) student-student Doctoral Students' Learning Success in Online-Based Leadership Programs Lee, Chang, and Bryan 68 relationship, (b) student-faculty relationship, and (c) student-non-teaching staff relationship. "Studentstudent relationship" refers to students' connectedness with their peers and the feeling of community within their leadership program; "student-faculty" to students' connectedness and ability to communicate with faculty; and "student-non-teaching staff relationship" to students' connectedness with and perceived helpfulness of the non-teaching staff.
The dependent variable, SLS, consists of three subfactors, (a) gain of knowledge and skills, (b) self-efficacy, and (c) persistence. "Gain of knowledge and skills" refers to students' perceived gain of knowledge and skills pertaining to leadership; "self-efficacy" to their ability to apply their knowledge and skills to their leadership practice and to conduct original research; and "persistence" to their commitment to finishing the program in their current institution. .936

Data Collection and Analyses
The OLSS was transposed in Qualtrics, an online survey software for collecting data. An introduction, containing the link to the survey, was sent via email to a comprehensive list of 239 online-based leadership doctoral program directors in three rounds of distribution with one reminder for each round. Program directors who accepted the participation invitation sent the survey link to their students and recent graduates. Directors who did not act on our invitation either did not communicate with the researchers at all or cited various reasons for their decline, such as institution IRB rules, too many study requests, program not beginning until the next year, lack of program participation, and lack of online components in the program.
Initially, 276 respondents participated in the survey. Two respondents did not consent to participate and

Results
This section reports three types of results: descriptive statistics, predictability of TF and RF on SLS, and effects of technological and relational subfactors on SLS.

Descriptive Statistics
Various subdivisions of 210 responses to the survey represent slightly different pictures of SLS, RF, and TF. Although there were no significant mean differences in gender, age, delivery model, and students' status, the mean scores of the TF differ by delivery models: 100% online model (4.01), blended model (3.86), and primarily face-to-face (3.42). In terms of RF, the mean score of 100% online students was 3.87, lower than the mean score of 4.21 of respondents in blended and face-to-face programs. The similar importance of the RF for respondents in blended and face-to-face programs was reinforced through text answers provided in the survey. Note. SLS (student learning success), TF (technological factor), RF (relational factor).

71
To investigate the relationships among TF, RF, and SLS, Pearson's correlation coefficients were calculated. Table 4 presents the correlation coefficients for the subvariables of the three main variables: i.e., TF, RF, and SLS. The correlational results from the survey indicated that all subfactors in SLS were significantly correlated with all subfactors of both TF and RF in a positive direction except for two subvariables. Namely, correlations between the flexibility of TF and the student-student relationship of RF (r = .052) and between the flexibility of TF and the student-faculty relationship of RF (r = .13) were not statistically significant.
Moreover, stronger correlations were found between RF and SLS compared to between TF and SLS. The most significant positive correlation was noted between the persistence of SLS and the student-faculty relationship of RF (r = .777, p <. 01). Note. SLS (student learning success), TF (technological factors), RF (relational factors), KS (gain of knowledge and skills), SE (self-efficacy), PE (persistence), US (usefulness), FL (flexibility), EU (ease of use), SS (student-student), SF (student-faculty), SN (student-non-teaching staff).
*p < .05, **p < .01 The significant correlations among the variables do not mean that all the variables have casual relationships, and thus it is necessary to undertake regression analysis to examine the relationships among the variables.

Predictability of Technological and Relational Factors on Student Learning Success
This section presents the results of Research Question 1: How do technological factors and relational factors separately and interactively predict doctoral student learning success in online-based leadership programs?
Multiple regression analysis was conducted to determine if technological and relational factors affected student learning success significantly in terms of gain of knowledge and skills, self-efficacy, and persistence.

72
According to the results of the multiple regression analysis, TF and RF together significantly predicted SLS (R 2 =.465, F = 89.903, p = .000). Moreover, TF and RF respectively affected SLS significantly. RF (t = 11.382, p = .000) especially affected SLS more significantly than TF (t = 3.209, p = .002). In addition, if a variance inflation factor (VIF) is 10 or more, it is assumed that there is a multicollinearity (Kutner, Nachtsheim, & Neter, 2004), and thus there is no multicollinearity between TF and RF (VIF < 10; see Table   5).

Effects of Technological and Relational Subfactors on Student Learning Success
This section reports on the results in response to Research Question 2: Which subfactors of the technological and relational factors are the best predictors of doctoral student learning success in onlinebased leadership programs? To identify which subfactors of the technological and relational factors were the best predictors of student learning success, another multiple regression analysis was performed.
According to the results of the multiple regression analysis, all the subfactors of both TF and RF predicted SLS significantly (R 2 =.500, F = 33.867, p = .000). However, the technological subfactors-usefulness, flexibility, and ease of use-and one relational subfactor between the student and the non-teaching staff separately did not predict SLS. Only two of the relational subfactors, namely student-student relationship (t = 4.436, p = .000) and student-faculty relationship (t = 6.591, p = .000), were statistically significant regarding the effects on SLS. The student-faculty relationship particularly was the best predictor of SLS (t = 6.591, p = .000). There is no multicollinearity between the subfactors of the TF and the RF (VIF < 10; see Table 6).

Discussion and Conclusion
Faced with the increasing importance of distance learning as a preferred means of obtaining a degree at the graduate levels, including the doctoral level (National Center for Education Statistics, 2018), all higher education institutions and programs must consider the impact of technology and relationships, individually and interactively, within the online environment. The intent of this study was to determine how TF and RF related to the SLS of students engaged in the U.S. doctoral leadership programs.

74
An analysis of the data collected from this study found that significant correlations exist, which confirms the importance of both technology and human relationships in the learning success of doctoral students in online-based learning environments. Persistence, students' determination to continue to completion, is most significantly related to RF for respondents from blended or 100% online programs. This result corresponds with similar studies that have established connectedness and social integration as critical to the likelihood of doctoral students persisting within the coursework and candidacy stages of the program (Kennedy et al., 2015;Martínez-Argüelles & Batalla-Busquets, 2016;Rockinson-Szapkiw et al., 2016).
Another finding indicates that all three RF (student-student, student-faculty, and student-non-teaching staff) were collectively and separately better predictors than the TF for doctoral SLS. Whether the success is defined as a gain of knowledge and skills, self-efficacy, or persistence, these results concur with similar studies (Kennedy et al., 2015;Lambie et al., 2014;Rockinson-Szapkiw et al., 2016). Interviews with doctoral students at a research-intensive university in New Zealand found that technology was an effective means of facilitating the development of learning communities to construct meaningful knowledge and share individual experiences (Lai, 2015).
Technology is important, but it seems to be a means to the end of student learning, secondary to relationships. Our study found that the student-faculty relationship was the subfactor with the strongest predicting power to SLS. The instructor is the pivotal participant in the online learning experience, helping to facilitate productive dialogue, encouraging the exploration of new concepts, and providing timely feedback (Augustsson & Jaldemark, 2013;Kumar, 2014). An integrated literature review by Hart (2012) identified connectedness, belonging, and support as important factors that went beyond the content to motivating students to overcome hardships and persist in the online-based environment. A grounded theory study of students in a limited-residency program found that the greatest factor for not completing the doctoral work, especially in the dissertation phase, was a lack of supportive interaction (Kennedy et al., 2015).
This is not to negate the correlation of TF with student success. Of the three subfactors of SLS, self-efficacy correlated most significantly with the TF of blended or online learning. One explanation is that this study surveyed doctoral students who have already experienced academic success. A meta-analysis of withinperson self-efficacy found that self-efficacy was a product of past performance rather than a predictor of future performance (Sitzmann & Yeo, 2013). The self-efficacy of doctoral students then increases as courses are completed and aligned with research opportunities (Lambie et al., 2014). The very definition of selfefficacy involves the ability of an individual to identify the contexts for which the individual has the skills and ability to succeed (Celik & Yesilyurt, 2013). There is an integration of knowledge and skills that doctoral leadership programs should be aware of to create successful technology-based learning opportunities that are associated with increased self-efficacy.
In summary, the results from this study lead to a conclusion that both TF and RF predict learning success as perceived by students enrolled in online-based doctoral leadership programs in the United States. The study found that RF predict SLS better than TF, particularly the student-faculty relationship. Distance education programs must purposefully develop support systems, such as the cohort model, that encourage 75 connectedness and social integration (Kennedy et al., 2015;Williams et al., 2019). Administrators, faculty, and staff of distance education programs must be prepared to facilitate communication using technology, and understand the importance of timely responses to students at all phases of the doctoral program (Gardner, 2009;Rockinson-Szapkiw et al., 2016).
This study has several limitations that might have affected the findings. Regarding the program and participant selection, the study had limited data caused by several uncontrollable conditions. Information on the individual institution websites was often incomplete or outdated, which made it difficult to accurately determine the online nature of the programs. This difficulty was compounded by the whole spectrum of terms that can be used that describe an online program (Anohina, 2005). In addition, a good number of eligible programs or participants were inaccessible due to their institutional or programmatic constraints and unresponsiveness of directors or student participants.
While this study found no difference by gender, status, or age, there was a gender imbalance with two-thirds of the respondents being female. A review of the literature finds mixed results with regards to gender and relational preference. A study of 12 online-based graduate courses found that female students felt more connected with their peers and perceived that they learned more than their male counterparts in the courses, while a study of students in Taiwan found that the differences were related to status in the college program (Hung, Chou, Chen, & Own, 2010;Rovai & Baker, 2005). Other studies, like this one, found no differences in the success or satisfaction of students by gender, status, or age (Cho & Kim, 2013;Martin, 2005). Lastly, this study engaged only programs based in the United States, creating an issue of the difficulty of generalization. However, similar studies in different contexts also have concluded that relationships are the critical factor in the success of students in the online-based educational environment (Fuller et al., 2014;Lai, 2015;Roach & Lemasters, 2006;Sohail & Shaikh, 2004).
Based on these limitations, further study is recommended to engage a more balanced set of participants by gender, age, and degree type. Secondly, further study could expand the research beyond the leadership discipline or the U.S. context, between disciplines, or among different contexts. Thirdly, qualitative studies around online doctoral leadership programs could provide a holistic understanding of programs and doctoral SLS by gaining multiple perspectives from program directors, faculty, students, and alumni beyond pre-selected variables such as TF and RF.