The Use of a Mobile Learning Management System at an Online University and Its Effect on Learning Satisfaction and Achievement

This study investigates online students’ acceptance of mobile learning and its influence on learning achievement using an information system success and extended technology acceptance model (TAM). Structural equation modeling was used to test the structure of individual, social, and systemic factors influencing mobile learning’s acceptance, and how said acceptance influences learning satisfaction and achievement. Unlike earlier TAM-related research that did not provide a broad view of technological acceptance and its impact on learning activities, the present study’s results highlight the relationship between behavioral intention/learning satisfaction and learning achievement. Additionally, this study tests the theoretical model of successful mobile learning by empirically accepting mobile learning management systems. The findings further imply that students at online universities have started to accept mobile technology as a new learning tool; consequently, its acceptance has influenced their learning achievement both directly and indirectly. These discoveries should facilitate a better understanding of students’ usage of mobile learning systems in higher education, and provide timely guidance for its development and implementation.


Article abstract
This study investigates online students' acceptance of mobile learning and its influence on learning achievement using an information system success and extended technology acceptance model (TAM). Structural equation modeling was used to test the structure of individual, social, and systemic factors influencing mobile learning's acceptance, and how said acceptance influences learning satisfaction and achievement. Unlike earlier TAM-related research that did not provide a broad view of technological acceptance and its impact on learning activities, the present study's results highlight the relationship between behavioral intention/learning satisfaction and learning achievement. Additionally, this study tests the theoretical model of successful mobile learning by empirically accepting mobile learning management systems. The findings further imply that students at online universities have started to accept mobile technology as a new learning tool; consequently, its acceptance has influenced their learning achievement both directly and indirectly. These discoveries should facilitate a better understanding of students' usage of mobile learning systems in higher education, and provide timely guidance for its development and implementation.

111
As mobile technology has matured sufficiently in recent years to support advanced learning activities, its adaptation for this purpose has spread globally. This is natural since students are surrounded by mobile technology in their daily lives. Additionally, the proliferation of advanced wireless technologies has facilitated learning on the go, whereby individuals can access educational content regardless of their location. Mobile technology also offers various opportunities for timely and active knowledge acquisition through the exchange of learning materials (Woodill, 2011;Jones, Scanlon, & Clough, 2013).
Due to its wide range of benefits (e.g., cost effectiveness, ubiquity, location-based services, and potential as a study aid), mobile learning is expected to play a significant role in a multitude of educational settings (Cheon, Lee, Crooks, & Song, 2012). Most importantly, mobile technology possesses great potential in offering rich multimedia experiences and resources of a varied nature while enabling students to learn without being restricted by time or location in both formal and informal educational settings (i.e., seamless learning) (Lam, Yau, & Cheung, 2010;Milrad, Wong, Sharples, Hwang, Looi, Ogata, 2013). Consequently, educational institutions specializing in open and distance learning have attempted to develop applications such as mobile learning management systems (LMSs) for students enrolled in e-learning courses. Indeed, the swift deployment of such systems by online educational institutions should be a priority given continuously increasing student demand.
The availability of mobile technology does not actually guarantee that it will be used in an educational setting (Hwang & Chang, 2011); similarly, the mere adoption of a new technology does not assure its learning effectiveness. Nevertheless, many educational institutions have attempted to support online learning activities, while researchers have also begun placing greater focus on the utilization of mobile LMSs. Still, mobile learning at institutions of higher education remains in its infancy (Park, 2011;Cheon et al., 2012). Research concerning mobile learning has generally focused either on its effectiveness as a learning aid, or on approaches to designing such systems (Chu, Hwang, Tsai, & Tseng, 2010;W. Wu, J. Wu, Chen, Kao, Lin, & Huang, 2012).
Additionally, existing studies have focused primarily on mobile learning as a potential type of informal learning (Wang & Chang, 2011;Jones et al., 2013;Martin & Ertzberger, 2013), or on its effectiveness in supplementing formal learning (Gikas & Grant, 2013). Studies examining situations in which all elements of learning occur through a mobile device are scant, however.
Furthermore, few studies have investigated the impact of mobile LMSs on student learning activities in regular courses at formal educational institutions. Further research is necessary to determine why students use mobile LMSs and their effect on learning satisfaction (LS) and achievement.
Recent research has adopted the technology acceptance model (TAM) as an explanatory tool for investigating the technological learning process (Park, 2009), allowing researchers to identify how students adopt mobile learning approaches (Park, Nam, & Cha, 2012). Notwithstanding the TAM's adaptability, arguments exist in favor of incorporating additional socio-cultural and 112 organizational factors into the model that were originally unexplored (Teo, 2009). Indeed, TAM's two constructs (i.e., ease of use and usefulness) may not fully capture the components necessary to predict students' acceptance of technology (Edmunds, Thorpe, & Conole, 2012); as such, other variables should be considered to provide a broader view and improved explanation of technological adoption (Legris, Ingham, & Collerette, 2003). These unexplored factors, which have not been studied in mobile learning contexts, should be investigated to identify influential factors involved in applying an information system success (ISS) model to gauge student intentions toward adopting mobile learning, satisfaction, and learning achievement (LA).
Moreover, a sparse amount of research has explored mobile LMS adoption and its influence on LS and achievement from the perspective of learning success. Therefore, this paper tackles the intention to implement mobile LMSs and the resultant influence on LS and achievement by combining the TAM and ISS model.

Background TAM
Researchers have spent many years attempting to develop and test models for predicting technology acceptance. In most m-learning studies, adaptations of the theory of reasoned action (TRA) (Fishbein & Ajzen, 1975), the theory of planned behavior (TPB) (Ajzen, 1991), and the TAM (Davis, 1989) have been proven to predict technology acceptance behavior successfully (Cheon et al., 2012). The TAM, which is derived from the TRA, has gained recognition as the most useful among them for investigating the acceptance of novel technologies, and it covers significant factors affecting their use. Moreover, since its introduction the TAM has been extensively tested and validated empirically by scholars in various fields and contexts to explain user beliefintention-behavior across a broad range of computer-related technologies (Davis, Bagozzi, & Warshaw, 1989;McCoy, Galletta, & King, 2007;Teo, 2009;Giesbers, Rienties, Tempelaar, & Gijselaers, 2013).
The TAM comprises factors affecting behavioral intentions in technology use, and demonstrates the effects of self-efficacy and outcome expectations (the perceived ease of use and usefulness of a technology [Davis & Venkatesh, 1996]) on attitudes toward technology use. Technology acceptance in this model entails four main factors: perceived ease of use (PEU), perceived usefulness (PU), attitudes toward technology use, and behavioral intention (BI). Among these predictors, PU and PEU are hypothesized to be the fundamental determinants of user acceptance, a notion verified through empirical support (Gibson, Harris, & Colaric, 2008). PU is the extent to which a person believes using a particular technology will enhance his or her job performance, 113 while PEU refers to the degree of simplicity a prospective user expects from a target system .

Extended-Technology Acceptance Model (eTAM)
Although the TAM has been successfully adapted to explain the acceptance and usage of various forms of technology, some researchers insist that further investigation is necessary to identify additional factors that may influence technology acceptance. Teo (2010), for example, advocates integrating technological complexity, computer self-efficacy, and environmental/organizational support, in addition to other relevant and unexplored factors. Critics of TAM, such as Legris et al. (2003), assert that other variables should be considered in order to facilitate a broader view and more thorough understanding of technology adoption. In other words, the eTAM is a prerequisite for developing a genuine understanding of technology use. SE was selected as an important construct based on the findings of Compeau and Higgins (1995), who reported that higher SE levels with respect to computers lead to greater behavioral intention and use of information technology. Furthermore, Lewis, Agarwal, and Sambamurthy (2003) examined the effects of SE and PI on PEU and PU among 161 university staff members, finding that PI exhibited a significant effect on ease of use and usefulness. In another study by Liu, Li, and Carlsson (2010), the TAM was applied to examine PI's effects on Chinese university students' intention to use mobile learning; the results revealed that, in the long term, PI positively influences PU. Furthermore, Park (2009) found that SN exhibit a significantly positive influence on PU.
Regarding social factors, Fishbein and Ajzen (1975) examined the relationship between SN, perceived pressure to perform a given behavior, and one's motivation to comply with said pressures; since then, researchers have focused on SN when attempting to demonstrate correlations between behavioral intentions. While early TAMs did not consider SN influential (Davis, 1989), models were later revised as the influence of SN on the acceptance of technology became well established (Venkatesh & Davis, 2000). Hence, contemporary researchers give credence to SN due to its proven, significant relationship to the intention to use mobile learning (Cheon et al., 2012).

114
Finally, this study considers relative advantage (RA) and SA as system factors. RA, which is rooted in Rogers' (2005) innovation diffusion theory, measures to what extent an innovation improved upon its predecessors. Although it was not deemed influential in early TAMs, it is currently considered a dominant factor affecting technology adoption since its importance was proposed by Venkatesh, Morris, G. B. Davis, & F. D. Davis (2003), who argue that while some researchers may insist that PU is similar to RA, differences become apparent between them in discussions concerning performance expectancy. Additionally, SA could be regarded as an important factor affecting BI. Park et al. (2012) insist that SA as an organizational factor is among the dominant exogenous constructs affecting BI toward mobile learning, since a wireless Internet connection is required to facilitate mobile learning activities. In terms of mobile learning employing the TAM, Park et al. (2012) investigated mobile learning intention among Korean college students and determined that attitude was the most important construct in explaining the acceptance of mobile learning. In contrast, Liu et al. (2010) found that PU and PI were the two most influential factors in adopting mobile learning among Chinese college students. As mentioned earlier, most prior research has focused on the general adoption of new technology related to mobile learning, not college students' adoption of mobile learning and its relation to satisfaction and LA specifically; this is because said studies primarily examine students' acceptance of mobile learning from the standpoint of intention (Cheon et al., 2012).

Mobile Learning with the TAM and ISS Model
Ultimately, research on the relationship between technology adoption and learner achievement would be more significant if the effect of new technology on learning were considered. Studies examining the path to LA that take advantage of the TAM and ISS model are influential in this regard.
By referring to literature involving mobile learning and ISS (e.g., Freeze et al., 2010;Park et al., 2012), this study developed an integrated model based on both the TAM and ISS model ( Figure   1), which is best suited for investigating students' intention to use mobile learning systems, in addition to their satisfaction and LA. That is, this study examines relationships among PU, PEU,

Participants and Context
The study's participants included 1,117 undergraduate students enrolled in a South Korean online university. Upon registering for courses students were offered to select between PC (traditional) or mobile-based learning, the latter being periodically augmented by in-class lectures. Mobile learners used devices such as smartphones and tablets to stream classroom lectures and post 116 questions or messages by means of the mobile LMS. Participant demographics, including gender, age, and grade level are shown in Table 1.

Data Collection
Questionnaires were distributed using the university's LMS and results were collected for fourteen days between May 20 and June 3, 2013. In total, 1,304 questionnaires were received, of which 187 were excluded from analysis that contained either blank or invalid responses (i.e., identical answers for each question). Hence, 1,117 surveys were analyzed.

Survey Instrument
A survey instrument was designed to measure ten constructs using a five-point Likert scale. The survey comprises two sections, the first of which contains questions concerning demographic factors, followed by 31 questions regarding the ten constructs. The ten constructs included two questions for SE, four questions each for RA and LA, and three questions each for PI, SN, SA, PU, PEU, BI, and LS. Three experts in related fields reviewed each question to verify content validity.
The definitions and sources for the ten constructs are provided in Table 2.

Results
Data analyses comprised two steps. First, the measurement model's convergent and discriminant validity was examined. Next, the hypotheses were tested by evaluating the path model and assessing model fit using various indices.

Convergent validity.
Three variables encompass the convergent validity test: each measure's item reliability, each construct's composite reliability, and the average variance extracted (AVE) (Fornell & Larcker, 1981). All indicator factor loadings for each measure's item reliability require significance (i.e., a value equal to or greater than 0.7). The results showed that all items' factor loadings were significant, ranging between 0.711 and 0.931; hence, all items obtained convergent reliability. As for composite reliability, it requires a value equal to or greater than .7. Composite reliability in this study ranged between .802 and .932, indicating that all constructs obtained composite reliability. Finally, AVE requires a value equal to or greater than .50; this condition was also met, with values exceeding .50. These findings, which are summarized in Table 3, conclusively demonstrate that the measurement model's convergent validity is adequate.

Discriminant validity measures if a construct differs with another construct and its indicators,
which is determined by examining whether the AVE's square root is over the correlation coefficient between constructs (Segars & Grover, 1998). Table 4 shows that the AVE's square root 120 is greater than the correlation coefficient between constructs, thereby validating the research instrument.

Model Fit and Hypothesis Testing
The chi-square ( ) score for testing model fit was 1845.162 (p<.001), invalidating the null hypothesis and necessitating another goodness-of-fit index (Bentler, 1989). Model fit was confirmed by testing the model fit index (Hu & Bentler, 1999), which considers parsimony but not sample size.
Root mean square error of approximation (RMSEA) was used as a measure of absolute fit, while the comparative fit index (CFI) and Tucker-Lewis index (TLI) were used to measure incremental fit. According to Bentler (1990) and Tucker and Lewis (1973), CFI and TLI values of .90 or more reflect good fit; comparatively, for RMSEA values between .06 to .08 are considered a good fit, and those below .06 excellent (Hu & Bentler, 1999). Based on these standards, model fit in the present study reflected a good-fit, with CFI, TLI, and RMSEA values of .945, .937, and .056 respectively.

121
For the hypothesis testing, the t-value results are shown in Figure 2. Table 5 shows the direct, indirect, and total effects of determinants on the endogenous variables.

Discussion and Conclusion
This study investigated online students' acceptance of mobile learning and its influence on LA using an eTAM and ISS model. Furthermore, SEM was employed to test the structure of individual, social, and systemic factors influencing the acceptance of mobile learning, and how its acceptance influenced LS and LA. Several implications were derived from the results, which are discussed below.
First, the results revealed that SE, PI, and SA have a significant influence on PEU. The determination that SE and PI significantly influences PEU supports the notion that learners' SE and PI positively influence technology use, a finding mirrored by Lewis et al. (2003). This implies that intrinsic personal factors such as SE and PI positively impact mobile LMS use. Moreover, the finding that SA has positive effect on PEU coincides with prior studies that adopted a TAM, such as Park et al. (2012). It further demonstrates the importance of systems that provide high-speed wireless Internet access, compatibility between mobile/PC-based LMSs, and a user-friendly design that facilitates searching for information and learning content using mobile devices.
However, SN and RA did not influence PEU, indicating that SN and RA as social and system factors respectively are not directly related to mobile LMS use, thus coinciding with previous studies (i.e., Venkatesh et al., 2003;Cheon et al., 2012)  Second, SN and RA exhibited a significant and positive influence on PU. This is in agreement with Legris et al. (2003), Park (2009), andHuang et al. (2014), who found a correlation between students' desire to remain current with societal changes engendered in technological development while integrating contemporary technology into their learning activities. Thus, based on the perception that rapidly embracing technology can aid one's survival in society, students may adopt a positive attitude toward mobile learning. The aforementioned influence of RA on PU also coincides with Venkatesh et al. (2003), who concluded that learners expecting mobile LMSs to outperform preexisting technology perceive its usefulness. Such individuals consider mobile learning a complementary alternative to traditional tools, which promote more effective and easier learning, and hence hold a positive outlook concerning their use. In contrast, SE, PI, and SA did not exhibit a positive or direct influence on PU. Thus, confidence and openness toward mobile LMSs does not necessarily imply a belief in its usefulness (Joo, Lee, & Ham, 2014).
Furthermore, SA does not significantly influence PU, a finding that is consistent with prior studies examining factors affecting mobile learning in Korea (Park et al., 2012 Third, PU is the most influential factor affecting BI in students' intention to use mobile learning systems, a finding that is in agreement with Park et al. (2012) andHuang et. al (2014). PU significantly and directly influences BI, while PEU and other exogenous variables (including SE) indirectly influence BI by means of PU. There are two possible interpretations for these results.
The first is that students adopt mobile learning based on its perceived usefulness, while the second is that since students enrolled in an online university are already very familiar with mobile learning, PEU has no direct effect on their intention to use such technologies. Thus, a key factor determining the acceptance of mobile learning is whether students are inclined not only to consider its ease of use, but other characteristics as well. Indeed, the importance of remaining current with technical trends in mobile learning cannot be overemphasized, and instructors must ensure that students are afforded a comfortable mobile learning environment while continuously receiving information of relative advantage to mobile learning (Han & Han, 2014). Additionally, unlike prior TAM research examining online distance and mobile learning, the present study reflects the ISS model, which empirically shows that the adoption of mobile technology contributes to successful learning. The acceptance of mobile LMSs as a new technology directly impacts student learning satisfaction and achievement. These results suggest that positive support from instructors and academic institutions and diffusion are necessary for the adoption of mobile learning in online distance learning; further research must be carried out in this regard.
In sum, the present study is significant in that it comprehensively examined factors partly considered by the TAM and eTAM. Specifically, it demonstrated a relationship between BI/LS and LA, a feat unaccomplished by prior TAM-related research that did not provide a broad view of technology acceptance and its impact on learning activities. Indeed, enrollees at online universities have begun accepting mobile technology as a new learning tool; in turn, this acceptance has influenced their LA both directly and indirectly. Furthermore, the results should elucidate student usage of mobile learning systems in higher education, aiding stakeholders such as instructors and academic staff in creating and developing similar systems and learning environments. Moreover, the results yield timely information for further development and implementation. As the application of mobile learning in education has been widely researched and is spreading rapidly to open distance learning, this study is timely and has great relevance for future educational research.

125
While this study bears significant implications for providing guidelines to support mobile learning systems, the generalizability of its results are limited. As one of the world's most technologically advanced countries, approximately 10% of Korea's formal universities operate online.
Consequently, the study's results reflect a unique context that may not be transferrable to other educational systems, and similar studies should be conducted in different educational contexts.
Second, the study is limited in terms of statistics, since LA reflected student perception rather than actual data. Analyses based on quantifiable data such as test scores could strengthen the study's results. Furthermore, not all variables that could significantly affect student adoption of mobile learning systems were examined. Therefore, the SEM results may have differed if other variables were considered. Finally, data analysis was based on self-reported information, which can be susceptible to response bias. From a methodological perspective, in-depth student interviews could reinforce the study's results by strengthening their perception and satisfaction of mobile learning systems.