International Review of Research in Open and Distributed Learning Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns

This study aims to apply a sequential analysis to explore the effect of learning motivation on online reading behavioral patterns. The study’s participants consisted of 160 graduate students who were classified into three group types: low reading duration with low motivation, low reading duration with high motivation, and high reading duration based on a second-order cluster analysis. After performing a sequential analysis, this study reveals that highly motivated students exhibited a relatively serious reading pattern in a multi-tasking learning environment, and that online reading duration was a Abstract This study aims to apply a sequential analysis to explore the effect of learning motivation on online reading behavioral patterns. The study’s participants consisted of 160 graduate students who were classified into three group types: low reading duration with low motivation, low reading duration with high motivation, and high reading duration based on a second-order cluster analysis. After performing a sequential analysis, this study reveals that highly motivated students exhibited a relatively serious reading pattern in a multitasking learning environment, and that online reading duration was a significant indicator of motivation in taking an online course. Finally, recommendations were provided to instructors and researchers based on the results of the study. This study aims to explore the effects of learning motivation on online reading behavioral patterns. In response to the research question (1), the study found that the high reading behavior duration group presented a higher ratio of intensive reading and a lower ratio of non-completion of the unit test. However, the other two groups (low reading duration with low motivation and low reading duration with high motivation groups) manifested a lower proportion of intensive reading behavior, higher proportion of skim reading behavior, and a higher proportion of non-completion of the unit. In response to research question (2), we discovered that, after taking a unit test, the low reading duration with low motivation and low reading duration with high motivation groups had a behavioral tendency to bypass the reading materials and instead participate directly in the next unit test. In addition, the behavioral patterns of the low reading duration with high motivation group showed the characteristic of interchange between intensive reading and multi-tasked reading. In summary, this study indicates that “ online reading duration ” in the online learning system is a strong indicator for reading motivation in students. In addition, this study proposes that multi-tasking learning should be considered in the future design of online learning courses to provide appropriate teaching strategies or platform functions. Learning motivation is an important factor in enhancing “ intensive reading ” in students. With the aid of online course materials, instructors can further


Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns
Jerry Chih-Yuan Sun, Che-Tsun Lin and Chien Chou  1492-3831 (digital) Explore this journal Cite this article Sun, J.-Y., Lin, C.-T. & Chou, C. (2018). Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns. International Review of Research in Open and Distributed Learning,19(2). https://doi. org/10.19173/irrodl.v19i2.2853 Article abstract This study aims to apply a sequential analysis to explore the effect of learning motivation on online reading behavioral patterns. The study's participants consisted of 160 graduate students who were classified into three group types: low reading duration with low motivation, low reading duration with high motivation, and high reading duration based on a second-order cluster analysis. After performing a sequential analysis, this study reveals that highly motivated students exhibited a relatively serious reading pattern in a multi-tasking learning environment, and that online reading duration was a significant indicator of motivation in taking an online course. Finally, recommendations were provided to instructors and researchers based on the results of the study.

Introduction and Background
In recent years, learning analytics has been a hot topic in information technology. It is a technology that allows for the analysis of highly complex information and deals with the professional storage, management, analysis, and imaging of such complex information (Chen, Chiang, & Storey, 2012). Furthermore, the technology of learning analytics involves administration, research, teaching, learning, and support resources (Long & Siemens, 2011). Led by the trend in the recent innovation of information technology, learning analytics in education is an application which gathers large amounts of data to process, collect, measure, and analyze the content or behavior of students generated in the online learning process. The information obtained from learning analytics can allow us to gain a more in-depth understanding and determine the best methods in learning and teaching. Learning analytics was listed on the NMC Horizon Report from 2011 to 2014 as an innovative and potential agenda item in higher education in recent years, underscoring its importance in education research (Johnson, Adams, & Cummins, 2012;Johnson Adams, Cummins, Estrada, Freeman, & Ludgate, 2013;Johnson, Becker, Estrada, & Freeman, 2014).
Learning analytics was formed through two coinciding trends: the increase in the application of virtual learning environments in education, and the expansion of data mining techniques used in business organizations (Agudo-Peregrina, Iglesias-Pradas, Conde-González, & Hernández-García, 2014). Based on this viewpoint, Agudo-Peregrina et al. (2014) believed that learning analytics is often regarded as educational data mining (EDM). The International Educational Data Mining Society defines EDM as follows: Educational Data Mining is an emerging discipline, concerned with developing methods for exploring the unique and increasingly large-scale data that come from educational settings, and using those methods to better understand students and the settings in which they learn in. The methods utilized by EDM include statistics, graphics technology, social network analysis (SNA), online analytical processes (OLAP), regression, cluster analysis, correlation analysis, and text mining (Black, Dawson, & Priem, 2008). Macfadyen and Dawson (2010) considered that learning behavior tracking logs, such as on-line hours, the number of interaction in discussion forums, and the duration and time required to complete the online test, were meaningful data for analysis. This study investigated learners' reading behavior in an online context through a behavioral sequence analysis, which is a technique of learning analytics. E-learning is a teaching format that combines the application of education and information technology, and which allows students and instructors to engage in learning activities at different temporal and spatial distances (Raab, Ellis, & Abdon, 2001). However, after reviewing students' online activities such as login frequency, frequency of accessing course materials, number of bulletin board messages posted, and number of synchronous discussions attended, Hung and Zhang (2008) found that the majority of the students tended to demonstrate passive learning behavioral patterns. In other words, most online learning activities involved reading of materials rather than posting messages or participating in synchronous discussions. As a result, Chen and Jang (2010) proposed to incorporate learning motivation into learning analytics to further explore the relationship between student's motivation, online learning behavior, and learning achievement. Learning motivation is a psychological variable that significantly impacts learning behavior, engagement, effort, and learning achievement (Bandura, 1977;Schunk, Meece, & Pintrich, 2013;Sun & Rueda, 2012). Learning motivation affects the students' choice of learning tasks, effort, and degree of perseverance that is invested in certain tasks, as well as the mental state of the student (Schunk et al., 2013). Deci and Ryan (1985) defined motivation both in terms of intrinsic motivation and extrinsic motivation Applying Learning Analytics to Explore the Effects of Motivation on Online Students' Reading Behavioral Patterns Sun, Lin, and Chou 211 according to the self-determination theory. Empirical studies have pointed out that both intrinsic motivation and extrinsic motivation influence learning attitude, self-learning efficacy, learning achievement, and overall acceptance of e-learning (Saadé, He, & Kira, 2007;Tseng & Tsai, 2010;Yoo, Han, & Huang, 2012).
Engagement is a cognitive process in learning tasks and an indicator of active participation, emotion, and other aspects in learning (Pellas, 2014). In online learning, online time spent and other behavior records represent the behavior engagement of the student, such as the frequency in which they log in and out of their online platforms, and the frequency in which they open and close reading materials, download and upload exercises, and post in internal discussion forums. "Online time" appears to have a positive impact on online learning achievement (Hu, Lo, & Shih, 2014;Ma, Han, Yang, & Cheng, 2015;Macfadyen & Dawson, 2010). Skinner, Furrer, Marchand, and Kindermann (2008) categorized learning engagement into behavioral engagement and emotional engagement. Behavioral engagement is defined as a continuous behavior that includes the effort spent on learning, attention, and continuity of behavior from the beginning to the execution of a learning task. Emotional engagement includes feelings such as interest, enjoyment, and frustration from defeat. Related studies have pointed out that learning motivation and user engagement levels have an impact on the actual learning behavior (Kong, Kwok, & Fang, 2012;Wang & Lin, 2007). As seen from the previous study results, both learning motivation and user engagement have a significant impact on the behavior process in online learning. However, existing research related to online learning motivation has primarily focused on behavioral aspects, such as outcomes and performance determined by one's motivation "after learning," and on the psychological aspect, such as emotions and one's level of satisfaction. Hou (2012b) suggested that gamers' behavioral patterns in educational games include collaborative learning and problem-solving. Existing studies have applied behavioral sequence analysis to investigate users' behavioral patterns during social interactions in an online learning context (Hou, 2012a;Hou & Wu, 2011;Sun, Kuo, Hou, & Lin, 2017), and behavioral patterns of knowledge construction from a cognitive perspective (Hou & Wu, 2011;Yang, Li, Guo, & Li, 2015). Furthermore, behavioral sequence analysis had been utilized in an online two-tier test strategy to diagnose students' misconceptions and to examine the pattern of students' test-learning behaviors (Yang, Chen, & Hwang, 2015). Lai and Hwang (2015) discovered that, compared to low-achieving students, high-achieving students tended to exhibit patterns of behaviors in which they repeatedly searched for additional learning materials, often attempted an input value, and observed the corresponding outcome. Chen and Jang (2010) discovered that there were no significant correlations between learning motivation and weekly login frequency, and the number of clicks on the online platform. However, changes in behavior during the learning process, as well as the relationship between learning motivation and behavioral patterns in an online environment still await further exploration, with extra attention needed to assess the differences among high and low motivated students. Therefore, the present study refers to the recommendation of Chen and Jang (2010) to combine learning motivation and user engagement to investigate the behavior pattern in online learning. This study aims to understand and compare the reading behavior patterns that students with different levels of motivation exhibit in order to provide recommendations to instructors and researchers for future use or design of online learning courses.
The sequential analysis in Bakeman (1997) is often used in peer discussions of the learning process, which includes factors such as behavior, the knowledge construction process, and problem solving skills 212 (Eryilmaz, Chiu, Thoms, Mary, & Kim, 2014;Hou, 2012aHou, , 2012bHou & Wu, 2011;Liu, Cheng, & Huang, 2011). However, aside from teacher-student and peer interactions, online learning also includes interactions between students and e-materials (Moore, 1989). Gil-Flores, Torres-Gordillo, and Perera-Rodríguez (2012) argue that the digital culture resulting from ubiquitous information communication technology has a significant impact on the learning environment, and digital reading has become a crucial skill that a student must possess in the contemporary learning environment. Liu (2005) revealed that the digital reading environment is a topic deserving of an in-depth study as the digital environment continues to evolve. In recent years, researchers have started applying sequential analysis to investigate the interactive process happening on the discussion platform among students (Hou, 2012a;Hou & Wu, 2011). However, in terms of online reading behavioral patterns (i.e., the way students read online course materials), further studies are still needed. Therefore, this study used a sequential analysis to investigate and compare the differences in behavior patterns of students in reading online learning materials and in assessing their learning motivation. The research questions posed to the participants were as follows: 1. What are the frequencies of different types of online reading behavior?
2. What are the differences in the reading behavioral pattern of online students with different type?

Research Procedures and Participants
This study adopted the cross-sectional survey research method. Cross-sectional survey research refers to the investigation of data collected at a specific point in time or a short period of time (Ann, 2006). This study adopted a convenient sampling method. The participants were graduate students from three national universities in Taiwan who were invited to participate in this research study via campus bulletins, email, and community website marketing. The study design is shown in Figure 1. Participants read the digital learning materials from each of the four units and then took a test. If their test scores were lower than 80, the participants could re-read the learning materials and try to pass the same test again. After passing the test, the participants were prompted to complete a post-test questionnaire that measured their levels of learning motivation. The research ethics course designed for the study aimed to ensure a mastery of basic knowledge. Therefore, we defined the passing threshold to be 80% for all unit tests and permitted learners to take the tests repeatedly. The participants were required to finish the entire course online. The system recorded participants' online learning behaviors during the learning and testing processes. The open period for the online course information collection was 2 months. After the researcher informed the participants that the online course would be open for 2 months, the participants were free to control their own study pace without interference from the researcher. Out of the 249 students enrolled, 170 completed the course.
After the invalid data was eliminated in the questionnaire review session, a total of 160 students had valid information; among them, 64 were female and 96 were male. The average age was 23.46 and the standard deviation for age was 2.46 years. The distribution of participants' schools and grade levels is presented in Figure 2. There were more participants from the schools of engineering and computer science (48%) than from other departments, and the majority of the participants were first-year master's students (57%).

Instruments
This study used the digital learning motivation scale proposed by Yoo et al. (2012) that measured employees' acceptance levels in digital learning and further categorized them into six sub-factors. There were 20 items in the original scale after translation into Chinese. Among the six sub-factors, "social influence" (three items) did not fit the online learning environment, as its online course did not include any interactive function among the teacher and students or among peers, and that items such as "(i)n general, the organization has supported the use of e-learning" did not fit the context of this study, in which our participants were asked to carry out the study autonomously, and was therefore eliminated. The scale used in this study was a 5-point Likert scale with 1 representing strongly disagree -no confidence and 5 representing strongly agree -full confidence. A confirmatory factor analysis revealed that the factor 214 loadings of three items were less than .40. Therefore, these three items were removed from the scale.  (Nunnally & Bernstein, 1994). The factor loadings of effort expectancy (three items) ranged from .59 to .81 (e.g., "So far, I think the e-learning platform is easy to use"). The factor loadings of attitude towards e-learning (four items) ranged from .55 to .80 (e.g., "using e-learning is fun").
Anxiety contained three items with reversed wording and factor loadings ranged from .77 to .83 (e.g., "I hesitate to use e-learning because of making a mistake"). The factor loadings of the two items of performance expectancy were .41 and .93 (e.g., "I think the e-learning tool is useful"). Facilitating conditions contained two items, with factor loadings of .71 and .77 (e.g., "I have sufficient knowledge to use the e-learning tool").

Online Learning Platform and Materials
Austin, Gorsuch, Lawson, and Newberry (2011)  with four options. There were buttons labeled "next page" and "previous page" in the system that allowed students to sequentially or repeatedly run the pages. The system interface also provided a unit navigation menu, allowing students to switch freely between the pages. Black et al. (2008) pointed out that combining questionnaires and actual log data can reveal the relationship between the mental states of learners and their actual behavior. Therefore, this study adopted learning motivation and reading duration as the base of the cluster analysis. Data collected in the study included the rating of the digital learning motivation scale, the total online reading time, and the behavioral coding logs. A cluster analysis and a behavioral sequence analysis were applied to the data analysis. Predictive

Data Analysis Method
Analytics Software (PASW) 22 was used to perform the cluster analysis. This study used learning motivation and e-textbook reading duration as the two variables for cluster analysis. In the first stage, Ward's method was used to determine the number of clusters as the preliminary optimal. In the second stage, K-mean was used to establish the grouping results. No significant correlation was found between learning motivation and e-textbook reading duration in the pre-test performed in this study (r = .04, p = .62), and no collinearity of variables was observed, indicating that the data was fit for grouping. Ward's method was adopted to divide students into two to five groups sequentially. The variance analysis of the clustering validity is listed in Table 1. There was a substantial increase of the coefficients of determination (R 2 ) in the transition from three to four groups, while the increase was not apparent in the transition from four to five groups. Although four and five groups had higher coefficients of determination, the number of participants in each group was 216 small (<30), yet there was a large discrepancy in the number of participants in each group. Therefore, we decided to use three groups for the K-means cluster analysis. After K-means clustering, the groups were named as follows: low reading duration with low motivation (LRLM group), low reading duration with high motivation (LRHM group), and high reading duration (HR group), representing three different types of student learners. ANOVA revealed that the scores of all three groups showed significant variances in terms of pretest learning motivation (F = 78.40, p < .01) and online e-textbook reading duration (F = 234.23, p < .01). A post hoc comparison showed that the learning motivation of the LRHM group was significantly greater than that of the LRLM group (p < .01) and the HR group (p < .01). Furthermore, the post hoc comparison showed that the learning motivation of the HR group was substantially greater than that of the LRLM group (p < .01). In addition, the reading duration of the HR group was significantly greater than that of the LRLM group (p < .01) and the LRHM group (p < .01), and no apparent difference was found in reading duration between the LRLM group and the LRHM group (p = .54). This confirms that the characteristics of the three groups were indeed different, and therefore proves validity of this grouping. After the students were classified into different groups, a sequential analysis was then used to examine the sequential behavioral pattern in each type of student. In this study, the online learning behavior was encoded based on the guidelines proposed by Bakeman (1997). A pilot test along with the observation and interview of two graduate students was conducted. The results indicated that the time the participants took to finish the course was shorter than expected. During the interview, the participants also indicated that the learning pace was not fast and the material covered was not difficult. Therefore, this study established 120 seconds as the average amount of time needed to flip through a page of intensive reading. If a page was turned before the learning material was finished playing, it would be classified as incomplete reading of the learning materials. To sum up the results from this pilot test, a definition of behavior codes used in this study is listed in Table 2. A self-developed program system was used in this study to automatically transform the behavior records into codes to avoid human error and unnecessary omissions. There were a total of 13,035 converted codes. We then used the Multiple Episode Protocol Analysis (MEPA) software to conduct a sequential analysis.
218 Table 2 Online Reading Behavioral Coding Scheme

Code Name Description Definition
A

Intensive reading
Finished flipping through learning materials and pressed "next page" button.
Students with time spent jumping to a page is more than 15 seconds but less than 2 minutes.

Multitasking reading
Finished flipping through learning materials and pressed "next page" button.
Students with time spent jumping to a page is more than 2 minutes but less than 20 minutes.

C
Skim reading Pressed "next page" button but did not finish flipping through learning materials.
Students with time spent jumping to a page is less than 15 seconds.

Passing the unit test
Took unit test and passed.
Students "submitted the test" and scored higher than 80.

Not completing the unit test
Took, but did not complete the unit test.
Students "entered test" record but did not have record of "submitted the test." F Offline Temporarily logged out from the learning platform.
Students with 20 minutes of gap time between two records and showed no record of other activity in between.

Results
To address research question (1), the sequential codes of each group were statistically organized based on the results of the cluster analysis conducted in this study. Related data are as shown in Table 3 and Figure   4. *Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group." *Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group." To address research question (2), the data was converted into an adjusted residuals table of sequential codes, as shown in Table 4, with the horizontal axis representing "start of behavior" and the vertical axis representing "end of behavior." For the z value shown in the adjusted residuals table, if the value was greater than 1.96, the change in sequential behavior of that group reached the significance level of p < .05 220 (Bakeman, 1997;Hou, 2012b); that is, there is a significant relationship between the order of sequence. The sequential behavioral pattern among the three groups in online reading is shown in Figure 5. *Note. LRLM group means "low reading duration with low motivation group"; LRHM group means "low reading duration with high motivation group"; HR group means "high reading duration group." 221 Figure 5. Diagram of sequential behavioral patterns in online reading of all three groups.

Online Students Exhibit Different Reading Behavior Frequencies
According to the behavior frequency pie chart (Figure 4), the allocation of behavior outcomes in the two groups of low reading duration is quite similar, but is considerably different from the high reading duration group. In the high reading duration group, 68.3% of students fell into the "intensive reading" category of the overall learning behavior, whereas the intensive reading behavior in the low reading duration with low motivation and low reading duration with high motivation groups was only 34% and 39.9%, respectively.
"Skim reading" behavior was only exhibited by 18.5% of the students in the high reading duration group, while the percentages of the other two groups were quite high -36.5% and 34%, respectively. Among the students in the group of high reading duration group, only for 2.4% exhibited behavior of "not completing the unit test," but the percentages of students in the other two groups who exhibited this same behavior was 14.5% and 13.8%. In comparing the differences in behavioral proportions due to different learning motivation factors, the results showed that reading duration could significantly affect learning behavior.
Students with high reading duration tended to be more seriously engaged in learning activities, and these students not only tended to exhibit page retention time spent on the e-textbook learning materials in accordance with expectations, but also tended to exhibit a relatively lower rate of occurrence in the behavior of "not completing the unit test." In addition, for both the group of low reading duration with low motivation and the group of low reading duration with high motivation, the next behavior after "passing the unit test" was "not completing the unit test" (D→E). However, such a sequence was not observed in the high reading duration group. Thus, we could see that after passing a unit test, the two low reading duration groups had the tendency to quickly transition into the test challenge in the following unit, rather than spending time reading the learning 222 materials in the following unit. Such results indicate that these two low reading duration groups seem to have a speculative mentality: they hoped to finish the course quickly and therefore skipped reading the learning materials and took the test right away. This result is consistent with the study of Hu et al. (2014) who pointed out that online learning time can significantly predict the learning performance.

The Online Students of Different Types Have Different Reading Behavior Patterns
Although the behavior of digital media multitasking is believed to lower task performance (Gardner, 2008), we found that the group of low reading duration with high motivation exhibited a different behavior sequence than the other two groups. The group of low reading duration with high motivation showed that the behavior of "intensive reading" tended to follow the behavior of "intensive reading" and "multi-tasking reading" (A→A、B→A), and "multi-tasking reading" behavior also tended to follow "intensive reading" behavior (A→B). The findings suggest that when multi-tasking under the "intensive reading" environment, students may have been delayed in flipping through the learning materials and were therefore encoded as performing "multi-tasking reading." However, when encoded as performing "multi-tasking reading," this meant that a student's attention had not completely detracted from the learning materials, and that he or she had returned to the page with "intensive reading" behavior. Other data in this study showed that the group of low reading duration with high motivation exhibited 944 instances of "intensive reading," whereas the low reading duration with low motivation group only showed 715 instances of "intensive reading." Therefore, for students who prefer the multi-tasking online learning environment, highly motivated students can easily switch back from multi-tasking to the learning materials and refocus their learning.
Many studies in the past have widely viewed learning motivation as an important factor in the actual learning behavior that is exhibited (Kong et al., 2012;Schunk et al., 2013). However, we found that highly motivated students may prefer to exhibit a motivated online multi-tasking reading behavior pattern; that is, in which students' learning behavior is characterized in terms of focused multi-tasking.

Conclusion, Limitations, and Future Research
This study aims to explore the effects of learning motivation on online reading behavioral patterns. In response to the research question (1), the study found that the high reading behavior duration group presented a higher ratio of intensive reading and a lower ratio of non-completion of the unit test. However, the other two groups (low reading duration with low motivation and low reading duration with high motivation groups) manifested a lower proportion of intensive reading behavior, higher proportion of skim reading behavior, and a higher proportion of non-completion of the unit. In response to research question (2), we discovered that, after taking a unit test, the low reading duration with low motivation and low reading duration with high motivation groups had a behavioral tendency to bypass the reading materials and instead participate directly in the next unit test. In addition, the behavioral patterns of the low reading duration with high motivation group showed the characteristic of interchange between intensive reading and multi-tasked reading. In summary, this study indicates that "online reading duration" in the online learning system is a strong indicator for reading motivation in students. In addition, this study proposes that multi-tasking learning should be considered in the future design of online learning courses to provide appropriate teaching strategies or platform functions. Learning motivation is an important factor in enhancing "intensive reading" in students. With the aid of online course materials, instructors can further 223 enhance students' learning motivation. We propose that a record function should be added for "online reading duration" in the online course platform in future online courses. Coupled with an interface with graphical presentation, instructors or system managers can better understand the learning situation so that flexible adjustments to the online learning materials can be made accordingly, and so that the effectiveness of the learning platform and student performance can be evaluated more accurately.
In terms of research limitations, we used the time spent on reading the online materials to define the student's behavior as "intensive reading," "multi-tasking reading," or "skim reading." However, these standards are not definitively certain in terms of effectiveness. As the length of each animation page in the learning materials varies, applying the same amount of time as the encoding standard may not fully explain the actual usage behavior in the students. Therefore, the analysis in this study may be limited. In addition, this study adopted a convenient sampling method to recruit graduate students from three national universities in Taiwan; therefore, readers should be cautious when applying our findings to the entire population. This study adopted reading duration as an indicator of behavioral engagement. Future research is suggested to further explore other influential factors of behavioral patterns in online learning, such as emotional and cognitive engagement, perceptual habits in reading, learning attitudes, gender, and academic background. Given that the research ethics course adopted in this study aimed to establish learners' basic knowledge, criterion-referenced tests that allowed repeated participation were utilized; therefore, the results of the test were not included in the discussion. Future studies are advised to adopt a topic with a more comprehensive question pool so as to explore the impact of online learning behavior patterns on learning performance. Finally, with the large amount of data saturation that exists today, using the back-end quantitative data on the server to undertake a massive analysis is a worthwhile direction to take in the developmental stages. We suggest that in subsequent research the correlation between the backend data in the server and the actual usage behavior in the front end should be explored through observation and interviews. Furthermore, we suggest that data triangulation should be applied to such an investigation to examine the actual reading situations and page retention time to serve as the foundation for massive data analysis and to further understand reading behavior patterns in online courses.