International Review of Research in Open and Distributed Learning An Internet Articles Retrieval Agent Combined With Dynamic Associative Concept Maps to Implement Online Learning in an Artificial Intelligence Course

Online learning has been widely discussed in education research, and open educational resources have become an increasingly popular way to help learners acquire knowledge. However, these resources contain massive amounts of information, making it difficult for learners to identify Web articles that refer to computer science knowledge. This study developed an Internet articles retrieval agent combined with dynamic associative concept maps (DACMs). The system used text mining technology to analyze keywords to filter computer science articles. In previous research, concept maps were manually constructed; in this study, such maps can be automatically and dynamically generated in real time. In a case study of a fundamental course of artificial intelligence, this study designed two experiments to compare students’ learning behaviors while using this system and the Google search engine. The results of the first experiment showed that the experimental group searched for more knowledge articles on computer science using this agent, compared to the control group using the Google search engine. The learning performance of the experimental group was significantly better than that of the control group, while the cognitive load of the experimental group was significantly lower than that of the control group. Furthermore, the results of the second experiment showed that the learning progress of students using the agent was significantly greater than that of students who used the Google search engine. This illustrates that the agent effectively filtered computer science articles, and DACMs helped students gain a deeper understanding of academic concepts and knowledge related to artificial intelligence.


Introduction
In recent years, online learning has been widely discussed in education research (Abdullah & Mirza, 2020).Some studies indicated that online learning can effectively improve students' learning performance in the classroom (Dashtestani, 2020) and learning motivation (Chaiprasurt & Esichaikul, 2013;Li & Tsai, 2017).With the rapid development of information technology, more and more teachers provide online learning platforms and tools in the classroom so that students can conveniently use the Internet to implement online learning (Lowenthal et al., 2015;Tsai et al., 2018).Also, students can effectively obtain massive open resources and learning materials through online learning (Isaac et al., 2019).Compared with traditional lectures, books, and course resources, students can obtain more information and resources through online learning (Alshahrani et al., 2017).However, online open resources usually contain massive amounts of information, which may make it difficult for students to identify the connection between these resources and learning contents.Meanwhile, with the rapid development of artificial intelligence, students need to learn the latest knowledge and concepts very quickly.
Recent research has indicated that concept maps can effectively help online students understand new concepts while learning new knowledge (Farrokhnia et al., 2019).Concept maps have also been regarded as a useful tool for structural knowledge representation (Hwang et al., 2013;Hwang et al., 2011).In recent years, concept maps have been applied to different courses to improve students' learning performance.For example, Hwang, Zou, and Lin (2020) used a question-posing approach based on concept mapping to explore ubiquitous learning about plants in elementary natural science courses.Chiou et al. (2017) discussed various concept mapping techniques for senior accounting students.Chiou et al. (2015) used multimedia animation combined with multidimensional concept maps to discuss multimedia animation courses in universities; the results of these three studies showed that concept maps effectively promoted students' learning performance in the classroom.However, in previous studies, concept maps were usually constructed manually (Chularut & DeBacker, 2004;Marzano & Miranda, 2020;Sun & Chen, 2016); maps were not generated automatically and dynamically in real time, even as sources and amounts of information increased.
Current research has indicated that online learning combined with concept maps can effectively improve students' learning performance.For example, Fatawi et al. (2020) indicated that learning online can improve student learning outcomes and engagement through concept maps.Hwang, Chang, et al. (2020) indicated that a problem-posing strategy guided by concept maps and adopted in an online learning environment improved the learning performance of students.Compared with students with lower critical thinking, students with a higher level of critical thinking have a more obvious improvement in their learning performance.According to the survey, this study found no existing online learning platform that automatically filtered Web articles related to computer science and dynamically generated concept maps related to computer science in real time.
Our study sought to address the limitations of previous research; we developed an Internet articles retrieval agent combined with dynamic associative concept maps (DACMs).Students connected to this system for online learning through the Internet.This system used text mining technology to automatically filter computer science articles, and the Apriori algorithm automatically and dynamically generated associative concept maps in real time.In the context of a fundamental course in artificial intelligence, this study aimed to address the gaps in existing research, conquer the technical limitations of specific learning tools, reduce students' cognitive load, and improve their learning performance.

Literature Review Online Learning
Many studies have indicated that online learning is a way to gain a learning experience through the use of certain technologies (Benson, 2002;Carliner, 2004;Conrad, 2002).Hwang, Wang, and Lai (2020) indicated that online learning can be independent of time and place, allowing students to watch and work with learning materials or multimedia videos ubiquitously.These online resources include text, pictures, videos, databases, and so on (Elbaum et al., 2002).Although many studies have confirmed the convenience of online learning, Doo et al. (2020) indicated that it requires effective learning methods to improve the quality of learning in the online environment.Therefore, this study proposed the use of DACMs with online learning activities as students searched for and read computer science articles autonomously.

Concept Maps
A concept map is a graphical tool commonly used to organize or express knowledge (Novak & Cañas, 2006).Students connected and integrated the concepts of new knowledge with their own knowledge through the use of concept maps (Chiou et al., 2015).Previous research has indicated that concept maps can effectively promote meaningful learning (Farrokhnia et al., 2019).In recent years, many researchers have confirmed the effectiveness of concept maps in education.For example, use of concept maps have improved students' learning performance (Hwang, Zou, & Lin, 2020), learning motivation (Hsu, 2019), problem solving (Whitelock-Wainwright et al., 2020), and critical thinking (Yue et al., 2017).In addition, Sun and Chen (2016) proposed dynamic concept maps with IRS to investigate the learning effects for students in anti-phishing education, and this map was gradually shown by the teachers' instructional explanations.This study showed that the use of dynamic concept maps with IRS by students can effectively improve their learning outcomes.Marzano and Miranda (2020) proposed a dynamap remediation approach that allowed users to specify nodes, relationships, and related content as they created new dynamic concept maps.However, these maps were manually constructed by the instructor in advance.For now, new concept maps cannot be automatically generated as open resources and information grow rapidly.
This study established an Internet articles retrieval agent combined with DACMs to realize the dynamic generation of associative concept maps in real time.Because this system was built on a server, DACMs were generated automatically and dynamically in real time whenever information was updated.These DACMs effectively conquered the limitation that meant instructors needed to manually construct concept maps in advance.This study designed two experiments to verify the effectiveness of the system in implementing online learning activities in a fundamental course on artificial intelligence, and was guided by the following research questions: 1.In Experiment 1, is the learning performance of the experimental group using the Internet articles retrieval agent combined with DACMs higher than that of the control group using the Google search engine? 2. In Experiment 1, is the cognitive load of the experimental group using the Internet articles retrieval agent combined with DACMs lower than that of the control group using the Google search engine?66 3.In Experiment 2, is the learning progress of the same group using the Internet articles retrieval agent combined with DACMs higher than those who use the Google search engine?

Method
In this study, an Internet articles retrieval agent combined with DACMs was used for online learning activities in a basic course on artificial intelligence in order to explore students' learning performance, cognitive load, and learning progress.Two experiments were designed to compare the learning behaviors of students using this system with those using the Google search engine.

Participants
A total of 75 college students in the department of computer science and information engineering at a university in Taiwan participated in this experiment.The ages of the participants ranged from 20 to 21 years.None of the students had previous experience with an online learning activity involving the use of an Internet articles retrieval agent combined with DACMs.

Research Procedure and Artificial Intelligence Learning Activity
Figure 1 shows the research procedure for Experiment 1, based on a quasi-experimental design method.
Of the 75 students who participated, 38 students were assigned to the experimental group, and 37 students were assigned to the control group.The experimental group used the Internet articles retrieval agent combined with DACMs in specific online learning activities, and the control group used the Google search engine.The same teacher instructed the online learning activities for both groups.The experimental time for each group was the duration of one class period (50 minutes).
Before the experiment began, the teacher introduced the experimental procedure to the students.When the experiment started, all students in both groups were given a pre-test to measure their knowledge of concepts related to a basic artificial intelligence course.Next, the teacher introduced the learning tools to be used for implementing online learning activities, namely the Internet articles retrieval agent combined with DACMs or the Google search engine.Each group then used the specifically designated learning tools to implement their learning tasks.Finally, all students in both groups were given a posttest and cognitive load questionnaire to explore whether there were significant between-group differences in terms of learning performance and cognitive load after using the designated learning tools.

Figure 1
Research Procedure for Experiment 1 Figure 2 shows the learning conditions for the experimental group who used the Internet articles retrieval agent combined with DACMs.In this learning task, the teacher assigned a keyword related to artificial intelligence.The experimental group searched for articles with this keyword using the proposed system, recorded the titles and the corresponding URLs, and read the articles they found.In addition to searching for computer science articles through this system, the experimental group visualized the relevance between each keyword and related keywords through DACMs.

Learning Conditions for the Experimental Group
The learning conditions for the control group who used the Google search engine to complete the same learning task performed by the experimental group.This ensured consistency between the two groups in terms of learning difficulty.The keywords that the control group used in their searches were the same as the experimental group.
Experiment 1 was designed to explore the learning performance and cognitive load of the two groups in terms of online learning activities.The teacher assigned the keywords that students had not yet learned.
In order to ensure that the analytical results of the post-test were correct and reliable, the pre-test answers were not given to the students.The students autonomously searched for articles associated with the keywords and read the computer science articles they found.

Participants
A total of 28 students in the department of computer science and information engineering at a five-year junior college at a university in Taiwan volunteered to participate in Experiment 2. Participants, who were selected based on availability and willingness to participate, ranged in age from 16 to 17 years.All students used the Internet articles retrieval agent combined with DACMs and the Google search engine in two stages to implement the online learning activities.Before the start of the online learning activity, none of the students had experience with the Internet articles retrieval agent combined with DACMs.

Research Procedure and Artificial Intelligence Learning Activity
Figure 3 shows the research procedure for Experiment 2 which was implemented in two stages for the duration of one class period.First, the teacher explained the experimental procedure and then introduced the two learning tools to the students.Next, all students took the first test (i.e., pre-test) to determine their level of prior knowledge before using any learning tools.In the first stage of the experiment, the students used the Google search engine for the assigned learning tasks.After completing the learning task, the students took the second test to assess their learning performance after using the Google search engine.In the second stage of the experiment, the students used the Internet articles retrieval agent combined with DACMs to complete the assigned learning tasks.After completing the learning task, the students took the third test to determine their learning performance after they used the proposed system.All the students used two learning tools, one in each of the two phases; three tests conducted in two stages explored whether there were significant differences in learning progress depending on the tools used.In Experiment 2, this study determined whether there were significant differences in learning progress when the students used two different learning tools to carry out online learning activities.When the students used the Google search engine, their learning progress was based on the difference in scores between the first test and the second.When the students used the Internet articles retrieval agent combined with DACMs, their corresponding learning progress was based on the difference in the scores between the second test and the third.In order to ensure that the analytical results were correct and reliable, the test answers were not provided to the students.The students autonomously searched for articles associated with the keywords and read the computer science articles they found.

System Architecture and Functionality
This study used text mining technology to develop an Internet articles retrieval agent combined with DACMs to improve students' learning performance in a fundamental course on artificial intelligence.
The system was built on the server, so students learned online through the Internet at any time.The system collected a total of 35,907 computer science articles that contained 1,751 keywords.Text mining technology automatically filtered the articles and provided main keywords from the articles, and the Apriori algorithm automatically and dynamically generated associative concept maps in real time.The unique advantage of this system was that it allowed students to quickly search through computer science articles for the latest information on artificial intelligence.The use of DACMs strengthened the keywords and concepts related to AI. Figure 4 illustrates the system architecture for this study.

Figure 4
System Architecture Diagram This system used the Web crawler module to retrieve an enormous number of online articles.The Ngram segmentation module pre-processed the Web articles and automatically filtered to select those that dealt with computer science.Then, the term frequency-inverse document frequency module automatically calculated the text weighting of each computer science article.Finally, each computer science article with main keywords were stored in the database on the server.In addition, the DACMs module used the Apriori algorithm to automatically determine the strength of association between keywords, so as to dynamically generate the associative concept map in real time.

Dynamic Associative Concept Maps (DACMs)
In previous studies, concept maps were not generated automatically; teachers constructed concept maps manually for specific courses or topics.Artificial intelligence is developing at a rapid rate, and the source of knowledge is no longer limited to textbooks.In order to enable students to obtain the latest information and concepts any time and any place, this study used DACMs based on the Apriori algorithm and text mining technology, so that concept maps were generated dynamically in real time.
Through this function, the proposed system solved the limitations of previous research in which concept maps were constructed manually.In addition, students also accessed and worked with dynamic associative concept maps with the latest AI information in real time.Compared to common concept maps, DACMs automatically and dynamically generated the strength of the correlations between keywords in real time.In addition to quickly understanding the relevance of various keywords through the DACMs, students also learned highly-related keywords to further enhance their learning performance.Figure 5 is an example of a DACM used in the proposed system.Students entered keywords, and the system generated dynamic associative concept maps in real time.The DACMs calculated the relevance between each keyword, and automatically connected and displayed the strength of relevance between keywords in the arrows and the association rule.In addition, students used the DACMs to learn new keywords, which promoted a deeper understanding of the keywords and enhanced their learning of a new concept.

Internet Articles Retrieval Agent
The Internet articles retrieval agent used text mining technology to automatically filter computer science articles so that students can avoid reading articles unrelated to computer science.Therefore, this agent provided students with an efficient way to search for computer science articles any time and any place, and also provided keywords for each computer science article.Figure 6 shows the operation interface for the Internet articles retrieval agent used to search for computer science articles.The system provided the title and link to a computer science article as well as main keywords for the article.Before they read each article, the students reviewed the main keywords and determined whether the article was suitable.Therefore, the computer science articles and main keywords provided by this system helped students gain a deeper understanding of new information related to artificial intelligence.

Figure 6
Operation Interface for the Internet Articles Retrieval Agent

Data Collection
Quantitative data were collected from the two experiments.In Experiment 1, data were collected from the pre-test, post-test, and cognitive load questionnaires filled out by 75 students, and in Experiment 2, from the first, second, and third test results collected from 28 students.The following provides details on the data collected.
• Test content: 10 keywords related to computer science were selected.The students indicated which keywords were highly relevant to artificial intelligence.Each question was worth 10 points, with a full score of 100 points.
• Cognitive load questionnaire: This study was modified from the measure developed by Hwang et al. (2013) and measured the cognitive load for the students in Experiment 1.This questionnaire comprised two dimensions with a total of eight items.Five items measured mental load and three items measured mental effort.All responses used a five-point Likert scale.

Data Analysis
SPSS statistical analysis software was used to analyze data from the two experiments.
• Experiment 1: Analysis of covariance (ANCOVA) and independent sample t test were used to analyze the learning performance and cognitive load of 75 students.In addition, Cronbach's alpha was used to check the internal consistency of the cognitive load questionnaire.The Cronbach's alpha for mental load was 0.82, and 0.85 for mental effort.These results indicated that the cognitive load questionnaire had good reliability.
• Experiment 2: Paired sample t tests were used to analyze learning progress for 28 students.

Learning Performance
Before adopting ANCOVA to analyze students' learning performance in Experiment 1, this study conducted a homogeneity regression to ensure there was no interaction between the independent variables and covariates.For the independent variables, the experimental group used the Internet articles retrieval agent combined with DACMs to implement online learning activities, and the control group used the Google search engine.The covariates included the pre-test taken by the two groups of students.These results showed that there were no significant differences in the interaction between the independent variables and the covariates (F = 2.81, p > .05).
Table 1 shows ANCOVA results for the post-test.The adjusted mean and standard error of the post-test for the experimental group were 53.09 and 2.48, respectively, and the adjusted mean and standard error of the post-test of the control group were 41.42 and 2.52, respectively.The results of ANCOVA showed that by excluding the influence of the pre-test, the post-test of the two groups reached a significant difference (F = 10.9, p < .01).This means that the students in the experimental group using the Internet articles retrieval agent combined with DACMs exhibited significantly better learning performance as compared to the control group using the Google search engine.In addition, we counted the number of computer science articles searched for by the two groups during the learning tasks.On average, each student in the experimental group found 9.24 computer science articles, whereas each student in the control group found 3.41 computer science articles.Clearly, the main keywords in each computer science article provided by the proposed system effectively helped students to determine the key points of the article from the start.This not only helped the students search for and read more computer science articles but also sped up their judgments as to whether the article was related to the search keywords.In addition, the experimental group significantly improved in terms of learning performance after using the proposed system.This suggests that DACMs promoted student understanding of the strength of the associations between keywords, and in turn, effectively enhanced their acquisition of professional information and concepts related to artificial intelligence.

Cognitive Load
An independent sample t test was used to analyze the cognitive load of the students during Experiment 1; Table 2 shows the results.The mean and standard deviations of cognitive load for the experimental group were 1.63 and 0.58, respectively, and the mean and standard deviation of cognitive load for the control group were 2.01 and 0.6, respectively.The independent sample t test results showed that the cognitive load in the two groups reached a significant difference (t = -2.79,p < .01).This means that the students in the experimental group had a significantly lower cognitive load than did the students in the control group using the Google search engine.In other words, when the students in the experimental group used the proposed system to acquire the professional information and concepts related to artificial intelligence, they did not experience an increase in cognitive load that influenced their learning process.This study also explored two dimensions of cognitive load.Table 3 shows the independent sample t test results for mental load and mental effort.The means of mental load and mental effort for the experimental group were 1.68 and 1.54, respectively, and the mean of mental load and mental effort for the control group were 2.09 and 1.87, respectively.The independent sample t test results showed significant between-group differences between mental load (t = -2.9,p < .01)and mental effort (t = -2.06,p < .05).This indicated that when the experimental group used the proposed system to complete the learning task, their mental load and mental effort were significantly lower than was the case for the control group using the Google search engine to complete the learning task.In other words, the Internet articles retrieval agent combined with DACMs effectively assisted the students in the experimental group to search for correct computer science articles related to the learning task, so they successfully completed it.In summary, there were significant between-group differences in both the analysis of cognitive load and in the analysis of mental load and mental effort.The Internet articles retrieval agent combined with average of 2.57 computer science articles.In Experiment 2, the number of computer science articles found by students using the proposed system and Google search engine were similar.

Discussion and Conclusions
In this study, an Internet articles retrieval agent combined with DACMs was used to improve students' learning performance in a fundamental artificial intelligence course while reducing their cognitive load when learning new information and concepts.In addition, the DACMs proposed in this study were based on the Apriori algorithm and text mining technology, which automatically and dynamically generated a concept map in real time and mitigated the limitations related to manual construction of concept maps found in previous studies.DACMs not only helped students understand the strength of relevance between keywords or concepts but also automatically and dynamically generated related concept maps with the latest information in real time; students used this system to learn online at any time.Two experiments were conducted to compare the learning behavior of students using the proposed system with the Google search engine.

Experiment 1
The results for Experiment 1 showed significant improvements in learning performance in the experimental group as compared to the control group.The results of this study confirmed that the Internet articles retrieval agent combined with DACMs significantly improved students' learning performance.It is thus inferred that DACMs effectively provided students with knowledge about the strength of the correlation between various keywords related to artificial intelligence and so helped students understand the latest information or concepts.At the same time, the students were able to use DACMs to recognize keywords that had not been learned previously, as well as to use the Internet articles retrieval agent to search for computer science articles.Compared to the control group using the Google search engine, the experimental group was able to search for more computer science articles; the system promoted students' learning of information and concepts related to artificial intelligence while they consolidated the latest information.
In addition, the cognitive load results showed that the experimental group's cognitive load was significantly lower than that of the control group.At the same time, the results showed that the mental load and mental effort exerted by the experimental group were significantly lower than that of the control group.The results of this study confirmed that the use of the proposed system to acquire professional knowledge and understand concepts related to artificial intelligence did not cause excessive cognitive load during the learning process.It is thus inferred that the system effectively enabled the students to search for computer science articles related to the learning tasks, and the DACMs made it possible for them to visualize the strength of the correlations between keywords.The students were able to easily understand and learn artificial intelligence keywords through this system, and they were also able to search for and read artificial intelligence computer science articles and successfully complete the learning tasks.

Experiment 2
The results of Experiment 2 indicated that the students using the Internet articles retrieval agent combined with DACMs had significantly greater improvements in learning progress compared to the control group using the Google search engine.The results of this study confirmed that the use of the Internet articles retrieval agent combined with DACMs enhanced the students' knowledge acquisition and understanding of relevant concepts to a greater degree than did the use of the Google search engine.
In addition, the teacher assigned different keywords in the two stages to compare the learning progress of the students using different learning tools.Although the number of computer science articles found by students using the proposed system was not very different from that found by the students using the Google search engine, it is worth noting that the quality of computer science articles the students searched for might affect the differences in learning effectiveness.The students using the proposed system to search for and read computer science articles exhibited significantly greater learning progress as compared to the students using the Google search engine.Therefore, Experiment 2 showed that, compared to the Google search engine, the proposed system provided artificial intelligence-related computer science articles more effectively, and made it possible for students to gain deeper insights into the latest information and concepts in the same amount of experimental time.
In summary, the results of this study can be used by educators to verify the importance of students' acquiring novel information and concepts in open educational resources.The system proposed in this study is open and free.Students can use this system to achieve online learning and distance education through the Internet.Furthermore, this study suggests that an Internet articles retrieval agent combined with DACMs can serve as a superior auxiliary learning tool for students using open educational resources, and students can acquire novel information and concepts using this system.

Table 1
ANCOVA Results for Learning Performance in Experiment 1

Table 2
Independent Sample t Test Results for Cognitive Load in Experiment 1

Table 3
Independent Sample t Test Results for Mental Load and Mental Effort