Challenges of Transitioning to e-learning System with Learning Objects Capabilities

In order for higher education institutions, which implements blended and/or online learning to remain competitive and innovative it needs to keep up with the cutting edge technological and educational advances. This task is usually very difficult, keeping in mind the budget constraints that many institutions have. This usually implies that existing open source solutions have to be used and adapted to individual needs of each institution. Keeping up with the current technological advances often brings not only financial challenges, but also transitional challenges that may put at risk learning quality and reputation of the institution, as well as performance of students. This work describes the features of the system, results and challenges of transitioning to e-learning system that displays learning materials through sequence of reusable learning objects (LOs) from the system that does not have these capabilities. The goal of such system is to increase reusability of learning content, and moreover, to increase online interactivity and communication between the instructor and students. Findings of this work reveal advantages, disadvantages and potential obstacle of implementation e-learning system with LOs and give an overview of suggestions for implementation improvements. These suggestions are given based on evaluation of implementation of new e-learning system with LOs, after the transition from the traditional e-learning system. Furthermore, based on the research of existing methodologies in the field of information systems, and the results of this research, this work proposes methodology for transferring into e-learning system with LOs.


Introduction
Rapid developments of information and communication technologies (ICT) are introducing new opportunities in the development of e-learning systems. Traditional e-learning systems are used to organize and publish learning materials, typically in the format of written lectures or video tutorials. However, the tendency for the new systems is leaning towards the personalization and interactivity of learning materials for each student (Chun, 2004). One of the ways to achieve this is by segmenting learning materials into smaller units, with the goal of being able to assemble lessons based on the already existing materials. In the context of e-learning materials, these smaller and reusable units are referred to as learning objects (LO). LOs can be used to guide students' learning by determining the sequence of LOs for each user based on their study curriculum and prerequisite knowledge (Cvetanovic & Raspopovic, 2012). LOs represent a good candidate in online learning for enhancement of learners' effectiveness, performance and experience. Usage of LOs and their repositories are said to offer many benefits to learners (AlMegren et al., 2013).
Much work has been done on analyzing how different types of e-learning systems influence learning experience and effectiveness. The focus of this paper is to identify potential problems and challenges in implementation of a new e-learning system with LOs when transitioning from a traditional e-learning system. The goal of this paper is to discover advantages, disadvantages, and potential obstacles of this transition and to propose suggestions for improvements. Transitional challenges that one institution may face are of interest, as this situation may pose difficulties that are not encountered when introducing the first e-learning system in the academic institution. It is considered important to integrate technology solutions in the process of teaching and learning and that the whole process should be appropriately designed (Cukusic et al., 2010).
According to literature research, much work has been done to propose different methods for implementing new information systems. While using a traditional Waterfall method can be time consuming and can put a big burden on the budget, new methods have been proposed to make this process more efficient. In recent decades, "agile" software development methodologies have been put forth as an alternative to traditional "waterfall" implementation; an "agile" implementation methodology has been put forth (Chan et al., 2009, Brhel et al., 2015. Agile implementation focuses on short cycles (sprints), in order to make faster progress, and to improve monitoring the progress of the implementation. The idea is to complete one part of the system at the end of each system, and to allow re-evaluation of the project's direction. This agile implementation methodology is usually used by SAP in the implementation of their Enterprise Resource Planning (ERP) packet systems. According to the SAP proposal (SAP AG, 2011), the implementation should start by identifying the most important functionalities (called "must have" functions) without which new systems cannot be put into operation. The next iteration allows for delivering the "nice to have" functions which are defined by the so called "delta list". We adopted this agile approach and improved this agile process to be more suitable for the implementation of the e-learning systems.
The paper is organized as follows. Section 2 presents e-learning systems and their integration with learning objects, along with methods of evaluation of e-learning systems. Section 3 presents the research methodology used for result analysis, while Section 4 presents the results and discussion of this research. Section 5 concludes the paper.

E-learning systems and their integration with LOs
Progress in the field of e-learning has been very slow due to the problems that are mostly associated with the poor interface design of e-learning systems (Zaharias, 2005). On the other hand, research confirms the high importance of design and employment of methodologies, tools, and learning environments for stakeholders (both teachers and students), as well as usability evaluation of e-learning applications (Granic, 2008). A significant and important part of an elearning system is the learning management system (LMS). LMS is the "front face" of the entire system and what users are able to see and interact with. An LMS is designed to organize and regulate the administrative tasks of schools and other educational organizations (Esther, 2008).
According to Watson and Watson (2007), an LMS provides functionalities beyond instructional content such as management tracking, personalized instruction, and integration across the system. However, the LMS does not constitute the entire e-learning platform, especially when considering e-learning systems that use LOs. In addition to an LMS, an e-learning platform that can work with LOs has a system for creation and editing of LO content; it allows retrieval and reuse of LO, and delivery of appropriate LO to the LMS.
Not every LMS supports implementation and presentation of learning materials through a series of LOs. The problem with LO-oriented systems is a potential gap between designing different levels of learning and designing and planning a higher-level lesson or course. There are many examples that reflect the problems, from course design down to LOs design. The LDSE project provides the support for learning design including specific learning activities, lessons, and full courses. LDSE points out that designs at the 'higher' layer should incorporate 'lower layer' designs (LDSE, 2009). JISC D4L program (Beetham, 2008) set out the model to address linking between higher-level designs, which include course/lesson planning and LOs design. This model was represented as four layers: course design at the highest level, session or lessons planning, activity design (things learners actually do), and designing LOs. The key weakness of this model was the separation between layers of learning activity and learning objects that was treated as unsustainable.
In designing a learning object layer, two categories of design principles can be implemented: structural and pedagogical. The aim of structural principles is to create self-contained LOs with reusable content. Structural design principles include cohesion and de-coupling (Boyle, 2003).
On the other hand, the aim of the pedagogical design principle is to create rich interactive learning experiences (Boyle, 2008). One system that graphically supports the design approach at the lesson/course level, and is inspired by IMS LD specification (IMS LD, 2003), is LAMS system (Dalziel, 2003). LAMS is based on the selection and sequencing of different pedagogical functions for presentation, discussion, and previously designed LOs.
For effective implementation of LOs within an e-learning system, it is recommended to have a set of assigned metadata for each LO. Metadata enable LO to be described, indexed, and searched.
Several metadata standards have emerged and can be used to describe LOs and their resources: IEEE LOM, ARIADNE and IMS. However, the high quality design and development of LOs, enabled by pedagogical principles, is more important than the LO's metadata description. A new concept of Generative Learning Objects (GLO) was introduced with pedagogical principles in LO design (Boyle, 2006a;Boyle, 2008). GLO establishes an underlying learning design that can be created, inspected, and adapted to meet learners' needs. Learning design tools can be classified in four groups: authoring, ontology-based, standards-based, and generic form-based tools (Charlton, 2010).
There are many authoring tools that allow for the creation and reusability of learning design.
Authoring tools can incorporate just structural, or a combination of both structural and pedagogical design principles. There are some authoring tools that should be mentioned such as Articulate Presenter, Snap, Raptivity, Xerte, Course Lab, Atutor, GLO Maker, and Microsoft content developer (LCDS). Generally, authoring tools should fulfill all features including pedagogy, reusability, and technology (Diwakar, 2013;Dag, 2014). For the purpose of this study, the DITA authoring tool was used.
Ontology enables the organization of learning materials around small pieces of semantically enriched LOs that can be easily organized into customized learning courses and delivered on demand to the user (Nejdl, 2001). It allows for associations between multiple LO repositories and solves the problem of metadata misconception. Ontologies are classified in three categories: (1) a content-domain ontology that solves the problem when semantically identical concepts are expressed by different keywords, (2) a content structure ontology that explicitly defines LO structures that may be useful in many cases when there is a need to reuse a specific part of an LO, rather than using the LO as a whole, and (3) context (didactic) ontologies formally specifying the educational/pedagogical role of a content unit (Stojanovic et al., 2001).
For the purpose of this study, IBM's Darwin Information Typing Architecture (DITA) was implemented as a content structure ontology. DITA provides two structure components: topics and maps. A topic can be used to describe an overview, summary, or particular concepts or tasks as parts of a lecture, exercises, assignments, case studies, etc. The map is used to specify the chosen learning design model. In recent years, learning design has moved away from a sequential waterfall model, where output from one phase serves as an input to the next, and becomes a more fluid and nonlinear process that relies on frequent evaluation and alternation. In this view, learning design should be seen as a complex adaptive system where one component in the design process influences all other components, not simply a sequential element (Ritchie, 1999).

E-learning evaluation
E-learning evaluation is not a one-time activity but rather a complex process parallel to e-learning development as well as its implementation (Dvorackova & Kostolanyova, 2012). A large number of e-learning evaluation models exist. One of the most cited and implemented is the DeLone and McLean model . The evaluation of an LMS must consider its usability and accessibility, as well as its didactic effectiveness in order to achieve educational goals. If an elearning system is not usable enough, it obstructs students' learning and the learners would spend more time learning how to use the software rather than learning the contents (Wong, 2003). The usability of the system is of great importance for e-learning implementation, and therefore, it is also important for the transition from one system to another. It is imperative to research and implement an appropriate methodology for the system usability.
Several studies have investigated the usage of learning objects and their impact on improving the teaching and learning process (Boyle, 2003;Cochrane, 2005;Van Zele, 2003). Khor (2014) showed that users' perception has a significant effect on the acceptance and adoption of usage of learning objects.
According to literature research, there are a number of methodologies for the evaluation of elearning systems: the methodology for systematic usability evaluation -SUE (Ardito et al., 2006); the hierarchy structure for evaluating web-based e-learning systems -WELS (Shee & Wang, 2008); the hexagonal e-learning assessment model -HELAM (Ozkan & Koseler, 2009); the three stages model (Cukusic et al., 2010); and a model for using e-learning systems for sharing experiences, communicating with others, and getting feedback from student's learning experiences (Wang & Chiu, 2011).

The reference model is the Layer 3 of the Institute of Electrical and Electronics Engineers (IEEE)
Learning Technology Standards Committee (LTSC) -standard IEEE P1484.1 LTSA defines three components: Processes (boundaries, services, inputs and outputs of the learning system), Stores (repository of data that can be accessed by users), and Flows (connectivity and the type of information exchanged). The standard IEEE P1484.1 LTSA as a mode defined by an authoritative institution is proposed as a framework that encourages portability and reusability, and provides a good basis not only for architectural and functional design, but also for evaluation of candidate  Learning process (Administration, Activities, Review of competency levels);  Evaluation/optimization (Planning, Realization, Analysis, Optimization/ Improvement).
In our research we analyze our suggested methodology for implementation of LO in e-learning systems by three components defined in the IEEE standard and through the seven process categories of EN ISO/IEC 19796-1. Analysis is conducted through a students' survey.

Functionalities of e-learning systems with and without LOs
An e-learning system that does not have the capability to present learning materials through LOs (Type A systems) is designed as a system with the static and non-LO oriented content. Learning materials for Type A systems contain the materials that are presented and prepared in the same manner for all students. The objective of such system is usually to transfer the knowledge, information or skills from the teacher to students.
LO-oriented e-learning systems (Type B systems) are based on finely granular LOs. LOs represent modular and logically independent reusable units, even though they may be related to other LOs in a larger scope. As discussed by Sicilia & Garcia (2003), reusability is an essential and arguably the most important characteristic of the reusable LOs. Drucker (2000) compares the characteristics of non-LO oriented systems and LO systems through several dimensions: delivery, responsiveness, access, symmetry, authority, personalization, adaptability, etc. By analyzing these two types of systems, the author concludes that they show completely different characteristics.
According to the research of Baki and Cakirogly (2010)  In the study of McCormick & Li (2005), teachers considered that LOs were beneficial tools for learning and that they could be used for scaffolding to understand concepts. Teachers were interested in using LOs in their classrooms. The preferences about teachers' LO use are consistent with the results of case study research about LO use in classroom environments (Ilomäki, Lakkala, & Paavola, 2006). The preferences are shaped through student, design, and contentbased causes. The results about preferences indicate that organizing, structuring, and guiding processes of the course are crucial. In conclusion, as Kay & Knaack (2007) pointed out, students may benefit from LOs if they are comfortable with LOs offering good learning control, useful content, and clear instructions.
In order to describe differences between Type A and Type B systems, two categories of functionalities were considered: preparation and usage of learning materials. Functionalities and differences between these two types of systems are described in Table 1 and are mainly based on the literature review and previous experiences. Even though they support the same categories of functionalities, the teaching and learning scenarios in Type A and Type B systems are performed in different manners. however, all students have to study the same learning contents.
Type B system is suitable for an adaptive instructional design model that relies on frequent evaluation and alternation. Students can access LOs in whatever sequence, not only in a linear one. This means that the system exposes more possibility of direct access to the chunk of knowledge contained in LOs.

Learning materials usage
Learning activities All assignments, assessments, and lectures are published as separate activities. Due to this, the learning material is not easily adaptive.
Due to the possibility of combining different LOs in different manners at the lesson level, Type B system is able to provide more flexibility to instructors when dealing with different types of students.

Interaction with instructor
Training and learning occur as separate activities, with defined starts and stops, so that students are not able to keep in touch with the instructor during the learning process. In addition, students' interactions with the instructor is asynchronous, as the student is not able to collaborate or interact with the instructor directly. Student is able to exchange opinions with the instructor only by emails, conference messages, and forums.
Learning is the integral component of the training process, and it is performed through the iterative process. During the iterations, students are able to collaborate directly with the instructor and other students in order to provide feedback that enables the instructor to continuously change and adapt learning content.

Learning assessment
Assessment is done through grading of assignments and tests. These assessments usually do not influence adjustment of learning materials.
During the learning process, instructor can analyze the current knowledge level of a student based on results of students' knowledge self-assessments and adjust learning material to their needs.

Methodology for transitioning to a new e-learning system
When transitioning from one e-learning system to another, one should consider an appropriate transition model that will allow learners who are used to using a Type A system to maintain and/or increase effective learning. In our case, the goal was to transition from the system that views learning materials as a whole, to a system that allows sequencing of different learning objects (LO), along with different learning activities. As previously mentioned, in this paper the system that can work with LOs is referred to as an LO-oriented e-learning system (Type B), and those that cannot as non-LO oriented (Type A).
Implementation of the new system and the transition from Type A to Type B system can be done in different ways. Since implementing a new e-learning system using the Waterfall method can be time consuming and costly, in our approach we combined Waterfall and agile methods resulting into a hybrid model. The goal is to allow for the implementation of the new system to be structured, adaptable, and flexible.

Hybrid transitional model
Based on the Waterfall and agile methods, we suggested a Hybrid transitional model, as shown in Figure 1. In this model the first phase of the Waterfall method is kept, as it is necessary to plan organizational and system changes, which have to be in line with the institution's goals and mission. In this model, we assume that learners' goals are set and are in alignment with institution's goals. In other words, we assume that the course is already planned out, as it existed In the case of a Type B system, "must have" functions were chosen to allow for an adaptive model of learning design at lesson and course level, and an adaptive teaching/learning process performed through the iterative process.
The following functions were determined as "must have functions" for the Type B system:  (Guduric et al., 2013).
 Learning (instructional design) -For higher level designs, two types of maps were used: lesson maps and course maps. Lesson maps, in addition to different topic types (specialized for overviews, content, summary), include two types of activities: students' self-assessment such as tests containing open questions, true/false, single select or multiple answer type questions, and activities that enable communication and interaction, for example, chat, wiki, forum, etc. In the initial phase of the project, basic functions of the old system were maintained and included in "must have" functions, which related to interactive questions in test format for self-assessment and discussion forums. The course map's purpose is to organize a sequence of learning materials, including LOs, weekly assessments (test, homework, exercises), and course projects. For lesson and course preparation, the LAMS system was used.
 Publishing -The publishing methods were chosen based on the previous learners' habits. Output formats were mainly chosen to be published on web using XHTML, as well as in PDF format. LAMS can be automatically instantiated as a Web-based learning session using the LAMS delivery infrastructure.
Before learning material is developed for an individual course, it is recommended that the base course is developed. The base course will be used to train other instructors on how to adapt and enhance their course materials for the new system.
In order for an e-learning system with LOs to be successful, it is necessary to use high-quality LOs that are structured, adaptable, and ultimately reusable in different contexts. Even though one may try to maintain modularity and logical independence of each LO, in actuality, group of LOs are usually developed in order to meet the needs of a certain learner, usually having in mind a particular course. These LOs are developed by course instructors and/or multimedia developers (Boyle et al., 2006). While course instructors are mainly in charge of developing content and pedagogical methods for presenting the materials, the multimedia developer's role is to enhance LOs with visual presentation under the instructor's supervision. What should be kept in mind is that interaction between course instructors and multimedia developers in this model represents an iterative process. Once the course instructor develops the content for LO, this content is forwarded to the multimedia developer. Another step can be incorporated in this iterative process in which the learning content should be first forwarded to the reviewer, who should be a domain expert. Even though involvement of the reviewer can ensure and improve academic quality of LO's content, at the same time it may increase the cost of the entire process. Once the instructor forwards the LO, the multimedia developer develops the multimedia based on the given specifications. In this iterative process, the resulting LO is given back to the instructor for a review. This iterative process between instructor, reviewer, and multimedia developer continues until all specifications and criteria are met, and the LO is published (Cvetanovic & Raspopovic, 2014).
Once LOs are published, the instructor can receive feedback from learners about the published material. More importantly, quality assurance and evaluation can be completed through involvement of the learner in the entire process, so that learning materials are tested with a certain group of students. This is referred to as "use before reuse" (Boyle et al., 2006).
Considering that these LOs are used in a "live" scenario, these evaluations can be done from a one-week period to the entire semester. This feedback and analysis should give a better input about things that should be changed, adapted, and enhanced throughout the semester. More importantly, from this type of evaluation, conclusions can be made as to how the entire system can be improved. In this evaluation process, "nice to have" requirements and functionalities are determined. These are then used for the next phase of system improvements. "Nice to have functions" were determined based on the faculty and student evaluation throughout the semester, and finally at the end of the semester, after the evaluation of the entire system was completed.
Our proposed hybrid model elements are based on ISO dimensions (Table 2).

Research Methodology
In this paper we propose a model for transitioning from a non-LO system (Type A) to an elearning system that supports generation, sequencing, and publishing of LOs along with different activities (Type B). The impact on students' learning and study process by transferring from a Type A to a Type B system was analyzed. The purpose of this study was to analyze effective ways of changing an e-learning system and how these changes influence satisfaction, learning experience, and usage of the system. Furthermore, it is of interest to analyze how this transition can contribute to student motivation and satisfaction. The examination of this evaluation was expected to reveal good practices, obstacles, and needs for future improvements. In the conducted case study at the academic institution, the Type A system was used for eight years, before deciding to change to a Type B e-learning system. During the first year of the transition to the Type B system, the process of transferring from the Type A to the Type B system was documented and an initial evaluation was performed. To illustrate the outcome of applying these practices, the illustrative case study examined the following:  Students' degree of satisfaction with the new system  Student satisfaction with published materials  Students' motivation and driving forces to be active (i.e., communication with peers, interaction with the instructor, etc.) Current literature and methods of developing, implementing, and evaluating information and elearning systems were employed (Boyle et al., 2006). This resulted in adapting an agile methodology for the development of course content in form of LOs and overall implementation of the Type B system, presented in the following section.
Faculty was asked to develop their courses in the form of a sequence of LOs and interactive activities. Both online and traditional students were asked to evaluate these LOs by using these materials in their courses for the entire semester. Data was collected throughout the semester through student interviews and at the end of the semester through a web questionnaire. Data from student interviews was used to improve the quality of learning content throughout the semester. Statistics comparing students' usage of the system and satisfaction with the new elearning system was analyzed. 184 students participated in this evaluation, out of which 93 were traditional, and 91 online students. These results were compared to the previous year's results, collected prior to the introduction of the Type B system. As a reference, the analyzed

Analysis of success factors of the implemented transition
The evaluation of the Type B system and its implementation was done continuously throughout the semester with traditional and online students. Traditional students used learning materials on the e-learning system in addition to traditional lectures and labs in the classroom in blended-style learning, while the online students used the same materials without attending the lectures. Each lesson on the e-learning system consisted of a sequence of different topics (LOs), interactive questions in test format, discussion forums, assessment tests, homework assignments, exercises, projects, and summarized learning materials in a written lecture form in PDF format. At the end of the semester, students were asked to give their opinion on their learning experience via a survey. Each survey question offered scaled marks from 1 to 5 (1-unsatisfactory, 2-sufficient, 3average, 4-very good, 5-excellent). 184 students participated in this evaluation; 93 were traditional, 91 were online students, while 108 were men and 78 women.
The student survey includes analysis of IEEE components Processes and Flows (survey questions for evaluation of students' degree of satisfaction shown in Table 3) and Stores (survey questions for evaluation of degree of satisfaction about the quality of learning materials published on the Type B system shown in Table 6).
Based on the average marks on the survey of students' satisfaction (Table 3), students were most positive about interaction and additional consults with instructors, 3.96 ±1.08, and their current learning motivation, 3.72 ±1.18. Similarly, they expressed their positive opinion about weekly course assignments, which helped them during their learning, 3.66 ±1.33 (with the median 4, which corresponds to a very good mark). Students expressed that the least contribution in helping them learn was provided by discussion forums on the Type B system, and that this type of activity did not contribute much to better understand the learning topic, 2.26 ±1.30, with the median of 2 (corresponding to a sufficient mark). These findings show that students' learning experiences are centered on problem-based learning and communication mainly with course instructors, and also with their peers. These results derive "nice to have functions" that should be considered within developing and modifying LOs. These "nice to have" functions point out that pedagogical methods that focus on increasing efficacy in communication with an instructor and classmates should be integrated with a Type B system. One possible approach to increase communication is to incorporate a social learning aspect within the Type B system, and foster problem-based learning through teamwork within this environment, guided and administered by the course instructor.

P1_10 How important is your interaction and additional consults with instructors
3.96 ± 1.08 4 Table 4 Result of survey questions based on the type of studying Results from Table 4 were analyzed based on the variable "type of study" (online and traditional).
It is clear that a statistically significant difference was established on most questions when two types of studies were examined. Traditional students were satisfied with established communication with classmates and they consider that this activity contributed the most in their learning process (P1_4 and P1_5, p<0.001). On the other hand, online students expressed the high importance of their interaction and additional consults with the instructor in their learning process (P1_10, p<0.01). At the same time, online students expressed the positive opinion about their activities on discussion forums, interactive activities, and exercises on the e-learning system (P1_8 and P1_9, p<0.001). Online students are also satisfied with the quality of online learning, and they think that their surrounding shares their opinion (P1_2 and P1_3, p<0.001). These results are in line with previously drawn conclusions about adding "nice to have" functions that will emphasize pedagogical methods that foster communication and problem-based learning, along with teamwork and group activities. One can notice that traditional students expressed lower satisfaction with the e-learning system. Lower satisfaction can be attributed to the fact that traditional students used the e-learning system in blended-style learning with the main way of learning being class lecture and the e-learning as an additional source (not the only source). For establishing the statistical significance a Mann-Whitney test was used. It was of interest to analyze whether the correlation exists between the analyzed parameters in order to be able to better improve the system and predict future results of improvements. As shown in Table 5, it is evident that a statistically significant positive correlation exists between most questions. The degree of correlation was classified as follows: small (from 0.10 to 0.29), moderate (from 0.30 to 0.49), and high (from 0.50 to 1). These results show that students who are satisfied with discussion forums are also satisfied with interactive activities and exercises (P1_8 and P1_9, p<0.001); students who have a positive attitude towards the type B system consider that discussion forums (P1_7 and P1_8, p<0.001), interactive activities, and exercises (P1_7 and P1_9, p<0.001) are very helpful for better success in studying and knowledge acquisition; students who have established good contacts with their classmates also feel that this connection contributes to their better success (P1_4 and P1_5, p<0.001); students who have a positive opinion about the quality of online learning also consider that their surrounding have the same opinion (P1_2 and P1_3, p<0.001). A second part of the survey attempted to analyze how satisfied students were with published learning materials (Table 6). Based on the average marks from this part of the survey, it is noticeable that the highest mark was given for written lectures in PDF format, which actually represent summarized materials and leave students without utilizing the full functionalities of the Type B system, 4.51 ±0.96, with median of 5 (corresponding to the highest mark). The lowest mark was given to discussion forums, 2.08 ±1.30, with median of 2. This result shows that students did not take in the new system fully, and that their tendency is to study based on the learning methods from the Type A system. This result is significant, as it should lead to the improvements and further development of interactive and self-assessment activities on the Type B system, which would emphasize more learning outcomes and significance of each LO and its activities in the students' learning process. On the other hand, even though the discussion forum received low marks, this does not necessarily mean that the forum represents a poor "methodical tool", but it could also signify that not enough students have used the forum, so the discussion never developed between students, as well as between students and teachers. This also leads to the need for further development of "nice to have" characteristics of the LO system that should be implemented for further improvements by including more LAMS activities, and even be considered as "must have" in the next iteration. To step away from the old system, consideration should be given to remove residual functionalities of the Type A system because it detracts students from fully embracing the new system, such as written lectures in PDF format.

Comparison of the results between Type A and Type B systems
In order to compare students' satisfaction with the Type B system, the results were compared with the analysis previously conducted for the Type A system. Students' level of satisfaction was classified as satisfied (when given marks 4 or 5) and unsatisfied (when given marks lower than 4).
It can be noticed that students' satisfaction decreased after transitioning to the Type B system.
Moreover, these results point out that introducing a new system with significantly different functionalities, such as a Type B system, can in the short term decrease students' satisfaction with learning experiences (Table 7). When introducing a new system, it can be expected that during the adjustment period, students may show the preferences towards the old style of learning; hence, it is important to carefully introduce novelties and tend towards their improvements. This is the reason for introducing "must have" functions as functions that are closer to the Type A system, to allow students an adjustment period. However, results show that this approach did not allow students to embrace new functionalities fully. Future steps in determining "nice to have" functions should move away from Type A functionalities, so that pedagogical methods that increase efficacy of learning through communication and problems solving can be further explored.

Conclusions
This paper proposed a hybrid transitional model for the implementation of an e-learning system that supports interactive learning through the sequencing of Los and interactive and selfassessment activities. The hybrid transitional model was based on the combination of waterfall and light agile methods. Content of learning materials from a non-LO oriented system served only as a baseline for developing content of LOs. The proposed hybrid transitional model required careful definition of the "must have" and "nice to have" functions. "Must have" functions have to be primarily based on the vital functionalities, without which the system cannot be put into operation. For the easier transition from the old to the new system, some of the "must have" functions were chosen based on the identified vital functions of the previous system. The main disadvantage of choosing functionalities of a non-LO oriented system to be a part of "must have" functions was instructors' high level of familiarity with the previous system and the time and training needed for adoption of the new approach and structure of LOs. They in some way tried to maintain basic functions of the old system, which does not allow for full utilization of the new model. Furthermore, residuals from the new system also represented an obstacle among students, who were used to written lectures in PDF format and had a hard time accepting new structured, segmented, and reusable LOs.
Development of LO content, peer review of LO, and multimedia development were recognized in the proposed model as a part of the Development/production ISO dimension. A suggested methodology for implementation of LO was analyzed using three components defined in the IEEE standard and through seven process categories of EN ISO/IEC 19796-1. This evaluation was also tightly connected with an assessment of an LO e-learning system from the students' point of view, and according to students' survey analysis, improvements of LO content with "nice to have" functions are determined. In order to test the proposed model, the evaluation of the hybrid transitional model was conducted on the case study based on the students' survey in order to verify "must have" and "nice to have" functions, the success of the transition, and to point out the key points of the system that can be improved. Analysis showed that students' learning experiences were centered on problem-based learning (weekly and project assignments) and communication mainly with course instructors, and also their peers. Traditional students gave high importance to establishing good communication with their classmates, while online students gave the same importance to their communication with the instructor and additional consults needed for their learning of the material. These findings point out that additional "nice to have" functions should be explored, in order to integrate pedagogical methods to suit the needs of the students. One approach to address this is to incorporate a social learning aspect within an LOoriented system and foster problem-based learning and communication through teamwork and group activities, guided and administered by the course instructor.
For the future iterations of improvements of the system, "must have" requirements should focus on development and implementation of more appropriate self-assessment activities, as students find them important. These activities are directly linked to how students perceive the importance of the e-learning system in their studying. Furthermore, future work should explore satisfaction and inputs of the system evaluation from the faculty and institution management. Moreover, it will also be of great benefit to compare students' academic success and results from non-LO and LO-oriented systems. These analyses will be of interest, once the institution has gone through several iteration of improvements and implementations of "nice to have" functions.