|
|
Anna Mavroudi1 and Thanasis Hadzilacos2
1Open University of Cyprus & University of Cyprus, 2Open University of Cyprus & Computer Technology Institute and Press “Diophantus”, Greece
Analysis, the first phase of the typical instructional design process, is often downplayed. This paper focuses on the analysis concerning a series of e-courses for collaborative adult education in semi-formal settings by reporting and generalizing results from the REVIT project. REVIT, an EU-funded research project, offered custom e-courses to learners in several remote European areas and received a ‘best practice’ distinction in social inclusion. These e-courses were designed and developed for the purpose of providing training in aspects of the learners’ professional domains related to the utilization of information and communication technologies. The main challenge was to prove that it is possible and economically feasible to provide meaningful training opportunities via distance education, by utilizing existing infrastructure (“revitalizing schools”) and by making use of modern digital technology affordances coupled with suitable distance learning techniques and Web 2.0 tools. ADDIE, the generic instructional systems design model, enhanced with a rapid prototyping phase, was put forth in order to allow stakeholders to interact with a prototypical e-course, which served as an introductory lesson and as a reference point, since its evaluation informed the design choices of all subsequent e-courses. The learning needs approach adopted in REVIT combined learner analysis, context analysis, and needs analysis into a coherent analysis framework in which several methods (observation, estimation, document analysis, survey, and dialogue) were exploited. Putting emphasis on the analysis phase and decoupling the design from the delivery of the e-courses facilitated adaptation and localization. Adaptation and localization issues concerning the adoption of the REVIT distance learning framework, taking into account the socio-cultural and pedagogical context, are discussed. A central result reported is that the analysis phase was crucial for the success of the whole endeavour and that carrying it out properly is not straightforward or easy. The analysis framework presented in this paper could be useful in other similar e-learning situations whose “educational-identity” also involves distance and adult learning in a semi-formal setting.
Keywords: Distance education; adult learning; instructional design; analysis in e-learning; collaborative e-class
Although analysis and evaluation are regarded as crucial phases in the instructional design process since “[they] concern the immediate needs of an organization” (Horton, 2003, p. 25), they are often overlooked in e-learning projects (Shelton & Saltsman, 2011, p. 567; Rothwell & Kazanas, 2008; Wang & Wilcox, 2006; Trolley, 2006). Spending too little time in analysis is a mistake commonly made not only by inexperienced (Cook, 2005), but also by expert instructional designers. Indeed, a research study (Visscher-Voerman & Gustafson, 2004) that investigated how expert instructional designers actually work with regard to the implication of the ADDIE model revealed that only 17% (4 out of 24) conducted a full analysis, while the others performed a limited one. Nevertheless, failing to fully explore the analysis phase in an e-learning project can bring about a negative impact on the relevancy and the quality of the educational endeavor (Shelton & Saltsman, 2011). The learning needs analysis is a key step of the instructional design process since the data gathered in this phase drive the nature of the intervention and its outputs inform the creation of learning objectives (Cook, 2005). It also has a significant impact in the continuous professional development of those receiving the training (Forbes et al., 2006). A limited number of case studies exist in the recent literature, which refer to the analysis phase in educational contexts related to continuous professional development in non-credit lifelong learning. Examples include a survey of the information needs of rural and non-rural primary care physicians (Gorman, 2001) and a study that examines the training needs identified in adult rural communities (Frossard & Frutos, 2011). Yet, these examples do not incorporate the analysis as a phase of their instructional design model.
The research question for REVIT was: Can we provide attractive, meaningful, and relevant educational opportunities to adult professionals living in remote areas (in Europe) utilising existing infrastructures, Web 2.0 affordances, and available human resources in a way that is sustainable and economically feasible? Consequently, the purpose of the paper is to systematically explore a multifaceted analysis strategy to address instructional design in the context of a scalable and economically feasible example that provides lifelong learning opportunities to people who are usually excluded from mainstream educational activities, such as the inhabitants of remote areas. A central result reported here is that the initial analysis phase was crucial for the success of the whole endeavour and that carrying it out properly is not straightforward or easy.
The REVIT project (REVITalizing Remote Schools for Life-Long Distance e-learning, http://revit.cti.gr, Lifelong Learning Programme/Transversal Programme/Key Activity 3: ICT-European Commission, EACEA, January 2009 to December 2011) was addressed mainly to adults who were interested in receiving further training in aspects of their current or envisaged professional domain; usually those were aspects related to the utilization of information and communication technology. For this purpose, e-courses were designed and developed based on the actual needs of the specific remote application areas, as these were elicited from local people in Cyclades (Greece), Fornetka (Poland), Ullava (Finland), Kula (Bulgaria), and Palaichori (Cyprus).
Figure 1 illustrates two points about the REVIT project: 1) the geographical distribution of the application areas and 2) the fact that in each application area a dyad of project partners existed. The five dyads (the dyad of project partners shown in the map of Figure 1 in Italy is an exception) co-operated closely throughout the project duration towards the completion of the main project goal, that is to provide training via distance and online education in aspects of the learners’ professional domains related to the utilization of ICT. These dyads were comprised from one partner located in the application area (usually the municipality of the area), which liaised with the locals, and one partner in the same country, which liaised with the project consortium (this partner could be a university or a research center or an adult training center).
The REVIT project placed a good deal of effort in conducting an extensive analysis and a comprehensive evaluation. The methodology and the results of the evaluation phase of the REVIT project are available online (Dagdilelis, 2010) while this paper focuses on the analysis phase. Design, development, and implementation can be treated as a ‘black box’, since the combination of the pre-analysis and the post-evaluation confirms that the needs were met, given the actual commitment of the learners.
Another similar endeavor that was running in parallel with the REVIT project was the SoRuraLL! project (Rurall Social Networking for Lifelong Learning, http://www.sorurall.eu) which investigated the potential for enhanced lifelong learning offered by social networking tools and platforms to those living in geographically and socio-economically disadvantaged rural areas. These projects exchanged ideas and experiences in domains of relevance (i.e., social applications for lifelong learning) in a set of conjoint events (meetings, conferences, etc.).
The many definitions of instructional design (ID) come from two different perspectives: a process or a discipline. Here the definition adopted is the following: “the entire process of analysis of learning needs and goals and the development of a delivery system to meet those needs. It includes development of instructional materials and activities; and tryout and evaluation of all instruction and learner activities” (Berger, 1996).
As a process, ID mostly complies with the ADDIE model (Visscher-Voerman & Gustafson, 2004), an acronym for analysis-design-development-implementation-evaluation. Although the ADDIE model has received much criticism (Bruce, 2004; Colón et al., 2000), most of the instructional systems design (ISD) models are based upon it (Allen, 2006). Contradictory as this may seem at first, a satisfying explanation is given in Stolzer et al. (2011, p. 26): The criticisms for being too systematic (i.e., cumbersome, inflexible, linear) refer to the ADDIE model as originally proposed which resembled the waterfall model of software development (MacCormack et al., 2003). Recently the model has evolved “into a more dynamic process in which the user can go back to the other phases as needed and revise steps as determined by the evaluation” (Stolzer et al., 2011, p. 26). In REVIT, ADDIE was enhanced with a rapid prototyping stage (Boulet, n.d) between design and development, as an extension of the design phase (Kruse, n.d). The model adopted in REVIT is shown in Figure 2, where its two key elements (dynamic nature and rapid prototyping phase) are depicted.
A central issue in instructional design is independence of phases, in particular from people involved. Each phase should be based exclusively on the output of the previous one and not depend on the same person being, for example, both designer and implementer. The several advantages of this independence come at the cost of process rigidity. Having the same person or team in several roles provides flexibility but hinders independence. In REVIT we opted for a more flexible approach, in which people working in each stage of ADDIE were encouraged to liaise with persons in the previous and subsequent phases. Thus, persons carrying out the needs analysis were different from the course designers and from those delivering the course (tutors), but were available for interaction beyond formal documents. In this way and by adding a rapid prototyping phase for the first delivery of each course we were able to uphold a dynamic nature of the REVIT instructional design model.
A prototypical e-course (i.e., a series of lessons that constituted a course named “Introduction to Web Technologies”) was implemented and evaluated after the completion of the high-level design of the e-courses. The evaluation results were used to make corrective actions in the overall design and also informed the development of all subsequent e-courses.
The REVIT distance learning framework is shown in Figure 3. The picture of the school depicts a basic idea for the sustainability of this European research project: the exploitation of the existing infrastructure and human resources of schools in the remote areas. For the residents of these rural areas, their often small multigrade school plays an important role as a centre of culture and education for the whole community. Moreover, research work, for example Miller (1993), illustrates the important role of rural schools in their communities and the promising approach of collaborative partnerships between communities and schools for community revitalization and survival. The school acted as a central place in the REVIT framework where the local moderators and the learners could get together during the delivery of the e-courses. The e-tutor, knowledgeable in the subject of the course, was the teacher delivering the e-courses at a distance. The local moderator was the person in charge of the learning process inside the classroom. This role was usually undertaken by one of the school teachers in the application areas. Their main duty was to foresee and solve problems that would threaten the learning process (like technology issues). A second level of national support for each participating country was provided by a university or a research organization. So learners were benefiting from four sources: the local moderator, the distant tutor (instructor), the subject matter expert (too expensive to be a tutor for a small audience), and the university partnership (that catered for the instructional design as a whole and provided the technological solutions).
As an e-learning project whose “educational-identity” involves distance and adult learning, the REVIT specifications and constraints were raised by an amalgam of knowledge about adult learning and distance education. Basic principles for adult learning (Rogers, 1996; Rogers, 2005; Jarvis, 2004; Knowles, 1990; Rosemary et al., 2006; Schön, 1987) posit that adult learners are autonomous, self-directed, goal-oriented, and relevancy-oriented. Adults need to attain practical learning results that are forthcoming, tangible, and directly relevant to their own experiences and vocational or other aspirations. To create a learner-centered curriculum which is closely related to real-life situations and also close to the learners’ needs and aspirations, the instructional designer needs to know these needs and aspirations. Our aim was to foster community inquiry through “pragmatic technology” (Hickman, 1990; Bishop et al., 2009) with the concern to design a learning environment that meets actual learning needs, assesses knowledge in terms of its practical usage, and accommodates the learners’ lived situations. Moreover, there are different learning styles or characteristic ways in which adults prefer to learn (Rosemary et al., 2006) and in order to engage all learners, it is required to vary the methods in which information is communicated.
The project involved a collection of e-courses that combined a variety of media, tools, and methods:
On the other hand, Moore’s work on transactional distance acknowledges that the separation in time and space between students and their tutors may have a profound effect on learning and teaching in a distance education setting. This transactional distance can be defined as the gap of understanding and communication between the teachers and the learners caused by the spatiotemporal separation “that must be bridged through distinctive procedures in instructional design and the facilitation of interaction” (Moore & Kearsley, 2005, pp. 223-224). Towards this end, the REVIT learning material was subjected to instructional design procedures that aimed at creating meaningful, practical, and relevant learning experiences. Also, the affordances of computer supported collaborative learning and e-learning 2.0 tools (mentioned above) were exploited in order to eliminate the transactional distance and to offer opportunities for dialogue and interaction. Social networking functionalities (Facebook, Twitter) were also incorporated in the REVIT distance learning framework to establish and maintain ties with experts, e-tutors, instructors, learners, and other communities.
Additionally, in alignment with the basic principles of learner-centered design – “understanding is the goal”, “motivation is the basis”, “diversity is the norm” (Soloway et al., 1994) – the topics of the courses were not predefined, but decided by the learners after constructive discussions with the instructional designers. An appropriate adjustment from what is academically valued in a knowledge domain to what is valued by the specific learners, while carefully examining learners’ characteristics, ensures that they will perceive content (rather its transformation into knowledge) as worth knowing (Sims, 2001; Parchoma, 2003).
The e-courses included topics like “e-business”, “organic farming”, “rural tourism” and others. Eleven e-courses were offered in total, ten designed for adults and one for pupils. Each e-course lasted for a semester (10-13 weeks).
The design of a course starts from the educational objectives. They rely on a learning needs analysis, that is a set of activities for assembling information about the learners, their needs, wants, and the environment in which learning will take place. Based on the educational objectives, lesson plans, materials, tests, assignments, and activities are developed. The learning needs analysis clarifies the purposes of the educational program.
Learner-centered and participatory models of instructional design emphasize learners’ characteristics – demographics, needs, preferences, and experiences, among others (Parchoma, 2003). Learning needs identification includes:
The analysis conducted for REVIT included investigation, elicitation, and commitment.
Investigation was a feasibility study on the local technical and social space. For the technical space an investigation of the available facilities in the application area (Tessmer, 1990) was conducted for several parameters and factors involving the technical infrastructure (Internet connections with adequate bandwidth and available computers, cameras, and microphones in the support environment, i.e., the school and homes of participants). Another important factor for REVIT was the adequacy, suitability, proximity, and availability in practice of the physical resources, mainly the school or community classrooms.
For the social space, in order to clearly form the profile of the learners, a few visits in the application area were organized. Except for learners’ demographics and needs, in the context of adult learning in informal settings using distance education, elements like learners’ sociocultural context (Vygotsky, introduced by Tudge & Scrimsher, 2003), motivation (Jones & Davis, 2011; Bandura, 1978; Salomon, 1981; Gagné, Briggs, & Wager, 1992), and expectations from the educational program (Jones & Davis, 2011) were considered as highly important parameters and, consequently, informed the design decisions. Since planning instruction with the learner’s expectations and needs in mind always helps in achieving the learning objectives and being aware of the fact that the learners have high expectations from the use of technology in distant courses (Jones & Davis, 2011), this was something that needed to be reclaimed as an educational requirement. Motivation has been reported frequently (for example, McNair & Quintero, 2008; Marcinkiewicz, 2011) as a key success factor of learning with the use of ICT at a distance. Similarly, the learners’ sociocultural context plays an important role in distance education in general (Valcke & Leeuw, 1998), but also specifically in our case where the application areas were remote villages in five European countries.
Geographical factors such as distances and accessibility always affect blended learning where students do whole physical meetings, albeit rarely. Relationships are always important and in small places more so. Availability varied daily (for teachers), weekly (in islands where the visiting boat schedule affects the life of the whole island), and yearly (for farmers). Local technical help (usually by a volunteer teacher, agronomist, or just a young person) can be a make or break factor at the beginning.
Elicitation was the main stage of the needs analysis, in which learners were helped by the instructional designer to conduct a self-diagnosis regarding the learning tasks that might be most helpful to them. Here the instructional designer acted as a counselor who helps the students identify the knowledge and skills that are most beneficial to them (Brookfield, 2009). Informal communication was encouraged at this stage by the project team since it could convey valuable feedback concerning the learners’ attitudes towards the educational venture at stake.
Elicitation involved a variety of interviewing and questioning techniques, like small group or public discussions, brainstorming (McDermott, 1982), and diagnostic questionnaires in the form of online survey (Patterson, 2009) or in the traditional, written (offline) survey method, in order to obtain relevant performance data in topics of interest (Rorgiquez, 1990). Samples of the online questionnaire used in the Polish application area, as well as a sample of the questionnaire handed out in written form to the learners of the Bulgarian remote area, are shown in Appendix A. More details about the description of the methodology in each country concerning this stage are described in Appendix B. In practical terms, after a few visits in the application areas, the instructional designer team was able to estimate topics of common interest and the actual number of the participants in each of them. The method of informal discussion (“speaking their language”) proved helpful as a means of detecting educational needs in informal or semi-formal settings, as well as potential barriers.
For REVIT, taking into account the sociocultural context meant:
Receive learners ‘commitment of participation (Rodriguez, 1990). Our aim was to agree on a learning contract. In theory, a learning contract is “a formal, written agreement … about what the learner will learn and how that learning will be measured” (Boak, 1998, p. 1). The use of learning contracts may foster the creation of online learning communities (Mahoney et al., 2000) and contribute to establishing trust and building a sense of community (Allan & Lawless, 2003). In REVIT practice, the learning contract was a good presentation of the learning goals aiming at reaching consensus about them with the learners (10-20 learners in each remote area). In small communities, oral commitment expressed in public can be much more binding than private written commitment or even financial commitment.
Gaining the learners’ commitment is an important process in promoting among the learners the sense of greater responsibility and control of their learning.
A prerequisite to asking the learners’ commitment was to discuss with them the following aspects:
From the learning needs analysis emerged that the first educational goal would be to provide the level of digital literacy required to use the REVIT learning tools and services effectively. It was expected that the learners wouldn’t have a good ICT background which was confirmed from the learner analysis. To remediate for this, the e-course “Introduction to Web Technologies” was the first to be designed and offered to all the perspective learners. Of a value by itself, this course helped reduce the learners’ cognitive barriers related to the lack of ICT-competence. The learning needs analyses conducted by five coordinated teams in the five remote areas converged into another 10 e-courses. For each e-course, the learning objectives reflected the competencies needed from the learners’ standpoint and were clearly stated in terms of knowledge, skills, and attitudes. For instance, in the “EFL Teachers using Web Technologies” the main educational aim was training EFL teachers to make successful lesson plans. The learning objectives were explicitly stated as follows:
The REVIT methodology for needs analysis of trainees proved to be well adapted and efficient, as judged a posteriori by the participants (see Table 1 and Table 2). The contacts with the locals have led to an authentic investigation of the local culture and the important points in the life of eventual participants. For instance, in Ios island (Greece) the time of the arrival of the boats (with tourists) was very important for the local economy and life and, indirectly, affected the synchronous courses. In most cases, the course topics were important for the professional and personal life of learners (see Table 3). Also, the initial assumption that the learners wouldn’t have clear ideas on how ICT would help them in their professional development was confirmed by the learning needs analysis, as well as that the learners living in the same remote area would strongly affect each other’s opinion about the value of the program. The latter information was provided by the local moderator who liaised between the project team in each application area and the locals and who was in constant contact with both sides (via physical meetings with locals and via computer mediated communication with the project team).
The questionnaires, addressed to trainees via an online survey component that was integrated in the REVIT DL platform, were answered by 136 people in total (80% of the total number of participants). The repartition of the answers to the countries is 24% from Bulgaria, 17% from Cyprus, 18% from Finland, 17% from Greece, and 24% from Poland.
A variety of interviewing and questioning techniques were used in the analysis process in order to provide a contextualized approach that would harness socio-cultural diversity and foster the elicitation of actual learning needs, while also studying previous work concerning lifelong learning (LLL) best practices in rural areas derived from previous projects. The REVIT team mainly examined projects that targeted the adoption of innovative techniques and entrepreneurship by adult professionals. Examples include: training initiatives of the Chambers of Commerce in Italy1, Ecologica2 (online learning for organic farming), and others. Moreover, the team built on the experiences gained by the MustLearnIT3 project , a venture also acknowledged as a “best practice in social inclusion” aimed to provide learning opportunities for children in remote rural communities in a way that included their teachers and the community as a whole. The impact of the MustLearnIT project has both inspired and triggered the REVIT project and the experiences gained by the former provided valuable insights on various aspects of the latter, like how to adapt the REVIT distance learning methodology to the context of school education (one of the REVIT e-courses was addressed only to pupils).
The flexibility in the needs analysis meant that various techniques were used in the participating countries. For example in Cyprus analysis of national statistical data concerning barriers in the uptake of LLL by adults living in remote areas clearly showed that the main reason for not participating in formal or informal LLL activities was lack of time due to family duties (41.4% in rural areas) followed by the fact that the training program was not compatible with the job time schedule (30.2% in rural areas).
On the other hand, in order for the REVIT project partners to follow similar methodological approaches as far as needs analysis was concerned, specific guidelines were formed early in the project indicating the most appropriate methods for examining educational needs of schools located in small, remote villages. Thus, the partners had to select one or more of the following methods:
An example: If prospective learners already had internet access and were digitally literate, then an online questionnaire would be more appropriate. With the use of these methods each partner country that participated in implementing courses in the REVIT project provided a study on
The project managed to reach the outcomes described below under tight timetables and economic constraints agreed before the start of the project with the funding agency. REVIT was considered a best practice in the field of social inclusion (U. Haller-Block, personal communication, 22 May, 2012) by its funding agency, the Education, Audiovisual & Culture Executive Agency of the European Commission (EACEA). The development of a learner-centered curriculum and the provision of the e-courses was one of the main priorities agreed between the funding agency and the REVIT research team. This gave the team the opportunity to conduct the extensive analysis described in this paper, speaking in economic terms (i.e., budget allocated for this purpose). The economic feasibility of the educational venture was achieved mostly by keeping low costs for the design, development, and implementation of the e-courses, while keeping high quality outcomes in each of these phases. Rapid prototyping has helped towards this direction and also the fact that in this particular e-learning project the duration of the e-course was at most three months. The subject matter experts were called to create the e-courses at a low cost, a ‘light’ version of the course material that was easier (since the course was addressed to beginners) and limited (since the course had limited duration) compared to the materials that they were using in their everyday teaching practices. The subject matter experts in some cases could also play the role of the e-tutor, depending on their availability and competencies. This also illustrates the principle of separation between the development and the implementation of the e-courses that was adopted from the start of the project.
Concerning the needs analysis, it is not very clear how local conditions could be determined precisely in all cases: discussions with as many as possible people of the target area, knowledge of the infrastructure, and the explanation of all mutual duties and obligations; all these elements seem to be necessary. However, it is true that this thorough needs analysis is difficult and may be too expensive for a large scale application. In a small community it is easier to contact eventual participants and to determine their needs with accuracy, while in larger areas it is more difficult. In this case, a balance should be established.
A significant number of drop-outs was observed (42 out of 212 adults that originally participated in the program). Even if, in general, a large percentage of drop-outs is expected in distance courses (Tyler-Smith, 2006; Wang et al., 2004) this still remains an important open question. Lack of time was an important factor for these dropouts, as the evaluation results revealed. Also, learners reported that the difficulty level of the learning activities was sometimes too challenging and other times too easy for them; or that the heterogeneity of the learners’ backgrounds made it impossible to balance interest and learning. Other examples relate to course length and computer availability: Even if the course length was appropriate for some, it certainly was not so for everybody; a computer at home does not guarantee ready access at convenient times. On the other hand, only a very large scale project which could vary these parameters in distinctively different ways and provide personalized instruction could give proper answers. It would also be interesting to revisit the areas a few years later and gauge (a) how learning has progressed over time and its impact on the participants’ professional lives (i.e., what learning has been retained and put to use) and (b) what the opinions of learners, drop outs, and non-participants are, in retrospect, about the value of the endeavor as an educational program. Finally, although the analysis resulted in topics that had an impact on the trainees’ professional and personal lives, it should be noted that this detailed analysis was a factor for a considerable increase of the total cost.
The know-how gained from the design and the implementation of the project can be capitalized as a “model” (or at least as elements of a model) for future projects or courses that could be organized under similar circumstances. This project is different from the usual distance learning courses (and that is the point where new, important know-how is gained) in that courses were not created in abstracto for some virtual community of users/learners, but were designed and constructed specifically for people living in these remote areas with educational needs defined through a thorough examination.
Although the e-courses were well-prepared, in some cases contradictory estimations from participants emerged. This phenomenon (contradictory statements) was observed for instance in the case of their preference for printed learning material, as opposed to digital learning material. Previous research (Moore & Kearsley, 2005, pp. 91-92) has actually confirmed the REVIT experience, that is, some learners prefer printed materials, giving reasons of reliability, portability, and habit. This put forth the pedagogical requirement of having the learning material available in various formats (including in printed form) and then letting the learners choose the most appropriate for them.
The participants (learners) in the e-courses represented a set of people with diverse characteristics (age, sex, profession, level of education, country) which is an indirect indicator of the reproducibility of the findings. Also, the components of each course were defined independently from the content, which was provided by experts of the field, and adapted to the standard forms of the e-courses, ready to be delivered at a distance. Finally a third person, a teacher or a local moderator, was responsible for the delivery of the course (the “teaching”). The separation of the form and the content may seem unusual, since in typical lessons the same person prepares the courses (form and content) and also delivers them (i.e., teaches). But the REVIT method produced models of reusable courses which have low cost and can be easily tested with several criteria (efficiency, conformity to a set of standards, etc.). These courses were often “re-localized”, that is adapted to local conditions (language, etc.) outside the borders of the country of their initial production, overcoming cultural and language barriers. It is worth noting that the process of localization of the didactic material was not a trivial one: It went far beyond a simple translation; it was an adaptation in the sense of transformation to the local culture and this was a demanding process.
It is clear, on the other hand, that the REVIT’s learners, living in remote areas, have particular sociocultural characteristics, affecting the success of a distant course, such as practicing traditional professions, and having a strong emotional “connection” to the members of their local community. The local communities were mobilized and affected positively by the REVIT educational venture. The participants’ dispositions seem to be the best assurance for the continuation and the extension of these courses – finally for the idea of lifelong learning. These side effects cannot be easily measured. However, the trainees, almost unanimously, declared that they prefer courses organized in groups and they also expressed clearly their preference for a specific group size: 3-5 persons. They had the feeling of belonging in a small class where it is easy to collaborate with each other, without being divided into sub-groups.
In order to provide some insights on the important aspect of the interactivity of the distance courses, a sample of five e-courses was selected randomly and examined. In particular, overall course design documents and specific lesson plans (usually 8-11 lesson plans per course) were analyzed. The evaluation rubric proposed by Roblyer and Wiencke (2003) in their seminal work titled “Design and Use of a Rubric to Assess and Encourage Interactive Qualities in Distance Courses” was used. It consists of a scale of interaction which ranges from low to high in conjunction with five interactivity dimensions: social (rapport-building), instructional, technological, learner driven, and instructor driven. The results can be summarized as follows (a four-point scale is used where the value 1 is assigned to low interactivity and the value 4 to high interactivity):
Finally, concerning the cost-effectiveness of the e-course design and development, it was enhanced by various parameters that can be summarized as follows:
The above choices not only minimized effort and cost, but also had a positive impact on the re-usability and the ease of localization of the e-courses. Methodologically speaking, after the completion of the analysis phase, the development of the course syllabus, and the ensuing lesson plans, learning activities and resources followed as a computer supported collaborative design process between the partners that had an interest in the specific e-course. Having a European perspective in the selection of the activities and the resources by keeping a balance between local needs, while having a European audience in mind, was a focal requirement for the sake of an easier re-localisation of the e-course.
Note: The analysis framework presented in this paper adheres to a balanced approach between local choices with a pan-European perspective and instructional design theory with practice, while keeping a certain flexibility level in the pathways taken during the analysis in each application area and also utilizing the facilities of the schools.
The REVIT project (beginning 1st January, 2009 and lasting 2 years) was funded by the Education, Audiovisual and Culture Executive Agency (EACEA) as a Lifelong Learning Programme (Key Activity 3: ICT). Tables 1, 2 and 3 included in the D6.2 “Evaluation Report” REVIT deliverable are originally authored by Dr. V. Dagdilelis (University of Macedonia, Greece) who was responsible for the internal evaluation actions of the project. The example concerning the EFL course (Section 3) is from a presentation by Dr. Nikos Sifakis (Hellenic Open University, Greece) and it is included in the REVIT deliverable D2.3 “Guidelines for using the REVIT Distance Learning Framework for Adult Education”.
Allan, J., & Lawless, N. (2003). Stress caused by on-line collaboration in e-learning: A developing model. Education & Training, 45(8), 564–572.
Allen, W. C. (2006). Overview and evolution of the ADDIE training system. Advances in Developing Human Resources, 8(4), 430-441.
Bandura, A. (1978). The self-system in reciprocal determinism. The American Psychologist, 33(4), 344–358.
Berger, C. (1996). Education 626: Educational software design and authoring. Retrieved from http://www.umich.edu/~ed626/define.html
Bishop, A. P., Bruce, B. C., & Jones, M. C. (2009). Community inquiry and informatics: Collaborative learning through ICT. In M. Carroll (Ed.), Learning in communities (pp. 3-5). London: Springer.
Boak, G. (1998). A complete guide to learning contracts. Hampshire, England: Gower Publishing Ltd.
Boulet, G. (n.d). Rapid prototyping: An efficient way to collaboratively design and develop e-learning content. Retrieved from http://www.guyboulet.net/site/docs/Rapid_prototyping.pdf
Bruce, J. (2004). Emergence and e-learning design: From artificial to natural selection. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education ( pp. 1286-1289). Washington, DC, USA: AACE.
Chen, I. (2011). Instructional design methodologies. In I. Association (Ed.), Instructional design: Concepts, methodologies, tools and applications (pp. 80-94). Hershey, PA: Information Science Reference.
Colón, B., Taylor, K. A., & Willis, J. (2000). Constructivist instructional design: Creating a multimedia package for teaching critical qualitative research. The Qualitative Report, 5(1-2). Retrieved from http://www.nova.edu/ssss/QR/QR5-1/colon.html
Cook, S. (2005). Learning needs analysis: Part 1: What is learning needs analysis. Training Journal, 64-68.
Dagdilelis V. (2010). D 6.2 evaluation report. REVIT project deliverable. Retrieved from http://revit.cti.gr
Edwards, R. (1995). Different discourses, discourses of difference: Globalisation, distance education and open learning. Distance Education, 16(2), 241-255.
Forbes, A., While, A., & Ullman, R. (2006). Learning needs analysis: The development of a tool to support the on-going professional development of multiple sclerosis specialist nurses. Nurse Education Today, 26(1), 78-86.
Frossard F., & Frutos M. B. (2011). Bridging the gap in rural life-long learning and digital development. In S. Sotiriou (Ed.), Proceedings of the European Distance and E-learning Network (EDEN) Open Classroom 2011 Conference (pp. 26-31). Hungary: EDEN Publishing.
Gagné, R. M., Briggs, L. J., & Wager, W. W. (1992). Principles of instructional design (4th ed.). Wadsworth Pub.
Gorman, P. (2001). Information needs in primary care: A survey of rural and nonrural primary care physicians. Studies in Health Technology and Informatics, 1, 338-342.
Hickman, L. A. (1990). John’s Dewey’s pragmatic technology. Bloomington, IN: Indiana University Press.
Hodell, C., & American Society for Training and Development. (1997). Basics of instructional systems development. Alexandria, Va: American Society for Training and Development.
Horton, W. (2003). E-learning tools and technologies: A consumer’s guide for trainers, teachers, educators, and instructional designers. Hoboken, NJ: John Wiley & Sons.
Jarvis, P., (2004). Adult education and lifelong learning: Theory and practice (3rd ed.). London: Routledge Falmer.
Jones, P., & Davis, R. (2011). Instructional design methods integrating instructional technology. In I. Association (Ed.), Instructional design: Concepts, methodologies, tools and applications (pp. 101-113). Hershey, PA: Information Science Reference.
Knowles, M. M. S. (1970). The modern practice of adult education (Vol. 41). New York: Association Press.
Knowles, M. S. (1990). The adult learner: A neglected species. Houston, TX: Gulf Publishing.
Kruse, K. (n.d.). Introduction to instructional design and the ADDIE model. Retrieved from http://www.transformativedesigns.com/id_systems.html
MacCormack, A., Kemerer, C.F., Cusumano, M., & Crandall, B. (2003). Trade-offs between productivity and quality in selecting software development practices. Software(IEEE), 20(5), 78- 85.
Marcinkiewicz, H. R. (2011). Planning for integrating technology. In I. Association (Ed.), Instructional design: Concepts, methodologies, tools and applications (pp. 207-218). Hershey, PA: Information Science Reference.
McDermott, F. M. (1982). Try brainstorming – a quick route to job analysis. Training/HRD, 19(3), 38-40.
McNair, S., & Quintero-Re, L. (2008). CONFINTEA VI: United Kingdom national report on the development and state of the art of adult learning and education (ALE). NIACE, Leicester. Retrieved from http://shop.niace.org.uk/media/catalog/product/C/O/CONFINTEA-VI-UK-Country-Report.pdf
Miller, B. (1993). Rural distress and survival: The school and the importance of “community.” Journal of Research in Rural Education, 9(2), 84-103.
Mahoney, S. E., & Harvell, T. J. (2000). Role of contracts in enhancing community building in web courses. Educational Technology & Society, 3, 3.
Moore, M., & Kearsley, G. (2005). Distance education: A systems view (2nd ed.). USA:Wadsworth.
Patterson, A. (2009). A needs analysis for information literacy provision for research: A case study in University College Dublin. Journal of Information Literacy, 3(1), 5-18.
Parchoma G.(2003). Learner-centered instructional design and development: Two examples of success. Journal of Distance Education, 18(2), 35-60.
Rodriguez, S. (1988). Needs assessment and analysis: Tools for change. Journal of Instructional Development, 11(1), 23-28.
Rogers, A. (1996). Teaching adults (2nd ed.). Buckingham: Open University Press.
Rogers, A. (2005). Non-formal education: Flexible schooling or participatory education? USA: Springer.Science+Business Media.
Rosemary, S., Caflarella, B., & Barnett, G. (2006). Characteristics of adult learners and foundations of experiential learning. New Directions for Adult and Continuing Education, 1994(62), 29-42.
Rothwell ,W. J., & Kazanas, H. C. (2008). Mastering the instructional design process: A systematic approach (4th ed.). Pfeiffer.
Salmon, G. (2002). E-tivities: The key to active online learning. USA: Routledge.
Salomon, G. (1981). Communication and education, social and psychological interactions, people & communication. Beverly Hills, CA: Sage.
Schön, D.,A (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.
Schwen, T. (1973). Learner analysis: Some process and content concerns. AV Communication Review, 21(1), 44-72.
Shelton, K., & Saltsman, G. (2011). Applying the ADDIE Model to online instruction. In I. Association (Ed.), Instructional design: Concepts, methodologies, tools and applications (pp. 566-582). Hershey, PA: Information Science Reference.
Sims, R.C. (2001). From art to alchemy: Achieving success with online learning. Retrieved from http://it.coe.uga.edu/itforum/paper55/paper55.htm
Soloway, E., Guzdial, M., & Hay, K. E. (1994). Learner-centered design: The challenge for HCI in the 21st century. Interactions, 1(2), 36-48.
Stolzer, A., Halford, C., & Goglia, J. (2011). Implementing safety management systems in aviation. Surrey, England: Ashgate.
Tessmer, M. (1990). Environment analysis: A neglected stage of instructional design. Educational Technology Research and Development, 38(1), 55-64.
Trolley, E. (2006). Lies About learning. Baltimore, Maryland: ASTD.
Tudge, J. R. H., & Scrimsher, S. (2003). Lev S. Vygotsky on education: A cultural-historical, interpersonal, and individual approach to development. In Educational psychology: A century of contributions (pp. 207-228).
Tyler-Smith, K. (2006). Early attrition among first-time eLearners: A review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking eLearning programmes. MERLOT Journal of Online Learning and Teaching, 2(2), 73–85.
Wang, G., & Wilcox, D. (2006). Training evaluation: Knowing more than is practiced. Advances in Developing Human Resources, 8(4), 528-539.
Wang, J., Wu, E., & Wong, P.C. (2004). Strategies for reducing dropout from distance education courses. In J. Nall & R. Robson (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 1705-1710). Chesapeake, VA: AACE.
Valcke, M., & Leeuw, F. (1998). Evaluating digital distance learning: Studies, practices and recommendations for evaluating distance learning activities and programs. Retrieved from http://users.ugent.be/~mvalcke/CV/worldbank.pdf
Visscher-Voerman, I., & Gustafson, K. L. (2004). Paradigms in the theory and practice of education and training design. Educational Technology Research and Development, 52(2), 69-91.
Sample of the Diagnostic Questionnaires
The online survey questionnaire (used in the Fornetka application area) was comprised of the following questions:
• Demographics (age, sex, field of working)
• Educational context (Readiness to participate in distance E-course in Primary School in Fornetka, Areas of interest concerning the distance E-course, I have the following ICT skills)
• ICT facilities (I have the following Internet connection at my home)
The questionnaire handed out to the learners in Kula (BG) in a written format contained the following fields:
• Demographics (Sex, age, occupation)
• Prior knowledge (What are your computer skills?)
• Willingness/Readiness (Would you be interested to attend e-courses at High school “Vasil Levski” in Kula?, If your answer to the previous question is “no”, would you attend e-courses from your home/work?)
• Interests/aspirations (What ICT skills are you interested in developing?, Which of the following topics of potential e-courses interests you most?, Would you be willing to participate in an e-course with fellow trainees from different age groups?)
Methodology Description in a Country Level
This section presents the needs elicitation methodology (method, instruments, protocols, participants/ respondents) in a country level.
A. Ullava, Finland
Method: online questionnaire
Protocol: The questionnaire was send via the teachers, at first to all the parents of the pupils in Veikko Vionoja and Rahkonen school. It was not possible to send the questionnaire to all the adults in the application area and this sample was chosen to present the needs of adults in Ullava. Teachers had in their disposal the email addresses of the parents. The data was collected electronically by SPPS/Mr. Interview software.
Instrument: online survey tool with 12 questions, 10 closed, multiple choice questions and 2 open-ended questions at the end of the questionnaire so that participants could elaborate about their interests and needs towards distance E-learning and videoconference based learning. The first 7 questions of the questionnaire are identical with the ones in the questionnaire that was administered in Fornetka, Poland (see previous section) i.e. demographics (3 questions), educational context (3 question) and ICT facilities (1 question).
Respondents: 27 persons completed the questionnaire successfully.
B. Fornetka, Poland
Method/Protocol: Physical meetings and discussions with the community of Fornetka, including the local people, the authorities and the local co-ordinator. The aim of the first meeting (semi-public) was to inform the community about the project and anticipate their interest to participate as the application area in Poland. The second meeting was organized with the perspective learners in order to investigate their needs. The third meeting was public with invited representatives of local authorities. A fourth public meeting was held at Fornetka, in order to finalize the list of the educational needs and get the commitment of the people that would participate in each course.
Instrument: Between the second and the third meeting an online questionnaire was administered to the perspective learners (see Αppendix A).
Respondents: 40 persons completed the questionnaire successfully.
C. Kula, Bulgaria
Method/protocol: The local project team together with the local moderator in Kula after having conducted questioning (using the online questionnaire) and interviewing (focus groups) during their visits in the area, identified the educational needs of the potential beneficiaries. A follow-up public meeting was scheduled for getting commitment from the participants in the e-courses. From the questioning, discussions and meetings conducted the educational needs were finally specified.
Instrument: A short questionnaire with 10 questions was disseminated among potential local beneficiaries (see Appendix A).
Respondents: 21 questionnaires were successfully filled in and returned to the local moderator in Kula.
D. Koufonissi and Ios islands, Greece
Method/protocol: a two-staged method was adopted, that is, Stage 1) interviews with local people and reports from local teachers, Stage 2) observation and dialogue methods (visits to both islands, organization of infodays and parallel collection of data needed). Also, contacts with the school directors (early in Stage 1) were established.
Instrument: Attending members of the local community in each island were requested to complete a questionnaire in order to determine the learners’ profiles. The learner profile questionnaire contained personal profiling information, educational and professional background, computer literacy information and further learning interests.
Respondents/participants: 129 preferences on a list of potential REVIT e-courses (but this number does not indicate the exact number of participants, since several persons expressed interest in more than one course).
E. Palaichori, Cyprus
Method/protocol: Members of the project team visited the community of Palaichori four times, in order to specify the educational needs of the locals. The aim of the first meeting was to inform the community about the project and anticipate their interest to participate as the application area in Cyprus. During the second and the third visits, meetings with perspective learners were organized and the methods of discussion and brainstorming were used in order to elicit their educational needs. Also, the team used the school infrastructure and equipment (internet connection, projector, speakers) in order to demonstrate the main components of the REVIT distance learning system, its tools and services. In the last meeting in Palaichori, the list of the educational needs was finalized, consensus was reached and the project team gained the learners’ commitment in the courses.
Instrument: A brainstorming protocol was used with notes taking from a member of the research team, in parallel.
Participants: 40 preferences on a list of potential REVIT e-courses (but this number does not indicate the exact number of participants, since several persons expressed interest in more than one course).
4 Sharable Content Object Reference Model (SCORM)
5 Learning object Metadata (LOM)
6 eLearning XHTML editor, http://exelearning.org/
7 Moodle CMS, https://moodle.org/
8 OpenSimulator, http://opensimulator.org