Research Areas in Distance Education: A Delphi Study

This study had three purposes: Firstly, to develop a categorization of research areas in distance education; secondly, to identify the most important research areas in distance education; and thirdly, to identify the most neglected research areas in distance education. Based on a literature review and a Delphi study, three broad levels or perspectives with 15 research areas were derived to organize the body of knowledge in distance education. Prospective researchers can use the results to identify gaps and priority areas and to explore potential research directions.


Résumé de l'article
This study had three purposes: Firstly, to develop a categorization of research areas in distance education; secondly, to identify the most important research areas in distance education; and thirdly, to identify the most neglected research areas in distance education. Based on a literature review and a Delphi study, three broad levels or perspectives with 15 research areas were derived to organize the body of knowledge in distance education. Prospective researchers can use the results to identify gaps and priority areas and to explore potential research directions.

Introduction
Research on distance education has been subject to consistent critique (Bernard et al., 2004;Moore, 1985;Perraton, 2000;Saba, 2000). It has even been characterized as "atheoretical and predominantly descriptive" (Perraton, 2000, p. 1). Research questions should be posed within a theoretical framework and embedded in a holistic structure of research areas within a discipline. However, in the field of distance education, a validated meta-structure of research topics is lacking, i.e., a map of research areas that would help to organize the body of knowledge in the field. Furthermore, the structure of a research discipline forms the foundation for identifying gaps and priority areas for researchers. With regard to distance education, Mishra (1998) makes a plea for "a comprehensive and cohesive structure internationally to provide a strong foundation to the discipline" (p. 281).
Many reviews of the distance education literature have been conducted in which authors developed categorization schemes that were mapped onto publications under review. By and large, there is general agreement on the basic research topics in distance education, such as student support, instructional design, educational technologies, or interactivity through media. However, the various attempts to describe the broad and interdisciplinary field of distance education show a disparate picture. 3 media and technology; (5) learners and learning; (6) institutional policy and management; (7) economics; (8) evaluation and programme evaluation; and (9) staff development. Panda (1992) furthermore provided a conceptual framework for distance education research based on the model of systems philosophy, distinguishing between input (e.g. courses, students, staff development), process (e.g. two-way communication, student support services, evaluation) and output variables (e.g., student achievement and satisfaction, effectiveness and efficiency of the entire system).
A similar study was conducted by Koble and Bunker (1997) for The American Journal of Distance Education (1987Education ( -1995 with the following classification: (1) theory, policy, and development; (2) media and delivery systems (effectiveness, evaluation, methods); (3) institution, staff, and management; (4) student psychology, motivation, and characteristics; (5) faculty participation and instructional process; (6) course design and curriculum development; and (7) student administration and support.
In the same year, Mishra (1997) published an analysis of periodical literature in distance education (1991)(1992)(1993)(1994)(1995)(1996) with seven major groups and 34 sub-groups of research topics (p. 49). The major groups in this classification are as follows: (1) distance education in perspective (including open learning, growth and development, equity and access); (2) students and their learning (including learner characteristics, drop out, cognition, student support, interaction, and feedback); (3) learning materials and related issues (including design and development of study materials, learning from media); (4) technology issues; (5) management issues (including planning, economics, evaluation, and quality assurance); (6) distance education theory, research and training/professional development; and (7) distance education in practice (including distance education institutions and teacher education). Berge and Mrozowski (2001, p. 7) adopted Sherry"s ten issues in distance education (Sherry, 1996) for their review of research in distance education between 1990 and 1999, with the following outcome: (1) redefining roles of key participants with focus on training (teachers, students, and site facilitators); (2) technology selection and adaptation; (3) design issues; (4) strategies to increase interactivity and active learning; (5) learner characteristics; (6) learner support; (7) operational issues (planning, administration, management, and economics); (8) policy and management; (9) equity and accessibility; and (10) cost/benefit trade-offs. 4 Rourke and Szabo (2002) applied nine categories for a content analysis of the Canadian Journal of Distance Education (1986)(1987)(1988)(1989)(1990)(1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001): (1) administration: budgeting, change, inter-institutional cooperation, staffing; (2) evaluation: judgment of the worth, value, success of a course, program, or courseware; (3) foundations: semantics, definitions, history, models; (4) instructional design: course, program, or product design, development, and delivery; (5) instructor characteristics: satisfaction, motivation, attitude, perceptions, teaching style, gender, demographics; (6) international perspectives: practices, procedures, role of distance education in specific countries; (7) learner characteristics: satisfaction, motivation, attitude, perceptions, learning styles, cognitive styles, attrition, gender, demographics; (8) technology and media: educational radio, educational TV, video, audio, computer conferencing; and (9) trends: discussion of the general direction or inclination of issues and topics in the field. Lee, Driscoll, and Nelson (2004, p. 227) developed a system consisting of six categories for grouping distance education literature: (1) design-related topics: needs assessment, course scheduling, course design, instructional strategy development, course material design, and visual design; (2) development-related topics: course support system and material development, web-based learning management system building, online tools development, and online testing system development; (3) management-related topics: learning resource management, troubleshooting, attrition rate, faculty and staff support, learner support, and technical support; (4) evaluation-related topics: program quality control, assessment of learning outcomes, benefits and cost analysis, return on investment, evaluation of supporting system; (5) institutional and operational-related topics: administration, academic affairs, accreditation, certification, policy, payment, and budgeting; and (6) theory and research-related topics: distance education theory building, review of literature, introduction to new research methods, culture and gender issues, learning style, history of distance education, and copyright law.
In order to identify distance education research priorities for Australia, Jegede (1994) developed a questionnaire that was validated by a group of 36 Australian distance education researchers and practitioners. The questionnaire contained 22 broad groupings of research areas: (1) theory and philosophy; (2) learner characteristics; (3) equity and access; (4) design and development of study materials; (5) instructional and communications technology; (6) tele-teaching and learning; (7) management and planning; (8) student support services; (9) development of students" study skills; (10) systems for the provision of feedback to students; (11) interactive multimedia; (12) discipline based context; In contrast to the listings of research topics described above, this paper reports on an attempt to structure the broad and interdisciplinary research areas in the field of distance education based on a systematic analysis of expert responses in a Delphi study.

Methodology
The Delphi technique was selected to develop a consensus among a group of experts on common areas that are or should be covered in distance education research: "It is important that participants be, in some sense, "experts" for valid results to be reached, and there should be sufficient participants that the extreme views of a few do not influence the results for the group disproportionately" (Charlton, 2004, p. 245). The essential element in the Delphi process is anonymity of participants when giving their opinion. The Delphi, then, alleviates problems that could be caused by domination of the group by a few prestigious or powerful individuals. According to Isaac and Michael (1995), the Delphi method of group interaction avoids the following disadvantages of face-to-face discussions: the bandwagon tendency, the vulnerability to manipulation, and the reticence on the part of individuals to change their minds in front of others.
A careful process was followed that resulted in three criteria for selecting experts, from which the eventual expert panel emerged. The three criteria were that the expert has achieved the following: a significant contribution to the distance education literature; at least 10 years of professional experience in the field of distance education; and a willingness to contribute.
All current members of the editorial boards of four major distance education journals were invited to participate. Furthermore, the invitation was sent to current and former faculty members in a master"s of distance education program in which the author of this paper is also teaching, among them Tony Bates, Börje Holmberg, Som Naidu, and Otto Peters. A personal interview was carried out with Professor Peters at the FernUniversität in Hagen.
The final expert panel comprised 25 individuals from 11 countries (Australia, Brazil, Canada, China, Fiji, Germany, Ireland, New Zealand, South Africa, UK, and USA). Nineteen of the 25 experts participated in both rounds of the Delphi study. The required time to collect the data was four months from September to December 2008.
The respondents (N = 19) reported an average of 27 years of professional experience in distance education, ranging from 10 to 40 years. Twelve of them (63 %) sit on editorial boards, and two are editors of prominent distance education journals. Six (32 %) of the experts were women and 13 (68 %) were men. The panel consisted of eight directors, ten professors, four senior researchers, one instructional designer, and one manager. Five panellists reported dual titles of professor and director or researcher and manager resulting in 24 professional titles.

6
In the first round of the Delphi study, the experts were asked via e-mail to list at least 10 issues or research areas that are most important for the development of theory and practice in distance education. Based on a qualitative text analysis of the responses using ATLAS.ti and a literature review on the topic, a categorization of research areas was developed. In order to identify priority areas, the second round of online questionnaire required participants to rate the research areas on a scale of importance ranging from 1 to 10, with 1 labeled "very low importance" and 10 labeled "very high importance." They were also asked to identify the most neglected research areas. This questionnaire was completed by 19 of 25 experts (76 % response rate).
The study is limited to English-speaking experts in distance education with access to the Internet. Therefore, the results may not be generalizable to certain non-English speaking settings or where the technical infrastructure does not allow access to the Internet. Due to the small group size complex statistical analysis cannot be done. The reported results are therefore descriptive and the median is chosen as the measure of central tendency to depict experts" ratings.

Results
Three research questions were addressed in this study.

Question 1: What are the research areas in distance education, and how can they be classified?
This question was addressed in round one of the Delphi study. The experts were asked via e-mail to list 10 research areas in distance education (DE). Table 1 shows a list of the most common research themes and their frequency when analysing the responses (N = 25).

Globalization of education and cross-cultural aspects
Aspects that refer to the global external environment and drivers, the development of the global distance education market, teaching and learning in mediated global environments and its implications for professional development.

Distance teaching systems and institutions
Distance education delivery systems, the role of institutional partnerships in developing transnational programmes, and 8 the impact of ICT on the convergence of conventional education and distance education institutions (hybrid or mixed-mode).

Theories and models
Theoretical frameworks for and foundations of distance education, e. g. the theoretical basis of instructional models, knowledge construction, interaction between learners, or the impact of social constructivism learning theories on distance education practice.

Research methods in distance education and knowledge transfer
Methodological considerations, the impact of distance education research and writing on practice and the role of professional associations in improving practice. Literature reviews and works on the history of distance education are subsumed under this area.

Management and organization
Strategies, administration, and organizational infrastructures and frameworks for the development, implementation, and sustainable delivery of distance education programmes. What is required for successful leadership in distance education? Distance education and policies relating to continuing education and lifelong learning and the impact of online learning on institutional policies, as well as legal issues (copyright and intellectual property).

Costs and benefits
Aspects that refer to financial management, costing, pricing, and business models in distance education. Efficiency: What is the return on investment or impact of distance education programmes? What is the impact of ICT on the costing models and the scalability of distance education delivery? How to provide cost effective but meaningful learner support.

Educational technology
New trends in educational technology for distance education (e.g. Web 2.0 applications or mobile learning) and the benefits and challenges of using OERs, media selection (e.g. synchronous vs. asynchronous media), technical infrastructure and equipment for online learning environments and their opportunities for teaching and learning.

Innovation and change
Issues that refer to educational innovation with new media and measures to support and facilitate change in institutions (e.g. incentive systems for faculty, aspects referring to staff workloads, promotion and tenure). 9

Professional development and faculty support
Professional development and faculty support services as a prerequisite for innovation and change. What are the competencies of online teachers and how can they be developed?

Learner support services
The infrastructure for and the organization of learner support systems (from information and counselling for prospective students through library services and technical support to career services and alumni networks).

Quality assurance
Issues that refer to accreditation and quality standards in distance education. The impact of quality assurance and high quality learner support on enrolments and dropout/retention, as well as reputation and acceptance of distance education as a valid form of educational provision.

Instructional design
Issues that refer to the stages of the instructional design process for curriculum and course development. Special emphasis is placed on pedagogical approaches for tutoring online (scaffolding), the design of (culturally appropriate) study material, opportunities of new developments in educational technology for teaching and learning (e.g. Web 2.0 applications and mobile devices), as well as assessment practices in distance education.

Interaction and communication in learning communities
Closely related to instructional design considerations is course design that fosters (online) articulation, interaction, reflection, and collaboration throughout the learning and teaching process. Special areas include the development of online communities and gender differences and crosscultural aspects in online communication.

Learner characteristics
The aims and goals of adult learners, the socio-economic background of distance education students, their different learning styles, critical thinking dispositions, and special needs. How do students learn online (learner behavior patterns, learning styles) and what competencies are needed for distance learning (e.g. "digital literacy")?
A clear and non-ambiguous separation of research areas into categories is not easy in all cases. Some areas are considered on different levels. Cross-sectional fields are concerned with issues that refer to quality assurance and evaluation, educational technologies, and cross-cultural aspects.
If the focus is placed on the evaluation or accreditation of whole institutions or programmes or, for example, the research is about the development of a general, institution-wide instrument to measure learner satisfaction, the issues are classified on the meso level: management, organization, and technology. However, if emphasis is placed on the evaluation of single courses, learning groups, or courseware (e.g., print-based study materials or multimedia applications), the topics are subsumed under the micro level: teaching and learning in distance education (evaluation as a step in the instructional design process).
The same applies to cross-cultural issues between the macro, meso, and micro levels. Crosscultural aspects have to be considered in international cooperation in the global education market (macro level) and in the planning, implementation, and management of transnational programmes to address the needs of an internationally dispersed student body, or "target group" to use the terms of management and marketing (meso level). Intercultural communication plays an important role in classes with mixed cohorts of students from all over the world. Of course this also has implications for competencies required by distance educators on the meso level (professional development and faculty support).
Finally, the application of educational technologies has implications on all three dimensions.
Issues that refer to access to educational technologies and infrastructure have to be considered on the global macro level. The introduction of new media for learning and teaching in institutions has a strong impact on educational management on the meso level, e.g., issues that refer to the organizational support and infrastructure for educational technologies, their costs, quality assurance, implications for organizational change, and the resulting need for professional development. Research that emphasizes the pedagogical opportunities that new media afford for teaching and learning falls into categories on the micro level (e.g., instructional design and mediated interaction and communication).

Question 2: What are the most important distance education research areas?
In the second round of the Delphi study the experts were asked to rate the 15 research areas described above on a scale of importance ranging from 1 to 10, with 1 labeled "very low importance" and 10 labeled "very high importance." The distribution of the experts" ratings on the importance of research areas in distance education is depicted in Figure 1. The box plot shows the median, which is less sensitive to extreme ratings by the small group of experts (N = 19) and the corresponding inter-quartile range (IQR). The IQR is defined as the difference between the twenty-fifth and the seventy-fifth quartiles, i.e., the tinted boxes in Figure 1 show the middle 50 % of scores (Field, 2005, p. 75).  Figure. 1: Distribution of experts" ratings on the importance of research areas in distance education. * scale ranges from 1=very low importance to 10=very high importance ** 1=Access, equity and ethics; 2=Globalisation of education and cross-cultural aspects; 3=Distance teaching systems and institutions; 4=Theories and models; 5=Research methods in distance education and knowledge transfer; 6=Management and organization; 7=Costs and benefits; 8=Educational technology; 9=Innovation and change; 10=Professional development and faculty support; 11=Learner support services; 12=Quality assurance; 13=Instructional design; 14=Interaction and communication in learning communities; 15=Learner characteristics A high level of agreement on the importance of a research area can be observed for management (6), innovation and change issues (9), professional development and faculty support (10), and instructional design (13); whereas, the widest range of ratings can be reported for issues that refer to learner characteristics (15).

Tabulation of Responses Regarding the Importance of Research Areas in Distance Education
Scale of importance** Researc h area* 13 * 1=Access, equity and ethics; 2=Globalisation of education and cross-cultural aspects; 3=Distance teaching systems and institutions; 4=Theories and models; 5=Research methods in distance education and knowledge transfer; 6=Management and organization; 7=Costs and benefits; 8=Educational technology; 9=Innovation and change; 10=Professional development and faculty support; 11=Learner support services; 12=Quality assurance; 13=Instructional design; 14=Interaction and communication in learning communities; 15=Learner characteristics ** scale ranges from 1=very low importance to 10=very high importance Table 4 ranks the importance of the research areas on the three levels. Due to the small group size (N = 19), the measure of central tendency chosen is the median. On the macro level (distance education systems and theories), the experts claim that the role of culture and cultural differences in global distance learning programmes and cooperation should receive much more attention. This includes the impact of cultural differences on leadership and The research areas identified in the present study as requiring a concentration of research activities seem to agree in the main with what Jegede (1994) recommended for Australia. Issues that refer to access and equity, the role of distance education in developing areas, student support services, evaluation, educational technologies, and instructional design are still important. The field has been dominated by research in the areas of technology, increased interactivity through new media, and instructional design issues (Berge & Mrozowski, 2001). However, there are also new fields or key areas that are, or should be, on top of the research agenda.
Many institutions are moving into the global education market to reach new target groups using online learning to "export" their knowledge. Therefore, globalization of education, cross-cultural aspects, and access, equity, and ethics are research areas that should receive more attention. This 15 is supported by the fact that the whole first section in the new International Handbook of Distance Education is devoted to "Diversity in Distance Education" (Evans, Haughey, & Murphy, 2008).
The results suggest that there is a shift from technology-centered research to areas that focus on management and change in distance education institutions. The emergence of online distance learning highlights a pressing need for educational institutions to embrace innovation and change. It can be concluded that all aspects related to educational management are growing in importance (strategy and leadership, change management, quality assurance, costs etc.). In order to guide practice we should not rely on under-informed trial and error, but on sound research on the effectiveness of managerial interventions for education innovation. Distance educators ranked the areas of innovation and change and quality assurance as the research areas to be given priority attention on the meso level. There was a high level of agreement on the importance of faculty support and professional development of those who are involved in the process of education innovation. This process should be driven by people and pedagogical concepts that apply new technological developments in a meaningful way for the purpose of adding value to teaching and learning. Technology is not an end in itself. Interestingly, the area of educational technology emerged on only the third rank.
As revealed in Tables 3 and 4, interaction and communication in learning communities was ranked the highest on the micro level. Future research emphasis should be on the pedagogical impact of educational technologies on teaching and learning and on the capacity of social software, Web 2.0 applications, and mobile devices to foster social interaction and to support flexible learning.
Based on the classification of research areas and the priority areas identified in this study, it would be interesting to carry out a new large-scale literature review to investigate research trends and gaps, methods, and authorship patterns in distance education research.