Review of Distance Education Research (2000 to 2008): Analysis of Research Areas, Methods, and Authorship Patterns

This paper presents a review of distance education literature to describe the status thereof and to identify gaps and priority areas in distance education research based on a validated classification of research areas. The articles (N = 695) published in five prominent distance education journals between 2000 and 2008 were reviewed for this study. The conclusion is that distance education research is strongly dominated by issues related to instructional design and individual learning processes; whereas, other important areas (e.g., innovation and change management or intercultural aspects of distance learning) are dreadfully neglected. There is a significant trend towards collaborative research and more qualitative studies. Over 80% of all articles originate from only five countries.


Résumé de l'article
This paper presents a review of distance education literature to describe the status thereof and to identify gaps and priority areas in distance education research based on a validated classification of research areas. The articles (N = 695) published in five prominent distance education journals between 2000 and 2008 were reviewed for this study. The conclusion is that distance education research is strongly dominated by issues related to instructional design and individual learning processes; whereas, other important areas (e.g., innovation and change management or intercultural aspects of distance learning) are dreadfully neglected. There is a significant trend towards collaborative research and more qualitative studies. Over 80% of all articles originate from only five countries. (Mishra, 1998, p. 281). Based on a validated classification of research areas in distance education through the systematic analysis of expert responses in a Delphi study (Zawacki-Richter, 2009), we conducted a literature review of previous distance education research published in five prominent distance education journals between 2000 and 2008 (N = 695 articles).
The aim of this paper is to address questions in the following three areas: • Issues in distance education research: What are the main research areas in distance education and how have they changed between 2000 and 2008? What are the most common research areas and where are there gaps in distance education research? • Research methods: Getting beyond the question of "any significant differences?" experts advocated more qualitative approaches in distance education research (Minnes, 1985;Saba, 2000). But can we observe a significant trend towards the application of more qualitative methods and mixed-methods designs (triangulation)? Is there an association between research methods and gender? And do distance education journals prefer to publish qualitative or quantitative studies? • Publication and authorship patterns: Researchers are more and more often involved in complex international collaborative projects. Is there a significant trend towards more collaboration among researchers in distance education? Who are the leading contributors of research papers and where do they come from? Do distance education journals tend to publish papers from their country of origin?
Based on the review of research areas and trends, the results can be used to explore the body of knowledge in distance education and to identify priority areas for future research projects.

Selection of Journals and Articles
Five journals were reviewed for this study: Open Learning (OL), Distance Education (DE), the American Journal of Distance Education (AJDE), the Journal of Distance Education (JDE), and the International Review of Research in Open and Distance Learning (IRRODL). They were selected because of their reputations as the most prominent and recognized journals in the field of distance education. With the exception of IRRODL, which was released for the first time in 2000, the journals have been used as data sources in previous studies (cf. Berge & Mrozowski, 2001;Lee, Driscoll, & Nelson, 2004;Koble & Bunker, 1997;Mishra, 1997;Rourke & Szabo, 2002;Scriven, 1991). Furthermore, the five journals were selected because of their wide scope of distance education research in contrast to more specialized journals such as the Online Journal of Distance Learning Administration or the International Journal of Distance Education Technologies.

Classification of Research Areas
In order to prepare this review, an international Delphi study was carried out in 2008 to develop a classification system for research areas in distance education. The Delphi technique was selected to develop a consensus among a group of experts on common areas that are or should be covered in distance education research. The essential element in the Delphi process is anonymity of participants when giving their opinion. The Delphi method, then, alleviates problems that could be caused by domination of the group by a few prestigious or powerful individuals (Charlton, 2004). According to Isaac and Michael (1995), the Delphi method of group interaction avoids the following disadvantages of face-to-face discussions: the bandwagon tendency, the vulnerability to manipulation, and the reticence on the part of individuals to change their minds in front of others. 1. Access, equity, and ethics: The democratization of access to distance education afforded by new media and by finding ways to deliver high-quality education to those who have limited resources and poor infrastructure; issues that refer to the (sustainable) provision of distance education in developing areas. What is the impact of distance education (e.g., via mobile learning) on narrowing the digital divide and what is the role of ICT (information and communication technologies) and/or OER (open educational resources) in terms of access to education? 2. Globalization of education and cross-cultural aspects: Aspects that refer to the global external environment and drivers, the development of the global distance education market, teaching and learning in mediated global environments, and the implications for professional development. 3. Distance teaching systems and institutions: Distance education delivery systems, the role of institutional partnerships in developing transnational programmes, and the impact of ICT on the convergence of conventional education and distance education institutions (hybrid or mixed-mode). 4. Theories and models: Theoretical frameworks for and foundations of distance education, e.g., the theoretical basis of instructional models, knowledge construction, interaction between learners, or the impact of social constructivism learning theories on distance education practice. 5. Research methods in distance education and knowledge transfer: Methodological considerations, the impact of distance education research and writing on practice, and the role of professional associations in improving practice. Literature reviews and works on the history of distance education are also subsumed within this area.
Meso level: Management, organization, and technology.
6. Management and organization: Strategies, administration, and organizational infrastructures and frameworks for the development, implementation, and sustainable delivery of distance education programmes. What is required for successful leadership in distance education? Distance education and policies relating to continuing education, lifelong learning, and the impact of online learning on institutional policies, as well as legal issues (copyright and intellectual property). 7. Costs and benefits: Aspects that refer to financial management, costing, pricing, and business models in distance education. Efficiency: What is the return on investment or impact of distance education programmes? What is the impact of ICT on the costing models and the scalability of distance education delivery? How can cost effective but meaningful learner support be provided? 8. Educational technology: New trends in educational technology for distance education (e.g., Web 2.0 applications or mobile learning) and the benefits and challenges of using OERs, media selection (e.g., synchronous vs. asynchronous media), technical infrastructure and equipment for online learning environments, and their opportunities for teaching and learning. 25 9. Innovation and change: Issues that refer to educational innovation with new media and measures to support and facilitate change in institutions (e.g., incentive systems for faculty, aspects referring to staff workloads, promotion, and tenure). 10. Professional development and faculty support: Professional development and faculty support services as a prerequisite for innovation and change. What are the competencies of online teachers and how can they be developed? 11. Learner support services: The infrastructure for and organisation of learner support systems (from information and counselling for prospective students about library services and technical support to career services and alumni networks). 12. Quality assurance: Issues that refer to accreditation and quality standards in distance education. The impact of quality assurance and high quality learner support on enrolments and drop-out/retention, as well as reputation and acceptance of distance education as a valid form of educational provision.

Micro level: Teaching and learning in distance education.
13. Instructional design: Issues that refer to the stages of the instructional design process for curriculum and course development. Special emphasis is placed on pedagogical approaches for tutoring online (scaffolding), the design of (culturally appropriate) study material, opportunities provided by new developments in educational technology for teaching and learning (e.g. Web 2.0 applications and mobile devices), as well as assessment practices in distance education. 14. Interaction and communication in learning communities: Closely related to instructional design considerations is course design that fosters (online) articulation, interaction, reflection, and collaboration throughout the learning and teaching process. Special areas include the development of online communities, gender differences, and cross-cultural aspects in online communication. 15. Learner characteristics: The aims and goals of adult learners, the socio-economic background of distance education students, their different learning styles, critical thinking dispositions, and special needs. How do students learn online (learner behavior patterns, learning styles) and what competencies are needed for distance learning (e.g., digital literacy)?
All 695 articles published between 2000 and 2008 in the five journals (OL, DE, AJDE, JDE, and IRRODL) were coded according to this classification scheme. Table 2 provides an overview of sample studies. They are representative of a certain research area and research method.

Classification of Research Methods
The majority of distance education research can be classified broadly as quantitative, qualitative, or of mixed design (triangulation), which employs both quantitative and qualitative approaches. For this review the research methods were classified as quantitative, qualitative, triangulation, or other (cf. Grant, Ward, & Rong, 1987).

26
Articles that used statistical analysis were classified as quantitative, from simple methods such as chi-square analysis to multivariate techniques. Qualitative studies were data-based articles that did not quantify data beyond frequency counts. Included in this category were case studies, interpretive ethnographies, grounded theory, and phenomenological studies. Other articles were usually descriptive, not data-based, theoretical papers. 27

Inter-Rater Reliability
Thirty articles were randomly selected to evaluate the coding decisions of the three coders (A, B and C) to determine inter-rater reliability using Cohen's kappa (κ) (Cohen, 1960), which is a coefficient for the degree of consistency among raters based on the number of codings in the coding scheme (Neumann, 2007, p. 326). Kappa values of .40 to .60 are characterized as fair, .60 to .75 as good, and over .75 as excellent (Fleiss, 1981;Bakeman & Gottman, 1997).
Coding consistency for the classification of research methods between rater A and B was κ = .855; between rater A and C, it was κ = .855; and between rater B and C, it was κ = .930 (median = .855). For the classification of research areas, coding consistency between rater A and B was κ = .672; between rater A and C, it was κ = .675; and between rater B and C, it was κ = .675 (median = .675). Therefore, inter-rater reliability can be considered as excellent for the coding of methods and as good for the coding of research areas.
The main source of coding discrepancies is the considerable overlap between some research areas so that a unambiguous classification of research areas was not easy in all cases. Some areas are investigated on different levels. Cross-sectional fields are those concerned with issues that refer to quality assurance and evaluation, educational technologies, and cross-cultural aspects (Zawacki-Richter. 2009). For example, cross-cultural aspects have to be considered in international cooperation in the global education market (macro level) and in the planning, implementation, and management of transnational programmes (meso level). Cross-cultural aspects also have implications for competencies required by distance educators (professional development and faculty support − meso level). Intercultural communication plays an important role in classes with mixed cohorts of students from all over the world (micro level).

Trend Analysis
In order to identify significant trends between the years 2000 to 2008 in the development of research areas, in the application of research methods, and with regard to cooperation among researchers (single vs. multiple-author papers), the frequencies of research areas, methods, and number of authors in the 695 articles under review were ranked for each year and correlated with the years. Significant rank correlations on the 5% confidence level are interpreted as significant linear positive or negative trends. Due to the small number of years (N = 9), Spearman's rho (ρ) was chosen as the non-parametric correlation coefficient.  With only 13 articles in research area 5 (research methods in distance education), the number of articles increased slightly, but significantly, at a low level (Spearman's ρ = .69, p < .05). Research area 6 (management and organisation) is strongly correlated with area 7 (costs and benefits), ρ = .88, p < .01., which might indicate that these two categories could be merged. Table 4 shows several noticeable peaks in certain research areas that were due to special issues on the topic, e.g., the special issue on "The Hybridzation of Higher Education: Cross National

Analysis of Research Methods
Distance education, in particular, and the teaching and learning process, in general, are complex matters. Many variables are involved in instructional settings, not to mention other elements involved in distance education, such as social, organizational, technical, and global issues affecting the theory and practice in the field.
Getting beyond the question of "significant differences," experts make a plea for more qualitative approaches in distance education research (cf. Minnes, 1985;Saba 2000) to capture a deeper and richer range of data. Garrison and Shale (1994) make a case for mixed methods research: "Researchers are realizing that in practice the methodologies can be viewed as complementary ….Researchers who advocate combining quantitative and qualitative methods are thus on solid epistemological ground" (p. 25). The link between qualitative and quantitative research is therefore often termed triangulation (cf. Neumann, 2007, p. 149). This approach has the advantage that a complex research field such as distance education can be explored from different perspectives (or angles), utilizing different instruments and methods, and the data gathered can be used to mutually validate the results.
However, what is the current status of the practice of distance education research? Table 5 shows the frequency tabulation regarding the methods applied in published studies. Interestingly there seems to be a trend towards more empirical research. Only 38.1% of all articles are descriptive in nature ("other") and 12.9% followed a mixed-method design (triangulation). However, this trend is not significant for the investigated period between 2000 and 2008 in the five journals. In contrast to the results of this study, Berge & Mrozowski (2001) classified 75.9% of articles published in journals as descriptive (N = 727 articles; AJDE, DE, JDE, andOL between 1990 and1999). In his review of 361 articles published between 1991 and 1996 in AJDE, DE, JDE, and the Indian Journal of Open Learning, Mishra (1997) reported the percentage of descriptive papers as 47.6.

35
Research into issues on the macro level (distance education systems and theories) is very descriptive, e.g., research area 3 (distance teaching systems and institutions, 77.4%), or research area 1 (issues of access, equity and ethics, 74.2%). As could be expected, the highest percentage of theoretical papers was found under research area 4 on theories and models of distance education (79.2%). The highest frequency of qualitative approaches can be reported for research area 14 (interaction and communication in learning communities, 33.6%). A large number of quantitative studies are carried out in research area 12 (quality assurance, e.g., development and application of instruments for program evaluation, analysis of drop-out, completion/noncompletion in distance learning programs, etc., 53.7%), and in research area 15 (learner characteristics, e.g., studies on learning styles, learner preferences, etc., 53.1%).

Authorship Patterns and Reference Characteristics
Collaboration.
With 15 research areas on three broad levels, as described in the Classification of Research Areas section, the complexity of the distance education discipline is enormous, and it continues to grow with the application of new and emerging information and communication technologies and their implications for educational systems and organizational change. Researchers are more and more involved in international collaborative projects. This phenomenon has accelerated during the last decade through the massive proliferation of computer-mediated communication and the development of new web applications which facilitate collaboration, social interaction, and negotiation of meaning (e.g., via social software or Web 2.0 applications). As an indicator of the extent of collaboration among researchers, the number of contributors per paper was analyzed. This positive trend since 2000 towards collaboration in distance education research is supported by the fact that Mishra (1997) reported a much lower percentage of multiple author papers of 38.5% for the period between 1991 and 1996 (N = 361 articles).
Year Percentages single-author articles multiple-author articles Leading contributors.
The total number of different authors who contributed to the 695 articles in this study was 1138. Table 9 lists the 48 authors who contributed to at least three articles. They are from Australia (5), Canada (17), China (5), Israel (2), Japan/South Korea (1), New Zealand (1), UK (8), and USA (9). Canada is very successful in distance education research, with 17 leading contributors coming from this country. Nine of these 17 researchers are affiliated with Athabasca University.

Country-wise distribution of articles.
For the analysis of the country-wise distribution of articles, the country of origin of the first author was taken into consideration (53 countries). Table 11 shows that the vast majority of articles (80.3%) come from only five countries: USA, Canada, UK, Australia, and China. Given the national bias of journals, it is no wonder that journals tend to publish more from their own country of origin. For example, the stated intent of AJDE is to explore "the great new field of study, research, and practice that is distance education in the Americas" (Moore, 1987, p. 1). This goal is truly met: AJDE in particular, and also JDE, have a strong North American focus: 80.4% of papers published in AJDE and 71.1% of papers published in JDE are contributed by US-American or Canadian authors respectively. Koble and Bunker (1997) found that 69.8% of authors in AJDE between 1987 and 1995 were from the USA and 20.9% were from Canada. Similar results are also reported in earlier studies by Calvert (1995) and Mishra (1997).

Number of references.
In 1985 Michael G. Moore, the editor of AJDE, lamented the lack of references to previous research and theoretical frameworks: "I believe the convention of reporting previous research as preface to current research is too neglected today" (Moore, 1985, p. 37).
The incidence of unreferenced articles and the number of references per article is taken as an indicator of the scholarliness of a journal (cf. Cline, 1982, p. 210;Mishra, 1997). Articles providing fewer than 10 references imply that "scholarship does not exist but is irrelevant or exists relevantly but is unknown" (Price, 1970, p. 8). Taking this as a benchmark, as was to be expected, all journals under review are on a high academic level, with a mean of 29 references per article. During the period of 2000 to 2008, only one paper out of 695 was published without any references in AJDE (Osiakwan & Wright. 2001). The record holder is McGreal (2004) with 128 references in a paper in IRRODL on copyright issues.
However, this point should not be pushed too hard. We do not claim that papers with 20 references are more scholarly than those with only 15, or that all articles with 20 references are of similar scholarliness.
• The AJDE clearly prefers to publish quantitative studies; whereas, DE accepted the highest number of qualitative studies, and JDE published the highest number of papers that followed a mixed method design. • More than 80% of all articles were contributed by authors from only five countries: USA, Canada, UK, Australia, and China. The first authors of the 695 articles under review came from 54 different countries. Interestingly, the journals publish more from their own country of origin. The most international journal is IRRODL with only 18.9% of authors coming from Canada; whereas, AJDE has a strong North American bias with over 80% of authors from the USA and Canada.
• A significant trend was found towards more collaboration among researchers in distance education. In the period between 2000 and 2008, the proportion of single-author papers was 44.2% compared to 61.5% of 361 articles published between 1991 and 1996 that were reviewed by Mishra (1997).

So Quo Vadis?
According to the experts' opinion in the Delphi study (Zawacki-Richter, 2009), there is a great need for more research on the role of culture and cultural differences in global distance learning programmes. Furthermore, co-operation among institutions should receive much more attention, including the impact of cultural differences on leadership and culturally complex student support systems, and there is a need for comparative research on distance learning systems (macro level). A lot of work still needs to be done on the meso level: In particular, experts on the panel highlighted aspects referring to leadership in distance education and strategy, management of change and innovation, costs, organizational development and infrastructure for online student and faculty support, professional development, and quality assurance. The experts claimed that empirical evidence is lacking on the pedagogical opportunities that Web 2.0 applications, mobile devices, and synchronous tools afford for teaching, learning, and assessment. This review of 695 articles published in five leading distance education research journals confirms the results of the pre-study. It reveals a strong imbalance between the three research levels. Distance education research is highly dominated by issues that refer to the micro perspective (teaching and learning in distance education), with over 50% of all articles focusing on interaction and communication in learning communities, instructional design, and learner characteristics. As anticipated according to the results of the pre-study, those areas that were said to require much more attention take the last three places (globalisation of education and crosscultural aspects, innovation and change, and costs and benefits) in the ranking of research areas with regard to the frequency of articles (Table 3).
A possible interpretation for this imbalance is that the selection of research themes might follow practical considerations, especially with regard to the availability of data. Of course, the analysis of interaction patterns in computer-mediated communication is a very interesting topic and the text-based data of synchronous communication in online conferences is saved in databases of learning management systems and is therefore readily available. In contrast, it is not surprising