International Review of Research in Open and Distributed Learning Beauty Lies in the Eye of the Beholder

Best practice can be deﬁned as that combination of structure, educational technology and content of a learning opportunity, which, in certain contexts and for particular groups of learners, is most likely to achieve the purposes of the main stakeholders. However, the rate of change of technological, political, economic, social and cultural contexts suggests that best practice may become a redundant concept, in that what is judged as best one day may not be so judged the next. This article considers what some signiﬁcant contributions to the literature on open and distance learning practice have to say about the development and provision of best practice and about the place of critical reﬂection by stakeholders. It also considers the challenges facing the development of best practice presented by change, concluding with the identiﬁcation of the most signiﬁcant areas of development yet to be made.


Introduction
How many of us could have anticipated the exponential growth of open and distance learning which we now see around us? What was initially seen as a form of learning provided by a few national universities which specialised in distance teaching is currently ubiquitous across all levels of learning and all forms of institutions.Worldwide there are now thousands of different providers of open and distance learning giving credence to the use of the term .global phenomenon to describe what is happening.Political, religious, military, commercial, industrial and educational organisations are among the agencies in over 102 different countries who offer open and distance learning courses (Calder & McCollum, 1998).We are not however describing the adoption of some static innovation.As new and different information and communications technologies become more widely available, they allow new delivery systems 1 to be tested and adopted and new groups of learners to be reached.Accurate data about this provision is however remarkably difficult to find.The lack of agreed definitions of terms, the lack of systematic, reliable and accessible data at all levels -local, national and international -and the paucity of rigorous research tends to be ignored in the rhetoric surrounding the expansion of open and distance learning.
Beauty Lies in the Eye of the Beholder 2 In spite of this proliferation of initiatives, or perhaps because of their very newness, there are many who would argue that open and distance learning does not yet constitute a recognised field of study.I suspect that in part this view is held because there is not yet an established and coherent body of knowledge which is recognised as representing the whole field.However the very nature of open and distance learning means that the boundaries to the field are still changing and look likely to continue to change well into the next millennium.
Much of this change is driven by those in government and industry who, in attempting to respond to the profound changes taking place in the global economy, place a perceived link between education, training and the economy at the centre of their thinking.Commentators such as Edwards (1997) have highlighted the "general shift towards technologically mediated and flexible forms of delivery to the extent where it is possible to argue that the boundary between "distance education" and "conventional education" is likely to disappear" (p.126).
At the same time, technological enthusiasts battle for the widest possible dissemination of their favourite technological innovations.The early adoption of these innovations by what have been termed visionaries is a high-risk activity.New methods of teaching have to be introduced, along with new forms of organisation and quality control of delivery systems and unfamiliar methods of learning.New stakeholder groups such as community leaders, employers, or, with in-company training, line managers, may be involved.While costs can reduce dramatically with high student numbers, at some point the question of output and achievement starts to be raised.
Open and distance education is now a mainstream and widespread phenomenon.The global spread and the diversity of practice which comprises open and distance education at the beginning of this new millennium is a source of both optimism and concern.It is a source of optimism in that the rapid spread and development of an innovation which challenges many of our preconceptions about teaching and learning suggests that there is an openness and receptivity to new ideas among the policy makers and strategic thinkers in many countries.It is a source of concern in that the development and introduction of many open and distance learning initiatives is driven by a desire to achieve simple low cost solutions to complex social and economic problems.In such contexts, the quality of the provision appears often as a fragile afterthought rather than as fundamental to its development.

What is meant by open and distance education?
The terms open, distance, flexible, and remote learning are used increasingly loosely to describe a growing and diverse variety of learning delivery systems.

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder

3
The question of just what these terms actually mean has been a source of considerable debate in the literature.The speed of change and the rapid introduction of new developments have presented a considerable challenge to those attempting to capture and to define key concepts.Increasingly the temptation for many is to use some terms interchangeably.There are however critical differences between them.Although many authorities have attempted to define the concept of distance education (Daniel, 1996;Dodds, 1995;Holmberg, 1986;Keegan, 1996;Moore, 1990;Wedermeyer, 1981), Holmberg's definition, first articulated in 1977, appears to have stood the test of time.He declared that: Distance education thus includes the various forms of study at all levels which are not under the continuous, immediate supervision of tutors present with their students in lecture rooms or on the same premises, but which, nevertheless, benefit from the planning, guidance and tuition of a tutorial organisation.(Holmberg, 1986, p. 2) In effect, the one common factor appears to be the use of media to enable the time separation and the geographical separation of the teaching process from the learning process.Issues of synchronicity (Daniel, 1996), two-way communication (Holmberg, 1986), the role of face-to-face support (Dodds, 1995), the influence of an educational organisation (Keegan, 1996), transactional distance (Moore, 1990), are all subsumed within this working definition.
It could be argued that the confusion between the terms open and distance was set early on with the establishment of the Open University (UKOU).This model of open and distance learning (ODL), developed in the late 1960s, was designed to give adults a second chance to study at degree level regardless of where they lived, regardless of their work or family responsibilities, and regardless of the lack of any prior educational qualifications.Lord Geoffrey Crowther, in his Installation Address as Vice Chancellor of the UKOU said that "we are open as to people, as to places, as to methods and as to ideas" (Open University, 1973, p. vii).Thus the ideas of openness rather than of distance were emphasised at this time.Key features of this early model of ODL included the design, development and production of courses through the use of teams of academics, educational technologists and media specialists; mass dissemination via a range of different media including print and broadcasting; further dissemination through the use of personal media such as audio and video; and personalised assessment and support to individual learners through the allocation of personal tutors.International practice during the next decade tended to focus on variations of the UK model with the establishment of centralised and government sponsored distance teaching universities such as the Universidad Nacional de Educacion a Distancia (UNED) in Spain, the Sukhotai Thammathirat Open University (STOU) in Thailand and the China Central Broadcasting and TV University (CRTVU) in China during the 1970s.Daniel (1996) has highlighted the appearance and the growth of these mega-universities, defined as distance teaching universities with over 100,000 students studying at degree level.

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 4 Recognition of the differentiation between the ideas of distance and of openness was relatively slow in coming.Not until the mid-1980s did authorities such as Lewis, Rumble, Scriven, Robinson and Carr attempt to disentangle what was meant by the term open learning.No consensus emerged.On the one hand were those who, having examined the difficulties in attempting a definition, drew back from the brink, concluding that "there exists no universally-agreed, adequate and comprehensive definition of open learning" (Webberley & Haffenden, 1987, p. 137).Others preferred to take a more inclusive approach to the problem.While Lewis (1986) conceptualised open learning in terms of a continuum in terms of the choices available to the learner, Carr (1990), and also Robinson (1989), drew a distinction between open access and open pedagogy.As Carr argued, the removal of barriers in terms of the location of the learning, or the pace at which it progressed had little relationship with the issue of learner centredness in deciding what and how the learner should learn.Thus it became clear that the terms open and distance were not synonymous, but did in fact refer to distinct and different ideas.This explained how some distance teaching provision could, in many respects, be considered closed (Guri-Rozenblitz, 1993).In the same way, open learning did not have to take place within a distance taught mode.
The terms flexible learning, distributed learning and e-learning are relative newcomers to the field, and again, are often used synonymously with the terms open and distance.The term flexible learning came via the vocational training field, and, as Smith points out, bears a distinct resemblance to the concept of open learning by virtue of two key determinants -"extended access to learning through the removal of barriers, and a philosophy of learner-centred provision where learner choice is the key" (Smith, 2000, p. 88).Meanwhile, the terms distributed learning in North America, and e-learning in the UK are used to describe integrated electronic distributed learning environments (or supported online learning) (Inglis, in press).

Best practice
Given the almost overwhelming range of practice on which would-be providers of open and distance education can draw, what can be said about best practice?How can it be identified, developed and provided?It must be said that relevance of the concept itself is open to debate.My own attempt at a definition would be along the following lines.
Best practice can be defined as that combination of structure, educational technology and content of a learning opportunity, which, in certain contexts and for particular groups of learners, is most likely to achieve the purposes of the main stakeholders.

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 5 This definition raises more questions than it answers for those looking for enlightenment.In particular there are the criteria which should be applied in assessing practice.Questions about the structure of the provision, the media to be used, the content of the provision, and the teaching approach used are usually to the fore.However major texts on practice also draw attention to the issue of learner support (Cookson, 1990;Rowntree, 1992;Simpson, 2000;Tait & Mills, 1999), to the institutional context and management of the provision (Moore, 1990;Robinson, 1989;Rumble, 1996), and to the necessity of evaluation over the life of the programme in relation to quality.At the same time the issue of costs and resource availability must be taken account of in the debate.
The application of these criteria by individual providers can present problems.While there is a considerable literature on each of these aspects, with guidance being given as to effective and efficient ways of designing, developing and producing materials, it is clear from reports of actual practice that providers new to the field are often unaware of the literature and the knowledge resources available to them.At the same time, the staff charged with designing and developing distance teaching materials, such as academics in higher education, lecturers and teachers in further education, community educators and vocational skills trainers, may have little or no background themselves in open and distance education and may thus have problems in identifying and articulating their own support needs (Abdullah, 1998).The urgent need for staff development in this area is only now becoming widely recognised.Texts which specifically address these needs are now beginning to appear (e.g., Latchem & Lockwood, 1998).
The other question lies with the issue of whether there is such a thing as best practice or whether a more realistic aim should be good practice.My definition of best practice above attempts to locate the complex set of variables which constitute practice within a framework which is constantly changing.For example, learning contexts at the national, institutional, local and individual levels vary continuously depending on current economic, social and political conditions.Stakeholders' aims, their purposes in establishing provision, and the measures of success they use will also vary.Thus, a particular form of provision which would be claimed by a particular set of stakeholders at one point in time to represent best practice, would not be regarded as such at another point in time.In contrast, good practice is a more robust concept.It is designed to allow for fluctuations and change, and for compromise between the aims of different stakeholders.
For example, the priority aim may be to provide higher education opportunities for school leavers from all sections of society, or for a wider range of employed workers, or to maximise numbers of people with specific skills, the application of skills or knowledge in a range of contexts, or the achievement of critical reflection and awareness of the transformational properties of learning.These purposes will vary with the context in which the provision is offered, and with the culture within which it is offered.The literature on the use of open and

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 6 distance approaches to vocational education and training suggests that, while there is evidence that there are a variety of practices provided by both the public sector and within companies, there is often little evidence, particularly in the private sector, of the planning and monitoring systems needed to assess the success of different approaches.
It is also by now well recognised that different groups of stakeholders perceive success in very different ways.Funders, for example, may focus on input measures such as numbers of learners who buy, or register for a course, or the number of employees to whom a CD-ROM is circulated.Providers may look at other measures such as net profits, client satisfaction ratings or relative rankings of their organisation by peer groups.Suppliers of communication channels or of materials may look at gross sales and market share.Learners themselves use criteria such as enjoyment, increased self-confidence and achievement of personal short-or long-term aims.The fact is that there can be a clear potential for conflict both between different stakeholders' aims and their criteria for success.

The place of critical reflection
While there are plentiful case studies about different examples of practice in the literature, there has been, to date, relatively little critical reflection on experience that has been shared publicly.However providers' concerns can be deduced to some extent from the issues raised in the literature.Three issues in particular are of interest in that reflection: the importance of organisational structure, learning technologies, and learner centredness.Debate on the issues they raise continues unabated.

Importance of organisational structures
There is no shortage of clear authoritative texts on the structure, composition, and development of open and distance learning courseware.The clear consensus is that in order to achieve effective teaching and learning, not just the design, but also the production and distribution of the teaching and learning materials have to be well planned and controlled.However, the tendency among both public and private providers is for the different systems which are necessary for the development, production, distribution and support of teaching and learning to be organised, and often operated separately.A common scenario is where one person or group of people design and prepare the course content, another person or group of people transfer that content to one or more media, another group bear responsibility for distributing the material to the users, and yet another group or groups actually interact with and support the students during their learning.Recognition of the need to consider issues of course development and production processes and systems, with a few notable exceptions, does not ap-

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 7 pear to have been addressed in the early literature.Otto Peters' contribution is perhaps one of the few outstanding exceptions with his pursuit of debate about what he termed "industrial forms of instruction" (Peter, 1993, p. 15).He saw distance education as the product of a particular period in the development of our culture in which "distance study must be carefully pre-planned, prepared and organised, and that there is a division of labour, a growing use of technical equipment to work with, and the necessity of formalised evaluations" (Peters, 1993, p. 15).He first applied this analysis to correspondence education in an original paper in German in the late 1970s, but subsequently a full debate developed about the use of what were termed Fordist and post-Fordist approaches in distance education (e.g., Farnes, 1993;Raggatt, 1993;Rumble, 1996).However recognition of the need for clear and firm management control and agreed upon avenues of communication for monitoring and evaluating progress is now becoming evident.Both Brown (1997) and Robinson (in press) have highlighted in particular the importance of the organisational dimension and the quality of the planning processes and evaluation processes and their effects upon outcomes of innovations in open and distance education.
A number of early writers drew attention to the need to consider the special processes that are used in designing, building, operating, and evaluating nontraditional institutions and programs in relation to the emphasis given to the different technologies and different media used in teaching at a distance (e.g., Rowntree, 1974;Wedermeyer, 1981).Not only was there a considerable interest in the learning technologies which were used, but there was a strong tendency towards focussing on the potential of different technologies rather than on their relative effectiveness with different types of learners in different contexts.Nipper's (1989) timely contribution of the idea of generations of distance education to such debates for example, focussed on the simple recognition of the changing use of technology for distance education.Even here, although the distinction between the first two generations of distance education was clear -with correspondence teaching as the first generation and multiple media as the second -Bates (1991) identified a lack of clarity about the defining characteristics of the third generation.The continuing debate about what constitutes third generation distance education has emphasised the importance of the distinction between the use of technology for distribution purposes, for example via postal services, broadcasts and the Internet, and its use for an interactive exchange between teacher or trainer and learner using audio and videoconferencing, and computer conferencing or e-mail.

Learning technologies
Concerned by the assumption that the latest technology was necessarily the best one to use, with scant regard for its particular strengths and weaknesses, researchers such as Bates attempted to introduce a note of reality into the debate about media choice and media use in distance teaching.His downbeat

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 8 conclusion almost two decades ago that "the greatest media development during [the UKOU's] 12 years of existence has been the humble audio-cassette" (Bates 1982, p. 11) drew early attention to the importance of the teaching and learning issues as well as the technological changes.The same message of caution continues in much of the literature today.In an American review of contemporary research on the effectiveness of distance learning in higher education, Phipps and Merisotis concluded that the higher education community still had much to learn about how technology could enhance teaching and learning at a distance, expressing concern over research which was driven by the information revolution.As they pointed out: computer mediated learning requires special skills of students and more sophisticated technical support if students are to interact fully.Questions that need to be asked include: What is the "quality" of the access?Does the student have the necessary skills to use the technology?What are the best ways to participate in synchronous communication?Is there adequate technical support?Perhaps most important, will the cost of purchasing a computer and maintaining software be prohibitive for a substantial number of students?(Phipps & Merisotis, 1999, p. 7)

Learner centredness
The debates around teaching and learning at a distance have drawn on a range of literatures.American and Canadian research on self-directed learning which was led by the work of Allen Tough (1979) and Malcolm Knowles (1975) has had a major influence on thinking about the learning process in adults.Authorities on adult education such as Stephen Brookfield, commenting on the research which had revealed the vast extent of self-managed and independent learning which adults undertook as part of their daily lives, remarked that "there is now much less likelihood that educators will presume that valid and valuable adult learning can occur only in the presence of an accredited and professionally certified teacher" (Brookfield, 1986, p. 149).Distance educators such as Evans (1994), and Morgan (1993) attempted to open up what they termed the world of the learner by investigating the factors which influenced students' learning and the student experience of the learning process.Literature in this field was underpinned by the work of researchers such as Kolb (1984), Pask and Scott (1972), Marton and Saljo (1976) and Entwistle (1981).Smith's (2000) review of the development and application of ideas on individual learning styles and approaches to learning shows the more recent growth of people working in the field.However, the impact of the work on learners and learning in relation to the development of ideas about the structuring of materials and the need for different types of support by different groups of learners is only now beginning to be felt in open and distance education.The importance of the age of the

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 9 student, their previous educational experience, the type of programme in which they are enrolled, and their cultural affiliation are all factors which have emerged as associated with the learners' approach to learning (e.g., Calder & Wijeratne, 1999;Kember 1999;Richardson, 1997).

Challenges
The challenges facing the development of best practice, or as I would prefer it, good practice, are many.But I would highlight three challenges in particular which need to be met if open and distance education is to achieve its potential.These are, to develop a better understanding of (a) the effects on curricula when presented through open and distance education, (b) the effects of cultural diversity in determining good practice, and (c) the challenge of access and equity.

Effects upon curricula
Concern about good practice in teaching at a distance initially focussed on adult learning at degree level.Curriculum design and development followed very much the traditional university curriculum.Any debate about the curriculum tended to revolve around the extent to which laboratory-based subjects such as applied sciences, technology, and medicine could be taught at a distance.While a whole literature exists on this topic alone, early solutions ranged from decisions not to include the subject at all, to including periods of residential laboratory work as part of the course, to devising and using home-based experimental work.As the technology available developed, more sophisticated solutions such as the virtual microscope and other forms of simulation have been introduced.Changes to the form and shape of the curriculum itself reflected changes in the locus of control.Whereas previously higher education provision had reflected the teacher-centred nature of existing curricula, economic and market pressures pushed providers towards a more client-centred curricula.Thus there appears to be a tendency among providers to construct curricula which can be presented as a series of increasingly shorter modularised courses which are planned to take days or weeks of part-time study rather than months or years.It could be argued that the curriculum of vocational education and training is similarly changing.For example the growth of competence-based training qualifications has encouraged the development of links between modules of training traditionally provided for induction, skills development or up-dating purposes for basic grade staff, to enable portfolios of work to be developed for assessment purposes.

International Review of Research in Open and Distance Learning
Beauty Lies in the Eye of the Beholder 10 Sensitivity to the effects of cultural diversity in determining good practice Attention has been increasingly drawn to the diversity of problems faced by those attempting to design and introduce good open and distance education practice.It increasingly appears that the importance of inherent differences in organisational cultures, academic cultures, education and training philosophies, and teaching and learning values and traditions within different cultural groups have not been adequately recognised by those attempting to transplant models of practice from other contexts.Robinson (1999), for example, has pointed out the need for distance educators "to understand more fully the cultural contexts of learners and to build better bridges into and out of cultures of learning"(p.45).Organisations which do not value the rather different demands of distance teaching from conventional face-to-face teaching can place unrealistic demands on their staff and on their organisational systems.Academic cultures which give a different emphasis to critical peer review, and with entrenched academic hierarchies may find the course team approach to developing materials less relevant than other approaches.Similarly, a culture of elitism rather than of learner centredness and inclusiveness will result in fundamental differences in the way in which student recruitment is undertaken, in the language or languages in which materials are prepared and support given, and in which student and programme evaluation findings are interpreted.At its simplest, are high drop-out and poor results interpreted as meaning that the students do not have the ability to cope, or that the provision does not meet learners' needs because of problems with its design and implementation?The growing interest in the ideas of Argyris and Schon (1978) and the development of a learning organisation suggest that there is a growing awareness of these issues.

The challenge of access and equity
Finally, the issue of access and inclusiveness is one which is coming increasingly to the fore in the developed, newly developed and developing worlds.Widening participation in education and training is now seen as an economic and social imperative.How this participation is to be achieved is, however, problematic, and certainly distance education is seen by many as the solution with most potential.However, it is clear from the funding regimes which have been put in place for many distance education programmes that the apparently low costs of this form of provision is the key attraction to many of those seeking to optimise student or trainee numbers on decreasing budgets.The fact that low variable costs per additional student recruited comes with the high fixed costs associated with initial course development, and with fixed variable costs where learner support is integral to the programme is often overlooked or ignored.There is a growing recognition of the frequent lack of resources -financial, personnel, time, facilities -which so often accompany commitments to produce "low cost" International Review of Research in Open and Distance Learning distance education (Robinson, in press).Paper after paper at international conferences bear testimony to the disastrous effects on the development of open and distance education programmes of inappropriate organisational cultures, inadequate communication infrastructures, and inadequately trained and prepared staff (e.g., Mugadzaweta & Benza, 1999;Murugan, 1999).
It should be pointed out that there was a distinct lack of early reliable cost estimates related to production and distribution.With the exception of people such as Rumble (1976Rumble ( , 1986Rumble ( , 1997) ) writing on cost effectiveness, there was little written about the organisation and administration of distance education.More recently, with the growth of the market in all forms of open and distance education, concern about real costs has come to the fore, with a recognition of the need for much greater transparency about all costs associated with any programme.For example, a recent review (Marchmont, 2000) of the costs of e-learning identified a number of different models for examining costs.While each of the models has its own strengths and weaknesses, among the points they make are the need to distinguish between different types of costs and benefits, to link costs to break-even points in relation to student numbers as well as to average costs, and to link cost structures to the different phases of project development.One example given by Marchmont (2000) is for a three-phase model that would cost each phase separately: planning and development, production and delivery, and maintenance and evaluation.Perhaps to that should be added a fourth phase of up-dating and revision or remake.The importance of identifying otherwise hidden costs associated with programmes, such as for instance, staff time for learner support is also emphasised.
Issues of access and equity are not limited to availability of resources, but the role played by resource availability is fundamental to the sustainability of good open and distance education practice.If genius is one tenth inspiration and nine tenths perspiration, then it can also be argued that good practice in distance education is one tenth presentation and nine tenths preparation.There are no short cuts to good practice.