Evaluation Criteria for Interactive E-Books for Open and Distance Learning

The aim of this mixed method study is to identify evaluation criteria for interactive e-books. To find answers for the research questions of the study, both quantitative and qualitative data were collected through a four-round Delphi study with a panel consisting of 30 experts. After that, a total of 20 interactive e-books were examined with heuristic inquiry methodology. In the final phase, the results of the Delphi technique and the heuristic inquiry results were integrated. As a result, four themes, 15 dimensions, and 37 criteria were developed for interactive e-books. Lastly, the results and their implications are discussed in this paper and suggestions for further research are presented.


Introduction
Open and Distance Learning (ODL) strives to provide effective, efficient, engaging, and enduring learning opportunities which are dependent on improvements and developments in information and communication technologies (ICT). There have been attempts to employ ICT to eliminate the limitations that derive from physical and psychological distance among learners, learning sources, and learning environments. The influence of ICT resulted in online and digital solutions that increase interaction in the learning process. Beldarrain (2006) suggests that technology has played a critical role in changing the dynamics of each delivery option over the years, as well as the pedagogy in ODL. As new technologies emerged, instructional designers and educators had unique opportunities to foster interaction and collaboration among learners, thus creating a true learning community.

59
As a result of these developments, e-books and, following that, interactive e-books have gained a wide interest and have been used as a valuable and viable medium in both traditional education and ODL. By realizing the potential of digital books, institutions of higher education have begun to provide interactive e-books for learners to be able to deliver the information in a more effective and attractive way. According to Rothman (2006), for distance educators as well as traditional classroom educators, digital books would not only enhance student access to information, but would also help revolutionize the processes of reading, analyzing, and researching.
As a response to current developments in the e-book and interactive e-book landscape, interactive e-books are defined and their pros and cons are explained in this study. Following that, interactivity in interactive e-books is discussed and finally evaluation criteria of interactive ebooks are explained based on a mixed method study in which Delphi technique and heuristic inquiry were used.

Aim of the Study
Interactive e-books are used for providing flexibility and presenting enriched content by means of hard and soft technologies. As an emerging technology, it is a necessity to define evaluation criteria of interactive e-books and contribute to relevant literature.
On this basis, the main purpose of this research is to develop an evaluation criteria checklist for interactive e-books. Within this perspective, research questions for this study are as follows: • What are the core themes to evaluate interactive e-books?
• What dimensions and criteria should the determined core themes cover?

Interactive e-Books
Defining Interactive e-Books Books are defined as "the first teaching machine" (McLuhan, 1964, p.174) and they are indispensable in the teaching/learning process (West, Turner, & Zhao, 2010). For centuries, books have been the catalyst of dissemination and transmission of knowledge. They paved the way of improvement, helped to evolve humankind and have evolved themselves. The year 1971 was a milestone for electronic books (e-books). Michael Stern Hart initiated the Project Gutenberg that year to encourage the creation and distribution of e-books (Hart, 2004) and created the first digital version of Declaration of Independence as the first e-book in history (Hart, 1992). Other developments such as the first digital-born hypertext fiction Afternoon in 1980, DOS-based e-books and the Runeberg Project in 1992, PDF 1.0 in 1993, E-ink Corporation in 1997, first handheld e-book reader in 1998, copyright/copyleft and Creative Commons in 2001, Kindle e-book reader by Amazon in 2007, and tablet PCs and smartphones at the beginning of the new millennium triggered the evolution and acceptance of digital books (Bozkurt, 2013;Bozkurt & Bozkaya, 2013a).
When comparing definitions, conventional books (c-books) can be defined as a set of written and printed sheets that include text and visuals. As a digital version of c-books, Rao (2003) defines ebooks as text in digital form, a book converted into digital form, digital reading materials, a book in a computer file format, an electronic file of words and images displayed on a device screen intended for more than solely reading e-books, or an electronic file formatted for display on dedicated e-book readers.
In 2011, introduction of the next-generation digital book required a new definition: interactive ebooks. In his TED Talk (Technology, Entertainment and Design), Matas (2011) introduced one of the first known interactive e-books, Our Choice, and promoted it as a next-generation digital book. Our Choice was a clear indicator of the future of digital books as the first full-length digital book that utilized various creative and innovative features. Some features of this interactive ebook are given in Table 1.  Talks) By examining 20 interactive e-books systematically, Bozkurt & Bozkaya (2013a;2013b) defined interactive e-books as an improved extension of digital books. According to their definition, interactive e-books are essentially digital book formats in which the user, the digital book, and the environment can interact reciprocally at a high level; digital book elements can communicate and interact among themselves and environment as well as users, and many communication channels are put in use at one and the same time. They also defined the digital book as a generic term that covers e-books, interactive e-books, and other digital book formats. A comparison of c-books, e-books, and interactive e-books are provided in Figure 1.  (Bozkurt & Bozkaya, 2013a;2013b).
According to this definition, it is salient that advanced interactive e-books are at the forefront of the digital book evolution which is tightly connected to technological innovation. It can be also seen that as a result of e-books' dependency on technology, the distinction between interactive ebooks and software and mobile applications is being blurred. However, these blurring borders can become distinct by applying design principles of interactive e-books and determining the purpose of the application as it refers to the user's electronic reading (e-reading) experience. In the ereading experience with interactive e-books, there are four type of interactions: interaction between environments (real and virtual environments), interaction among the digital book elements, interaction with other users, and interaction with the user: • Interaction among the digital book elements: Interactive e-book elements can communicate and interact among themselves. This refers to the interconnectedness of interactive e-book elements (e.g. synchronously retrieving data within the book as in the interactive charts).
• Interaction between environments: Interactive e-books can further communicate with real and digital environments. The sensors of an interactive e-book reader (accelerometer, barometer, compass, fingerprint reader, gesture sensor, GPS, A-GPS, GLONASS, gyroscope, heart rate monitor, ambient light sensor, proximity sensor and more) can gather information from a physical environment, for example, geo-location information or data retrieved from online databases.
• Interaction between digital book and user: In addition to the invisible cognitive interaction which occurs while reading in any kind of book, it refers to the tangible interaction between an interactive e-book and a user (e.g. detecting user gestures for navigation or tracking eyes to keep the screen on in reading mode).
• Interaction with other users/online communities: Users can interact with a specific online community by tracking relevant hashtags related to the content within an interactive e-book or they can share all or specific parts of the book on their own online social networks.

Pros and Cons of Interactive e-Books
Interactive e-books have been preferred increasingly, especially from the beginning of 2000 onwards for the advantages they provide. Some of the advantages of interactive e-books are listed in Table 2. Table 2 Pros of Interactive e-Books

In terms of users/readers:
• interactive e-books are portable; you can carry a whole library on one device; • they are searchable; readers can find what they need easily; • they have an enormous capacity to store a great deal of information in a single book; • they have annotating support; readers can edit, add notes, add bookmarks or highlight without harming the original work; • they are customizable; users can tweak the style according to their needs; • they are durable; they do not have a short shelf-life span; • virtual libraries can be created; they are always available with a mobile device or through cloud computing; • if they do not have Digital Rights Management (DRM) restrictions, they are shareable; • they make reading accessible for the individuals with special needs; • they are cheaper than printed books; they are printable; they can be used like a c-book (if it is allowed); • they are convertible; they can be used in different formats; • they can be hyper-linked for additional sources; • they are easy to use with natural user interface and gesture based computing; • they are easy to read; they can be read in the darkness; they can be read aloud automatically with text to speech; • they are portable; there are no weight or bulk constraints; • they defeat attempts of censorship; • they support multimedia content and enhance e-reading experiences; • they are reusable infinitely, and they promote e-reading.

In terms of authors:
• interactive e-books are easy to publish; • they empower self-publishing; • authors can have feedback from readers and update their book instantly.

In terms of libraries and other educational institutions:
• interactive e-books reduce maintenance cost; • allow user statistics if featured; • can be protected with DRM options; • save physical space; • a single book can be used by many; • readers can take service from libraries 7/24; • infinite circulation is possible as interactive e-books do not wear out.

In terms of publishers and retailers:
• interactive e-books are environment friendly; • they can be delivered almost instantly; • they are convenient, they take up no physical space; • their publishing speed is faster and publishing cost is cheaper than c-books.
As well as having advantages, interactive e-books have some disadvantages. However, the following disadvantages in Table 3 mostly derive from external reasons, not from the inherent features of interactive e-books. Table 3 Cons of Interactive e-Books In terms of reading device: • resolution of the screens is a problem and longer reading time fatigue eyes; on the other hand, new display technologies promise real-like high quality display experiences; • compatibility of interactive e-books is another problem; there has yet to be a widely accepted universal format that allows all books to be read on any device; • reading devices need power so they cannot provide continuous e-reading experience and are limited to battery life.

In terms of piracy:
• they can be hacked and can be used easily in a manner that violates copyright.

64
In terms of tactile experience: • many readers simply enjoy and prefer the smell, weight, and page turning sound of print books.

Interaction in ODL
Interaction, as a complex and multifaceted concept in all forms of education, fulfills many critical functions in the educational process (Anderson, 2003). It appears to be one of the most critical instructional elements (Kearsley, 1995) and it was highlighted that interaction is a necessary ingredient for a successful learning experience in ODL (McIsaac & Gunawardena, 1996).
According to Dewey (1916), who used the word "transaction" instead of "interaction" to emphasize the relationship between organism and environment, interaction is the defining component of the educational process that occurs when the learners transform the inert information passed to them from another and construct it into knowledge with personal application and value. Moore and Kearsley (1996) focused on interaction in distance learning and described learner-content, learner-learner, and learner-instructor interaction, while Hillman, Willis, and Gunawardena (1994) additionally described learner-interface interaction. Moore and Kearsley (1996) further stated that effective teaching at a distance depends on a deep understanding of the nature of interaction and how to facilitate interaction through technologically transmitted communications. These ideas inspired instructional designers and educators not only to design interactivity in learning process, but also to design interactive learning tools such as interactive e-books.

Interaction Design: The Flow between Action and Reaction
Interaction design (IxD) and Human Computer Interaction (HCI) are terms that have been used interchangeably. Currently, there has been a growing interest in the structure and nature of interaction design in education and academia. Influenced heavily by ICT in education, interaction design became another discipline engaging in education, particularly in online and digital learning experiences.
According to Wagner (1994), within the ODL perspective, interaction is reciprocal events that require at least two objects and two actions, and it "occurs when these objects and events mutually influence one another" (p. 8). Silver (2007) defines interaction design as a blended endeavour of process, methodology, and attitude. Lowgren (2013) states that interaction design is about shaping digital things for individuals' use.
In essence, interaction design is a system view. All the elements in an interactive system should be designed for a purpose (Bozkurt & Bozkaya, 2013a). That's why interaction design uses five dimensions to have a broad view and cover all elements of an interactive system. On this basis, the first four dimensions of interaction design were introduced by Smith (2007) and the fifth dimension was added by Silver (2007).
• 1 st dimension (Words): Words are interactions that users employ to interact.
• 2 nd dimension (Visual representations): Visual representations, which include typography, diagrams, icons, and other graphics, are the things that the user interacts with on the interface.
• 3 rd dimension (Physical objects or space): It defines the space or objects with which or within which users interact.
• 4 th dimension (Time): Time is the dimension within which users interact. For example, content that changes over time such as sound, video, or animation.
• 5 th dimension (Behavior): Behavior defines the users' actions, their reactions to the interface and how they respond to it. Behavior is about including action, or operation, and presentation, or reaction.
According to Fischer and Coutellier (2005), there are three types of interaction: cognitive, sensorial, and pure physical interactions. It is important to define cognitive interaction since any type of books, including digital ones, are sources of information that require a cognitive interaction to process information and then construct knowledge. Interactive e-books in new generation mobile devices cover all these interactions. On this ground, in an interactive e-book design process, these three types of interaction should be designed in agreement with planned objectives. However, another important issue to decide upon is intended interaction level in interactive e-books.

Interaction Levels
The distinguishing feature to define and categorize book types (c-book, e-book or interactive ebook) is the level of interaction they exhibit. Interactive Multimedia Instruction (IMI), or as it is otherwise known, Interactive Courseware (ICW), has been developed over the years by the U.S Department of Defense (DoD) (1999). In the ICW model, there are four major levels of interactivity which are defined as the degree of student's involvement in the instructional activity. The four levels of interactivity identified by the ICW are provided in Table 4. Table 4 Interactivity Levels and Definitions
The student acts solely as a receiver of information.
The student makes simple responses to instructional cues.
The student makes a variety of responses using varied techniques in response to instructional cues.
The student is directly involved in a life-like set of complex cues and responses.

Related Research
The current study attempts to examine a topic which has not been extensively researched as interactive e-books are a recent emerging technology. Among the few available studies, Wilson and Landoni (2002)  Previous research has generally focused on e-books which are usually in a single file format and include basic multimedia elements with low interaction. Therefore, it can be argued that there is a need to develop evaluation criteria for interactive e-books which include a combination of many multimedia components with high interactivity, and the current study intends to fill this gap.

Theoretical Framework
Throughout this research, four theories were used to establish a sound base to start and provide a clear lens through which we can look, enhance our interpretation, generate valid ideas, and give meaning to the research findings. The theories applied in this research were the theories of Independent Study, Transactional Distance, Multimedia Richness, and Multimedia Learning.
Theory of Independent Study: According to Wedemeyer (1981), the essence of distance education is the independence of the learners. Wedemeyer's Independent Study Theory emphasizes learner independence and adoption of technology as a way to implement that independence (Simonson et al., 2003). According to the theory, learning can occur in spite of the time-space barriers and learning should be individualized by providing wider choices to learners; learning responsibility belongs to learners themselves and they learn at their own paces.

Theory of Transactional Distance:
This theory is originally an extension of the Theory of Independent Study. Inspired by Dewey's term of transaction, the Transactional Distance Theory refers to the cognitive space between instructors and learners in an educational setting. Moore (2007) argues that transactional distance is a typology of all education programs having this distinguishing characteristic of separation of teacher and learner. According to Moore (1993), transactional distance is a psychological and communication space to be crossed, a space of potential misunderstanding between the inputs of instructor and those of the learner. The key concepts of the theory are dialogue, structure, and learner autonomy.

Theory of Media Richness:
Developed originally by Daft and Lengel (1984), the Media Richness Theory is based on the contingency theory and the information processing theory.
According to the theory, "the more equivocal a message, the more clues and data are needed to understand it, and media richness theory places communication mediums on a continuous scale that represents the richness of a medium and its ability to adequately communicate a complex message" (Carlson & Zmud, 1999, p. 155). The main idea of Media Richness Theory was expressed as "the more learning that can be pumped through a medium, the richer the medium" (Lengel & Daft, 1988, p. 226). According to the theory, the richness of the media is influenced by four criteria (Daft & Lengel, 1984): (1) Capacity for immediate feedback; (2) capacity to transmit multiple cues; (3) language variety; and (4) the capacity of the medium to have a personal focus.
Theory of Multimedia Learning: Proposed by Richard Mayer, this theory explains learning with multimedia from the perspectives of educational psychology and e-learning (Ataizi and Bozkurt, 2014) and claims that individuals learn better when multimedia messages are designed in ways that are consistent with how the human mind works (Clark & Mayer, 2011;Mayer, 2002).
The theory has three main assumptions (Mayer, 2002): (1) there are two separate channels for processing information: the auditory and visual channels (Dual Coding Theory); (2) each channel has a limited capacity (cognitive load); and (3) learning is an active process of filtering, selecting, organizing, and integrating information through association with previous experiences.

Research Design
The study was designed as an embedded mixed model research to provide a better understanding of the research problem. The purpose of the embedded design is to collect quantitative and qualitative data simultaneously or sequentially. In the embedded design, a secondary form of data is used to augment or provide additional sources of information not provided by the primary source of data (Creswell, 2004). Throughout this research, the Delphi technique (primary source of data) and heuristic inquiry method (secondary source of data) were applied to obtain and analyze data.
Because the topic was an emerging one dealing with different expertise areas such as instructional design and interaction design, and the number of the studies in the literature was insufficient, the Delphi technique was preferred as a primary source of data to be able to get expert opinions from different disciplines. The Delphi Method is based on a structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback (Adler & Ziglio, 1996;Dalkey & Helmer, 1963;Koçdar & Aydın, 2013). As a highly flexible problem-solving process, it fits for situations where evidence based practice is dependent on expert opinion (Sandrey & Bulger, 2006). Expert panel members provide feedback, revise judgments, and contribute to the development of agreed-upon practicesall with complete anonymity (Flippo, 1998). Within this perspective, the basic assumption of the Delphi Method is that the informed, collective judgment of a group of experts is more accurate and reliable than individual judgment (Clayton, 1997;Ziglio, 1996). The key characteristics of the Delphi technique are defined as anonymity of respondents, controlled feedback process, and statistical response (Fowles, 1978).
User experience (UX) is the essence of the interaction design. Thus, in addition to theoretical and practical experiences of Delphi panelists, researchers employed heuristic inquiry to harness additional research findings which can be explored by directly engaging research questions. On this ground, heuristic inquiry was preferred as a secondary source of data. Heuristic inquiry is an experience-based technique for problem solving, learning, and discovery. Douglass and Moustakas (1985) define heuristic inquiry as a search for the discovery of meaning and essence in significant human experience. The heuristic inquiry is an adaptation of phenomenological inquiry, yet it requires the involvement of the researcher in a disciplined pursuit of research process (Hiles, 2001;Djuraskovic & Arthur, 2010).

Sampling
The selection of panel members is considered to be critical for the Delphi process, which is directly related to the focus or objectives of the research (Sandrey & Bulger, 2006). Interactive ebooks are a final product of different procedures and different expertise. Therefore, it is important to select experts for a purpose to apply their knowledge to a certain problem on the basis of criteria, which are developed from the nature of the problem under investigation .
For this research, the participants are required to be experts in one of the following areas: digital books, digital publishing, content design, instructional design, interface and layout design, or elearning. It is further required that they have a background in academic research or experience working in the field. Through literature review and snowball sampling, 55 experts were invited to the research. A total of 30 experts expressed their intention of participating. A highly representative Delphi panel of 30 experts, from well-respected institutions and companies, who have published research and/or had practical and theoretical experience, was constructed to contribute to the validity and reliability of research findings.
In heuristic inquiry, as the secondary form of data, 20 distinguishing interactive e-books were selected. A set of criteria were defined to be able to examine representative samples that had interactive features peculiar to interactive e-books. The interactive e-books included in the heuristic research were those that were most downloaded, awarded, and had positive reviews by the critics. Book samples that exhibited different features in terms of the interaction level and genre were selected to have maximum variation sampling (Appendix A).

Procedure
Before initiating Delphi rounds, a pilot study was carried out with three doctoral students. Delphi The third round was completed in two weeks with the participation of all experts of the Delphi panel.
• Fourth round: In the final Delphi round, experts of the Delphi panel were requested to reevaluate four themes, 19 dimensions, and 49 criteria that met the predefined agreement levels with five-point Likert items that emerged in the third round. Percentage, median, and IQR statistics used in the third round for consensus were also provided for each item to give statistical feedback. At the end of the final round, items with a mean below 90 percent, median of below 4, and IQR of over 1 were discarded again. As a result of the Delphi study, four themes, 14 dimensions, and 33 criteria were obtained. The final round was completed in two weeks with the participation of all 30 experts of the Delphi panel.
In the heuristic inquiry, a total of 20 interactive e-books were downloaded into a tablet computer.
The interactive e-books were selected purposefully from distinguished examples, most of which have notable awards and reviews (Appendix A). Each book was used and examined in four themes at the end of the Delphi study. First of all, the validity of the Delphi findings were checked by examining the applicability of the criteria which emerged in the Delphi rounds, using 20 interactive e-books. Following that, researchers systematically noted their experiences as well as the features observed. This process was conducted between March and May 2013. All the data gathered were coded, categorized, and put into themes using content analysis. The findings that matched with the findings gathered in the Delphi study were eliminated to assure that the same research findings were not replicated. It is salient that the four new criteria emerged from heuristic inquiry are mostly related with user experience which requires direct interaction with the products that are investigated. For that reason, it is believed that these four criteria didn't emerge during the Delphi rounds. Lastly, four new criteria were defined and associated with relevant themes and dimensions. At the end of the Delphi and heuristic research processes, a total of four themes, 15 dimensions, and 37 criteria were identified.

Data Collection and Analysis
In Delphi studies, qualitative data can be analyzed using content analysis techniques . The data in the first and second Delphi rounds of this study were analyzed using content analysis. The data were coded, categorized, and put into themes (see sample in Appendix B). The findings obtained were converted into short, simple sentences and presented as questionnaire items in the third and fourth Delphi rounds.
For the quantitative data in the third and fourth rounds, statistical methods were used. Consensus in a Delphi study is subject to interpretation. Consensus can be decided if a certain percentage falls within a prescribed range (Miller, 2006). Some researchers suggest that a level of at least 70 percent agreement is enough to call a consensus; on the other hand, there is no certain percentage defined and it changes according to the scope of the research topic. Other statistics used in Delphi studies are measures of central tendency (means, median, and mode) and level of dispersion (standard deviation and interquartile range) . To be able to get robust and reliable research results, a combination of different statistics were defined as the consensus level for the research. Percentage (80% for the third round and 90% for the fourth round), IQR, and median were used as statistical indicators of consensus (Table 5). In the heuristic inquiry, both qualitative and quantitative data were gathered systematically through real life experiences. The data were then analyzed using content analysis. The findings were used to support and to check the Delphi study findings and to discover new findings concerning the evaluation criteria of interactive e-books.

Limitations
The findings of this research is limited to current Web and mobile technologies in addition to capacities of mobile devices used in the research. For this reason, new themes, dimensions, and criteria can be added in time with the emergence of new technologies or new learning approaches.

Reliability
The data in the first and second round of Delphi were coded by one of the authors (Rater A) of this study. Another author (Rater B) who was experienced in the field also rated the data of first and second rounds. Cohen's Kappa (κ) was calculated to check the inter-rater reliability of the first two Delphi rounds. Inter-rater reliability between Raters A and B for the first round was κ = .918 (95% CI, .8551 to .9817), p<.0005 and κ = .951 (95% CI, .9040 to .9981), p<.0005 for the second round. Altman (1991) proposed that the extent of agreement can be qualified as poor (< 0.20), fair (0.21 to 0.40), moderate (0.41 to 0.60), good (0.61 to 0.80), and very good (0.81 to 1.00). Thus, the reliability of ratings for the first and second round Delphi data can be considered as very good.

Results
The overall findings of this study are presented in Table 6. In the following table, research findings were organized as themes, dimensions and criteria. The criteria were associated with the most relevant dimensions and themes, but these criteria may intersect and overlap with other dimensions or themes. Table 6 Interactive e-Book Evaluation Criteria

CONTENT Presentation
• Clear and fluent language usage • Effective narration features • Preparing content with a theoretical framework that supports learning objectives

Richness
• Richness of multimedia components • Balance of information density

Motivation and Attractiveness
• Attractiveness of the content • Content design appropriate for characteristics of the target audience

Assessment and Evaluation
• Providing mechanisms to users enabling them to assess their own learning process

Integrity , Coherence and Connectivity
•  McLuhan's (1964) widely known phrase, "The medium is the message," means that the content of a mediated message is secondary to the medium (West et al., 2010). McLuhan did not ignore the importance of the content, but he deliberately pointed out the ability of the medium that shapes the message. In this study, the medium's ability, that is to say interactive e-books, to shape the message became apparent in four basic themes that emerged as a result of the data analysis: content, interface, interaction, and technology.
• Content: This theme presents the pedagogic perspective of the evaluation criteria. The criteria in this theme highlight the importance of learning/instructional design. It is clear that interaction is not provided with solid technologies but with design of the content.
From the selection of action verbs in the content to the theoretical framework applied, all are related with content theme.
• Interface: This theme is the intersection point where user and interactive e-book contact.
The theme of interface can be interpreted as the showcase or the face of the interactive ebook. The criteria in this theme consider aesthetic and visual design properties. The art of the designer appears in this theme and the criteria are related with usability features of the interactive e-book.
• Interaction: This theme determines the interactivity level of the digital book by using interaction design. The questions of interactivity such as what, when, where, how, why, and who are answered in this theme. However, it should be noted that interaction is not provided merely by using technology, but it is also provided by presentation of the content, design of the interface, and use of hardware capabilities of the e-book reader device.
• Technology: This is a challenging theme in contrast to the previous soft themes of evaluation criteria. Hardware features and their functions are related with this theme. On the other hand, in this study usually hardware features and functions rather than hardware were the locus of the study in interactive e-books, which is basically a software technology.
Comparing the results of the current study with  and Diaz (2003), in addition to content and interface themes previously defined, two additional themes appeared: interaction and technology. E-books which were examined by  and Diaz (2003)  In other words, interactive e-books promise more than reading experience; they promise e-reading experience which includes cognitive, sensorial, and physical interactions.
It is also seen that the interactive e-book criteria that emerged in this research are coherent with basic principles of four theories which indicate interactive e-books' value for ODL: • Theory of Independent Study: From correspondence study to computer-based distance learning, books have been the primary learning material, whether they are conventional or electronic. The purpose of using distance learning materials is to provide autonomy to learners so that they can learn independently. By providing a range of choices to individualize the act of learning, interactive e-books stand as an effective option for distance learners.
• Theory of Transactional Distance: Interactive e-books serve in increasing the level of interaction to eliminate physical and psychological barriers between learners and learning process and also provide individualized learning experiences by utilizing educational technology. The premise of interactive e-books lies in their ability to act like a learning environment more than a knowledge source in which meaningful learning can occur.
• Theory of Media Richness: The essence of the interactive e-books lies in the interaction provided. Interactive e-books use various media to increase and support interaction of the content using hard and soft technologies. The Media Richness Theory offers a solid walkthrough in order to design interaction and communication to augment learning through interactive e-books.
• Theory of Multimedia Learning: Meaningful learning through multimedia requires instructional design. Principles of Multimedia Learning Theory provides a proven guideline for designers of multimedia learning and helps to improve design of an interactive e-book to facilitate learning.

Conclusion and Future Directions
In this study, through a mixed research design in which Delphi technique and heuristic inquiry were used to gather data, four themes, 15 dimensions, and 37 criteria were developed to be able to evaluate interactive e-books. As a significant learning material, this study revealed important aspects of interactive e-books to ease access to information and to increase interaction, which are needed for meaningful learning experiences for distant learners.
In this study, a set of criteria was proposed to evaluate interactive e-books. Interactive e-books as a learning material encompasses many facets such as interaction design, instructional design, and interface design. Furthermore, hardware and software capabilities are directly related with the proposed evaluation criteria. Thus, the criteria revealed in this research can be developed according to changing needs of learners, and new hard and soft technologies. It is believed that the findings gathered in this research will: • help develop and evaluate interactive e-books, • help instruction and learning designers, software developers, and content providers by means of themes, dimensions and criteria identified in this research, • be a guide to identify the strengths and weaknesses of an interactive e-book, and • be a base for future research.
As a result of the Delphi study, heuristic design and literature review, the following implications can be taken into consideration for future research: • Interactive e-books use multimedia elements which provide rich communication opportunities. However, as a learning material, there is a need to identify the principles of multimedia use and design in interactive e-books to enhance learning experiences and to provide effective, efficient, and engaging learning opportunities.
• Interactive e-books use different communication channels and new technologies to reach users. Individuals with special needs (visual impaired, hearing impaired and orthopedically handicapped) gain advantages with rich communication channels, gesture based interaction, and natural user interface of new generation devices. Thus, it is advised to implement future research using universal design principles to meet needs of individuals with special needs.
• Interactive e-books are technology-oriented learning materials. As a source of learning and practicing, semantic and adaptive systems can be integrated to increase independency and autonomy of learners. Therefore, future research can focus on semantic and adaptive systems in interactive e-books to help users track their progress, receive feedback, and better assess and evaluate the level of learning.
As a final remark, this study does not claim that interactive e-books are superior to c-books or ebooks; conversely, it claims that interactive e-books are a good and flexible alternative to be able to provide individualized learning opportunities. On this basis, the themes, dimensions, and criteria can be used as a checklist to identify strengths and weaknesses of interactive e-books and can be helpful to guide researchers who are interested in interactive e-books as a learning material. It is believed that well designed interactive e-books can significantly contribute to learning process in an effective and efficient way.