Enriching Higher Education with Social Media: Development and Evaluation of a Social Media Toolkit

While ubiquitous in everyday use, in reality, social media usage within higher education teaching has expanded quite slowly. Analysis of social media usage of students and instructors for teaching, learning, and research purposes across four countries (Russia, Turkey, Germany, and Switzerland) showed that many higher education instructors actively use social media for private purposes. However, although they understand that their students also use it for learning purposes, and instructors sense the potential of social media in teaching, they mostly refrain from doing so due to various barriers. In response, an openly accessible trilingual Social Media Toolkit was developed which analyzes the teaching scenario with several questions, before suggesting, based on an algorithm, the best matching class of social media, complete with advice on how to use it for teaching purposes. This paper explains the rationale behind the toolkit, its development process, and examines instructors’ perceptions towards it.


Introduction
While social media usage becomes ubiquitous in everyday life, the same cannot be said for its application for educational purposes. In a recently finished EU FP7 project (Era.Net RUS) called Social Media as Catalyser for Cross-national Learning (SoMeCat), research was performed on the attitudes towards, and the use of social media among students and instructors in higher education (HE) for teaching, learning, and research purposes. Although there were some objections, most students, and to a slightly lesser degree the instructors interviewed, were positive towards utilizing social media for teaching and learning purposes in HE. Surprisingly, although various potential benefits of social media (SoMe) utilization for HE were sensed, in particular in comparison to traditional learning management system (LMS) usage, only very few instructors actually used it in their teaching, although many used it for private or research purposes. One major reason stated by instructors was an unsureness of how to soundly integrate SoMe in teaching from an instructional design perspective. Additionally, many instructors were unsure how to use SoMe in educational settings. Furthermore, instructors were found to be unaware of relevant support at their institution.
Based on the research findings, an electronic resource Social Media Toolkit was developed and placed online (http://socialmediaforeducation.org/). Available in three languages, it addresses two major barriers for using SoMe in HE for educational purposes: Firstly, from several questions it analyzes the instructional scenario where SoMe could be utilized and, based on an algorithm, suggests the bestfitting SoMe application. Secondly, instructional guidelines are provided on how to use the suggested technology in an educational setting. Providing this aid, the toolkit supports instructors in tapping the potential of SoMe utilization for educational purposes and increases its usage (social media application for learning and research purposes whilst part of the SoMeCat project, it is not within the scope of this paper). The purpose of this current study is to present the framework and design of the toolkit and explore users' perceptions.

Social Media in Education Literature Review
Social media is increasingly used in higher education (Dahlstrom& Bichsel, 2014) and has much potential for the educational context (Blazer, 2012). SoMe increases interaction, enhances communication, collaboration, active participation, allows for information sharing, discussion (Kilis, Rapp, & Gülbahar, 2014a), producing SoMe content (Johnson et al., 2013), and enhances critical thinking skills (Ajjan & Hartshorne, 2008;Mason, 2006;Mazman & Usluel, 2010). It also promotes individualization and facilitates personalized learning (Dabbagh & Reo, 2010). Furthermore, students can enhance self-regulation (Matzat & Vrieling, 2016) by tracking their own and peers' learning (Gammon & White, 2011). Similarly, Cao, Ajjan, and Hong (2013) offer practical and pedagogically supported arguments for SoMe, stating that "the primary argument for adopting social media in teaching is that social media applications provide multiple formats, directions and channels of communication, which can improve educational outcomes" (p. 583). Hung and Yuen (2010) demonstrate in their empirical study that SoMe opens wide opportunities for community of practice and holds "promise for enhancing students' sense of classroom community" (p. 713). Bosman and Zagenczyk (2011) explain SoMe in education from the point of the revised Bloom's Taxonomy of learning objectives and argue that SoMe tools extend possibilities for achieving different cognitive levels of learning, introducing examples of corresponding tools for each level. Cao et al. (2013) also verified one hypothesis; proposing that SoMe in teaching positively affects student learning. Taylor, King, and Nelson (2012) conducted a survey to explore practices and problems of academic use of SoMe; with results indicating that students have positive attitudes and value SoMe in online courses.
However, there are some barriers for effective SoMe integration reported by researchers. Hemmi, Bayne, and Land (2009) warn about volatile and challenging characteristics of SoMe that may influence established relations and principles in both teaching and learning, and should be taken into account while repurposing SoMe for education. Schroeder, Minocha, and Schneider (2010) argue that there are not only benefits, but threats and weaknesses of SoMe usage in education, e.g., increased workload for instructors and students, reliability, illegitimate use should be properly considered when instructors plan to use SoMe within courses. Regarding technology integration, Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, and Sendurur (2012) differentiate between two clusters of barriers: First-order barriers were defined as those that were external to the teacher and included resources (both hardware and software), training, and support. Second-order barriers comprised those that were internal to the teacher and included teacher's confidence, beliefs about how students learned, as well as the perceived value of technology to the teaching/ learning process. (p. 423) Various studies have shown that the latter caused greater challenges to technology integration than the former. Additionally, Hew and Brush analyzed literature published from 1995 to 2006 and identified: (1) resources, (2) knowledge/ skills of teachers, and (3) attitudes/ beliefs as the most frequently cited barriers (cited in Ertmer et al., 2012, p. 424). However, the major obstacle stated was a lack of knowledge (second-order barrier) and skills of how to effectively integrate SoMe from an instructional design perspective. Rogers-Estable (2014)  In summary, SoMe offers potential, but also poses barriers to achieving the goal of supporting instructors tapping the potential offered by SoMe in HE. Barriers are related to the skills and needs of teachers, whereas potential is more applicable for students. Therefore, it can be concluded that solutions should be provided to support teachers overcoming possible barriers to effective SoMe integration in HE for educational purposes.

Review of Project Outcomes
The SoMeCat project examined usage patterns and perceptions toward SoMe in education of 37 instructors and 107 students from four countries (Turkey: 12 instructors, 40 students; Germany: 10 instructors, 20 students; Russia: 10 instructors, 40 students; Switzerland: 5 instructors, 5 students).
The research indicated that few instructors were aware of the potential benefits of SoMe in education, and how to realize them. In parallel, although most students used SoMe daily, they were not used to or were against using SoMe for instructional purposes. However, when used properly, SoMe can create effective learning environments for students, with environments they value for individual gain, whilst effective use fosters instructors' awareness of potential benefits for both instructor and student. The research indicated that SoMe, as an effective learning environment, can support traditional courses.
SoMe makes it easier to reach students, to communicate with them at any time, both generally and beyond the confines of the classroom, and encourages peer learning, group work, and socialization among students. The findings are similar to findings of previously reported literature.
Our research also revealed some barriers. Instructors are both enthusiastic and skeptical about integrating SoMe within teaching and have pedagogical concerns about quality and student success.
Instructors stated a need for a roadmap on what to do, how to do it, and where to start. They lack relevant information on the process and are progressing only through self-motivated individual effort.
Again, findings were also parallel with the literature.
In response, the Social Media Toolkit was designed as a guide for instructors wanting to effectively integrate SoMe media within their courses. Employing universal terminology, geographic variations (Russia, Turkey, Germany, and Switzerland), various cultures, disciplines, and diverse requirements were addressed in order to ensure uniform comprehension of concepts by all instructors (users). The toolkit was designed to be straightforward and uncomplicated, preventing the sort of misunderstandings revealed from our research and as found in the literature. The following details the development and validation process.

Specification of the Social Media Toolkit
Based on the literature about social media integration, it was decided which core functions the toolkit should provide and which barriers cannot readily be influenced. While, at least initially, instructors' skills cannot be increased, their knowledge regarding SoMe utilization in HE teaching can, with support of the toolkit. Concerning factors revealed by Cao et al. (2013), the toolkit should primarily address task-technology fit, the most decisive factor explaining SoMe utilization of instructors: "The decision has to be based on the fit and compatibility between existing teaching tasks and social media technology capability, as the diffusion of innovation suggests" (p. 584). The toolkit can reduce the risk of SoMe utilization perceived by some instructors (Cao et al., 2013).
Core factors the toolkit should address were mainly the basic components of the teaching-learning process. Mao (2014) proposed that "it is important for educators to critically evaluate the adoption of social media in education rather than being driven by the latest craze in the technology market" (p. 222). It is important that social media technology should be used in conjunction with other course material and aligns with course objectives, assignments, and assessments (Callens, 2014).
Based on the barriers stated by instructors on deciding to use SoMe in HE and the literature discussed, the toolkit focused on two core functions: 1) Identification and display of best fitting social media class for the instructors teaching scenario, and 2) Provision of guidelines on using the technology for educational purposes.  Discussion), three of which are teacher-centered and three student-centered, were the instructional methods selected due to their common usage (Arends, 2011;Borich, 2013;Burden & Byrd, 2013).
Instructional method is important for instructional design and the selection of any corresponding social media tool. To ensure effective guidance, the toolkit employs instructional methods together with detailed explanations, including phases of implementation and a general summary.
Knowledge levels. The second dimension of the framework, knowledge levels, refers to learning goals based on the cognitive dimension of Bloom's revised taxonomy (Remembering, Understanding, Applying, Analyzing, Evaluating, Creating), the strong theoretical framework used both by educators and researchers (Anderson et al., 2000).
Explanation of each knowledge level, sample assessment types, and activities are provided separately to increase user understanding. Instructors can read explanations of each knowledge level, or review sample activities in order to determine the knowledge level the instructor wants students to achieve.
Based on these findings, plus three other domains (instructional models, assessment, content), the most suitable social media is suggested, allowing instructors to integrate SoMe into their educational settings. Assessment. The fourth dimension of the framework is assessment. Type of assessment has been addressed in many studies (e.g. Bennett, 1993;Birenbaum & Tatsuoka, 1987;Birenbaum, Tatsuoka & Gutvirtz, 1992;Shohamy, 1984;Traub, 1993) and results have indicated that types of assessment have different effects on student performance and learning. This reason was considered an important point in the decision for SoMe type, and so two general classification choices (Alternative Assessment, Classic Assessment) aim to match requirements of the diverse target group, although the literature also has other classifications of assessments -the first has assessments types as summative or formative (Garrison & Ehringhaus, 2011), while the other has classical and alternative (Presley & McCormick, 2007). Since there is no common paradigm for different countries' educational systems, classical and alternative assessments were used to develop the toolkit, being the more general terms of classification.
Classification of social media. In providing classifications for the toolkit, different kinds of SoMetools were examined and classification made according to Bower et al.'s (2010)  Scoop.it matrix to provide the best-matched social media type(s). Following posting the results, the toolkit also provides related tutorials, containing both general and educational information.
The end product has a graphical interface, and an individual platform developed using open-source software (as no open-source web application met the design requirements). The basic technology utilizes a LAMP server, a combination of Linux (server), Apache (web server), MySQL (data storage), PHP (programming language), PHP-based framework yii, and JavaScript library jQuery.  In summary, based on research findings and literature, a framework for social media has been developed. Social media has been classified and pattern-matching performed to constitute a decision matrix for an algorithm to determine the most appropriate social media type, based on instructor preferences. The toolkit then suggests appropriate social media types, supported by sample cases.

Teaching Preferences of Users
In a previous descriptive study of the social media toolkit, teaching preferences of instructors from a pedagogical point of view (frequency of knowledge levels, content types, instructional methods, and assessment methods) were determined; along with the respective social media suggestions (Kilis, Gülbahar, & Rapp, 2016). In that study, quantitative data was collected from 583 instructors in 39 countries. The findings indicated that instructors mostly prefer to teach at the applying and then the understanding knowledge level, by using text-based materials and designing their courses on problem-based or presentation models. They mostly favor alternative assessment methods, such as portfolios and group works, over classical assessment methods. The findings gave evidence that the social media toolkit is suitable for any instructor, from any discipline or culture, who wants to transform their traditional course to that of one supported by social media.

Methodology
The social media toolkit was designed and implemented based on research results and existing literature. The reliability of the decision matrix, the basis for the suggestions of SoMe classes to instructors, was then studied and verified, and finally, user acceptance was evaluated. The current study was designed as a descriptive study, gathering data to describe and explain events, and then to organize, tabulate, and depict the data (Glass & Hopkins, 2008). The aim of this current study is to explore the perceptions of instructors about SoMe toolkit. Instructors were requested to use the SoMe Toolkit and then answer six open-ended questions; hence, this study employs a qualitative approach to investigate perceptions towards the tool.

Data Collection and Sampling
Participants were selected based on purposive sampling. Instructors that use SoMe in HE were selected; the sample comprising 34 voluntary users from three countries (Turkey, 15; Germany, 10; Russia, 9). The research questions devised for this study are as follows: The data was collected using a feedback form given to users at the end of their review of the toolkit.
The form consists of six open-ended questions matching the research questions. The qualitative data was then analyzed inductively based on content analysis.

Reliability and Validity of the SoMe Toolkit
A pilot study was conducted to ensure the reliability and validity of the SoMe toolkit. Reliability of the algorithm that constitutes the base for the suggestions made was tested with data collected from 19 participants across Turkey. The participants include instructors, faculty members, and PhD students who are experienced in instructional/educational technology. All of them have taught with the support of social media in their courses, and therefore determined as qualified to be in the sample for this study. Measured with interrater agreement (IRA) method using Fleiss Kappa value due to having nominal categories, it revealed 0.83 and 0.86 for each social media type, and 0.76 to 0.90 for each domain, which indicates almost perfect agreement (Cohen, 1960;Fleiss, 1971;Landis & Koch, 1977) for social media type and the four domains. The toolkit was therefore deemed as reliable. Validity of the toolkit was measured via 16 expert opinions sought from four countries (Turkey, 5; Germany, 3; Switzerland, 4; Russia, 4). Experts included instructors and faculty members considered qualified in teaching with social media. Data was collected via online feedback embedded in the toolkit website, with experts each asked six open-ended questions. The experts' opinions indicated the toolkit appropriate in terms of content and usability. Content validity of the toolkit was ensured, and overall, the toolkit was found as reliable and valid (Kilis, Rapp, & Gülbahar, 2014b).

Perceptions about Scope/Theoretical Background of Toolkit Questions
Twelve of 29 users were satisfied by the scope of the SoMe toolkit, whereas 15 added suggestions to enhance it, and two responded negatively. Suggestions with the number of users in the parentheses accordingly included: the requirement for more specific questions (n=2), information about the sample size (n=2), and instructors' preferences and familiarity about social media types (n=2). One Turkish user stated, "Not more questions, but more distinctive questions are needed. An instructor will want to use all the options presented here in his/her course." One German user suggested to "Include the current state of knowledge of the students and some institutional characteristics." One Russian user suggested, "It's necessary to ask questions about information sources in the learning process (library, internet, original works, etc.)." In general, users found that the questions in the toolkit were efficient but could be enhanced.

Perceptions about Instructional Tips
Users responded very positively to the instructional tips in the SoMe toolkit. Almost all participants stated they found it very useful and facilitative. Few made suggestions to enhance the instructional tips or about their needs. On positive responses, one Turkish user stated "Yes, I think they are pretty useful in that they provide a short but detailed description along with the examples of various uses," one German user stated "The tips are already very helpful in view of broadening horizons. They also contain concrete ideas of tools and how to use them," and one Russian user said that Instructional tips offered in the tool contain detailed, structured information about the social media that are mostly used in practice. They propose most effective, rational variants of actions as applied to the teaching process, with the help of the tips it is possible to define the structure, focus in studying a topic, conducting lessons, etc.
As for suggestions, one Turkish user stated, "I think which platforms support what kind of activities should be clearly shown. I think many instructors are unaware of the potential of the platforms. The model should include a clearer link between activities and the type of platforms." One German user said, "Maybe instructional tips should be more detailed and appear on the screen step-by-step according to the new questions coming up." Overall, users were very positive about instructional tips.

Perceptions about Additional Information
Users

General Thoughts about the Toolkit
Of the 34 participants, 27 responded positively about the toolkit overall. One Turkish user stated, "Declaration of social media types according to educational context is very informative; these kinds of applications can improve the number of potential users." One German user said, "It is a good help in identifying a variety of options; one can implement a simple and cheap solution without limiting the functionality in a fundamental way." One Russian user declared that: The tool solves one problem -saving valuable time that is spent studying social media and becoming familiar with the work of each tool, what is necessary for a precise choice of a suitable tool(s) from the variety. Therefore, instructors who not only understand the material to be taught but also master internet technologies are needed. The tool permits an instructor to choose in a short time the most suitable type of social media for the course and after that to focus on studying and applying the tools. Thank you for your work, excellent idea!!! In addition, some made suggestions to enhance the toolkit in general; for example, one Turkish user said, "A very useful website, but more examples on educational use would enhance the site content," and a German user suggested adding a function to save proposals and the possibility to add own scenarios or to have own practice evaluated.
Overall, users found the toolkit supportive and useful within educational settings; however, they still require more examples and sample uses.

Perceptions about the Usability of the Toolkit
Of the 34 participants, 25 favored the usability of the toolkit, whilst others offered some suggestions.
One Turkish user said the "Design and colors are appropriate, navigation is easy. The system has a high usability factor" and a Russian user said, "I think the tool is quite useable, easy to use. First of all, it's easy to use because any user can perform a search (navigate) and find what s/he is looking for." Other suggestions include a Turkish user who stated that "Sometimes I want to switch between the page and the pop-up, but I have to close the pop-up in order to read the previous screen." On the same issue, one German instructor said, "It is difficult to navigate back from the results page to previous questions." Alternatively, two German instructors stated that "There is too much text before one can start the query wizard." All Russians participants were very positive and made no suggestions. In terms of usability, it appears that no problems exist and users had no difficulty with navigation and access to information. In conclusion, the developed toolkit was accepted as appropriate and easy-touse by the majority of respondents.

Additional Comments
For this last question, 15 of the 34 participant users responded. Some Turkish instructors valued the project and research, with one stating, "This project could be enhanced through collaboration with researchers who study ontology and personalized educational environments." On the other hand, one German user stated "Good idea! I am looking forward to seeing how the toolkit will be improved." A Russian instructor wondered about new tools and questioned "Are you going to explore whether there are new, more suitable social media? Are you planning to update the content, namely instructional tips and tools (if there are new, more practical)?" One German instructor reported that "The difference in the percentage of the results for the six social media classes was low. The proposed social media types are too general." Another German instructor was surprised by the suggestion made by the toolkit and claimed it required further testing of the algorithm, which has already be performed.
Problems with data security/privacy related to the suggested services were raised by only one instructor.
This developed toolkit is an initiative and will continue to be developed. Some deficiency fixes and/or new requirements are possible, and in time, additional research may further develop the toolkit.
Overall, the feedback is very valuable to the continued improvement of the SoMe toolkit, enriching it to the benefit of all educators.

Discussion and Conclusion
Social media offers great potential for education purposes in HE. In this paper, the most critical barriers to SoMe integration in HE were identified. Those that could potentially be addressed by an open accessible resource (SoMe toolkit) were singled out with the aim to support instructors tap into the potential that social media offers to enrich teaching. The focus was on providing two key features for supporting instructors: (1) analyzing the teaching scenario and suggesting the best-match class of social media from an instructional design perspective, and (2) providing guidance on the suggested technology's usage for instructional purposes. As no similar product exists, there were no guidelines on how to develop such a resource and which parameters to take into consideration. This was a potential limitation, although as shown, the approach is theory driven. Consequently, to a certain extent, the choices (such as "why those four questions") is subjective and therefore open to discussion.
One crucial step was to evaluate the toolkit, with both the validity of the decision matrix (the basis for the algorithm making the SoMe class suggestion), and the usefulness from users' perspectives.
Considering all the participants, and the fact that they were from three different countries, the feedback was generally very positive. Useful suggestions were received for potential future enhancements, especially in terms of pedagogical issues. In summary, users were very positive toward the toolkit, finding it useful and easy-to-use. They also offered valuable directions for future development, and as a next step, a section where instructors can upload best-practice examples of SoMe utilization in HE for discussion may be useful, and could even kick-start a Community of Practice. As research only extended to users in some European and Asian locations within the scope of the project, generalizability of the results was limited to some extent.
However, in the development of the toolkit, quite diverse requirements of the target group were considered, even though this current study is limited due to a small sample size. Further research with a larger sample size from different disciplines and/or cultures is therefore recommended. However, the generalizability of the results was enhanced by having chosen participants from three countries.
Whether or not the toolkit also matches the requirements of other users was not tested; a potential enhancement or future research. Moreover, examining the level of confidence of instructors prior to, and after using the toolkit, and the likelihood of them using social media as part of their next course would be beneficial.
With regard of the evident challenges that need to be addressed by educational institutions, and in particular the instructors, in benefitting from using social media tools in teaching/learning (Greenhow & Lewin, 2016), the developed toolkit could be seen as advantageous; a guide to steer them in the right direction. Since being an original attempt to provide guidance to instructors regarding the effective selection and integration of SoMe into their HE courses, the SoMe Toolkit is expected to be valued and used in higher education. Instructors that want to implement online or blended-learning courses are expected to benefit from the toolkit: They can either supplement their courses with different social media services or completely migrate their courses onto social media platforms in accordance with their requirements. For example, they can easily form student groups to work in Wikis for any shared goals in order to enhance peer-to-peer interaction (Kaplan & Haenlein, 2010). Alternatively, they can create an effective discussion environment within any online/blended learning environments (English & Duncan-Howell, 2008;Mazer, Murphy, & Simonds, 2007). The online instructors can benefit from SoMe in order to create effective collaborative learning communities. The positive effect of SoMe on the students' community of inquiry was addressed more recently (Lim & Richardson, 2016;Öztürk, 2015); contributing to the students' social, cognitive, and teaching presence through the facilities of social networking services. Dissemination and enhancements to the toolkit can provide an insight to