Enriching the Web of Data with Educational Information Using We-Share

This paper presents We-Share, a social annotation application that enables educators to publish and retrieve information about educational ICT tools. As a distinctive characteristic, We-Share provides educators data about educational tools already available on the Web of Data while allowing them to enrich such data with their experience using technology in the classroom. We-Share evaluation entails an empirical study where 23 educators enriched tool descriptions available on the Web of Data out of their own experience. The results suggest that experiential annotations published by educators using We-Share improve the satisfaction and confidence of other educators when discovering and selecting ICT tools. Further, most educators found We-Share an easy-to-use application suitable to share and retrieve information about educational ICT tools.


Introduction
One of the problems that educators need to face when designing a technology-enhanced learning situation is the selection of Information and Communication Technologies (ICTs) tools ("tools" hereinafter) to support it. This is a critical decision (Gómez-Sánchez, Bote-Lorenzo, Jorrín-Abellán, Asensio-Pérez, & Dimitriadis, 2009) that requires them to be informed about not only the functionality of the tools available or their technological features, but also about their suitability to the contexts (Kurti, Spikol, & Millard, 2008) and the pedagogical intentions of the learning situation. As several educators typically employ search systems in order to be informed about the tools available (Conole, 2008;Madden, Ford, & Miller, 2005), several tool search systems have been proposed for them to find this information (e.g., Cool Tools For Schools). The precision of these search system is higher than general-purpose ones (e.g., Google) because they focus on a specific domain; however, they suffer from common data sustainability problems that limit their utility (Ruiz-Calleja, Vega-Gorgojo, Asensio-Pérez, Bote-Lorenzo, Gómez-Sánchez, & Alario-Hoyos, 2012): the isolation of their datasets, which increases the overall effort required to sustain them; and the well-known cold-start problem (Maltz & Ehrlich, 1995), which hinders the collection of data from the users in sociallyconstructed registries. In order to overcome these problems we developed SEEK-AT-WD (Ruiz-Calleja, Vega-Gorgojo, Asensio-Pérez, Bote-Lorenzo, Gómez-Sánchez, & Alario-Hoyos, 2014), an infrastructure that exploits the data available on the Web to automatically create-˗and periodically update--its tool dataset.
SEEK-AT-WD currently exposes more than 7000 different tool descriptions that could be exploited by educational search systems. These descriptions include functional and technical characteristics of the tools. However, existing data about tools in the Web of Data do not include information about how tools are employed in educational contexts, which hinders the support provided by this data to the educators. As an example, an educator may discover several concept map tools out of the data already published on the Web, but it will be hard for her to select an appropriate one for her classroom unless she knows which ones have already been used in similar educational contexts. SEEK-AT-WD is technically prepared to provide this education-specific information (Ruiz-Calleja et al., 2014) but how to collect it remains a problem. A possible solution is to combine the Social Web and the Web of Data advantages in a social-semantic approach (Mikroyannidis, 2007). Such an approach involves the federation of a tool dataset to the Web of Data while supporting the community of educators when enriching or curating the dataset. Thus, the resulting dataset will be automatically updated from noneducational data sources and, at the same time, it will contain education-specific information published by its community of users.
How to enable the community of educators to contribute to the sustainment of the dataset publishing educational-specific information is key for this social-semantic approach. With this aim, the present paper proposes We-Share, a social annotation application that relies on SEEK-AT-WD to support educators to publish, enrich, and consume semantic descriptions of tools. We-Share offers a convenient and easy-to-use web interface that allows educators to annotate and search tools for their needs. Further, this paper provides empirical evidences of the usefulness of the social information published with We-Share. Specifically, it describes a case study where 23 higher education practitioners from multiple disciplines published information on the Web of Data using We-Share and use it later on to discover and select tools. Thus, this proposal assesses the feasibility of enriching data from the Web with social educational information and also motivates the use of third-party data from the Web for educational purposes.
The rest of the article is structured as follows: the next section depicts current social and semantic approaches followed to sustain registries of educational tools. Then, the following section presents the design and development of We-Share. The paper continues by including the description of the empirical study and the discussion of its results. Finally, the most important conclusions are presented in the last section.

Crawling the Web of Data to Gather Educational Tool Descriptions with SEEK-AT-WD
The Linked Open Data movement promotes the publication of data on the Web with an open license and following the Linked Data principles (Berners-Lee, 2006). These principles define a set of best practices for data publication that facilitate the consumption and integration of datasets. Thousands of data providers, including educational ones (d'Aquin, 2016), are linking their datasets on the Web of Data and third parties are reusing this data to build applications.
Some registries of educational tools publish their content on the Web with an open license, such as Cool Tools For Schools or Moodle Plugins Directory. However, reusing their data requires a big effort because they use their own data formats and schemas. This fact hinders the data integration and makes them suffer data sustainability problems, as all their data should be publish by their community of users, thus increasing the overall effort of getting tool descriptions. Further, they suffer from the so-called "cold-start" problem since, at the beginning, these registries include very few tool descriptions, which refrains the community of educators from spending their time publishing information in them. In this regard, a step forward has been taken by the ROLE Widget Store and Ontoolsearch (Vega-Gorgojo, Bote-Lorenzo, Asensio-Pérez, Gómez-Sánchez, Dimitriadis, & Jorrín-Abellán, 2010), which publish the functional and administrative characteristics of several tools as Linked Open Data. Even though, as they are unable to import data from third-party registries, the same sustainability problems still remained.
Reusing existing Linked Data can help avoid the cold-start problem and the isolation of datasets . This is the approach followed by SEEK-AT-WD (Ruiz-Calleja et al., 2014), which uses the descriptions of tools available on the Web of Data to create, and periodically update, a dataset of educational ICT tools. Once a week a crawler extracts tool descriptions from the Web of Data and relates them to an education-specific vocabulary, called the SEEK Ontology (Ruiz-Calleja et al., 2014). This ontology defines a vocabulary that can be used to describe tools, reviews related to these tools, and educational contexts. By January 2016, SEEK-AT-WD retrieved more than 7000 different tool descriptions, which, once related to the SEEK Ontology using ontology mapping techniques (Choi, Song, & Han, 2006), are offered back on the Web of Data to be used by third-party applications (note that not all these descriptions were available on the evaluation presented in this paper since new tool descriptions appeared on the Web of Data since then).
SEEK-AT-WD currently offers its data through a browsing interface and a querying interface that follow the Linked Data principles. As these interfaces are not suitable for end users, a search system is required to enable educators to construct and submit queries to SEEK-AT-WD, as well as to visualize the results obtained. Another limitation of SEEK-AT-WD is the current lack of data published on the Web about the educational use of the tools, which is specially relevant for the appropriate selection of tools by educators (Gómez-Sánchez et al., 2006) (Kurti, Spikol, & Millard, 2008). Specifically, the SEEK Ontology includes a set of concepts to describe educational reviews of the tools and the educational contexts where these tools were employed (Ruiz-Calleja et al., 2014). Even if no data related to these concepts can be currently obtained from the Web, the possibility of semantically describing educational contexts is a very interesting feature of SEEK-AT-WD, since they could eventually be exploited when selecting tools and when reviewing them (Gómez-Sánchez et al., 2006;Kurti, Spikol, & Millard, 2008). This paper explores the possibility of enabling educators to publish this kind of information on SEEK-AT-WD and, as a consequence, on the Web of Data, following a social-semantic approach (Mikroyannidis, 2007).

Social Annotation of Educational ICT Tools for the Web of Data
The Social Web (O'Reilly, 2007) is a web paradigm in which users move from a purely informationconsuming role to be information publishers. One of its best achievements is to collect and aggregate knowledge into community-and domain-specific sites (Gruber, 2008;O'Reilly, 2007), which are commonly called "collective knowledge systems" (Gruber, 2008). Several examples can be found where collective knowledge systems are employed to recommend tools to educators. They all have to face the problem of who and how describes the educational use of the tools so that such information can be incorporated in their datasets to facilitate the discovery and selection of tools. Some tool registries, such as Schoolforge, separate the users that publish tool descriptions (e.g., tool providers) from those that consume them (e.g., educators). Tool providers are expected to contribute since they are interested in advertising their tools and they deeply understand their functionality and their technical characteristics. However, tool providers cannot always anticipate the educational affordances of their tools. In other tool registries, such as Cool Tools For Schools, the community of educators is not only expected to obtain tool descriptions, but also to publish descriptions of the tools they use in their classrooms. Finally, there are other registries (e.g., ROLE Widget Store) that support both roles in contributing to the data publication, thus enabling tool providers to publish their tools and educators to enrich the tool descriptions with education-specific information.
However, all these socially-constructed tool registries are isolated data silos and, hence, they suffer from the abovementioned sustainability problems. Furthermore, the educators can only review tools by stating a numerical rating or writing an unstructured text. None of these socially-constructed registries include a set of concepts to explicitly state the characteristics of the educational use of the tools, as SEEK-AT-WD does with the inclusion of the educational contexts in its vocabulary. This lack of structure in the reviews simplifies their publication but has some well-known disadvantages (Mikroyannidis, 2007). For instance, they lack a coherent categorization scheme (e.g., it is not possible to list all the reviews that refer to blended scenarios) and their concepts cannot be used to express formal queries (e.g., it is not possible to submit queries such as "I need a voting tool that has been employed to support a face-to-face debate").
The idea of merging social annotation with semantic descriptions has already been explored in the educational domain (Jovanovic, Gasevic, Torniai, Bateman, & Hatala, 2009), although this kind of applications typically behave as isolated data silos. Some examples of applications that combine data from the Web with information published by their community of users can be found in other domains.
For instance, Revyu (Heath & Motta, 2007) is a simple application that enables its users to publish on the Web of Data reviews of any entity. Other example is presented by Heitmann and Hayes (2010), who proposes to use data from the Web to mitigate the data acquisition problems of current recommender systems in a similar way as SEEK-AT-WD does. This kind of systems can be accessed through an application that hides the complexity of publishing semantically-structured Linked Data to their users. In the case of SEEK-AT-WD, such an application is required to enable educators to socially publish and curate semantic descriptions of tools as well as to review them.

We-Share
We-Share is a social-semantic annotation application that enables educators to publish information about tools on the Web of Data using the SEEK-AT-WD infrastructure (see Figure 1). We-Share supports the publication of tools, educational reviews, and educational contexts (as described by the SEEK Ontology), as well as updating the already existing ones. This data publication entails the management of user accounts to state the authorship of the information published on We-Share and the edition rights of each user. Further, We-Share enables educators to retrieve (either browsing or querying) and visualize the data published on SEEK-AT-WD in a single environment, as recommended for social annotation applications (Neuwirth, 1990). We-Share also offers a usable user interface that hides the complexity of managing semantic data since educators are not required to have technical skills.

Logical Architecture of We-Share
The logical architecture of We-Share consists of the set of components shown in Figure 2, which are arranged in a three-tier architecture. This architecture explicitly separates the elements that conform the user interface, the presentation tier, from those that implement the processes needed for the system to provide the required functionality, the business logic tier, and those that persist the data generated and consumed by these processes the data tier. It is also noteworthy that all the information about tools managed by We-Share is stored in SEEK-AT-WD so as to be publicly available on the Web. Nonetheless, the information related to We-Share users is stored in a separate dataset since their privacy should be guaranteed. The separation of concerns stressed by the proposed architecture enables the modification of the visual appearance of the user interface without affecting the other components of the system. We-Share is a data annotation and consuming application (see Figure 1) that directly uses SEEK-AT-WD data interfaces without requiring to modify the infrastructure. As a consequence, the Query Manager and the Browse Manager not only retrieves data from SEEK-AT-WD, but also parse it to be presented by the presentation tier. Similarly, the Data Storing Module aligns the data published by the users to the SEEK Ontology before submitting this data to the infrastructure.

We-Share user Interface Design
The design of the user interface of the system is specially challenging since it should facilitate educators to interact with tool descriptions while hiding the complexity of their computational representation. Four different user interfaces need to be designed for We-Share, as shown in Figure 2.
The Query Interface of We-Share is based on a former search system called Ontoolsearch (Vega-Gorgojo et al., 2010), which proposes a direct-manipulation interface to guide educators in the construction of formal queries. It includes a graph-based interface that depicts the hierarchy of concepts defined by the ontology, thus enabling educators to browse these hierarchies and select the concepts that will be used to filter results. In a similar way, the Query Interface depicts the hierarchies of tools, educational tasks and artifacts defined by the SEEK Ontology. Additionally, this interface includes two forms that enable educators to filter the tool descriptions using the characteristics of the educational contexts where the tools have been employed, the technical characteristics of the tools, or the keywords contained in their description. The descriptions obtained are presented to the educators as a list of tool names, listing first those that have been reviewed. As an example, Figure 3 shows part of the tool graph included in the search interface. Educators can select concepts from this graph, add them to their query, and use them to filter results.
The Browse Interface enables educators to browse SEEK-AT-WD knowledge base by representing the descriptions of its tools and educational contexts. It also enables users to access collections of entities, such as the list of tools that belong to a specific tool type or the ones that support a specific educational task. As an example, Figure 4 shows part of the description of Wikispaces as represented by We-Share. This description includes data that was extracted by SEEK-AT-WD from the Web of Data and the educational reviews published by educators. Each review includes a textual description, and the main characteristics of the educational context where the tool was employed. It is also noteworthy that We-Share offers information that was initially obtained from diverse non-educational external sources of the Web of Data and educational-specific information published by educators.

Prototype Implementation
We developed a prototype of We-Share as a Drupal-based web application, which is currently available at http://seek.cloud.gsic.tel.uva.es/weshare/. The development effort was highly reduced using Drupal since it includes user and data management facilities that can be directly taken, or easily adapted, to We-Share needs. Furthermore, Drupal natively supports the publication of RDF data and its architecture follows the same three-tier architecture as We-Share does. Technical details about We-Share development can be found at (Ruiz-Calleja, Vega-Gorgojo, Asensio-Pérez, Bote-Lorenzo, Gómez-Sánchez, & Alario-Hoyos, 2013).
Different versions of the We-Share prototype were tested by several educators. They used We-Share to search and publish tool descriptions providing useful feedback that helped to improve its interface.
The data published by these tester educators was available during the evaluation of We-Share, even if none of them took part in this evaluation.

Evaluation
The aim of this evaluation is to assess whether it is possible for We-Share to gather useful information from the community of educator for SEEK-AT-WD data sustainment. The evaluation follows the CSCL-EREM framework (Jorrín-Abellán & Stake, 2009) to guide the collection of evaluation data, its analysis, and its interpretation. CSCL-EREM distinguishes from other evaluation frameworks since it is evaluand-oriented, meaning that it guides researchers to focus on what is being evaluated. It highly encourages researchers to gather and analyze data about the evaluand using a mixed methods approach (Tashakkori & Teddlie, 2003), which combines the gathering and analysis of both qualitative and quantitative data, and is especially suitable to understand social phenomena.
Specifically, this evaluation adapts the mixed evaluation method described in (Martínez, Dimitriadis, Rubia, Gómez-Sánchez, & de la Fuente, 2003). This method combines quantitative and qualitative data gathering techniques and triangulates the data gathered to comparatively analyze them in order to reach the global conclusions of the evaluation. Thus, the evaluation process is enriched as multiple perspectives about the same evaluand are obtained.
We also followed the anticipated data reduction procedure proposed by (Huberman & Miles, 1994) in order to conceptually organize the data gathered. This data was arranged according to three topics, defined in Table 2. The first one aims to detect the usability problems that may arise due to the fact that We-Share reuses information from the Web of Data. The other two topics assess the impact of the social information collected by We-Share. This social information illustrates how the tools were used in other learning situations pursuing specific pedagogical objectives, and is also used, in conjunction to other information collected from the Web of Data, by educators to discover and select tools. Hence, we evaluated whether this social information increases the satisfaction of the educators when they discover tools and whether it increases their confidence when they select a specific tool for their classrooms.
Twenty-three higher education educators were voluntarily involved in this evaluation. They first published information on We-Share. Then, the impact of the information published was assessed through a formal comparison of We-Share versus a search system that does not manage social information. This comparison is based on the interactive track of the Text REtrieval Conference (TREC) workshops (Lagergren & Over, 1998), which are commonly used in the evaluation of interactive information retrieval systems. Figure 5 graphically represents the design of the proposed experiment. This design involves two differentiated phases: a first part, tool publication, where participants used We-Share to publish information, and a second part, tool information retrieval, where the same participants searched for tools using two search systems. As participants belong to universities from different countries, an asynchronous e-mail-mediated experiment was designed, as described next.

Experimental Setup
First, the participants received an e-mail, asking them to fill in a questionnaire (Q-knownTools) and enumerate some of the tools they had educational experience with. Then, each participant received an e-mail with a video tutorial about the data annotation in We-Share and the list of the tools she was asked to add or to review. From the tools she had experience with, each participant was asked to review those that had previously been published on SEEK-AT-WD, and to add a description and a review to those that had not been published yet. After using We-Share to publish information, the participants filled in another questionnaire (Q-publication) that contained a Likert-Scale that asked for their perceived usability of We-Share regarding data publication. The second part of the experiment included a formal comparison between We-Share search system and a non-social search system implemented as a stand-alone application described in (Vega-Gorgojo, Ruiz-Calleja, Asensio-Pérez, & Jorrín-Abellán, 2012). The latter also collects data from SEEK-AT-WD but does not manage the social information included in the infrastructure. Six search tasks (TA-TB-TC-TD-TE-TF) were chosen for the study (see Table 1). As the search tasks were intended to be representative of real information needs, they were selected from the pool of questions posed by educators to evaluate the initial dataset of SEEK-AT-WD (Ruiz-Calleja, et al. 2012). Table 1 Overview of the Search Tasks used in the Experiment ID Brief description TA I would like a tool to edit video clips (it would be better if it also allows to edit audio and images). This tool has to be free. TB In an on-line course I ask my students to do a written exercise and then review the work of two of their partners. I would like a tool to support this peer review activity. TC I would like a tool that facilitates meetings between people that are physically in the same place and other people who are not. The communication cannot be based on textual messages. TD I am a teacher of a course of engineering. I would like a tool to make questionnaires to my students, as well as to ask their opinions. TE I would like to recommend my students to use LaTeX and, as most of them are Windows users, I would like to recommend a LaTeX editor that works on Windows. TF In a face-to-face laboratory I want my students to create a concept map with the most important concepts of the subject. I would like a tool for them to do this task.
When the second part of the experiment began, each participant received an e-mail with a video tutorial that explained how to search tools with both systems and four information needs that they were asked to satisfy. The search tasks assigned to each participant and the performance order were randomly selected. The participants were asked to search for tools that would satisfy the information need and to select the one they would employ in the given situation. They were asked to fill in a questionnaire (Q-search) for each information need where they reported their satisfaction on the tools discovered and their confidence on the tool selected in a 1 to 10 scale (these were the metrics used to compare the performance of both search systems), the tool they chose, the reason why they chose that tool, and any other observation they wanted to state. When they finished all the search tasks they completed another questionnaire (Q-exit) where their global opinion about We-Share utility and usability was reported using a Likert-Scale.
The satisfaction on the tools discovered and the confidence on the tools selected were computed for each observation. The factors that were considered to influence the satisfaction on the tools discovered were the search system (the primary factor) and the search task. On the other hand, the confidence on the tool selected was considered to be affected by the search system (the primary factor), the search task and the satisfaction on the tools discovered. A quantitative analysis of the results was carried out, which included an Analysis of Variance (ANOVA) model, thus addressing the question of whether these two metrics were significantly different using both search systems.
Furthermore, a deeper comprehension of the search processes was obtained by analyzing We-Share logs and the opinions of the participants collected from the different questionnaires. In this study, 23 educators from 12 different universities and 4 different countries participated as nonrewarded volunteers, although 2 of them did not completed the second part. The participants did not have any kind of relationship among themselves and most of them did not know each other. They had teaching experience in Computer Science (10), Telecommunications Engineering (9), Pedagogy (3), and Psychology (1). They all had experience using Social Web applications and employing technology for educational purposes.

Findings on the Data Publication Process
The participants published 23 tool descriptions, 39 educational contexts, and 80 educational reviews.
All this data was combined with the data available in SEEK-AT-WD, which had been previously Enriching the Web of Data with Educational Information Using We-Share Ruiz-Calleja, Asensio-Pérez, Vega-Gorgojo, Gómez-Sánchez, Bote-Lorenzo, and Alario-Hoyos 258 obtained from the Web of Data or published by the tester educators who provided feedback about We-Share interface, as Table 3 summarizes. The participants' opinion about the data publication was collected and the information they published was analyzed. Table 4 collects the results of the Likert-scale questionnaire regarding the data publication in We-Share (Q-publication). It can be seen that most of the participants agreed or completely agreed that We-Share is an easy-to-use annotation application. Most of them understood the meaning of its menus and they found it easy to interact with them. Additionally, the publication of tool descriptions and reviews in We-Share was not perceived as a difficult task by most of the participants. In fact, they properly published tool descriptions, reviews, and educational contexts, and all the educators made a correct use of We-Share.
Most of the comments posed by the participants regarding the data publication were positive. A few participants complained that We-Share requires to understand the SEEK Ontology in order to properly classify tools and relate them to educational contexts. This "cognitive cost" (Mathes, 2004) is a well-known problem that appears when users relate the information they publish to a given vocabulary, since they have to do the effort of understanding such vocabulary. Despite this, participants socially agreed on the meaning of these concepts by reading some examples of tools and educational contexts that had already been published in We-Share. As the same participants that complained about this aspect, agreed or strongly agreed that We-Share is a usable application, it can be considered that they understood this cognitive cost as part of the learning process required to annotate tool descriptions with We-Share.

Findings on the Tool Discovery Process
During the second part of the evaluation, a total amount of 84 tool discovery processes were carried out. In 82 of them, the participants could find useful tools to satisfy the search tasks requested. The two discovery processes that could not be completed corresponded to the first time two different participants used the We-Share search system, so they may be due to the learning process required to get used to We-Share. Moreover, only in four observations, the satisfaction when discovering tools reported was lower than 5 out of 10. An ANOVA model was fitted for the satisfaction with the factors considered in the design. A valid model was obtained and it showed that the difference of the participants' satisfaction using both search systems was not statistically significant. In order to further understand the impact of the social information on the participants' satisfaction about the tools discovered, the mean of the satisfaction was calculated for each of the search tasks, as shown in Table 5. It can be seen that when the social information is available the satisfaction is higher, although only slightly better results are obtained.
The queries submitted by the participants were also analyzed. 255 queries were submitted for 42 search tasks without social information and another 256 for 42 search tasks with social information.
The quantity of search tasks required to satisfy the information needs and the type of filters used were very similar in both search systems. In fact, when the social information was available only 25.1% of the queries submitted used concepts related to educational contexts to filter results. As most of the queries submitted by the participants with both search systems used the same filters, in most cases the results obtained were the same. This fact explains why the satisfaction on the tools discovered is so similar. In questionnaire Q-exit the participants were also asked about their opinion of the usability of the We-Share search system (see Table 6). As a general rule, the participants found it easy to search for tools using We-Share. Specifically, 14 out of 21 participants considered easy to obtain tool descriptions using We-Share and 18 out of 21 found it an easy-to-use search system. Additionally, some participants made comments about the usability of the search system. Some of them explicitly said that they liked the interface. Nonetheless, a few participants pointed out two well-known usability problems: their need to understand the SEEK Ontology and the difficulty to translate a search task into a boolean expression with precise semantics. These are two well-known problems in information retrieval systems that manage structured data that were also detected in other search systems (e.g., Vega-Gorgojo et al., 2010).

Findings on the Tool Selection Process
During the second part of the evaluation process the participants also reported their confidence on the tools they selected on each search task. An ANOVA model was fitted with the participants' confidence on the tools selected and the factors considered in the design. Then, the ANOVA assumptions were checked in order to assess its validity (Jain, 1991). For a 90% confidence interval the difference of the confidence of the tools selected with We-Share with respect to the non-social search system is in the interval (0.0283,1.4053) with mean 0.7168. This difference shows that the search system employed has a statistically significant influence on the confidence of the tools selected by the participants, having better results We-Share. Note. Each tool selection can be based on several types of arguments. Table 7 shows the type of arguments reported by the participants when selecting tools using both versions of the search system (note that each tool selection can be based on several types of arguments). It can be seen that when the social information was not available, the functionality of the tools, and the previous experience of the participants were the two main arguments for their selection.
On the other hand, when the social information was available, these two arguments were also very frequent, but many educators also based their selections on the reviews and the educational contexts related to the tool descriptions. Interestingly, when the tool selection was based on the experience of other educators, the reported confidence was usually higher. In fact, only one of them had a confidence lower than 7 out of 10, and their arithmetic mean (8.20/10) was higher than the mean of the confidence in the rest of the experiments (7.24/10). There were also differences on the tools selected by the participants when the social information was available. Table 8 shows, for each search task, the fraction of the tools selected that had reviews attached to them (note that when using the non-social search system the participants did not know which tools had reviews). When the social information was not available, educators selected tools with reviews in 19 out of 41 cases, while these tools were selected in 32 out of 41 cases when the social information was available (68.4% of increase).
Q-exit asked participants about their perception of the data added by the educational community using We-Share. Fifteen out of 18 participants agreed or completely agreed that the data published by the community of educators is apparent to them. Additionally, this questionnaire asked participants for their perceived utility about We-Share. Fifteen out of 21 participants found We-Share a useful tool in their job, while 18 out of 21 participants agreed or completely agreed that the time spent to discover and select tools using We-Share is reasonable.

Discussion of the Evaluation Results
The evaluation entailed a group of educators who used We-Share to publish, enrich, and retrieve tool descriptions from the Web of Data. Some information related to the usability of We-Share was collected, showing that the participants did not find special problems to annotate tool descriptions, nor for coherently combine the information they publish with the tool descriptions already available on the Web. Further, the problems detected when participants searched tools using We-Share were very similar to those detected in other evaluations where end users submit semantic and boolean queries to a manually-created dataset. These are remarkable findings since they show that tool descriptions from the Web of Data can be combined with social educational-specific information to support educators when discovering and selecting tools. Thus, We-Share can exploit the sustainability advantages of reusing updated data from external sources of the Web while it is still able to provide educational-specific information about tools.
Another important conclusion of the evaluation is related to the impact of the social information when discovering tools. The reported satisfaction of educators when discovering tools slightly increased when the social information was available. The analysis of the queries submitted by the participants showed that the participants did not make extensive use of the parameters that filter tools depending on their educational use. For this reason the social information was not fully exploited during the experiment. Even though, the social information had an impact on the participants when selecting tools. When this social information was available, their confidence when selecting ICT tools significantly increased. Further, many participants justified their tool selection based on this social information and they tended to select those tools that had been reviewed by other educators.
All these findings support the conclusion that We-Share is useful to enrich with educational-specific information the information about tools that is published on the Web of Data. Nonetheless, the results of this evaluation are affected by several aspects. One of the most important ones is the SEEK Ontology, which defines the parameters to describe tools and educational contexts and restricts the formal queries that can be submitted to SEEK-AT-WD. Another aspect is the data obtained by SEEK-AT-WD from the Web of Data, which was gathered from non-educational data sources and it is sometimes difficult to relate the data gathered from them to the SEEK Ontology (Ruiz-Calleja et al., 2014). The evaluation may also be conditioned by the higher-education context and by the fact that most of the educators who participated in the happenings had a technical background. The impact of these two aspects was not assessed in the study carried out.

Conclusions and Future Work
This paper proposed We-Share, an application that allows educators to publish, annotate, and retrieve tool descriptions available on the Web of Data by making use of the SEEK-AT-WD infrastructure. A working prototype of We-Share was developed and used in an empirical experiment where a group of 23 educators participated. The evaluation showed that We-Share is perceived as an easy-to-use application that is useful to satisfy realistic information needs regarding educational tools.
Interestingly, it was shown that the social information collected had a positive impact, both when participants discovered and selected tools, but this impact was higher in the tool selection process. It is also remarkable that no usability problems were detected on We-Share due to the origin of the data it manages.
These results lead us to conclude that We-Share supports the community of educators to take advantage of the Web of Data. Rich semantic descriptions of ICT tools can be automatically obtained from the Web of Data and further enriched with educational-specific information. Thus, We-Share combines updated information about the technical and administrative aspects of the tools with experiences of their educational use. In addition, as the data published by We-Share is also available as Linked Open Data, it can be further exploited by other educational applications, thus allowing to share information among educational communities. The combination of these technological and pedagogical perspectives in the information provided by We-Share about the tools is recognized by well-known theoretical frameworks, such as TPACK (Koehler, Mishra, Kereluik, Shin, & Graham, 2014), as one of the crucial aspects to be promoted in Teacher Professional Development (TPD) efforts aimed at the effective integration of ICT in education.
One of the future research lines is indeed the exploration of the capabilities of We-Share in TPD programs aimed at training non-experienced teachers in the use of ICT tools for innovative teaching situations. In this sense, We-Share is understood as a shared repository to collect the innovative use of ICT tools for learning purposes and to promote the spread of innovations inside a community of educators. It would also be very interesting to integrate the social annotations published through We-Share to other social annotations of educational tools that may appear in the Web of Data.