Teachers’ Use of Education Dashboards and Professional Growth

Education dashboards are a means to present various stakeholders with information about learners, most commonly regarding the learners’ activity in online learning environments. Typically, an education dashboard for teachers will include some type of visual aids that encourage teachers to reflect upon learner behavior patterns and to act in accordance to it. In practice, this tool can assist teachers to make data-driven decisions, thus supporting their professional growth, however, so far, the use of education dashboards by teachers has been greatly understudied. In this research we report on two studies related to the associations between the use of education dashboards by elementary school teachers and the teachers’ professional growth. We used the framework defined by the International Society for Technology in Education’s (ISTE) Standards for Educators. In the first study, we took a quantitative approach (N=52 teachers), using an online self-report questionnaire, and found that the use of dashboards is positively associated with professional growth in the dimensions of facilitator, analyst, designer, and citizen. In the second study, we took a qualitative approach (N=9 teachers), using semi-structured interviews, to shed light on the mechanisms through which teachers benefit from the use of education dashboards.


Introduction
The use of educational technologies has become mainstream in many teaching and learning settings.
Specifically, the use of computing devices and online learning environments by students in grade schools today appear to be the norm in many countries (Allen & Seaman, 2017;Richardson et al., 2013;Wishart, 2017). Whether the online learning component is integrated as part of the school-based activities or as homework assignments (in most cases, it is a combination of both), teachers face the need to deal with hybrid learning spaces, which include the physical classroom and the online learning environment. While teachers are trained for observing (and responding to) what is happening in the physical environment, where students' actions are normally visible and communication with students is primarily spoken, they often lack the means to observe and respond to what is happening under the hood of the virtual environments. Indeed, teaching experience was found to be negatively correlated with teachers' communication self-efficacy and self-directed learning in online teaching (Hung, 2016).
Evidently, in such hybrid learning spaces, teachers face challenges referring to many facets of the teaching profession. In previous studies, we have shown that one-to-one computing programs (where each student has their own portable computer) may affect pedagogical behavior, for example in the ways in which teachers prepare themselves to teach lessons, how they choose to divide the lesson time between different teaching strategies, or how they assess students (Hershkovitz & Karni, 2018;Hershkovitz, Merceron, & Shamaly, 2019).
In that light, it may be useful to think of teachers as decision-makers. Indeed, teachers constantly make decisions -before, during, and after lessons -to better support their students and themselves. Such decisions may include: which content to focus on; how to best engage the students with content; whether to, or how to, partition the classroom into smaller learning units; or how to evaluate a students' learning. In many cases, such decisions are taken based on the teacher's experience and understanding of the situation, and not necessarily on empirical evidence (Annerstedt & Larsson, 2010;Hay & Macdonald, 2008;Vanlommel, Van Gasse, Vanhoof, & Van Petegem, 2017).
However, when teaching in hybrid spaces, teachers could use data about learners that is gathered automatically and continuously. Specifically, many online learning systems log students' activity in three dimensions: The action taker (who?), the action itself (what?), and the action time (when?).
Analyzing these data, it is possible to calculate a plethora of measurements of the learning process (Lang, Siemens, Wise, & Gašević, 2017). As the use of such systems has grown world-wide, it has become clear that these data should be made accessible to teachers in a way that is easy to understand and to act upon. One of the most common ways to communicate such data to teachers is via an education dashboard, that is, a display which presents education stakeholders with data-driven information regarding teaching and learning processes. Education dashboards are considered as practical tools for teachers to meet the daily challenges they face and to promote the use of technology as an integral part of their teaching (Rienties, Herodotou, Olney, Schencks, & Boroowa, 2018;Xhakaj, Aleven, & McLaren, 2016;Yoo, Lee, Jo, & Park, 2015).
Indeed, many learning management systems and learning platforms today offer teachers with dashboards that provide them information about various aspects of their students' learning; for example, some popular platforms include: BrainPop (http://www.brainpop.com), Khan Academy (http://khanacademy.org), Google Classroom (http://classroom.google.com), and Canvas (http://www.canvas.net). However, evidence of their use and the impact on teachers' professional development is still inadequate. Recent preliminary findings demonstrate how education dashboard can affect teachers' pedagogical decision-making and improve interactions with their students (Ezzaouia, Tabard, & Lavoué, 2020;Molenaar & Knoop-van Campen, 2018;Tissenbaum & Slotta, 2019).
The current paper aims at bridging this gap. We take a broad perspective on teachers' professional development, in accordance with the understanding that professional development is an ongoing process that crosses a teacher's professional and personal circles and is not limited to institutional training (Collinson et al., 2009;Nabhani & Bahous, 2010). Therefore, we aim at understanding how teachers can benefit from technological tools, not only in their daily work but also in a broader sense of their ongoing professional development, may contribute to a more effective integration of technology into schools and to more effective professional development.

Background Teachers' Professional Development
Current teacher professional development is a process that starts with a teacher's pre-service training and concludes at the end of the teacher's professional career or at the teacher's retirement, and includes various forms of education, both formal and non-formal (Avidov-Ungar, 2016;Collinson et al., 2009).
As such, the term professional development refers to various aspects of teaching and to different periods in the teacher's career. Among others, these include the deepening of the teachers' understanding of pedagogies and of the students' learning; the development of responsibility and commitment to the teaching profession; and the ways technology may impact teaching and learning (Darling-Hammond & McLaughlin, 1995;Hargreaves, 2005).
Consequently, there is a plethora of models that describe teachers' professional development. They do not necessarily consider it as a linear, step-wise experience, but rather as an ongoing, dynamic, multicontextual endeavor that involves multiple circles in which the teacher belongs (e.g., school, professional communities, and home; Ferreira, Ryan, & Tilbury, 2007;Grossi, Oliveira, Barbosa, & Oliveira, 2016;Yurtseven Avci & O'Dwyer, 2016). In recent years, it is agreed, therefore, that teachers' professional development presents a continuum of life-long learning throughout a professional teaching career (Collinson et al., 2009); moreover, it is closely linked to both their professional and personal identity and exists not only as part of institutional training (Nabhani & Bahous, 2010).
Additionally, one should note that current age, a teacher's identity comprises of more than just delivering content during lessons (no matter what pedagogical or technological tools and techniques they use). Danielson's (2011) Framework for Teaching details four domains based on which teaching should be assessed; these dimensions-planning and preparation, classroom environment, instruction, and professional responsibility-portray a broad view of the teaching profession. Even more prominently, the Standards for Educators, presented by the International Society for Technology in Education (ISTE; 2007), set a general framework that describes seven standards for today's teachers.
The standards are: 1. Learner: Teachers continually improve their practice by learning from and with others, and explore promising practices that leverage technology to improve student learning; 2. Leader: Teachers seek out opportunities to take the lead in order to promote students' learning and their own teaching; 3. Citizen: Teachers inspire students to contribute to the digital world positively and to participate in it responsibly; 4. Collaborator: Teachers dedicate time to collaborate with colleagues and students, in order to improve their practice, to share ideas, and to solve problems; 5. Designer: Teachers design learning activities that are authentic and learner-driven, and that recognize and accommodate learner variability; 6. Facilitator: Teachers facilitate learning with technology to support student achievement; and 7. Analyst: Teachers understand and use data to drive their instruction and to promote students' learning (Trust, 2018).
We chose this framework to study teachers' professional development because of its broad view, and because it directly refers to the teachers' roles in the digital era.

Education Dashboards for Promoting Data-Driven Decision Making
Data-driven decision making refers to collecting, understanding, and analyzing educational data, and is considered an integral part of a teacher's professional conduct (Mandinach, 2012). Using data for educational decision making is not something new, as teachers have been using grades, students' work, and behavioral data since very early schooling days (Mandinach, 2012;Schifter, 2014). Overall, teachers use such data to evaluate their class and individual students, and also to reflect upon their own teaching (Light et al., 2005;Molenaar & Knoop-van Campen, 2018), and also to communicate with various stakeholders, such as their students or their students' parents, or the school's educational and management teams.
In hybrid learning spaces, the amount of data per student have dramatically increased (Mandinach, 2012), and nowadays, teachers need to handle a wide range of data gathered from various educational technologies (Schifter, 2014;Xhakaj et al., 2016). Additionally, the pressure on them to make accurate decisions keeps on growing (Faria et al., 2014). This has led to an increase in research and development of education dashboard, particularly to support teachers who teach in blended environments Yoo et al., 2015).
To put it simply, an education dashboard is a display that presents educational stakeholders with datadriven information (usually visual) regarding teaching and learning processes in such a way that will promote reflection on their behavior patterns and help them to adjust their actions accordingly Yoo et al., 2015). Education dashboards could contain various types of information, such as an overview of the course activity, time per tasks, students' skills, misconceptions, test results, social interaction, and students' current and historical progress in the course (Charleer, Klerkx, Duval, De Laet, & Verbert, 2016;Matuk et al., 2016).
Many studies have shown that education dashboards can be used as a decision-making tool that supports teachers in planning their curricula, evaluating the class knowledge level, and the tracking of individual students (Molenaar & Knoop-van Campen, 2018;Schifter, 2014;Xhakaj, Aleven, & McLaren, 2017). Furthermore, it was shown that the use of dashboards had led teachers to better tailor their teaching (in terms of both content and style) to students' needs, to collaborate more effectively with their colleagues, and to reflect upon their own professional conduct and abilities (Light, Wexler, & Heinze, 2005;Schifter, 2014;Schwendimann et al., 2017;Xhakaj et al., 2016). That is, this process may be seen as an integral part of a teacher's continuous professional development. However, evidence regarding the impact of education dashboards on the development of a teacher's skills is still limited (Gillet et al., 2017;Schwendimann et al., 2017), and as such, this will be the focus of our study. In this research, we address the following questions: 1. What are teachers' perceptions of using education dashboards for their own professional development?
2. What insights do teachers gain and what actionable steps do they take from examining the information in education dashboards?

Methodology Study 1 (Quantitative, Self-Report Questionnaire)
This study was quantitative in nature (N=52), involving teachers' self-report, via an online questionnaire, of their experience using a specific education dashboard (to their choice), doing so through the lens of their professional development (see details under Instrument below). We Research population. Participants were teachers (N=52, 50 females and 2 males) in public elementary schools acorss Israel who met the inclusion criteria: they used at least one education dashboard as part of their teaching practice during the school year previous to data collection.
Participants were aged between 25 to 66 years (M=41.5, SD=8.4), and had 2 to 42 years of teaching experience (M=15.8, SD=8.9). Of the participants, 11 (21%) started using the learning environment to which they referred in the questionnaire during the previous school year (2017/8); 13 participants (25%) started using that learning environment's education dashboard during the previous school year (2017/8). The remaining participants had used the learning environment or the education dashboard for longer. Most of the participants reported using an online learning environment for Mathematics (26, 50%), with less use reported for subjects in Science (5, 10%), Language (6, 12%), Social Sciences (2, 4%), or for multiple topics (13, 25%).
Instrument. The main part of our questionnaire was based on ISTE Standards for Educators (Trust, 2018). These standards are designed around seven themes: Learner, Leader, Citizen, Collaborator, Designer, Facilitator, and Analyst, each explicitly defining a set of actionable, measurable items. For example, under the first theme, Learner, there are three items (1a-1c), the first of which is: "Set professional learning goals to explore and apply pedagogical approaches made possible by technology and reflect on their effectiveness." Overall, there are 24 items.
Based on these Standards and items, we developed 23 self-report items (two items of the Collaborator dimension were merged into one questionnaire item) that directly connect the use of education dashboard to the various themes. For example, based on item 1a from the Standards, we phrased the following questionnaire item: "Using this dashboard assists me in setting professional goals to develop my pedagogical ability." Questionnaire items were phrased as short as possible, while keeping the meaning of the original item. Items were ranked on a 5-point Likert scale (from 1 "Not at All" to 5 "To a Large Extent"). The items were reviewed by a few education experts to make sure they correctly reflected the nature of the original standard; this process had been taken for a few rounds until full agreement was achieved between the experts. Each variable then was calculated based on the mean of its corresponding items. In order to test for reliability, and as we have only a few items for each dimension, we calculated average inter-item correlations instead of using the more common Cronbach's alpha test.
We used the non-parametric Spearman's test to get the acceptable values of 0.46-0.79. The full list of items, including the inter-item correlation values, is provided in Table 1.
When filling out the questionnaire, after explaining what we meant by an "education dashboard," participants were asked to mention one online learning environment they used with the education dashboard that they had had experience with, and to refer to this dashboard while responding to the remaining items. We then made sure that the mentioned dashboards were indeed eligible, that is, that they were considered an education dashboard as defined in the literature. Overall, participants had referred to nine different educational systems, which represent most of the certified platforms in use in Israel (approved by the Israeli Ministry of Education), in various disciplines. By referring to multiple types of education dashboards and various disciplines, we allow for an increased variance in our population, which contributes to the generalization of our findings. Additionally, the online questionnaire was used to collect data about the background variables. Using this dashboard contributes to the creation of social responsibility and empathic behavior in my classroom (3a). Using this dashboard allows me to establish a learning culture that promotes curiosity and criticism (3b). Using this dashboard allows me to mentor my students on how to use learning materials in a safe and effective way (3c). Through using this dashboard, I model my students how to manage personal data and data privacy (3d). Collaborator (ρ=0.61) As a result of using this dashboard, I collaborate with colleagues to create authentic learning experiences for my students (4a). Using this dashboard allows me to collaborate and co-learn with my students about effective uses of technology (4b).
Using this dashboard allows me to emphasize the cultural diversity in my classroom to students, parents, and colleagues, thereby making them all partners in the learning process (4d). Designer (ρ=0.79) Using this dashboard allows me to personalize learning experiences for my students that encourage independent learning (5a). Using this dashboard allows me to design learning activities that foster active and deep learning (5b). Using this dashboard allows me to apply instructional design principles to promote student engagement and assist their learning (5c). Facilitator (ρ=0.72) Using this dashboard allows to foster a culture where students take ownership of their learning, individually and collectively (6a). Using this dashboard helps my student to foster learning strategies (6b). Using this dashboard allows me to create challenging, innovative learning opportunities for my students (6c). Using this dashboard allows me to foster creative discourse about ideas, knowledge, and connections (6d). Using this dashboard allows me to provide my students with diverse ways to demonstrate their ability (6a).

Analyst (ρ=0.64)
Using this dashboard allows me to give my students formative and summative assessments (6b). Using this dashboard allows me to encourage my students' self-direction by communicating with students, parents and education stakeholders (6c).

Study 2 (Qualitative, Semi-Structured Interviews)
Research tools. In this qualitative study, which was part of a broader study of technology orchestration in the mathematics classroom, we used a semi-structured interview. The design of the interview allowed to deviate from the protocol and capture the natural behavior and insights of the participants. During the interview, we presented to the participating teachers three types of data visualization of students' activity that were presumably taken from an online learning environment for mathematics; the visualizations were fabricated by the research team. These visualizations were a scatter plot, a table, and a bar chart; these types were chosen as they are among the most common information visualizations (Khan & Khan, 2011). The scatterplot presented students' activity, based on the average score (x-axis) and time on tasks (y-axis), where each student was represented as a circle ( Figure 1); the table presented detailed information about the activity of each student in 6 dimensions: the number of completed tasks, the average score for a task, the average time for a task, the task with the highest score, the task with the lowest score, and last seen ( Figure 2); lastly, the bar chart presented the number of students that master each topic (e.g., decimal fraction, percentage, ratio, scale etc.; Figure   3).   Research population. Nine mathematics teachers, all females aged between 27 to 52 years with 4 to 29 years of teaching experience, participated in the study. All participants were teaching in public elementary schools from various geographical areas in Israel at the time of the study. Some of the teachers had additional roles in or outside of school, such as a mathematics teachers' trainer or a mathematics coordinator. Participating teachers were recruited in a snowball fashion, with the inclusion criteria being that they integrated the online environment for mathematics in their teaching (either in the classroom or for homework assignments) at least once a week. A description of the research population is given in Table 2. presented to the teachers via Google Drawings. All the interviews were fully transcribed before analysis.
Qualitative content analysis was conducted under our theoretical framework for assessing teachers' professional development. The unit of analysis was teachers' statements related to professional development. We used the Direct Content Analysis method (Hsieh & Shannon, 2005) with seven variables derived from ISTE's Standard for Educators; specifically, statements about professional growth were categorized into seven groups: learner, leader, citizen, collaborator, designer, facilitator, and analyst. The first author had coded one interview-transcript, and then reviewed it with the third author; these two authors discussed the coding and came to an agreement regarding conflictual coding schemes. The remaining interviews were then coded by the first author.

Findings Study 1
We found no associations between age or teaching seniority with either of the dependent variables.
Due to the small population size and the distributions of the research variables, which are not necessarily normal, we used non-parametric tests. Specifically, we used Mann-Whitney U test for examining associations between the dependent variables and participants' experience with the education dashboard; this test is valid also in cases of unequal sample sizes and unequal variance of compared groups (Mann & Whitney, 1947). We find that, on average, the teachers with a longer experience with an education dashboard-compared with those who have a year or less experiencescored higher on Facilitator (M=3.9, SD=1.0, and M=2.9, SD=1.1, respectively, with Z=2.56, at p<0.05) and Analyst (M=3.8, SD=1.0, and M=3.0, SD=1.2, respectively, with Z=2.14, at p<0.05). (We report on Z score, which is calculated based on an approximation to the standard normal distribution and is served as the basis for the p-value calculation.) These differences denote large effect sizes of d=0.78, and d=0.7, respectively. Results are summarized in Table 3.

Study 2
Analyzing the interviews conducted in this study, we were able to map teachers' statements to three of ISTE dimensions: designer, analyst, and facilitator. We were unsuccessful in finding statements that referred to the other four dimensions.
Designer. Recall that this dimension refers to the design of authentic, learner-driven activities, and environments that recognize and accommodate learner variability. Under this dimension, participants mostly referred to the accommodation of learner variability based on insights gained from examining the information presented to them. Commonly, teachers suggested to divide the class into groups, based on knowledge level: "I would divide the kids according to the tasks…and after that I would build the groups" (T5); "I am applying differential instruction. I am taking each group according to their knowledge, according to their presented points, and I am teaching them accordingly […] promoting them or strengthening them according to their needs" (T4); "Here [table] I know which groups to create and give them tasks according to their success or un-success" (T8).
Some teachers referred specifically to those students who seemed to be struggling the most, differentiating them from the rest of the class, and focusing on the ways they could be assisted and supported, for example, by clarifying the tasks to them, giving them individual lessons, extra exercises, and extraordinary attention: "First, I think to explain the tasks [to the struggling students]. There seems to be some misunderstanding of the tasks [among them]" (T7); "So, it is worth one or two individual hours for those children who have difficulties with [the subject]" (T1); "Working individually [with the struggling students], or in a group during class, or as an extra practice" (T9); "[There are] children who need more support and more help, so that I will keep an eye on them, and I will notice them more, so I can see if they get along" (T3).
Some teachers referred to instructional design principles regarding the struggling students, mostly by creating a personal work plan that is based on "checking the questions that were there" (T5) in order to support "students who need reinforcement" (T6).
On the other side of the scale, and to the least extent, some teachers referred to supporting those students who seem to be excelling by supplying them with challenging tasks. Some ideas on that directions include: "Giving them extra tasks in the classroom, like riddles, more challenging tasks, higher level tasks" (T1); "There are many [students] who scored many points in a short time. It means that we can challenge them with extra tasks or raise the level of difficulty" (T9); and "[This student], scored excellent grades in 3 out of 5 tasks, maybe it required out-of-the-box thinking, so I will give him challenging tasks" (T2).
Alongside the reference to the two extremes, teachers also referred to the class' overall knowledge level and suggested instructional principles that mainly aim at the average, mostly by adding more class-time on topics with which the majority of the class struggled, and implementing instructional intervention: "If more than half of the class does not know a particular topic, I would repeat it in another way, with the entire class." (T5); "The topics with the lowest number of children [achieving proficiency]: "circle," "calculate volume," "calculate area" -I would teach and repeat it in the class, because most of the class does not know it, half of the class" (T4); "On less successful topics, I would re-teach or re-practice...
repeating the basics with them" (T9); "I would review task number four and would try to figure out what is that task. Then I would teacher that material using questions with the same style" (T3).
Analyst. This dimension refers to teachers' use of data as a driving force of their instruction.
Indeed, teachers were appreciative of the data-driven insights: "A child that does not like to work in the classroom gets [a full score] on a task. [The dashboard] reveals another aspect of the child, which we do not always see" (T6); "It adds to the understanding of what you do not see during lessons, and then, when you actually get the data at home, you can manage and process it correctly" (T9); "The table clearly presented the students' skills, and how you as a teacher can strengthen and give feedback to specific children" (T2).
Furthermore, examining the data presented to them, teachers were able to identify necessary assessment components: "If I want to build a formative assessment for a student, I must use this table" (T5).
Others suggested additional, more fine-grained student evaluation: "I would take the topics and make a summary table to see which topics are well understood, and which topics I should strengthen" (T2); "I could map out the student's' skills" (T3).
Some of the teachers explicitly mentioned how data would help them to communicate with students and parents to encourage student ownership of their learning: "I need to know in which tasks the student struggled, because when I talk to a parent or with another teacher, I need a very accurate picture of the child" (T8); "[Using the dashboard I can] ask the child: why didn't you do the task? Where did you struggle?" (T6); "I saw that [the student] did [the task] quick and that he was wrong. I would talk to him about it, and I would try to direct him so that he will do it more slowly" (T9): When you see data on a child [regarding the

Discussion
In this paper, we reported on two complementary studies of the role of education dashboards by teachers in their professional development. We refer to "professional development" in a very broad sense, as a life-long endeavor that takes place throughout the teacher's career and relates to her or his professional and personal lives at large. In many ways, our two studies complement each other, as findings from our qualitative study (N=9) shed light on those from the quantitative study (N=52), and the quantitative study helps us emphasize the more prominent findings from the qualitative study.
Overall, the studies reported here suggest that teachers' experience with education dashboards may positively contribute to the extent to which they grow professionally as facilitators, analysts, and designers, as defined by ISTE Standards for Educators. While the quantitative findings emphasize the associations with the facilitator and analyst dimensions, the qualitative findings support the contribution to the designer and analyst dimensions, and to a lower extent, the facilitator dimension.
From the similarities and differences between the findings from these two studies, we would like to highlight some important issues.
First, education dashboards may indeed support teachers' decision-making, hence their professional growth, regarding teaching in the traditional sense. The impact on the dimensions of designer and facilitator is well understood, as education dashboards usually present teachers with information on students' learning vis-à-vis the students' content understanding. Note that of the seven dimensions defined in the Standards for Educators, these two are the ones that directly relate to the traditional teachers' responsibilities, that is, to her or his teaching content in the classroom. So, it is of no surprise that the current findings reinforce previous studies in that sense (Molenaar & Knoop-van Campen, 2018;Schifter, 2014;Xhakaj et al., 2016). Indeed, when teachers think of their needs from the "ultimate" dashboard, they usually think of it as augmenting their traditional teaching (Holstein, Second, the positive impact on teachers' sense of being an analyst is worth highlighting. Data literacy has been mentioned as an important skill for today's educators (Mandinach & Gummer, 2013), and it was shown that data-related interventions may improve this skill (Reeves & Honig, 2015). We showed that the very use of data-intensive digital platform may improve teachers' data literacy. Of course, this finding does not underestimate or undermine such interventions, but rather emphasizes the importance of authentic, routinely, long-lasting usage of data.
Finally, that we did not observe an impact of the use of education dashboards on the other dimensionsnamely, learner, leader, citizen, and collaborator-may be an evidence to the yet unfulfilled potential of education dashboards as a means for teachers' professional development on a broad level. This may be a result of either a design issue or lack of training (or both). In order to enhance the effectiveness of the use of education dashboard by teachers, it is also recommended to design them according to teachers' needs (Demmans Epp, Phirangee, & Hewitt, 2019), to train teachers on the use of data for their professional use, and to supply them with ongoing support on that topic (Rienties et al., 2018). We plan to continue studying the ways in which education dashboards could help in promoting teachers professionally, to make this decision-supporting tool as effective as possible in all aspects of teaching and learning.
This study is, of course, not without limitations. First, it was situated in a single country, characterized by a specific culture of education, technology, and implementing technology in schools; more than that, it is limited to a particular sub-population (elementary school teachers, mostly women, teaching primarily mathematics), which may have unique characteristics. Therefore, our findings should be validated by similar studies in other disciplines and in other countries. Additionally, the sampled population is not to be considered as representing the whole teacher population in the discipline/country discussed here. Despite these limitations, we feel that the contribution of the current study is of importance for promoting a more effective use of education dashboards, and to enhance teacher training and teachers' professional development on that topic.