The Heart of Data Wise
HGSE Professors Richard Murnane, Kathryn Boudett,
and doctoral student Elizabeth City
The Data Wise process is the subject of a popular book from Harvard Education Press (see the companion Usable Knowledge article, "Using assessment to improve instruction"). The logic of Data Wise evolved in an HGSE course taught by Richard Murnane and Kathryn Boudett. This article describes insights drawn from continuing research about how to best teach educators to use the Data Wise improvement process.
In a yearlong HGSE course called Data Wise, principals and teachers from the Boston Public Schools are asked to explore an area of low student achievement in their data, dig into multiple data sources to illuminate a learner-centered problem of understanding underlying low test score performance, investigate the ways in which instructional practices might contribute to issues of understanding, and collaboratively come to a determination of better practice to improve student learning. At the conclusion of the 2005-2006 academic year, the Data Wise teaching team spent the summer following its own advice: As part of an ongoing effort to improve their own instruction, the team used data from participant interviews, focus groups and course evaluations to determine what participants learned and how their schools benefited from enrolling in the course. The summer's research has led to revisions in the course focus and structure, and to a set of recommendations for schools – particularly schools struggling with low performance – seeking to implement the Data Wise model on their own.
The Data Wise Theory
What is powerful about the improvement process laid out in the Data Wise book is that movement through the various tasks is designed to do more than integrate data into the culture of the school: it is designed to turn schools into learning organizations capable of continuous introspection and improvement. The model encourages the development of an interlocking system of teams, a schedule with time for teacher collaboration and peer observation, and mechanisms by which teachers are held accountable for implementing new instructional strategies. What happens when school teams consisting of a principal, a teacher or other administrator, and a Harvard graduate student enroll in a class designed to support schools in implementing the Data Wise process? For some schools, participating in the Data Wise course provided an opportunity for authentic school improvement. In others, , however, a set of real world problems presented both cultural and organizational barriers to change.
Real World Problems
Research indicated that some participating schools faced structural and cultural challenges to implementing the Data Wise process, including the following:
- Principals are overwhelmed.
The Data Wise improvement process encourages principals to spearhead this important work. The research about the class indicated that many school leaders, particularly those in struggling schools, lack the time necessary to work with their teams on using data to improve instruction.
- Teachers were not held accountable for changing their practice.
The research showed that many schools lacked a "closed feedback loop" in the process of examining instruction. When administrators and principals were able to observe teachers in action and provide feedback, they seldom had the time to follow up on recommended changes or areas of concern.
- Principals found changing practice akin to "moving mountains".
Even if school leaders were able to find the time to close the feedback loop with teachers, this is clearly not the final step in changing instructional practice. Several leaders stated that dislodging experienced teachers' firmly held beliefs about instructional practice was a daunting task, and work with one resistant teacher could monopolize an entire year.
- Teachers were not comfortable giving critical feedback to their peers.
An integral component of the Data Wise model is that teachers will come to shared conclusions about what constitutes good instructional practice in a given area and hold each other accountable for its implementation. The research indicated that even if teachers were comfortable with and logistically able to spend time watching each other teach, they were seldom comfortable giving feedback that was not positive.
Real World Implementation
In light of these findings, the teaching team has adapted its strategy for teaching struggling schools how to implement the Data Wise improvement process. We have determined that there is a set of qualities that can not be expected to evolve as a by-product of the data work, but must be explicitly cultivated along with it. Although we call term them "Data Wise Practices" here, these practices underpin the success of a wide range of whole school improvement initiatives.
This year's Data Wise course will incorporate the following changes, which may also prove beneficial to schools struggling with implementing the model on their own:
- Begin with a Data Inventory and a Data Wise Practices Self-Assessment
The Data Wise book recommends that schools begin by compiling a Data Inventory of all the data sources available at the school. The research shows that educators could benefit from an initial step of "taking the temperature" of their school with regard to Data Wise Practices, as well. To what degree does the school already have a collaborative culture, system of interlocking teams or closed feedback loop? Which will pose the most challenge to implement? In what order should they be addressed? The goal here is to make sure schools understand the importance of Data Wise Practices up front.
- Attack structural concerns first
Educators often talk of going after "low-hanging fruit" or those changes that are easiest to put in place. Concrete, structural practices like a system of teams (Instructional Leadership Team, grade level, subject area, data team, etc.) with a reliable channel of information flow between them are probably easier to tackle than more subtle issues of culture or relational trust. School leaders might work on making these changes first and build off these successes when moving on to trickier issues.
- Adapt Data Wise protocols to address cultural concerns
There are no quick ways to engender relational trust or raise teachers' comfort levels with giving critical feedback to their peers. Cultural change requires tough conversations held in a safe environment with skillful facilitation. The Data Wise model provides many protocols designed to achieve these very ends in the context of data work; these protocols can be readily adapted to the work of cultural change.
- Pare down principal's role to the necessary minimum.
Principals already stretched too thin need not be involved in all Data Team activity. At the bare minimum, principals should be cognizant of the types of questions that can be answered with data and the types of analyses and presentations that are possible using basic PowerPoint and Excel. Research over the years has confirmed that principals' leadership and endorsement of the process is critical to the success of the model, but that principals can delegate the technical analysis to a well-trained Data Team.
- Bring Coaches or Teacher Leaders into the process
Administrators responsible for a fifty-person faculty may not realistically be able to provide the ongoing support and accountability required to change teacher practice. Principals might consider integrating instructional coaches and teacher leaders into the Data Wise work, tasking them with closing the feedback loop and holding teachers accountable for implementation of new strategies. As a collaborative culture grows within the school and faculty takes ownership of instructional improvement strategies, teachers will also come to hold each other accountable for effective classroom practice.
Data Wise Practices
This research did not challenge the basic order or content of the eight steps of the Data Wise improvement process. Instead, it helped to bring to the fore the value of making explicit the "Data Wise Practices" that must accompany these tasks if the work is to lead to meaningful school change.