“You can have data without information, but you cannot have information without data.”
This well-known quote by Daniel Keys Moran illustrates the importance of effective data collection and analysis, which create the foundation for informed decision-making and targeted improvements.
It can apply directly to implementing Multi-Tiered Systems of Support (MTSS), which are shown to lead to improved long-term academic achievement, increased high school completion rates, decreased disruptive conduct, and minimized unnecessary placements.
The complexities of this system demand an integrated, evidence-based approach to teaching and learning practices driven by strong teaming structures and data use to make strategic MTSS decisions. Many districts are in the process of creating a cohesive system that aligns assessments, evidence-based instructional methods, intervention protocols, and data-driven decision-making processes for this purpose.
One of these districts is Hardin Public Schools, located about an hour outside of Billings, Montana, on the edge of the Crow Nation Reservation. I have been part of a Regional Educational Lab Northwest (REL Northwest) project team that has been working with Hardin and other Montana districts to support these efforts. (REL Northwest is operated by WestEd in partnership with Marzano Research.) Hardin’s journey to implement a clear, effective data analysis protocol to better inform their MTSS in reading (MTSS-R) offers insights for other districts looking to improve their MTSS-R structures and implementation.
The Starting Point: Rich Data, Limited Insights
When the REL Northwest team first arrived, we found a district with so much data that it was difficult to use effectively.
This type of challenge is not unique to Hardin. Many districts find themselves drowning in data and in need of tools and processes to turn that data into informed, actionable insights. The consequences of data overload can be significant: missed opportunities for early intervention, inefficient use of resources, and students who don’t receive the targeted support they need to succeed.
Developing Strong Data Literacy Practices
Together, the REL Northwest team first focused on developing strong data literacy practices among the staff, drawing on REL Northwest’s Continuous Improvement for a Multi-Tiered System of Support in Reading facilitator’s guide, which provides valuable evidence-based resources and best practices.
Reflection and Problem of Practice Identification
The Hardin literacy leadership team began by conducting an MTSS-R self-reflection worksheet, which guides teams in evaluating their practices across four essential components:
1) evidence-based instruction and interventions,
2) comprehensive literacy screening and assessment,
3) a tiered delivery system, and
4) continuous data-based decision making.
This exercise helps leaders to compare their current practices to best practices and to identify strengths and areas for improvement.
After a root cause analysis of the areas for improvement, the literacy leadership team identified their high leverage problem of practice: the need for a clear, organized way to synthesize and act on student data to inform improved teaching practices across the district.
Initiating the PDSA Cycle
With the problem of practice clearly defined, Hardin and the REL Northwest team initiated the first phase of a PDSA (Plan, Do, Study, Act) cycle. Here’s how we implemented each step:
- Plan: We collaborated with the district literacy leadership team to develop an action plan. This plan outlined specific goals, success metrics, and implementation steps for creating and using a data analysis protocol.
- Do: We supported the district in creating the protocol, which they then implemented during data meetings for grade-based teams. Teachers engaged in specific conversations about student literacy data and drew concrete takeaways from it, such as students they might have overlooked for intervention supports.
- Study: The REL Northwest team collected and analyzed data on the effectiveness and use of the new data analysis protocol. This included surveying participants about their experiences and examining how well the protocol helped translate data into informed, actionable insights.
- Act: Based on the analysis from the Study phase, we used an Adapt, Adopt, Abandon flowchart to make decisions about next steps and strategic action plans.
Dr. Devon Flamm, District Curriculum Director at Hardin, found the PDSA cycle particularly effective, saying it helped keep the group outcome-based: “By being focused, we were able to identify exactly what we needed to accomplish.”
Adapt, Adopt, Abandon
During the Act phase of the cycle, the Adapt, Adopt, Abandon flowchart from REL Northeast & Islands proved to be an especially valuable tool for the district. This simple but powerful resource helped the literacy leadership team evaluate their work so far:
- Adapt: They identified which aspects of the new data analysis protocol were working well but could be improved with some tweaks.
- Adopt: They determined which new practices or tools they wanted to incorporate into their regular data analysis protocol.
- Abandon: They recognized which elements were not serving them well and should be discontinued.
Initial Results
Teachers’ responses were overwhelmingly positive. When asked if they would continue to use Hardin’s new protocol even if the district didn’t require it, over three-fourths of respondents said they would. Staff shared that the added structure helped facilitate much-needed conversations and kept meetings student centered. Furthermore, district administrators shared that the conversations they were having around student achievement were richer and more in-depth than they had been previously.
This widespread buy-in revealed just how helpful protocols like these can be for fostering challenging discussions and building consensus, critical elements in establishing a robust MTSS-R that are otherwise difficult to achieve.
With these promising results, the REL Northwest team supported the literacy leadership team in conducting a second PDSA cycle to refine the protocol based on suggestions from the teachers. When the district rolled out the data-backed updates to the protocol, they received similar levels of positive engagement and feedback.
Continuous Improvement Through PDSA Cycles and Data Meetings
These first few PDSA cycles are just the beginning. By establishing a systematic approach to data analysis and decision-making, Hardin Public Schools has set the foundation for long-term continuous improvement in their MTSS-R structures and systems.
Each subsequent PDSA cycle will allow the district to refine their data analysis protocol further, ensuring that it remains effective and aligned with their evolving needs.
Based on these strong successes, Hardin is now using this data analysis protocol not only at the district level, but at all levels—all the way down to schools’ individual team-based data meetings. These team-based data meetings focus on data-informed problem-solving and improving student outcomes through shared accountability, regular meetings, established norms, and formalized decision-making and feedback processes.
With an abundance of data, the school level team-based meetings narrow down what learning targets or literacy skills a specific student needs support with. Perhaps most significantly, they are able to discover students in need of supports who previously went unidentified.
Dr. Flamm noted the broader impact: “Expanding our data meeting protocol into looking at Tier I instruction next year, the resources will be used throughout that process to keep all staff focused, participating, and moving collectively towards a unified goal.”
Hardin Public Schools’ journey with REL Northwest support demonstrates the power of developing a clear, consistent approach to data analysis and data-driven decision making. Using these tools and processes made it possible for Hardin to shift from having “data without information” to getting accurate, actionable insights from data, enhancing their ability to support student literacy through their MTSS-R structures and fostering a more collaborative district culture overall.
This blog is part of a series highlighting REL Northwest’s work with districts to improve their MTSS-R implementation. Through these partnerships, we’re helping districts transform their approach to data, leading to more effective interventions and, ultimately, better outcomes for students. Read the rest of the series and check out REL Northwest’s blogs here and here.
Additional Source
Bailey, T. R. (2018). Is MTSS/RTI here to stay? All signs point to yes! Center on Multi-Tiered System of Supports, American Institutes for Research. https://mtss4success.org/blog/mtssrti-here-stay-all-signs-point-yes
