Fidelity Assessment and Implementation Drivers

Fidelity assessment is a key outcome of implementation done well.  In Module 2: Implementation Drivers, it sits at the top of the implementation triangle. It is the link between implementation supports, consistent delivery of an innovation, and reliable outcomes for students.

An Implementation Team is accountable for assuring adequate supports for teachers and other practitioners using an innovation. Thus, while fidelity assessments include observations of teacher behavior in an education setting, the results reflect how well the Implementation Team is supporting teachers (e.g., with training and coaching and with efforts to improve administrative supports for teachers using an innovation as intended).

  • If teacher instruction is improving rapidly, the Implementation Team should be congratulated for assuring effective supports for teachers.
  • If teacher instruction is poor, the Implementation Team is accountable for providing more effective supports for teachers.
  • If the Implementation Team is struggling, state and district leaders are accountable for improving the functions and effectiveness of the Team.

For example, fidelity data may show teachers 1-8 are providing high quality instruction while teachers 9-18 are not faring very well. The Implementation Team can analyze the information to figure out why the differences are occurring. Did different trainers train the high and low performing teachers? Do they have different coaches? Did different assessors do one set of assessments versus the other? Perhaps in this example the high fidelity teachers had Coach #1 and the lower fidelity teachers had Coach #2. This clearly points to a coaching problem so the Implementation Team immediately goes to work on improving Coach #2 coaching skills. On the other hand, if teachers 1 – 18 were all struggling with particular instructional practices, to one degree or another, then the Implementation Team goes to work improving the training and coaching for all teachers.

For leaders in education, fidelity is not just of academic importance.  The use of a fidelity measure helps leaders and others discriminate implementation problems from innovation problems and helps guide problem solving to improve outcomes. As shown in Table 1, information about fidelity and outcomes can be linked to possible solutions to improve intended outcomes (Blase, Fixsen, and Phillips, 1984; Fixsen, Blase, Metz, & Naoom, 2014).

Without a fidelity assessment leaders and Implementation Teams have no idea where to direct their improvement efforts.  When good student outcomes are achieved, there is no clear idea of what should be repeated to achieve those outcomes for all students. When poor student outcomes occur, leaders are left wondering what to do to “fix the problem.” Fidelity assessments help leaders make effective and efficient use of scarce resources to improve education outcomes for students.
 

Table 2: Fidelity scores as a diagnostic tool.

 

 

High fidelity

Low fidelity

 

Good Outcomes

 

Celebrate!

Re-examine the innovation
and
Modify the fidelity assessment

 

Poor Outcomes

 

Modify the innovation

Start over

 

As shown in Table 2, the desired combination is high fidelity use of an effective innovation that produces good outcomes. When high fidelity is linked consistently with good outcomes it is time to celebrate and continue to use the innovation strategies and implementation support strategies with confidence. The second best quadrant is where high fidelity is achieved, but outcomes are poor. This clearly points to an innovation that is being done as intended, but is ineffective. In this case, the innovation needs to be modified or discarded.

The least desirable quadrants are those in the low fidelity column where corrective actions are less clear. Low fidelity in combination with good outcomes points to either a poorly described innovation or a poor measure of fidelity. In either case, it is not clear what is producing the good outcomes. Low fidelity associated with poor outcomes leaves users in a quandary. It may be a good time to start again — to develop or find an effective innovation and develop effective implementation supports.  The most efficient strategy may be to first improve fidelity and then reassess outcomes so that “babies are not thrown out with bathwater.”

Implementation is in service to using effective innovations to realize intended outcomes.  Implementation Drivers are designed to improve the skill levels of teachers, principals, and staff and produce greater benefits to students. Implementation Drivers “drive” successful implementation of innovations.

Fidelity assessments: Prognosis

A good fidelity assessment requires a demonstration that the measure is highly correlated with student outcomes. This is specified in 4.b of the Usable Innovation Criteria. This means that fidelity scores today predict student outcomes in the future.

If high fidelity today predicts student outcomes in the future, this is good news! This means fidelity measures bring the future into the present. Instead of lamenting poor student outcomes next year when the annual data come out, Implementation Teams can assess fidelity today and provide improved supports for better instruction today. Implementation Teams know how to help improve fidelity today by improving coaching for teachers and helping school administrators support teacher instruction more effectively. Thus, efforts to improve fidelity today help assure improved student outcomes in the future.