Fidelity Assessment and Usable Innovations

The lack of adequately defined programs based on best available evidence is an impediment to implementation with good outcomes (e.g., Vernez and colleagues, 2006).  To improve student outcomes on a useful scale, innovations need to be teachable, learnable, doable, and assessable in typical education settings.  Usable Innovations provide the content that is the focus of training, coaching, and fidelity assessments.  Usable innovations provide the reasons for changing roles, functions, and structures in schools and districts to more efficiently, effectively, and consistently produce intended outcomes.

Module 6: Usable Innovations provides an overview of the four criteria that define an effective innovation.  A key criterion is:

4. Evidence of effectiveness; Practical performance assessment

  1. The performance assessment relates to the education innovation philosophy, values, and principles; essential functions; and core activities specified in the practice profiles; the performance assessment needs to be a feasible method (e.g. a 10-minute classroom walkthrough observation ratings) that can be done repeatedly in the context of typical education settings
  2. Evidence that the education innovation is effective when used as intended
    1. There are data to show the innovation is effective
    2. A performance (fidelity) assessment is available to indicate the presence and strength of the innovation in practice
    3. The performance assessment results are highly correlated (e.g. 0.50 or better) with intended outcomes for students, families, and society

Fidelity assessments should be standard practice in education.  From an implementation point of view, any innovation (evidence-based or standard practice) is incomplete without a good measure to detect the presence and strength of the innovation as it is used in education practice (4.b. above ). 

Fidelity assessments should be directly linked to the usable innovation components.  In particular, the essential functions and the Practice Profiles that operationalize those functions provide information to guide the development of fidelity assessment items.  Usable innovations are doable and assessable in practice. 

For example, fidelity of Positive Behavioral Innovations and Supports (PBIS) can be assessed because it is a well-defined innovation.  On the other hand, Annual Yearly Progress Standards describe student outcomes but not teacher instruction.  In cases like this, Implementation Teams can begin by relying on generic fidelity assessments that are related to evidence-informed teacher instruction practices generally.  Even though these measures are not related specifically to an innovation they can be used frequently (e.g., six times a year for every teacher) to assess the effectiveness of implementation supports for teachers.