Autism Evidence 1. Who do you believe? A model for understanding the development of evidence for interventions in autism

Many experts regularly pontificate about which educational and psychological interventions are “evidence-based” for children with autism. The problem is that there is no one right answer. So, there can be as many perspectives on the evidence base in autism as there are experts. Who do you believe? The purpose of this series of blogs is to describe a tool that may help you draw your own conclusions and/or to empower you to ask those experts more probing questions about the evidence.

A continuum of evidence

All psycho-educational interventions for children with autism are complex – they normally involve several components, might target several outcomes, involve the child and other people, and will typically occur in several environments and over a period of time. Several people have pointed out that it can be helpful to think about a process for the development of evidence for complex intervention that can be represented as a continuum. Thornicroft, Lempp and Tansella (2011) described a model of the process of development of evidence that I find very helpful 

[see http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=8361312&fulltextType=ED&fileId=S0033291711000109]

I have re-drawn Thornicroft et al.’s mode above representing their phases of evidence development and emphasizing increasing evidence from left to right.

The idea of this model is that basic science or theory should inform interventions in the first place. In essence, this is because any intervention should be informed by the latest understanding of a “disorder” or difficulty and interventionists should be able to say what they are trying to do! The fact that Phase 0 is called Basic Science is important. For me, this means that it is not enough to be able to describe what you are trying to do with an intervention – your story needs to be evidence based. For example, you might be informed by common difficulties or strengths in children with autism, the way that children with autism learn, or typical child development (i.e., what are the key early skills children with autism need to learn to enable them to build more complex skills).

An intervention should also be very clearly defined or described (Phase 1) and ideally written down in detail in the form of a manual for those using the intervention. The competencies needed to deliver an intervention probably also need to be described and training/supervision models developed. There would also need to be ways to check whether someone is properly delivering an intervention (this is called “fidelity”). A new intervention can then be subjected to initial testing – trying it out, and measuring some outcomes. What outcomes to measure would also be closely related to the underlying theory – what are you trying to change?

At some point during Phase 1 and in preparation for Phase 2, we might also expect to see a Pilot Randomized Controlled Trial (RCT) conducted. Using this research design, children with autism would be randomly assigned to receive the intervention or to receive another intervention (e.g., existing practice) or even no particular intervention at all. A pilot RCT tests out everything ready to run a definitive RCT at Phase 2. Pilot RCTs are not designed to answer the question of whether an intervention works.

During Phase 2 of the development of evidence, research studies (probably large scale RCT studies) are carried out to ask whether the intervention can work. This is a question of efficacy – given well-resourced, well-trained and supervised interventionists delivering an intervention with fidelity, CAN the intervention be shown to work? It is very important at this stage to recognize that studies are not designed to tell you whether an intervention will work when rolled out into practice.

Too often, the development of evidence in the field of autism stops at Phase 2. Or at least, experts focus on this aspect of evidence when they define what are evidence-based interventions. From Thornicroft et al.’s model, you can see that this is far from the end of the story. The next step at Phase 3 is to carry out tests of the effectiveness of an intervention. CAN an intervention work is an important question, but whether it still works in practice and under less ideal conditions is the question of whether the intervention can be effective. Phase 3 research studies will often be RCTs again. The focus, for example, might be on whether if you train people who train others to deliver the intervention and they do this in typical school settings using teachers or classroom assistants, do you still get positive outcomes.

Phase 4 for me is about then how do you scale up the delivery model to deliver the intervention as a part of standard services. In health contexts, people often talk about care pathways. So, how can the new intervention be incorporated into an existing care pathway and still be delivered in a way that leads to positive outcomes? What conditions are needed in practice to make sure that the intervention still “works”? In educational contexts, questions such as how an intervention can be delivered in classrooms and how does it dovetail with the curriculum? All of these sorts of questions need to be subjected to research study.