Autism and Evidence 4. Does ABA work for children with autism?

 This blog must be read after blogs 1-3 in this series where I describe a way of thinking about “evidence”, and also where I have explained what I think ABA is when applied to working in educational contexts with children with autism. I am assuming that you have read these - they are the context for the blog. Please don’t read the current blog out of context!

The first thing to point out is that the form of my question is just plain wrong. ABA is an applied science. So asking if ABA “works” is like asking if Physics works or Ecological Science works. We have to get more specific. The question needs to be posed about particular ABA-based interventions. So, it is more appropriate to ask whether PECS works for children with autism. It is also appropriate to ask whether Positive Behaviour Support works in reducing behaviour problems in children with autism. Thinking about some of our recent research, we have asked whether an Internet reading programme based on ABA principles works for children with autism, and whether using a buzzer prompting devise in a child’s pocket can help to improve their social initiations towards peers.

In this blog, I do not intend to review everything. For the moment, I will focus on whether a comprehensive ABA-based educational model leads to good outcomes for children with autism. So, my question might be about the evidence for Early Behavioural Intervention (whether Intensive or not).

A little more about my perspective

I also want to be clear about some of the reasons I am writing this particular blog (and the series). I am not a behaviour analyst. I am an academic psychologist and researcher working in the field of intellectual disability (learning disability using the UK health terminology) and autism. I have studied behaviour analysis and carried out a good deal of research on ABA-based interventions for children with autism. However, I am no spokesperson for ABA. I do hope that I have a contribution to make as someone who has some understanding of ABA but also more broadly psychology and perhaps policy/practice in the field of ID and autism.

It is important to say that I am convinced that there is something important in ABA interventions to support children with autism. I am frustrated both by the “pro” and the “anti” ABA camps out there. My opinion is that those pro ABA in the field of autism have been poor at communicating their work and also sometimes poor at working with other autism professionals and experts, including those who are autistic. I have to perhaps include criticism of myself within this too. I am trying hard though, and this blog is a part of that effort. Those anti ABA, in my opinion, have not taken the time to fully inform themselves about ABA, fail to recognize the need to specify the question about what ABA-based interventions might work, and are preventing children and young people with autism benefiting from something useful.

What am I “pro” then? I’m pro making a positive difference to the lives of children and young people with autism. I suspect anyone reading this blog will say the same thing. So, for goodness sake let’s talk and collaborate and stop getting sidetracked away from the most important agenda. Many children out there with autism even in the UK still receive no specialized support, people do not understand their condition, and when children receive some services there is poor practice (ABA and non-ABA alike). Children and young people also face that cliff at transition to adult services. The cliff is partly about even fewer services for adults, but also a real lack of continuity from an educational focus to…err…what?

The evidence for comprehensive Early Behavioural Intervention - a summary

My summary of the evidence can be represented by this single picture that only makes sense in the context of my earlier blogs:

Let me explain. A comprehensive ABA-based educational model for young children with autism (early - so perhaps up to around 7/8 years of age) is informed by basic scientific findings about learning processes, and also by research suggesting that children with autism can fail to develop typically during the early years. Comprehensive ABA models are also informed by research describing how children develop typically, and by autism research showing what skills early on might predict later positive outcomes (e.g., joint attention and other social skills, communication skills). These various aspects of scientific research have informed clearly described comprehensive long-term educational programmes much of which is described in manuals for interventionists. These interventions are conceptually rooted in behavioural theory.

There are plenty of studies exploring outcomes from the delivery of these described interventions, some of which include comparison or control conditions. Lovaas’ (1987) seminal study might be viewed as a part of this effort. Thus, Phases 0 and 1 in the evidence continuum are covered pretty well.

I am not aware of a definitive randomized controlled trial study yet published that was designed as a test of the efficacy of comprehensive early behavioural intervention. That is why I have a red cross at Phase 2 of the evidence continuum. Why is this? It is probably safest to say that no-one really knows. There may be many reasons. Historically, for example, when behaviour analysts were first delivering and evaluating comprehensive ABA-based intervention for children with autism (actually 30 years ago now or more) there may have been less clarity about the importance of efficacy RCTs. Another problem has always been what the comparison should be. As I pointed out in my third Blog in this series, ABA-based methods are probably being used as a part of pretty much all educational practice in the field of autism. So, how different might best practice and comprehensive ABA actually look?

This touches on an ethical point some researchers have made - that random allocation to an intervention lasting two years or more is ethically complex when one already has some evidence that a comprehensive ABA-based model may be beneficial. The control group could not benefit from early intervention once that period has passed for them. So, that possibility in their life would be lost. Others argue that RCTs of longer and more intensive interventions in the field of autism can indeed be carried out. There is no disagreement that this is possible (see Dawson Rogers et al ESDM study mentioned in my second Blog in this series, for an example) - that RCTs CAN be carried out. However, there is still an ethical debate to be had.

To be clear, the fact is that I am pretty sure there is no knock-out efficacy RCT of comprehensive ABA-based educational intervention for young children with autism. Commentators who point this out are absolutely correct. We do not disagree. However, the point in describing the evidence continuum to you is that we know this is not the whole story. Anyone stopping at this point without providing the full picture is doing a poor job of informing people about the evidence. The lack of an efficacy RCT is perhaps a gaping hole, but it does not invalidate the remainder of the evidence.

So, the full picture is that there are quite a lot of Phase 3 outcome studies relating to comprehensive ABA-based educational models. There are several effectiveness studies, including some use of randomized designs. For example, Smith, Groen and Wynn in 2000 published a paper where they had randomly allocated children to a comprehensive ABA-based early intervention which was quite intensive and also delivered by experts, or to an intervention where parents were trained in similar ABA skills (and they delivered fewer intervention hours). The children in the expert delivery group had better outcomes. There were still some positive outcomes for the parent training group too. What I would take from this study is that these may be two different delivery models for early behavioural comprehensive intervention that can be clearly described. Both may be viable models depending on resources available and family capacity, but if the studies results are replicated we might conclude that an expert delivered model is likely to be more effective. This sort of research could inform how intervention could be delivered on a large scale.

Comprehensive Early Behavioural Intervention - implementation research

What is pretty much unique in the field of autism interventions is that there are also data at the implementation end of the evidence continuum for ABA-based comprehensive early intervention. Several groups in Canada, for example, have delivered large scale (i.e., hundreds of children with autism) early behavioural intervention models and also evaluated outcomes and published the results. Large scale delivery is not simple, and several problems needed to be solved along the way including how to train enough interventionists in the first place, and how to ensure that services were delivering good quality intervention across large geographic areas. The outcome studies are very encouraging in terms of making a difference to the skills of young children with autism. Google (other search engines are available!) the work of Adrienne Perry, Nancy Freeman and colleagues in Ontario, and Isabel Smith, Susan Bryson and colleagues in Nova Scotia.

We have also recently published two implementation studies with colleagues in Wales [Grindle, C. F., Hastings, R. P., Saville, M., Hughes, J. C., Huxley, K., Kovshoff, H., Griffith, G. M., Walker-Jones, E., Devonshire, K., & Remington, B. (2012). Outcomes of a behavioral education model for children with autism in a
mainstream school setting. Behavior Modification, 36, 298-319], and in Norway [Eldevik, S., Hastings, R. P., Jahr, E., & Hughes, J. C. (2012). Outcomes of behavioral intervention for children with Autism in mainstream pre-school settings. Journal of Autism and Developmental Disorders, 42, 210-220]. In both studies, we described how ABA comprehensive models can be delivered within existing services (mainstream pre-schools, and a special class within a mainstream infant school), and showed that outcomes were better than existing “education as usual” models.

The main point here is that it would be a mistake to evaluate these research studies as if they were designed to test the efficacy of comprehensive ABA-based education for children with autism. They should properly be considered as Phase 4 research - developing/describing and evaluating the day-to-day delivery of comprehensive ABA intervention methods.

More implementation research (practice-based evidence) needed

We cannot wait 20+ years for basic science to inform new interventions and for these to go through the process of gathering evidence right to the implementation stage. More evidence is needed NOW to inform the design and development of educational services for children with autism. It would also be premature to base these service design and delivery discussions on evidence about interventions that have only reached the stage of being tested for their efficacy (Phase 2). Such interventions are very encouraging in terms of their outcomes, but we have no idea whether they will prove effective and whether they will deliver good outcomes on a large scale.

Intriguingly, ABA-based intervention, especially for young children with autism has been around so long that there are examples and data around about successful implementation. Service providers and commissioners must take notice of these unique data partly for what they say about ABA-based interventions but also the lessons that might be learned in relation to the delivery of other interventions.

Just as I advocated earlier in this blog for pro and anti ABA camps to work together towards a common goal, researchers and service providers need also to collaborate on implementation research. This need is urgent. Educational services and professionals need to clearly define their models and identify the evidence informing what they are trying to do. What is the rationale for your educational model? Researchers then need to turn to more applied research projects, devising ways to evaluate these defined models. We can then start to have evidence-based educational service delivery in the field of autism.

Funders of autism research can also play a role here. They all seem to recognize that there has been too much emphasis on funding basic science on autism and less attention given to intervention. However, it is important to use the evidence continuum model to inform funding of good quality research at all Phases. A focus only on Phase 2 efficacy trials will take a long time to have impact on children with autism, their families, and educators. Yes, children with autism and their families deserve the highest quality intervention research. However, what is the point if this work does not or cannot inform practice and services?