Are education systems interested in using evidence in practice?

 

I could have written this blog about any school system, but I work at Bangor University in Wales, our research team has carried out education research (especially in special education in Wales), and there is some debate in Wales this week about using technology to deliver educational outcomes for young children. Follow this link for a BBC news story about increased reading skills and self esteem in children working using tablet devices in their educational environment: http://www.bbc.co.uk/news/uk-wales-22816389

Others have already asked what exactly is the evidence for the outcomes discussed in the story. It is not clear if a controlled research comparison has been conducted (let alone a gold standard randomised controlled trial evaluation), and almost certainly we do not know (if there are indeed proven positive outcomes) which active ingredient led to any effects. The teaching is delivered using tablet devices and the changes seem to have been attributed to the technology. However, it seems clear that there is a peer teaching element involved, perhaps any instruction delivered via computers as teaching machines is more consistent and directly available to every child than is typical in a classroom, and almost any new initiative can be driven forward by enthusiastic teachers and/or children? So, if there are positive outcomes, there may be many explanations and it matters what the explanation is if we are to understand the implications for Welsh education.

My point is not to be grumpy and critical in this blog (although there are clearly many unanswered questions). However, Welsh Government minister for education Leighton Andrews AM has suggested in the media that all schools in Wales can learn from the tablet-based education work in the Swansea area. First point is that such a recommendation is unwarranted until we know WHETHER the teaching approach works, and then HOW it works (what is actually the active ingredient). The remainder of this blog is designed to make a couple of other points.

A cautionary tale

Over a period of five years, a research and practitioner team from the School of Psychology at Bangor University (myself, Dr Carl Hughes, Dr Corinna Grindle, Maria Saville) worked with other colleagues and in partnership with Westwood School in Flintshire and both Flintshire and Wrexham education authorities to develop and evaluate a comprehensive educational model for young children (Keystage 1) with autism. We took evidence from the existing research literature on educational approaches for children with autism, especially comprehensive behavioural models (see one of my other blogs for a summary - http://profhastings.blogspot.co.uk/2012/12/autism-and-evidence-4-does-aba-work-for.html)

Our main task was to use this evidence and to translate it in two main ways. First, much of the existing research evidence focused on home-based or specialist clinic-based early behavioural educational intervention. We were working in a school setting, and a mainstream one at that. Second, no existing research focused on how the behavioural model could be dovetailed with the requirements of a national curriculum and a broader education system. I cannot pretend that this translation was easy, but the behavioural intervention team, teachers and classroom assistants, the school, and the education authorities pulled together to make it work. Although we drew on much of the existing evidence-base and the intervention approaches developed elsewhere, two considerable changes were made to what had gone before: an emphasis on mainstream educational integration for all of the young children with autism, and an exercise to match the evidence-based behavioural curriculum with the National Curriculum in Wales.

Not only was the practice developed into a model that dovetailed into the system (thus a great example of translation of evidence-based methods into practice), but the outcomes for the children with autism were excellent – when compared with similar children with autism receiving “education as usual” elsewhere in the UK (see http://bmo.sagepub.com/content/36/3/298.abstract). This research has proved popular – it is currently the fourth most read paper from the journal in which it was published over the past 12 months. So, someone likes it.

The practice and research team were also able to develop new practice and carry out evaluation of this practice to contribute to the evidence base. An excellent example is a paper we wrote that is now available online and will be published later this year: http://onlinelibrary.wiley.com/doi/10.1002/bin.1364/abstract. This project focused on adjusting an evidence-based mainstream online reading programme for children with autism and evaluating outcomes. The children with autism gained up to 3 years of reading age over online reading instruction period (sounds just as impressive as the Swansea data, right?!). This is not a controlled research study, but the point is that practice and evidence was being developed, and also within the spirit of the mainstream provision. Thus, the reading instruction used a mainstream evidence-based programme; not one developed specially for children with autism.

What has happened to this Westwood school autism provision now? Unfortunately, the education authorities decided to close it down (http://www.bbc.co.uk/news/uk-wales-politics-14331574).

The cautionary tale is that evidence of outcomes is clearly not enough for excellent practice to survive in the Welsh education system. Even the experience of taking existing evidence-based practices and translating them to be delivered within the existing education system is not enough. However, it seems that nice ideas perhaps without strong evidence (in the Swansea tablets case perhaps) do survive and thrive in the education system.

Although the Westwood school model was closed down, there is a massive legacy from the project. First, there are still further reports on new evidence-based practices for children with autism that will be published from the research over the coming few years. Second, the staff (our University experts, and also the classroom assistants we trained) have also gone on to make a big impact elsewhere. For example, special schools in Conwy, Gwynedd, and Anglesey have been using the online reading programme to teach children with intellectual disabilities to read; and other schools have evaluated the numeracy teaching model adapted by the Westwood project team. Third, the practice and evidence developed at Westwood is being used internationally (in England, Norway, Switzerland, and France amongst others). Fourth, and most importantly, the children with autism and their families who were educated by the Westwood team got a cracking education and learned a great deal of stuff whilst they were there. I hope that it has continued to make a difference to their lives.

Some priorities for the future

I hope that this blog doesn’t come over like a whinge about the closure of the Westwood autism education project. Of course, we are disappointed that the project was closed down. However, it has sent ripples of positive practice and great outcomes for disabled children all around the world. So, Wales can be truly proud that a small nation can do things sometimes to make a difference across the world (thanks to Wayne Crocker of Mencap Cymru for that “quote” today, but in a different context – I’ve just borrowed it).

What can Welsh Government do to ensure that evidence-based practice in education in Wales gets rolled out within the education system?

First, it is clearly important that there are ways to recognise excellent work that actually comes with strong evidence. The Westwood autism project is an example – the model and evidence is good enough outside Wales, so why not within? Are education experts just not aware that the data are there and that people have already developed a translational model that has been proven to work? I imagine that this applies to other research evidence of practices within the Welsh education system – that good evidence is simply not being used.

Second, there probably does need to be a stronger evidence culture within Welsh education (and in most other education systems – this is not just about Wales). This is needed at a number of levels. Policy makers may need to develop more of an understanding of how evidence works and how important evidence is. Teachers and schools also need to take some responsibility to understand evidence, how it is produced, and its limitations and strengths. This can start with the initial training of teachers, but it is an ongoing professional development issue. Finally, teachers need the tools to be able to evaluate what they do day-to-day within their classroom to ask “is what I am doing working FOR THIS CHILD?”. Such tools exist using ongoing simple data collection in the classroom, and are taught on the Applied Behavioural Analysis postgraduate programmes at Universities in Wales (at Bangor – Dr Carl Hughes is the programme director, and the University of South Wales – Dr Jenn Austin is programme director). Another Wales strength – so make use of it!

Third, where is the investment in education research and education researchers within Wales? Considerable investment is clearly needed to ensure that the Welsh education system generates high quality evidence, to ensure that education policy is evidence-based, and that our children get the high quality educational input that they need. There is a large investment in health and social care research within Wales via NISCHR. However, we know that many health and social problems are related to poor education and much inequality is related to poorer education. Thus, education is key to the well-being of everyone in Wales and actually important presumably to the long term economic burden of health and social care problems. So, the case for funding education research and education researchers within Wales is both direct and indirect.