“How do I know?”: Some thoughts on PBS accreditation

High quality delivery of evidence-based practices is a good thing

There are evidence-based, “consumer”, and likely economic arguments for ensuring that any approach to intervention/support is implemented with fidelity. By fidelity here, I am primarily thinking about high quality delivery of an intervention. Any evidence-based practice needs to be delivered with a high degree of quality (i.e., “done properly”) to have the best outcomes and to deliver anything like the promise of any more tightly delivered original research-based evaluations of that practice. That’s the evidence-based/research argument.

My “consumer” argument is simple and actually is the most important. In this case, people with intellectual disabilities and their families and carers deserve high quality support and intervention. Thus, they also deserve access to the highest quality Positive Behavioural Support.

The economic case derives from the first point about achieving better outcomes with better implementation (higher quality delivery of supports and intervention). These days, there is considerable interest in an “invest to save” perspective: if services can invest in wider scale implementation of evidence-based practices of various kinds, the high costs of picking up serious problems later might be avoided. In the case of people whose behaviour is described as challenging, perhaps this could include avoiding high cost placements in inpatient settings, shorter stays in such settings, shorter term use of high staffing ratios, and making sure people can stay in their own local community with security of residence (thus avoiding the costs of multiple moves between services and settings). Please note that any simplistic economic case being made here is not in my view the most important, but it is a reality we have to address.

Any economic models for investment in new services, in training for support staff, or the delivery of evidence based practices generally assumes at least a reasonably effective impact from the services, training or intervention. Basically, there is an assumption that whatever is done will be done well.

Current “standards” for Positive Behavioural Support

There may be several ways to improve the chance of high quality implementation of evidence-based practices, but the key features identified in the research literature relate to organisational commitment, good quality training of staff and ongoing expert supervision, and the articulation of clear standards for individuals and services/teams to meet.

How does this relate to the current situation in the UK around PBS? The key is in the answer to some apparently simple questions: How do I know that this practitioner is an expert in PBS? How do I know that this PBS training course will ensure I develop high quality PBS competencies? How do I know that this service/this team “does” high quality PBS?

The answer to these very important questions at the moment is that you have to decide for yourself/work it out yourself. The PBS Academy has produced a very detailed framework describing PBS competencies (see: http://pbsacademy.org.uk/pbs-competence-framework/). An answer to the “How do I know?” questions could be linked to this framework such as: “I received training based on the PBS competencies framework”, or “This training course is mapped onto the PBS competencies framework”. Unfortunately, this does not tell us very much because we naturally then need to ask something like “but who says it was any good?”. The PBS Academy has also published series of resources that help different stakeholders work out the answers to the “How do I know?” questions themselves. Thus, there are tools for people with intellectual disabilities, for family carers, for support staff, for service providers, for commissioners, and soon for people charged with inspecting the quality of services.

Personally, I think that these PBS Academy resources are helpful not least because they were produced with people from each of the stakeholder groups and are designed to address the particular nature of their “How do I…?” questions. These resources are in any case produced under a Creative Commons license so that they can be easily taken on board and amended by anyone who wants to use them. As I have said before, please do use and adapt these resources to make them work for you.

In terms of individuals delivering PBS, some professionals will point to the international Behavior Analyst Certification Board (BACB) and their levels of certification. These are indeed clear standards, and you should expect anyone with these credentials to know all about the Applied Behaviour Analysis aspects of PBS. However, behaviour analysis certification is not UK-based nor necessarily sensitive to the UK context, it is not specific to PBS, and of course ABA is a core part of PBS but not the whole of PBS. So, this may be useful but I do not think that it is a full solution and it primarily addresses the highest level expert practitioner only. There are University courses in ABA in the UK at postgraduate level in particular. Graduates from these programmes are likely to understand much more of UK context and may also meet the BACB certification requirements.

Interestingly, some UK Universities from 2016/17 are bringing in new PBS-specific routes through their postgraduate training in ABA, and others (some of the same ones too) have been delivering “challenging behaviour” courses for several years. Other PBS/challenging behaviour training courses have been developed that are accredited against a relevant UK qualifications framework. All of this training is delivered by respected education providers and is “accredited” somewhere within the UK public education provider/system. Although this is a good thing and they are undoubtedly good courses, there is a still no nationally agreed standard for PBS training against which these existing training courses can be tested. In addition, there are many PBS training providers who offer their own quality assurance (and several take this seriously) but they are conflicted as training providers, and there is no national standard for PBS training against which they can be checked.

UK accreditation for Positive Behavioural Support

Currently, in the UK, there are no nationally agreed standards for PBS. Does this matter? My argument is that it certainly does. First, PBS is recommended as a part of the plans going forward to address the Transforming Care agenda. It is not the whole solution to the challenge of challenging behaviour, but it is a part of it. Second, NICE guidance and quality standards are also clear about the need for functionally-based approaches (essentially, PBS). Thus, it is important to ask about quality and standards of PBS as a recommended approach.

Some may argue that setting standards and then having an accreditation process will cost money and that we cannot afford this. This is likely true – accreditation does cost money. How else though can we set proper standards and ensure high quality without such a process? Relying on people to sort it out for themselves (the current situation) is not appropriate. What is needed is a national accreditation process that can be trusted so that people do not have to sort it out themselves.

What if it does cost more money? Don’t the people affected by the transforming care programme and care scandals deserve high quality supports? In addition, if we can raise standards and somehow ensure higher quality supports we won’t have to use the sticking plaster high cost responses that we use when things go wrong. In the end, we may have invested and indeed saved!

The good news is that the PBS Academy has secured some funding and contributions of time from people to write/develop PBS accreditation standards for individuals, providers/services, and for PBS training courses. Thus, we’ll get on with this writing process using a similar methodology to that we have used before. The resulting standards will also be released as always using a Creative Commons license – hopefully by Spring 2017. This will mean that whoever might be tasked with PBS accreditation in future will have a head start. If the powers that be do not finally see the wisdom in commissioning PBS accreditation nationally, then at least the standards will be there for those committed to high quality PBS to use as a community.

Who is Challenging Who(m)? Using research evidence on the perspectives of people with intellectual disabilities and carers

If you ever read these blogs, you might remember some research I talked about a couple of years ago now about "554 voices". This was about results from syntheses (systematically bringing together the findings across several research studies) of qualitative (mostly interview) research about challenging behaviour from the perspective of either people with intellectual disabilities or their carers.

A key question with research on the perspectives of people with intellectual disabilities or their carers is how we actually use the data to change things is some way. There are many ways to think about this, but in this blog I want to focus on a couple of points in relation to our research papers entitled "I'm not a patient, I'm a person" and the review of carer research "He's hard work, but he's worth it" and how we've gone on to use these.

That's NICE?

One way in which we're delighted that our reviews and sytheses have been used is that they were carried out and published just at the right time to influence a number of National Institute for Health and Care Excellence (NICE) guidelines focused on children and adults with intellectual disabilities. The evidence included in the NICE guideline on "challenging behaviour" had chapters devoted to our reviews and syntheses of qualitative research with people with intellectual disability and with carers.

This is all lovely (very NICE, we might say) and hopefully the recommendations in the challenging behaviour guideline, influenced by our research findings, will make a difference to children and adults with intellectual disabilities and their families and carers. The problem is that it is hard to see the potential impact of the research through this route. I'm thinking that this may be necessary, but it won't be sufficient to stop there. I do think as researchers we also need to work to more direct translate research into practice somehow. So, how have we done that?

Empowering carers with PBS resources

Not everyone will be happy with the leap from our synthesis of carer research specifically to Positive Behavioural Support (PBS). However, PBS is a part of the supports and services potentially on offer to carers with a relative whose behaviour has been labelled as challenging. Working as a part of the Positive Behavioural Support Academy and in partnership with the Challenging Behaviour Foundation, we recently produced some practical tools for family carers. A short introduction to these resources and links to the free downloads can be found via a short blog on the Paving the Way website, so I won't repeat that information in detail here. We will try to evaluate the usefulness of these resources for family carers. The important thing for now is that, specifically in relation to PBS, there are some resources to help family carers deal with services and professionals, recognise what is "good" in relation to PBS practice, and hopefully to get high quality support for their relative. These practical issues were all common themes in the research.

Who's Challenging Who(m)?

Having heard a number of individual stories from people with intellectual disabilities whose behaviour had been labelled as challenging, and carried out our systematic research, it was clear that a piece missing from training for social care and other support staff and professionals was explicitly the perspective of people with intellectual disabilities. Working with people with intellectual disabilities, we designed and carried out a pilot test of a half day training course for support staff using the findings from the review research and also some of the personal perspectives of the co-trainers with intellectual disabilities.

Since that pilot, we have changed a number of things abuot the Who's Challenging Who training. The key first point is the way that we describe the training. Originally, we talked about co-trainers with an intellectual disability delivering the training to staff alongside a co-trainer who did not have intellectual disability. Now the people with intellectual disability are the trainers and we have a facilitator working with them to support them and the training session. We have also introduced more of the personal stories of each trainer - integrated within the broader research findings. So, a second key point is that the training is not just about one person's perspective, but it does include that perspective. The training attempts to give a wider perspective about the views of people with intellectual disability drawing on the research evidence. A third issue is that some staff looking at the training wanted included an official perspective on "challenging behaviour" such as definitions used within services and/or recommendations from the NICE guidance (see above). We resisted this and instead we're now very clear that the Who's Challenging Who training is ONLY about the perspective of people with intellectual disability and is nothing about any other perspective. A final change to the training since the pilot research has been the introduction of two short telephone coaching sessions for people going back to their residential settings to change things after the training course.

How are we testing if this makes a difference? For this, we are fortunate to have research funding from the National Institute for Health Research School for Social Care Research to carry out a large scale randomised controlled trial evaluation of the training. In the first phase of this project, we recruited 58 residential settings to take part. If you have services in England especially (potentially also into Wales) and might be interested in the Who's Challenging Who training, we are now recruiting the final 60 services for the second phase - recruiting up to July/August 2016. Check out information about the project and an online expression of interest form. Every participating service gets the training, some within a few weeks and some a few months later.

The simple existence of a Behaviour Support Plan (BSP) is not enough

A Behaviour Support Plan is not the same as Positive Behavioural Support

In the Positive Behavioural Support Competencies Framework for the UK (see: http://pbsacademy.org.uk/pbs-competence-framework/), there is a central role for a document called a Behaviour Support Plan (or BSP). This document is the culmination of the process of detailed assessment, including work with people whose behaviour is being described as challenging and their carers (family, and paid where relevant). The BSP should include a description of what is going on, and why it is happening (if you like, a formulation). The BSP should also specify the variety of intervention approaches to be implemented, as agreed with the person and other stakeholders. There should also be a process for gathering information about whether the implemented plan is working and if not how the plan can be reviewed in a responsive way and re-developed where necessary.

The PBS competencies framework is very clear about all of the work that needs to go on around the BSP to ensure that PBS is going to be done properly. The PBS Academy definition of PBS also makes it clear that the BSP is only a part of the framework we know as PBS. Bringing together key information in a BSP is a core of the “Process” parts of the definition of PBS from Gore et al. (2013). Importantly, these process aspects make up four of the 10 components of PBS.

Thus, BSPs are important but they are not the whole of PBS. If all 10 PBS components were equally weighted (this is just for effect you understand), then a good BSP is only 40% of PBS.

The dangers of the simple presence of a BSP being an indicator of quality

It should be clear by this point in the blog that if you want to work out if a person is receiving high quality Positive Behavioural Support, you cannot rely on the simple presence of a Behaviour Support Plan as your only piece of evidence. I am not suggesting that this happens, although from some informal comments lately it is possible that this is happening for people whose behaviour is described as challenging.

Of course, the BSP should be considered as only one aspect of overall PBS quality. It is an important part. Even simply for the BSP, in my mind there are at least three quality sub-components:

1.  It matters how a BSP has been developed both from a values and an evidence perspective. First, and rooted in values and rights, a BSP must have been developed properly with the person whose behaviour has been described as challenging, and with their carers (family, and also paid carers). Second, and from an evidence-based perspective, if this involvement or co-production is done properly the BSP will have what we would call improved Contextual Fit. Good contextual fit might essentially mean that the BSP makes sense to everyone and addresses issues important to all stakeholders in a way that might fit within the context in which the person lives. Increased contextual fit is associated with better outcomes.
2. As I sneakily introduced earlier, it matters how “good” the BSP actually is. As an extreme example, you could see a BSP where the interventions described are entirely based on aversive strategies. This is completely inappropriate and should not be in a good BSP. So, you do have to open up the BSP box and check what’s inside. Although they are not perfect, and more research and development is needed to improve them, there are tools to audit the quality of BSPs. One such tool that we’ve used in research already is the BSP-QEII and the very detailed manual for this can be found for free on the internet: http://www.pent.ca.gov/beh/qe/bipqe.html
3. Finally, it matters whether the BSP is actually being used. The document has to be “living” – continually reviewed and revised, but also actually making a difference to what people do in practice.

You might agree with these points but then ask me what can be done about some of these issues. In addition to the existence of a quality tool for auditing BSPs already available out there, we’ve been working on some practical tools and methods to address involvement/co-production of BSPs and also the issue of how to get BSPs used in practice.

How do we involve people with learning disabilities in their BSP?

There will be many areas of good practice out there I am sure designed to involve people with learning disabilities in their PBS planning, and I encourage people to share these. This was a core issue for the PBS Academy in producing various PBS-related resources recently released free online. See the following link for the tool that can be used with people with learning disabilities: http://pbsacademy.org.uk/people-with-learning-disabilities/

If you don’t like aspects of this tool, don’t just whinge about it. Take it and improve it, and then release the revised version for everyone for free. All we ask is that the original source is recognised and that you make any revised tool freely available for anyone to use and go on to adapt further. This is the point of the Creative Commons license under which these materials were released.

How can we make sure that Behaviour Support Plans are being used?

One way of checking on whether BSPs are being used in practice is to check out what you (family carer, provider etc) are seeing in terms of PBS practice. Again, the PBS Academy web pages include some tools to help you with this: http://pbsacademy.org.uk/

Such tools are important but do not really address the key problem of implementation of the interventions in a BSP. Fidelity/integrity (i.e., consistency, quality) of implementation of any intervention is crucial to success. We can have a great plan, but unless the plan is actually followed we’re not going to have successful outcomes. It is easy to criticise BSPs in two ways here. First, people just don’t follow them even when they have been carefully produced and are of good quality. Second, from the other side, some “expert” comes in and writes a very nice document but that’s it – that’s all we get. Thus, the BSP can lie dormant in a cupboard or drawer until someone wants to make sure that one exists at which point it is brought out, dusted down, and shown to an inspector or other person.

Interestingly, this key implementation question has been rarely directly considered in PBS research. Of course, there are related research areas (such as the idea of ensuring contextual fit). However, researchers have typically considered that the problem lies somewhere with the carers who are in a position to implement a BSP. This isn’t explicit of course, but implied by the research that has been carried out to date. Researchers have considered what the barriers might be to implementing behavioural interventions such as PBS – many of these are about the broader environment and not carer factors. Researchers have also examined staff technical, theoretical or practice knowledge and ways to improve this; or considered their beliefs about behaviours that challenge. My own work in this area suggests that carers are perfectly able to articulate appropriate ways to think about challenging behaviour. However, they still might in the moment act in a way that is counter to a clear BSP. One factor in this is likely to be the emotional demand of caring for individuals who have behaviours described as challenging. That’s a whole other story, but not one to be ignored as I have indicated previously in this blog: http://profhastings.blogspot.co.uk/2013/05/winterbourne-view-will-happen-again-and.html

Alison Branch (Northumberland Tyne and Wear Foundation NHS Trust), Carl Hughes and Mike Beverley (Bangor University), and I have just had a research paper accepted for publication in the Journal of Intellectual and Developmental Disabilities that takes a different perspective. We considered that carers may not be best prepared to actually implement a BSP because no-one directly teaches them what to do. Yes, there may be some discussion and demonstration by an “expert”. However, in the moment of real world work to be able to successfully implement an intervention strategy you need to be already fluent so that you almost “automatically” respond appropriately. So, Alison took individual BSPs and broke them down into their small components and used flashcards and fluency-based teaching procedures to help carers get to know the content of the BSP. Carers were taught to teach themselves, using several short one minute practices over several days, spending on average 78 minutes of their time. Compared to a control group, those following the self-directed learning procedure performed much better on late tests of their knowledge of the BSP and its application in hypothetical scenarios.

This was an initial research study of an innovative practice, and as yet we’ve not moved on to be able to test whether such a teaching practice actually also leads to better outcomes for people with learning disabilities. However, if you’re interested in this the paper will be published sometime later this year hopefully and in the meantime I’m sure you can find one of us online and email us requesting a copy!