Reducing unwarranted variation – Do we really know what we are talking about?
Blog 6 in the Quality Matters: From Insight to Impact series
Quality Improvement (QI) is critical to delivering on the NHS Plan. This series of articles from Associate Directors, Ian and Laura, provides a personal and practical insight into how QI is needed more than ever.
Veran Patel, Director, Health & Social Care
“Reducing Unwarranted Variation” has become one of the most popular phrases in the transformation / improvement world. It just spills off the tongue to describe why we need to change something or what we will achieve if we do.
And who can argue with that?
After all, we all know that the NHS is riddled with variation. And indeed the godfather of modern improvement science himself, Edwards Deming famously stated “If I had to reduce my message for management to just a few words, I’d say it all had to do with reducing variation”
So, it must be true. And yes, it makes sense. But, like so many other improvement ideas and concepts, so many of us lack the deep understanding of what that little phrase really means or what indeed Edwards Deming was suggesting with his statement. I include myself within this bracket. It’s actually only after quite a number of years of thinking about it and digesting it do I think I now actually get it!
In our improvement training, we give our delegates a little brain teaser…you may like to try it…
Let’s say there are two NHS services, in different trusts but delivering the same service. Both need to achieve a performance target of 95% of patients treated within 18 weeks of referral (RTT).
Service A has good standard work that is consistently applied to all patients. Their data suggests they have a standard deviation of 2 weeks.
Service B is a bit of a different story. The service is driven very much by individual ways of working and there is very poor standardisation. Unsurprisingly, their standard deviation is 5 weeks with some patients navigating through the pathway very quickly and others taking much, much longer.
The question we pose is this – what mean average pathway time does each service need to achieve to meet its RTT performance target? (stop reading now if you want to have a go).
It’s perhaps a bit unfair to ask you to do this, not least because if you were in the training we would have just talked about standard deviation (which you need to know to be able to work it out!). However, the answer is: Service A = 14 weeks and Service B = 8 weeks. Why is this significant? Well, because Service A is far more consistent, they are afforded much longer to get patients through their pathway than Service B. So yes, whilst reducing unwarranted variation no doubt improves the patient and staff experience it is also the way we should be seeking to achieve our performance targets.
What does this mean in practice? Not so long ago we were asked to support some improvement work around a colorectal cancer pathway. The aim was to improve their pathway performance and seek to ensure that key performance targets were met, namely: 85% of patients receiving treatment within 62 days of referral and; the faster diagnosis target of 75% patients receiving a diagnosis within 28 days.
The cancer improvement team had been working hard on trying to make improvements with little impact on performance. They had been applying what I describe as the ‘traditional’ method: that is to monitor the progress of all patients along the pathway and then focus on developing solutions when ‘breaches’ occur (i.e. patients go beyond the desired timeframe for a part of the pathway). The logic behind this type of improvement is obvious, however I think it is ultimately flawed.
- It doesn’t tell you where within the pathway the problem arises. You know you have a problem when the breach occurs, but it could have been something much earlier in the pathway that led to the delay. The result is that we guess what we need to change and rarely make a positive impact on performance.
- You focus on preventing breaches, rather than reducing variation on a day-to-day basis. So, you might add extra checks (more work) along the pathway to avoid breaches as opposed to looking at and stabilising the daily work along the pathway that might reduce unwarranted variation.
Our approach was to deeply understand the pathway for all patients with a positive cancer diagnosis over a12 month period – and by understand we mean chart their entire journey from a time perspective, with all its warranted clinician variation so we could answer both what happened (in detail), how long it took and what level of unwarranted variation existed within the pathway. From this exercise, we learnt a number of things:
- Getting hold of the data was extremely difficult. Extracting key time stamps from systems was impossible, unless it was one of the key performance indicators. So, we were forced to extract it manually by going through records, patient by patient. It’s not an unfamiliar story for the NHS – we are awash with data that we need to provide for performance purposes, but we have little that help us to understand how well our systems and processes are working (until it’s too late of course and we suffer breaches). But without this data it is impossible to understand the level of variation we have in our processes beyond the anecdotes of staff and patients.
- There was LOTS of variation! In fact, high variation was the norm rather than the exception. Some ‘lucky’ patients were expedited through the pathway in timely fashion, a bit like you are lucky on a journey when all the traffic lights happen to be green. But if one or two of those lights were amber or red, and that was normally the case, the chances of pathway performance being met for that patient fell sharply. For the most part clinicians and administrators were unable to explain the variation, so it certainly appeared unwarranted. But having this deep understanding of the entire pathway and its variation allowed us to start conversations around:
- what is really going on here?’
- why is this part of the process not working?
- why do we set our process to do this step in 7 days when we could do it in one?
- how might we fix it?
- what do we need to measure, to know it is improving?
And yes, there were certainly things that our partners could do to support – primary care, pathology, etc but it became undeniable that there were in house improvements to be made to standardise procedures and reduce variation.
So, in conclusion – do we understand what we mean by reducing unwarranted variation? I’d say, sort of. Whilst we know it’s a good thing to do, I’m not sure we understand how vital it is in our pursuit of our performance targets. We certainly don’t always have the data to hand to map out all our pathways to help us fully understand them, which is quite an admission when you think about it. And of course, extracting that data and observing processes to fill in the blanks, is going to take time to do; and time is something people are not awash with. But we would argue that the significant time spent gathering monthly performance data and say, cancer pathway breeches, would be better directed on seeking to improve performance, not measure it.
Laura Woodward and Ian Railton are Quality Improvement (QI) experts and are Associate Directors at TIAA. This series of fortnightly blogs are their insights into a long career and successful track record of working with healthcare organisation to improve productivity and better outcomes for patients.
Click below to catch up on the other blogs in the series –
Quality Matters: From Insight to Impact; Before solving a problem, you must first understand it
Quality Matters: From Insight to Impact 7 Traits of an Improver
Quality Matters: From Insight to Impact – Control Versus Autonomy – Seeking the Balance
Quality Matters: From Insight to Impact – Easier, Better, Faster, Cheaper