How to make improvements that really work
The symptom is visible. The cause is hidden. The constraint is structural — meaning it is produced by the design of the system itself: the boundaries, the budget lines, the incentive structures, the accountability arrangements. It will keep producing the same outcome regardless of who is in the role or how hard they work. The contradiction is assumed away — meaning the tension that keeps the constraint in place is treated as an unavoidable fact of life rather than a question worth asking. The moment it is named explicitly, it often dissolves.
And the natural human response — act on what’s visible, react to what’s loudest, fix what you can reach — almost always addresses the wrong thing. Every great improvement framework is a method for bridging that gap between what’s visible and what’s true.
“If we keep working the way we are doing it, the outcome will be the same.”
Syd Stewart — StepChangeAnalysis.com
You have tried to make things better. Perhaps it worked briefly — then the metrics drifted back. Or the results never came at all. Perhaps you have changed the initiative before it had time to work, because the pressure to show progress was too great to wait. You are asking three questions that most improvement tools cannot answer: is this the right kind of change? How do I know if anything has really changed? And: why do improvement actions eventually fail even when everyone is trying hard?
Built for healthcare improvers who need honest evidence of change.
Test your time-series data, avoid tampering, and know what to do when nothing changes.
Helping people think better about improvement.
Every concept page on this site exists to answer a question that persistent problems raise. The blue concept links throughout the site are thinking tools — follow them when a step raises a question you want to understand more deeply. The goal is not to provide answers but to equip you to ask the right questions, in the right order.
Start with the 7-step improvement method — or go to The Right Questions for the 54 diagnostic questions mapped to each step.
The problem with standard improvement measurement
Most improvement programmes are measured with run charts, RAG (Red-Amber-Green) status ratings, or simple before-and-after comparisons. These tools answer the question: did the metric move? They cannot answer the more important question: has the system structurally changed?
A metric can move for many reasons — seasonal variation, a one-off event, a change in recording practice, random fluctuation around an unchanged mean. Standard charts cannot distinguish between genuine structural improvement and noise. This means that improvement programmes are regularly declared successful when nothing has changed, and abandoned when something genuinely has.
The three questions you should already be asking
If you work in healthcare improvement, you will recognise the Model for Improvement (Langley et al., 1996) — the three questions that underpin the PDSA cycle used across the NHS and internationally. Bootstrap CUSUM and the Deming framework on this site sit directly inside that model:
What are we trying to accomplish?
Have you identified the right goal? Is the constraint correctly identified? Are you addressing the root cause or the symptom?
How will we know a change is an improvement?
Has the process structurally shifted to a new mean — or are you seeing variation around an unchanged level? Standard run charts cannot tell you. Bootstrap CUSUM can.
What change can we make that will result in an improvement?
Are you intervening at Level 1 (output), Level 2 (process), or Level 3 (system)? Only Level 3 interventions produce structural change that Bootstrap CUSUM will confirm.
Bootstrap CUSUM sits specifically in the Study phase of the PDSA cycle (Shewhart’s original cycle, developed by Deming and formalised by Langley et al.) — the phase where you ask whether what you did actually worked. It is the most honest tool available for that question.
Why improvement actions eventually fail
The data on this site documents four reasons why well-intentioned improvement actions fail to produce lasting structural change. Each is visible in the Bootstrap CUSUM results of real NHS and public sector data:
1. The constraint was never identified
You optimised everything except the bottleneck. Goldratt’s insight: you cannot improve throughput by optimising anything other than the constraint. The constraint on NHS dementia diagnosis was not GP referral rates — it was the absence of a proactive, coordinated system. The target addressed the output and left the constraint untouched.
2. Purchased behaviour, not system design
Deming’s 12th Point: incentive schemes purchase behaviour rather than embedding it. The behaviour disappears when the incentive is removed — taking the metric with it. The NHS dementia CQUIN is a precise demonstration: the rate rose while the incentive was active, then drifted when it changed.
3. Reinforcing loops were never broken
Some systems maintain their poor performance automatically through self-sustaining causal loops. The NHS A&E system has been in decline for 15 years despite repeated policy interventions — because the reinforcing loops that drive the decline were never structurally addressed. Bootstrap CUSUM finds four stages of decline. Not one intervention is visible as an improvement.
4. Tampering before results appear
Deming identified tampering as one of the most common causes of persistent failure: intervening again before the first intervention has had time to work. Level 3 structural changes have long lag times. Abandoning the intervention during the lag and replacing it with something new resets the clock. The political cycle almost guarantees this will happen.
Try it on your own data
The free Bootstrap CUSUM tool
Paste your CSV data. Run Bootstrap CUSUM and X-mR analysis. Find out whether your improvement programme has produced structural change — or whether you are seeing variation around an unchanged system mean.
Browser-based · No data uploaded · Works on any time-series data · Any sector
▶ Open the StepChange AnalyzerWorks in any sector — not just the NHS
The mission is to give people better tools and a better mental model to ask better questions about whether things have improved through structural change — in any organisation, any sector, any context where improvement is attempted and measurement matters.
Worked examples from real data
Every article on this site applies Bootstrap CUSUM to real data and asks the same questions: what has structurally changed, when, and why? Each one demonstrates what the tool finds — and what the Deming and systems thinking framework reveals about why things are the way they are.
Why nothing has worked: NHS A&E
15 years of monthly data. Four structural stages of decline confirmed by Bootstrap CUSUM. Not one policy intervention visible as an improvement.
The 66% dementia diagnosis target
Bootstrap CUSUM on 9 annual observations. The constraint was never addressed. Two reinforcing loops. The PM’s Challenge did not produce detectable structural change.
GP appointments: flat since 2018
Doctor contact rates statistically unchanged through COVID, lockdowns, and £1bn/year workforce expansion. Bootstrap CUSUM finds the flat line the policy narrative obscures.
Never events that never stopped
Wrong-route medication administration declared “wholly preventable” in 2011. Bootstrap CUSUM finds a flat process at 17.5 events per year. The engineering solution exists and has not been implemented.
ULEZ worked — but not when you think
Bootstrap CUSUM finds the air quality signal appeared 18 months before ULEZ was introduced. What does that tell you about attributing improvement to a specific intervention?
The hydrogen plant problem
How residual CUSUM catches what standard SPC misses in a process industry setting. Bootstrap CUSUM applied to chemical engineering data.
The thinking framework behind the tool
Bootstrap CUSUM answers Question 2: how will we know a change is an improvement? But Questions 1 and 3 require a different kind of thinking — the kind that W. Edwards Deming, Brian Joiner, Eliyahu Goldratt, and Peter Senge developed over decades of working with systems that resisted improvement.
The core ideas are these: most problems are caused by the system, not the people in it (Deming’s 94% rule). The constraint drives throughput, not the non-constraints (Goldratt). Level 3 interventions address the system; Level 1 and 2 address its outputs (Joiner). Reinforcing loops are self-sustaining — they require structural intervention at a specific node to reverse, not pressure on the output (Senge). Constancy of purpose over long time horizons is the prerequisite for structural change (Deming’s first Point).
The Glossary explains each of these concepts in plain English. The Why nothing changes page provides the full five-stage diagnostic method for finding the root cause and designing a structural response.
“Give people better tools and a better mental model to ask better questions about whether things have improved through structural change — and to understand why, when they haven’t.”