No Change Detected: What to Do Next
Bootstrap CUSUM found no statistically significant step change. Before reacting — or not reacting — three questions need honest answers: is the system genuinely stable, is that stable level acceptable, and if you applied an intervention, did it actually reach the constraint? The answers lead to very different next steps.
If no change is detected, default to waiting (don’t tamper): set a review date, improve the theory/measure/test, and re-run after enough new data points.
No change detected has five possible meanings — the system is stable and acceptable; a change point is forming but not yet confirmed; the intervention was at the wrong Joiner level; the wrong constraint was addressed; or the expected lag has not elapsed. Each has a different next step.
If Bootstrap CUSUM (StepChange Analyzer) finds no change point, the default action is: do not tamper. A flat line means the system is in common cause variation — reacting to it as if it were a special cause will increase instability, not reduce it.
No change detected has five possible meanings: the system is stable and acceptable; a change point is approaching but not yet confirmed; the intervention addressed the wrong constraint; the intervention was at the wrong Joiner level; or the expected lag has not yet elapsed. Each has a different next step. Only one of them is 'do nothing'.
☰ Contents
What “no change detected” means
A Bootstrap CUSUM result of no change point means the algorithm found no statistically significant, sustained shift from one stable level to another. The data has been fluctuating — but within the range that a stable system produces. In Joiner and Deming’s terms, you are looking at common cause variation: the normal, expected output of a process behaving consistently with its own design.
This is not a failure of the method. It is an honest answer. The system has not structurally changed — and the method has correctly not detected a change that did not happen.
A flat Bootstrap CUSUM line tells you the system is stable. Combined with knowledge of what interventions have been applied and when, it tells you whether those interventions reached the structural constraint or not. That is precise, actionable information — even if it is not the answer you wanted.
First checks — before reacting
🔎 Three questions to answer before acting
Bootstrap CUSUM needs sufficient data on both sides of any potential change point to detect it reliably. If your data series is short — fewer than 12–18 monthly observations, or fewer than 24 weekly observations — a genuine structural shift may not yet be detectable. The algorithm is conservative by design: it requires the post-shift period to be long enough to confirm the shift is sustained, not a fluctuation.
If the data window is short: extend it. Upload more historical data if available, or wait and re-run as more observations accumulate. Do not conclude there is no change until you have sufficient data to detect one.
Bootstrap CUSUM detects a change point only when the cumulative sum has crossed the statistical threshold with sufficient confidence. If the cumulative sum is trending strongly in one direction but has not yet crossed the threshold, a change point may be imminent — detectable in the next one to three periods as more data arrives.
If this is the case: this is Path B — the “decide to wait” situation. See No change detected: decide to wait (don’t tamper) for the decision rules. The key principle is the same: do not intervene while waiting for a signal that may be forming.
Before concluding the system is in common cause variation, check for special causes that the CUSUM may have absorbed without flagging as a structural shift. A spike followed immediately by an equal and opposite movement, for example, may cancel out in the cumulative sum without triggering a change point — but it is still a special cause event worth investigating.
Plot the raw data alongside the CUSUM output. Any data point that looks dramatically out of range, or any sustained run in one direction that then reversed, warrants investigation as a potential special cause before treating the system as purely stable. See Special Cause Variation for the full checklist.
Which situation are you in?
Once the three checks above are satisfied — sufficient data, not approaching a threshold, no hidden special causes — the “no change detected” result means one of three things. Identify which before deciding what to do next.
| Situation | The signal | Correct response |
|---|---|---|
| A. Stable at an acceptable level — no intervention applied | The system is performing within the range you need. No change point is expected because nothing has changed — and nothing needs to. | Hold steady and monitor. Continue running Bootstrap CUSUM as data arrives. The next genuine change point — improvement or deterioration — will be detectable against this clean, stable baseline. Do not introduce changes in response to month-to-month fluctuation. See below. |
| B. Stable at an acceptable level — intervention recently applied | A programme or change was introduced but no change point has appeared yet. The intervention may be too recent for the data to show a shift, or the system may need more time to respond. | Wait — do not tamper. Extending the observation window is the correct response. Adding further interventions before a change point is confirmed muddies the baseline and makes future attribution impossible. See decide to wait for the decision rules on when to wait and when to act. |
| C. Stable at an unacceptable level — intervention applied, no change point | The system is consistently producing results below the required standard. An intervention was applied. Bootstrap CUSUM shows no change. The intervention did not reach the constraint. | Move up a level. The intervention was Level 1 or Level 2 applied to a Level 3 constraint — or it addressed the wrong constraint entirely. The investigation below applies. See why no change point appeared. |
| D. Stable at an unacceptable level — no intervention applied | The system is performing poorly and has been doing so consistently. This is common cause variation at the wrong level — the system is doing exactly what it is designed to do, and the design is wrong. | Redesign the system. Joiner’s common cause highway: you cannot improve the output without changing the system. See Common Cause Variation — Improving a Stable System for the framework. |
The temptation to “do something” with a system that is performing acceptably is itself a risk. Deming called reactions to common cause variation tampering — and demonstrated with the funnel experiment that tampering consistently makes a stable system worse, not better. Each intervention adds variation. The system becomes less predictable. Staff learn that numbers trigger reactions regardless of whether those numbers contain a real signal.
The correct action is to run Bootstrap CUSUM continuously as a monitoring tool. The next real change point — whether an improvement from a future initiative or a deterioration from an emerging constraint — will be detectable against the clean baseline. The flat line is not a problem. It is the condition that makes future change points interpretable.
If you applied an intervention — why no change point?
An intervention that does not produce a Bootstrap CUSUM change point is not merely disappointing — it is precise, diagnostic information. Work through the following in order.
⚒️ Diagnosing a failed intervention
Goldratt’s Theory of Constraints states that improving anything other than the binding constraint produces minimal improvement in system throughput. The most common reason an intervention produces no change point is that it addressed a non-binding constraint — a real problem, but not the one that is limiting system performance.
Ask: what is the one thing, if resolved, that would most improve this metric? Is that the thing the intervention addressed? If not, the no-change result is the expected outcome. The intervention may have been worthwhile for other reasons — but it was not going to move this metric.
Using Joiner’s levels of fix: Level 1 fixes the output (temporary symptom relief). Level 2 fixes the process within existing organisational boundaries. Level 3 fixes the system — changes structural conditions that may cross organisational or political boundaries.
If the constraint sits at Level 3 but the intervention was at Level 1 or Level 2, no change point will appear in the outcome data. The process may have improved. The people implementing it may have worked harder. The metric will not have shifted — because the structural constraint was not reached. The NHS A&E record is the clearest example: 15 years of Level 1 and Level 2 interventions applied to a Level 3 discharge constraint, with no improvement change point in 184 monthly observations. See Why Nothing Has Worked.
Some structural interventions take time to produce a detectable change point. Social care capacity changes take 6–18 months to appear in discharge data. Workforce pipeline changes take 2–5 years to appear in staffing ratios. Prevention programmes take 5–20 years to appear in emergency admissions. If the intervention genuinely addresses the right constraint at the right level, the lag before a Bootstrap CUSUM change point appears may be longer than the review window.
Define the expected lag before implementing the intervention. If you are still within that lag window: wait, and re-run Bootstrap CUSUM as each period’s data arrives. If you are well beyond the expected lag: the intervention has not reached the constraint.
A well-designed intervention at the right level applied to the right constraint can still fail to produce a change point if it was not implemented as intended. Check the process measures: did the intervention actually happen, at scale, consistently, for long enough? A partial implementation produces a partial signal — which Bootstrap CUSUM may correctly not flag as a structural change point, because the system did not structurally change.
Run Bootstrap CUSUM on the process measures (implementation fidelity, coverage, compliance). If the process measures show no change point either, the intervention was not implemented. If they do show a change point but the outcome does not follow, the intervention was implemented but did not address the right constraint.
An improvement intervention applied at the same time as an adverse system change — increased demand, a workforce reduction, a new policy adding administrative burden — can produce two simultaneous change point signals that partially or fully cancel each other out. The CUSUM may show no net structural change even though both effects are real.
Check whether any significant adverse changes were introduced around the same time as the intervention. If so, stratify where possible: run Bootstrap CUSUM on subgroups or sites that were exposed to the intervention but not to the adverse change, and compare.
📝 The pre-committed prediction — use it next time
The most powerful use of Bootstrap CUSUM for interventions is prospective: before implementing, state explicitly what change point you expect — direction, the metric it should appear in first (the leading indicator), approximate timing, and the confidence threshold you will use as the test. This pre-commitment prevents the common error of interpreting any subsequent data movement as evidence the intervention worked.
If you did not make a pre-committed prediction before this intervention, make one for the next. A no-change result from a pre-committed prediction is unambiguous: the intervention did not produce what it was expected to produce at the level it was expected to produce it. That is honest and precise. Without the pre-commitment, a no-change result can always be explained away.
Run it again — recommended re-tests
| Re-test | Why | What to look for |
|---|---|---|
| Extend the data window | A genuine structural shift may not yet be detectable if the post-intervention period is short. Add more historical data at the start, or wait for more observations at the end. | Re-run monthly as data arrives. If a change point is forming, the cumulative sum will be trending consistently in one direction. If it is not trending, the intervention has not moved the system. |
| Lower the confidence threshold | A weak signal may not cross the 95% threshold but may be visible at 80% or 90%. This does not confirm a structural change — but it may indicate a marginal shift worth monitoring. | A change point at 80% but not 95% is a weak signal. It may strengthen as more data arrives, or it may disappear. Do not act on it as if it were confirmed structural change. |
| Run on the leading indicator | If the outcome metric shows no change, check whether the leading indicator has moved. A change point in the lead indicator without a corresponding outcome change point indicates the causal chain is intact but the lag has not yet elapsed. | Lead indicator change point present, outcome not yet changed: wait for the lag period. Neither changed: the intervention did not reach the constraint. |
| Stratify by subgroup or site | An aggregate no-change result may mask genuine change in a subgroup. Run Bootstrap CUSUM separately on sites, patient groups, or time periods where the intervention was applied most fully. | Change point in high-fidelity implementation sites only: the intervention works but was not implemented consistently enough to move the aggregate. Change point nowhere: the intervention does not work at this level. |
| Run on process measures | Did the intervention actually happen? Run Bootstrap CUSUM on implementation data — training completion, protocol compliance, referral rates. A change point in process without a change point in outcome confirms the intervention was delivered but did not reach the constraint. | No process change point: the intervention was not implemented. Process change point, no outcome change point: implemented but wrong level or wrong constraint. |
(1) The system is stable and acceptable — no action needed, monitor. (2) A change point is forming but not yet confirmed — wait, don’t tamper. (3) The intervention addressed the wrong constraint — return to constraint identification. (4) The intervention was at the wrong level — move up a Joiner level. (5) The lag has not yet elapsed — wait for the expected timeframe before concluding failure. Only one of these requires doing nothing. The others each have a precise next step.
Re-run the analysis
Extend your data window, add the leading indicator series, or test at a different confidence threshold. Upload to the StepChange Analyzer and run Bootstrap CUSUM again.
▶ Open the StepChange Analyzer