The Innovator’s Dilemma
The old product line is profitable enough to keep demanding attention, but declining enough to eventually pull the business down with it. That is the dilemma. It has defeated companies far larger and better resourced. Every rational short-term decision — maintain the profitable base, customise to win the next deal, staff the support queue — accumulates into an irrational long-term outcome: the constraint on the future is the success of the past.
☰ Contents
Christensen’s framework
Clayton Christensen, in The Innovator’s Dilemma (Harvard Business School Press, 1997), documented a precise pattern: well-managed, successful companies, following rational management principles, consistently failed when facing disruptive new technologies or markets. Not because they made bad decisions. Because they made good decisions — the decisions that made sense given their existing customers, existing cost structures, and existing incentive systems.
The disruptive innovation typically starts in a market the incumbent does not serve — smaller customers, lower price points, simpler requirements. It is inferior on the metrics the incumbent’s existing customers care about. So the incumbent ignores it. Rationally. The disruptor improves. The disruptor’s market grows. The disruptor moves upmarket. By the time the incumbent responds, the disruptor has captured the mainstream market and the incumbent is retreating to a shrinking premium segment.
The incumbent cannot invest in the disruptive technology because its existing customers do not want it, its financial model does not support investment in lower-margin new markets, and its best people are rightly focused on serving the profitable existing base. Every correct short-term decision makes the long-term problem worse. The dilemma is that good management — listening to customers, focusing on high-margin work, investing in sustaining improvements — is precisely what causes failure in the face of disruption.
Applied to any specialist service business: the profitable installed base demands attention. The high-complexity, high-margin bespoke work is where the expertise lives. The simpler, standardised new market looks less attractive. Every developer-hour spent on the base is a developer-hour not spent on the new product. The constraint compounds. The erosion continues. The new product never gets finished.
The dilemma stated precisely
The dilemma is not a strategy failure. It is a system structure problem — which is why it recurs across industries, decades, and management generations. The same pattern appears in disk drive manufacturers (Christensen’s original research), steel mills, telecommunications, publishing, and specialist B2B software companies. The structure is always the same:
- The existing product serves a demanding customer base that generates most of the revenue
- That customer base pulls development effort toward complexity, customisation, and sustaining improvements
- The new market requires simplicity, standardisation, and a lower price point
- Simplicity and standardisation feel like going backwards to the people who built the complex product
- The financial model rewards the complex product (higher margin per unit) over the simple one
- The simple product never gets the sustained investment it needs before being deprioritised for the next urgent base maintenance request
In Causal Loop Diagram terms, the dilemma is a reinforcing loop: profitable base demands attention → best people work on base → new product underdeveloped → new product less competitive → more customisation required to win deals → more base complexity → more base demand for attention. Each cycle makes the constraint tighter. The loop does not break by itself. It requires a structural intervention — a deliberate decision to shield the constraint from the base and point it at the new product.
The five Expert Traps
In any organisation caught in the Innovator’s Dilemma, the same five traps recur. Each is rational in isolation. Together they make repeated attempts at new product development fail in the same way each time.
The Innovation Trap
New product development is captured by the most demanding existing customers — the lead users who want more features, more complexity, more customisation. The product grows toward the expert end of the market. It becomes expensive to build, expensive to implement, and too complex for the simpler market segment it was meant to serve. Products slow down, features pile up unfinished, and the eventual product is a premium offering that competes on the old playing field rather than creating a new one.
The Customisation Trap
Saying yes to bespoke requests is a survival reflex. Each customisation generates revenue and satisfies a customer. Each also adds a variant to the installed base that must be maintained, upgraded, and supported separately. Over time the base contains dozens of variants, each requiring specialist knowledge. Developer time is consumed by maintaining variants rather than building the new product. The customisation trap is the operational expression of the Innovator’s Dilemma at the project level.
The Quality Spiral
As the installed base grows in complexity, the volume of advice-seeking support cases grows faster. Customers with complex, bespoke configurations need expert help for edge cases that no documentation covers. The support burden per customer rises. Failure demand compounds. Developer time that should go to the new product is consumed by the quality maintenance demands of an increasingly complex base. Each new bespoke sale makes the spiral worse.
The Attention Trap
The most senior, most expert person in the organisation is also the sole sales channel, the escalation point for complex support, the product vision holder, and the relationship manager for key accounts. Divided attention produces partial performance across all roles and excellent performance in none. The new product initiative that requires sustained, focused strategic thinking competes with operational urgency every day — and urgency always wins. This is the constraint that Newport’s Deep Work identifies: creative work requires uninterrupted concentration, and the attention trap systematically destroys it.
The Dilemma Itself
The profitable product prevents investment in the disruptive one. Not because the organisation does not understand the need. Because the financial model, the incentive structure, and the resource allocation process all rationally direct investment toward the highest-margin, most-certain work — which is always the existing product. The new product is always one deprioritisation away from another year of delay. The constraint is the structure of the organisation itself.
In the 7-step improvement method, the five traps are the symptoms listed in Step 1. The Behaviour Over Time chart for each trap traces the decay or stagnation pattern. Steps 3 and 4 (root cause and dominant constraint) converge on the same answer for all five traps: the absence of a structure that separates the people who maintain the existing product from the people who build the next one.
Von Hippel and lead users
Eric von Hippel’s lead user research identifies a paradox that is directly relevant to the Innovator’s Dilemma. Lead users — customers at the extreme end of the adoption curve who are ahead of market trends — are the source of the most valuable product insights. But they are also the source of the Innovation Trap.
The distinction that matters is the direction of the lead user’s innovation. Von Hippel’s original insight is that lead users who are pulling the product toward increasing complexity, customisation, and premium features are pointing at the incumbent’s existing market — they want a better version of what already exists. That is a sustaining innovation path, not a disruptive one.
The lead users who point toward the disruptive opportunity are different: they are the customers getting exceptional results with simpler configurations. They have found ways to achieve the core outcome without the complexity. They are the Bright Spots in the customer base — positive outliers whose results are statistically distinguishable from the mean, not because they have more resources or more customisation, but because they have found a simpler, more reliable approach.
The wrong question: what do our most demanding customers want us to build next? This produces the Innovation Trap — more complexity, more features, more customisation.
The right question: which customers are achieving the best outcomes with the simplest configurations, and what are they doing that others are not? This produces the Blueprint hypothesis — the standardised offer that delivers value reliably, affordably, and without bespoke development.
Von Hippel corrected: lead user insight → Blueprint hypothesis → mainstream validation. Not lead user insight → bespoke feature build → premium product for existing market.
The Innovator’s Solution
Christensen’s follow-up book, The Innovator’s Solution (co-authored with Michael Raynor, 2003), identifies what successful disruptors do differently. The key insight is that the disruptive product is not a stripped-down, inferior version of the incumbent’s premium offering — it is a product designed for a different job-to-be-done, at a price point that makes it accessible to customers who currently have no solution at all.
Applied to a specialist B2B service business, the Innovator’s Solution has five characteristics:
- Simpler, not stripped-down. The new product does fewer things, but does the most important thing reliably and without requiring expert implementation. It is not a worse version of the complex product. It is a better solution to a simpler problem.
- Fixed scope, not open-ended. Every sale is a configuration of a known product, not a scoping exercise for a new project. Delivery time is weeks, not months. Risk is bounded for the buyer.
- Priced for a different market. The new product targets customers who currently have no solution, not customers who could afford the premium product if they tried harder. The price point creates a new market rather than competing in the existing one.
- Validated by lead users, not designed for them. The customers whose simple configurations are producing exceptional results (the Bright Spots) are the validation of the Blueprint hypothesis, not the design brief for the next bespoke feature.
- Built by ring-fenced people. The new product cannot be built by the same people who maintain the existing product. The shielding structure — Goalies protecting Builders, Toolmakers ring-fenced from operational work — is the structural precondition for the Innovator’s Solution to be built at all.
Where this fits in the 7-step method
📈 The Innovator’s Dilemma in the improvement method
Step 1 (List symptoms) — The five traps are the symptom list. No net growth, high support burden, slow new product development, single sales channel, implementation complexity. Each symptom traces a recognisable pattern on the BOT chart: stagnation for revenue, decay for installed base value, stagnation for new contract value despite effort.
Step 2 (Measure) — Bootstrap CUSUM on new contract value confirms the stagnation pattern — flat for several years at an undesirable level. The dilemma has been active for longer than it feels. The flat line is its statistical signature.
Steps 3–4 (Root cause + constraint) — The Causal Loop Diagram of the five traps reveals the reinforcing loop. The dominant constraint is the absence of a structure that separates operational work from new product development. This is Level 3 — a system redesign, not a process fix.
Step 5 (Complete solution) — Applying the necessary but not sufficient test: the Investigator role is necessary. The Toolmaker ring-fence is necessary. The Blueprint product is necessary. All together they are sufficient — they break the reinforcing loop by introducing a structural separation between the operations that sustain the dilemma and the development that resolves it.
Step 6 (PICK) — The Investigator appointment and Toolmaker ring-fence are Implement — high impact, low effort relative to their unlock value. The Blueprint is Challenge — broken into stages, each a complete PDSA cycle. The Kill item is unscoped customisation without a portfolio policy.
Step 7 (PDSA) — Pre-specify the Bootstrap CUSUM prediction: an upward change point in new contract value within 18–24 months of the structural separation. The Study step is not “does it feel like we are making progress?” It is: has a change point appeared?
Bootstrap CUSUM signature of the dilemma
The Innovator’s Dilemma has a specific Bootstrap CUSUM signature across multiple series simultaneously:
- New contract value — stagnation pattern. Flat for years despite repeated initiatives. No upward change point. The reinforcing loop has neutralised each initiative before it could produce structural change.
- Support case volume — gradual upward drift or stagnation at high level. Failure demand compounding as base complexity grows. The upward change point coincides with major customisation waves, not with improvement initiatives.
- New product release cadence — stagnation or intermittent bursts. New features appear and then stop as operational pressure pulls developers back. No sustained upward trajectory.
- Base revenue — slow decay. EMR displacement, regulatory change, demographic shifts. A gradual downward change point that is easy to deny for years before Bootstrap CUSUM makes it undeniable.
The combined picture — four series, each showing its characteristic pattern — is the statistical portrait of a system caught in the Innovator’s Dilemma. The diagnosis does not require a consultant. It requires Bootstrap CUSUM and the willingness to look at all four series together.
The Innovator’s Dilemma sits at Steps 1–4 of the 7-step improvement method — the diagnostic phase. See Standardisation vs Customisation for the solution, Focus & Prioritisation for the structural mechanism, and the Strategic BOT chart for the full picture of how the six solution waves build across a five-year horizon.