General manager Quote by Jez Humble
““As discussed in Chapter 2, data gathered from A/B tests by Ronny Kohavi, who directed Amazon’s Data Mining and Personalization group before joining Microsoft as General Manager of its Experimentation Platform, reveal that 60%–90% of ideas do not improve the metric they were intended to improve. Thus if we’re not running experiments to test the value of new ideas before completely developing them, the chances are that about 2/3 of the work we are doing is of either zero or negative value to our customers — and certainly of negative value to our organization, since this work costs us in three ways. In addition to the cost of developing the features, there is an opportunity cost associated with more valuable work we could have done instead, and the cost of the new complexity they add to our systems (which manifests itself as the cost of maintaining the code, a drag on the rate at which we can develop new functionality, and often, reduced operational stability and performance).””
About This Quote
Source Book: Accelerate: The Science of Lean Software and DevOps, 2018
Most ideas fail to improve metrics; without experiments, most work adds no value and incurs hidden costs.
In simple terms: Ideas often don’t work; testing saves resources.
Key Takeaway
Test ideas before full investment.
Themes
experimentation
risk management
efficiency
Mood
analytical
cautious
Type
advice
warning
When to use this quote
- product development
- feature rollout
- resource allocation
Key Concepts
A/B testing
lean methodology
cost analysis
Questions to Reflect On
- How can we balance speed and testing?
- What metrics truly matter?
A Different Perspective
Testing can delay delivery and may not capture long‑term effects.
