2008 Quote by Christopher D. Manning
““In principle, more analytic power can be achieved by varying multiple things at once in an uncorrelated (random) way, and doing standard analysis, such as multiple linear regression. In practice, though, A/B testing is widely used, because A/B tests are easy to deploy, easy to understand, and easy to explain to management.””
About This Quote
Source Article: Blog post on experimental design, 2023
Using many random variations can theoretically give more insight, but simple A/B tests are preferred for ease of use and communication.
In simple terms: Random multi‑factor tests are powerful but complex; A/B tests are simple and popular.
Key Takeaway
Choose the method that balances insight with practicality.
Themes
experimental design
statistics
business
Mood
analytical
practical
Type
advice
explanatory
When to use this quote
- product development
- marketing campaigns
- software feature testing
- policy evaluation
Key Concepts
randomization
multiple regression
trade‑offs
Questions to Reflect On
- Can you justify the added complexity to stakeholders?
- When does simplicity outweigh precision?
A Different Perspective
Complex designs need more data and expertise, risking misinterpretation.
![“That so far the material has been dealt with in a rather subjective way provokes the question whether a means can be found of handling it objectively. [...] This chapter considers the applicability of the statistical tests employed by Wilson and the general problem whether the Linear B data are…” quote by Jennifer K. McArthur](https://sms4smile.com/quotes/img-quotes/a76/jennifer-k-mcarthur-57498_decipherment_that-so-far-the-material-has-been-dealt_2073765.jpg)
