--- title: "Meaning & Key Takeaway of: Beware of the problem of testing too many…" description: "\"Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained unde..." image: "https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png" --- # Beware of the problem of testing too many hypotheses; the… ![“Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained under duress may not be admissible in the court of scientific opinion.” quote by Stephen Stigler](https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png) [Download](https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png) [Open image](https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png) > “Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained under duress may not be admissible in the court of scientific opinion.” Stephen Stigler ## About This Quote This interpretation was drafted with AI assistance. It is one reading of the quote, not the author's own explanation. Over‑testing data to fit hypotheses yields false confidence; rigorous, pre‑registered methods protect scientific credibility. **In simple terms:** Testing too many ideas can corrupt results. Key Takeaway Limit hypotheses before data collection. Themes scientific integrity methodology bias rigor reproducibility Mood critical analytical cautious Type Confession:0 Court:0 Data:1 Duress:0 Hypotheses:1 Hypothesis:1 May:0 Opinion:0 Problem:1 Science:1 Testing:1 Torture:0 Testing Hypotheses:1 Hypotheses Torture:0 Torture Data:0 Confessions Obtained:0 When to use this quote - clinical trials - social science studies - machine learning model validation Key Concepts statistics philosophy of science research ethics Questions to Reflect On - What safeguards can you implement against data dredging? - How do you balance exploration with confirmation? A Different Perspective Pre‑registering limits flexibility and may miss novel insights. ★ ★ ★ ★ ★ No ratings yet [Confession](https://sms4smile.com/quotes/about/confession/) [Court](https://sms4smile.com/quotes/about/court/) [Data](https://sms4smile.com/quotes/about/data/) [Duress](https://sms4smile.com/quotes/about/duress/) [Opinion](https://sms4smile.com/quotes/about/opinion/) [Problem](https://sms4smile.com/quotes/about/problem/) [Science](https://sms4smile.com/quotes/about/science/) [Torture](https://sms4smile.com/quotes/about/torture/) ## Related quotes > [“Everything you do in customer development is centered around testing hypotheses.”](https://sms4smile.com/quotes/cindy-alvarez/everything-you-do-in-customer-development-is-centered-around-testing/) [Cindy Alvarez](https://sms4smile.com/quotes/author/cindy-alvarez/) > [Evidence-based reasoning underpins all scientific thinking, and it involves testing hypotheses or theories against data. Validating a theory requires replicable measurements from independent groups…](https://sms4smile.com/quotes/priyamvada-natarajan/evidence-based-reasoning-underpins-all-scientific-thinking-and-it/) [Priyamvada Natarajan](https://sms4smile.com/quotes/author/priyamvada-natarajan/) > [If you torture the data long enough, it will confess.](https://sms4smile.com/quotes/anonymous/if-you-torture-the-data-long-enough-it-will-confess/) [Anonymous](https://sms4smile.com/quotes/author/anonymous/) > [Experimental investigation, to borrow a phrase employed by Kepler respecting the testing of hypotheses, is "a very great thief of time." Sometimes it costs…](https://sms4smile.com/quotes/john-william-draper/experimental-investigation-to-borrow-a-phrase-employed-by-kepler/) [John William Draper](https://sms4smile.com/quotes/author/john-william-draper/) > [“Contrary to the rules of philosophers of science, who advise testing hypotheses by trying to refute them, people (and scientists, quite often) seek data…](https://sms4smile.com/quotes/daniel-kahneman/contrary-to-the-rules-of-philosophers-of-science-who-advise-testing/) [Daniel Kahneman](https://sms4smile.com/quotes/author/daniel-kahneman/) > [“That you could loathe yourself for acting under duress and still crave the very things you'd done.”](https://sms4smile.com/quotes/kit-rocha/that-you-could-loathe-yourself-for-acting-under-duress-and-still/) [Kit Rocha](https://sms4smile.com/quotes/author/kit-rocha/) > [“Defence lawyers use the term "duress" to describe the use of force, coercion or psychological pressure exerted on a client in the commission of…](https://sms4smile.com/quotes/emily-thorne/defence-lawyers-use-the-term-duress-to-describe-the-use-of-force/) [Emily Thorne](https://sms4smile.com/quotes/author/emily-thorne/) > [People never change because they are under threat or under duress. Never. They change because they see something that makes their life seem valuable…](https://sms4smile.com/quotes/robert-downey-jr/people-never-change-because-they-are-under-threat-or-under-duress/) [Robert Downey, Jr](https://sms4smile.com/quotes/author/robert-downey-jr/) > [Although the mass of the people accepted the white man's God, either under physical duress or because he seemed more powerful than their own…](https://sms4smile.com/quotes/penny-lernoux/although-the-mass-of-the-people-accepted-the-white-man-s-god-either/) [Penny Lernoux](https://sms4smile.com/quotes/author/penny-lernoux/) > [To experience commitment as the loss of options, a type of death, the death of childhood's limitless possibility, of the flattery of choice without…](https://sms4smile.com/quotes/david-foster-wallace/to-experience-commitment-as-the-loss-of-options-a-type-of-death-the/) [David Foster Wallace](https://sms4smile.com/quotes/author/david-foster-wallace/) > [ “\[John Jarndyce\] rubbed his head so constantly that not a single hair upon it ever rested in its right place” ](https://sms4smile.com/quotes/charles-dickens/john-jarndyce-rubbed-his-head-so-constantly-that-not-a-single-hair/) [Charles Dickens](https://sms4smile.com/quotes/author/charles-dickens/) > [At a time when going to college has never been more important, it's never been more expensive, and our nation's families haven't been in…](https://sms4smile.com/quotes/anonymous/at-a-time-when-going-to-college-has-never-been-more-important-it-s/) [Anonymous](https://sms4smile.com/quotes/author/anonymous/) > [Songs sung under duress are often very powerful.](https://sms4smile.com/quotes/henry-rollins/songs-sung-under-duress-are-often-very-powerful/) [Henry Rollins](https://sms4smile.com/quotes/author/henry-rollins/) > [ Once over there \[Balkans\], I felt extremely patriotic. Here are these people, from 18-year-olds to military veterans, enduring real duress for the cause of… ](https://sms4smile.com/quotes/sheryl-crow/once-over-there-balkans-i-felt-extremely-patriotic-here-are-these/) [Sheryl Crow](https://sms4smile.com/quotes/author/sheryl-crow/) > [With war and famine and flood and special effects films, when you do somebody under duress, you have to be really be inventive and…](https://sms4smile.com/quotes/peter-weir/with-war-and-famine-and-flood-and-special-effects-films-when-you-do/) [Peter Weir](https://sms4smile.com/quotes/author/peter-weir/) ## More by Stephen Stigler > [“Cournot noted that if one averaged the respecticvee sides of a collection of right triangles, the resulting figure would not be a right triangle...”](https://sms4smile.com/quotes/stephen-stigler/cournot-noted-that-if-one-averaged-the-respecticvee-sides-of-a/) Stephen Stigler Explore all 2 Stephen Stigler quotes ## More Confession quotes > [If You Love Someone, Be Brave Enough To Tell Them., Otherwise Be Brave Enough To Watch Them To Be Loved By Someone Else.](https://sms4smile.com/quotes/bilal-zahoor/if-you-love-someone-be-brave-enough-to-tell-them-otherwise-be-brave/) [Bilal Zahoor](https://sms4smile.com/quotes/author/bilal-zahoor/) > [A love to Christ which is so cowardly and selfish that it is unwilling to proclaim by a public confession its faith in Him…](https://sms4smile.com/quotes/abbott-eliot-kittredge/a-love-to-christ-which-is-so-cowardly-and-selfish-that-it-is/) [Abbott Eliot Kittredge](https://sms4smile.com/quotes/author/abbott-eliot-kittredge/) > [Boys say its great..boys say its fine... 9 months later boys say its not mine..!](https://sms4smile.com/quotes/simi-ngr/boys-say-its-great-boys-say-its-fine-9-months-later-boys-say-its-not/) [Simi Ngr](https://sms4smile.com/quotes/author/simi-ngr/) > [The characters in my novels are my own unrealized possibilities. That is why I am equally fond of them all and equally horrified by…](https://sms4smile.com/quotes/milan-kundera/the-characters-in-my-novels-are-my-own-unrealized-possibilities-that/) [Milan Kundera](https://sms4smile.com/quotes/author/milan-kundera/) > [Notoriety and public confession in the literary form is a frazzler of the heart you were born with, believe me.](https://sms4smile.com/quotes/jack-kerouac/notoriety-and-public-confession-in-the-literary-form-is-a-frazzler-of/) [Jack Kerouac](https://sms4smile.com/quotes/author/jack-kerouac/) > [In Confession, Jesus welcomes us with all our sinfulness, to give us a new heart, capable of loving as he loves.](https://sms4smile.com/quotes/pope-francis/in-confession-jesus-welcomes-us-with-all-our-sinfulness-to-give-us-a/) [Pope Francis](https://sms4smile.com/quotes/author/pope-francis/) > [A man does not have to feel less than human to realize his sin; oppositely, he has to realize that he gets no special…](https://sms4smile.com/quotes/criss-jami/a-man-does-not-have-to-feel-less-than-human-to-realize-his-sin/) [Criss Jami](https://sms4smile.com/quotes/author/criss-jami/) > [“I was going to make a confession. Then I remembered I don't make them very well. We still need to see the monkeys." "You…](https://sms4smile.com/quotes/nora-roberts/i-was-going-to-make-a-confession-then-i-remembered-i-don-t-make-them/) [Nora Roberts](https://sms4smile.com/quotes/author/nora-roberts/) > [Yelena, you've driven me crazy. You've caused me considerable trouble and I've contemplated ending your life twice since I've known you." Valek's warm breath…](https://sms4smile.com/quotes/maria-v-snyder/yelena-you-ve-driven-me-crazy-you-ve-caused-me-considerable-trouble/) [Maria V. Snyder](https://sms4smile.com/quotes/author/maria-v-snyder/) > [“in other words, his story was part boast, part confession.”](https://sms4smile.com/quotes/ben-mezrich/in-other-words-his-story-was-part-boast-part-confession/) [Ben Mezrich](https://sms4smile.com/quotes/author/ben-mezrich/) > [“I have a confession to make.121 Almost every time I tune in to religious radio or TV, I want to change my religion. I”](https://sms4smile.com/quotes/brian-d-mclaren/i-have-a-confession-to-make-121-almost-every-time-i-tune-in-to/) [Brian D. McLaren](https://sms4smile.com/quotes/author/brian-d-mclaren/) > [A catechism is simply a tool for teaching the fundamentals of the faith. Unlike a creed or confession, a catechism uses questions and answers.…](https://sms4smile.com/quotes/kevin-deyoung/a-catechism-is-simply-a-tool-for-teaching-the-fundamentals-of-the/) [Kevin DeYoung](https://sms4smile.com/quotes/author/kevin-deyoung/) > [“I’m beginning to get the feeling that confession is what we need in order to forgive ourselves.”](https://sms4smile.com/quotes/shane-kuhn/i-m-beginning-to-get-the-feeling-that-confession-is-what-we-need-in/) [Shane Kuhn](https://sms4smile.com/quotes/author/shane-kuhn/) > [“We need repentance. You see, repentance is not only going to a priest and confessing. We must free ourselves from the obsession of thoughts.…](https://sms4smile.com/quotes/elder-thaddeus-of-vitovnica/we-need-repentance-you-see-repentance-is-not-only-going-to-a-priest/) [Elder Thaddeus of Vitovnica](https://sms4smile.com/quotes/author/elder-thaddeus-of-vitovnica/) > [You said, I love you. I said it too. The only difference is, I didn't lie to you.](https://sms4smile.com/quotes/anonymous/you-said-i-love-you-i-said-it-too-the-only-difference-is-i-didn-t-lie/) [Anonymous](https://sms4smile.com/quotes/author/anonymous/) [Browse all 909 Confession quotes ](https://sms4smile.com/quotes/about/confession/) ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Quotation", "text": "Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they are to confess, but confessions obtained under duress may not be admissible in the court of scientific opinion.", "creator": { "@type": "Person", "name": "Stephen Stigler" }, "isPartOf": { "@id": "https://sms4smile.com/quotes/#website" }, "url": "https://sms4smile.com/quotes/stephen-stigler/beware-of-the-problem-of-testing-too-many-hypotheses-the-more-you/", "about": { "@type": "Thing", "name": "Confession" }, "@id": "https://sms4smile.com/quotes/stephen-stigler/beware-of-the-problem-of-testing-too-many-hypotheses-the-more-you/#quotation", "image": { "@type": "ImageObject", "@id": "https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png", "contentUrl": "https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png", "url": "https://sms4smile.com/quotes/i/911/stephen-stigler-107708_confession_beware-of-the-problem-of-testing-too-man_404003.png", "width": 1080, "height": 1080, "caption": "“Beware of the problem of testing too many hypotheses; the more you torture the data, the more likely they…” — Stephen Stigler", "creditText": "SMS4Smile Quotes", "creator": { "@id": "https://sms4smile.com/quotes/#organization" } } }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sms4smile.com/" }, { "@type": "ListItem", "position": 2, "name": "Quotes", "item": "https://sms4smile.com/quotes/" }, { "@type": "ListItem", "position": 3, "name": "Stephen Stigler", "item": "https://sms4smile.com/quotes/author/stephen-stigler/" }, { "@type": "ListItem", "position": 4, "name": "Beware of the problem of testing...", "item": "https://sms4smile.com/quotes/stephen-stigler/beware-of-the-problem-of-testing-too-many-hypotheses-the-more-you/" } ], "@id": "https://sms4smile.com/quotes/stephen-stigler/beware-of-the-problem-of-testing-too-many-hypotheses-the-more-you/#breadcrumb" }, { "@type": "WebSite", "@id": "https://sms4smile.com/quotes/#website", "name": "SMS4Smile Quotes", "url": "https://sms4smile.com/quotes/", "publisher": { "@id": "https://sms4smile.com/quotes/#organization" } }, { "@type": "Organization", "@id": "https://sms4smile.com/quotes/#organization", "name": "SMS4Smile Quotes", "url": "https://sms4smile.com/quotes/", "logo": { "@type": "ImageObject", "url": "https://sms4smile.com/quotes/wp-content/themes/sq/assets/icons/icon-512.png", "width": 512, "height": 512 } } ] } ```