Investing Quote by Nick Bostrom
““Another lesson is that smart professionals might give an instruction to a program based on a sensible-seeming and normally sound assumption (e.g. that trading volume is a good measure of market liquidity), and that this can produce catastrophic results when the program continues to act on the instruction with iron-clad logical consistency even in the unanticipated situation where the assumption turns out to be invalid. The algorithm just does what it does; and unless it is a very special kind of algorithm, it does not care that we clasp our heads and gasp in dumbstruck horror at the absurd inappropriateness of its actions. This is a theme that we will encounter again.””
About This Quote
Source Book: Superintelligence by Nick Bostrom, 2014
Algorithms follow instructions literally, so faulty assumptions can cause disastrous outcomes when conditions change unexpectedly.
In simple terms: Programs act on assumptions without understanding context.
Key Takeaway
Validate assumptions before automating.
Themes
technology
risk
automation
assumptions
Mood
cautious
analytical
Type
warning
philosophical
When to use this quote
- financial trading
- AI deployment
- software development
- risk assessment
Key Concepts
logic
robustness
system design
Questions to Reflect On
- How can we detect hidden assumptions?
- What safeguards prevent rigid logic from causing harm?
A Different Perspective
Assumptions may be hidden and hard to test.
![“There is more scholarly work on the life-habits of the dung fly than on existential risks [to humanity].” quote by Nick Bostrom](https://sms4smile.com/quotes/img-quotes/8d3/nick-bostrom-10771_life_there-is-more-scholarly-work-on-the-life_560392.jpg)
