Computers Quote by Nick Bostrom
““If some day we build machine brains that surpass human brains in general intelligence, then this new superintelligence could become very powerful. And, as the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species would depend on the actions of the machine superintelligence. We do have one advantage: we get to build the stuff. In principle, we could build a kind of superintelligence that would protect human values. We would certainly have strong reason to do so. In practice, the control problem—the problem of how to control what the superintelligence would do—looks quite difficult. It also looks like we will only get one chance. Once unfriendly superintelligence exists, it would prevent us from replacing it or changing its preferences. Our fate would be sealed.””
About This Quote
Source Book: Superintelligence: Paths, Dangers, Strategies by Nick Bostrom, 2014
If machines surpass human intellect, their goals could dictate humanity’s future, making alignment crucial and difficult.
In simple terms: Powerful AI could control humanity’s destiny, so we must align it with human values.
Key Takeaway
Align AI with human values before it becomes uncontrollable.
Themes
AI safety
existential risk
control problem
Mood
cautious
urgent
reflective
Type
philosophical
warning
When to use this quote
- governance design
- AI research
- public policy
- risk assessment
Key Concepts
Superintelligence
value alignment
single point of failure
policy making
global governance
Questions to Reflect On
- What safeguards can we design now?
- How do we ensure global cooperation on AI safety?
A Different Perspective
Alignment may be impossible after deployment; single chance to get it right.
![“There is more scholarly work on the life-habits of the dung fly than on existential risks [to humanity].” quote by Nick Bostrom](https://sms4smile.com/quotes/img-quotes/8d3/nick-bostrom-10771_life_there-is-more-scholarly-work-on-the-life_560392.jpg)
