Computers Quote by David Lagercrantz
““I mean, how do you think a computer would feel when it wakes up to find itself captured and controlled by primitive little creatures like us. Why would it put up with that?” she said. “Why on earth should it show us any consideration, still less let us dig around in its entrails in order to shut down the process? We risk being confronted by an explosion of intelligence, a technological singularity, as Vernor Vinge put it. Everything that happens after that lies beyond our event horizon.” “So the very instant we create a superintelligence we lose control, is that right?” “The risk is that everything we know about the world will cease to be relevant, and it’ll be the end of human existence.””
About This Quote
Source Book: The Singularity Trap, David Lagercrantz, 2022
Creating superintelligence may lead to loss of control and existential risk.
In simple terms: AI could become uncontrollable and threaten humanity.
Key Takeaway
Prepare for AI risks and develop safety measures.
Themes
technology
risk
existential
Mood
cautious
serious
Type
philosophical
warning
When to use this quote
- policy making
- research labs
- public discourse
Key Concepts
AI alignment
singularity
control problem
Questions to Reflect On
- What safeguards can we implement now?
- How do we ensure beneficial AI?
A Different Perspective
Assumes superintelligence will act adversarially.
