The Things That Could Go Very Wrong
An honest inventory. Not doom — attention.
Being optimistic about the future requires being clear-eyed about the ways it can fail. This is not a list of scares. It is a list of things a serious civilisation would already be preparing for.
Concentration of power
A world in which a handful of firms or states control the frontier models is a world in which most of us are guests. Preventing that is a political project, not a technical one.
Cognitive dependence
Skills we do not practise, we lose. A generation raised entirely on AI-mediated thinking may find that some human capacities atrophy in ways that are difficult to reverse.
Loss of shared reality
Personalised media, synthetic evidence, and infinite content mean any two people can inhabit incompatible worlds. Democracy assumes a common substrate. That substrate is being eroded.
Systemic accidents
Complex systems fail in ways their designers did not predict. Autonomous agents interacting at machine speed will produce crashes we have not yet imagined. Preparing means slowing down where it counts.
Questions worth arguing about
- ◆What single failure keeps you up at night?
- ◆What would a serious pause look like, and who could enforce it?
- ◆If the worst case happens, what would you want to have already built?