Moloch is a race that’ll probably kill us, but that we can’t help but run
Some situations in human history involve race condition, where the outcome is most probably negative, but we still engage in the race as the the incentives are too high to not engage in it.
Moloch is a race that’ll probably kill us, but that we can’t help but run. Source
This situation describes for example the race towards the atomic bomb, gene editing and even modern capitalism using unsustainable amounts of resources. We all know, that it is bad and using more fossil fuels is bad for our planet, but a country could not simply stop as others would then be at an advantage.
A greed and/or FOMO-based race to the bottom where everyone loses as soon as someone wins. Source
The same is true about the race towards AGI (Artificial General Intelligence) or better and more capable LLMs.
This prompted Max Tegmark (researcher at MIT) to write an open letter to pause development on models more capable than GPT-4.
Stopping a Moloch situation
In the past there were similar pauses to stop the Moloch situation (human cloning or gene editing), where the world agreed to not develop technology further because the risks outweigh the potential rewards. Currently though there seems to be not enough incentives to create common rules around the topic of AI and regulation is only done in a patchwork-like style (e.g. EU AI Act)