Cambridge University’s Centre for the Study of Existential Risk (CSER) recently released a new Civilization 5 mod all about reducing the threat superintelligent AI can pose against mankind, The Verge reports.
As The Verge points out, CSER was founded in 2012 to explore "various global catastrophes capable of collapsing civilization or wiping out humanity altogether." These dangers are referred to as "existential threats." One of these threats, the CSER thinks, is a superintelligent artificial intelligence system smarter than humans that ultimately chooses they aren't necessary to its system's needs.
"We had the idea in the center that we wanted to do outreach for the idea of superintelligence – to get people with the right skillset interested, grow the field of people who care about AI safety, and test our own ideas,” CSER postdoc researcher Shahar Avin told the outlet.
CSER created two mods for Civilization 5 and its DLC Brave New Worlds where players focus on building a superintelligent AI. However, despite tasking players with developing the AI, they must keep check on an "AI risk" counter that slowly ticks up. If the counter fills up, somewhere in the game's world an AI is out of control and the game is over.
"You play through this long arc of history, but it all ends if you don’t manage your technology right," Avin said.
To fight losing, players can use resources for building AI safety labs. As The Verge points out, more research is the key to survival.
“If you choose to go down the AI path, you need to make sure you have more safe AI research than rogue AI research,” Avin told The Verge. “Investment in AI safety is in some sense altruistic, and we try to replicate that.”
Avin added he noticed something "surprising" in the mod: the negative impacts geopolitical issues can have on allocating resources to AI research in the mod. "Let’s say you’re stuck between two aggressive civilizations. It becomes very difficult to manage AI risk, because your resources are devoted to fighting wars," the researched said.
"There are these problems where you need altruism and strong international cooperation. Things that seem short-term now could end up having a very significant effect in the future," Avin added.
The Superintelligence mod is available now on Steam.