Here’s a new, non-Steampunk game I’m working on. The idea is that the Technological Singularity is coming: the moment when a smarter than human AI gets created, then recursively self improves itself to the point where it easily takes over the universe. This will either be the best or worst thing ever, depending on whether the AI is set up to be “friendly” or not-that is, whether the AI is set up such that as it recursively self improves, it maintains a utility function that values the same things humans value, instead of, say, being made by a shortsighted paperclip manufacturer who wants it to “make as many paperclips as you can” and so it turns the entire universe into paperclips. So you’re trying to finish researching Friendliness before either of three tracks to super AI finish-Hardcoding it, a Whole Brain Emulation, or a simple Seed AI that can take off.

I have two ideas for the gameplay so far. This is going to be a cooperative game. Most cooperative games-Forbidden Island, Pandemic-seem based around collecting matching sets of cards and then turning them in for a bonus. I don’t want to just copy these, but I want to think about it for a while, to see where it leads me. So far what I’m thinking is that the board would consist of a few cities, each with a few cards (one for each of the three AI tracks) at them, and if you wanted to do “preventative research” in one of them, you would do a three card monte style game to grab one, trying to get the particular one you want. Then you could trade in a few of those to prevent research from being accomplished on that kind of AI that turn.

The second mechanic, which is more original and I like more, would be that you would choose a character, and each would come with a custom die. Every turn everyone would roll it, and then try to guess what the total is. You would give a confidence interval, and then have a reward if it lies within it. The idea here is that there could be some limited information sharing, so people could try to do proper Bayesian updates on the info they have, and then be rewarded for being well calibrated on their estimates. So for instance, it could work like this-you pick a number that you’re 80% sure the total is equal to or above. If it is above and within 5, then you get a Friendliness point, and if it’s above and not within 5, nothing happens, and if it’s below, you roll a Disaster Die. Tweak the numbers so that actually being right 80% of the time is the best option, make a few ways to share information (you can look at someone else’s die if they’re in the same city?) and this becomes good practice for updating and calibration, which would be pretty cool.


Obviously this is in the early stages, but I thought I’d share my thoughts so that anyone interested in how I go about designing games can check it out.

In case you aren’t aware, this game is heavily inspired by my work with the Singularity Institute and Center for Applied Rationality.

Leave a comment

Leave a comment

Your email address will not be published. Required fields are marked *