Happy new year! Last month (last... decade?) I was at Dagstuhl, a research center in Germany that hosts computer science seminars where people come together to discuss the future of fields like artificial intelligence. I spent the week with games researchers, designers, mathematicians and more to think about what the future of games and AI will look like. This is the writeup of the third workgroup of the week, which I hosted, titled Designing Impossible Games. As with Day 2, I’ve also written a short summary of what the other groups got up to at the end, with names of people you can ask for more information.

My pitch for the workgroup was to design games that were impossible for humans to play but easy for AI, or vice versa. Of course, it’s easy to imagine games that fit into either category - a game where you tell a joke is already quite hard for AI (although some comedian AI do exist!) for example, and a game that requires you to memorise 5,000 numbers is impossible for most humans. But there are many different ways a game can become hard for either group, and exploring the borders and edges of this is interesting. I also proposed it because I was itching to do some design, and I think having practical workgroups like this is really useful somewhere like Dagstuhl, as it’s refreshing and a bit different from what (most of us) normally do.

We broke into three smaller groups for an hour or so and tried to prototype some ideas as fast as possible. I won’t describe the games in too much detail here, but we are planning to write up the instructions to the games and post them online at some point, so people can play them.

Left: Balancing dice. Right: Rotating hexes. Click to enlarge.

One group designed a dexterity game about balancing dice within a time limit, and then challenging other players to match them. The dice balancing alone was really impressive, but the randomised time limit introduces a kind of bidding mechanic - you want to stack more dice to make it harder for everyone to match, but if you try to do too much you might fail to do it yourself. The dexterity element makes this very hard for modern AI and robotics - a specialised machine might manage it, but this is still something humans are far better at.

The group I was in tried to design a game that was hard for humans but easy for AI. We discussed several aspects that might affect this, but focused on a game where the consequences of an action were hard to visualise. The games we designed involved a board of large tiled hexes split into segments that pieces could move across. The hexes could also be rotated, which would trigger new rules and change the connectivity of the game board depending on what configuration the hex tiles ended up in. With a bit of iteration in the afternoon, we ended up with some quite interesting but challenging games.

The third group, like the first, focused on games that were hard for AI, and looked at language and creativity. They sketched out many different ideas, a lot of which emphasised common knowledge or a shared creative space between two or more players. For example, setting up a creative conundrum by one player, and then other players contribute creative extensions, and the first player judges the quality of each. Creativity is already tough for AI, but adding co-operation on top makes it extra challenging, and emphasises understanding how other people see the world (or being able to imagine it). The best game they designed (in my opinion!) was a collaborative association/mindreading game where both players must guess each other’s next move to end the game - but they can only try it once, resulting in a tense sequence of voting to end the game (or not) based on how confident you feel after each turn.

In addition to designing a lot of games and thinking about different aspects of challenge, some of our group also spent time thinking about how to break down these game elements more formally and think about them in isolation. We’re hoping to put it all together into a short paper some time in the future, with some evaluation of both the games we designed as well as more well-known games, broken down by the kinds of challenge they set for players. It was really refreshing and interesting to think about these topics (and also very fun to design and play games while doing so). Thanks to Nathan, Vanessa, Tommy, Raluca, Hui, Emily, Diego and Alex for joining me and making things!

Other Workgroups

At the end of each day we all give a short report on what we got up to. I took a few notes on each group so that you can get a rough flavour for what topics are being discussed here. I’ve also noted down the group leader in each case, so you can contact them if you want more info.

Evolutionary Computation in Games (Dan Ashlock)

This group had a lot of discussion about the current state (and past) of the field, and did a sort of informal survey - who does it, where is it used, what new openings are there? They discussed new kinds of representation that might be applicable to games, shared tips and hacks for better performance, and discussed future ideas for work in the area.

Discovering a Diversity of Skills (Jacob Schrum)

An extension of a workgroup from a previous day, this time they extended their experimentation into a simple art domain, using a data set of lily pictures and trying to reproduce the source picture with certain restrictions or constraints. Still in the early prototyping, but the group felt like they were on their way to something interesting. They plan to build some simple RL systems to try out exploring the space a bit more, and hopefully write something up in the future.

Human Co-operation and Competition (Katja Hoffman)

Another extension of a workgroup from a previous day. Using their findings from earlier in the week, they wanted to look at ‘zero-shot’ co-operation, where people don’t know who they are co-operating with. The data from their first experiment was more interesting than they had anticipated, so they build an online co-ordination game to gather more data. They discussed the idea of a co-ordination benchmark, and built a bot API to begin making AI that can play the game with other AI, or humans.

Meta-Dagstuhl: Past Trends and New Directions (Julian Togelius)

This is the fourth Dagstuhl, and there have been other events too like Shonan and Banff. This group revisited ideas from a previous panorama paper surveying the field, and looked at how these events work. They found some interesting data, like statistics on how many games were designed at each event (2019 was a new record for this), and data on diversity of topics. Underexplored topics include testing, audio and contemporary boardgames.

Learning to Learn (Spyridon Samothrakis)

How can we build systems that learn to adapt other systems which are learning another task? This sounded like a really interesting group that looked at new structures and patterns for learning systems, and discussed what aspects of such systems could be tweaked or adjusted by a higher-level ‘learning tutor’. A lot of theory and discussion, moving in a promising direction.

Day 4 - Generative Design in Minecraft

On Day 4 I attended a group led by Christoph Salge about the Generative Design in Minecraft competition, but this was a largely practical course so I won’t write up a separate post about it (however I am hoping to write a separate thing about the competition in the future). It was a very long and tiring week, and I pretty much collapsed when I got home, but I had a lot of good discussions with people and made some exciting plans. Thanks to everyone I worked with or talked to during the week!

LTR: We Didn't Playtest This: Legacies (thanks Raluca), Forward Models Is Fun, Hand Turkeys from last Dagstuhl, and Edward de Bono's eSport. Click to enlarge.