The intersection of artificial intelligence and play


By Phaeda Boinodiris  |  June 18, 2018


The way that we think about AI is colored by popular culture and science fiction. AI promises to create a vastly more productive and efficient economy and if properly harnessed can vastly improve our lives. But what is the future of AI and Play? There are many representations in science fiction of the intersection of Artificial Intelligence and Play.

Black Mirror (Above Image) 
I recently watched an episode of the NetFlix show Black Mirror called “Playtest” in which an AI both projects gameplay and mines content for games directly from a human brain in order to optimize engagement. 
(See YouTube video)


Ender's Game

Ender’s Game

Another example is from the book Ender’s Game. The main character Ender is cerebrally controlling a game called the Mind game in which he is in effect honing key skills making him better fodder for recruitment. (See YouTube video)


Star Treck Voyager

Star Trek Voyager

And of course, we must include Star Trek’s Holodeck as an example of play.
These examples of AI-enabled play are not as far away from us as you might think. There is not one single way in which play and AI is intersecting but many ways.

(See YouTube video)


How AI and Play are intersecting

We have examples of AI playing games. Kasparov completing against IBM’s Deep Blue. Watson completing against people in Jeopardy. Google’s Deep Mind AI besting the ancient strategy game of Go and now tackling StarCraft. Blizzard and the team at DeepMind have even created a special StarCraft visualization layer that has been adapted for AI.


..learning patterns through gameplay

There are many examples of AI actually learning patterns through gameplayFold-It, the protein folding game out of the University of Washington is a great example of this.

We have examples of AI augmenting gameplayin completely new ways. Consider Hazardous Games’ Resequencing Engine or their newest engine Amalgam which allows developers to easily combine objects, rules, logic, and behaviors to create new game mechanics and game objects. See an example of the finished product via their Cyberdefense Strategy game.

There are examples of AI helping game players get better. Consider the game of football where now coaches are turning to AI and data science to improve their games. IBM’s Watson is already coaching Fantasy Football leagues.


Contemporary AI’s four main categorizations and what it means for augmenting play

Let’s consider current era AI’s 4 main areas of focus as we ponder what it means for play.
 

Language

Back to the utopian world of Star Trek. Consider how the crew of Star trek used natural language to converse with their onboard computer. They were able to ask the computer questions about not only the diagnostics of their ship but also about the history and nature of the planets they were passing by.  The Star Trek ship computer is what inspired Watson’s creator. If you are building a game with a rich history, you can train Watson on the backstory of your game and converse with NPC characters in the game naturally. The AI will recognize the context of the words.

You can integrate AI APIs into a game in order to access other information which might be relevant. We worked with a high school out in Texas that re-skinned Minecraft and wanted the AI to communicate to in-game nanobots. (See Medical Minecraft.)

Mining personalities to curate custom experiences


Chewy

We can mine both structured and unstructured datasets in order to find personality patterns. I can use AI to analyze Abraham Lincoln’s personality by studying his written speeches. I can use AI to mine all of Donald Trump’s twitter feeds to determine his personality too. When you have personality insights, you can use this as a means to tailor experiences to an individual… Imagine that you are about to create a character in an RPG when it asks you for your Twitter ID. The AI mines your twitter content and decides to assign you as a troll based on how you behave in real life, complete with the big hairy smelly feet.
 

People ultimately really love to learn about themselves- that’s why people keep playing those ridiculous Facebook quizzes (Which Star Wars character are you, etc etc).  We can even make our own Harry Potter Sorting hat the decides based on your profile which house you REALLY belong to.
 

We have APIs that have the ability to recognize what emotion you are expressing. So imagine that you can curate an experience to optimize a particular emotion. You can imagine how this could be used for games that train salespeople!


Data Insights

We can glean insights from data to achieve game play balancing and in effect serve up a level of a game that is neither too easy nor too difficult for a play in order to achieve maximum flow. This is in effect what we at IBM are doing with Sesame Street using Watson’s Tradeoff Analytics API. This can be used to create custom learning pathways for people based on their skillsets and preferences.

And remember the sci fi games that connect directly into your brain? That is here too. Here is a video of how IBMers in Ireland tricked out the old toy Hungry Hungry Hippos to be played solely by players going into semi meditative states and becoming very very calm. We actually have high schools now building these newly smart toys using old toys in their MakerSpaces and play-testing them at children’s hospitals.


Vision

Contemporary AI can describe what is in an image and be 95% accurate, which means… with this capability I would have the power to put on AR headgear and the AI can KNOW that there is a table in front of me and have the behaviors of the AI driven characters behave in ways that they know there is a table there too.  The AI systm is trained to recognize what a table looks like.
 

  • Imagine having a camera that can guess at what you are building with your building blocks and make recommendations for you. Developers can do this now with something called Watson Similarity Search API.
     
  • Researchers have programmed drones to recognize which bird of prey is trying to attack them so that they play a sound that is uniquely irritating to that bird to scare it away.
     
  • One company is building an AR disaster preparedness game for kids. A child will be able to scan his or her room and the AI will recognize where the exits are so that the app may run through highly personalized what if scenarios.

Speech

We all know what a chatbot is right? Well with the APIs we have available today, we can make Empathetic conversation bots. Bots that are able to use natural language but then use all of this other data about a user like, personality, sentiment, tone, and other data that may come from sensor devices to curate a custom experience. In VR, we can use speech as a means to bring objects from libraries directly into a VR world.


What this means for Business

We are all swimming in a deluge of data, sometimes feeling like we are drowning in numbers and algorithms that don’t feel like they seemingly connect to us.  Organizations are hiring data scientists in order to find insights, but rarely is the actual end user, the consumer of the insights, considered. These end users may look at the revealed insight and say ‘Why should I care? How does this effect me? Why should I believe you? Now what?

 

When you pair artificial intelligence and play you are actually well on your way to making Big Data actionable- getting end users to actually be motivated to care, to be optimally engaged.

On making Data Science Actionable
I had a government agency a couple of months ago invite me to interview their first data scientist. I remember scratching my head thinking ‘Why am I here?’, after all I was the only outsider invited to help with the interviews.  The last set of questions for the scientists were of this nature:

You realize that the data that you will be curating has enormous socio-economic importance to our citizens. It may influence which zip code they decide to move to, which programs they support, and even who to vote for. How will you curate your data in such a way where it can be consumed not just by in house stakeholders, but also community leaders and most importantly the citizenry. “  It was then and there that I realized that they too were looking at using techniques from games and or PLAY as a means to make data science actionable. So consider everything that I have described to you can be used not only to optimize entertainment but indeed to change mindsets, motivate behaviors, and get us all to be more productive in the way we make decisions.