Training an AI

How do you train an AI to play the game better than before? This is a tough question but I’ll try to tell you a bit of how we’re doing it with Rising Empires. But let’s start with examining what type of AI we have.

In December 2012 I wrote a post called ‘Strategic AI overview‘. In this post I claimed that the AI used in Rising Empires is an ‘utility-based’ AI. An utility-based system is flexible and doesn’t have a pre-defined set of triggers that decide when to do what. Instead, it considers all potential actions and evaluates what is good and what is bad. When this is done it selects the actions which are most appropriate to the situation. Rising Empires is a very complex game and at any time there will be multiple possible actions. Depending on the AI’s resources it might refrain from doing anything, just do one thing or start a number of new projects in a single turn.

Each action has a priority value. Each race has a basic set of values, but the behavior of each empire also depends on its religion and ‘personality’. What I want to optimise is the basic set of values for the race. The religion and personality modifiers are not meant to make the AI better, they are meant to make the AI play a role in order to make the game more interesting for the player.

This is a list of the actions and their start values:
Build Army Reserve: 3.00
Build Scout: 3.00
Conquer Settlement: 3.00
Need Garrison: 3.00
Guard Cave Opening: 3.00
Increase Population: 3.00
New Settlement: 3.00
Raid: 3.00
Repel Incursion: 3.00
Standing Army: 3.00

But how will I optimise these basic set of values? My idea is to let the AI develop itself using genetic algorithms. I’ve created a huge battleground with room for 10 empires on the Surface and 10 in the Netherworld. All empires on the Surface will be Humans and all in the Netherworld will be Greenskins. Each game will run for 200 turns and the top three of both the Human and the Greenskin AIs are selected to continue in the next game. To develop the AI I’ll do a crossover between AI 1+2, 1+3 and 2+3 of each race. Then I’ll do the same crossovers as before but also mutate one of the values. For the tenth AI I’ll add a fully random one.

Crossover – When doing the crossover I will randomly select values from each of the two parents. For example build scout might be taken from parent 1 and conquer settlement from parent 2.
Mutation – When I mutate a set of values a random value will be increased or decreased with 1 (value must be between 0.5 and 10).
Random set – All values are randomly set between 0.01 and 10.

It’ll be interesting to see how this turns out. Will the values for Humans and Greenskins be similar or will they develop differently? With each game running on a random map and with the possibility that even a poor AI might be lucky and capture a capital and win a game, I’m not 100% sure of success in this. But by studying the values from the winners of each game I should be able to make enough conclusions to be able to improve the performance of the AI of all races.

Part two can be found here.