Artificial Intelligence

Due:  Monday, October 16

In this homework you will experiment with reinforcement learning.  The directory ~echown/courses/370/chess contains files pieces.cc and pieces.hh which will learn to play an end-game in chess with no prior knowledge of the game.  The end-game for the homework is one in which one side has only a King left and the other side has a King and a Queen.  The two possible results are a Draw (when the side with the King takes the Queen) and a win for the side with the Queen.  The reinforcement learning system interacts with a chess system.  It gets state information to update its Q function, and it gets the number of available moves in each state.  It has no information whatsoever about chess.  Your goal in this homework is to experiment with training regimes for the system in order to maximize performance along several dimensions.  Ideally a good system should do at least the following things well:

  1. It should converge in a reasonable length of time
  2. It should perform well against a variety of opponents
  3. It should minimize the lengths of solutions

What to hand in

I want to see what program configuration you have chosen as the best one and I want support for your choice.  Support should come in several forms: arguments about why one choice is better than another, and data which backs up your arguments.  You should provide a coherent description of how you went about finding good settings. You are free to modify the program to collect extra data if you so desire.   Try to present your data in a coherent and forceful fashion.  Charts or graphs could be very helpful. No coding is necessary for this assignment. The program is done in C++ for efficiency sake. A Java version is also in the directory, but runs much slower.