Graham Kendall

(Academic Year 2005/06)

Introduction

This page details the third year individual projects that are/have been supervised by Graham Kendall who is a lecturer in the Department of Computer Science at The University of Nottingham.

This page contains details of projects supervised in the academic year 2005/06.

The main project page can be found here.

Downloadable Dissertations

Some of the projects are available for download, so that you can view them. This facility is for current third year students, being supervised by me, who can use previous dissertations for their own research and also to look at previous dissertations to see how to lay them out etc. (it also saves the inevitable requests I receive asking to see a "good" project from last year, with the risk that I never see it again!).

As such, these files are password protected and, to get the username/password you need to EMAIL me. People outside of the university will not be given access.

In addition, I cannot give access to those students outside of my tutor group. You should approach your own tutor to get copies of previous dissertations they have supervised.

Note : I cannot make the files available in formats other than those already supplied.

3rd Year Projects Supervised in 2005/06

1. Playing Blackjack to Win - Evolving a Successful Blackjack Strategy Using Multi-Parameter
   Card Counting Techniques by Mark Thomas Paul Gale (pdf : 385KB)
   Abstract : In 1956 Roger Baldwin et al. published a paper entitled The Optimum Strategy in Blackjack. Using statistical analysis they proved that of all the casino games, Blackjack offered the best odds of winning if a player adopted a specific playing strategy. This strategy was further developed in the 1960s by Edward Thorp and Julian Braun of IBM. Using computer simulation Thorp was able to produce a system of play that affords the Blackjack player roughly even-odds with the casino, that system was called “Basic Strategy”. Thorp also considered the effects on the player of each of the different cards when removed from the deck. He assigned a negative value to the ‘good’ cards and a positive value to ‘bad’ cards. This was the basis of card counting, adding these values as the cards are exposed during game play indicated when the game is favourable to the player, at which time the player should increase their bet. Card counting has long been the main focus for Blackjack theorists ever since, attempting to ever increase a player’s edge over the casino. The majority of the research into Blackjack strategies and counting techniques has been carried out using computer simulation of hands and mathematical reasoning. Yet at the same time in the field of game playing, machine learning approaches have been applied to several other games, in order to develop an expert player. In 1997, IBM’s project Deep Blue beat the reigning world Chess Champion, Gary Kasparov, 3.5 to 2.5. This man vs. machine competition has also been extended to Checkers, with David Fogel’s Anaconda which gained grand master status and completed six perfect games against a commercially available checkers program. Gerald Tesauro used temporal difference learning to develop TD-Gammon for the game of backgammon. TD-Gammon was able to play to an ‘advanced’ level, losing only one point in a series of 40 games against a world-class Bill Robertie. The following is the report for a year long academic dissertation with the aim of applying similar machine learning techniques, as applied to many other game playing problems, to evolve a series of neural networks to develop an expert Blackjack player. This document contains a literary review of related work, both in the field of machine learning and game playing, as well as material directly related to Blackjack. It also includes the design for the learning architecture to be used for the development of the player, as well as a full description of the implementation, finally it includes the results of all experimentation with the players and an appraisal of the entire project.
   
   
2. Contrasting the Blackjack Playing Strategies Learnt Using Evolving Neural Networks when Applied to Different Rule Sets by Andrew James Mitchell (pdf: 1186KB)
   Abstract: The game of blackjack owes its popularity to its simplicity and the easy manipulation of odds. Strategies have been devised that allow the player to significantly reduce the house odds, and in some playing conditions can even offer an approximately break-even wager. The specific house edge is very much dependent on the rule-set being used, but these differ radically from casino to casino and country to country. The casino may use a single deck or multiple decks. They may reshuffle whenever desired, or when processing a fixed percentage of cards. They may limit the player to doubling down on 10 or 11 as opposed to any two cards. They may limit the number of times a player may split cards, or whether or not the player may split 10s. The following paper is a significant literature review on all related topics followed by a basic design and implementation of a project that will compare and contrast the differing strategies learnt by evolving neural networks under various sets of rules. I will finalise by discussing the results found and evaluate my performance throughout the project.
   
   
3. ???????? by Sarah Morris (pdf: 2359KB)
   Abstract: In 2001, David Fogel published a several papers detailing his creation of a Checkers player that using Darwinian evolution, taught itself to play at an expert level without human knowledge. As well as using this principle of evolution the system was not given human knowledge to advance its play and instead had to decide which features of the game were important. Further to this to evolve the system was only given a score after a set of games and was not told how this score was calculated. Despite this lack of human input the player taught itself to play checkers at an expert level. It is the aim of this investigation to see whether the architecture Fogel described can be adapted to the game of Texas Hold’em to produce a player with a similar level of dominance in its field. The report follows the creation of this player known online as ldyMarmalade25 through its journey from creation to application. Although the player did not achieve a high level of dominance winning only fifty six percent of hands it stayed in for on low money tables, the player clearly developed its own features for the game and improved throughout the evolutionary process.
   The investigation shows how the system reacted to various types of player both human and electronic and whilst it will never be a world champion system it certainly exhibits signs of learning, an important characteristic of intelligence. The systems ability will never match that of encyclopedic systems like the chess program Deep Blue, but this project shows the true meaning of “artificial intelligence”, by allowing the system to learn to complete the task by itself.
   
   
4. An Investigation into Automated Poker Play, specifically Opponent Modelling, within the Domain of Poker by Greg Pakes (pdf: 1492KB)
   Abstract: Creating a programme that automatically plays Online Poker, without any human intervention, has been the goal for many Poker Players for the past few years. Similarly, the creating of a unique world class computer Poker player has also attracted much research interest. A world class computer player has been a target that has proved to be out of reach for the time being. There are several attributes that have been identified as crucial for any poker player to be successful. It is fair to suggest that a Poker robot must share the same qualities. One of which is effective opponent modelling. This is the ability to model opponents, giving yourself a good estimate of how your opponent will play, formed solely by observations made at the table. This work will focus on using two Opponent Models to maximise the accuracy shared between them allowing a Poker robot to play at a reasonable level.
   I have successfully managed to create an automated Poker Player to play online. This player uses some opponent modelling that builds on previous proven ideas.
   
   
5. PokerFace: A Universal Interface for Gathering and Analysing Data from Poker Games in Real Time by Jack Ryan (pdf: 698KB)
   Abstract: Inspiration for this dissertation came from a desire to create opponent modelling systems for poker. When researching these systems it became apparent that there is currently no easy way to test these systems in real poker games. This dissertation documents the development of a program designed to allow the gathering of data from poker games involving real money, and present this data in such a way that it is useful for the development and testing of poker playing and oppponent modelling algorithms. The creation of this system has involved some complex pieces of programming, including a system to gather data directly from the computer display, and the creation of two parsing systems. Described within this dissertation is a system that seperates data gathering and data storage, and a language: GenHoldem that acts as a means of transferring data from the gathering system to the storage system. This language is designed to be as minimal as possible, and thus it should be an easy task to create automatic translators to convert other poker notations to this format.
   The system produced is currently not of a standard where it can be used for the gathering of data, but it does serve as a proof of concept for my design. Instructions for running the current system are in the ‘README.txt’ file included with the source code.
   
   
6. Evolving a Cribbage Player through Self Play by Neil Sculthorpe (pdf: 216KB)
   Abstract: This dissertation attempts to evolve a Cribbage playing algorithm through the medium of self play. The algorithm is not given any knowledge of Cribbage tactics; it is only permitted to learn from its experience of playing games. By using a cross between reinforcement learning and supervised learning, the algorithm builds up a database of possible situations, each having an associated move. This move is that which the algorithm believes to be the correct move to make in each instance. This dissertation summarises the related work in the field of computer game playing, describes the system that has been constructed, and then evaluates the success of the system. It is shown that the algorithm successfully learns from its past games and improves its own performance, before reaching a constant ability cap for the system.
   
   
7. Computer Poker Player and the Effects of Opponent Modelling by Christakis Sofroniou (word: 879KB)
   Abstract: Poker is a challenging game to develop a strong computer player for. It presents interesting challenges to overcome in order to make a strong player, challenges such as dealing with imperfect knowledge, multiple competing agents, risk management, agent modelling, deception and unreliable information, challenges which are rarely or not ever seen in the perfect information games that computers have been used to solve. The aims of this project were to produce a competent poker player, and to improve this player further by implementing opponent modelling into the playing strategy. The competent player was produced based on potential profit, hand strength and potential, and two further parameters that modified how tight and aggressive the player played. Further to this a second player was produced that evaluated opponent potential hand strengths based on an evolving weight table of previous actions made by each opponent, and a fuzzy algorithm to compensate for dealing with imperfect information. The second player performed well in the time span, learning accurately which hand strengths are likely to be played by the opponent in the pre-flop and post-flop betting rounds. Using these models the player was able to improve play over time to the point where it was winning and losing approximately the same amount of playing chips, and with testing results suggesting that the player would improve to become stronger than those that didn't use opponent modelling the initial aims were met.
   
   
8. An Online Poker Application by David Tolman (pdf: 101KB)
   Abstract: It’s the biggest craze to hit online gaming. Online Poker Applications are a big hit to Poker players around the world. Now without having to leave their homes, players across the world play poker with other players across the world online. I am going to design and implement an online poker game. Players across a network or the internet will be able to pit their poker expertise against each other. If the poker players like Tournaments, be it low stakes or high stakes, this poker client should accommodate.