Poker
Research Page
Poker
|
|
This page contains a history of computers playing poker. This review is currently done from an academic point of view but will be expanded to cover the history of the game in due course.
You might also want to look at my home page (with navigation bar) or my game playing research page. You might also be interested in poker articles or, if you are from the media, you might be interested in the poker media page.I am a lecturer at The University of Nottingham where I lecture on artificial intelligence and specialise on how computers can learn to play games without being told how to play (i.e. they are given no information as to game strategy, instead the computer has to "learn" the strategy for themselves). From an academic point of view I have published several articles on this research area (see my game playing articles).
The page contains references to the relevant academic literature (please let me know (by EMAIL) if I have missed any papers).
I have also listed some of the poker sites you might find of interest. If your site is not here, EMAIL me, and I'll include it.
This page is part of my game playing web site.
Poker has a long research history with von Neumann and Morgensten [VON44] experimenting with a simplified, two-player, version of poker in their seminal work on economics. They recognized that accomplished poker players regularly adopt bluffing as part of their game, which would have to be catered for in any automated poker player.
Findler [FIN77] studied automated poker, over a 20 year period. He also worked
on a simplified game, based on 5-card draw poker with no ante and no consideration
of betting position due to the computer always playing last. He concluded that
dynamic and adaptive algorithms are required for successful play and static
mathematical models were unsuccessful and easily beaten. Other than von Neumann's
and Findler's work research within poker playing has been limited. There have
been three main research groups working on the subject, all relatively recently.
One of the groups is headed by Jonathan Schaeffer (of Chinook fame). He and
members of his research team have developed ideas which have led to the release
of Loki, which is, arguably, the strongest automated poker playing program to
date. It is still a long way from being able to compete in the World Series
of Poker, an annual event held in Las Vegas, but initial results are promising.
Schaeffer's work concentrates on two main areas [BIL98a], [SCH99]. The first
is to make betting decisions using probabilistic knowledge [BIL99] to determine
what action to take (fold, call or raise) given the current game state. Billings
et. al. also uses real time simulation of the remainder of the game that allows
the program to determine a statistically significant result in the program's
decision making process. Schaeffer's group is also looking at opponent modeling
[BIL98b]. This allows Loki to maintain a model of an opponent that it uses to
decide what betting decisions to make. This is done on the assumption that players
adopt differing styles and any automated player has to be able to exploit this.
The Gala system developed by Koller and Pfeffer [KOL97] allows games of imperfect
information to be specified and solved, using a tree based approach. However,
due to the size of the trees they state "…we are nowhere close to
being able to solve huge games such as full-scale poker, and it is unlikely
that we will ever be able to do so."
Luigi Barone and Lyndon While have also recently carried out research into the automation of poker [BAR98], [BAR99], [BAR00]. They recognise that there are four main types of poker player : Loose Passive, Loose Aggressive, Tight Passive and Tight Aggressive players ([BAR99], BAR00]). In their first paper [BAR98] they suggest using evolutionary strategies as a way of modelling an adaptive poker player. They use a simple poker variant where each player has two private cards, access to five community cards and there is only one round of betting. This initial work incorporates three main areas of analysis; hand strength, betting position and risk management. The work demonstrates how a player that has evolved using evolutionary strategies can adapt its style to two types of table (loose and tight). In [BAR99] they develop their 1998 work by introducing a hypercube which is an n dimensional vector, used to store candidate solutions. The hypercube has one dimension for the betting position (early, middle and late) and another dimension for the risk management (selected from the interval 0..3). At each stage of the game the relevant candidate solutions are selected from the hypercube (e.g. middle betting position and risk management 2) and the decision is made whether to fold, call or raise. To make this decision the hypercube entry holds seven real numbers which are used, via suitably defined functions, to produce probabilities for each choice. It is the seven real values that are evolved depending on whether the player won the hand or not. Barone reports that this poker player improves on the 1998 version and is able to adapt to different playing styles. Their 2000 work [BAR00] extends the dimensions of the hypercube to include four betting rounds (pre-flop, post-flop, post-turn and post-river) and an opponent dimension so that the evolved player can choose from the hypercube which type of playing it is up against. The authors report this player out performs a competent static player.
Kendall and Willdig [KEN01] have carried out some investigations into an adaptive
poker player. Their work alows a poker player to evolve by playing games and
adapting its play based on whether the adaptive player wins or loses. They show
that a player that, initially, plays poorly is able to adapt its play over time
so that it eventually is able to win against its opponents.
Poker is interesting, from an artificial intelligence research point of view,
as it is a game of imperfect information. Unlike chess and checkers, games such
as poker have some information that is unseen. Poker also contains other unknowns
such as the playing styles of the other players who may use bluffing (and double
bluffing) during the course of the game. Unlike complete information games where
the techniques to solve the games (computational power allowing) have been known
and understood for a long time (such as mini-max search and alpha-beta pruning),
games of imperfect information have not received the same sort of analysis and,
doing so, could prove relevant to many other areas such as economics, on-line
auctions and negotiating.
Last Updated 26th Mar 2001