Upping the Ante: Top Poker Pros Face Off vs. Artificial Intelligence

20-Day Contest at Rivers Casino in Pittsburgh Begins Jan. 11

Wednesday, January 4, 2017 - by Byron Spice

Four of the world’s best professional poker players will compete against artificial intelligence developed by Carnegie Mellon University in an epic rematch to determine whether a computer can beat humans playing one of the world’s toughest poker games. 

In "Brains Vs. Artificial Intelligence: Upping the Ante," beginning Jan. 11 at Rivers Casino, poker pros will play a collective 120,000 hands of Heads-Up No-Limit Texas Hold’em over 20 days against a CMU computer program called Libratus.                

The pros — Jason Les, Dong Kim, Daniel McAulay and Jimmy Chou — are vying for shares of a $200,000 prize purse. The ultimate goal for CMU computer scientists, as it was in the first Brains Vs. AI contest at Rivers Casino in 2015, is to set a new benchmark for artificial intelligence.

"Since the earliest days of AI research, beating top human players has been a powerful measure of progress in the field," said Tuomas Sandholm, professor of computer science. "That was achieved with chess in 1997, with Jeopardy! in 2009 and with the board game Go just last year. Poker poses a far more difficult challenge than these games, as it requires a machine to make extremely complicated decisions based on incomplete information while contending with bluffs, slow play and other ploys."

A previous CMU computer program, called Claudico, collected fewer chips than three of the four pros who competed in the 2015 contest. The 80,000 hands played then proved to be too few to establish the superiority of human or computer with statistical significance, leading Sandholm and the pros to increase the number of hands by 50 percent for the rematch.

"I’m very excited to see what this latest AI is like," said Les, a pro based in Costa Mesa, Calif. "I thought Claudico was tough to play; knowing the resources and the ideas that Dr. Sandholm and his team have had available in the 20 months since the first contest, I assume this AI will be even more challenging."

Brains Vs. AI is sponsored by GreatPoint Ventures, Avenue4Analytics, TNG Technology Consulting GmbH, the journal Artificial Intelligence,Intel and Optimized Markets, Inc. Carnegie Mellon’s School of Computer Science has partnered with Rivers Casino, the Pittsburgh Supercomputing Center (PSC) through a peer-reviewed XSEDE allocation, and Sandholm’s Electronic Marketplaces Laboratory for this event.

"We were thrilled to host the first Brains Vs. AI competition with Carnegie Mellon’s School of Computer Science at Rivers Casino, and we are looking forward to the rematch," said Craig Clark, general manager of Rivers Casino. "The humans were the victors last time, but with a new AI from the No. 1 graduate school for computer science, the odds may favor the computer. It will be very interesting to watch and see if man or machine develops an early advantage."

Les said it’s hard to predict the outcome. Not only is the AI presumably better, but the pros themselves are playing better.

"From the human side, poker has gotten much tougher in the last 20 months," Les said. That’s because pros generally have embraced publicly available game theory tools that have elevated game play, he explained.

"Though some casual poker fans may not know all of them, Les, Kim, McAulay and Chou are among the very best Heads-Up No-Limit Texas Hold’em players in the world," said Phil Galfond, a pro whose total live tournament winnings exceed $2.3 million and who owns the poker training site Runitonce.com. Unlike the multi-player poker tournaments popular on television, professional one-on-one No-Limit Texas Hold’em is often played online.

"Your favorite poker player almost surely wouldn't agree to play any of these guys for high stakes, and would lose a lot of money if they did," Galfond added. "Each of the four would beat me decisively."

The Libratus AI encompasses new ideas and is being built with far more computation than any previous pokerbot, Sandholm said. To create it, he and his Ph.D. student Noam Brown started from scratch.

"We don’t write the strategy," Sandholm said. "We write the algorithm that computes the strategy." He and Brown have developed a new algorithm for computing strong strategies for imperfect-information games and are now using the Pittsburgh Supercomputing Center’s Bridges supercomputer to calculate what they hope will be the winning strategy.

"We’re pushing on the supercomputer like crazy," Sandholm said, noting they have used around 15 million core hours of computation to build Libratus, compared with the 2-3 million core hours used for Claudico. That computing process will continue up to and during the contest.

Claudico’s favored strategy was limping, a poker term for getting into a hand by calling, rather than raising or folding. Sandholm said that Libratus also will limp sometimes.

"It will make many types of weird moves — we know that already," he added.

Libratus is a Latin word, meaning balanced and powerful. It was chosen  because the program’s algorithm incorporates new technology for attaining what game theorists call a Nash equilibrium. Named for the late Carnegie Mellon alumnus and Nobel laureate John Forbes Nash Jr., a Nash equilibrium is a pair of strategies (one per player) where neither player can benefit from changing strategy as long as the other player’s strategy remains the same.

One of Libratus’ new technologies is a faster equilibrium-finding method. It identifies some paths for playing a hand as not promising. Over time, the algorithm starts to ignore those bad paths.

"We found that this is not just faster, but that the answer is better," Sandholm said.

Another change has to do with endgame strategies. During last year’s contest, the pros noticed Claudico was making some all-too-obvious bluffs that they were able to exploit. Rather than rely on abstractions for endgame play as Claudico did, Libratus will use the Bridges computer to do live computations with a new endgame-solving approach and algorithm.            

Head’s Up (two-player) No-Limit Hold’em is an exceedingly complex game, with 10¹⁶⁰ (the number 1 followed by 160 zeroes) information sets — each set being characterized by the path of play in the hand as perceived by the player whose turn it is. That’s vastly more information sets than the number of atoms in the universe.

The AI must make decisions without knowing all of the cards in play, while trying to sniff out bluffing by its opponent. As "no-limit" suggests, players may bet or raise any amount up to all of their chips.

Solving such a game has many real-world applications in areas also characterized by incomplete and misleading information, such as business, military, cybersecurity and medicine, Sandholm said. The algorithms are not poker specific but rather apply to a myriad of decision-making situations of incomplete information.

"Extending AI to real-world decision-making, where details are unknown and adversaries are actively revising their strategies, is fundamentally harder than games with perfect information or question-answering systems," said Nick Nystrom, senior director of research at PSC. "This is where it really gets interesting."

In February 2016, an earlier AI developed by Sandholm and Brown won both categories of Heads-Up No-Limit Texas Hold’em in the Annual Computer Poker Competition, announced at the Association for the Advancement of Artificial Intelligence conference in Phoenix.

The easier game of Head’s Up Limit Hold’em, which has 10¹³ information sets, has been near-optimally solved by a computer poker group at the University of Alberta, headed by CMU alumnus Michael Bowling.

To ensure that the outcome of the competition is not due to luck, the four pros will be paired to play duplicate matches — Player A in each pair will receive the same cards as the computer receives against Player B, and vice versa. One of the players in each of these pairs will play on the floor of the casino, while his counterpart will be isolated in a separate room.

For this second installment of Brains Vs. AI, the pros have agreed to increase the number of hands to improve the chance of reaching statistical significance, that is,ruling out with high confidence the possibility that either the humans or the computer win by just getting lucky. To do so, the pros will play more days and will "two-table," playing two hands simultaneously.

Play will begin at 11 a.m. each day at Rivers Casino and end around 7 p.m. The public is welcome to observe game play, which will be in Rivers’ Poker Room.  

The site of the competition, Pittsburgh’s Rivers Casino, opened in 2009 and has been named "Best Overall Gaming Resort" in Pennsylvania by Casino Player Magazine for seven years straight. No one under age 21 is permitted on casino property.

For more information, Contact:
Byron Spice | 412-268-9068 | bspice@cs.cmu.edu