Libratus, an artificial intelligence, just beat the world's top poker players by a margin of $1.7 million. (Not real money, sadly for the scientists behind the AI.) "I thought we had a 50-50 shot, but to have such a huge victory, I would have never guessed," said Tuomas Sandholm, the professor at Carnegie Mellon behind Libratus.
"We have proven that the best AI is better than the best humans," he said.
This is the second time a system from his lab has won a poker world championship, and the first time in the most difficult "heads up / no limit" version of the imperfect information game.
This isn't Sandholm's first time building a system that beats the market. His background includes founding CombineNet. Before its acquisition in 2010, the company commercialized over 800 of the world's largest combinatorial auctions, with over $60 billion in total market size and over $6 billion savings. He's also proud that his algorithms run the UNOS kidney exchange, which covers 2/3 of the U.S. market for kidney transplants.
Making an AI game master
Libratus is a system of systems designed to work with imperfect information in three steps. Step one is learning the game. "We give the AI a description of the game. We don't tell it how to play," says Noam Brown, a Ph.D grad student and researcher on the Libratus team.
As Libratus computed game after game against itself in training, the program reinforced patterns that led to successful outcomes. In addition to its pattern recognition, the Carnegie Mellon team built a second system that focuses on the current game and runs potential end-game scenarios.
The more artificial intelligence plays, the more it adapts to the opponent
By solving forward for potential end game scenarios, Libratus continually refines its strategy to the specific stage of play and new information. "Instead of just solving the endgame once, we are solving it every time the opponent makes a move in the endgame. So we can actually take the opponent's bet sizes into account. And we do what we call safe endgame solving, taking into account the opponent's mistakes so far," explains Sandholm. The approach is the topic of a paper he released recently.
Finally, once a day, a third system reviews the day's play for predictable patterns. "Based on what holes the opponent found in our strategy, the AI will automatically see which of those holes have been the biggest and the most frequently exploited. And then overnight on a supercomputer, it will compute patches to those pieces of the strategy, and they're automatically glued into the main strategy," Sandholm explains.
Predicting the future of AI
"These algorithms work for any imperfect information game," says Sandholm of the recent game-changing artificial intelligence poker victory. "And by game, I don't mean recreational. I mean these games can be very high stakes, like business-to-business negotiations, military strategy planning, cybersecurity, finance, medical treatment planning of certain kinds."
He predicts future AI will be business centric. "These are really for a host of applications, really any situation that can be modeled theoretically as a game. Now that we've shown that the best AI's ability to do strategic reasoning in an imperfect information setting has surpassed that of the best humans, there's really a strong reason for companies to start using this kind of AI." Learn more about what he's doing in this recent talk: