World's leading artificial intelligence outsmarted in Go with just a 'trivial trick'

 "No move wins with move", the trick to defeat the world's top Go AI turns out to be surprisingly simple.

Go was the first field to be disrupted by artificial intelligence (AI). Before 2016, the world's best human Go players could still confidently defeat the strongest AI systems. But that changed with DeepMind's AlphaGo , which used deep neural networks to teach itself to play at a level that humans could not match. Most recently, the KataGo system has also gained popularity as an open-source Go-playing AI system that easily defeated top human Go players.

AlphaGo has made many human players "regretful".

But last week, a team of AI researchers published a paper outlining a method for beating KataGo , using adversarial techniques that exploit the AI ​​system's blind spots . By playing unexpected moves outside of KataGo's training range, a Go program that was determined to be much weaker—one that even amateur players could beat—was able to force KataGo to lose.

Adam Gleave, a PhD candidate at UC Berkeley, said he and his team developed what AI researchers call an 'adversarial policy.' In this case, the researchers' strategy was to use a combination of neural networks and a search tree method (also known as Monte-Carlo search) to find the right moves.

With its world-class prowess, KataGo learned Go by playing millions of games against itself. But that was still not enough experience to cover every possible situation, which left room for loopholes due to 'unwanted behaviour'.

'KataGo generalizes well to many novel strategies, but it gets worse the further it gets from the games it has seen in training,' said Gleave. 'We discovered a 'no-distribution' strategy that KataGo is particularly susceptible to, but there are likely many others.'

The new tactic causes KataGo (white) to lose even though the captured territory seems much larger.

Gleave explains that, in a Go game , the 'confrontation policy' works by placing the first piece in a small corner of the board. He then provides a link to an example where the opponent, controlling the black pieces, plays mostly in the upper right of the board. This allows KataGo (playing white) to occupy the rest of the board, and then places a few easily captured pieces in its territory.

"This fools KataGo into thinking it has won," says Gleave, "because its territory is much larger than its opponent's. But the bottom left territory (in the photo above) doesn't actually contribute to its score, because of the presence of black pieces there, making the territory not fully controlled."

Overconfident of a win—the system assumes it will win if the game ends with a score—when there is no more land on the board to expand, KataGo will make a pass, and its opponent will also pass. This signals the end of the game in Go, and the two players will stop to calculate the score.

The subsequent tally was not as KataGo expected. As the report explains, its opponent received points for its corner territory, while KataGo did not receive points for its territory, because that area still contained the presence of opponent pieces.

After Go, many other games have also been overwhelmed by AI such as chess, Chinese chess.

While this strategy may seem difficult to beat for KataGo, it is relatively easy for amateur human players to beat. Simply put, the strategy's sole purpose is to attack an unforeseen vulnerability in KataGo. And it is a demonstration that a similar situation can occur in almost any deep learning AI system.

'The study shows that AI systems that appear to operate at human levels often do things in a very unfamiliar way , and can therefore fail in ways that are surprisingly simple to humans,' Gleave explains. "This result is interesting in Go, but similar failures in safety-critical systems could be very dangerous."

Imagine an AI in the field of self-driving cars encounters an extremely unlikely situation that it did not expect, such as allowing humans to trick it into performing dangerous behaviors. ' This research highlights the need for better automated testing of AI systems to find worst-case errors, not just average-case performance,' Gleave said.