DeepMind doesn't generally comment on these sorts of things, but I think they knew it would be strong, just not how strong.
It is hard to accurately gauge the playing strength of such a program for the following reasons:
* Go is not solved, it doesn't really even admit a good heuristic for close games.
In chess, given enough time, we can do a pretty good job of searching the possible outcomes from a given position, and even if we can't evaluate all the possible endgames, evaluation functions exist that give us an idea of which side is ahead when we terminate the search.
Go has a much higher branching factor (more moves available at each turn, making the search more expensive) and short of a catastrophic blunder, it's hard to quantify who is ahead at each point in time[1].
So we can't (in general) quantify optimal play, and therefore cannot quantify how much AlphaGo (or anyone else) deviates from it.
* One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning.
According to some people I've talked to, this can be suboptimal, but there are methods of addressing this[2].
Note that you can't just change the objective function from "winning" to "win by as much as possible" without potentially reducing AlphaGo's strength.
* The value function learned by a deep net is hard to interpret, partly because it encodes information about the possible futures arising from a position, and partly because it involves a tremendous number of calculations to compute.
We do not know what it is representing at the intermediate levels-- techniques that aim to visualize or cluster unit activations can provide a bit of insight, but there's always the possibility that we're interpreting the patterns incorrectly because we're trying to fit it into the framework of "what would a human think?".
Further, the representation is somewhat monolithic-- we can't tweak the value of one thing without changing the values of others.
In chess engines, we might modify the material value of, say, a knight, without affecting the value of a rook.
In a convolutional net, if we adjust the value of one position, it will tend to affect the value of many others.
* We can attempt to quantify its strength by comparing it to other programs, but AlphaGo has already crushed other programs (99% win rate), so all that it tells us is that it is stronger than those programs.
Essentially, without the ability to perform a significant amount of searching, or gauge strength via margin-of-victory, or to examine the program from other angles, it's hard to gauge just how good the program is.
The only thing we can do is throw skilful opponents at it, and see if it fails eventually[3].
In its current incarnation, though, it seems like its playing strength is just going to be "stronger than you".
---
1. Hence the need for Deep Reinforcement Learning-- we learn how valuable each position is based on the results of the positions that it can lead to.
2. By modifying the search to maintain a given margin once it has been attained-- but I do not work at DeepMind, nor am I an expert on MCTS like some of my colleagues, so I don't know if this could adversely affect AlphaGo's overall strength.
3. We might be able to get a better idea by having extremely strong players play against weaker versions of AlphaGo (ones with less computing power) and then sort of telescope upwards once a good baseline has been set, but it remains to be seen if the current level of invincibility is due to it not having been around long enough for its weaknesses to become apparent[4].
4. How would an AI researcher play against it? I've talked to a few people, and the answers have been: (a) play the game to the conclusion, don't resign; (b) attempt to take the game off-the-rails into parts of the state-space it hasn't really explored before (although this is risky because the human player is more likely to make mistakes in these cases as well), and (c) let it get ahead somewhat so that it "relaxes" (see the points about margin), allowing you to catch up and overtake it.
The way to find out how strong AlphaGo is is to start giving handicap and see when the game becomes even. A difference in rank of 1 stone corresponds roughly to a 2:1 advantage. It is traditional in long-running matches to increase the handicap by 1 after one player wins three consecutive games. I haven't heard any talk of that here -- I'm sure Lee Sedol wants the chance to beat AlphaGo in an even game -- but it would be interesting. It would also be interesting to see how AG plays when it thinks it is behind.
That might work; the concern is that this would take it too far off of the task it was trained on.
That is, if it doesn't have a lot of experience being significantly down, then it won't play nearly as well when trying to catch up-- but that doesn't matter in even games because it never gets that far behind.
You're right that it would be interesting to see, though-- we need to get better at understanding these sorts of systems, at least until they can start optimizing themselves.
Alternatively, we might train a different agent (OmegaGo?) to try to win by the largest margin possible-- if it works as well as AlphaGo, then that might give us some more insight into how strong both programs are.
>One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning.
One interesting implication of this is that it very much applies to perfect information games--not games that involve an element of luck or, indeed, pretty much anything that involves the real world. In those latter environments, you absolutely want to run the score up or otherwise take small chances to gain an immediate advantage to create a cushion against future unexpected events (even if they slightly hurt your odds if those events don't occur).
> are willing to sacrifice margin of victory to maintain or increase their odds of winning. According to some people I've talked to, this can be suboptimal,
I suspect this is less suboptimal when a computer is involved since a computer never gets tired or makes an attnetion-based mistake.
DeepMind doesn't generally comment on these sorts of things, but I think they knew it would be strong, just not how strong.
It is hard to accurately gauge the playing strength of such a program for the following reasons:
* Go is not solved, it doesn't really even admit a good heuristic for close games. In chess, given enough time, we can do a pretty good job of searching the possible outcomes from a given position, and even if we can't evaluate all the possible endgames, evaluation functions exist that give us an idea of which side is ahead when we terminate the search. Go has a much higher branching factor (more moves available at each turn, making the search more expensive) and short of a catastrophic blunder, it's hard to quantify who is ahead at each point in time[1]. So we can't (in general) quantify optimal play, and therefore cannot quantify how much AlphaGo (or anyone else) deviates from it.
* One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning. According to some people I've talked to, this can be suboptimal, but there are methods of addressing this[2]. Note that you can't just change the objective function from "winning" to "win by as much as possible" without potentially reducing AlphaGo's strength.
* The value function learned by a deep net is hard to interpret, partly because it encodes information about the possible futures arising from a position, and partly because it involves a tremendous number of calculations to compute. We do not know what it is representing at the intermediate levels-- techniques that aim to visualize or cluster unit activations can provide a bit of insight, but there's always the possibility that we're interpreting the patterns incorrectly because we're trying to fit it into the framework of "what would a human think?". Further, the representation is somewhat monolithic-- we can't tweak the value of one thing without changing the values of others. In chess engines, we might modify the material value of, say, a knight, without affecting the value of a rook. In a convolutional net, if we adjust the value of one position, it will tend to affect the value of many others.
* We can attempt to quantify its strength by comparing it to other programs, but AlphaGo has already crushed other programs (99% win rate), so all that it tells us is that it is stronger than those programs.
Essentially, without the ability to perform a significant amount of searching, or gauge strength via margin-of-victory, or to examine the program from other angles, it's hard to gauge just how good the program is.
The only thing we can do is throw skilful opponents at it, and see if it fails eventually[3]. In its current incarnation, though, it seems like its playing strength is just going to be "stronger than you".
---
1. Hence the need for Deep Reinforcement Learning-- we learn how valuable each position is based on the results of the positions that it can lead to.
2. By modifying the search to maintain a given margin once it has been attained-- but I do not work at DeepMind, nor am I an expert on MCTS like some of my colleagues, so I don't know if this could adversely affect AlphaGo's overall strength.
3. We might be able to get a better idea by having extremely strong players play against weaker versions of AlphaGo (ones with less computing power) and then sort of telescope upwards once a good baseline has been set, but it remains to be seen if the current level of invincibility is due to it not having been around long enough for its weaknesses to become apparent[4].
4. How would an AI researcher play against it? I've talked to a few people, and the answers have been: (a) play the game to the conclusion, don't resign; (b) attempt to take the game off-the-rails into parts of the state-space it hasn't really explored before (although this is risky because the human player is more likely to make mistakes in these cases as well), and (c) let it get ahead somewhat so that it "relaxes" (see the points about margin), allowing you to catch up and overtake it.