Limitations of Opponent Masking

Building off the summer project in this project we present evidence that opponent masking underperforms in environments, like Go, which are heavily dependent on reacting to the opponent. We show this limitation by first deploying opponent masking in an ideal scenario in Go. We then repeat the experiment using a novel rules-based defence. Opponent masking was unable to defend the target model, however, due to the ease of the test scenario, the rules-based defence achieved a 100% win-rate. We believe opponent masking failed because it removes too much information from the model’s input space. Naively reducing the degree of masking improves performance, suggesting approaches that minimise the number of inputs masked could be effective