So far it seems to me like research on game-playing AI has not carried over well to solving problems that people care about. Deepmind's success in applied settings doesn't really seem to have benefitted from any meaningful way from the game-playing research as far as I can tell. What are the best examples of this so far that people are aware of?
The problem is that the expectation of being able to transfer this stuff over is more hype driven than anything else. People hear "AI beat the best person in the world at something intellectually hard" and think that it should be smarter than people in other ways too.
That doesn't diminish work on playing games. If they had released a chess or go challenge where the board is just bigger or more pieces, that would be dumb. But this challenge is a game that is a tiny bit closer to real "problems that people care about." Solving this won't get us to problems that people care about either, but it'll get us closer. It's only an incremental step, but that's okay.
Yeah I agree. I just wanted to highlight that to me the idea that doing better at games is advancing AI in a meaningful way is definitely overhyped.
Sometimes I think that the progress in games seems kind of orthogonal to progress in using machine learning to solve real world problems, because anytime you have a game it automatically gets you essentially infinite labeled training data set (each game has a score/outcome, and there are essentially infinite possible games). So as long as the compute scales up enough, any game humans can play will be solvable.
I wouldn't say that makes games orthogonal to real world problems. That's what makes them good stepping stones. Risk free "cheap" testing makes for fast research.
I totally agree about the ability to just skirt sample complexity. It's a tough one, made tougher by how early stage this work really is. We want bots to be able to match human ability and match human learning. Though they're put together, they're have very separate concerns.
For matching human ability, we're just beginning to learn techniques to get bots able to master hard tasks (e.g. incomplete information games, atari games, picking objects up). Those bots mostly learn waaaaaay slower than people. But never mastering is worse than slowly mastering, so it's early days.
On the other hand, you have people working on efficient learning. This is the question you're getting at with compute scaling arbitrarily-ish. It's more impressive if it can master a game after only playing it a small number of times. People are definitely working on this too, but for even simpler tasks. There's a lot of work right now in contextual bandits on learning fast, and that's a kind of baby-RL task. Even there, simulation tasks are super important because you really need a counterfactual to say whether you're doing well compared to alternatives.
I don't think it's orthogonal. Instead, there is a bias in news reporting. If someone manages to put one factory worker out of a job due to automation, it's not worth a headline. If someone beats the top human at some game, there'll immediately be a headline. People are being put out of jobs by machines as we speak.
Yes but computers can't process infinite data. Go has more possible state than atoms in the galaxy. It is not just about the data it is about novel solutions for interpreting the data and predicting useful states.
I mentioned this in another comment but two notable applied applications would be AlphaFold [1] (greatly outscoring SOTA on major protein folding problem), and their data center efficiency [2] which has reducing their data centre cooling bill by 40%. I expect to see many more applications in the coming years as the algorithms get to a more useful point. I think (as does Demis Hassabis) there are going to nearly endless applications in biosciences, weather prediction, material design, and much much more.
