Well I think the state of the art is quite a varied thing. It’s a moving feast. Ask five different people, they’ll give you probably eight different examples of what the state of the art is. But there are certain avenues that are beginning to get a lot of attention.

If you think about the work that’s being done at DeepMind, which is probably a good example. They started off looking at getting machine learning models to play, what apparently seems to be rather trivial tasks, video games, in particular Atari games. That’s an interesting problem because it combines some of the deep neural network supervised learning models, with another technology called reinforcement learning. Which is the idea when we address problems where we don’t get feedback straight away as to whether we are right or wrong.

A classic example, you make a move in a game of chess and you don’t find out till four hours later. But that was a critical move that cost you the game. 

They were combining the latest trend in using very big neural network models with many layers. There are two reasons we can do that now. One, is we have very powerful computers and we have a lot of parallelisation. A lot of work is done on GPU’s, graphical processing units, and indeed, Google have their own tensor processing units or TPU’s. This means we can number crunch all this data.

Again, data is the key thing. These models have huge numbers of parameters. It’s all been exposed to statistics in the past, or the concepts required for parameter estimation. You need huge amounts of data that correspond with huge amounts of parameters in your models.

So we are able to use much more complex models because we now have the huge amounts of data that we need to be able to fit those models in a statistically robust way.