While experimenting with enhancements of the prediction model of Guess.js, I started looking at deep learning. I’ve focused mainly on recurrent neural networks (RNNs), specifically LSTM because of their “unreasonable effectiveness” in the domain of Guess.js. In the same time, I started playing with convolutional neural networks (CNNs), which although less traditionally, are also often used for time series. CNNs are usually used for image classification, recognition, and detection. Controlling MK.js with TensorFlow.js You can find the source code for this article and MK.js in my GitHub account. I haven't shared the dataset that I used for training but feel free to collect your own and train the model as described below! Also, to get an idea of how everything works together, feel free to play around with the widgets below. After playing around with CNNs, I remembered an experiment I did a few years ago, when the browser vendors introduced the getUserMedia API. In this experiment, I used the user’s camera as a controller for playing a small JavaScript clone of Mortal Kombat 3. You can find the game at my GitHub account. As part of the experiment, I implemented a basic posture detection algorithm which classifies an image into the following classes: