. Github link to the project hw. Facebook Messenger recently received an update that allows users to play games within the App, amongst them was a vertical scrolling game — , which was getting pretty competitive within my social group. Endless Lake In this article I’ll be talking about the processes I used and challenges I faced whilst building a bot for the game Endless Lake. endless lake I don’t like having free time I recently finished a side project and was feeling rather empty inside, and so I decided to build a simple bot to play this game with the following two rules enforced: The inputs for the robot must either be the or a processed version of the raw pixel values. raw pixel values from the screen No hard-coding rules. Or in other words, it’ll attempt to learn how to play the game by watching a user play it. bot? bot. was chosen as the language of choice as I was familiar with it and it has a ton of machine- libraries that integrates with it. Python learning Mouse/Keyboard Event handler This is really crucial in collecting data and enabling the bot to play the game. Fortunately there’s a really easy to use library which fits the bill — . pyuserinput Screen Capture Capturing a region of the screen at a good rate (at 30 fps) was already a challenge due to Python’s interpretive nature. Well-known libraries such as or was able to take a screenshot of the region every 0.1 seconds or so (or around 10 fps), which was too slow and only 1/3 of the speed which I originally wanted. least pyscreenshot PIL/Pillow I was ready to give up until I stumbled onto this talking about . Installation was a pain in the ass but was worth it. I could capture a region of my screen at around 200 fps, a 20 fold increase! korean blog python-wx If you would like to have fast screencapture using Python, have a look at the . code Pre-processing I decided to use OpenCV as it had a suite of that suited my needs. After grabbing the screen region, I used to figure out the contour of the game window, and crop it so it only captures the game window and nothing more. tools Otsu’s dynamic thresholding I then divided the game window into a grid (which can be changed), used some to figure out where the platform and player was relative to each other. NxN simple RGB thresholding Since I didn’t really care anything that was too far ahead or too far behind the player, I decided to just take into account 6 rows in front of the player. These rows will be used as inputs for our neural network. A visualization of the inputs can be seen below. (Red=Water, Green=Platform, Blue=Player) Grid area depicts the inputs for our neural network ML I decided to use due to its sheer simplicity and ease-of-use. Of course you can use other techniques such as random trees, SVMs, KNN, etc. Neural Networks from scikit-learn Does it even work? Errr well, it does, with enough data. Here’s how it performs with 1 training set: 6 training sets: 10 training sets: TL;DR: More data = more performance If you have any questions just email me at . kendricktan0814 at gmail.com
