Run the Dobble Challenge on MaaXBoard

Intro

In part one of this tutorial, we learned about the math behind the card game Dobble and looked at the dataset.

In part two, we created and augmented our Dobble Card dataset, trained a Dobble-playing machine learning model, and tested it on a real life game between human and machine!

Part three picks up where we left off, but this time we will try running our Dobble Game on specialized hardware - the MaaXBoard!

Prerequisites

See my previous tutorials to do the following:

1. Get MaaXBoard setup in headless mode (about an hour)

2. Install OpenCV (about 2 hours).

3. If you have the MIPI-CSI Camera, plug it into the board, blue side facing the board's ethernet port. Otherwise, plug your USB webcam into the USB port.

UPGRADE OR DOWNGRADE TENSORFLOW?

Models that are generated on Tensorflow 2.3.0 and beyond are not compatible with Tensorflow 2.1.0. Because we generated the model on Tensorflow 2.4.0 (the latest as of today), we'll either have to 1.) build our own wheel for Tensorflow on Arm 2.2.0 and above, or 2.) we'll have to rebuild our model on a PC running Tensorflow 2.1.0.

Option 1: Upgrade to Tensorflow 2.3.0

In the Tensorflow installation tutorial I wrote last spring, I included a current version of the prebuilt .whl file, which at the time was 2.1.0. Tensorflow is now at version 2.4.0. I just wrote a tutorial that shows how to build Tensorflow from source for use on Arm, so you can follow it to build a Tensorflow wheel for MaaXBoard. The project also includes the .whl file for Tensorflow 2.3.0 that you can download.

Copy your Tensorflow wheel file over to the MaaXBoard:

scp tensorflow-2.3.0-cp37-none-linux_aarch64.whl ebv@[YOUR-IP]:

If you've already installed a different version of Tensorflow on your MaaXBoard and would like to keep it, you can simply create a new virtual environment:

mkvirtualenv tf23 -p python3
workon tf23
pip install tensorflow-2.3.0-cp37-none-linux_aarch64.whl

If you run "pip show tensorflow" from within your virtual environment, it should now show Tensorflow 2.3.0

Now that we have the latest version of Tensorflow installed, we can install the Keras and kaggle on our MaaXBoard:

pip install keras
pip install kaggle

Option 2: Build an old version of the model and run it on Tensorflow 2.1.0

To rebuild the model, follow the instructions in the tutorial Training The Dobble Challenge, but this time you'll have to uninstall tensorflow and keras and reinstall the correct versions.

On your PC:

pip uninstall tensorflow
pip uninstall keras
pip install tensorflow==2.1.0
pip install keras==2.3.1

You will also need to uninstall and reinstall an older version of h5py:

pip uninstall h5py
pip install h5py==2.10.0

Verify that you have the correct version of Tensorflow by typing:

which tensorflow

You'll also need opencv and kaggle installed.

Once your new model has been built, copy it to your MaaXBoard:

scp dobble_model.h5 ebv@[YOUR-IP]:

You'll need an older version of keras that still works with Tensorflow 2.1.0

pip install keras==2.3.1

Link OpenCV to your new virtual environment

If you created a new virtual environment to install Tensorflow on, link OpenC to it

cd ~/.virtualenvs/tf23/lib/python3.7/site-packages/ 
ln -s /usr/local/lib/python3.7/site-packages/cv2/python-3.7/cv2.cpython-37m-aarch64-linux-gnu.so ./cv2.so

You'll also need to install imutils:

pip install imutils

You'll also need to install numpy:

pip install numpy

Test to make sure it installed correctly:

python -c 'import cv2; print(cv2.__version__)'

Give yourself camera permissions

MaaXBoard's linux version doesn't allow non-root users to access serial devices, like MIPI-CSI or USB cameras. You'll have to grant your user permissions. Run:

ls -l /dev/video*

This will tell you the ownership and the permissions. You should now be able to run sudo chmod for any cameras you have to grant permission to non-root users:

sudo chmod -R a+rwx /dev/video1
sudo chmod -R a+rwx /dev/video0

Note: This change will only take be in effect until the MaaXBoard reboots. To create a more permanent change, while you're still logged in as root, edit /etc/rc.local file to include this line (use video1 instead of video0 you're using a USB camera) just before "exit 0":

sudo chmod -R a+rwx /dev/video0

Once you're finished, run:

sync
sudo reboot

Download and unzip the code

git clone https://github.com/albertabeef/dobble_buddy
cd dobble_buddy

If you are running the included model using Tensorflow 2.2 or above, unzip the model:

unzip dobble_model.zip

If you are using the MIPI-CSI camera, open dobble_detect_live:

nano dobble_detect_live.py

Uncomment the first line and comment the second line so it looks like this:

input_video = 0 # laptop camera

#input_video = 1 # USB webcam

You'll also have to remove both occurences of this line, as Debian doesn't like the characters in '1':

After edting, save the file.

Pull the dataset from kaggle

Because the MaaXBoard only has 2GB of RAM and as much non-volatile memory as your SD card brings (max 64GB), it's not practical to augment datasets or train a model on the MaaXBoard.

Instead, we run what's known as inference. Inference just means running the pre-trained model on a device.

Even though we won't be running training, the live script still requires the folder dobble_deck01_cards_57 from the dataset in order to show reference images. Unfortunately there's no way to download a single folder from Kaggle, so we'll have to download the full dataset. It is 201MB, so make sure you have at least that much space on your MaaXBoard by typing df -h. Edit the dobble_dataset_download.py file to include the kaggle API key you created. Then run it to download the dataset.

python dobble_dataset_download.py

You can check to make sure you downloaded the correct dataset by running:

python dobble_dataset_overview.py

Optional: Set up the camera mount and position your cards

I found this very practical and easy to print camera mount on Thingiverse that works well with the Maaxboard Camera. I'm using it with OpenBeam, so I can slide it up and down to adjust it, but it would also be feasible to use it simply by screwing it to a wooden frame.

Run inference

Finally, the moment we've all been waiting for. Run the dobble_detect_live script on your MaaXBoard:

python dobble_detect_live.py

And that's it! You should be able to play Dobble using your MaaXBoard!

References

The Solution

MaaXBoard SD card image with all of the above work already done for you as a reference solution: MaaXBoard DobbleBuddy SD Card Image

The Dobble Dataset

The Dobble dataset, available on kaggle:

Kaggle - Dobble Card Images

The Dobble Challenge

Getting started with machine learning for the Dobble card game using TensorFlow.

Hackster - The Dobble Challenge

Training The Dobble Challenge

Train a machine learning model that can play Dobble (Spot-It) against you. This is part two of The Dobble Challenge.

Hackster - Training The Dobble Challenge