In part one of this tutorial, we learned about the math behind the card game Dobble and looked at the dataset.
In part two, we created and augmented our Dobble Card dataset, trained a Dobble-playing machine learning model, and tested it on a real life game between human and machine!
Part three picks up where we left off, but this time we will try running our Dobble Game on specialized hardware - the MaaXBoard!Prerequisites
See my previous tutorials to do the following:
1. Get MaaXBoard setup in headless mode (about an hour)
2. Install OpenCV (about 2 hours).
3. If you have the MIPI-CSI Camera, plug it into the board, blue side facing the board's ethernet port. Otherwise, plug your USB webcam into the USB port.UPGRADE OR DOWNGRADE TENSORFLOW?
Models that are generated on Tensorflow 2.3.0 and beyond are not compatible with Tensorflow 2.1.0. Because we generated the model on Tensorflow 2.4.0 (the latest as of today), we'll either have to 1.) build our own wheel for Tensorflow on Arm 2.2.0 and above, or 2.) we'll have to rebuild our model on a PC running Tensorflow 2.1.0.Option 1: Upgrade to Tensorflow 2.3.0
In the Tensorflow installation tutorial I wrote last spring, I included a current version of the prebuilt .whl file, which at the time was 2.1.0. Tensorflow is now at version 2.4.0. I just wrote a tutorial that shows how to build Tensorflow from source for use on Arm, so you can follow it to build a Tensorflow wheel for MaaXBoard. The project also includes the .whl file for Tensorflow 2.3.0 that you can download.
Copy your Tensorflow wheel file over to the MaaXBoard:
scp tensorflow-2.3.0-cp37-none-linux_aarch64.whl ebv@[YOUR-IP]:
If you've already installed a different version of Tensorflow on your MaaXBoard and would like to keep it, you can simply create a new virtual environment:
mkvirtualenv tf23 -p python3
pip install tensorflow-2.3.0-cp37-none-linux_aarch64.whl
If you run "
pip show tensorflow" from within your virtual environment, it should now show Tensorflow 2.3.0
Now that we have the latest version of Tensorflow installed, we can install the Keras and kaggle on our MaaXBoard:
Option 2: Build an old version of the model and run it on Tensorflow 2.1.0
pip install keras
pip install kaggle
To rebuild the model, follow the instructions in the tutorial Training The Dobble Challenge, but this time you'll have to uninstall tensorflow and keras and reinstall the correct versions.
On your PC:
pip uninstall tensorflow
pip uninstall keras
pip install tensorflow==2.1.0
pip install keras==2.3.1
You will also need to uninstall and reinstall an older version of h5py:
pip uninstall h5py
pip install h5py==2.10.0
Verify that you have the correct version of Tensorflow by typing:
You'll also need opencv and kaggle installed.
Once your new model has been built, copy it to your MaaXBoard:
scp dobble_model.h5 ebv@[YOUR-IP]:
You'll need an older version of keras that still works with Tensorflow 2.1.0
Link OpenCV to your new virtual environment
pip install keras==2.3.1
If you created a new virtual environment to install Tensorflow on, link OpenC to it
ln -s /usr/local/lib/python3.7/site-packages/cv2/python-3.7/cv2.cpython-37m-aarch64-linux-gnu.so ./cv2.so
You'll also need to install imutils:
pip install imutils
You'll also need to install numpy:
pip install numpy
Test to make sure it installed correctly:
python -c 'import cv2; print(cv2.__version__)'
MaaXBoard's linux version doesn't allow non-root users to access serial devices, like MIPI-CSI or USB cameras. You'll have to grant your user permissions. Run:
ls -l /dev/video*
This will tell you the ownership and the permissions. You should now be able to run sudo chmod for any cameras you have to grant permission to non-root users:
sudo chmod -R a+rwx /dev/video1
sudo chmod -R a+rwx /dev/video0
Note: This change will only take be in effect until the MaaXBoard reboots. To create a more permanent change, while you're still logged in as root, edit /etc/rc.local file to include this line (use video1 instead of video0 you're using a USB camera) just before "exit 0":
sudo chmod -R a+rwx /dev/video0
Once you're finished, run:
Download and unzip the code
git clone https://github.com/albertabeef/dobble_buddy
If you are running the included model using Tensorflow 2.2 or above, unzip the model:
If you are using the MIPI-CSI camera, open dobble_detect_live:
Uncomment the first line and comment the second line so it looks like this:
input_video = 0 # laptop camera
#input_video = 1 # USB webcam
You'll also have to remove both occurences of this line, as Debian doesn't like the characters in 'ï»¿1':
After edting, save the file.Pull the dataset from kaggle
Because the MaaXBoard only has 2GB of RAM and as much non-volatile memory as your SD card brings (max 64GB), it's not practical to augment datasets or train a model on the MaaXBoard.
Instead, we run what's known as inference. Inference just means running the pre-trained model on a device.
Even though we won't be running training, the live script still requires the folder dobble_deck01_cards_57 from the dataset in order to show reference images. Unfortunately there's no way to download a single folder from Kaggle, so we'll have to download the full dataset. It is 201MB, so make sure you have at least that much space on your MaaXBoard by typing
df -h. Edit the dobble_dataset_download.py file to include the kaggle API key you created. Then run it to download the dataset.
You can check to make sure you downloaded the correct dataset by running:
Optional: Set up the camera mount and position your cards
I found this very practical and easy to print camera mount on Thingiverse that works well with the Maaxboard Camera. I'm using it with OpenBeam, so I can slide it up and down to adjust it, but it would also be feasible to use it simply by screwing it to a wooden frame.
Finally, the moment we've all been waiting for. Run the dobble_detect_live script on your MaaXBoard:
And that's it! You should be able to play Dobble using your MaaXBoard!
MaaXBoard SD card image with all of the above work already done for you as a reference solution: MaaXBoard DobbleBuddy SD Card Image
The Dobble Dataset
The Dobble dataset, available on kaggle:
The Dobble Challenge
Getting started with machine learning for the Dobble card game using TensorFlow.
Training The Dobble Challenge
Train a machine learning model that can play Dobble (Spot-It) against you. This is part two of The Dobble Challenge.