Detect and monitor the states of a 3D printer with TinyML

A model trained on Edge Impulse Studio and deployed with the RAKwireless WisBlock to detect the states of an Ender 3 v2 printer. The inferences are shared over BLE to a Blues Wireless Cellular Notecard connected to a Raspberry Pi. Then the Notecard sends the states to the cloud.

WisBlock Hardware Setup

First, connect the RAK4631 WisBlock Core to the Base Board (if not connected yet), and ensure its connection with the tiny compatible screws. Then, you need to connect the BLE and LoRa antennas. We will not cover the LoRa connectivity in this tutorial. Still, using it while using the module is recommended to avoid any RF damage to the transceiver on it.

One of the most relevant components for the model we will train is the accelerometer to detect the printer's states. So, place it in one of the slots on the back side of the Base Board, and secure it with a screw. Then, grab the 3D-printed case, put everything together, and connect the USB cable to your computer.

After that, this tiny guy will be ready to be flashed with our first firmware!

Preparing the WisBlock to collect motion data

Open WisToolBox and click on CONNECT DEVICE. Wait until the module is detected and press CONNECT.

Navigate to the FIRMWARE tab within the Dashboard view and use the UI options to upload the RAK4631_DataForwarder.ino.zip firmware package.

When the firmware update process is successful, you can go to the ADVANCED tab and open the console available in WisToolBox to view the serial logs. The serial logs for this firmware are the acceleration values for the three accelerometer axes. At this point, you can move the device to check the values vary accordingly.

Lastly, you will need to disconnect the WisBlock from WisToolBox because we will need the serial port free for the following steps.

Creating the project on Edge Impulse Studio

Go to Edge Impulse Studio and login into your account. If it's your first time, you can create an account to start building your first model.

After you log in, create a new project. I called this one Monitoring 3D Printer States. Specify the Accelerometer data type from the list shown when the project is created. Then, you can close the onboarding wizard and navigate to the Devices section. Our WisBlock will be listed there in a few minutes!

Running the data forwarder

You need to install the Edge Impulse CLI to use the data forwarder. When you finish the installation, open a new terminal window in your computer and start the data forwarder using the following command:

edge-impulse-data-forwarder

It will prompt for your credentials and to select the project to connect the device connected. When you see the projects listed, press enter to choose the Monitoring 3D Printer States we created in the previous steps.

The data forwarder tool will detect the data frequency and request the names of the values. You can enter accX, accY, and accZ to identify the values read from the accelerometer. Then, you can press enter to use the address shown as the device name. Lastly, you will get a confirmation that the device is connected to the project.

Return to the Edge Impulse Studio and check the device is now listed in the Devices section. The address is also used as the ID. We will need this address later when connecting the WisBlock and the Raspberry Pi over BLE. I will refer to it as wisblock_ble_address.

Collecting data to train the model

Go to the Data acquisition section, check the device is on the list, and enter a label for the data to collect. The goal is to collect several samples for the states we want to detect. For example, I defined three states for my model Off, Standby, and Printing.

I will not go deep into the impulse design because that will depend highly on the printer to monitor and how the hardware to measure the motion is mounted. However, I definitely recommend you review the Continuous motion recognition tutorial later if you haven't done so.

The good news is you can continue with this tutorial because I prepared another firmware package with my model deployed. The source code of the package is available here, so you can freely use it as a reference to deploy a model to monitor your own 3D printer or any other use case it comes to your mind. Furthermore, the studio project is also public, so you can access it here.

Model deployment

You are already familiar with the process of updating the WisBlock firmware. Let's use WisToolBox this time to flash the Monitoring_3D_Printer_Impulse_Deployment.ino.zip package.

The console logs will show more information about the inferencing process and the corresponding results.

When the firmware runs, the WisBlock will be available to receive BLE connections from a BLE host. When the connection with the host is established, it will share a state based on the inferencing results as a string in the following format +EVT:EI_Inference,Ender_3,{state},{value}. You can inspect the source code to better understand what's happening.

Notecard + Raspberry Pi Setup

Several of the things I will quickly show are well documented in the Blues Wireless documentation.

Connecting the Notecard to the Raspberry Pi using the Notecarrier Pi is a really straightforward process.

The interface between the Notecard and the Raspberry Pi is over I2C. So I installed an image of Raspberry Pi OS using Raspberry Pi Imager. Then I logged in to the system, enabled the I2C, and started to play with the Notecard following the Quickstart guide.

Notehub project creation

Go to the Blues Wireless Notehub, log into your account, or create an account if it's your first use. Then, create a new project and copy the ProductUID.

Interfacing the WisBlock with the Notecard

The python script I wrote to interface with the WisBlock and send the data to the Notehub is also available in the repo. You can copy its content to a new file in the Raspberry Pi. Before executing the script, you must install some packages using pip.

• note-python The Python API for the Notecard
• bluepy The Python interface to Bluetooth LE on Linux

After installing the packages, set the productUID and ble_address with your values. You will find it in the first lines of the script.

When you are ready, run the script with the command

python send_data.py

The script will establish a connection with the Notecard, then set the parameters to communicate with the Notehub and try to connect with the WisBlock over BLE to get the string with the state reported. Some execution logs will be printed in the terminal. If this process is successful, you go back to the Notehub and check a new device is listed in the Devices section.

Routing the 3D printer states to Ubidots

Click on the routes section, and let's start routing the states we are getting in the Notehub to Ubidots.

You need to create a new Device that I call 3Dprinter States here. Then go back to the Routes section in the Notehub, click on Create Route, select the first option General HTTP/HTTPS Request/Response, and enter a name.

Copy and paste the URL https://industrial.api.ubidots.com/api/v1.6/devices/sensor-data/. Add at the end of the URL the device API label. Then, copy your API token and paste it as the value of an additional header named X-Auth-Token. Then, include the data.qo Finally, select JSONata Expression to transform the data and include the timestamp by entering the following lines:

{

"Off": {"value": body.Off, "timestamp": when * 1000},
"Printing": {"value": body.Printing, "timestamp": when * 1000},
"Standby": {"value": body.Standby, "timestamp": when * 1000}

}

The image below shows a dashboard to visualize the data reported.

Some ideas to go deeper

While preparing this guide, a few ideas that can be useful to you to expand some of the things I cover here to other scenarios came to mind.

For instance, you could use some elements of this architecture to monitor other machines that also move or vibrate. In that case, the Raspberry Pi could host several sensing devices placed on those machines and still receive their data wirelessly over BLE.

One more example could be those cases where there is no access to WiFi or even a wired internet connection. The cellular module used in the guide could continue transmitting the states of the machines if there's coverage for any cellular operator.

Lastly, the hardware I used for sensing and running the ML model is also ready to transmit over LoRa, so you eventually could replace the Raspberry Pi and use a LoRaWAN network to transmit the data.

Don't forget the comments section is available to ask questions, report issues, or express your ideas.

Thanks for reading! 😊