Movement Recognition with Tiny ML

Story

I've been more interested in Machine Learning for Embedded Devices in recent years. I have taken some courses, read several articles, and tested some frameworks with development boards.

Now, I know one promising ML framework called Neuton IA. The Neuton platform enables users to use the system to predict various outcomes based on statistics. This is made possible by processing the user data and using it to train a machine learning model. It has easy steps for training and deploying a model in a microcontroller.

This article will present the implementation of a movement recognition application.

Select dataset for training

The data used in this project was collected from the Arduino Nano 33 board via USART. An application was developed to read the data from the IMU sensor and print it in CSV format.

The dataset has four different target(model output) variable values.

• Idle - 0
• Up And Down - 1
• Left and Right - 2
• Rotate - 3

There are 600 accelerometer and gyroscope readings per movement, with a sampling rate of 120Hz, resulting in a monitoring window of 5 seconds.

Train your model

After collecting the data, it's time to train the Machine Learning model.

In Neuton configure the task type as multiclass classification. Multiclass classification is used to predict one value of a set of possible outputs. For example in this case predict one movement of four possible movements.

For the model metric, configure Accuracy.

Enable the DSP Option for automatic processing and feature extraction from gyroscope and accelerometer data. This will help detect the movement better, as each movement has its frequency and variance in the data.

After the model is trained, you can visualize the metrics and evaluate if it's good for your application.

Accuracy represents how accurately class is predicted. If from 100 predicted records 73 have been assigned a correct class, then the accuracy will be 0.73 or 73%. The higher the value is, the better.

Balanced Accuracy accounts for imbalanced classes where (for example) one class may be represented by 10% of samples, a second class may be represented by 60% of samples and the other N classes are represented by the remaining 30% of samples.

Precision is the fraction of relevant samples among the retrieved samples. The precision score reaches its best value at 1 and the worst score at 0. Precision is the ability of the classifier not to label as positive a sample that is negative. Use this metric when the priority is to find only relevant samples without mistake even if you may skip some of the relevant samples.

Deploy and Make predictions

You can download the C library from the Neuton platform and embed it on your firmware project.

Steps:

• Copy all files to your project
• Include "neuton.h" on your application file

#include "neuton.h"

• Set the data input for your model and predict the result

Use the same data format you used for training the model.

In this case, 600 accelerometer and gyroscope measures

#define NUM_SAMPLES 600

typedef struct {
float acc_x;
float acc_y;
float acc_z;
float gyro_x;
float gyro_y;
float gyro_z;
} vector_t;

vector_t data[NUM_SAMPLES];

.
.
.
/* Collect the data */
.
.
.

if(i == NUM_SAMPLES) {
    /* Set model input */
    if (neuton_model_set_inputs((input_t*)data) == 0) {
        uint16_t predictedClass;
        float *probabilities;
        /* Run inference */
        if (neuton_model_run_inference(&predictedClass, &probabilities) == 0)
        {
            // code for handling prediction result
            Serial.print("Detected gesture: ");
            if(predictedClass == 0) {
                Serial.println("Idle");
            } else if(predictedClass == 1) {
                Serial.println("Up and down");
            } else if(predictedClass == 2) {
                Serial.println("Left and right");
            } else {
                Serial.println("Rotate");
            }
            Serial.print("Probabilities: ");
            for (int j = 0; j < neuton_model_outputs_count(); j++)
            {
                Serial.print(probabilities[j]);
                Serial.print(" ");
            }
            Serial.println();
        }
    }
}

Connect your board to serial monitor software and test your model!

Detected gesture: Idle
Probabilities: 0.95 0.05 0.00 0.00 
Detected gesture: Idle
Probabilities: 0.99 0.01 0.00 0.00 
Detected gesture: Idle
Probabilities: 0.46 0.45 0.09 0.00 
Detected gesture: Up and down
Probabilities: 0.00 0.89 0.11 0.00 
Detected gesture: Idle
Probabilities: 0.86 0.14 0.00 0.00 
Detected gesture: Up and down
Probabilities: 0.00 0.66 0.34 0.00 
Detected gesture: Up and down
Probabilities: 0.00 0.50 0.50 0.00 
Detected gesture: Rotate
Probabilities: 0.00 0.01 0.00 0.99 
Detected gesture: Idle
Probabilities: 1.00 0.00 0.00 0.00

Have fun!