Published September 12, 2022 © CC BY

Early detection of workers falls with Machine Learning

Fall detection prototype with Arduino clients, Raspberry Pi server and Machine Learning.

IntermediateFull instructions provided8 hours2,103

Early detection of workers falls with Machine Learning

Things used in this project

Hardware components

Arduino Nano 33 BLE Sense

Raspberry Pi 3 Model B

Tp4056 charger

Rechargeable Battery, 3.7 V

Software apps and online services

Arduino IDE

Autodesk Fusion

Edge Impulse Studio

Story

“I'm going, down, down, down, down, down. Yes, I've got my feet in the window. Got my head on the ground” (Freddie King)

Client server prototype to detect falls with Machine Learning

Falls could be dangerous in any situation but for certain working scenarios, consequences are definitely worst, therefore came the idea of developing a Machine Learning fall detection/report system.

Arduino 3d printed case

Each worker will have a small TinyML device in charge of detecting falls with accelerometer data and reporting to a server through Bluetooth. The server will be a Raspberry Pi running a Python script that scans specific BT announcements, parse fall information and store it into a SQL Lite database for reports and alerts.

Client device

Electronics part of the client is easy: just a battery, a TP4056 and the Arduino Nano BLE 33 Sense. The board has an on board accelerometer and enough processor to run Edge Impulse library.

Battery and TP4056

. I have installed ArduinoBLE library in Arduino IDE from Sketch, Include Libraries, Manage Libraries, “ArduinoBLE”

. I have dowloaded Edge Impulse Firmware from this link https://cdn.edgeimpulse.com/firmware/arduino-nano-33-ble-sense.zip Unzip, connect Arduino BLE with microUSB cable, double click Reset button and run flash_window.bat (or flash_mac.command or Linux)

Flashing Edge Impulse firmware

. From Edge Impulse Data Acquisition, I have selected WebUSB and Inertial sensor. I have obtained Stand and Fall samples. Then those samples were splitted and 20% were assigned to test category.

Edge Impulse Data Explorer

. I have designed an Impulse with 1500ms window size, 150ms window increase, 100HZ frequency. Spectral Analysis with just 3 axis accx, accy, accz. Keras classification and 2 output features: Stand and Fall. For Neural Network, 50 training cycles with 0.0005 learning rate, Autobalance dataset and 20% validation worked fine.

Impulse design

After model testing I have exported the Arduino Library with the Machine Learning Model. I have imported this library Zip file from Sketch, Include, Add Zip.

Model testing

The library has an example ready to send accelerometer data to the Machine Learning model and obtain inferences. I have used the continuous mode example and added the functions to advertise the fall with Blutetooth.

Every fall is advertised with this format:

advertiseFall("Fall-"+worker+"-"+String(myCounter));

Example: Fall-Smith-1922

Client 3d printed case

3d slicer

You have to 3d print 2 parts with PLA, no support, attach a strap and you are ready to go.

To download 3d files of Fall Detection system please check this link

Server device

I have downloaded Raspberry Pi Imager from https://www.raspberrypi.com/software/

Then I have burned an Image of Raspberry PI OS Lite – no desktop is required - to a microSD card

I have inserted the microSD to a Raspberry Pi 3 and completed the setup (entering user and password)

Client device

Then I have executed raspi-config

$ sudo raspi-config

and enabled SSH and entered WiFi Credentials.

I have installed this requisites:

$ sudo apt install bluetooth libbluetooth-dev
$ sudo apt-get install python3-pip
$ pip3 install pybluez
$ sudo apt-get install libbluetooth-dev bluez bluez-hcidump  libboost-python-dev libboost-thread-dev libglib2.0-dev
$ sudo pip3 install gattlib
$ sudo apt-get install bluetooth libbluetooth-dev
$ sudo python3 -m pip install pybluez
$ sudo pip3 install termcolor

I have downloaded all the Python scripts to the Raspberry Pi. You can do that with FTP, GIT Clone and even $ sudo nano and copy file contents.

I have executed databaseSetup.py to create SQL Lite table.

CREATE TABLE FALL (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, Timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, name TEXT, worker TEXT)

And with everything ready, I have executed the scan script:

$ sudo python3 scan.py

Scan script screenshot

Server scan script

Other scripts included are: clearDatabase.py (removes all records) chart.py (creates a chart with database records)

Server database browsing

To auto-execute the script at start a cronjob could be added

$ sudo crontab -e
@reboot sudo python /home/pi/scan.py /home/pi/scanLog.txt

A script to create charts is also included. Below is the exported chart jpg

Chart

Being a prototype, security was not considered but of course you should for a production/working scenario. Use a hashed key before considering the BT advertise as authentic. Then, filter for SQL Injection before any database query. Sample code has been included inside the scan script.