Early detection of workers falls with Machine Learning

“I'm going, down, down, down, down, down. Yes, I've got my feet in the window. Got my head on the ground” (Freddie King)

Falls could be dangerous in any situation but for certain working scenarios, consequences are definitely worst, therefore came the idea of developing a Machine Learning fall detection/report system.

Each worker will have a small TinyML device in charge of detecting falls with accelerometer data and reporting to a server through Bluetooth. The server will be a Raspberry Pi running a Python script that scans specific BT announcements, parse fall information and store it into a SQL Lite database for reports and alerts.

Client device

Electronics part of the client is easy: just a battery, a TP4056 and the Arduino Nano BLE 33 Sense. The board has an on board accelerometer and enough processor to run Edge Impulse library.

. I have installed ArduinoBLE library in Arduino IDE from Sketch, Include Libraries, Manage Libraries, “ArduinoBLE”

. I have dowloaded Edge Impulse Firmware from this link https://cdn.edgeimpulse.com/firmware/arduino-nano-33-ble-sense.zip Unzip, connect Arduino BLE with microUSB cable, double click Reset button and run flash_window.bat (or flash_mac.command or Linux)

. From Edge Impulse Data Acquisition, I have selected WebUSB and Inertial sensor. I have obtained Stand and Fall samples. Then those samples were splitted and 20% were assigned to test category.

. I have designed an Impulse with 1500ms window size, 150ms window increase, 100HZ frequency. Spectral Analysis with just 3 axis accx, accy, accz. Keras classification and 2 output features: Stand and Fall. For Neural Network, 50 training cycles with 0.0005 learning rate, Autobalance dataset and 20% validation worked fine.

After model testing I have exported the Arduino Library with the Machine Learning Model. I have imported this library Zip file from Sketch, Include, Add Zip.

The library has an example ready to send accelerometer data to the Machine Learning model and obtain inferences. I have used the continuous mode example and added the functions to advertise the fall with Blutetooth.

Every fall is advertised with this format:

advertiseFall("Fall-"+worker+"-"+String(myCounter));

Example: Fall-Smith-1922

Client 3d printed case

You have to 3d print 2 parts with PLA, no support, attach a strap and you are ready to go.

To download 3d files of Fall Detection system please check this link

Server device

I have downloaded Raspberry Pi Imager from https://www.raspberrypi.com/software/

Then I have burned an Image of Raspberry PI OS Lite – no desktop is required - to a microSD card

I have inserted the microSD to a Raspberry Pi 3 and completed the setup (entering user and password)

Then I have executed raspi-config

$ sudo raspi-config

and enabled SSH and entered WiFi Credentials.

I have installed this requisites:

$ sudo apt install bluetooth libbluetooth-dev
$ sudo apt-get install python3-pip
$ pip3 install pybluez
$ sudo apt-get install libbluetooth-dev bluez bluez-hcidump  libboost-python-dev libboost-thread-dev libglib2.0-dev
$ sudo pip3 install gattlib
$ sudo apt-get install bluetooth libbluetooth-dev
$ sudo python3 -m pip install pybluez
$ sudo pip3 install termcolor

I have downloaded all the Python scripts to the Raspberry Pi. You can do that with FTP, GIT Clone and even $ sudo nano and copy file contents.

I have executed databaseSetup.py to create SQL Lite table.

CREATE TABLE FALL (id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, Timestamp DATETIME DEFAULT CURRENT_TIMESTAMP, name TEXT, worker TEXT)

And with everything ready, I have executed the scan script:

$ sudo python3 scan.py

Scan script screenshot

Other scripts included are: clearDatabase.py (removes all records) chart.py (creates a chart with database records)

To auto-execute the script at start a cronjob could be added

$ sudo crontab -e
@reboot sudo python /home/pi/scan.py /home/pi/scanLog.txt

A script to create charts is also included. Below is the exported chart jpg

Being a prototype, security was not considered but of course you should for a production/working scenario. Use a hashed key before considering the BT advertise as authentic. Then, filter for SQL Injection before any database query. Sample code has been included inside the scan script.

Demo

You can fast forward to the actual demo or view the entire episode with English captions

Important links for this project

Sign up for Edge Impulse Machine Learning platform (free for developers)

Download Raspberry Pi OS image

Download enclosure 3d files

Download client and server files

Interested in other Machine Learning projects?

• LRAD triggered by Computer Vision
• YouTube Playlist with Machine Learning Projects for Arduino and Raspberry

TedxTalk about Arduino and Raspberry inventions

Social Media

Are you a maker/engineer/tech/art enthusiast? Let's get in contact

https://www.instagram.com/ronibandini/

https://twitter.com/RoniBandini