Software apps and online services
Hand tools and fabrication machines
The entire world is going through troubled times, with millions of lives lost to the current pandemic. Not to mention the huge amounts of money invested in curtailing the spread of this pandemic and tracking it. While the new vaccines will soon end this pandemic, humanity has collectively realized the need to buckle up its healthcare facility. We need to change how we perceive healthcare monitoring as something required only after the symptoms are visible and not before. However, healthcare monitoring is not cheap and most health monitors measure very few vital stats which does not cover the overall health of a person. Hence, we felt the need to develop a device that measures 4 vital parameters of a person -
- Body Temperature (BT)
- SPO2 levels
- Heart Rate (HR)
- Heart Rate Variability (HRV)
All these parameters are vital in showing the health of our respiratory and cardiac machinery and together, they can be used to detect many symptoms or conditions like -
- Hypoxemia (Low SPO2 levels)
- Arrythmia (Irregular Heart Beats)
- Atrial Fibrillation (High Heart beat and High HRV)
- Fever (High body temperature)
- ... and many more like anxiety, insomnia, etc.
These conditions in turn, when coupled with each other can help us idetify the probability of the subject having a particular disease of even him being susceptible to one based on his/her vitals.
Another problem faced during patient monitoring is carelessness. When we perform a certain activity on regular basis, we start exhibiting a certain amount of carelessness towards it. It is important that we show a very quick response to any potential health risk scenario. For this very reason, I have used the Neosensory buzz in this project. While it is very difficult to notice minor changes in the vitals using just vibrations, It is more than enough to get a rough idea of the value of each vital and make the doctor or overseer alert in case of any emergency.Solution:
Our Solution is very simple. We will develop a low-cost device, that can be kept at any location to measure the aforementioned vital stats. This device can be easily reused by multiple people and since it uses IR rays it might even work on fingers covered by a thin glove to some extent. A continuous stream of people will keep measuring their vitals at the entry point of the facility, which will be stored in an online server accessible from anywhere by authorized personnel. The device will show the subject his/her vital stats as well as help a doctor feel this data in the form of vibrations. This will help in privacy protection might be anywhere in the world. The doctor will be notified of the patient's vitals in the form of vibrations and the location of measurement. In case of any alert, the doctor can then contact the facility and take further steps to assess the subject directly. Using edge impulse, we have developed multiple models to categorize each signal received for a particular vital stat into high, normal and low as required. These models feed the labels to our app, and the app user can see this data as well as the labels from each model. The app provides a connection to the buzz and maps the vital stats into different vibrations for each of the 4 motors on the Neosensory Buzz.
We have used multiple components as stated at the beginning of the project. This is how we assembled the entire system:
As you can clearly see, the device was developed using the Node MCU controller. It first provides an open Wifi, connecting to which you can proceed to 192.168.4.1 and enter your wifi details. These details are saved in EEPROM and reused every time the device boots up. The device measures SPO2 and HRV continuously, while HR and body temperature are measured over an interval, which can be changed. The device then sends this data to the Heroku server at regular intervals. For details pertaining to the server-side or app side development, refer to the ReadMe page of the attached Github repository. The Device can be seen in action here -
Next we come to the Edge Impulse development.Data Collection
We collected data using the edge impulse data forwarder, by making people wear the client device while exercising, deep breathing, etc. We also used hot objects to create high-temperature data. We compared all this data with ground truths, like a mercury thermometer and a commercial pulse oximeter. We then used all this data to train our edge impulse model. All this data was collected after talking to doctors and physicians so that the data is as reliable and practical as possible. We then increased the size of the data by modifying the actual values and duplicating them. This gave us a lot more useful data to train our model on.Edge Impulse
Edge impulse is a wonderful way of making light classification models using signal processing. We used it to develop 4 models:
- HRV model (High, medium, low)
- SPO2 model (Normal and low)
- HR model (High, normal, low)
- Temperature (High and Normal)
These accounted for 10 labels and 4 separate models running together. The labels might have different names at a few places, eg. high spo2 instead of normal spo2 but it does not really make any difference.
Here are the images from our edge impulse training:
As you can see, we first tried combining all labels into a single model, but a person could have multiple labels together (eg. low spo2, low HR and high temp). Hence we developed 4 separate models, 1 for each vital. The images show data visualization for all of them. The training results are also shown for a few models. The edge impulse dashboard makes it super easy to develop such kinds of mini models that can be deployed on the edge or on the server-side. I went for server deployment as it was easier and faster. I used a simple NN architecture as it was sufficient for our classification and I wanted to keep my model as simple as possible.App development and Neosensory Buzz integration
The Neosensory buzz is a truly wonderful device, I especially like its app and music mode. Here's me with it and my app!
I developed a simple app interface for the project, that took data from the Heroku backend and connected with the Neosensory Buzz. The Buzz-Beat App has been made using Android Studio with Java as the backend language. The app works in two stages, the first stage is like a landing page that is displayed when the app is opened, the second is displayed after the connection to the buzz is established when it is brought near the phone.
For the connection between the android app and Neo Sensory buzz, the SDK is built on top of the Bluetooth blessed library which is a light-weight low energy-based Bluetooth utility that made a lot of heavy work easy. Once connected, the app works with GET request on the custom node.js server, a fter every 1 second interval a GET request is processed and the JSON at server is collected and displayed as text-view onto the App's screen. The value received is somewhat like this:
Out of these, the first four are the data that needs to be sent to the buzz on the hand of the doctor/individual wearing it. The values are mapped using a custom range mapping formula that is explained excellently on this StackOverflow answer. In basic terms, the vitals are mapped to 40-255, which is the range for the Neo Sensory buzz motors. Finally, these values are sent to the buzz via Bluetooth, in realtime, they are updated and sent over in quick intervals and the insight from the Edge Impulse model is also printed on the app.
The Heroku backend receives data as follows:
All this as you can see happens in real-time, with timestamps. We have tested this system practically as seen in the following video -Conclusion and Roadmap
I hope that this project will make everyday healthcare more affordable. With the hardware cost excluding the buzz costing only around 15$, this project has a very high ROI and utility. It provides good quality healthcare monitoring and alerts the necessary authorities to take action if a subject's condition is deteriorating. I am looking forward to collecting data on patients suffering from heart and lung diseases and try to diagnose the early onset of the disease using edge impulse.
As is visible, I am almost done with prototyping and research. I am still trying to figure out a few problems like delayed classification by the edge impulse models that are displayed on the app, erroneous sensor readings, etc. However, none of them are major problems and they should be solved by Mid Feb. I will then approach some authorities in my country and abroad which will help me get permissions for actual patient data. Using this permission, I will be able to actually verify if my system can identify if a patient has arrhythmia or hypoxia (for example). This will help me get credibility and data that will be permissible to get CE and FDA certifications. I plan to launch my product by April this year. I am looking forward to developing this project to its full potential!