By performing some post signal processing on the data collected with Walabot (an 18-antenna array radar), I have developed a system for localizing up to 5 different stationary people simultaneously, and also detecting if someone has fallen.
Falls among the elderly population are the leading cause for injuries and hospitalizations. Fall detection has previous been approached through wearable technology and camera-based systems. However, both of these systems have major limitations since wearables can cause discomfort, while the camera-based systems compromise the privacy of the user, require a field-of-view and are costly to set up and maintain.
Walabot is the perfect alternative! It does not require a direct line-of-sight and can be programmed to only detect people, and not other objects such as furniture. Its ultra-wide band (UWB) technology is also very useful as it provides high-speed data transmission.Signal Processing Overview
Walabot is a MIMO radar and the raw data that it returns represents the impulse response and thus, it provides us with a time-of-flight (TOF) profile of each scan. TOF is defined as the time it takes the transmitted wave to reach a reflector and then travel back to the receiver. With this information and the fact that these waves travel at the speed of light, we can calculate the round-trip distance. This round-trip distance localizes the person to an ellipse whose foci are the transmit and receiver antennas. This principle is the foundation used by my system for initially localizing people and then detecting a fall.
Here's a short video that briefly outlines the process of localizing people and detecting a fall:
The system uses two python scripts; one for collecting data and one for making ellipse heatmaps. The heatmaps are an essential part of signal processing because they use the raw signals obtained from Walabot to show areas with strong signal intensity, which indicate the presence of a person.
Vertical plane heatmaps represent the yz-plane, while the horizontal plane heatmaps represent the zx-plane. The intersection point of these two planes is determined to localize a person. A person would be located some z-distance from the Walabot and that z-vlaue would be the same in heatmpas for both planes. One of the vertical plane heatmaps is used to find this z-value. Afterwards, the corresponding x-value is found by using one of the horizontal plane heatmaps.Step 1: Data Collection with Walabot
The python script 'DataCollection' is used to collect data. To ensure that background reflections from furniture do not clutter the obtained signals, an initial calibration is performed before people enter the room. The area in front of the Walabot should be clear of all moving objects or people when the calibration is performed. A message is displayed in the terminal once the calibration is complete.
Once calibration is complete, the data from each frame is written to a csv file named ‘raw_data’. The frame number is also displayed in the terminal. It is important to take note of two specific frame numbers: the first frame is where all people have been standing in the range for ~10s and the other one is when someone has fallen. The Walabot requires some time to stabilize the signals it receives from the people and therefore, the frame where all people are detected are usually 10s after people enter the range.Step 2: Making the Heatmaps
Here, the python script 'Heatmap' is used for making the ellipse heatmaps. This script reads the data from the 'raw_data' csv file and hence, it should be in the same directory as this script. There are 4 parameters that need to be specified: i) the frame number for the stabilized signals, ii) the antenna pair, the iii) the upper limit of range and iv) the lower range limit. The range limit is set by the number of points to be used; each scan from Walabot provides 8192 data points for each frame, which corresponds to a distance of ~12m. Depending on how many data points are used, the range for heatmaps can be manipulated easily. Initially, the lower limit of the range should be set to 0; this is modified later when a section of the heatmap is required to be removed once a person is localized.
Since we will be comparing data from the horizontal and the vertical planes, we need to first make the heatmaps for these planes. In total, 9 heatmaps will be required; the antenna pairs that provide data for the horizontal plane are 1, 2, 9 25, and 26 and the antenna pairs for the vertical plane are 7, 14, 32, and 34. See the images below to see the location of the antenna pairs on the Walabot board.
i) Optimal Horizontal Plane Heatmap:
Examine the maximum signal intensity value of the signals by looking at the color-bar, and choose the heatmap of the antenna pair with the highest intensity value.
ii) Optimal Vertical Plane Heatmap:
To determine if a person is in the left vertical plane or in the right vertical plane, the intensity values for signal strengths are examined. The following flowchart outlines the steps for the two different cases.
Now that we know which heatmaps to use, we can look at their intersection points to localize the first person. The flowchart given below organizes the steps for three possible cases.
The heatmaps below show an example of how to find the max intensity z-value from the vertical plane and how to find its corresponding x-value in the horizontal plane. In this case, the optimal vertical plane was made with data from antenna pair 32, thus, the right side of the heatmap is considered when looking for the corresponding x-value.
The person can now be localized using the z- and the x-values; be sure to record the signal intensity value of the person from the chosen horizontal plane.Step 6: Remove Localized Person from Heatmap
One of the biggest challenges for localizing people is the 'Near-far Problem'. People at different distances from the radar have different intensities; thus, people farther away often don’t show up in the heatmap. To overcome this issue, the section of the heatmap corresponding to the person closest to Walabot should be removed, which can be done by updating the lower limit of the heatmap range:
An example of a heatmap before and after the first localized person is removed:
Repeat Steps 2-6 until all people in the range are localized and don't forget to keep track of every person's signal intensities. Note that in step 6, a section of the heatmap wil need to be removed every time a person is localized.Step 8: Fall Detection
Walabot is able to detect the changes in signal intensities due to the occurrence of a fall and it usually does so ~2-3 frames after someone has fallen. Therefore, to detect a fall, the acquired signal intensity values for the localized people should be compared to the new data every 3 frames. Once again, a heatmap of the new frame is required to perform some signal analysis. The detailed steps are given below: