The Cough-E Machine

As work-forces return and public areas open up in the wake of the coronavirus pandemic, it is essential that we keep these spaces clean and hygienic so that it can remain a safe environment for everyone.

The Cough-E machine aims to aid and partially automate this process, with a device that can detect when someone has coughed in a certain area or room, advising the relevant personnel that the area needs to be cleaned as a result of this event.

The backbone of this idea is the development of a machine learning model that detects coughs that can also run on the AWS IoT EduKit (M5 Core2) device locally.

Demo Video

Step 1: Collecting Data and training ML Model

This process involved gathering a range of audio clips depicting what coughs sounded like, and contrasting that data with a range of sounds you may hear in a regular indoor environment, such as talking and rustling.

This was performed on Edge Impulse platform which provided AutoML functionalities on a friendly web interface.

Step 2: Exporting ML model to Arduino

Once the model was established, it was exported in the form of an Arduino library, which then was adapted to work on the edge on the M5 Core2, utilising the device’s local resources for the model’s operation.

We needed to ensure that the microphone input was captured in the right format and size for the edge inference to perform correctly. This took a bit of trial and error to get right.

Step 3: Setting Up AWS Back End

We created back end AWS IoT resources, from AWS IoT Core Thing, AWS IoT Topic Rule, DynamoDb, API Gateway, Lambda, S3, CloudFront for the server side processing and web app. This was coded using AWS CDK which allows us to create configurable AWS infrastructure as code (IaC).

Step 4: Sending Data to AWS IoT Cloud

Each device is mapped to an AWS IoT Thing and connects to AWS IoT Core via MQTT. As part of the mutual authentication security, the AWS back end needs to validate that each device is what it claims to be. The AWS IoT Thing certificate and private key specific to each device (so they can be deactivated if a device or credential is compromised) is deployed to the device.

When an event (cough or clean) is detected, the device will push the event to AWS IoT topic which will save the data to the Dynamo DB.

M5 Core2 supports 2.4Ghz WiFi out of the box (other communication module such as LTE, NB-IoT, LoraWan, can be purchased as add-on if required) so we used the built-in WiFi for simplicity.

Step 5: Detecting nearby BLE

The clean event is recorded by the cleaner tapping their BLE card to the M5 Core 2 device. For the quick hackathon demo, we are not being selective and any BLE within -50 RSSI will trigger the clean event.

This is where we struggled a bit with the memory limit of the M5 device as it kept running out of memory when WiFi, MQTT and BLE are running at the same time. The device comes with 8MB PSRAM. However, most of the libraries run on the much faster and smaller SRAM, which is only 520KB on the device.

We ended up switching to a more memory efficient BLE library (NimBLE rather than the standard ArduinoBLE) and quickly releasing connections and memory that are not actively required.

Step 6: Building Web UI for Admin Users

We built a React Web App (using AWS serverless stack) that displays the status of the locations and list the events linked to each location.

Thank You

Thank you for reading our submission. The team had a great time and learned a lot while doing this and will be looking forward to apply some of the new technologies we learned in our workplace.