Ever worried whether or not you brushed your teeth thoroughly enough? Have you ever wandered off with your thoughts and forgot about the time, suddenly being uncertain if you've been brushing for 30 seconds or 5 minutes? And have you ever forgotten whether or not you brushed your teeth last night? Worry no more, the tooth fairy's got you covered. With this combination of an app for your phone and code for your Arduino Nano 33 BLE Sense you can now be worry free too. This setup uses machine learning to detect whether you are brushing your teeth and starts counting the time. And after you're done it even remembers that you did it!Project Origins and Goals
The idea for this project got realized as part of a school project for our Internet of Things university class. The goal was to create an IoT product that utilized machine learning on an Arduino Nano 33 BLE Sense which would then feed the data to a user terminal via Bluetooth. Of course we also created an Android app so that the recieved data could be displayed in a neat and user-friendly manner.
The project as such is split into three larger tasks:
- Creating the machine learning model
- Creating the mobile phone app
- Enabling Arduino's Bluetooth connectivity
We started our work by gathering data that could then be fed to a machine learning algorithm. The website Edge Impulse has everything one could wish for when making such projects with many handy tools. One of them is a really simple way of gathering the necessary data. All one has to do is to simply connect their mobile device and any recordings of its sensory information can be uploaded directly into the ML project on the website. At the end we managed to record data with a time record of two hours in different situations, from brushing with the water flowing in the sink or shower to brushing while the washing machine is on.
The creation of the machine learning model was then a relatively simple task. After choosing a processing and a learning block of an impulse we were able to train and test the model. In the end its prediction accuracy reached about 80%. The details of the model's training and testing can be seen on the bottom two images.
After the model was built all we had left to do was to export it onto our Arduino devices. As with everything else on Edge Impulse, that process too was straightforward and simple. After deciding whether or not to use optimizations (which we did as it only reduced the accuracy to 70% which we believe is still acceptable while improving recource usage to a huge degree) we let the website build a.zip library with the click of a button.
The built ML model library is now fully ready to be implemented into Arduino IDE.Arduino Firmware
Creating a program for the model to run on an Arduino board is by itself fairly easy as well. The procedure is to simply import the.zip library built by Edge Impulse, open the microphone_continuous example from the examples list, and upload it to your device. In its default state however, you can only use it to send data to your computer via the USB cable and IDE's serial monitor. As one of the main project's goals is being able to stream data to a mobile device, we had to do better. To implement BLE connectivity we decided to use code from another IDE's example: battery monitor. While requiring a bit more programming know-how it was still a relatively simple process of copying and pasting appropriate lines of code to the right places and then editing a few of strings. The final thing we did was adjusting the output of the predictions so that the board would send a 1 when it detected teeth brushing and 0 when it wouldn't. The full program is listed under the Attachments section but on the images bellow there are highlights of the changes done to the original Edge Impulse code.
The basic functionality of the application that we wanted to achieve was for it to be able measure the amount of time a person would be brushing their teeth as well as storing the date and time of the last brushing. As the project developed we started adding additional features: settings, where the user could specify their name and other details for personalization, the ability to write down the date of the last visit of the dentist as well as the date of the last toothbrush replacement and more. The application's user interface is shown on the images below:
As mentioned, we have created the code in MIT App Inventor. As we progressed through our project the application's code rose in size dramatically. That is why we will only provide one piece of the code as an example but if you are interested in the inner workings of the app we invite you to download the provided MIT Inventor project file and start exploring and experimenting yourself.