Emotion and Navigation Wearable Device with Spresense

The story

I'm an engineer and right now I'm finishing my master degree. As a part of my thesis I started to work with emotions and that motivated me to start a project where bike riding could be manage by emotions.

Have you ever feel angry or sad? what if you ride your bike and the system let you to choose a route and a destination to improve your feelings or to feel as you want?

I'm a big fan of ride my bike to go everywhere, to my work, to my university and every place I can, but also I'm a big fan of the great views when you're riding. Here is where I got the idea of an autonomous navigation system with the ability of change your route based in the emotions you would like to feel.

For example: I live in a city with a river and It makes me feel happy and motivated, so If I feel sad, then the system should recommend me to go through this way. But what if I started my ride and in the middle of the way I get stressed? then the system should recommend me a ride with a views like the river.

Here, there is a problem that I detected, If I have to pay attention to the view and the spots, It is necessary to have an autonomous navigation system. For cars exist many options, but for bike there are few system available. So, the first part of the project consist in create an autonomous and free hands navigation system for bikes. The second part consist in create a ML system to identify emotions an run it in a mobile device and the third part of the project consist in integrate everything with a navigation system based on google maps.

Yes! until here is a big project and I'm finishing a part of it for my degree thesis and other as a hobby. Although I know it is a big project, but with every free minute between study and work, I work in this project.

Process (What I have Done)

For the moment I'm working in two parts: voice direction system for bike and the ML emotion recognition.

For the ML recognition system I used an Empatica E4 wristband (https://www.empatica.com/en-int/research/e4/) to get the biosignals from many situation to tag some emotions. The theory says that It is possible to determine your emotion vector between 4 quadrants, as a vector in a plane. This model is referred as Arousal-Valence Emotion plane and getting some biosignals data from different emotions let me to train a classification system.

In this moment I'm training my model with Edge Impulse to run that model in a mobile, but I'm still improving the testing the training data. So, I'm expecting to continue uploading the progress in that part.

In the other hand, I'm working in free hands, voice navigation system for my bike ride the the Sony's Board Spresense. I'm using the advantages of the board sound management to run a ML system to say turn the lights if you say right, left or stop.

That system is the first part of the whole project and here is the progress.

2) Free hands voice navigation system

The first time I started to test the mics, I bought two generic mics in a local store, but these mics were so bad and they didn't work. So after many attemps, I had to buy new mics based on sony recommendation. I bought two on mouser ( https://www.mouser.es/ProductDetail/CUI-Devices/CMC-5044PF-A?qs=WyjlAZoYn52uDuCfafch4A%3D%3D ) I say that, just in case you want to test generic mics, but keep in mind that the local store didn't get me any datasheet or something like that.

By other hand, I started to train a ML model to identify the following orders: "left", "right", "stop" and everything else that do not correspond with my desired instructions. I followed the Edge Impulse example of train a ML system to react to one phrase like "ok google" or "hello Siri". But I will post it in other tutorial to make this shorter.

For the moment, I'm still programming, waiting for new mics and expecting to finish my thesis and this project. So, if I have new progress I will post it here. :D

I already worked with neopixel leds and I posted the miniproject here:

https://www.hackster.io/djsilverav/ws2812b-and-spresense-86b414

Thanks for the opportunity in the contest and I would like to post something to show that I tried to work as much as I could. But as many projects I sized so bad the goals, I got problems and I'm still trying to solve it. But don't worry, I'm still working on it.