Software apps and online services
Cheering Watch" to keep an eye on your motions
Getting up in the morning is hard! You live alone, but you need someone to cheer you up! It happens sometimes, doesn't it?
We have created "Cheering Watch," a device that uses the M5StickV's built-in acceleration sensor to estimate your motions and cheer you up according to your motions.
Cheering Watch not only shows you what time it is, but also adds "cheering" to your daily life.
My name is Cheering watch!!Imaging acceleration data
The acceleration data acquired from the IMU of the M5StickV is converted into an image. This conversion converts the acceleration data into 8x8 RGB format image data, where X-axis = R, Y-axis = G, and Z-axis = B, so that the data advances 8 dots in the X direction and then 8 dots in the Y direction.
When motion is detected by the acceleration sensor, a moiré pattern appears. The moiré pattern depends on the number of vertical and horizontal pixels in the image, the sampling rate of the sensor, and the periodicity of the hand motion.
The data output from the MPU needs to be converted because the minus is a complement.
accel = i2c.readfrom_mem(MPU6886_ADDRESS, MPU6886_ACCEL_XOUT_H, 6)
accel_x = (accel<<8|accel)
accel_y = (accel<<8|accel)
accel_z = (accel<<8|accel)
With M5StickV, save more and more motion data in image data while moving in various ways.
Let's try various ways to move. The resulting image data will be used to train classifications in TensofFlow/Keras.
What you have to be careful about is that the location and orientation of the installation should be fixed when you are learning.
The images of the acceleration data, obtained from the M5StickV and your movements, should be placed in a folder with a different name for each activity.
The more data you take, the better the accuracy of deep learning, so just do lots and lots of different movements! Save image data to SD card.
Install TensorFlow / Keras for Ubuntu
This session use Ubuntu20.04 or Windows Subsystem for Linux.
Install Miniconda and create a Python environment. The Miniconda installer is downloaded from the Miniconda website.
Install Python, TensorFlow, Keras, etc. on Miniconda.
conda create -n ml python=3.6 tensorflow=1.14 keras pillow \
numpy pydot graphviz
Install kendryte nncase
conda activate ml
nncase converts learning data created with Keras or TensorFlow into KPU learning data kmodels.
nncase github: https://github.com/kendryte/nncase/
tar -Jxf ncc-linux-x86_64.tar.xz
A model of acceleration is created by deep learning. Acceleration images are classified into M5StickV SD cards for each activity. Learn with Keras from classification and images at CNN. The learning results are saved with Kmodel.
Run the PYTHON program on Ubuntu as follows command.
"my_model.kmodel"file named A is generated.Kflash GUI
Write m5stickv firmware and kmodel with tools.
Cheering Watch is equipped with a function that emits a voice in Wav file when you make a predetermined move.
Cheering Watch learns your movements and cheers for you when you want your friends to cheer for you! Cheering Watch will learn your movements and cheer for you when you want it to.
A friendly face is displayed to help you.This face rotates like a gimbal to the tilt. In addition, audio is output according to the motion.
m5stickv cannot be connected to the Internet. So connect it to M5StickC and connect to the Internet. Send motion data to an ambient cloud service in Japan.
The m5stickc software creates programming with arduino.Receive from URT and send to AMBIENT.It also gets the current time from the network and displays the time.