ABC-Buddy

Intro

This project is created in context of the lecture “IoT ApplicationsPrototyping” at the Brunel University / TAE Esslingen. It covers the invention of an own IoT Idea and Application from the scratch solving a real-world problem. In the lecture the MSc students learn about signal processing, pattern recognition and scientific analysis in context of IoT applications.

Smartphone Sensing Framework

The Smartphone Sensing framework (SSF) provides a collection of modules anda quick way to develop context aware apps on android in an experimetal way.

IoT Prototyping Framework

The IoT Prototyping Framework (IoTPF) is a collection of tools, modules andsamples with the aim to empower students and developers creating full-stack IoT prototypes in a short time period.

Contact:

Idea and Application: Vinay Kumar Puligilla, Pascal Walter, Alisa Jäkle

Lecture, SSF and IoTPF: dionysios.satikidis@gmail.com

Tools:

https://github.com/MrDio/Smartphone-Sensing-Framework

https://github.com/MrDio/IoT-Prototyping-Framework

Brunel University London / TAE Esslingen 2018

This project was developed by students of Brunel University London as part of the Embedded Systems Engineering course.

- Kristin Findeis, Matthias Gebert, Christian Holder & Büsra Yagbasan

Motivation

For some years now, those responsible in the education sector and parents of kindergarten/primary school children have been dealing with the issue of phasing out the compulsory learning of joined-up writing. In Finland it was decided years ago that the handwriting has had to make way for computer courses. In order to prevent the handwriting from disappearing completely one day, a combination of old and new is to be created in people's lives through the playful use of an increasingly significant technology. In contrast to other applications designed for children, which are often criticised by educational experts, the use of the mobile phone as a magic wand should arouse the child's interest and prevent them from constantly staring at the screen.

The basic idea is simple, by repeated movements of the mobile phone the spelling of the individual letters is to be internalised.

Features of the Application

The application has two different modes:

• Learn-Mode: In this mode, the user can learn a specific letter by selecting it. The user is then shown a video for drawing the letter. After the user has drawn the letter, the application evaluates the recording and displays to the user whether he has written the letter correctly or not.

• Recognize-Mode: In this mode the user can write any letter in the air. The application compares the record with all letters and shows the user which letter he has written.

State Machine

The figure below illustrates the states of the application:

The application consists of three states. The default-state is the idle-state. If the user starts the recording, the application changes to the recording-state. After 4 seconds, the application switches to the analysis-state. In this state the mathematical calculation are done. Finally, the application prints the result and changes to the idle-state.

Implementation

Recording

When the application is put into recording mode, the user has one second to prepare for the movement. After one second a signal indicates that the movement can be started and from this moment on the recording will take place for four seconds. When the four seconds have elapsed, another signal appears indicating the end of the recording phase. The actual recording of the combined acceleration vector is done using the implementation of the Smartphone Sensing Framework. Ten data values are collected per second, whereby one data value is a so-called tick. For the analysis the data of the individual axes are collected in order to calculate the combined vector with which the Mean Squared Error is later calculated.

Analysis

As already mentioned, the Mean Squared Error method was used for the analysis. In this procedure, as explained in a, the deviation of an input data set from the data sets stored for the characters is calculated. The input data set in this case is the acceleration captured during the recording phase. While the comparison data sets are optimal accelerations of motion sequences created by the developers. Each character has 40 comparison data sets, with ten data sets from each developer. The following equation is used for the calculation itself.

In this context, n corresponds to the number of values in both data sets, which is set to 40 by the recording of four seconds. Furthermore, the values x and y describe the individual values at the position I in the respective data set. For this, x corresponds to the input and y to the comparison data set. Since this is a deviation respectively an error calculation, the result should be relatively small. For this reason, the lowest value of each character is stored.

If the learning function is selected, the calculated value belonging to the character selected is compared with a threshold value. This threshold value was created individually for each character by trial and error tests. The letter was correctly drawn into the air if the calculated value is smaller than the threshold. The table shows the thresholds for the different characters.

Final Application

The final application is presented below: