Digital Life Tracking

1. The Idea

The Digital Life Tracking App is designed to enable users to track their daily tasks and the time spent on them. To make it more convenient to trigger an activity, the start and end of an activity may be triggered in different ways. Possible activation mechanisms could be:

• Two claps

• moving the smartphone in a certain way (gestures)

• activation based on GPS position

• pressing a button

Users should be able to define their own activities and select an activation function from a list and assign it to one of their activities. The user should then be able to display his history in form of charts. A separate device (particle photon board) could also be used to trigger a specific activity.

2. The Deep Dive

For the prototype in the context of the assigment we decided to focus mainly on the detection of double clapping as an activity trigger. From the original idea a concept for the following app was developed:

Digital Life Tracking

The app should allow a user to divide his daily routine into categories of activities and to measure the time required for each activity. This enables him to get an overview of his time invested in various activities.

3. Data Analysis

The aim of this work is to generate an application that starts a timer that measures the time of a certain activity by clapping into the hands. The timer is stopped by the repeated clapping and the time is saved together with a corresponding activity. This creates a history for the user in which he can see how long which activity was performed.

The problem to be investigated is the proper detection of clapping noises. It must be possible to distinguish between background noises such as coughing or knocking. This presents a major challenge, as they are difficult to distinguish from a technical point of view.

A large number of sound samples were recorded for clapping, tapping, coughing and speaking and the respective frequency spectrum was compared. In order to get a variation of the sounds different people have executed these tasks. In addition, various software tools are used to analyse the frequencies.

The result of this work is that the analysed noises partly happen on the same frequencies. A distinction can therefore only be made in the duration of the tones. Using individual algorithms for this would become too complex, as many cases have to be covered. It therefore makes more sense to use pattern recognition frameworks. When enough time is available, the use of pattern recognition frameworks makes the most sense. The devices that were used reliably detect the noises and the mathematical calculations are also possible without problems on the systems.

4. Implementation

4.1 Clap Detector

The ClapDetector creates a new AudioDispatcher with a buffer size of 1024 bytes and a sample rate of 20kh and registers itself as AudioProcessor. Thus, the AudioDispatcher calls the AudioProcessor Process Handler with 1024 amplitude values every 0.02 seconds. For each time period, a Fast Fourier transformation is created using the Tarsos library. The number of peaks is counted for each new frequency spectrum. The peaks are detected by iterating over each frequency and calculating the decibel values for each magnitude. The decibel value is then compared with a fixed threshold. All decibel values that exceed the threshold are counted. The ClapDetector stores only the last two PeakCounter values. To detect a clap, the second last peak counter must be smaller than the last and the current one smaller than the last. If this is the case, an increase and decrease has been detected in several frequency ranges and the handler is called.

4.2 State machine

There are a total of four states in which the app can be in.

• Idle: The initial state when starting the app and the state after an activity tracking has been completed.

• StartPrecondition: If the app is in the idle state and a clap is detected, the state machine switches to this state. When switching to this state, a timer is started which defines the time window in which the second clap must occur in order to switch the state to TrackingActivity. If the timer expires before another clap is detected, the state machine switches back to the idle state.

• TrackingActivity: After a second clap is detected while the timer of the start precondition has not yet elapsed, the state machine changes to this state and starts capturing the time by saving a time stamp.

• StoppingPrecondition: If the state machine is in the TrackingActivity state and a clap occurs, then the state machine switches to this state, which behaves in the same way as the StartingPrecondition, except that on a successful second clap, it changes to the idle state and the tracking of the current activity is ended.

4.3 UI Design

Since there was not enough time to implement the actual user interface during development later on, the app only has the development UI as it is currently available.

5. Current State

The current state of clap detection causes some false positives. Often voices are also recognized as clapping, or flicking with fingers or coughing. Since there was not enough time to implement pattern recognition using a neural network, the app recognizes every pitch in a frequency range as a clap. Frequently it happens to the user that when trying to clap twice, one of the claps is not loud enough because he has not hit his hands properly and therefore just a single clap is detected.

If one examines the current state of the app for its usability, it is obvious that it would probably be quite unfavorable for the user experience if false positives ended the timer unintentionally, since the user would be currently busy with an activity and not immediately notice it.

6. Project Outlook

To work on the problem further, some debugging functionality within the app would be good, so it would be faster to try different parameters instead of rebuilding the app over and over again. Data analysis has shown that clapping is very difficult to detect. Compared to clapping, a clear pattern can be seen in the frequencies of a whistle sound. For the future of the app it might be better to detect a whistle instead of a clap to toggle the time tracking.

7. Presentation