Rate your driving with machine learning

We have created a web application that helps drivers determine their quality of driving based on a machine learning model. We have realised that a lot of mobility services such as Uber, Bolt, eCabs do not have an objective rating system. Instead they use subjective opinions of their customers to determine the quality of their drivers. In order to change that we have created a web application that is able to measure driver’s quality based on an inbuilt smartphone accelerometer.

Our web app is availible online, make sure you are using your phone tho: https://smoothtaxisaxy.herokuapp.com/

The project is made out of 3 components. First we built a machine learning model to detect good and bad driving patterns. Then we exported the model with the help of Web Assembly and third, we built a web app using javascript and html.

1. Edge impulse

For our application we wanted to create a machine learning model that would classify turns, braking and idle driving. We acquired the data with the help of a smartphone accelerometer, where we drove around recording gentle and harsh turns, brakings and idle driving. We first labelled every pattern that we obtained and then cropped them. After data acquisition we built a classification model with the nearest neighbour classifier. We wanted the model to separate among 7 different scenarios:

• Good left turn
• Bad left turn
• Good right turn
• Bad right turn
• Good braking
• Bad braking
• Idle driving

The model’s classification succession was 66, 5%, which you can also see from image 1.

2. Web Assembly

We wanted to build a web application because we knew every modern phone has an inbuilt accelerometer that can be used to our advantage. For that purpose we deployed our model from Edge Impulse to a web format with the help of Web Assembly. It came with a Javascript exported classification model, some Web Assembly code and a Python web server. When starting the server, you were taken to a simple website with one input field for classification. If you were to paste previously obtained accelerometer data into the input line, you got a json response classifying the impulse and values telling you the probability of that classification.

{     anomaly: 0,
results: [

{label: "desno grdo zavijanje",
value: 0.0625 },

{label: "desno lepo zavijanje",
value: 0.04296875},

{label: "levo grdo zavijanje",
value: 0.03125},

{label: "levo lepo zavijanje",
value: 0.046875},

{label: "mocno zaviranje 📷",
value: 0.01953125},

{label: "sibko zaviranje 📷",
value: 0.14453125},

{label: "voznja",
value: 0.65234375}

]}

3. Building a Web app

So far in order to classify the data, one had to paste the features into the input field. However we wanted the classification to happen automatically every 5 seconds while one is driving. We did that by implementing an event listener called DeviceMotionEvent in Javascript. This event listener fires every time a change in device acceleration is detected.

window.removeEventListener("devicemotion", handleMotion);

Even so there were a few challenges we needed to overcome. We first changed our Python server into a Node.js one, since everyone was more familiar with Node. Then in order to test the application, we needed to deploy our code to Heroku so that we were even able to access the device's accelerometer (laptops do not have one). The connection to the server had to be encrypted with SSH, so we had to be very cautious to always write an extra “s” to the heroku link so that the connection was “https”. It was also necessary to ask a user for accelerometer permission.

if (
DeviceMotionEvent &&
typeof DeviceMotionEvent.requestPermission === "function"
) {
DeviceMotionEvent.requestPermission();
}

Everytime a DeviceMotionEvent detects acceleration, it fires a function called handleMotion.

var tabelcaSpelca = [];

function handleMotion(event) {
tabelcaSpelca.push(event.accelerationIncludingGravity.x);
tabelcaSpelca.push(event.accelerationIncludingGravity.y);
tabelcaSpelca.push(event.accelerationIncludingGravity.z);

if (tabelcaSpelca.length > 900) {
klasificiraj(tabelcaSpelca);
tabelcaSpelca = [];
}
}

Its main purpose is to collect acceleration data in an array. After 5 seconds, which equals 900 array elements, we classify the data by calling a function Klasificiraj.

function klasificiraj(tabelcaSpelca) {
var classifier2 = new EdgeImpulseClassifier();
classifier2.init();
let res = classifier2.classify(tabelcaSpelca);
...}

This function calls our exported classification model and returns previously mentioned json classification output.

To make an application user friendly we made a counter that counts how many good/bad turns and brakes you made. We first find the classification with the maximum probability and then increment the counter for that event.

...
badRightTurnOverall = 0;
var max = 0;
var max_label;

// find classification with the maximum probability 
for (var i = 0; i < res.results.length; i++) {
if (max == 0 || res.results[i].value > max) {
max = res.results[i].value;
max_label = res.results[i].label;
}
}

//increment the counter
if(max_label == "desno grdo zavijanje"){
badRightTurnDaily++;
document.getElementById("badRightTurnOverall").innerHTML = badRightTurnOverall;
} else if(max_label == "desno lepo zavijanje") 
{...} //we do so for every label

There is also an overall score of one’s driving that is calculated in percentages. It shows you the percentage of how many of the total turns and brakes you did that were classified as good. The driver is then able to see how smooth his or her driving really is and the data can also be used by companies like Uber to objectively rate a taxi driver. (see image 2)