Ruker - Wheelchair Terrain Tracker

Wheelchair users often do not know the condition or surface type of the roads and sidewalks before starting a journey. The travel experience can vary significantly depending on whether the path is made of smooth asphalt, cobblestone, or damaged pavement. Uneven surfaces, cracks, and deteriorated roads can make navigation uncomfortable, difficult, or even unsafe, creating additional challenges for everyday mobility.

Machine learning for road classification

We used the Edge Impulse platform to train our machine learning model using data collected by ourselves. Most of the data was gathered in Ljubljana during real-world wheelchair movement.

The dataset focused on three main terrain categories: smooth, tough and rough terrain. Smooth terrain mainly consisted of asphalt roads, tough terrain includes gravel roads and paths that are paved with stone slabs, meanwhile rough terrain focuses on very bumpy roads like the ones paved with granite cubs and damaged roads.

In addition, we introduced a separate idle classifier to detect when the user was stationary and not moving. This helped improve the accuracy and reliability of the terrain classification system.

Although the model achieved very high accuracy, this result may be influenced by the limited amount of collected data. Even so, the model showed strong and reliable performance in real-world testing scenarios.

App

We decided to build the app for Android phones for two main reasons. First, Android is the most widely used mobile operating system in the world. Second, the Edge Impulse platform provides an Android library that can be easily integrated into an existing Android project.

All sensor data is processed locally on the device, meaning no data is sent to external servers unless the user explicitly chooses to share their route. This also gives the app very low latency, so terrain predictions appear in the user interface almost instantly.

For the map functionality, we used the Google Maps API mainly because of its extensive documentation and ease of integration. We also use Firebase because users need to create an account in order to store their recorded paths in a database. This keeps the amount of data stored locally on the phone to a minimum.

Using a central database also allows us to build a community-driven map, which is intended to become the main feature of our application. However, since the app currently relies on crowdsourced data, we do not yet have enough user data for this feature to be useful to an average user.

With the app, users can record routes wherever they want. Once they are satisfied with a route, they can stop the recording and choose to either discard it, save it privately, or make it public so it can be included in the community map.

Users also have full control over their privacy settings and can change any of their public routes back to private at any time.

Further improvements

We mainly made this as a proof of concept. As such our prototype and code probably have quite a lot of space for improvement.

The data was gathered mainly by one source and because of that it was not large. We believe that with a larger dataset, collected by more people, we could improve the accuracy and which types of terrain should go under each color. Since edge impulse focuses on edge devices that have very small storage space, the larger model will not take a lot of space and not make the device slower.

One key feature that is currently missing is the smart navigation. The final version of this app should be able to look at all the uploaded paths and pick the shortest and smoothest one to a desired destination. This of course means that we need a lot of different uploaded paths from users, to make it work.

Perhaps a button to add obstacles like stairs and uncrossable paths would also be a good addition, next to just automated terrain tracking.