VisionSense

Inspiration

Our team was discussing the advances in wearable technology, and how it helps us stay connected while making daily life easier in aspects like health, navigation, and safety. Eventually, we began to wonder how wearable technology could play these same roles in the lives of different user groups. We considered the design specifications and changes for groups that cannot use the sense of sight- primarily people with visual impairment or in occupations like mining/mountaineering that can at times be in pitch darkness.

Functionality

Step (1) Environmental Assessment

The TELUS DevKit contains a Avnet Quectel BG96 Shield, a STMicro NUCLEO-L296ZG MCU Board, and a STMicro X-NUCLEO-IKS01A2 Sensor Board, assembled as show below. Sensory data from the users environment (e.g. temperature, pressure, humidity, latitude/longitude, location, etc.) is outputted and sent over LTE to the Azure IoT Hub; the data is simultaneously stored in Azure Cosmos DataBase (cloud).

Step (2) Obstruction Detection

The Raspberry Pi board is connected to 3 Ultrasonic Sensors, through which it determines any obstructions within a 4 metre range from the user's head. This data is wirelessly communicated to an online server via WiFi, that can then provide input to the Arduino system. The system is also equipped with a help button that sends an emergency text message to an emergency contact with just a simple press of a button by the user.

Step (3) Haptic Direction

The Arduino uses input communicated wirelessly from the Ultrasonic sensors to set custom vibrations that can communicate any obstructions, along with their range and direction relative to the user. Each Ultrasonic Sensor provides input to one motor. Placement of motors is intuitively along the same side as US sensors, so users can easily estimate obstacles based on haptic sense. This component allows users to act upon all the information gathered from our system to seamlessly move around without any hand-held aids.

Challenges

• Setting up the Python Wireless Connection for Data collection
• Drawing out (and accordingly building) sound circuitry to ensure all UltraSonic Sensors provide accurate data
• Connecting Telus Devkit to Azure
• Stably building the housing components in wearable form for the systems
• Interfacing Raspberry Pi with Arduino

Accomplishments of Which We Are Proud

• We did not have actuators or vibrators available, so we really had to understand the inner workings of Stepper Motors and rewire them (as well as format the code accordingly) to have them work as vibrators using input from the Raspberry Pi
• Setting up the environments for all the systems-on-a-chip was a time-consuming and tedious job with much debugging. Getting through that took many hours, but we managed to successfully interface and integrate all components of our tech
• The data output by hardware was a bit challenging to work with, especially in the case of the UltraSonic Sensors. However, with much testing and documentation review, we did eventually achieve proper obstacle detection using these sensors

Future Steps and Improvements

• making the wearable technology smaller and easier to wear
• using computer vision with an embedded camera that communicates (using audio) any pre-determined tags to user if they appear before the user