AudioGuide - AI-Powered Environmental NavigatorStory
Read moreFor millions of people with visual impairments, navigating outdoor environments presents daily challenges. While traditional aids like white canes and guide dogs provide invaluable assistance, modern technology offers opportunities to enhance outdoor navigation and independence. AudioGuide aims to bridge this gap by creating an AI-powered environmental navigation system using the DFRobot UniHiker board.
The ProblemPeople with visual impairments face several challenges when navigating outdoor environments:
- Difficulty identifying obstacles and hazards in real-time
- Limited ability to read signs and text in the environment
- Challenges in understanding spatial relationships and distances
- Reduced confidence in exploring new areas independently
AudioGuide transforms the powerful UniHiker board into an intelligent navigation assistant. By combining computer vision, AI processing, and intuitive audio feedback, it provides real-time environmental information to users, enhancing their spatial awareness and confidence in outdoor navigation.
Unihiker Kit
python
python
Copy
import cv2
import numpy as np
import tensorflow as tf
import gtts
import pygame
import serial
import time
import RPi.GPIO as GPIO- Object Detection System:
python
python
Copy
def detect_objects(frame):
# Convert frame to blob for neural network
blob = cv2.dnn.blobFromImage(frame, 1/255.0, (416, 416), swapRB=True, crop=False)
net.setInput(blob)
# Get detections
layer_outputs = net.forward(output_layers)
# Process detections
class_ids = []
confidences = []
boxes = []
for output in layer_outputs:
for detection in output:
scores = detection[5:]
class_id = np.argmax(scores)
confidence = scores[class_id]
if confidence > 0.5:
# Process detection coordinates
center_x = int(detection[0] * width)
center_y = int(detection[1] * height)
w = int(detection[2] * width)
h = int(detection[3] * height)
# Rectangle coordinates
x = int(center_x - w / 2)
y = int(center_y - h / 2)
boxes.append([x, y, w, h])
confidences.append(float(confidence))
class_ids.append(class_id)
return boxes, class_ids, confidences- Audio Feedback System:
python
python
Copy
class AudioFeedback:
def __init__(self):
pygame.mixer.init()
self.audio_cache = {}
def generate_audio(self, text):
if text not in self.audio_cache:
tts = gtts.gTTS(text)
temp_file = f"temp_{hash(text)}.mp3"
tts.save(temp_file)
self.audio_cache[text] = temp_file
return self.audio_cache[text]
def play_audio(self, text):
audio_file = self.generate_audio(text)
pygame.mixer.music.load(audio_file)
pygame.mixer.music.play()
while pygame.mixer.music.get_busy():
continue- GPS Navigation:
pytho
python
Copy
class GPSNavigator:
def __init__(self, port='/dev/ttyUSB0'):
self.ser = serial.Serial(port, 9600, timeout=0.5)
def get_location(self):
while True:
data = self.ser.readline().decode('ascii', errors='replace')
if data.startswith('$GPGGA'):
parts = data.split(',')
if len(parts) >= 6 and parts[2] and parts[4]:
lat = float(parts[2][:2]) + float(parts[2][2:]) / 60
lon = float(parts[4][:3]) + float(parts[4][3:]) / 60
return lat, lon
return None, NoneHardware Assembly:
- Mount the camera module to the UniHiker board
- Connect GPS module to UART pins
- Wire ultrasonic sensors to GPIO pins
- Install emergency button
- 3D print the case and assemble components
- Hardware Assembly:
Mount the camera module to the UniHiker board
Connect GPS module to UART pins
Wire ultrasonic sensors to GPIO pins
Install emergency button
3D print the case and assemble components
Software Setup:
- Install required Python packages
- Configure system autostart
- Set up error logging
- Calibrate sensors
- Software Setup:
Install required Python packages
Configure system autostart
Set up error logging
Calibrate sensors
Testing:
- Verify object detection accuracy
- Test audio feedback clarity
- Validate GPS accuracy
- Check emergency button functionality
- Perform battery life testing
- Testing:
Verify object detection accuracy
Test audio feedback clarity
Validate GPS accuracy
Check emergency button functionality
Perform battery life testing
- Power on the device using the power button
- Wait for the startup sound confirmation
- Put on the bone conduction headphones
The system will automatically begin:
- Detecting objects and obstacles
- Providing audio feedback
- Tracking location
- Monitoring for emergency button press
- The system will automatically begin:
Detecting objects and obstacles
Providing audio feedback
Tracking location
Monitoring for emergency button press
- Single beep: Object detected
- Double beep: Obstacle in path
- Triple beep: Emergency mode activated
- Voice descriptions: Environmental information
Enhanced Features:
- Machine learning for improved object recognition
- Integration with mapping services
- Crowd-sourced safe path database
- Weather API integration
- Enhanced Features:
Machine learning for improved object recognition
Integration with mapping services
Crowd-sourced safe path database
Weather API integration
Hardware Optimizations:
- Reduced power consumption
- Smaller form factor
- Waterproof casing
- Extended battery life
- Hardware Optimizations:
Reduced power consumption
Smaller form factor
Waterproof casing
Extended battery life
Software Enhancements:
- Personalized audio feedback
- Offline operation mode
- Enhanced emergency features
- Social features for community support
- Software Enhancements:
Personalized audio feedback
Offline operation mode
Enhanced emergency features
Social features for community support
AudioGuide demonstrates how accessible technology can enhance independence and confidence for people with visual impairments. The project:
- Provides real-time environmental awareness
- Enhances safety during outdoor navigation
- Increases independence in daily activities
- Creates opportunities for social interaction
- Builds confidence in exploring new environments
Special thanks to:
- DFRobot for the UniHiker board
- The visual impairment community for valuable feedback
- Open-source contributors to the libraries used
- BUILD2GETHER 2.0 for the opportunity to develop this solution
Comments