Published August 30, 2023 © MIT

GestoVoice: Gesture to Voice Conversion

Gesture Vocalizer: Transforming sign language to speech with a smart glove and app, empowering inclusive communication. #InnovationForAll

IntermediateProtip2 hours1,554

Things used in this project

Hardware components

Arduino UNO

HC-05 Bluetooth Module

Flex Sensor

SparkFun Triple Axis Accelerometer and Gyro Breakout - MPU-6050

Software apps and online services

MIT App Inventor

Arduino Bluetooth Text to Speech

Hand tools and fabrication machines

Soldering iron (generic)

Solder Wire, Lead Free

Premium Female/Male Extension Jumper Wires, 40 x 6" (150mm)

Tape, Double Sided

Multitool, Screwdriver

Story

Introducing the initiative known as #GestoVoice; Gesture, to Voice Conversion by Riza Mohamed T. This remarkable project was inspired by an encounter with the sister of a deaf grandmother shining a light on the significant communication barriers faced by those who rely on sign language. It underscores the importance of inclusivity and empathetic understanding. Driven by compassion and a relentless desire to empower the hearing impaired this endeavor combines cutting edge technology with warmth to create a solution.

At the core of #GestoVoice lies a blend of state of the art components seamlessly integrated together. The essence of this innovation can be felt through sensors, a coordinating accelerometer, an Arduino microcontroller that showcases computational brilliance and a Bluetooth module for seamless wireless connectivity. These elements harmoniously come together within a glove transforming it into something

Imagine this glove as more than an assembly of parts; it serves as a bridge, between worlds translating gestures into eloquent voices. The journey begins as these gestures flow through the pathways of the Arduino microcontroller, where sophisticated algorithms intricately decode their nuances with remarkable precision to reveal their inherent meanings.

However this transformation is only the beginning.

The infused information soars through the air traveling wirelessly to an Android application where the enchantment intensifies. Within this realm the information undergoes a captivating transformation. Emerges as spoken words.These once-muted gestures are now vocal, articulating thoughts, feelings, and intentions with unparalleled clarity and resonance.

#GestoVoice isn't merely a project; it's a proclamation of progress. It's a testament to the endless possibilities that arise from the confluence of human empathy and cutting-edge technology. With every gesture translated into speech, it dismantles barriers and paves the way for a more inclusive, connected society. The creator's vision is brought to life through #GestoVoice, showcasing not just technical prowess, but also the transformative potential of technology to foster compassion, connection, and positive change.

The Circuit Design of GestoVoice

Components Needed:

Arduino Board (e.g., Arduino Uno)
Flex Sensors (4)
MPU6050 Accelerometer
Bluetooth Module (HC-05 or HC-06)
LED and Resistor (220Ω)
Jumper Wires
Step-by-Step Guide:

Arduino Setup:

Connect your Arduino board to your computer using a USB cable.
Open the Arduino IDE on your computer.
Connecting Flex Sensors:

Connect one leg of each flex sensor to the 5V pin on the Arduino.
Connect the other leg of each flex sensor to the analog pins (A0, A1, A2, A3) on the Arduino.
Connect a 10kΩ resistor from each flex sensor's leg to the ground (GND) pin on the Arduino.
Connecting MPU6050:

Connect the VCC and GND pins of the MPU6050 to the 5V and GND pins on the Arduino.
Connect the SDA pin of the MPU6050 to the A4 (analog pin 4) on the Arduino.
Connect the SCL pin of the MPU6050 to the A5 (analog pin 5) on the Arduino.
Connecting Bluetooth Module:

Connect the VCC pin of the Bluetooth module to the 5V pin on the Arduino.
Connect the GND pin of the Bluetooth module to the GND pin on the Arduino.
Connect the TXD pin of the Bluetooth module to the RX pin (pin 0) on the Arduino.
Connect the RXD pin of the Bluetooth module to the TX pin (pin 1) on the Arduino.
Connecting LED:

Connect the anode (longer leg) of the LED to pin 13 on the Arduino.
Connect the cathode (shorter leg) of the LED to a 220Ω resistor.
Connect the other end of the resistor to the GND pin on the Arduino.
Uploading the Code:

Copy and paste the provided Arduino code into the Arduino IDE.
Make sure you've selected the correct Arduino board and COM port from the Tools menu.
Click the "Upload" button to upload the code to your Arduino board.
Testing the Circuit:

Once the code is uploaded, open the Serial Monitor in the Arduino IDE.
You should see output indicating the gesture and corresponding spoken response based on the sensor readings.
Bend the flex sensors and tilt the MPU6050 to see how the responses change.

Code

Programming the Heart of GestoVoice

Programming the Heart of GestoVoice

C/C++

Gesture to Voice Conversion using Flex Sensors and MPU6050
This code snippet is part of the "GestoVoice: Gesture to Voice Conversion" project. It combines flex sensors and an MPU6050 accelerometer to translate hand gestures into spoken words.

Components Needed:
- Flex Sensors (adc1, adc2, adc3, adc4)
- MPU6050 Accelerometer

How to Use:
1. Ensure you have the necessary hardware components connected as specified.
2. Upload this code to your Arduino board.
3. The program reads values from flex sensors and accelerometer.
4. Based on the flex sensor values and accelerometer data, the program detects different gestures.
5. When flex1 or flex2 is bent, and the accelerometer is tilted in a certain direction:
- Different spoken responses are generated and displayed in the Serial Monitor.
- For example, tilting the accelerometer in one direction while bending a flex sensor might trigger "HELP," "WASHROOM," "MEDICINE," or "FOOD."
6. The LED on pin 13 (ledd) is used to indicate gesture detection.

Made by Riza Mohamed T

// GestoVoice: Gesture to Voice Conversion
// Made by Riza Mohamed T
// scl-----A5   Sda-----A4 

#include <Wire.h>
#include <MPU6050.h>
MPU6050 mpu;
#define adc1 A0
#define adc2 A1
#define adc3 A2
#define adc4 A3
#define ledd 13
int flex2=0,flex1=0,flex3=0,flex4=0;
void setup() 
{
  
 pinMode(ledd,OUTPUT);
 digitalWrite(ledd,LOW);
 Serial.begin(9600 );
 Serial.println("Initialize MPU6050");
 while(!mpu.begin(MPU6050_SCALE_2000DPS, MPU6050_RANGE_2G))
 {
  Serial.println("Could not find a valid MPU6050 sensor, check wiring!");
  delay(500);
 }
 checkSettings();
 
}
void checkSettings()
{
 
 Serial.println();  
 Serial.print(" * Sleep Mode:            ");
 Serial.println(mpu.getSleepEnabled() ? "Enabled" : "Disabled");
 Serial.print(" * Clock Source:          ");
 switch(mpu.getClockSource())
 {
  case MPU6050_CLOCK_KEEP_RESET:     Serial.println("Stops the clock and keeps the timing generator in reset"); break;
  case MPU6050_CLOCK_EXTERNAL_19MHZ: Serial.println("PLL with external 19.2MHz reference"); break;
  case MPU6050_CLOCK_EXTERNAL_32KHZ: Serial.println("PLL with external 32.768kHz reference"); break;
  case MPU6050_CLOCK_PLL_ZGYRO:      Serial.println("PLL with Z axis gyroscope reference"); break;
  case MPU6050_CLOCK_PLL_YGYRO:      Serial.println("PLL with Y axis gyroscope reference"); break;
  case MPU6050_CLOCK_PLL_XGYRO:      Serial.println("PLL with X axis gyroscope reference"); break;
  case MPU6050_CLOCK_INTERNAL_8MHZ:  Serial.println("Internal 8MHz oscillator"); break;
 }
 Serial.print(" * Accelerometer offsets: ");
 Serial.print(mpu.getAccelOffsetX());
 Serial.print(" / ");
 Serial.print(mpu.getAccelOffsetY());
 Serial.print(" / ");
 Serial.println(mpu.getAccelOffsetZ());
 Serial.println();
}

void loop()
{
  flex1=analogRead(adc1);
  flex2=analogRead(adc2);
  flex3=analogRead(adc3);
  flex4=analogRead(adc4);
  delay(10);
  Vector rawAccel = mpu.readRawAccel();
  Vector normAccel = mpu.readNormalizeAccel();
  Serial.print(" Xnorm = ");
  Serial.print(normAccel.XAxis);
  Serial.print(" Ynorm = ");
  Serial.print(normAccel.YAxis);
  Serial.print(" Znorm = ");
  Serial.println(normAccel.ZAxis);
  Serial.print(flex1);
 Serial.print("-");
  Serial.print(flex2);
  Serial.print("-");
  Serial.print(flex3);
  Serial.print("-");
  Serial.print(flex4);
  Serial.println("-");
  delay(1000);
  if(flex1<70)
  {
   digitalWrite(ledd,LOW);
   if(normAccel.XAxis>5){Serial.println("HELP");}
   else if(normAccel.XAxis<-5){Serial.println("WASHROOM");}
   else if(normAccel.YAxis>5){Serial.println("MEDICINE");}
   else if(normAccel.YAxis<-5){Serial.println("FOOD");}
  }
  else if(flex2<70)
  {
   digitalWrite(ledd,LOW);
   if(normAccel.XAxis>5){Serial.println("What is your name");delay(500);}
   else if(normAccel.XAxis<-3){Serial.println("WASHROOM");}
   else if(normAccel.YAxis>3){Serial.println("MEDICINE");}
   else if(normAccel.YAxis<-3){Serial.println("FOOD");}
  }
  else
  {
   digitalWrite(ledd,HIGH);
   if(normAccel.XAxis>5){Serial.println("THANK YOU");}
   else if(normAccel.XAxis<-5){Serial.println("WATER");}
   else if(normAccel.YAxis>5){Serial.println("SORRY");}
   else if(normAccel.YAxis<-5){Serial.println("HELLO");} 
  } 
}