Published April 22, 2025 © GPL3+

I Turned My Hands Into a Virtual Steering Wheel!!

🎮 " Steering Wheel with Just Your Hands!" 🚗 Tilt to steer, thumbs to drive or brake—no controller needed. Just a webcam! 🖐️🚗💻

AdvancedFull instructions provided3 hours65

I Turned My Hands Into a Virtual Steering Wheel!!

Things used in this project

Hardware components

Laptop or PC

Webcam or External Cam

Software apps and online services

Microsoft VS Code

OpenCV

Story

Full Video On Youtube

What Is This Project?

Imagine you're playing a car racing game… but instead of using a joystick or keyboard, you just move your hands in front of a camera to drive the car! This project makes that possible using a webcam, Python.

You can:

👐 Turn your hands left or right to steer the car.
👍 Move your thumb down to speed up or slow down. It’s like magic! But it’s actually made using something called Computer Vision.

This project is designed for complete beginners and fun enthusiasts who want to control a car using hand gestures with a webcam. The program uses MediaPipe to track your hand, calculates the angle between your wrist, thumb, and index finger, and based on that angle, it sends keyboard commands to the game.

No fancy equipment needed—just a webcam, your hand, and a game that uses the keyboard (like pressing A/D to steer).

Let’s dive into how it works in the simplest way possible.

Supplies

Here’s everything you need:

️ A laptop or desktop computer (Windows, macOS, or Linux)
A webcam (built-in or USB)
Python installed (version 3.7 or later)
Python packages:
opencv-python
mediapipe
pyautogui

Step 1: Install Python

Install Python

If you haven’t already:

Go to https://www.python.org/
Click Download Python
Install it like any other app

Step 2: Install the Libraries

OpenCV Library

MediaPipe Library

PyAutoGui Library

These are the special helpers for Python:

Open Command Prompt (Windows) or Terminal (Mac/Linux)
Type the following and press Enter:

pip install opencv-python mediapipe pyautogui

What Are These Libraries?

Let’s understand them like superheroes:

🧠 Mediapipe – This smart tool sees your hands and finds points like your thumb, wrist, and fingers.
👀 OpenCV – It helps us use the webcam and draw things on the screen.
⌨️ PyAutoGUI – It presses keys like W, A, S, D for you.

Step 3: Copy the Code

this took me days to figuring out and I am giving you the code for free!!!

Here’s the magic program. You can open Notepad, or better: install VS Code.

Step 4: Explaination of Code and Important Settings in Code Before Using It

🔍 Explaining the Key Code Lines

✅ Importing Tools

import cv2 # To use webcam and show visuals

import mediapipe as mp # To detect hands

import math # To calculate angles

import pyautogui # To press keyboard buttons automatically

import time # To handle timing for button presses

✅ Initial Settings

hands = mphands.Hands(...)

This initializes MediaPipe Hands to detect up to 2 hands.
It finds where your hands are and tracks their movement.

✅ Detecting Angle Between Hands

def calculate_angle(x1, y1, x2, y2):

...

Calculates the angle between your left and right hands.
This tells whether you're turning left, right, or going straight.

🧠 Important Settings :

🛠 turn_sensitivity = 0.01

This was used in a previous version to make turning more or less sensitive.
BUT in your current code, it's not used anymore, so we can remove it or ignore it.

🔁 straight_threshold = 6

This value decides how much your hand angle must change before it counts as turning.

If the angle is less than -6, it means turn left (A key).
If the angle is more than +6, it means turn right (D key).
If the angle is between -6 and +6, it means go straight (do nothing).

This helps avoid small hand shakes being treated as turns.

⏱️ tap_duration = 0.01

This sets how quickly to release the A or D key after tapping it.

0.01 means the key is held down for just 0.01 seconds (very short).
This makes the car turn in small steps, so you don’t turn too much with a small hand movement.

If you want smoother, longer turns, increase this value to something like 0.05.

🖐️ Detecting Thumb Up or Down

def is_thumb_up(hand_landmarks):

return hand_landmarks.landmark[4].y < hand_landmarks.landmark[3].y

Checks if the tip of the thumb is above the second joint (on the screen).
This is used to detect:
Left thumb up = stop accelerating
Right thumb up = stop braking

🚗 Controls Using Hands

🟢 Accelerate (W key)

if not is_thumb_up(left_hand):

pyautogui.keyDown('w')

If your left thumb is down, press the W key (accelerate).
If your left thumb goes up, release the W key (stop).

🔴 Brake (S key)

if not is_thumb_up(right_hand):

pyautogui.keyDown('s')

If your right thumb is down, press the S key (brake).
If your right thumb goes up, release the S key (stop braking).

↔️ Turn Left or Right (A and D keys)

if angle < -straight_threshold:

pyautogui.keyDown('a')

# release after tap_duration

Angle < -6: Press A (turn left)
Angle > +6: Press D (turn right)

These are short taps, so the car turns briefly and doesn't stay turning forever.

🧹 At the End

pyautogui.keyUp('w')

pyautogui.keyUp('s')

pyautogui.keyUp('a')

pyautogui.keyUp('d')

When you exit the program, it releases all keys to avoid the car getting stuck going forward or turning.

✅ Final Thoughts

🎮 What You Can Control

Step 4: Explaination of Code and Important Settings in Code Before Using It🔍 Explaining the Key Code Lines✅ Importing Toolsimport cv2 # To use webcam and show visualsimport mediapipe as mp # To detect handsimport math # To calculate anglesimport pyautogui # To press keyboard buttons automaticallyimport time # To handle timing for button presses

✅ Initial Settingshands = mphands.Hands(...)This initializes MediaPipe Hands to detect up to 2 hands.It finds where your hands are and tracks their movement.

✅ Detecting Angle Between Handsdef calculate_angle(x1, y1, x2, y2):...Calculates the angle between your left and right hands.This tells whether you're turning left, right, or going straight.

🧠 Important Settings :

🛠 turn_sensitivity = 0.01This was used in a previous version to make turning more or less sensitive.BUT in your current code, It's not used anymore, so we can remove it or ignore it.

🔁 straight_threshold = 6This value decides how much your hand angle must change before it counts as turning.If the angle is less than -6, It means turn left (A key).If the angle is more than +6, It means turn right (D key).If the angle is between -6 and +6, It means go straight (do nothing).This helps avoid small hand shakes being treated as turns.

⏱️ tap_duration = 0.01This sets how quickly to release the A or D key after tapping it.0.01 means the key is held down for just 0.01 seconds (very short).This makes the car turn in small steps, so you don’t turn too much with a small hand movement.If you want smoother, longer turns, increase this value to something like 0.05.

🖐️ Detecting Thumb Up or Downdef is_thumb_up(hand_landmarks):return hand_landmarks.landmark[4].y < hand_landmarks.landmark[3].yChecks if the tip of the thumb is above the second joint (on the screen).This is used to detect:Left thumb up = stop acceleratingRight thumb up = stop braking🚗 Controls Using Hands

🟢 Accelerate (W key)if not is_thumb_up(left_hand):pyautogui.keyDown('w')If your left thumb is down, press the W key (accelerate).If your left thumb goes up, release the W key (stop).

🔴 Brake (S key)if not is_thumb_up(right_hand):pyautogui.keyDown('s')If your right thumb is down, press the S key (brake).If your right thumb goes up, release the S key (stop braking).

↔️ Turn Left or Right (A and D keys)if angle < -straight_threshold:pyautogui.keyDown('a')# release after tap_durationAngle < -6: Press A (turn left)Angle > +6: Press D (turn right)These are short taps, so the car turns briefly and doesn't stay turning forever.

🧹 At the Endpyautogui.keyUp('w')pyautogui.keyUp('s')pyautogui.keyUp('a')pyautogui.keyUp('d')When you exit the program, it releases all keys to avoid the car getting stuck going forward or turning.✅ Final Thoughts🎮 What You Can Control

Step 5: Important Settings in Code Before Using It

🧠 Important Settings :

🛠 turn_sensitivity = 0.01

This was used in a previous version to make turning more or less sensitive.
BUT in your current code, it's not used anymore, so we can remove it or ignore it.

🔁 straight_threshold = 6

This value decides how much your hand angle must change before it counts as turning.

If the angle is less than -6, it means turn left (A key).
If the angle is more than +6, it means turn right (D key).
If the angle is between -6 and +6, it means go straight (do nothing).

This helps avoid small hand shakes being treated as turns.

⏱️ tap_duration = 0.01

This sets how quickly to release the A or D key after tapping it.

0.01 means the key is held down for just 0.01 seconds (very short).
This makes the car turn in small steps, so you don’t turn too much with a small hand movement.

If you want smoother, longer turns, increase this value to something like 0.05

Code!!

import cv2
import mediapipe as mp
import math
import pyautogui
import time

mp_drawing = mp.solutions.drawing_utils
mphands = mp.solutions.hands

cap = cv2.VideoCapture(0)
hands = mphands.Hands(static_image_mode=False, max_num_hands=2, min_detection_confidence=0.7, min_tracking_confidence=0.5)

def calculate_angle(x1, y1, x2, y2):
    angle = math.degrees(math.atan2(y2 - y1, x2 - x1))
    if angle < 0:
        angle += 360
    if angle > 180:
        angle -= 360
    return angle

def is_thumb_up(hand_landmarks):
    return hand_landmarks.landmark[4].y < hand_landmarks.landmark[3].y

pressing_w = False
pressing_s = False

turn_sensitivity = 0.01
straight_threshold = 6
tap_duration = 0.01 # Still used, but for logic-based tap, not sleep

# Tap state for A and D keys
tap_a_time = 0
tap_d_time = 0
tap_in_progress_a = False
tap_in_progress_d = False

while True:
    ret, image = cap.read()
    if not ret:
        break

    image = cv2.resize(image, (480, 360))
    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    results = hands.process(image)
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)

    current_time = time.time()

    if results.multi_hand_landmarks and len(results.multi_hand_landmarks) == 2:
        hand1 = results.multi_hand_landmarks[0]
        hand2 = results.multi_hand_landmarks[1]

        if hand1.landmark[0].x < hand2.landmark[0].x:
            left_hand = hand1
            right_hand = hand2
        else:
            left_hand = hand2
            right_hand = hand1

        x1, y1 = left_hand.landmark[0].x * image.shape[1], left_hand.landmark[0].y * image.shape[0]
        x2, y2 = right_hand.landmark[0].x * image.shape[1], right_hand.landmark[0].y * image.shape[0]

        mp_drawing.draw_landmarks(image, left_hand, mphands.HAND_CONNECTIONS)
        mp_drawing.draw_landmarks(image, right_hand, mphands.HAND_CONNECTIONS)
        cv2.line(image, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)

        thumb1_x, thumb1_y = left_hand.landmark[4].x * image.shape[1], left_hand.landmark[4].y * image.shape[0]
        thumb2_x, thumb2_y = right_hand.landmark[4].x * image.shape[1], right_hand.landmark[4].y * image.shape[0]

        cv2.putText(image, "Accelerator", (int(thumb1_x), int(thumb1_y) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 1)
        cv2.putText(image, "Brake", (int(thumb2_x), int(thumb2_y) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 1)

        angle = calculate_angle(x1, y1, x2, y2)
        cv2.putText(image, f'{int(angle)} degrees', (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 0, 0), 1)

        # Tap simulation for left and right
        if angle < -straight_threshold and not tap_in_progress_a:
            pyautogui.keyDown('a')
            tap_a_time = current_time
            tap_in_progress_a = True
            cv2.putText(image, "Left", (10, 210), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)
        elif angle > straight_threshold and not tap_in_progress_d:
            pyautogui.keyDown('d')
            tap_d_time = current_time
            tap_in_progress_d = True
            cv2.putText(image, "Right", (10, 210), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)

        # Release A key after tap_duration
        if tap_in_progress_a and (current_time - tap_a_time > tap_duration):
            pyautogui.keyUp('a')
            tap_in_progress_a = False

        # Release D key after tap_duration
        if tap_in_progress_d and (current_time - tap_d_time > tap_duration):
            pyautogui.keyUp('d')
            tap_in_progress_d = False

        # Acceleration
        if not is_thumb_up(left_hand):
            if not pressing_w:
                pyautogui.keyDown('w')
                pressing_w = True
        else:
            if pressing_w:
                pyautogui.keyUp('w')
                pressing_w = False

        # Braking
        if not is_thumb_up(right_hand):
            if not pressing_s:
                pyautogui.keyDown('s')
                pressing_s = True
        else:
            if pressing_s:
                pyautogui.keyUp('s')
                pressing_s = False

    cv2.imshow('Hand Tracker', image)
    if cv2.waitKey(1) & 0xFF == 27:
        break

if pressing_w:
    pyautogui.keyUp('w')
if pressing_s:
    pyautogui.keyUp('s')
pyautogui.keyUp('a')
pyautogui.keyUp('d')

cap.release()
cv2.destroyAllWindows()

Credits

Loyd Lobo

3 projects • 14 followers

Hello!!! I like to innovate,think and fabricate... i have an youtube channel!! u can see my projects!!

Embed the widget on your own site

I Turned My Hands Into a Virtual Steering Wheel!!

I Turned My Hands Into a Virtual Steering Wheel!!

Things used in this project

Hardware components

Software apps and online services

Story

What Is This Project?

Supplies

Step 1: Install Python

Step 2: Install the Libraries

What Are These Libraries?

Step 3: Copy the Code

this took me days to figuring out and I am giving you the code for free!!!

Step 4: Explaination of Code and Important Settings in Code Before Using It

🔍 Explaining the Key Code Lines

✅ Importing Tools

✅ Initial Settings

✅ Detecting Angle Between Hands

🧠 Important Settings :

🛠 turn_sensitivity = 0.01

🔁 straight_threshold = 6

⏱️ tap_duration = 0.01

🖐️ Detecting Thumb Up or Down

🚗 Controls Using Hands

🟢 Accelerate (W key)

🔴 Brake (S key)

↔️ Turn Left or Right (A and D keys)

🧹 At the End

✅ Final Thoughts

🎮 What You Can Control

Step 5: Important Settings in Code Before Using It

🧠 Important Settings :

🛠 turn_sensitivity = 0.01

🔁 straight_threshold = 6

⏱️ tap_duration = 0.01

Code

Code!!

Credits

Loyd Lobo

Comments

Related channels and tags