Motion controller for people with limited arm function

Introduction:

This project shows you how to build a motion controller used for playing video games. This is specifically aimed towards people with physical disabilities such as muscular dystrophy or some other condition which limits arm movement or any disability that limits sitting in upright position on a chair. Take a look at the videos below to see it in action:

People with physical disabilities often have difficulties using conventional gaming peripherals. It might be due to limited arm function or some other disability that doesn't allow them to sit at the desk and play games like other people do. One very important peripheral PC gamers use is the mouse. However, mouse requires a flat surface to work properly and requires user to sit on a chair in upright position and extend arms all the time while gaming. This induces an unnecessary disadvantage for the people that have limited arm movement or fatigue or cannot sit in an upright position. Therefore, I decided to make a controller that is intuitive to use and can effectively replace mouse for those individuals. Just like a typical mouse, sensitivity can also be adjusted based on specific needs. Also, a person's hand needs not to be in a specific position or orientation to use the device which mitigates disadvantages of a typical mouse. People with limited hand movement can also use this by adjusting sensitivity to their specific needs.

How to use it?

Plug the device like any other mouse, OS will detect it as a peripheral. Now, when you want to enable the mouse, just touch the button on the top of the device and the cursor will move when device will detect motion. The touch button is useful because not all the time you need to control the mouse cursor and unintended motion at these times can cause mouse cursor to move. This effectively mitigates this scenario.

Hardware used:

M5stack's atom s3 - a very tiny and compact development board based on ESP32-S3 microcontroller was used as main controller to build the project.

TTP223 touch sensing module was also attached on the board for accessible touch input. M5stack board was perfect for this project as it is a very tiny board with an inbuilt IMU sensor (MPU 6886).

TTP223 is a simple touch sensing module with single digital I/O pin for reading touch input. This sensor was attached on the front side of m5stack atom-s3 development kit using glue. The jumper wires were used to interface the sensor by connecting to I/O headers exposed on the bottom side of the development kit.

Theory and working:

M5stack atom-s3 have inbuilt MPU 6886 6-Degrees of Freedom Inertial Measurement Unit (IMU). This means that it has capability to measure linear acceleration as well as angular velocity on all 3 axes. This means that we can utilize angular velocity measurement to control cursor on screen when we rotate IMU around an axis. This rotation naturally occurs when user points towards a section of screen and therefore we can convert these readings into cursor position on the screen. This results in a motion controller that can be used as a controller in video games. The device is also equipped with a push button that functions as a mouse button. Finally, at the top of device, a touch sensor is attached. This touch button acts as a button to enable / disable cursor movement due to motion. The purpose of enabling / disabling cursor movement is to mitigate effects of gyro drift and only allowing to move mouse w.r.t moving when user wants. This ensures that user's hands are never in an uncomfortable position while playing a game.

Steps to reproduce the project:

• Interface the TTP223 sensor according to the schematic provided using jumper wires and attach sensor on front of M5stack board like this:

• Download Flash Download Tool from the following link and extract files: https://www.espressif.com/en/support/download/other-tools
• Download the binary files provided in this project.
• Open Flash Download Tool, Select Chip Type: ESP32-S3. Leave all other settings to default. Load binary files provided, set following offset values. Carefully match offsets to file names as provided in the image below. Also select all three files to flash. If everything is fine, all three sections will be green. Finally, select your COM port on which M5Stack board is attached and flash:

• After successfully flashed, press reset button on M5stack once. You should see your cursor moving if you place your finger on TTP223 touch button and move the controller. You are done if you just needed to use the device with provided firmware. However, if you want to build the code or want to do changes in the code, you need to install ESP-IDF v5.1 along with Vscode. If you want to build code yourself, you can skip previous steps of using flash download tool.
• Details on installing and configuring ESP-IDF is provided on following link: https://docs.espressif.com/projects/esp-idf/en/v5.1.2/esp32/get-started/index.html
• Once you have ESP-IDF 5.1 installed and configured, clone the project from following github repository: https://github.com/yahya-khalid/motion_controller
• You first need to configure the project by opening ESP-IDF terminal into root directory of the project and pasting following command:

idf.py menuconfig

• Following terminal will come up, Select Motion Controller Configuration and Enable Gyro Mouse:

• In the following menu, I have added options to configure buttons, touch and sensitivity from the menu. However, default values work as it is. Only change if you really want to change hardware configuration or sensitivity.

• Exit the menu and build the code using following command:

idf.py build

• To flash the code, paste the following command with your desired COM port (replace x with your COM port number):

idf.py -p COMx flash

Lastly, press M5stack board reset button and it should now be detected as a peripheral in windows drivers.

What's next?

I intend to pursue this project if enough interest is shown. Currently, it's in very initial stages of prototyping (First commit to github infact). Following are the ways it can be improved:

• Software support for configuring macros and key bindings.
• Software configuration for different types of I/Os such as joysticks, physical buttons, touch buttons, force and pressure sensitive sensors, IMU e.t.c.
• Sensor fusion of accelerometer and gyro for better results and drift compensation. Also adding motion based gestures.
• An integrated display support for configuration on runtime.