Universal Touch-Free Keypad Adaptor for Existing Keypads

Idea

Nowadays we are all facing the Covid-19 pandemic; we know that there are multiple ways to get infected and one of these is by touching someone who is infected or by touching something that has been touched by someone who is infected. In fact the the virus resists several hours on certain surfaces.

During our daily live we touch many objects that had been touched by others before. When we get into an elevator, for example, we have to press a button to select the desired floor.

This project is taught for all those machines that relay on a traditional keypad (elevators self-service petrol station, public water dispenser), where the user has to physically touch buttons of a keypad, exposing himself and others to the risk of getting infected. This adaptor can be configured for every keypad layout and than it has to be mounted on top of the traditional keypad. It provides a touch-free interface (a matrix of Infrared sensors) where the user does not need to touch the device because the presence of his finger is detected when it is 2-3 cm away; then the adaptor will take care of pressing the physical button. The other type of interaction is through bluetooth: in this case the user has to connect his phone to the adaptor by scanning the associated QR code and then he can interact with the keypad through the application, which reproduces on the display a digital copy of the original keypad. If you press a button from the application, the adaptor will press that button on the keypad of the machine.

The way this project could help preventing viruses spreading is that it removes the contact between the machine and the user.

This project can also be seen as a "bridge" from the world of today where we still need to physically interact with machines, to a possible world of tomorrow where every interaction is touch-less, maybe even machines controlled with our mind.

Block diagram

The main controller is an Arduino MKR 1010 WiFi, which takes care of communicating with the application on the phone through BLE and reading the IR Matrix (the touch-free interface). It also communicates with the Arduino Pro Micro through I2C.

The secondary Controller is the Arduino Pro Micro. This little controller handles the motors and the encoders. Its job is to receive the target position data from the main controller and then move the axes in order to reach those target positions.

the Application has been developed using Flutter, a framework that allow to write application in Dart and then to generate the App file for many different Operating Systems (IOS, OSX, Android, etc.)

Hardware

The Hardware has been developed using Fusion 360. It consists of a 3-axis system: the axes X and Y are moved by two DC brushed motors, and the Z axis I moved by a stepper motor. To the servo motor is attached the tip that physically presses the button on the keypad.

All the hardware, except the stainless steel bar, screws and bolts, motors etc. has been 3D printed using PLA.

The hardware can be easily assembled by looking at the Fusion360 design. https://a360.co/2YJQkPy this is the link for viewing the design in the browser and for downloading

Axes Encoders

The encoders of the X axis and Y axis are integrated in the structure. In particular on every gear support there are 2 IR Emitters and 2 Receivers and the gears have gaps that when it is rotating, produce an output from one IR Receivers which is 90 degrees out of phase respect to the other.

The Z axis does not need of an encoder since it is already integrated in the servomotor.

Bluetooth Connection

To connect the application with de keypad adaptor, the user only has to scan the associated QR code. The way the connection works is as follow: when the BLE module of the adaptor advertises the other devices his presence, it sends some data, in particular what we care about is the manufacturer data. Instead of real manufacturer data, the keypad sends a ID code which is the same ID generated by the QR code when read. The application starts scanning for all the surrounding bluetooth modules and creates a list of them. Once a QR code of a valid ID is scanned, the applications starts comparing the scanned ID with the discovered bluetooth devices manufacturer data. If it finds a device with the same ID, the application connects to it.

Keypad Configuration

The main goal of this project was to make the keypad configuration as easy as possible. In fact there are hundreds of different keypad layouts, and this adaptor is designed to work with any of them. The simplest way to configure the keypad that I could imagine was to take a picture of it. in fact this is how the basic of the configuration process. The full configuration procedure is explained in the video, the following is a brief description of the needed steps:

First step: apply the tag sticker 4-5 cm over the top of the keyboard. The tag acts as the reference point for the whole system.

Take a picture of the keypad: once the user takes a picture of the keypad, the image processing process starts. In particular a Gaussian blur filter is applied, then edge detection algorithm and at the end a close path search algorithm. The result is an image containing the boundaries of the keypad buttons.

Than the user has to select which of the closed-path boundaries detected are really buttons. In fact there could be other close path geometries in the image. After selecting a path the label of the button must be provided.

After selecting all the buttons, the configuration need to be saved by pressing the "Next" button on the top right corner and after double checking the layout on the "Save" button to apply the changes.

Infrared Sensors Matrix

The infrared sensor matrix is the touch-free interaction method. In fact this technology can detect a finger (or more generally an object) before then it touches the button. The infrared emitters and receivers are mounted in the holes on the top of the walls of the structure. On two adjacent sides of the matrix structure there are the IR emitters and on the other two sides there are the IR receivers.

The Matrix isn't directly controlled by the main controller (Arduino MKR 1010) because there are 10 emitters and 10 receivers that have to be controlled and the would take a lot of GPIO's. Instead there are two circuits: one for controlling the IR Emitters and the other for controlling the receivers. The first one is composed of two 3-8 channels decoder and the second consists in two 8-3 channels demultiplexer. This greatly reduce the number of GPIO that are needed to control the matrix to only 5 (4 output to select which pair of IR activate and 1 for reading the value of the selected pair receiver).

Store Keypad Layout Configuration

The keypad configuration needs to be permanently stored in a memory, so that it can be loaded at startup. Unfortunately the Arduino MKR 1010's microcontroller does not have an integrated EEPROM memory, so I decided to use an external one which communicate with the main controller through the I2C bus. The EEPROM IC is the STMicroelectronics 24C04WP. It has 4kbit (512 Bytes) of available memory which are enough to store up to 45 buttons.

Why two micro controllers instead of one?

The choice of dividing the control of BLE and the movement though motors and encoders comes from the first tests. In fact the original setup was with all managed by the Arduino MKR 1010. The problem with that configuration was that some BLE interrupts routines were too slow compared to the time between two consecutive encoders interrupt routines; so some encoder ISR was missing resulting in a misalignment of the axes. The separate controller instead has the only job of taking care of the motor and the encoders, thus being way more accurate

Circuit implementation

The circuit has been implemented on a proto-board.

wait, but where is the Arduino Micro?? Well, for some space constraints the Arduino Micro is right underneath the Arduino MKR!

I really hope that you enjoyed my project and you found it interesting or useful. I also hope that in the future this project could help fighting viruses spreading.

Thank You for Reading!