Digilent Arty S7 Accelerometer-based Windows mouse

In a little bit more than a year ago I got my first FPGA, the Digilent CMOD S7. FPGAs are quite flexible and can be used as programmable logic devices, but also as microcontrollers, since the programmable logic can be used to build a microcontroller. The programmable logic is flexible enough to allow you to design your own microcontroller, but this would be very time consuming. There are many microcontroller architectures that are available for FPGAs, but 2 are very popular and well supported by Xilinx: Microblaze and the ARM Cortex-M.

As I only had experience with the lowest level side of FPGAs (Verilog), I thought it would be a good idea to learn a bit about how to build a Microblaze microcontroller and program it to read sensor data from a PmodNAV (A 9-axis motion sensor + barometer PMOD board).

I first build a Microblaze softcore and connected it to the PMOD block like this:

The designed microcontroller uses the FPGA internal memory, and connects 4 LEDs, 4 push buttons, Digilent's PmodNAV and the virtual serial port. After building the microcontroller, one can program it using Vitis, an Eclipse-based IDE.

While experimenting with an program example from a webinar of Adam Taylor, I noticed that my mouse pointer tended to randomly jump across the screen. At the beginning I was a bit annoyed as I thought I would have to buy a new mouse, but later I found that Windows was for some reason detecting the FPGA virtual serial port as a serial mouse. The mouse pointer issue gave me an idea: to use the FPGA accelerometer and buttons as a pointer device.

I began exploring serial mouse protocols, but I could not get Windows to detect the FPGA as a serial mouse. I ran into a few problems: I couldn't get any information on the communication protocol of the mouse detected when running the webinar example, and apparently Windows could recognize the serial mouse only when rebooting. After a few reboots I abandoned my attempts of trying make Windows detect the FPGA as a mouse, and decided to write my own Windows driver.

Using the example as template I modified it to read the FPGA buttons, turn the LEDs on when they get pressed, and increased the reading speed of the PmodNAV accelerometer. The button and accelerometer data was packaged in a 10 byte packet. Only the first byte of the packet has the bit 7 set to 1, the other bytes have the bit 7 set to 0. This is done to synch the packets at the receiving end. The X, Y, Z gyro coordinates were encoded as a 17 bits (1 sign bit + 16 fixed point bits) so each axis required 3 bytes, the button status was sent in a single byte.

The driver was programmed using C++ and the win32 library, it was not a low-level driver, just a console program that read the serial port and updated the mouse pointer every time it read a new packet.

After some tweaking and the addition of acceleration I was able to use the accelerometer-mouse to move windows, and press icons and buttons. The final result was not as easy to use as a mouse, but it was by no means terrible; I would say it was probably easier to use than my Thinkpad's pointing stick, but harder than its touchpad.

Here is a small video that shows the board being used as a complete mouse replacement to play a game: