3D Rendering with MEMS IMU Data Using Spresense

This project is using the Spresense main board with an inertial measurement unit to render a 3D object in real-time. The result is then presented on a display.

System overview

There are three general parts in this project:

• Spresense main board
• MEMS gyro/accelerometer (Bosch BMI160 two board options listed in Hardware components section)
• TFT display (generic ILI9340 320x240)

This application rotates the text "BMI160" which is the name of the inertial measurement sensor used. Initially the 2D dimensional coordinates of the letters are defined in a set of tables. There is one table for each object since each letter can consist of several objects. For example letter "B" has a contour object plus two hole objects that together form the character.

During the initialization, the 2D objects are copied to an offset to create an object with depth.

During the main loop the gyro data is read together with a precise time stamp. The raw gyro data from the BMI160 gyro is showing the angle speed. Since the delta time is the time between the current sample and the last read sample, the absolute angle value can be calculated by accumulating all the changes in rotation. This application is tracking the rotation in three axes.

Once the absolute angle is calculated, the 3D image is rendered by vector rotation in the room.

When the board is laying still the image is rotated back to it's initial state using a PID controller with only the proportional part being non zero. This gives the rotation a smooth feel.

Hardware

The hardware setup in the project is identical to the setup used in: Spresense camera stablisation project. Please refer to that project for the details.

Software setup

This project have been developed using the NuttX based Spresense SDK, which is more advanced than the Arduino IDE environment. The source files for this project are located in the folder spresense/examples/bmi160_rotation in the build environment. The Spresense SDK is supported under Ubuntu on a PC.

Step by step instructions to build and flash the software:

$ git clone --recursive git@github.com:TE-KarlKomierowski/spresense.git
$ cd spresense
$ git checkout bmi_demo
$ cd sdk/
$ tools/config.py -k release
$ make buildkernel
$ tools/config.py examples/bmi160_rotation
$ make
$ tools/flash.sh -c /dev/ttyUSB[x] -b 1152000 nuttx.spk #Change [x]

In the default configuration the TFT display is configured to be connected directly to the main board. This can be changed to connect to the extension board instead. This is done via menuconfig "make menuconfig" under Board Configuration->LCD SPI connection (Extension board: SPI4).

The serial output from the board will contain VT-100 terminal commands to make the data possible to read for a human. Minicom can be configured to support VT-100 terminal emulation. Start minicom and type Ctrl+z and then type T to enter the terminal settings. Pressing A will change Terminal Emulation. When you selected VT102 you should find the output data from Spresense stable and not scrolling.

Conclusion and potential future features

The 3D image have been manually defined which is very time consuming. It would be very convenient to create a tool that could convert a common CAD model format into a format that could be interpreted by Spresense, maybe something like STL or STEP file format. FreeCAD can export solids to a JSON format which is pure human readable text. This format seems to be easy to parse to create an 3D object for Spresense.