Portable Thermal Imaging with the MiniZed

Introduction

I have previously looked at the top end of the low cost thermal imaging solution with the use of the FLIR Lepton coupled to HDMI displays or the Avnet touch screen display,

Recently I came across two components from pimoroni which I thought would be very interesting to make a compact portable thermal imaging solution based around the MiniZed. These are the 1.3 Inch Colour TFT LCD which has an SPI interface and the MLX90640 IR Thermal camera which provides 32 pixels by 24 lines thermal image.

Both of these can be made to connect to the MiniZed with a little thought and would produce a simple low cost thermal solution.

Connecting the Thermal Camera and LCD

The thermal camera uses a I2C interface, while the LCD uses an SPI interface, both of these can be easily implemented with the MiniZed PS. However connecting the devices required a little thought as neither can be directly plugged into the board.

This is due to the pin out of the I2C camera and the LCD, none of the ground and power pins align directly with the power and ground pins on the MiniZed. Both peripherals will work from 3v3.

I want to connect the thermal imaging camera to the Pmod interface and the LCD screen on the shield connector. This allows the ground and signal pins to align with those available on the MiniZed.

For the power interfaces I will connect them via a flying leads to appropriate power connections on the Pmod and Shield connectors.

This means that we need to solder on the connectors in different directions, the ground and signal connections down from the bottom of the board while the power pin points up from the top of the board.

Vivado Design

The design of the project is pretty simple in Vivado, we need to first create a new project which targets the MiniZed.

Select RTL as the project type

Select the MiniZed

Once completed select finish

Create a new block design and add to it the Zynq processing system

Run the block automation to configure the project

Reconfigure the Zynq PS to have both the I2c and SPI as Extended MIO. This makes the signals available to the programmable logic such that we can route them to the Pmod and Shield interfaces.

Once the EMIO are available in the block diagram we need to make the SPI and IIC signals external

Along with the SPI the LCD also uses two additional signals BL which is the backlight of the LCD and DC which indicates if the SPI transaction contains command information or data information.

To ensure the signals are routed to the interfaces required we need to define a constraints file.

set_property PACKAGE_PIN P11 [get_ports {BL[0]}]
set_property PACKAGE_PIN R11 [get_ports {DC[0]}]


set_property PACKAGE_PIN M11 [get_ports SPI0_MOSI_O_0]
set_property PACKAGE_PIN M10 [get_ports SPI0_SCLK_O_0]
set_property PACKAGE_PIN N9 [get_ports SPI0_SS_O_0]

set_property IOSTANDARD LVCMOS33 [get_ports {BL[0]}]
set_property IOSTANDARD LVCMOS33 [get_ports {DC[0]}]
set_property IOSTANDARD LVCMOS33 [get_ports SPI0_MOSI_O_0]
set_property IOSTANDARD LVCMOS33 [get_ports SPI0_SCLK_O_0]
set_property IOSTANDARD LVCMOS33 [get_ports SPI0_SS_O_0]

set_property PACKAGE_PIN N11 [get_ports IIC_0_scl_io]
set_property PACKAGE_PIN P14 [get_ports IIC_0_sda_io]
set_property IOSTANDARD LVCMOS33 [get_ports IIC_0_scl_io]
set_property IOSTANDARD LVCMOS33 [get_ports IIC_0_sda_io]


set_property SLEW FAST [get_ports IIC_0_scl_io]

Software Development

The software development on this application could be pretty in depth as both the LCD and the Temp Sensor require quite considerable software frame works. Rather helpfully there are two software repositories which we can fork and modify to work with the Zynq. The main modification we need to make it to map in the correct I2C or SPI drivers at the bottom of the provided technology stacks.

The first library is the Melexis mlx90640-library which is provided by the thermal imager manufacturer. This is designed to work with a Arm Mbed type device but the software is nicely structured and documented.

https://github.com/melexis/mlx90640-library

The software is structured such that the low level I2C commands are contained in one of two files either I2C Driver or SWI2C Driver. The SWI2C driver implements a I2C driver by bit banging ports and is not needed for this application. The I2C Driver includes a I2C drivers for the Mbed type devices. It is these drivers we need to update to work with the Zynq I2C driver. In effect we are swapping out the mbed I2C driver calls with the Zynq I2C driver calls.

This allows the API file to be able to used unchanged with just the drivers replaced. Note on reads the MLX90640 requires the I2C driver to perform a repeated start and it must also be the only device on the I2C bus.

Original MBed I2C interface

int MLX90640_I2CRead(uint8_t slaveAddr, uint16_t startAddress, uint16_t nMemAddressRead, uint16_t *data)
{
    uint8_t sa;                           
    int ack = 0;                               
    int cnt = 0;
    int i = 0;
    char cmd[2] = {0,0};
    char i2cData[1664] = {0};
    uint16_t *p;
    
    p = data;
    sa = (slaveAddr << 1);
    cmd[0] = startAddress >> 8;
    cmd[1] = startAddress & 0x00FF;
    
    i2c.stop();
    wait_us(5);    
    ack = i2c.write(sa, cmd, 2, 1);
    
    if (ack != 0x00)
    {
        return -1;
    }
             
    sa = sa | 0x01;
    ack = i2c.read(sa, i2cData, 2*nMemAddressRead, 0);
    
    if (ack != 0x00)
    {
        return -1; 
    }          
    i2c.stop();   
    
    for(cnt=0; cnt < nMemAddressRead; cnt++)
    {
        i = cnt << 1;
        *p++ = (uint16_t)i2cData[i]*256 + (uint16_t)i2cData[i+1];
    }
    
    return 0;   
}

Updated I2C read function with the Zynq I2C interface replacing the original

int MLX90640_I2CRead(uint8_t slaveAddr,uint16_t startAddress, uint16_t nMemAddressRead, uint16_t *data)
{

    int cnt = 0;
    int i = 0;
    u8 cmd[2] = {0,0};
    u8 i2cData[1664] = {0};
    uint16_t *p;

    p = data;
    cmd[0] = startAddress >> 8;
    cmd[1] = startAddress & 0x00FF;
    XIicPs_SetOptions(&Iic,XIICPS_REP_START_OPTION);
	XIicPs_MasterSendPolled(&Iic, cmd,2, IIC_SLAVE_ADDR);
	//while (XIicPs_BusIsBusy(&Iic)) {
		/* NOP */
	//}

	XIicPs_MasterRecvPolled(&Iic, i2cData,2*nMemAddressRead, IIC_SLAVE_ADDR);
	while (XIicPs_BusIsBusy(&Iic)) {
		/* NOP */
	}
	XIicPs_ClearOptions(&Iic,XIICPS_REP_START_OPTION);

    for(cnt=0; cnt < nMemAddressRead; cnt++)
    {
        i = cnt << 1;
        *p++ = (uint16_t)i2cData[i]*256 + (uint16_t)i2cData[i+1];
    }

    return 0;
}

This allows us to make functions calls to the API which will configure and read data from the MLX90640 sensor.

To be able to work with the display we can also adopt a STM32 based driver for the ST7789V driver which is used by the 1.3" LCD.

This library is available here https://github.com/Floyd-Fish/ST7789-STM32

Again we need to convert the lower level SPI drivers from the HAL layer used by the STM32 to the Zynq SPI.

Original STM32 HAL layer

static void ST7789_WriteCommand(uint8_t cmd)
{
ST7789_Select();
ST7789_DC_Clr();
HAL_SPI_Transmit(&ST7789_SPI_PORT, &cmd, sizeof(cmd), HAL_MAX_DELAY);
ST7789_UnSelect();
}

Updated Zynq Command

static void ST7789_WriteCommand(uint8_t cmd)
{

	XGpioPs_WritePin(&Gpio, DC, 0);
	XSpiPs_SetSlaveSelect(&SpiInstance_EMIO, 0x00);
	XSpiPs_PolledTransfer(&SpiInstance_EMIO, &cmd, NULL, sizeof(cmd));

}

With the SPI and I2C drivers updated the drivers have to be tested to ensure the LCD and MLX90640 can work as expected.

Connecting a Oscilloscope to the Pmod Interface allows the scoping of the I2C and SPI lines

The main software application performs the following steps to configure the system.

• Configure the Zynq Peripherals
• Initialize the Display - Set the background color to all black
• Grab the EEPROM Parameters from the MLX90640
• Extract the parameters from the EEPROM Dump

Inside the main loop the following functions are performed

• Get a frame of Data from the MLX90640
• Get the temperature of the Array
• Calculate the Temperature for each element in the array
• Output each array element over RS232 for analysis and double checking
• Convert the array temperature to a RGB color
• Display on the LCD Display

As the resolution of the display is only 32 by 24 I used a simple scaling algorithm to scale up the display to the 1.3" LCD.

MLX90640_DumpEE (0x33, eeMLX90640);
	MLX90640_ExtractParameters(eeMLX90640, &mlx90640);
    while(1){


    	MLX90640_GetFrameData(0x33,mlx90640Frame);
    	Ta = MLX90640_GetTa (mlx90640Frame, &mlx90640) - TA_SHIFT;
    	MLX90640_CalculateTo(mlx90640Frame, &mlx90640, emissivity, Ta, mlx90640To);
    	for(int i =0; i <24 ;i++){
    		for (int y= 0; y < 32; y++){
    			printf(" %.2f ", mlx90640To[y+(i*32)]);
    		}
    		printf("\n\r");
    	}
    	ST7789_Fill_Display(&mlx90640To);
    }

Scene Temperatures for each element in the display

We can convert this temperature into a color range for display to show the heat in the image.

How we do this is we create a color map which contains 255 different colors. We then scale the temperatures in the image to fit one of the 255 colors and display it on the screen.

Running the algorithm and imaging hot elements shows the following image

When the imager is looking at ambient temperature

Imaging a cup of tea.

Wrap Up

This has been a fun project which shows how the MiniZed can be used to create a low cost, portable thermal imaging solution.

You can find the project files here https://github.com/ATaylorCEngFIET/MiniZed_Thermal