FPGA Based Image System Dual Cameras SP701

Introduction

One of the great benefits of the FPGAs is that we can implement parallel image processing streams. To do this we do not need an expensive FPGA, we can use one from the cost optimized range such as Spartan 7 or Artix 7. For this project I am going to show how we can generate a simple image processing application which runs two cameras in parallel.

To do this I will be using a SP701 development board and the Digilent PCAM Adapter. The PCAM adapter provides interfaces for up to four PCAMS. However, when we take into account the need for a MicroBlaze processor and MIG for the frame buffers in DDR3 we do not quite have enough logic to support four PCAM streams running concurrently at 1080P 60 FPS.

The SP701 is a great development board with a Spartan 7 100 device and CSI and DSI interfaces along with Dual Ethernet and Pmod. The board also provides a Low Pin Count FMC connector to which we will connect the PCam Adapter.

Vivado

The design, to get system up and running quickly we are going to start with a example project from Xilinx. To open the reference project we need to first create a project targeting the SP701

Once the project has been opened, create a new block diagram

Once the block diagram is opened, add to the block diagram a MIPI CSI2 IP block

To open the design right click on the CSI2 block an select Open IP Example design.

It is to this reference project that we are going to be working with. The first thing to do is to remove the DSI output path. This will free up logic resources in the FPGA, for our image processing platform.

The next step is to add in the following elements to create a second image processing pipe line.

• CSI2 IP Block
• Register Slices & concatenation
• Sensor Demosaic
• VDMA
• AXI Switch

The completed design should look like the below

This second image processing pipeline is identical to the first with the exception of a setting in the CSI2 IP

Original CSI2 IP Settings

Settings in the added CSI2 IP

DMA Memory settings

Sensor Demosaic Settings

AXI4 Stream Switch

The clocking has different upstream and downstream clocks

Before we can build the project we need to move the I2C pins used to configure the MIPI CSI processor on bard to the FMC connection. We also need to define the Inputs for the MIPI pins also in the FMC.

set_property PACKAGE_PIN C4 [get_ports {mipi_phy_if_0_data_lp_p[1]}]
set_property PACKAGE_PIN B4 [get_ports {mipi_phy_if_0_data_lp_n[1]}]
set_property PACKAGE_PIN B10 [get_ports {mipi_phy_if_0_data_lp_p[0]}]
set_property PACKAGE_PIN A10 [get_ports {mipi_phy_if_0_data_lp_n[0]}]
set_property PACKAGE_PIN E11 [get_ports mipi_phy_if_0_clk_lp_p]
set_property PACKAGE_PIN E10 [get_ports mipi_phy_if_0_clk_lp_n]
set_property PACKAGE_PIN B5 [get_ports {mipi_phy_if_0_data_hs_p[0]}]
set_property PACKAGE_PIN A5 [get_ports {mipi_phy_if_0_data_hs_n[0]}]
set_property PACKAGE_PIN C12 [get_ports {mipi_phy_if_0_data_hs_p[1]}]
set_property PACKAGE_PIN C11 [get_ports {mipi_phy_if_0_data_hs_n[1]}]
set_property PACKAGE_PIN B7 [get_ports mipi_phy_if_0_clk_hs_p]
set_property PACKAGE_PIN B6 [get_ports mipi_phy_if_0_clk_hs_n]


set_property PACKAGE_PIN C9 [get_ports {mipi_phy_if_1_data_lp_p[1]}]
set_property PACKAGE_PIN C8 [get_ports {mipi_phy_if_1_data_lp_n[1]}]
set_property PACKAGE_PIN C3 [get_ports {mipi_phy_if_1_data_lp_p[0]}]
set_property PACKAGE_PIN C2 [get_ports {mipi_phy_if_1_data_lp_n[0]}]
set_property PACKAGE_PIN F8 [get_ports mipi_phy_if_1_clk_lp_p]
set_property PACKAGE_PIN E8 [get_ports mipi_phy_if_1_clk_lp_n]
set_property PACKAGE_PIN B12 [get_ports {mipi_phy_if_1_data_hs_p[0]}]
set_property PACKAGE_PIN B11 [get_ports {mipi_phy_if_1_data_hs_n[0]}]
set_property PACKAGE_PIN D11 [get_ports {mipi_phy_if_1_data_hs_p[1]}]
set_property PACKAGE_PIN D10 [get_ports {mipi_phy_if_1_data_hs_n[1]}]
set_property PACKAGE_PIN A8 [get_ports mipi_phy_if_1_clk_hs_p]
set_property PACKAGE_PIN A7 [get_ports mipi_phy_if_1_clk_hs_n]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_p[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_n[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_p[0]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_n[0]}]

set_property DIFF_TERM TRUE [get_ports mipi_phy_if_0_clk_hs_p]
set_property DIFF_TERM TRUE [get_ports mipi_phy_if_0_clk_hs_n]


set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_p[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_n[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_p[0]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_n[0]}]

set_property DIFF_TERM TRUE [get_ports mipi_phy_if_1_clk_hs_p]
set_property DIFF_TERM TRUE [get_ports mipi_phy_if_1_clk_hs_n]
#csi-gpio
#set_property PACKAGE_PIN AF12 [get_ports {GPIO_sensor_tri_o[0]}]
#set_property IOSTANDARD LVCMOS25 [get_ports {GPIO_sensor_tri_o[0]}]
#csi-iic1
set_property PACKAGE_PIN F17 [get_ports IIC_sensor_scl_io]
set_property IOSTANDARD LVCMOS25 [get_ports IIC_sensor_scl_io]
set_property PULLUP true [get_ports IIC_sensor_scl_io]
set_property PACKAGE_PIN F18 [get_ports IIC_sensor_sda_io]
set_property IOSTANDARD LVCMOS25 [get_ports IIC_sensor_sda_io]
set_property PULLUP true [get_ports IIC_sensor_sda_io]
set_property PACKAGE_PIN C4 [get_ports {mipi_phy_if_0_data_lp_p[1]}]
set_property PACKAGE_PIN B4 [get_ports {mipi_phy_if_0_data_lp_n[1]}]
set_property PACKAGE_PIN B10 [get_ports {mipi_phy_if_0_data_lp_p[0]}]
set_property PACKAGE_PIN A10 [get_ports {mipi_phy_if_0_data_lp_n[0]}]
set_property PACKAGE_PIN E11 [get_ports mipi_phy_if_0_clk_lp_p]
set_property PACKAGE_PIN E10 [get_ports mipi_phy_if_0_clk_lp_n]
set_property PACKAGE_PIN B5 [get_ports {mipi_phy_if_0_data_hs_p[0]}]
set_property PACKAGE_PIN A5 [get_ports {mipi_phy_if_0_data_hs_n[0]}]
set_property PACKAGE_PIN C12 [get_ports {mipi_phy_if_0_data_hs_p[1]}]
set_property PACKAGE_PIN C11 [get_ports {mipi_phy_if_0_data_hs_n[1]}]
set_property PACKAGE_PIN B7 [get_ports mipi_phy_if_0_clk_hs_p]
set_property PACKAGE_PIN B6 [get_ports mipi_phy_if_0_clk_hs_n]


set_property PACKAGE_PIN C9 [get_ports {mipi_phy_if_1_data_lp_p[1]}]
set_property PACKAGE_PIN C8 [get_ports {mipi_phy_if_1_data_lp_n[1]}]
set_property PACKAGE_PIN C3 [get_ports {mipi_phy_if_1_data_lp_p[0]}]
set_property PACKAGE_PIN C2 [get_ports {mipi_phy_if_1_data_lp_n[0]}]
set_property PACKAGE_PIN F8 [get_ports mipi_phy_if_1_clk_lp_p]
set_property PACKAGE_PIN E8 [get_ports mipi_phy_if_1_clk_lp_n]
set_property PACKAGE_PIN B12 [get_ports {mipi_phy_if_1_data_hs_p[0]}]
set_property PACKAGE_PIN B11 [get_ports {mipi_phy_if_1_data_hs_n[0]}]
set_property PACKAGE_PIN D11 [get_ports {mipi_phy_if_1_data_hs_p[1]}]
set_property PACKAGE_PIN D10 [get_ports {mipi_phy_if_1_data_hs_n[1]}]
set_property PACKAGE_PIN A8 [get_ports mipi_phy_if_1_clk_hs_p]
set_property PACKAGE_PIN A7 [get_ports mipi_phy_if_1_clk_hs_n]

set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_p[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_n[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_p[0]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_0_data_hs_n[0]}]

set_property DIFF_TERM TRUE [get_ports mipi_phy_if_0_clk_hs_p]
set_property DIFF_TERM TRUE [get_ports mipi_phy_if_0_clk_hs_n]


set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_p[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_n[1]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_p[0]}]
set_property DIFF_TERM TRUE [get_ports {mipi_phy_if_1_data_hs_n[0]}]

set_property DIFF_TERM TRUE [get_ports mipi_phy_if_1_clk_hs_p]
set_property DIFF_TERM TRUE [get_ports mipi_phy_if_1_clk_hs_n]

#csi-gpio
#set_property PACKAGE_PIN AF12 [get_ports {GPIO_sensor_tri_o[0]}]
#set_property IOSTANDARD LVCMOS25 [get_ports {GPIO_sensor_tri_o[0]}]

#csi-iic1
set_property PACKAGE_PIN F17 [get_ports IIC_sensor_scl_io]
set_property IOSTANDARD LVCMOS25 [get_ports IIC_sensor_scl_io]
set_property PULLUP true [get_ports IIC_sensor_scl_io]
set_property PACKAGE_PIN F18 [get_ports IIC_sensor_sda_io]
set_property IOSTANDARD LVCMOS25 [get_ports IIC_sensor_sda_io]
set_property PULLUP true [get_ports IIC_sensor_sda_io]

Once this is completed we are able to generate and build the project and export the XSA for software development.

The utilization of the device is below

Software Development

Once the XSA has been exported, we can create a new Vitis embedded project which and hello world application.

From the hello world application BSP settings we can import the MIPI CSI2 example project.

It is this project which we need to make some changes to.

The first is in setting the sensor up and communication with the sensor over IIC. The CSI2 on board has a direct I2C connection from the FPGA. The FMC however does not as such we much control a I2C switch to route the I2C to the FMC I2C connector. the PCAM also has a I2C switch which routes the I2C around the PCAMs as they all have the same address.

This can be changed in the sensor configuration function provided within fucntion_prototpye.c

This selects the Muxes prior to the configuration being run

extern int SensorPreConfig(int pcam5c_mode) {


  u32 Index, MaxIndex, MaxIndex1, MaxIndex2;
  int Status;
  SensorIicAddr = SENSOR_ADDRESS;

  u8 SP701mux_addr = 0x75;
    u8 SP701mux_ch = 0x40;

    u8 PCAM_FMC_addr = 0x70;
    u8 PCAM_FMC_ch = 0x01;


    Status = XIic_SetAddress(&IicAdapter, XII_ADDR_TO_SEND_TYPE, SP701mux_addr);
    if (Status != XST_SUCCESS) {
  	return XST_FAILURE;
    }

    WriteBuffer[0] = SP701mux_ch;
    Status = AdapterWriteData(1);
  	if (Status != XST_SUCCESS) {
  	  printf("sp701 mux failed\n\r");
  	  return XST_FAILURE;
  	}

    Status = XIic_SetAddress(&IicAdapter, XII_ADDR_TO_SEND_TYPE, PCAM_FMC_addr);
    if (Status != XST_SUCCESS) {
  	return XST_FAILURE;
    }

    WriteBuffer[0] = PCAM_FMC_ch;
    Status = AdapterWriteData(1);
  	if (Status != XST_SUCCESS) {
  	  printf("pcam mux failed\n\r");
  	  return XST_FAILURE;
  	}



  Status = XIic_SetAddress(&IicAdapter, XII_ADDR_TO_SEND_TYPE, SensorIicAddr);
  if (Status != XST_SUCCESS) {
	return XST_FAILURE;
  }


  WritetoReg(0x31, 0x03, 0x11);
  WritetoReg(0x30, 0x08, 0x82);

  Sensor_Delay();


  MaxIndex = length_sensor_pre;
  for(Index = 0; Index < (MaxIndex - 0); Index++)
  {
    WriteBuffer[0] = sensor_pre[Index].Address >> 8;
	WriteBuffer[1] = sensor_pre[Index].Address;
	WriteBuffer[2] = sensor_pre[Index].Data;

    Sensor_Delay();

	Status = AdapterWriteData(3);
	if (Status != XST_SUCCESS) {
	  return XST_FAILURE;
	}
  }


  WritetoReg(0x30, 0x08, 0x42);


  MaxIndex1 = length_pcam5c_mode1;

  for(Index = 0; Index < (MaxIndex1 - 0); Index++)
  {
    WriteBuffer[0] = pcam5c_mode1[Index].Address >> 8;
	WriteBuffer[1] = pcam5c_mode1[Index].Address;
	WriteBuffer[2] = pcam5c_mode1[Index].Data;

    Sensor_Delay();

	Status = AdapterWriteData(3);
	if (Status != XST_SUCCESS) {
	  return XST_FAILURE;
	}
  }


  WritetoReg(0x30, 0x08, 0x02);
  Sensor_Delay();
  WritetoReg(0x30, 0x08, 0x42);


  MaxIndex2 = length_sensor_list;

  for(Index = 0; Index < (MaxIndex2 - 0); Index++)
  {
    WriteBuffer[0] = sensor_list[Index].Address >> 8;
	WriteBuffer[1] = sensor_list[Index].Address;
	WriteBuffer[2] = sensor_list[Index].Data;

    Sensor_Delay();

	Status = AdapterWriteData(3);
	  if (Status != XST_SUCCESS) {
		return XST_FAILURE;
	  }
  }


  if(Status != XST_SUCCESS) {
    xil_printf("Error: in Writing entry status = %x \r\n", Status);
    return XST_FAILURE;
  }

  return XST_SUCCESS;

}

As we have added a second Demosaic we also need to update its configuration prototpye

int demosaic()
{
  demosaic_Config = XV_demosaic_LookupConfig(DEMOSAIC_DEVICE_ID);
  XV_demosaic_CfgInitialize(&InstancePtr, demosaic_Config,
		                           demosaic_Config->BaseAddress);
  XV_demosaic_Set_HwReg_width(&InstancePtr, 1920);
  XV_demosaic_Set_HwReg_height(&InstancePtr, 1080);
  XV_demosaic_Set_HwReg_bayer_phase(&InstancePtr, 0x3);
  XV_demosaic_EnableAutoRestart(&InstancePtr);
  XV_demosaic_Start(&InstancePtr);

  demosaic_Config1 = XV_demosaic_LookupConfig(DEMOSAIC_DEVICE1_ID);
  XV_demosaic_CfgInitialize(&InstancePtr1, demosaic_Config1,
		                           demosaic_Config1->BaseAddress);
  XV_demosaic_Set_HwReg_width(&InstancePtr1, 1920);
  XV_demosaic_Set_HwReg_height(&InstancePtr1, 1080);
  XV_demosaic_Set_HwReg_bayer_phase(&InstancePtr1, 0x3);
  XV_demosaic_EnableAutoRestart(&InstancePtr1);
  XV_demosaic_Start(&InstancePtr1);
  return XST_SUCCESS;

}

The final stage is to set up the second DMA, here care must be taken in the DDR3 to ensure frames do not over lap with each other.

int vdma_hdmi() {

  InitVprocSs_CSC(1);

  ResetVDMA();

  RunVDMA(&AxiVdma, XPAR_AXI_VDMA_0_DEVICE_ID, HORIZONTAL_RESOLUTION, \
		  VERTICAL_RESOLUTION, srcBuffer, FRAME_COUNTER, 0);

  RunVDMA(&AxiVdma1, XPAR_AXI_VDMA_1_DEVICE_ID, HORIZONTAL_RESOLUTION, \
		  VERTICAL_RESOLUTION, srcBuffer1, FRAME_COUNTER, 0);

  return XST_SUCCESS;

}

We also need to comment out any code which was used by functions such as DSI and Test Pattern Generator.

The main code also needs updating to control the AXI Switch under command of the serial port.

/******************************************************************************
* Copyright (C) 2018 - 2022 Xilinx, Inc.  All rights reserved.
* SPDX-License-Identifier: MIT
*******************************************************************************/

/*****************************************************************************/
/**
*
* @file xmipi_sp701_example.c
*
* <pre>
* MODIFICATION HISTORY:
*
* Ver   Who    Date     Changes
* ----- ------ -------- --------------------------------------------------
* X.XX  XX     YY/MM/DD
* 1.00  RHe    19/09/20 Initial release.
* </pre>
*
******************************************************************************/
/***************************** Include Files *********************************/

#include "xparameters.h"
#include "xiic.h"
#include "xil_exception.h"
#include "function_prototype.h"
#include "pcam_5C_cfgs.h"
#include "xstatus.h"
#include "sleep.h"
#include "xiic_l.h"
#include "xil_io.h"
#include "xil_types.h"
//#include "xv_tpg.h"
#include "xil_cache.h"
#include "stdio.h"
#include "xaxis_switch.h"



/************************** Constant Definitions *****************************/


#define PAGE_SIZE   16
#define XAXIS_SWITCH_DEVICE_ID		XPAR_AXIS_SWITCH_0_DEVICE_ID

#define IIC_BASE_ADDRESS	XPAR_IIC_2_BASEADDR

#define EEPROM_TEST_START_ADDRESS	0x80

#define IIC_SWITCH_ADDRESS 0x74
#define IIC_ADV7511_ADDRESS 0x39
//XV_tpg_Config		*tpg1_Config;XV_tpg_Config		*tpg1_Config;
//XV_tpg				tpg1;
//XV_tpg				tpg1;
typedef u8 AddressType;

typedef struct {
	u8 addr;
	u8 data;
	u8 init;
} HDMI_REG;

#define NUMBER_OF_HDMI_REGS  16
HDMI_REG hdmi_iic[NUMBER_OF_HDMI_REGS] = {
	{0x41, 0x00, 0x10},
	{0x98, 0x00, 0x03},
	{0x9A, 0x00, 0xE0},
	{0x9C, 0x00, 0x30},
	{0x9D, 0x00, 0x61},
	{0xA2, 0x00, 0xA4},
	{0xA3, 0x00, 0xA4},
	{0xE0, 0x00, 0xD0},
	{0xF9, 0x00, 0x00},
	{0x18, 0x00, 0xE7},
    {0x55, 0x00, 0x00},
    {0x56, 0x00, 0x28},
    {0xD6, 0x00, 0xC0},
    {0xAF, 0x00, 0x4},
	{0xF9, 0x00, 0x00}
};

u8 EepromIicAddr;		/* Variable for storing Eeprom IIC address */

int IicLowLevelDynEeprom();

u8 EepromReadByte(AddressType Address, u8 *BufferPtr, u8 ByteCount);
u8 EepromWriteByte(AddressType Address, u8 *BufferPtr, u8 ByteCount);



/****************i************ Type Definitions *******************************/

typedef u8 AddressType;

/************************** Variable Definitions *****************************/

extern XIic IicFmc, IicAdapter ;	/*  IIC device. */

//HDMI IIC
int IicLowLevelDynEeprom()
{
  u8 BytesRead;
  u32 StatusReg;
  u8 Index;
  int Status;
  u32 i;
  EepromIicAddr = IIC_SWITCH_ADDRESS;
  Status = XIic_DynInit(IIC_BASE_ADDRESS);
  if (Status != XST_SUCCESS) {
	return XST_FAILURE;
  }
  xil_printf("\r\nAfter XIic_DynInit\r\n");
  while (((StatusReg = XIic_ReadReg(IIC_BASE_ADDRESS,
				XIIC_SR_REG_OFFSET)) &
				(XIIC_SR_RX_FIFO_EMPTY_MASK |
				XIIC_SR_TX_FIFO_EMPTY_MASK |
				XIIC_SR_BUS_BUSY_MASK)) !=
				(XIIC_SR_RX_FIFO_EMPTY_MASK |
				XIIC_SR_TX_FIFO_EMPTY_MASK)) {

  }


  EepromIicAddr = IIC_ADV7511_ADDRESS;
  for ( Index = 0; Index < NUMBER_OF_HDMI_REGS; Index++)
  {
    EepromWriteByte(hdmi_iic[Index].addr, &hdmi_iic[Index].init, 1);
  }

  for ( Index = 0; Index < NUMBER_OF_HDMI_REGS; Index++)
  {
    BytesRead = EepromReadByte(hdmi_iic[Index].addr, &hdmi_iic[Index].data, 1);
    for(i=0;i<1000;i++) {};	// IIC delay
	if (BytesRead != 1) {
      return XST_FAILURE;
	}
  }


  return XST_SUCCESS;

}


/*****************************************************************************/
/**
* This function writes a buffer of bytes to the IIC serial EEPROM.
*
* @param	BufferPtr contains the address of the data to write.
* @param	ByteCount contains the number of bytes in the buffer to be
*		written. Note that this should not exceed the page size of the
*		EEPROM as noted by the constant PAGE_SIZE.
*
* @return	The number of bytes written, a value less than that which was
*		specified as an input indicates an error.
*
* @note		one.
*
******************************************************************************/
u8 EepromWriteByte(AddressType Address, u8 *BufferPtr, u8 ByteCount)
{
  u8 SentByteCount;
  u8 WriteBuffer[sizeof(Address) + PAGE_SIZE];
  u8 Index;

  /*
   * A temporary write buffer must be used which contains both the address
   * and the data to be written, put the address in first based upon the
   * size of the address for the EEPROM
   */
  if (sizeof(AddressType) == 2) {
	WriteBuffer[0] = (u8) (Address >> 8);
	WriteBuffer[1] = (u8) (Address);
  } else if (sizeof(AddressType) == 1) {
	WriteBuffer[0] = (u8) (Address);
	EepromIicAddr |= (EEPROM_TEST_START_ADDRESS >> 8) & 0x7;
  }

  /*
   * Put the data in the write buffer following the address.
   */
  for (Index = 0; Index < ByteCount; Index++) {
	WriteBuffer[sizeof(Address) + Index] = BufferPtr[Index];
  }

  /*
   * Write a page of data at the specified address to the EEPROM.
   */
  SentByteCount = XIic_DynSend(IIC_BASE_ADDRESS, EepromIicAddr,
				WriteBuffer, sizeof(Address) + ByteCount,
				XIIC_STOP);

  /*
   * Return the number of bytes written to the EEPROM.
   */
  return SentByteCount - sizeof(Address);

}


/******************************************************************************
*
* This function reads a number of bytes from the IIC serial EEPROM into a
* specified buffer.
*
* @param	BufferPtr contains the address of the data buffer to be filled.
* @param	ByteCount contains the number of bytes in the buffer to be read.
*		This value is constrained by the page size of the device such
*		that up to 64K may be read in one call.
*
* @return	The number of bytes read. A value less than the specified input
*		value indicates an error.
*
* @note		None.
*
******************************************************************************/
u8 EepromReadByte(AddressType Address, u8 *BufferPtr, u8 ByteCount)
{
  u8 ReceivedByteCount;
  u8 SentByteCount;
  u16 StatusReg;

  /*
   * Position the Read pointer to specific location in the EEPROM.
   */
  do {
	StatusReg = XIic_ReadReg(IIC_BASE_ADDRESS, XIIC_SR_REG_OFFSET);
    if (!(StatusReg & XIIC_SR_BUS_BUSY_MASK)) {
	  SentByteCount = XIic_DynSend(IIC_BASE_ADDRESS, EepromIicAddr,
					  (u8 *) &Address, sizeof(Address), XIIC_REPEATED_START);
    }

  } while (SentByteCount != sizeof(Address));
  /*
   * Receive the data.
   */
  ReceivedByteCount = XIic_DynRecv(IIC_BASE_ADDRESS, EepromIicAddr,
		                                          BufferPtr, ByteCount);

  /*
   * Return the number of bytes received from the EEPROM.
   */

  return ReceivedByteCount;

}


/*****************************************************************************/
/**
 *
 * Main function to initialize interop system and read data from AR0330 sensor

 * @param  None.
 *
 * @return
 *   - XST_SUCCESS if MIPI Interop was successful.
 *   - XST_FAILURE if MIPI Interop failed.
 *
 * @note   None.
 *
 ******************************************************************************/
int main() {
  int Status;
  int pcam5c_mode = 1;
  int usr_entry ,prev_sel;
  int default_input;
  int dsi_hdmi_select = 0;

  Xil_ICacheDisable();
  Xil_DCacheDisable();
  XAxis_Switch AxisSwitch;
  XAxis_Switch_Config *ASWConfig;

  ASWConfig = XAxisScr_LookupConfig(XAXIS_SWITCH_DEVICE_ID);
  XAxisScr_CfgInitialize(&AxisSwitch, ASWConfig,ASWConfig->BaseAddress);
  XAxisScr_RegUpdateDisable(&AxisSwitch);
  XAxisScr_MiPortDisableAll(&AxisSwitch);
  XAxisScr_MiPortEnable(&AxisSwitch, 0, 0);
  XAxisScr_RegUpdateEnable(&AxisSwitch);

  xil_printf("\n\r******************************************************\n\r");
  xil_printf("\n\r**           SP701 Example Design            **");

  Status = IicLowLevelDynEeprom();
  if (Status != XST_SUCCESS) {
    xil_printf("ADV7511 IIC programming FAILED\r\n");
    return XST_FAILURE;
  }
  xil_printf("ADV7511 IIC programming PASSED\r\n");


  //Initialize FMC, Adapter and Sensor IIC
  Status = InitIIC();
  if (Status != XST_SUCCESS) {
	xil_printf("\n\r IIC initialization Failed \n\r");
	return XST_FAILURE;
  }
  xil_printf("IIC Initializtion Done \n\r");

  //Initialize FMC Interrupt System
  Status = SetupFmcInterruptSystem(&IicFmc);
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rInterrupt System Initialization Failed \n\r");
    return XST_FAILURE;
  }
  xil_printf("FMC Interrupt System Initialization Done \n\r");

  //Set up IIC Interrupt Handlers
  SetupIICIntrHandlers();
  xil_printf("IIC Interrupt Handlers Setup Done \n\r");

  Status =  SetFmcIICAddress();
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rFMC IIC Address Setup Failed \n\r");
	return XST_FAILURE;
  }
  xil_printf("Fmc IIC Address Set\n\r");

  //Initialize Adapter Interrupt System
  Status = SetupAdapterInterruptSystem(&IicAdapter);
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rInterrupt System Initialization Failed \n\r");
    return XST_FAILURE;
  }
  xil_printf("Adapter Interrupt System Initialization Done \n\r");

  //Set Address of Adapter IIC
  Status =  SetAdapterIICAddress();
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rAdapter IIC Address Setup Failed \n\r");
	return XST_FAILURE;
  }
  xil_printf("Adapter IIC Address Set\n\r");

  Status = InitializeCsiRxSs();
  if (Status != XST_SUCCESS) {
    xil_printf("CSI Rx Ss Init failed status = %x.\r\n", Status);
	return XST_FAILURE;
  }


  dsi_hdmi_select = 0;
  //using default_input var to compare same option selection
  default_input = 1;
  //SetupDSI();
  resetIp();
  EnableCSI();
  GPIOSelect(dsi_hdmi_select);

  Status = demosaic();
  if (Status != XST_SUCCESS) {
	xil_printf("\n\rDemosaic Failed \n\r");
	return XST_FAILURE;
  }

  CamReset();

  //Preconifgure Sensor
  Status = SensorPreConfig(pcam5c_mode);
  if (Status != XST_SUCCESS) {
	xil_printf("\n\rSensor PreConfiguration Failed \n\r");
	return XST_FAILURE;
  }
  xil_printf("\n\rSensor 1 is PreConfigured\n\r");
  WritetoReg(0x30, 0x08, 0x02);

  //Preconifgure Sensor
  Status = SensorPreConfig1(pcam5c_mode);
  if (Status != XST_SUCCESS) {
	xil_printf("\n\rSensor PreConfiguration Failed \n\r");
	return XST_FAILURE;
  }
  xil_printf("\n\rSensor 2 is PreConfigured\n\r");
  WritetoReg(0x30, 0x08, 0x02);


  Status = vdma_hdmi();
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rVdma_hdmi Failed \n\r");
	return XST_FAILURE;
  }

  Status = vtpg_hdmi();
  if (Status != XST_SUCCESS) {
    xil_printf("\n\rVtpg Failed \n\r");
	return XST_FAILURE;
  }


  Sensor_Delay();
  xil_printf("\n\rPipeline Configuration Completed \n\r");

  while(1) {

    xil_printf("\r\nPlease Select Camera(1 or 2) + ENTER:");


    usr_entry = getchar();

   char b;
   scanf("%c", &b);// This will take ENTER key



	switch(usr_entry) {

	  case '1':
		  xil_printf("\n\rSwitching to Camera 1\n\r");
		  XAxisScr_RegUpdateDisable(&AxisSwitch);
		  XAxisScr_MiPortDisableAll(&AxisSwitch);
		  XAxisScr_MiPortEnable(&AxisSwitch, 0, 0);
		  XAxisScr_RegUpdateEnable(&AxisSwitch);
		  break;

	  case '2':
		  xil_printf("\n\rSwitching to Camera 1\n\r");
		  XAxisScr_RegUpdateDisable(&AxisSwitch);
		  XAxisScr_MiPortDisableAll(&AxisSwitch);
		  XAxisScr_MiPortEnable(&AxisSwitch, 0, 1);
		  XAxisScr_RegUpdateEnable(&AxisSwitch);
		  break;

	  default:
		  xil_printf("\n\rSelection is unavailable. Please try again\n\r");
		  break;
	}

  }
  return XST_SUCCESS;

}

Testing

To test this on the board we first need to set the FMC VADJ correctly. This is controlled by a MSP430 on the SP701.

We can download the application from here

https://www.xilinx.com/products/boards-and-kits/sp701.html#resources

Once the application is downloaded, connect via the USB JTAG and read the board voltages. You will see the VADJ is 0v select set VADJ 2.5V and read back the voltages again for verification.

NOTE - We have to do this every time we power cycle the board.

Once we have the FPGA VADJ set correctly we can runt he application when connected to a HDMI output and see the image on the display.

Selecting the image using the application.

As we are not using all four cameras we can use the PCB slots to feed through the flexi cable to hold the imagers.

Wrap Up

Image processing in FPGA is fun, now we have a dual camera system and a little bit of logic resources remaining we can also see what image processing we add into it.

The design is available on my GitHub