Empowering DUNE: AXI Quad SPI using Petalinux 2022.2 in KRIA

Before start, please review this tutorial.

In the DUNE experiment, the photon detection system, DAPHNE, has undergone an evolution that has led to the utilization of SOM (System on Modules). This transition aims to achieve greater compactness in hardware and software task implementations, increased system control, and enhanced operational security. Given the context of the DUNE experiment, where this system will be situated 1 km underground in the Fermilab caverns, rendering subsequent system servicing challenging, a high level of security and robustness in the system implementation is imperative. This is where the KRIA KR260 takes center stage as the chosen SOM to manage the photon detection system's operation.

In this tutorial, I will guide you through the necessary steps to include the AXI Quad SPI peripheral in the PL for communication with various SPI interface chips. This peripheral connects to the PS through the AXI port, where it will be assigned an address in the Petalinux memory map. Simultaneously, a driver managed by the Device Tree will be mounted, enabling this hardware to be visible in the list of devices in /dev, allowing interaction through the SPIDev driver. Once the project is configured in Vivado 2022.2, synthesis and implementation are completed, the bitstream is created, and the platform is exported, we'll create the Device Tree Overlay and send it to the KRIA KR260. We will then implement the embedded hardware and control the peripherals through the AXI port from Petalinux 2022.2.

Also, please visit the following link containing important information: - Configuration of SPI in Linux.

Summary

This post serves as a continuation of the previous one, utilizing the same Vivado 2022.2 project. It focuses on incorporating specific hardware IPCores for SPI communication within the KR260 board.

Implementation of AXI_Quad_SPI Peripheral

To implement SPI using the Kria Robotics board, we will add the IP Core AXI Quad SPI. We locate it in the IP Core searcher and add it to the project.

Configuring the IP Core involves double-clicking on the block. In this setup, we will only modify the Mode to Standard, set Transaction Width to 8, and select Enable Master Mode.

Next, we establish the IPCORE connection. In this case, we right-click on the SPI_0 label and select Make External. It's also essential to connect the interrupt of the SPI to the PS side of the ZYNQ UltraSCALE+ through the Concat IP block.

In this case, the size of the concat block is expanded, allowing for a direct connection from Concat to the AXI Interrupt Controller, which adjusts automatically. This connection subsequently leads to the PS.

Constrains definition

In this example, we will use the PMOD1 connector to connect the SPI peripheral to the exterior of the KRIA.

Create the constrains file comms.xdc using the following pinout

We also have the definitions of UART and I2C because this project is a continuation of this tutorial.

##################### PMOD 1 Upper ################################
set_property PACKAGE_PIN H12 [get_ports {kria_uart_rxd}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_uart_rxd}] 
                                                         
set_property PACKAGE_PIN E10 [get_ports {kria_i2c_sda_io}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_i2c_sda_io}] 
set_property PULLUP TRUE [get_ports {kria_i2c_sda_io}]
                                                         
set_property PACKAGE_PIN D10 [get_ports {kria_spi_io0_io}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_spi_io0_io}] 
                                                         
set_property PACKAGE_PIN C11 [get_ports {kria_spi_ss_io[0]}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_spi_ss_io[0]}] 
                                                         
##################### PMOD 1 Lower ################################
set_property PACKAGE_PIN B10 [get_ports {kria_uart_txd}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_uart_txd}] 
                                                         
set_property PACKAGE_PIN E12 [get_ports {kria_i2c_scl_io}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_i2c_scl_io}] 
set_property PULLUP TRUE [get_ports {kria_i2c_scl_io}]
                                                         
set_property PACKAGE_PIN D11 [get_ports {kria_spi_sck_io}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_spi_sck_io}] 
                                                         
set_property PACKAGE_PIN B11 [get_ports {kria_spi_io1_io}]   
set_property IOSTANDARD LVCMOS33 [get_ports {kria_spi_io1_io}] 

Once you reach this stage, you can proceed with the synthesis, implementation, and bitstream generation processes, which will then be loaded onto the PL side. This procedure generates an address map for the AXI interface. Subsequently, this address interface can be accessed from the PS side using the devmem command in Linux.

As seen, the address assigned to the AXI_Quad_SPI is 0x80080000. We can interact with this address using the devmem command, which will be demonstrated later in this tutorial.

Upon completing the compilation, we generate the bitstream and export the platform with the name kria_spi_base.xsa.

Generation of Device Tree Overlay for the PL

In the root folder of the project, we will add the following files:

• xsct_config.tcl

hsi::open_hw_design kria_spi_base.xsa
createdts -hw kria_spi_base.xsa -zocl -platform-name kr260_spi -git-branch xlnx_rel_v2022.2 -overlay -compile -out ./dtg_kr260_v0
exit

This file contains the commands that will be executed by the Xilinx Source Commands Tool to create the Device Tree.

Remember to create the shell.json file:

nano shell.json

Then, copy and paste the following content into shell.json:

{ 
    "shell_type": "XRT_FLAT", 
    "num_slots": "1" 
}

Next, in a terminal, execute the following commands:

source /tools/Xilinx/Vitis/2022.2/settings64.sh
cd ./Kria_SPI/
xsct xsct_config.tcl

This will create the pl.dtsi file, which contains information about the device tree. This file is located at ./dtg_kr260_v0/dtg_kr260_v0/kr260_spi/psu_cortexa53_0/device_tree_domain/bsp/pl.dtsi. We must modify this file to include SPIDev support in the device tree. The file will originally contain the following code block describing the AXI_Quad_SPI:

axi_quad_spi_0: axi_quad_spi@80080000 {
	bits-per-word = <8>;
	clock-names = "ext_spi_clk", "s_axi_aclk";
	clocks = <&zynqmp_clk 71>, <&misc_clk_0>;
	compatible = "xlnx,axi-quad-spi-3.2", "xlnx,xps-spi-2.00.a";
	fifo-size = <16>;
	interrupt-names = "ip2intc_irpt";
	interrupt-parent = <&axi_intc_0>;
	interrupts = <0 0>;
	num-cs = <0x1>;
	reg = <0x0 0x80080000 0x0 0x10000>;
	xlnx,num-ss-bits = <0x1>;
	xlnx,spi-mode = <0>;
};

Then, we must modify this block of code to add support for SPIDev by adding the following code block. It should look like this:

axi_quad_spi_0: axi_quad_spi@80080000 {
	bits-per-word = <8>;
	clock-names = "ext_spi_clk", "s_axi_aclk";
	clocks = <&zynqmp_clk 71>, <&misc_clk_0>;
	compatible = "xlnx,axi-quad-spi-3.2", "xlnx,xps-spi-2.00.a";
	fifo-size = <16>;
	interrupt-names = "ip2intc_irpt";
	interrupt-parent = <&axi_intc_0>;
	interrupts = <0 0>;
	num-cs = <0x1>;
	reg = <0x0 0x80080000 0x0 0x10000>;
	xlnx,num-ss-bits = <0x1>;
	xlnx,spi-mode = <0>;
	spidev@0x00 {
		status = "okay";
		compatible = "rohm,dh2228fv";
		spi-max-frequency = <25000000>;
		reg = <0>;
	};
};

Once this configuration is completed, we proceed to create the device tree pl.dtbo using the following commands:

• DT_generation.sh

echo ""
echo ""
echo "#############################################################################"
echo "Compilacion del device tree"
echo "#############################################################################"
echo ""
echo ""

source /tools/Xilinx/Vitis/2022.2/settings64.sh
cd ./Kria_SPI/

dtc -@ -O dtb -o ./dtg_kr260_v0/dtg_kr260_v0/kr260_spi/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./dtg_kr260_v0/dtg_kr260_v0/kr260_spi/psu_cortexa53_0/device_tree_domain/bsp/pl.dtsi
cd ./spi_file_transfer/
cp ../dtg_kr260_v0/dtg_kr260_v0/kr260_spi/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./ & cp ../Kria_SPI.runs/impl_1/kria_bd_wrapper.bin ./
mv kria_bd_wrapper.bin kr260_spi.bit.bin
mv pl.dtbo kr260_spi.dtbo

echo ""
echo ""
echo "#############################################################################"
echo "Envio de archivos a la Kria"
echo "#############################################################################"
echo ""
echo ""

echo petalinux | scp kr260_spi.bit.bin kr260_spi.dtbo shell.json petalinux@<Kria IP address>:/home/petalinux

Run PL Design on KR260

This part of the tutorial is execute in the KRIA KR260, you need to connect to it through SSH using the follow command and your access password:

ssh petalinux@xilinx-kr260-starterkit-20222

Once the files have been successfully transferred to the KR260, create a directory within the /lib/firmware/xilinx directory using the same name as provided to the device tree blob and.bin file. Move these files into this newly created directory:

sudo mkdir /lib/firmware/xilinx/kr260_spi 
sudo mv kr260_spi.dtbo kr260_spi.bit.bin shell.json /lib/firmware/xilinx/kr260_spi

At this stage, the PL design will appear like an accelerated application using the xmutil commands:

sudo xmutil listapps

Unload the default application, then load the PL design, which flashes the PL design’s bitstream into the PL and loads its device tree overlay:

sudo xmutil unloadapp 
sudo xmutil loadapp kr260_spi

Once here, you can verify the mounting of the SPI device in the PS by using the following command:

ls /dev | grep spi

You should receive the following response:

spidev3.0

Where spidev3.0 corresponds to the added peripheral to control the SPI from the PS.

It is also necessary to grant the necessary permissions to execute and control the peripheral:

sudo chmod 777 /dev/spidev3.0

With this, you now have access to control the AXI_Quad_SPI peripheral from the PS.

Testing AXI Quad SPI

To conduct this test, we will establish the following connection and create a loop between the MOSI and the MISO to read data sent from the PS.

We will install the spidev package for Python using the following command:

python -m pip install spidev

Python Code

Create a file named spitest.py containing the following script.

#!/usr/bin/env python3
# vim: expandtab ts=4 sw=4

import spidev

# Load the SPI object
spi = spidev.SpiDev()   
# Open the SPI device in the Bus 3 and the device 0
spi.open(3, 0)  
# Configure the clock to 5 MHz   
spi.max_speed_hz = 5000   
# Configure the SPI mode  
spi.mode = 0b01  

# message codification 
data = 'FABIAN'.encode('utf-8')  
# Trnasfer, this do the W/R operation with just one CS change
rec=spi.xfer(data)   
print(rec)

The code returns [70, 65, 66, 73, 65, 78], which represents in ASCII characters as ['F', 'A', 'B', 'I', 'A', 'N'].

We can also observe the pulses sent by the peripheral through a Logic Analyzer.

Using the devmem command to interact with SPI

It is possible to interact with the SPI peripheral using the devmem command in Linux. To do this, it is necessary to know the address assigned to the peripheral (in our case 0x80080000) and the register mapping of the IP. This information is available in the IP's official documentation. Below is the register map extracted from the official documentation:

And associated with this, there's a bit map which I invite you to consult in the official documentation so that you can configure the SPI to best suit your application. Below is a code snippet demonstrating the interaction with the AXI_Quad_SPI using devmem.

• spi_test.sh

#!/bin/sh

# 1. Basic Initialization
echo ""
echo "Read and clear interrupts"
devmem 0x80080020                                # to clear interrupts - 1
devmem 0x80080070 32 0xffffffff                  # clear SS - 1
devmem 0x80080060 32 0x180 # 0x1e6                       # inhibit transaction -1

# 2. Setup for multi-byte
# ~> Default is fine: devmem 0x8008001C 32 0x00000000           # configure DGIER (Device Global Int En Reg) - 2
# ~> Default is fine: devmem 0x80080028 32 0x00000000           # configure IPIER (IP Int En Reg) - 2

# 3. and 4.
devmem 0x80080060 32 0x00000000          # write data to master SPICR

# 5. Select the slave
devmem 0x80080070 32 0xfffffffe          # set SS - 5

# 6. and 7. Send Transaction
devmem 0x80080068 8 0xDE
devmem 0x80080068 8 0xAD
devmem 0x80080068 8 0xBE
devmem 0x80080068 8 0xEF
devmem 0x80080068 8 0x40
devmem 0x80080068 8 0x48

# deactivate SS
# devmem 0x80080070 32 0xffffffff          # set SS - 5

echo "read status"
devmem 0x80080064

echo "read Data"
devmem 0x8008006C
devmem 0x8008006C
devmem 0x8008006C
devmem 0x8008006C
devmem 0x8008006C
devmem 0x8008006C

echo ""
echo "Read out buffer occupancy: x \+ 1 = number of bytes"
devmem 0x80080074                                        # read out fifo - 6 and 7

echo ""
echo "Send buffer"
devmem 0x80080060 32 0x86                                # write data to master SPICR - 8

echo ""
echo "Read and clear interrupts"
devmem 0x80080020

echo ""
echo "Read and clear interrupts"
devmem 0x80080064                                        # Read SPISR reg - 11 and 12

Run this code in the terminal using the next command:

bash spi_test.sh 

This is the last in a series of three tutorials where we will demonstrate how to configure the UART, I2C, and SPI interfaces from the PL and utilize them with the PS.