RPi+PMOD Connector GPIO with Custom PL Design in Kria KR260

One of the features I was excited to get with the Kria KR260 for my robotics applications that the Kria KV260 didn’t have was the extra I/O connectors to drive various peripherals such as motors. Given that I/O mapping on an FPGA can be a bit more complex than your typical microcontroller board like the Raspberry Pi or BeagleBone, I decided to do a writeup covering how I map the I/Os of a new AMD-Xilinx development board.

Furthermore since the Kria is a SoM (System on Module), I/O mapping on it has an added layer of complexity given that it spans over two separate boards: the Kria K26 base board, and the KR260 carrier board. So knowing where to look to find the info needed to fully map out a signal from a pin on a PMOD or the RPi header all the way to a package pin on the Kria’s K26 FPGA itself can seem skewed for first-time users.

Finally, the PMODs and RPi header on the KR260 are connected to the programmable logic of the K26 versus the MIO/EMIO pins that most of the other peripherals are connected to (as outlined in my previous post here. This requires a modification of the block design (or addition of HDL code) in Vivado to add the logic for the desired interface to each of the connectors. Which means a new bitstream will be generated and corresponding hardware node in the device tree will also be required.

For those not familiar, the boot process of the Kria’s embedded Linux image boots the ARM-core processors of the Zynq MPSoC in the K26 FPGA before flashing a bitstream onto its programmable logic (PL). Thus any hardware nodes in the device tree for things located in the PL have to be loaded as a device tree overlay later, otherwise the boot process would freeze looking for device tree nodes that aren’t present in the system yet since the PL has yet to be programmed. This is why there is no device tree packaged into the default boot binary (boot.bin) for the Kria boards you download from the K26 SoM wiki.

Now it’s definitely possible to reconfigure a PetaLinux project built from the KR260/KV260 BSP to change the boot sequence to program the PL during boot and package a device tree into the boot binary. It’s also possible to just create a PetaLinux project from scratch configured in this manner as well, but a PetaLinux project for the Kria boards is still complex enough that it’s much easier/faster to modify a project created from a BSP (which is why I will not do a project tutorial for KR260/KV260 PetaLinux project built from scratch, 99% of it would just be copy+pasting stuff from a project built using the BSP). However, I’ve found the easiest solution is to load the device tree overlay the exact same way as I would for an accelerated application and be on my merry way, which is what I will be demonstrating in this project.

Download KR260 Schematic

The schematic for the KR260 carrier board (XTP743) can be found here. It must be downloaded directly from AMD-Xilinx’s site since a user agreement must be digitally signed prior to downloading.

The schematic for the Kria K26 base board is not publicly available given it’s a commercial product and not just a development part. However, there is a master pinout constraints file included in the download of the KR260 schematic that provides the necessary information for signal mapping.

Modify Vivado Block Design

As I mentioned previously, since the PMOD and RPi headers are connected via the PL, the block design in Vivado needs to be updated to add the specific interface desired for each connector. Constraints file(s) also need to be added to specify the package pins on the FPGA. I’m starting with the Vivado project I previously created for the KR260 in this previous project post.

I’m going to just use AXI GPIO block for everything, I’m not going to cover the RPi special functions at the moment, but you would simply add the appropriate IPs for each special function like I2C or SPI. So I added 5 AXI GPIO IP blocks to the block design: four are one channel of 8 bits and the fifth is one channel of 28 bits.

Once added, run the corresponding connection automation that appeared, leaving all selections for clock as the default.

Validate the block design and save it, then generate the block design again.

Create a new HDL wrapper by right-clicking on the block design file in the Hierarchy tab of the Sources window and selecting Create HDL Wrapper…

Select to either allow Vivado to auto-manage it or allow for user edits. I personally chose the latter so I could make the code a bit more readable since I don’t like the way that Vivado auto-generates the code for AXI GPIO that’s configured as inout signals, but this has zero impact on how the design actually functions and is just a personal choice to make it easier for myself to use with other custom code in the future.

Add Constraints

With the signal names pulled up to the top level HDL file, they need to be connected to specific package pins on the FPGA via constraints file(s).

I added two new constraints files to the design (Add Sources > Add or create constraints > Create File), one for the PMODs and one for the RPi header. Everything could definitely be in the same constraints file, it complies the same way regardless. I personally find it easier to keep things organized by separating out the different type of connectors into separate constraints files.

The tricky thing about the IO of the PMOD on the KR260 is that the PMOD IO numbering is different from the physical connector’s pin numbering is done, especially since pin 1 and pin 8 on the connector match up with PMOD IO1 and PMOD IO8, but the rest of the connector’s pin alternate sides while the PMOD IOs go across the connectors:

This is what prompted me to create a master pinout spreadsheet to keep track of everything (which I do for all of my more complex design anyways):

Since the PMOD IO signal names go across the connectors, that’s how I structured them in the constraints file with each broken up into the “upper” and “lower” IOs (also see constraints files attached below).

I did the same to keep track of the RPi header IO numbers relative to the 40-pin connector’s pin numbers, and the FPGA package pins:

You’ll notice I also went back and added the Sysfs GPIO numbers to the spreadsheet once they registered in the Linux system (explained in last section). I highly recommend some sort of overall documentation system as such for designs like this because this would be a nightmare to reverse engineer a few weeks/months later.

Generate Bitstream with.bin File Included

Since I’m just using standard AXI GPIO to interface with the PMODs and RPi header, I’m going to just use the Sysfs drivers straight from the KR260’s command line to drive them in my initial testing (since my main goal at the moment is to make sure I just have everything mapped out right).

This means I can generate the bitstream’s.bin file straight from Vivado and skip Vitis (recall that the.bin file is created in Vitis for accelerated applications). I can also write a basic (not accelerated) Linux application any time later and still utilize the.bin generated from Vivado as well.

By default, Vivado generates only a.bit file for the bitstream, which contains extra configuration info that’s not needed in this case since the ARM processor is already booted when the bitstream is being flashed onto the PL. So the option for Vivado to also generate a.bin file for the bitstream needs to be enabled. Open Settings from the Flow Navigator window, then in the Bitstream tab, enable the option for -bin_file (this will cause a <design_name>.bin file to be generated and output into /<Vivado project>/<Vivado project>.runs/impl_1/):

Click Apply, then OK to close the Setting window. Run synthesis, implementation, and generate bitstream for the design, as well as export the platform the same way as before.

Update PetaLinux with New XSA

Import the new XSA into either an existing PetaLinux project (I’m using the same project from my previous post here) or a new PetaLinux project created using the KR260 BSP:

~$ source /tools/Xilinx/PetaLinux/2022.1/settings.sh
~$ cd ./Kria_KR260/linux_os/
~/Kria_KR260/linux_os$ petalinux-config --get-hw-description ../

Since I’m using my existing project, I just exited the system configuration editor without making any changes and rebuilt the PetaLinux project:

~/Kria_KR260/linux_os$ petalinux-build

Next package a WIC image for an SD card:

~/Kria_KR260/linux_os$ petalinux-package --wic --images-dir images/linux/ --bootfiles "ramdisk.cpio.gz.u-boot,boot.scr,Image,system.dtb,system-zynqmp-sck-kr-g-revB.dtb" --disk-name "sda"

Then flash an SD card with the WIC image either using the command line, or program like balenaEtcher.

Generate Device Tree Overlay for PL Design

The device tree blob containing the overlay nodes needs to be compiled for the design, which is easily done by using the Xilinx Software Command Line Tools (XSCT):

~/Kria_KR260$ source /tools/Xilinx/Vitis/2022.1/settings64.sh
~/Kria_KR260$ xsct

Open the exported XSA from Vivado and use the createdts command to create the device tree source files for the PL design:

xsct% hsi::open_hw_design ../kria_base.xsa
xsct% createdts -hw ../kria_base.xsa -zocl -platform-name kria_kr260 -git-branch xlnx_rel_v2022.1 -overlay -compile -out ./dtg_kr260_v0
xsct% exit

After exiting XSCT, use the standard Linux device tree compiler (dtc) to compile the source files into the needed device tree blob:

~/Kria_KR260$ dtc -@ -O dtb -o ./dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtsi

Transfer PL Design Files to KR260

I like to create a folder to copy all of the necessary design files to that need to be uploaded to the KR260 for an accelerated application or PL design like this (note: the Kria_KR260 directory is the top level directory of the Vivado project):

~/Kria_KR260$ mkdir -p gpio_file_transfer
~/Kria_KR260$ cd ./gpio_file_transfer/

It’s here I’ll create the description file, shell.json, for the design:

~/Kria_KR260/gpio_file_transfer$ gedit shell.json

And copy+paste the following to shell.json:

{
    ”shell_type" : “XRT_FLAT",
    "num_slots": "1"
}

Then copy the generated device tree blob and.bin file into the folder:

~/Kria_KR260/gpio_file_transfer$ cp ../dtg_kr260_v0/dtg_kr260_v0/kria_kr260/psu_cortexa53_0/device_tree_domain/bsp/pl.dtbo ./
~/Kria_KR260/gpio_file_transfer$ cp ../Kria_KR260.runs/impl_1/kr260_top.bin ./

Rename the device tree blob and.bin file to the same thing (the only difference should be their respect file extensions (also change the.bin extension to.bit.bin):

~/Kria_KR260/gpio_file_transfer$ mv kr260_top.bin kr260_gpio.bit.bin
~/Kria_KR260/gpio_file_transfer$ mv pl.dtbo kr260_gpio.dtbo

Then, with the KR260 booted up and running with the new SD card image generated in the previous steps and connected to the local network, transfer the new PL design files to it:

~/Kria_KR260/gpio_file_transfer$ scp kr260_gpio.dtbo kr260_gpio.bit.bin shell.json petalinux@<KR260’s IP address>:/home/petalinux

Run PL Design on KR260

After the files have been successfully transferred to the KR260, create a directory in the /lib/firmware/xilinx directory with the same name as was given to the device tree blob and.bin file and copy them into it:

xilinx-kr260-starterkit-20221:~$ sudo mkdir /lib/firmware/xilinx/kr260_gpio
xilinx-kr260-starterkit-20221:~$ cp kr260_gpio.dtbo kr260_gpio.bit.bin shell.json /lib/firmware/xilinx/kr260_gpio

At this point, the PL design will show up just like an accelerated application would using the xmutil commands:

xilinx-kr260-starterkit-20221:~$ sudo xmutil listapps

Unload the default application then load the PL design which flashes the PL design’s bitstream into the PL and loads its device tree overlay:

xilinx-kr260-starterkit-20221:~$ sudo xmutil unloadapp
xilinx-kr260-starterkit-20221:~$ sudo xmutil loadapp kr260_gpio

When the device tree overlay loads, the terminal will print out as such indicating the new device tree nodes for each of the AXI GPIO IP blocks are now present in the system:

Testing the GPIO

Using the Sysfs driver, list out the GPIO available in the system:

xilinx-kr260-starterkit-20221:~$ ls /sys/class/gpio/
export  gpiochip0  gpiochip440  gpiochip468  gpiochip476  gpiochip484  gpiochip492  gpiochip500  gpiochip508  unexport

You can print the label for each to determine what each gpiochip number correlates to:

xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip440/label
80050000.gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip468/label
80040000.gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip476/label
80030000.gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip484/label
80020000.gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip492/label
80010000.gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip0/label
zynqmp_gpio
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip500/label
slg7xl45106
xilinx-kr260-starterkit-20221:~$ cat /sys/class/gpio/gpiochip508/label
firmware:zynqmp-firmware:gpio

The Zynq MPSoC MIO pins are gpiochip0, gpiochip508 is the ZynqMPSoC modepin GPIO controller, and gpiochip500 is the KR260’s I2C GPO reset controller. These three gpiochips are standard to the KR260 and will always be there, and you don’t need to mess with them.

The other gpiochips (gpiochip440, gpiochip468, gpiochip476, gpiochip484, and gpiochip492 in my case) all have the corresponding AXI GPIO’s address from the Vivado block design address editor as their label. So fro example, I can now see that the AXI GPIO for the RPi header (address 0x8005_0000) is gpiochip440. And since I have that AXI GPIO IP configured for 28 inout pins, that means RPi_GPIO0 on the AXI GPIO for the RPi header is GPIO 440 in the Sysfs driver, RPi_GPIO1 is GPIO 441, and so on up to RPi_GPIO27 which is GPIO 467.

I attached 8 LEDs to the 8 I/O of PMOD4 which is gpiochip476 and exported each IO as an output pin. Which again means PMOD4 IO1 is 476 through PMOD 4 IO8 which is 483:

xilinx-kr260-starterkit-20221:~$ echo 476 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio 476/direction
xilinx-kr260-starterkit-20221:~$ echo 477 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio477/direction
xilinx-kr260-starterkit-20221:~$ echo 478 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio478/direction
xilinx-kr260-starterkit-20221:~$ echo 479 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio479/direction
xilinx-kr260-starterkit-20221:~$ echo 480 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio480/direction
xilinx-kr260-starterkit-20221:~$ echo 481 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio481/direction
xilinx-kr260-starterkit-20221:~$ echo 482 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio482/direction
xilinx-kr260-starterkit-20221:~$ echo 483 | sudo tee /sys/class/gpio/export
xilinx-kr260-starterkit-20221:~$ echo out | sudo tee /sys/class/gpio/gpio483/direction

I then simply went through each toggling them on and off to verify the expected LED turned on/off:

xilinx-kr260-starterkit-20221:~$ echo 1 | sudo tee /sys/class/gpio/gpio476/value
xilinx-kr260-starterkit-20221:~$ echo 0 | sudo tee /sys/class/gpio/gpio476/value
.
.
.
xilinx-kr260-starterkit-20221:~$ echo 1 | sudo tee /sys/class/gpio/gpio483/value
xilinx-kr260-starterkit-20221:~$ echo 0 | sudo tee /sys/class/gpio/gpio483/value

The GPIO will stay present in the system so long as the PL design is loaded as the current application. You can run any basic (not accelerated) Linux application utilizing them at this point. If the KR260 is power cycled, it will manually need to be reloaded.

After testing LEDs on the other PMOD connectors, I did discover that the grounds for each of the PMODs is not common to each other even though they appear to be on the schematic (meaning I had to connect a ground pin from each PMOD to my breadboard if I wanted to drive an LED from that respective PMOD). So if your external circuit is depending on a common ground between each of the PMODs and RPi header, it will be up to that external circuit to make those connections.