Vitis-AI pytorch design example on the Kria KV260

Story

The Kria KV260 vision ai starter kit reased by Xilinx, is a development platform for the Kria K26 SoM (System on Module). The platform is released to provide developers with an easy access into "advanced vision application development", ultimately enabling users with little to none hardware design knowledge to get their hands on a hardware acceleration platform.

Personally I have no hardware design experience, but am interested in using FPGA hardware acceleration in the testing of on-site neural networks in physics experiments. Therefore I will be testing the use of a simple neural network on the KV260. While setting up the development starter-kit, I noticed the out of the box experience, was not so out of the box. Let me walk you through the process of setting up the development board for Vitis-AI development, and along the way showcase some of the bugs and errors that I encountered. Firstly I will give a list of the complete setup and its details.

Setup resources details

• Linux Ubuntu 20.04.3 LTS image

The latest 22.04 image which is also supported for the kv260 as listed by the website, did not boot, Therefore the 20.04 version was used. To use the latest 22.04 image, the firmware has to be updated first.

• Vitis-AI version 2.0 Github

This is not the latest version of Vitis AI, since the DPU architecture on the kv260 used for this project was not compatible with the latest Vitis AI verison.

• DPU B3136 architecture

The B4096 DPU architecture is only since recently supported by the Vitis-AI software and can be found on github:Xilinx/kria-apps-firmware/kv260/benchmark-b4096/. This architecture was not native to the linux image and thus the B3136 version was used. The DPU target architecture is DPUCZDX8G_ISA0_B3196_MAX_BG2

• Pytorch flow tutorial

The Pytorch flow for Vitis-AI tutorial by Vitis-AI hosting the MNIST classification of handwritten digits is a great project to test the FPGA hardware flow. Github: Xilinx/Vitis-AI-Tutorials/Design-Tutorials/09-mnist_pyt/

Load Linux image on SD card

It is recommended to run an embedded Linux image on the KV260 that is specifically made for the KV260 in combination with the Kria SoM. The Xilinx web page Getting started with Kria KV260 Vision AI starter kit hosts a link to download the Linux image. The latest version of Ubuntu 22.04 LTS is recommended by Xilinx, for this version to work, the firmware of the kria SoM has to be updated first. In this project I opted for the slightly older but still supported image version of Ubuntu, 20.04.3 LTS.

After choosing the image Linux image of choice, the image is to be burned on a micro SD card of at least 16GB. Burning the SD card image can be done with the Balena Etcher tool, or a disk image flash software of choice.

Note that the different embedded versions of Ubuntu come with varying integrated tool sets. For example, the xmutil command that is native to the 22.04 Ubuntu version, and is used to query the FPGA, is available through xlnx-config --xmutil in the command terminal for the 20.04 version of Ubuntu.

Getting Started with Kria KV260 Vision AI Starter Kit

After succesfully flashing the OS image on the SD card, it is time to start up the KV260. You can follow the detailed guide on the xilinx website on how to boot the starter kit, and get linux up and running. It is possible to either connect with the Kria board using the usb port, and using putty or similar software to connect to the kria board. This is not necessary since the GUI desktop of Linux can be broadcasted on an external display using the HDMI port.

Starting FPGA application

When the KV260 is successfully booted into the GUI desktop, it is ready to test the FPGA power with an application. For the Ubuntu 20.04.3 LTS image, the NLP-SmartVision Application is recommended to get started with the FPGA. This application allows the user to correctly set up the FPGA hardware structure, and test if everything is correctly installed, before quering the chip with the coming project.

The NLP-Smartvision application, and how to get it started is explained in the wiki, nonetheless here is a walk-through to get it going. First the application has to be installed:

sudo snap install xlnx-nlp-smartvision

This installs the application and its firmware using snap, into the correct folder. /lib/firmware/xilinx

Now we can load the nlp-smartvision application onto the FPGA. To see a list of all the currently available applications:

sudo xlnx-config --xmutil listapps

If the nlp-smartvision application is installed correctly it should now appear on the list. To load the application into the hardware, we use the command,

sudo xlnx-config --xmutil loadapp nlp-smartvision

The application can be unloaded using unloadapp instead of loadapp.

Now we are ready to run our first application on the FPGA. The nlp-smartvision application is able to run using input from USB camera by using the -u flag or the provided MIPI camera by using the -m flag. However to just test the application it is also possible to provide the application with just one image. For this example an image of a face was taken from google and analyzed using the provided densebox_640_360 model:

xlnx-nlp-smartvision.nlp-smartvision --test image.jpeg densebox_640_360

The application successfully recognized a face in the picture. We successfully ran our first accelerator application on the application. However we are interested in more than just an integrated demo. Therefore we will now use the Vitis-AI library to run one of the examples to demonstrate that it is possible to use the KV260 for custom AI applications and machine learning projects.

Vitis-AI example

setting up pytorch with correct dpu fingerprint

Running the Vitis-AI example is explained well on the github page, however it is tricky to correctly compile the projects for the KV260. For this project the 09-MNIST-pytorch example was chosen.

It is possible to customize the neural network model to test the difference the model makes on the performance. This is done in the common.py file. Be aware that not all types of pytorch layers are supported for the FPGA architecture.

After training and testing the model it is time for the compiling of the model. This is depended on the device that you are targeting, in our case the KV260, but also the architecture of the loaded application on the device. In our case, we loaded the NLP-smartvision application to direct the FPGA's DPU architecture. This enables the use of the B3136 DPU architecture. However it is possible to maximize the power of the KV260 by loading in a B4096 DPU architecture, but this is not released by the Ubuntu 20.04.3 version, rather for the latest 22.04 version.

When compiling the project we have to specify our target architecture, we will do that by specifying the fingerprint of the target chip:

{fingerprint: "0x1000020f6014406"}

This is done by editing the arch.json file in the /opt/xilinx/targetkv260/arch/ folder. However it is easier to specify a different file in the compile.sh file since the arch.json file in the conda environment resets after closing the environment.

If the model is now compiled succesfully for the KV260, it is ready to be deployed on the target. This can be done in numerous ways but the easiest is to scp the required files to the target, which requires both devices to be connected to the same internet connection.

scp -r /folder ubuntu@<ip address>:~/target_kv260

Now for running the model on the KV260, open terminal and cd into the correct source folder /target_kv260. To query the FPGA chip for the project we use the command on target:

xbutil query

Finally run the python file app_mt.py with the -m tag and specify the number of threads.

python3 app_mt.py -m CNN_kv260.xmodel -t 3

This will mount the application on the FPGA architecture using 3 threads. The result will look something like this:

It looks like we successfully completed the example MNIST recognition using pytorch on the Kria kv260 SoM! Do note that this project was ran not using the latest Ubuntu image at the time. This can be resolved by first updating the firmware of the Kria kv260, using the Ubuntu 20.04 version and then mounting the newer Ubuntu 22.04 version.

After some testing of different neural network structures the FPGA acceleration was clear as can be seen in the figure below. CPU times where on single thread using Intel core i7 8th gen chip.

Now that this proof of concept worked, I will further explore the possibilities of using vitis-ai in combination with the Kria kv260 to test the latency of neural networks in physics related problems.