Industrial Inspection using Xilinx Kria

Background

The introduction of Ubuntu for Kria has allowed new use cases for the Kria SoM by allowing many existing applications to be easily used on the Kria. This also provides a more familiar evironment for many people who are not too famliar with the traditional FPGA tools, which can get very overwhelming at first.

In this project we will leverage the ubuntu image to enable use of Kria SoM for an Industrial image processing use-case. We will be using some commercially available machine vision libraries instead of OpenCv. This is helpful when customers need quick solutions built in a matter of days/weeks and customer is ok with paying enough to cover licensing costs for commercial libraries, etc. Note that this is not a sponsored post.

Preparing Hardware

For this project, we need:

• Kria kv260 kit
• MicroSD card (16 GB recommended)
• Kria Power adapter
• Linescan camera + Encoder
• Linescan light + controller
• Motion control system (Servos, mechanics, motion controller, etc)

To begin with, since setting up all the machine subsystems (industrial linescan camera, lighting controller and motion control system) to work with Ubuntu will take quite some time, we will start with some images acquired from a system that is already set up and working. This way we can get down to the fun part right away.

Prepare Ubuntu Image

To install ubuntu on Kria, go to :

https://ubuntu.com/download/xilinx

Ubuntu is supported on multiple Ultrascale MPSoC boards/kits, so you will need to download the correct image by selecting the "Xilinx Kria KV260 Vision AI Starter Kit" and clicking "Download 64-bit". The download is slightly smaller than 1GB in size.

After this, go to the Kria Getting Started page to complete the rest of steps. This page contains two paths: Ubuntu or Petalinux. We will take the Ubuntu path to move forward.

Installing Halcon

Since the Kria uses Arm Cortex-A53 based application processing unit (APU), which is a 64-bit processor, we will need to download Halcon for 64-bit ARM-based platforms, which we can do from the embedded vision page (require account creation)

https://www.mvtec.com/products/embedded-vision

Since I have an existing Halcon 18.11 license, I will download the 18.11 version. For operating system, choose "Linux for Arm" and for Architecture, choose "aarch64-linux"

Once the options are chosen, the relevant files will appear below. You will have option to download the "Full version" or "Runtime version". Full version is more suitable when you need the ability to edit or develop Halcon scripts via HDevelop interactive environment on the target device. Runtime version is more suitable for deployment scenarios. The deployment licenses are much cheaper for the Runtime version also.

In my case I downloaded the Full version since I want the ability to be able to develop/edit Halcon scripts on the Kria itself. The installer cost me 1.8GB in disk space. The same page also gives a link to the halcon installation guide.

Next copy the downloaded tar.gz file to the Kria filesystem (I used a usb drive) and extract to a local folder. Next we open a terminal, navigate to the extracted folder and execute the install script.

sudo ./install-linux.sh

The script automatically detects aarch64-linux as the suitable platform version and selects aarach64-linux for installation. So far so good. Moving forward, the setup will require accepting the license agreement and selection some options. I used all default options. The installation took around 10 mins and ended with a successful installation, and some messages on things to do post installation. However, it leaves behind a file ".profile_halcon" to setup all of these on Kria terminal.

To execute this script, open a terminal, change directory to /opt/halcon

cd /opt/halcon

Then use the following command in a command prompt

source ./profile_halcon

This completes the installation and setup process.

Running Application

When trying to execute my machine vision scripts, I ran into a licensing issue. Halcon requires hidraw support in the ubuntu image to be able to access the hid dongle. However, the Kria ubuntu image is built without that. After posting question on the discussion board, and also talking to Halcon(MvTec) support, I was told that I need to rebuild the kernel with hidraw support turned on. The procedure to do this is given at this link:

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/2287796272/Rebuilding+the+Certified+Ubuntu+for+Xilinx+Devices+20.04+LTS+Kernel+from+Source

So next we will try to rebuild the kernel with hidraw support.