Petalinux vs. Baremetal: Run Your FPGA Video Design!

A: Overview

In FPGA-based embedded system design, you may have noticed two common terms: bare-metal, bare-bones, or standalone design, and Linux design. Bare-metal designs are built on top of the bare-metal software stack. This is also known as a Standalone OS or Standalone Board Support Package (BSP). AMD-Xilinx provides a bare metal software stack as part of the AMD Vitis™ software platform. The Standalone BSP gives you a simple, faster, single-threaded environment that provides basic features such as standard input/output and access to processor hardware features. The BSP and included libraries are configurable to provide the necessary functionality with the least overhead. For time-critical applications, the bare-metal design flow is significantly adopted.

https://docs.amd.com/r/en-US/ug1137-zynq-ultrascale-mpsoc-swdev/Bare-Metal-Software-Stack

On the other hand, Linux designs are built on top of the well-known Linux OS. To create Linux applications, AMD-Xilinx provides the Petalinux tool, which is an embedded Linux Software Development Kit (SDK) targeting FPGA-based system-on-a-chip (SoC) designs or FPGA designs. The tool contains everything necessary to build, develop, test, and deploy embedded Linux systems. For commercial feature-rich applications that don't need time precision, the Linux design flow is adopted predominantly.

The PetaLinux tool contains Yocto Extensible SDK, XSCT, and toolchains, and PetaLinux Command Line Interface (CLI) tools.

eSDK contains the platform(Microblaze, Zynq-MPSoC, Zynq 7000) specific SDK files.

Petalinux CLI includes the commands, such as petalinux-build, petalinux-config, petalinux-create, etc...

For more information, check: https://docs.amd.com/r/en-US/ug1144-petalinux-tools-reference-guide/Introduction

In this design article, rather than going deep into each OS, we will briefly see how the baremetal and petalinux design goes, considering a generic FPGA video design. I am using Vitis/Petalinux 2021.1, and for design deployment, I am using the ZCU102 Development Board.

B: Hardware Generation/Platform Generation/XSA Generation

In FPGA-based design, the foremost step is to create a proper hardware design for the target FPGA board. We start creating a block design in Vivado IP Integrator (VIPI). For simplicity, I have created a generic capture pipeline design. The design has the following hardware block (major only). The design steps have been skipped. To know the process of creating vivado projects and adding IPs in the VIPI, follow this project: https://www.hackster.io/nikilthapa/4k-tpg-video-streaming-in-kria-kv260-baremetal-part-1-c0c9d6

Video Test Pattern Generator (VTPG) IP

* The IP is used as a video source. In a real design scenario, the TPG can be replaced by a real hardware, for example, MIPI Camera or HDMI/DP/VGA/SDI video interfaces.

* The IP customization is included below:

* Resolution: 4096x2160 (Default)

* PPC: 2

* Data Width: 8-bit

* Background and Foreground: Default

Video Frame Buffer Write (VFBWR) IP

* The output of the TPG IP block is connected to the VFBWR IP. The IP buffers/stores source (TPG) frames into the specified memory location for later use cases.

* The IP customization includes;

* Resolution: 3840x2160

* Samples per clock: 2

* Maximum Data Width: 8

* 8 Bit Video Formats: Check all of them (for now)

Zynq UltraScale+ MPSoC Block

* This represents a Processing System, which has APUs. We run our Petalinux or Baremetal application here.

* Don't forget Run Block Automation, once it is added to the VIPI.

* The customization includes;

* Enabling S_AXI_HP0_FPD to provide FBWR IP PS-DDR access.

* Enabling only M_AXI_HPM0_FPD interface to configure IPs through AXI-Lite interfaces.

* Enabling EMIO GPIO to provide software reset signals.

* Enabling DP Interface to display TPG frame (Automatically enabled when performing "run block automation").

Other IP blocks

* Slice is used to slice the desired GPIO bit from the EMIO GPIO. We use EMIO GPIO 0 and 1 bits to supply software reset signals to TPG and FB WR IPs, respectively.

We use the PS default clock to drive our pipeline. Once connected properly, we will have the block design, as shown in the following image. This design does not have any floating ports or pins. We don't have to worry about the constraint mapping from the userside. We proceed with the hardware generation.

Once generation completes, we export it as an XSA file. While exporting, give the file name as you like.

C: Software Application

Here, we will create bare-metal and Linux applications one after another and do tests on the hardware.

C.1.1. Baremetal Application

This application uses a Standalone OS. This is the lighter OS that has the necessary libraries, including Xilinx IP's bare-metal drivers. The bare-metal application is built on top of the platform project. We use the Vitis SDK IDE to create, build, and run the bare-metal application. The individual project creation steps have been skipped. Follow this project to know the project creation steps and build the project in a similar manner. https://www.hackster.io/nikilthapa/4k-tpg-video-streaming-in-kria-kv260-baremetal-part-2-62fca5

# Platform project is created by using an XSA file. We use the same XSA file that we exported from Vivado.

# We need to build the platform project first.

# The application creation is done by selecting the platform project, we created in the previous steps. Furthermore, we choose one of the available application templates to start writing application code. We can select an Empty Application template.

# In the application coding, we will cover four configurations, as described below.

## PS EMIO GPIO Configuration is mandatory to control the reset line of TPG and VFBWR IPs through the slice IP block

## TPG Configuration is done to initialize and configure TPG with 1920x1080 video resolution, RGB color format, and colorbar test pattern IP.

## VFBWR Configuration is done to initialize and configure the VFBWR IP to buffer the TPG stream into a specific location of DDR memory. The stream parameters (resolution, color format) must match the TPG.

## DP Configuration is necessary to configure the PS DP interface in Non-Live mode and then read the TPG frame from the buffer address and display it through the PS DP interface. For the DP configuration, we utilize the PS DPDMA baremetal example.

All of the IP baremetal drivers and their examples can be found here. https://github.com/Xilinx/embeddedsw/tree/xlnx_rel_v2021.1/XilinxProcessorIPLib/drivers

# The following shows the main function that houses all the configuration functions we covered above.

#include "xv_tpg.h"
#include "xv_frmbufwr.h"
#include "xvidc.h"
#include "xdpdma_video_example.h"
#include "xgpiops.h"

void main(){

    print("PS Gpio Configuration\n");
    config_psgpio();

    print("TPG Configuration\n");
    config_tpg();

    print("VFBWR Configuration\n");
    config_fbwr();

    print("PS DP Configuration\n");
    config_psdp();

}

C.1.2. Build & Deployment

In baremetal, we build the baremetal application and run the design by connecting the JTAG cable. This is the most usual method for the development and prototyping stages. However, for the deployment stage, we can still generate an SD Card boot image and run the design using by boot.bin file.

We select the first switch configuration from the table below to set in the JTAG mode.

Visit the Output section of this article to see the output.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

C.2.1. Linux Application

Here, we cover Linux application, for which we are using Petalinux 2021.1. Unlike Baremetal application, we create an entire application using a Linux terminal and run the design by entering commands using the UART on the host machine.

The Linux application starts by building the petalinux project first. And we will cover the following steps.

Note that in this section of the article, the text contained within the code canvas is the petalinux commands entered in the Linux terminal.

# We source petalinux 2021.1

$ source <path-to-installed-PetaLinux>/settings.sh

# Creating Petlinaux project from template

$ petalinux-create --type project --name <project-name> --template zynqMP
$ cd ./<project-name>

# Configuring petalinux project with custom XSA file

$ petalinux-config --get-hw-description=<path-to-XSA-file>/<file-name>.xsa --silentconfig

# Initial Petalinux build can be done to check device tree generation. The device tree is an important part of the Linux-based design through which Linux OS gets the peripheral hardware configuration. The configuration of TPG and VFBWR hardware blocks is conveyed by the pl.dtsi.

# Once the build completes, we can open pl.dtsi file. In most cases, the pl.dtsi contains the complete device tree element of IPs automatically. If any of the elements are missing, we have to add them by using system.user.dtsi file.

Check the code section to see the pl.dtsi.

# Now, we can proceed with the rest of the configurations.

# System Configuration

$ petalinux-config

## Setting MACHINE_NAME in DTGSettings. As we are using the ZCU102 board, we need to set the name as "zcu102-rev1.0". You may also need to change the revision version, based on your board revision version.

## Boot Image Packing Configuration: As we are only booting the images without needing any file read/write or permanent package installation, we create minimal boot images, where we don't need to have a "root" partition in SD. We do configurations as follows

### Root filesystem type: INITRD

### INITRAMFS/INITRD Image name: petalinux-image-minimal

### Kernel Bootargs: Edit to add CMA value to 1000M. Depending on the video resolution, color format, we may also need to increase the CMA value.

# Kernel Configuration

$ petalinux-config -c kernel

## Unlike baremetal applications, we generally don't write the code for TPG and VFBWR IPs; rather, we enable drivers for them in Petalinux kernel configuration. These are the Linux drivers, provided by Xilinx for their IPs. For more details on Linux drivers, check this: https://github.com/Xilinx/linux-xlnx

### Enabling TPG driver: Device Drivers>Multimedia support>Media drivers>V4l platform devices > Xilinx Video IP >

In many cases, the Xilinx IP drivers are already enabled. However, we need to make sure the Xilinx Video Test Pattern Generator is enabled.

# RootFS Configuration

$ petalinux-config -c rootfs

As we need to capture and display the frame, we need to include a few File System and Petalinux packages, as follows;

## Adding capture-related driver packages: Filesytem Packages > misc > v4l-utils

v4l-utils, libv4l, and media-ctl packages are to be enabled.

## We can also enable Yavta packages, which will enable us to capture the frame for early verification.

## Adding display-related driver packages: Filesystem Packages > x11 > base > libdrm

libdrm, libdrm-tests, and libdrm-kms are to be enabled.

## To be able to use the GStreamer application, we need to enable GStreamer packages: Petalinux Package Groups> packagegroup-petalinux-gstreamer

## We also need to enable v4l2 utility packages: Petalinux Package Groups>packagegroup-petalinux-v4lutils

## Enabling auto-login: This enables us to directly log in to the Linux project without entering the credentials. For this reference design, we can enable it by Image Features>auto-login.

C.2.2. Build and Deployment

In the above section, we covered the basic configuration for our FPGA design. We can proceed with the building process.

$ petalinux-build

Unless you get tool-related or version-related or OS-related or petalinux-dependencies issues, there should be no problem in the petalinux-build itself.

Once the petalinux build completes, we need to generate a boot.bin file by packaging fsbl, uboot, pmufw, and bitstream files.

Command to generate boot image.

$ petalinux-package --boot --format BIN --fsbl images/linux/zynqmp_fsbl.elf --u-boot images/linux/u-boot.elf --pmufw images/linux/pmufw.elf --fpga images/linux/system.bit --force

We will copy the following files into the SD card.

image.scr, boot.bin, and image.ub

These files are sufficient for testing and verifying our design.

We are ready to run the design on the board. But before that, we need to set the boot configuration in SD mode by the following SW configuration. The configuration details can be found in the user guide.

Check the Board Setup section for board configuration.

UART Boot Log

Once we set up the UART and power the board, we see the boot logs rolling without any issue. Once boot completes, we start configuring the capture and display interfaces through a series of commands.

Media Topology/MediaGraph

# media-ctl -p /dev/media0

We can see the media graph of our design.

Media controller API version 5.10.0
Media device information
------------------------
driver          xilinx-video
model           Xilinx Video Composite Device
serial
bus info
hw revision     0x0
driver version  5.10.0
Device topology
- entity 1: vcap_v_tpg_0 output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0010000.v_tpg":1 [ENABLED]

- entity 5: a0010000.v_tpg (2 pads, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:RBG888_1X24/0x0@1/30 field:none colorspace:srgb]
pad1: Source
[fmt:RBG888_1X24/0x0@1/30 field:none colorspace:srgb]
-> "vcap_v_tpg_0 output 0":0 [ENABLED]

Note: to be able to see the media graph, the TPG and VFBWR IPs must be probed successfully. The following commands can be entered to confirm whether devices are successfully probed or not

# dmesg | grep tpg
# dmesg | grep frmbuf

You must see these logs if probed/registered successfully.

....
xilinx-video amba_pl@0:vcap_v_tpg_0: device registered
....
....
xilinx-frmbuf a0000000.v_frmbuf_wr: Xilinx AXI frmbuf DMA_DEV_TO_MEM
xilinx-frmbuf a0000000.v_frmbuf_wr: Xilinx AXI FrameBuffer Engine Driver Probed!!
...

If you still have issues, make sure pl.dtsi is properly generated and has the necessary IP device tree elements.

Now, we have to configure the media graph with proper resolution and format. As we are working with 1920x1080 resolution with RGB color format, we use the following commands for the configuration.

# media-ctl -v -d /dev/media0 -V "\"a0010000.v_tpg\":0 [fmt:RGB888_1X24/1920x1080 field:none]"

This sets our TPG-VFBWR pipeline in 1920x1080 resolution with 24-bit RGB format.

Note: We need to set the resolution that our display supports. To check the device-supported resolution, check it with the Modetest command in the following section.

Display Configuration

# To be able to see the TPG frame on the display monitor, we need to do the display configuration. Note that we are leveraging PS-DP. Make sure the DP monitor is connected. To get display information, we use the modetest command.

# modetest -M xlnx

It should print the display information without any issues. As the ZCU102 PS DP supports a maximum of 1920x1080@60Hz resolution, we see the supported resolution details in the log.

Encoders:
id  crtc  type  possible crtcs  possible clones
42  41  TMDS  0x00000001  0x00000001
Connectors:
id  encoder status    name    size (mm) modes encoders
43  42  connected DP-1            530x300   25  42
modes:
index name refresh (Hz) hdisp hss hse htot vdisp vss vse vtot)
#0 1920x1080 60.00 1920 2008 2052 2200 1080 1082 1087 1125 148500 flags: phsync, pvsync; type: driver
...
...
...
CRTCs:
id  fb  pos size
41  46  (0,0) (1920x1080)
#0 1920x1080 60.00 1920 2008 2052 2200 1080 1082 1087 1125 148500 flags: phsync, pvsync; type: driver
props:
25 VRR_ENABLED:
flags: range
values: 0 1
...
...
Planes:
id  crtc  fb  CRTC x,y  x,y gamma size  possible crtcs
39  0 0 0,0   0,0 0         0x00000001
formats: VYUY UYVY YUYV YVYU YU16 YV16 YU24 YV24 NV16 NV61 GREY Y10  BG24 RG24 XB24 XR24 XB30 XR30 YU12 YV12 NV12 NV21 XV15 XV20
props:
9 type:

# Configuring the display.

# modetest -D fd4a0000.display -s 43:1920x1080-60.00@BG24 -w 40:"g_alpha_en":0&

# We can confirm/test the display configuration by the following GStreamer command, using the video test screen as a source.

# gst-launch-1.0 videotestsrc ! video/x-raw, width=1920, height=1080, format=RGB, framerate=60/1 ! queue max-size-bytes=0 ! kmssink bus-id=fd4a0000.display

The following picture shows the test screen output.

Running the Pipeline

We have both capture and display interfaces working.

Let's execute the final GStreamer commands to run and display the source frame.

# gst-launch-1.0 v4l2src device="/dev/video0" io-mode=2 ! video/x-raw, width=1920,height=1080, format=RGB, framerate=60/1 ! queue max-size-bytes=0 ! kmssink bus-id=fd4a0000.display

Through TPG's v4l2 driver configuration, we can also set different test patterns as well as enable the moving box.

D: Board Setup

E: Output

We see exactly the same 1920x1080 test pattern on the monitor from both bare-metal and Linux applications.

F: What's Next?

Here, I used the same hardware design but for different application design approaches. This reference article provides a brief overview of the differences between bare metal and Linux applications running on the ZCU102 Development board. We will cover similar designs in detail in the upcoming days.

Don't forget to Like, Share, Comment, and Follow.