Empowering DUNE: A Guide to Starting with KRIA KR260, Vivado

Before start, please review this tutorial.

In the DUNE experiment, the photon detection system, DAPHNE, has undergone an evolution leading to the utilization of System on Modules (SoM). This transition aims to achieve increased compactness in hardware and software task implementation, enhanced system control, and operational security. Considering the experiment's context, the system will be situated 1 km underground in the Fermilab caverns, making future system servicing challenging. Consequently, stringent security measures and robustness are required in the system's implementation. This is where the KRIA KR260 takes center stage as the chosen SoM to manage the photon detection system's operations.

This tutorial will demonstrate the essential steps to begin working with the KRIA KR260 for high-performance embedded system projects. It will utilize Vivado tools to build the entire Programmable Logic (PL), establish connections with designed embedded hardware, and integrate with the Processing System (PS), necessitating specific configuration from Vivado. Additionally, the tutorial will cover integration with Petalinux, a Linux-based operating system developed by Xilinx for managing FPGA systems.

Vivado serves as the integrated development environment (IDE) tailored for Verilog or VHDL, focusing on the hardware development phase within Xilinx FPGA's programmable logic design. Vitis, on the other hand, acts as a software development IDE, enabling the creation of bare-metal or Linux-based C/C++ applications. These applications operate on either a physical ARM-core processor or a soft-core processor integrated into the Xilinx FPGA. Lastly, PetaLinux constitutes a collection of software tools derived from the open-source Yocto project. These tools are instrumental in crafting embedded Linux images specifically designed for Xilinx FPGAs.

Creation of the Vivado project

The initial step involves launching the Vivado tool. This can be achieved by sourcing the software in Linux and executing the following code to access the IDE:

source /tools/Xilinx/Vivado/2022.2/settings64.sh
vivado

Once the Vivado tool is open, proceed with the creation and setup of the project by following these steps:

• Select Create Project to begin the project creation process.
• Provide the project with a suitable name; for instance, let's name it Kria_KR260.
• In the Project Type window, ensure the options Project is an extensible Vitis platform and Do not specify sources at this time are checked. This marks the initial requirement for a hardware-accelerated design.
• Move to the Part page and navigate to the Boards tab. Click on Refresh, then search for KR260. Once located, single-click on the Kria KR260 Robotics Starter Kit row and proceed by clicking Next.
• The final page presents a summary of the chosen configurations for the new project. Click on Finish to initiate the creation of the new Vivado project.

Create Block Design

ChatGPT

The initial step in a new Vivado design process is to create a block design. This block design functions as a graphical programming tool where each block represents an HDL code. These blocks are configured and interconnected using this graphical user interface (GUI).

Within the Project Manager section, under the IP Integrator tab, select the Create Block Design option and provide it with the desired name in the subsequent pop-up window.

After that, an empty block design window will open:

To begin, the first component required is the processor, particularly for the KRIA, which integrates a physical ARM-core processor on the Processor System (PS) side. To add this, click the + button and search for "zynq".

Double-click on the Zynq MPSoC IP to incorporate it into the block design. This IP block allows users to customize hardware settings for the ARM-core processor within the KRIA SoM. Upon addition, a green banner will appear at the top of the Diagram window, providing the option to Run Block Automation. This automation will apply specific KR260 board presets to the Zynq MPSoC IP block.

• Click the Run Block Automation link and ensure that the Apply Board Preset option is selected in the window that appears. Then, click OK.
• After running the Block Automation, the port assignments within the Zynq UltraScale+ block will be adjusted to align with the presets of the KR260 board.

Modifying the IP Core Configuration

We suggest performing a basic configuration to enable hardware acceleration. This involves configuring the hardware in Vivado to ensure future access from the software in Vitis/PetaLinux while applying customizations to the Zynq UltraScale+ IP.

To customize an IP within the block design, double-click on it to access the configuration window.

Here are the steps to modify the configuration:

• Disable the two full-power high-performance AXI ports within the block design.
• Enable the low-power high-performance AXI port.

This action ensures that the two full-power high-performance AXI ports within the block design will not be utilized in any future updates of this hardware design. We aim to reserve these ports exclusively for hardware acceleration purposes later on.

It's essential to note a slightly counter-intuitive aspect of hardware acceleration within Vivado: disabling a component in the block design does not entirely deactivate it across the entire project. It remains an option that can be enabled in the Platform Setup step, thereby making it accessible to the software in Vitis.

Adding the Clocking Wizard IP

The subsequent step involves integrating a clocking wizard IP into the design to function as a clock source controlled by the Zynq UltraScale+ IP. This enables the clocking wizard to be enabled in the Platform Setup step, thereby becoming accessible to the software in Vitis for hardware acceleration.

Follow these steps:

• Double-click on the clocking wizard IP to access its customization options.
• Within the Output Clocks tab, enable the required number of output clocks, specifying the desired rates. Additionally, adjust the polarity of the reset to match that of the reset signal originating from the Zynq UltraScale+ IP block (which is active low).
• Connect the pl_clk0 output from the processor to the clk_in1 input of the clocking wizard IP.
• Connect the pl_reset0 output from the processor to the resetn input of the clocking wizard IP.

Integrating the Processor System Reset IP

To proceed, add Processor System Reset IP blocks individually for each output clock of the clocking wizard IP. Establish the following connections:

• Connect the ext_reset_in of each Processor System Reset IP to the pl_reset0 output of the Zynq UltraScale+ IP.
• Connect the clk_outn output of the clocking wizard IP to the slowest_sync_clk of each respective Processor System Reset IP.

Ignore all connection automation options as they will become irrelevant once the specified connections above are established.

Incorporating the AXI Interrupt Controller IP

To enable interrupt functionality for hardware-accelerated designs, an AXI interrupt controller IP needs to be added. The AXI interface is a standard connection interface utilized by Xilinx for IP blocks in hardware designs due to its high configurability and automation capabilities in the connection process.

Follow these steps:

• Add the AXI interrupt controller IP to the design.
• Double-click on the block and modify the Interrupt Output Connection from Bus to Single.
• Utilize the Run Connection Automation option to automate the configuration and connection of the AXI interrupt controller IP to the Zynq UltraScale+ IP through the AXI interface.
• Establish the connection by linking the irq output from the AXI interrupt controller IP to the pl_ps_irq input of the Zynq UltraScale+ IP.

Additionally, during the configuration of the connection automation, select the desired clock for the AXI interface that will serve as the default for the hardware acceleration design. In this case, use the 200 MHz clk_out2 from the clocking wizard IP. Ensure consistency by setting the clock source for the driving bridge IP, slave interface, and master interface to the same clock source as the one chosen. This alignment ensures synchronization among different components using the same clock source within the design.

Platform Configuration

As previously mentioned, to grant access to hardware resources for software utilization in Vitis for hardware acceleration, they must be enabled within the Platform Setup tab of the block design.

• The initial section in the Platform Setup tab involves selecting the AXI ports to be accessible in Vitis. Enable all except the low-power high-performance AXI port on the Zynq UltraScale+ IP. Additionally, enable 8 general-purpose Master AXI interfaces from the regular AXI interconnect. Take note of any names set in the SP tag field, as these identifiers will be used in Vitis software to target specific AXI ports.
• Following that, choose the clocks to be available in Vitis. In this scenario, only enable the two output clocks from the clocking wizard, setting the 200 MHz clk_out2 as the default clock.
• Enable the irq output from the AXI interrupt controller IP in the Interrupt section.
• Lastly, specify the desired name, vendor name, and version number for the platform.

Apply Basic Configuration for PS Peripherals in KR260 Platform

The primary configuration required involves enabling respective ports for each peripheral on the Zynq MPSoC chip. Open the block design within the Vivado project and double-click on the Zynq UltraScale+ IP block to access its configuration window.

Note: MIO pins are dedicated interface pins on the Zynq MPSoC chips connected directly to the ARM-cores. These pins are routed to specific package pins of the chip, inaccessible to the PL (Programmable Logic) of the FPGA, and are unchangeable, eliminating the need for manual mapping of signal names to pin numbers in a constraints file.

Note: EMIO pins are routed from the PS side through the PL side to the external ports of the FPGA. These pins traverse through the PL side to any FPGA pin, making them interchangeable, requiring signal name mapping to pin numbers in a constraint file.

Here are the steps:

• Begin with the I/O Configuration tab. Under Low Speed > Memory Interfaces, verify the QSPI settings.
• Under Low Speed > I/O Peripherals, enable I2C1 on MIO pins 24 - 25.
• Confirm the PMU settings.
• Enable SPI1 on MIO pins 6 - 11.
• Enable UART1 on MIO pins 36 - 37.
• Enable GPIO0 MIO and GPIO1 MIO.
• Enable both system-wide watchdog timers SWDT 0 and SWDT 1 under Processing Unit.
• Enable all four triple timer counters (TTC0 - TTC3), with the first outputting its wave out signal to EMIO.
• Under High Speed for the RJ45 Ethernet ports on the KR260, enable GEM 0 on GT Lane0 and GEM 1 on MIO pins 38 - 49, with its MDIO on MIO pins 50 - 51.
• Under USB0, enable USB0 on MIO pins 52 - 63 and USB 3.0 on GT Lane2.
• Under USB1, enable USB1 on MIO pins 64 - 75 and USB 3.0 on GT Lane3.
• Configure the reset pins for the USB ports to use separate MIO pins with an active low polarity. Assign USB 0 reset to MIO pin 76 and USB 1 reset to MIO pin 77.
• Enable the DisplayPort on MIO pins 27 - 30 with a Single Lower lane selection on GT Lane1.
• Under General > Fabric Reset Enable, increase Number of Fabric Resets from 1 to 4.

Skipping the SATA or PCIe peripherals as they lack physical routing on the Kria or KR260 PCBs.

The Clock Configuration and DDR Configuration were handled by the block automation for the Zynq MPSoC IP, which applied the Kria board presets.

Proceed directly to the PS-PL Configuration tab (PS = Processing System, PL = Programmable Logic).

In the end, the block design tab should have check marks on each of the enabled peripherals:

To see how each of these peripherals connect on the KR260 PCB, check out and download its schematic here. There is a very helpful block diagram overview on the second page.

CPU Fan Driver IP Block Design

To control the CPU fan on the Kria using the necessary PWM signal, the wave output from TTC0 is utilized. Out of the 3 bits available in the TTC0 wave output signal, only bit 0 is required.

Add an Xslice IP block and configure it to accept a 3-bit input, outputting specifically the least significant bit (LSB), bit 0:

Connect the Din input of xslice_0 to emio_ttc0_wave_o of the Zynq MPSoC IP.

Subsequently, right-click on the Dout pin of xslice_0 and opt for the Make External option. This action generates a port pin and automatically establishes the connection between the Dout pin and this newly created port. Rename the port to fan_en_b, and then regenerate the layout.

Please validate the block design and save it once more. Then, proceed to select Generate Block Design from the Flow Navigator window.

Following that, create a new HDL wrapper and choose the option to allow Vivado to auto-manage (refer to the previous instructions on how to create an HDL wrapper).

CPU Fan Constraint

To link the fan_en_b pin in accordance with the PCB design for the KR260 board (refer to the schematic), create the fan_pinout.xdc constraints file and input the following code. Then, save the constraint file:

set_property BITSTREAM.GENERAL.COMPRESS TRUE [current_design] 

#Fan Speed Enable
set_property PACKAGE_PIN A12 [get_ports {fan_en_b}] 
set_property IOSTANDARD LVCMOS33 [get_ports {fan_en_b}] 
set_property SLEW SLOW [get_ports {fan_en_b}] 
set_property DRIVE 4 [get_ports {fan_en_b}]

Validate and Save the Block Design

After incorporating and linking the desired IPs in the Diagram and completing the Platform Setup configuration, it's essential to validate and save the entire block design. In the Diagram tab of the block design, click the checkbox icon to initiate a design validation.

A critical warning may arise concerning the unconnected input interrupt on the AXI interrupt controller, which can be safely disregarded in this context.

A critical warning may arise concerning the unconnected input interrupt on the AXI interrupt controller, which can be safely disregarded in this context.

After clicking OK to dismiss the critical warning, click the save icon in the menu bar to save the block design.

Disable the Incremental Synthesis

The design checkpoint files created by the default incremental synthesis option in Vivado cause issues with the hardware acceleration workflow. Therefore, before initiating any synthesis run, it's crucial to disable this feature.

To disable incremental synthesis:

• Navigate to the Synthesis tab by selecting Settings in the Flow Navigator window.
• Click on the three dots located next to the incremental synthesis option.
• From the subsequent pop-up window, choose the option to Disable incremental synthesis.
• Click OK to apply the new settings.
• Additionally, in the Bitstream menu, select the bin checkbox to generate both *.bit and *.bin files simultaneously.
• Click OK to confirm the changes and return to the Vivado project.

Generate Block Design

The subsequent step in a Vivado project, especially when it's an extensible Vitis platform, involves generating the block design during its initial run.

Proceed by selecting Generate Block Design from the Flow Navigator window. Change the Synthesis Options from Out of context per IP to Global.

Finally, click Generate to initiate the process.

Create HDL Wrapper

After completing the block design, validating, saving, and generating it, the next step involves creating an HDL wrapper file to instantiate it within the overall design. Fortunately, Vivado offers the capability to automatically generate and update this wrapper.

In the Sources window, right-click on the block design file. Select the option to Create HDL Wrapper...

Next, choose the setting that allows Vivado to manage the wrapper and enable automatic updates, then click OK in the pop-up window.

Following a brief moment, the HDL wrapper file will appear in the Sources window.

Generate the Bitstream

It's time to generate a bitstream for the design. For those new to FPGA design, the workflow typically involves three main steps: synthesis, implementation (place & route), and finally, the creation of a bitstream. Synthesis converts HDL code to a transistor logic configuration, implementation (place & route) deploys the logic design onto a specific FPGA chip, and the bitstream is the resulting file used to program the FPGA with that specific logic configuration.

While you can manually execute each of these steps, Vivado is designed to detect and execute any previous steps that might be outdated or not yet run.

In a new project, upon selecting to generate a bitstream from the Flow Navigator window, Vivado will automatically recognize the need to execute synthesis and implementation beforehand:

Click OK to initiate the runs for synthesis, implementation, and bitstream generation.

Export the Platform for Software Development

To utilize the hardware design in Vitis and/or PetaLinux for software development, it must be packaged into a platform file.

Select the option Export Platform from the Flow Navigator window.

In the platform packaging window, choose Hardware for Platform Type (as Kria currently lacks emulation support), and ensure to check the option to include the bitstream in Platform State.

Finally, provide the platform with the desired name, vendor/version information, description, and select the preferred output directory for the exported platform.

At this stage, this Vivado project serves as an ideal foundation for integrating custom hardware into the design and subsequently importing it into Vitis and/or PetaLinux. The next tutorial will cover the process of importing this exported platform into PetaLinux 2022.2 and incorporating hardware acceleration support into the kernel/root filesystem.