Getting Started with AMD Spartan™ 7 FPGA in PetaLinux 2023.2

In my previous post, I showed how I created hardware designs on the AMD Spartan™ 7 SP701 development platform depending on whether I was planning on developing C code applications for the MicroBlaze soft-processor with or without a Linux OS running on it. So I'm following up with this project tutorial to show how to create the Linux OS to run on the MicroBlaze soft-processor using PetaLinux if choosing that hardware design route.

Again, this project series is specific to the 2023.2 version of the AMD FPGA toolset. I'm using an Ubuntu 22.04 host development environment, for which I've documented the entire 2023.2 AMD FPGA tools installation process here.

Create PetaLinux Project

Just like in AMD Vivado™ design tools with board preset files, a PetaLinux project can be created with the initial configuration already done for a specific FPGA development board by using the BSP (board support package) for that board. The PetaLinux BSPs for AMD FPGA development boards are in the same downloads area as the PetaLinux installer located here.

To create a PetaLinux project from a BSP, use the -s flag with path to the.bsp file:

~$ petalinux-create -t project -s ../Downloads/xilinx-sp701-v2023.2-10140544.bsp

The -name flag can still be used for a project created from a BSP if a custom name for the project is desired, otherwise the PetaLinux project name defaults to the name of the.bsp file.

Update Constraints with QSPI Flash Parameters

As I mentioned in my previous project write-up for creating the Spartan™ 7 SP701 development platform hardware design in Vivado to build a PetaLinux project on, I prefer to just use the Vivado project created by the PetaLinux BSP as a starting point. Then I can just modify it to add anything my specific project needs.

This Vivado project is the hardware design that the PetaLinux BSP uses to create all of the files in the ./pre-built directory of the PetaLinux project. The Vivado project itself is located in ./hardware/<BSP name>/, so in the case of the 2023.2 SP701 BSP it is located in ./hardware/xilinx-sp701-2023.2.

While this hardware design is always complete enough for a basic JTAG boot for testing, I found that the specific configuration in the constraints file for the QSPI flash that ultimately passes the necessary SPI bus information to the boot loader was missing.

To get to these particular settings to configure them, an up-to-date Implemented Design has to be open in the Vivado project. So launch an implementation run if it shows as out-of-date when the Vivado project is open, and select the option to open the implemented design in the pop-up window when it completes.

Once the Implemented Design is open, select Settings from the Flow Navigator window. Then in the Bitstream tab, select the option to Configure additional bitstream settings.

Under Configuration Modes, I found that only the default JTAG boundary scan option was enabled (this is required for all boards for the JTAG to communicate with it so leave it enabled).

But the configuration mode for QSPI flash's SPI bus wasn't specified. The data bus for the flash chip of the AMD Spartan™ 7 SP701 FPGA is 4 bits wide and the Spartan™ 7 FPGA is the master device on that SPI bus, so also enable the option for Master SPIx4:

Then switching to the Configuration tab, make sure the Configuration Rate (MHz) is set to 3, then set the Configuration Voltage to 3.3 and the Configuration Bank Voltage Selection to VCCO.

Since the SP701's flash is 1GB in size, the default 24-bit address values won't be able to index its full size (max size of a 24-bit address is 128MB). So the option to Enable SPI 32-bit address style needs to be set to YES.

Then set the SPI's Bus width to 4. It is possible to use 1 or 2, but this means the data bus of the SPI interface to the flash chip is narrowed down to one or two bits respectively, which means flashing the QSPI chip will take longer, as well as Linux boot times will also take longer.

Note: If the bus width is set to 1 or 2, the configuration mode needs to be changed to Master SPI x1 or Master SPI x2 respectively to match.

Click OK on each of the settings windows to save the edits, then close the Implemented Design. A prompt will appear notifying that edits have been made to the design and will ask if the changes should be added to the existing constraints file or written to a new one (I chose to add it to the existing constraints file, but either option works).

After opting to save the edits to the Implemented Design and closing it, open the constraints file to verify the following lines were appended to it:

set_property CFGBVS VCCO [current_design]
set_property CONFIG_MODE SPIx4 [current_design]
set_property CONFIG_VOLTAGE 3.3 [current_design]
set_property BITSTREAM.CONFIG.CONFIGRATE 3 [current_design]
set_property BITSTREAM.CONFIG.SPI_BUSWIDTH 4 [current_design]
set_property BITSTREAM.CONFIG.SPI_32BIT_ADDR YES [current_design]

After validating the new constraints are indeed present in the constraints file, run synthesis, implementation, and generate a new bitstream.

Once the new bitstream has been successfully generated export the hardware XSA file so that it can be pulled into the PetaLinux project in the next step so the software will be aware of the new hardware hooks to the QSPI that were just added.

Be sure to select the option to include the bitstream in the exported XSA file, otherwise none of the necessary changes made above will make it into the PetaLinux project.

I personally chose to export this XSA to a different location than the one the original XSA was exported to just so I have the original to revert back to if need be. So I chose to export it to the top level directory of the PetaLinux project for easy access:

Import Hardware from Vivado™ Design Tools

To import the new hardware definition into the PetaLinux project, the PetaLinux system hardware configuration editor is launched using the --get-hw-description flag with the path to the target XSA file (but to do not actually specify the XSA file, just the path):

~$ cd ./xilinx-sp701-2023.2/
~/xilinx-sp701-2023.2$ petaliunx-config --get-hw-description ./path_to_xsa/

Configure Hardware Parameters

Since this PetaLinux project is created from the BSP, there isn't a need to modify anything in the system hardware configuration editor for a successful boot. Changes can be made for a particular use case to add new peripherals, but be careful as some are hard requirements for the topology of the target development board.

I personally like to disable TFTP boot if I have no plans to use it as it cuts down both on the PetaLinux project build time and the overall final root filesystem size.

The option for TFTP boot is located under Image Packaging Configuration.

It is also worth noting here that since everything is going to be living in the QSPI flash of the SP701 and therefore packaged into the boot binary file, the only two options that can be used for the Root filesystem type are INITRD or INITRAMFS since they can be integrated/built into the kernel image (that's then packaged into the boot binary).

The default is INITRD, and since the QSPI flash is 1GB on the SP701 I'm not worried about running out of space so I didn't both changing it to INITRAMFS for the extra compression.

The PetaLinux BSP for the SP701 has also taken care of the configuration for the extra support required for integrating an INITRD filesystem into the kernel. So there is nothing that needs to be modified in the kernel settings for a successful boot.

Configure Kernel

If there is the need to modify the kernel settings to add support for a certain peripheral or something, the kernel configuration editor for the PetaLinux project can be brought up with the following command:

~/xilinx-sp701-2023.2$ petaliunx-config -c kernel

Until I decide which Pmod and FMC boards I'm using with the SP701 for my next project, I'm not modifying them from the defaults as set by the SP701's BSP.

Configure Root Filesystem

Likewise, if there is a need to modify the root filesystem to add additional libraries or packages, the root filesystem configuration editor for the PetaLinux project can be brought up with the following command:

~/xilinx-sp701-2023.2$ petaliunx-config -c rootfs

Build PetaLinux Project

With the PetaLinux project configured to build the Linux image needed for the SP701, it's time to build the project:

~/xilinx-sp701-2023.2$ petaliunx-build

A "clean" or first time build does take the longest, but subsequent builds only compile detected changes.

Should the need arise to clean a PetaLinux project in order to run a full new build, the following command can be used:

~/xilinx-sp701-2023.2$ petaliunx-build -x mrproper

Build SDK

While it's possible to write a Linux application in a text editor then build it into the PetaLinux project with its Makefile in the apps recipe of the PetaLinux project (PetaLinux is Yocto-based by the way), there is also the option to develop the application in the Vitis Unified IDE.

In order for the Vitis Unified IDE to be able to build an application for the specific Linux OS we just built for the SP701, a sysroot needs to be created for it. This is basically a dummy copy of the root filesystem that will be running on the SP701 but is contained in a directory on the host PC that we can point Vitis to for Vitis to run builds with.

To create a sysroot for a PetaLinux project, run another build with the --sdk flag:

~/xilinx-sp701-2023.2$ petaliunx-build --sdk

For the details of how to build a Linux application to run on the SP701 using the Vitis Unified IDE, see my other project write-up in this series here.

Test Linux Image on AMD Spartan™ 7 SP701 Development Platform with JTAG Boot

Before I spend the time to build a boot binary, flash the QSPI of a development board, or anything like that in a PetaLinux project, I like to verify that the Linux image can indeed boot and work correctly over JTAG first. This also narrows my scope during troubleshooting later on.

Connect the SP701 to the host PC via either the J5 USB port or a JTAG adapter on J3. Change directories into the build output directory of the PetaLinux project (./images/linux/) then use the petalinux-boot command to program with FPGA with the bitstream:

~/xilinx-sp701-2023.2$ cd ./images/linux/
~/xilinx-sp701-2023.2/images/linux$ petalinux-boot --jtag --fpga

Once the bitstream is programmed on the FPGA, boot the Linux kernel:

~/xilinx-sp701-2023.2/images/linux$ petalinux-boot --jtag --kernel

Open a serial terminal application connected to the second indexed USB port created by the SP701 from the J5 USB port (the J5 USB port is required to view serial output, but the JTAG interface can go over it or the J3 JTAG port). Also be sure that SW13 is in the JTAG boot mode configuration as shown below:

Since the Linux OS booted without issue via JTAG, this tells me that any issues that occur after I program the QSPI flash and try to boot from it will be related to either how I packaged the boot binary file or how I configured the QSPI flash settings.

Create Boot Binary

Since the only external non-volatile memory on the SP701 board is the QSPI flash memory chip, this means everything has to be booted from that QSPI flash and therefore QSPI boot mode is implemented.

This in turn means everything for the Linux system needs to be packaged into the boot binary to be flashed onto that QSPI. For this design, that includes the following:

• The FPGA bitstream - system.bit
• The first stage boot loader (FSBL) for the MicroBlaze - fs-boot.elf
• The second stage boor loader for Linux (U-Boot) - u-boot-s.bin
• The kernel image (the root filesystem is built into this with the INITRD/INITRAMFS filesystem type selected) - image.ub
• Boot script - boot.scr

All of these files are generated by PetaLinux when the project is built in the output directory ./images/linux/.

Since the MicroBlaze uses BRAM as its local cache and that's memory where the FSBL runs for it, a download.bit file is created for the FSBL with the.mmi file from Vivado and the FSBL's.elf file. This necessitates the Vivado tools being sourced to the environment before using the petalinux-package command to create the boot binary:

~/xilinx-sp701-2023.2$ source /tools/Xilinx/Vivado/2023.2/settings64.sh

Given that all of the source files that are being packaged into the boot binary are located in ./images/linux/ of the PetaLinux project, I changed directories into it first.

I chose the MCS file type for the boot binary because while a.mcs file is larger than a.bin file, I like having the extra file integrity of a checksum at the end of each line of the file, especially when I can afford the extra space in a large flash chip like the 1GB one on the SP701.

I also specified the flash size of 1GB (1024 because the --flash-size flag assumes MB values) and the SPIx4 interface.

~/xilinx-sp701-2023.2$ cd ./images/linux/
~/xilinx-sp701-2023.2/images/linux$ petalinux-package --boot --format MCS --fsbl fs-boot.elf --fpga system.bit --u-boot u-boot-s.bin --kernel image.ub --boot-script boot.scr --flash-size 1024 --flash-intf SPIx4

Again, if the SPI bus width was set to 1 or 2 in the pervious steps, be sure to match it here too with SPIx1 or SPIx2.

I learned the hard way that if any errors appear here about not being able to auto-detect parameters from the bitstream file then the resultant MCS file probably won't boot because the proper hardware settings haven't made it to the system.bit file.

I had some weird caching issue where PetaLinux wasn't updating the bitstream file system.bit in the output directory so I had to clean and rebuild the project to resolve it.

Program QSPI Flash on AMD Spartan™ 7 SP701 Development Platform

With the boot binary successfully generated, make sure the boot mode pins SW13 [4:2] MODE[2:0] are still set to JTAG mode and open Hardware Manger in Vivado, select Open Target, and connect to the SP701 board (power cycle the board though if the Linux image booted via JTAG is still running on it thought):

Auto Connect will work if the SP701 is the only board connected to the host PC, otherwise select Open New Target and follow the prompts to connect to the SP701 board.

Once connected, the only device that appears is the Spartan FPGA itself, so the QSPI flash chip needs to be manually added to the Hardware list. Right-click on the Spartan FPGA (labeled xc7s100 in the hardware list) and select the option to Add Configuration Memory Device:

The part number for the QSPI flash chip on the SP701 is mt25ql01g-spi-x1_x2_x4. Use the search function to locate it in the Add Configuration Memory Device pop-up window:

Click OK and the QSPI flash will then appear in the hardware list under the Spartan 7 FPGA.

Program the MCS file created in the previous step by right-clicking on the QSPI flash chip in the hardware list and selecting Program Configuration Memory Device.

In the pop-up window, browse to the boot binary created in the previous steps with PetaLinux for the Configuration File:

Click OK to start programming the QSPI flash. This will take a few minutes, and longer if the SPI bus width is set to 1 or 2.

Once programmed successfully, close hardware manager in Vivado and power off the SP701 board (just turn off power switch SW11).

Boot on Hardware

While the SP701 is powered off, change the boot mode pins SW13 [4:2] MODE[2:0] from JTAG to QSPI boot:

This did have me tripped up for a minute as when I initially referred to table 6 in the SP701 Evaluation Board User Guide (UG1319) it calls out SW13 [4:2] QSPI mode [2:0] as 001.

What I had failed to notice however in table 5 above it, is that the OFF position of each switch maps to 1/high and the ON position of each switch maps to 0/low. So I had the polarity reversed and spent a day wondering what in the world was wrong with my design that kept it from booting at all.

It's also worth noting that switch one (SW13 [1]) is connected to the INIT_B pin of the Spartan 7 FPGA, so it is important that it is on the off position as shown in my photo above.

Otherwise the INIT_B pin of the FPGA is shorted to ground which will prevent it from booting up, holding it in the configuration reset state indefinitely.

Once SW13 is set to QSPI boot mode, power on the SP701 board, and again open a serial terminal application connected to the second indexed USB port created by the SP701 from the J5 USB port (I like using Putty or TeraTerm) to see the boot process.

And that's it for the development of the Linux OS itself for the SP701, see the next project tutorial here for developing a C application to run on this OS on the SP701 using the Vitis Unified IDE.

Disclaimer/Disclosure

Whitney Knitter is an independent engineer, hired as an independent contractor by AMD. Although AMD sponsored this project write-up, including engineering hours and writing production, the opinions expressed are those of Whitney Knitter, which may not reflect the positions, strategies, or opinions of AMD. GD-5.

AMD, the AMD Arrow logo, Spartan, Vitis, Vivado, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this video are for identification purposes only and may be trademarks of their respective owners.