Connecting an SSD to an FPGA with PetaLinux

When we design a project with Zynq where we will use Petalinux as the main OS, in general, all the configurations we make, or the files we create, are stored in DDR modules, which are based on volatile memory. If we want to save data to non-volatile memory, we have the option of saving data to SD card, but for some applications where high data throughput is needed, SD cards can be slow.

Zynq devices has a lot of interfaces, including gigabit interfaces on the ports GTX, GTH or GTR. These ports can act as Sata interface, USB3.0, PCIe, and is this last one which we will use to communicate an NVME SSD hard disk. Let's start!

Downloading TE0802 board files using XHub Stores.

On last version of Vivado, Xilinx added a board download manager. Using it, we can download the board files directly from Vivado, without search on the board vendor webpage or Github. To use it, on the Vivado welcome page, we have to click on XHub Stores.

Accept the next window and then, the XHub stores will be opened. Next on Boards tab, we have to find the board we will use. In this case, we will navigate to the Trenz Electronic folder, and find TE0802.

Then right click, and install, and the board will be installed, and a green tick will appear.

Now we have the board installed, and the XHub store window can be closed.

Creating the Vivado project and the block design.

The next step is create the project, to do that, on the Vivado welcome page, we have to click on Create Project.

Write a name for the project, in my case te0802_nvme

Select RTL as project type, and we will not add any code or constraints. Clicking on Next button we will see the part selection window. On that window, we will open the Boards tab, and find the board we have just install.

Clicking on Next button, we can see the project summary, and then, the project will be opened.

Now, we will create the block design by clicking on Create Block Design on the left column.

Select a name for the block design, and the block design will be opened. Then we have to add the Zynq device to our Block Design. To do that, click on + icon and then search Zynq on the pop up window.

Once added the device, we have to apply the board preset to configure all the interfaces and the DDR, for it, click on Run Block Automation.

And, on the opened window, ensure that Apply Board Preset will be select, and then click on OK.

This step will apply to the Zynq all the configurations to peripherals that the manufacturer has include on the board. If we open the Zynq configurator by double click on the Zynq device, we can see all the peripherals that the manufacturer has enabled on the device. For this project, we have to be sure that PCIe peripheral is enabled.

To ensure that, we can click over PCIe Gen2 x1/x2/x4 peripheral, and ensure that the peripheral is enabled, and the Lane selected is correct, in this case, Lane 0.

Also, on clock configuration tab, we can assign a Clock to each GT peripheral, according to the clock used on the hardware. In this case, for PCIe peripheral, the RefClk0 has to be selected, with a frequency of 100MHz.

Once verified the configuration, again on block design, we will connect the AXI LPF clock, and audio clock to pl_clk0.

Then verify by clicking on the tick on the top of the Block design. If all was good, there are neither errors or critical warnings on the design.

Finally, on sources tab, Generate HDL Wrapper by right click on the block design file.

And ensure that the option Let Vivado manage wrapper and auto-update is selected.

When the wrapper is done, the next is Generate Bitstream.

When the bitstream is generated, click on cancel on the pop up window.

And then go to File > Export > Export Hardware

Select Fixed as Platform Type, and Includebitstream on the platform.

Click on Next button, check where Vivado will save the.xsa file and Finish.

Creating the Petalinux project.

To create any Petalinux project, we will need a machine with a Linux distribution installed. In my case I use Ubuntu as a main OS, but if it is not your case, you can use a virtual machine and follow the step on UG976 to install Petalinux.

Once Petalinux is installed on our machine, we will go to the petalinux folder, and we will add the environment variables to the path.

cd <petalinux_install_dir>
source ./settings.sh

The result on the terminal will looks lithe this.

Next we are going to create the petalinux project with the name te0802_nvme_ptlnx.

petalinux-create --type project --template zynqMP --name te0802_nvme_ptlnx

Now, we go to the petalinux project folder just created, and we will connect the hardware definition we have create on Vivado to this Petalinux project.

petalinux-config --get-hw-description ~/xilinx/te0802_nvme/

Then a terminal GUI will be open. On this configurator, the only confguration that I have change is the IP address, because I will connect the board directly to the computer, so there aren't a DHCP server. To change to Manual IP address we will go to

Subsystem AUTO Hardware Settings -->
Ethernet Settings -->
[ ] Obtain IP address automatically = N
(192.168.1.10) Static IP address
(255.255.255.0) Static IP netmask
(192.168.0.1) Static IP gateway

Now, exit and save.

Next, we will configure the kernel, to do that, we have to execute the next command

petalinux-config -c kernel

After a while, a menu will be opened. At that point we have to navigate to

Device Drivers > NVME Support

And select the options NVM Express block device and NVME Target Support.

Then Exit and Save the configuration.

Next we will add some functions to rootfs. The command to configure the rootfs

petalinux-config -c rootfs

Now, on the menu, ensure that pciutils is selected.

Filesystem packages > console > utils > pciutils

Now, add fsdisk, blkid and util-linux

Filesystem packages > base > util-linux

Then exit and save.

Finally, we can build Petalinux

petalinux-build

After a while...

Now we have to package our Petalinux distribution, to do that we have to navigate to images > linux, and execute the petalinux-package command.

cd images/linux
petalinux-package --boot --fsbl zynqmp_fsbl.elf --u-boot

Now we will copy image.ub, BOOT.bin and boot.src from images/linux folder to an SD card.

Insert the Sd card into the TE0802 and ensure that the boot configuration is good for SD boot. As we had change the IP address, we can connect to the board through ssh.

ssh root@192.168.1.10

Performance test.

Once we will check if the SSD has been detected by Petalinux, by executing the next command

fdisk -l

On the bottom of all DDR modules, we will find the disk.

In my case, the disk has a created a partition, but in case of the new disk, we have to create the partition. To do that we will check if the SSD has been detected as block device.

lspci

Then, with the name that appears, we have to create a partition.

fdisk /dev/nvme0n1p1

Type n to create a new partition, then type p, then 1 to create a new primary partition, and the rest will remain as default. Finally, type w to write the configuration to the disk. Now, when we execute again lspci, we can see the SSD and the partition created.

Last we will perform an speed test to our device using the dd command. To perform a write test, the command we have to execute is the next.

time dd if=/dev/urandom of=/dev/nvme0n1p1 bs=2M count=1000

and to read test

time dd if=/dev/nvme0n1p1 of=/dev/null bs=2M count=1000

These commands will return the time spent in the write operation and read operation.

Regarding write operation, the board has spent 32.698 seconds to move 2Mbits 1000 times, so the speed can be calculate as 1000*2Mb/32.698s = 61Mb/s

Read operation will spent 9.319 on the same operation so the speed can be calculated as 1000*2M/9.319 = 216Mb/s

If we compare these numbers, with the datasheet numbers of the Corsair MP510 (3.1Gb/s read, 1.05Gb write), maybe they can be disappointing, but, the SSD will achieve the maximum performance using its 4 lanes. Also, the test I did is done on Petalinux, so the speed will depend on how the operating system manages the read and write operations.

To me, this results are quite good, and an important thing is that with this project, we will have a non volatile memory for store data from sensor, or a webpage.