Image Convolution with Altera FPGAs

Sponsors

This article is to provide a basic and interactive example for Altera's devices, tools, and IP. The Agilex 7 development board was provided by Altera. Please note that I am not employed by Altera, nor do I provide official support for these products. The information provided here is intended solely for informational purposes, and all opinions expressed are my own.

Overview

Altera's Field Programmable Gate Arrays (FPGAs) are a powerful choice for video processing and AI applications, offering real-time performance and high efficiency. These devices employ 2D Finite Impulse Response (FIR) filters that work in a manner similar to the convolution layers in Convolutional Neural Networks (CNNs). Instead of learning weights through training, FPGAs use fixed, predetermined coefficients to perform specific tasks—such as edge detection—by harnessing parallel processing and pipelining. A common method involves a 3x3 convolutional kernel, a small matrix that slides over an input image, multiplying its values element-by-element with each corresponding 3x3 region and summing the results to produce a single output pixel. This process, repeated across the entire image, enables the extraction of key features like edges, blurring, or sharpening based on the kernel's configuration.

Altera FPGA

Any Altera FPGA with JTAG should work with this setup, no other interfaces are used.

List of tested FPGAs/ Boards

• Agilex 7 with Altera DK-SI-AGI027FC
• Arria 10 with Gidel HawkEye-20G-48
• Max 10 with TerAsic DECA
• Any Agliex FPGA with JTAG

JTAG provides a simple way to read and write the FPGA's internal memory of the Avalon Addressable Bus.

Quartus Project Setup

Depending on the target platform, using Quartus Lite/Std/Pro prepare a basic project. If the target needs special configurations like the Agilex 7's power sequencers, refer to any example projects and user guides provided by the Eval Board Vendor.

Platform Designer

This example is generated using the Platform Designer. Follow this Altera Video to understand the basics of the tool [link]. The design uses a JTAG to Avalon master to drive the data transfers and configure the 2D-Fir IP core. Below is the schematic output of the Platform Designer

The line by line instructions for the block diagram can be found on GitHub [link]. This uses all free IP cores provided by Altera. Each core has online documenation to explain the diferent paramters, configurations, and addressage registers.

Here is a basic explaination of each IP used:

• Clock Bridge: Defines the frequency of the input clk
• Reset Bridge: Schronizes the reset signal with the clk domain
• JTAG to Avalon Master Bridge: provides a simple JTAG interface to read and write to any of the available addresses
• System ID: Provide a basic known value to verify the read interface
• On-Chip Memory: Used as Read and Write buffers for the filter operations
• Modular Scatter-Gather DMA: Provides DMA function to and from memory
• 2D-FIR II: Provides a Convolutional kernel that operates on the Avalon Stream Data
• Avalon-ST Adapter: used to configure the bus latency between DMA engines and the 2D-FIR filter

2D FIR

The 2D-FIR II IP provides a customizable convultion that operates on the Avalon Stream data. There is a set of static paramters defined in the Platform Designer and a set of dynamic paramters defined during operation over the Avalon Master interface. These paramters are defined in the documenation [link].

These are the static configurations used in this example:

• One Color Plane: Grayscale
• Resolution set to fit On-Chip memory (Ex. 320 x 200)
• 8 bits per pixel
• 3 by 3 taps for Kernel (Convolution)
• Signed Fixed point coefficents (Sign, 1 integer bit, 7 fraction bits)
• Remove blanking (only 1 frame at a time)

You can use many 3x3 kernals for various functions such as line detection, embossing, and sharpen. Using the following link to interactivly play with different kernels

[Kernal Examples]

JTAG Access & Python Jupyter

JTAG is often used for FPGA configuration and debug features. It is the most common interface across all FPGA platforms and allows this example to be easily ported to a different target platform. Using the Quartus TCL shell, the JTAG to Avalon master can be controlled. To simplify this process, a TCL to Python interface is provided.

Follow the Python Notebook to configure the Python to TCL to JTAG interface and demo reading and writing to the FPGA [link].

Quartus Signal Tap

Using the Quartus Signal Tap debug features, you can monitor the Avalon Stream data during operation. This is great to see how the low level data is formated and verify that the 2D-FIR filter is working as expected.

After opening the Signal Tap GUI, these following signals can be added to the debug list. After the nodes are selected, the Bitstream can be re-generated to include these debug signals.

The din_valid is used as the trigger to capture data from the Avalon Stream.

Full Demo

The Python Notebook goes through the various steps of the full demo. This includes:

• Writing and Reading Frames for the On-Chip Memory
• Setup for continous DMA transactions to and from On-Chip memory
• Configuring the 2D-Filter for Custom kernels.

Direct Memory Access to Processor Driver

While this design could use a processor either as a Soft Processor NIOS-V or SOC ARM processor with driver software, this example wanted to provide the low-level details of how each core works. This example has used direct register addresses and masks to configure the system and perform the various operations.