Dynamic Traffic Light system on Kria KV260

INTRODUCTION

Traffic Jams are a major problem towards the sustainable goals of Smart Cities. We all have every once a while waited patiently across a junction for the traffic lights to turn green or being frustrated with the slow crawl of traffic in peak hours when we want to reach the office on time or go home from a hard day's work. We all have suffered a lot in traffic jams. Traffic Jams cost several Billion $ annually, several Billion hours of wasted time as well as Billions of Liters of wasted fuel leading to tremendous Economic Loss, wastage of time which could have been spent on other productive activities as well as loss to the environment. The aim of the project is to solve this problem using Dynamic Traffic Light System.

our Kria KV260 Basic Accessory Pack includes :

• Barrel Jack Power supply (12V, 3A)
• MicroSD card [16GB UHS-1]
• Micro-USB to USB-A cable
• AR1335 IAS camera module
• Ethernet cable
• HDMI cable to connect to a monitor

Building Petalinux

Follow the guide for building a petalinux project here

1) Download PetaLinux Tools Installer 2021.1 PetaLinux Download page

2) 2021.1 tools eSDK update 1 here

3) Download the Kria K26 Starter Kit 2021.1 Update 1 BSP from the PetaLinux Download link

Now this wic image is 6 GB by size.

Flashing the SD card

Now flash the image into SD card using Balena Etcher. SD card can then be inserted into the kria Board

Setup the KV260

For running the petalinux build on KV260 board, camera, Display monitor and serial USB port is attached to the device. Follow the guide for getting started with the setup with board connections here

First to communicate through the serial port we will be using TeraTerm

1) Find your correct port for KV260 (COM6 in my case)

2) Change Baud rate to 115200 under setup>serial port and reboot the board

3) Login with default Petalinux. For password or root user follow user guide here

Xmutil

Xmutil package group - xmutil handles loading of the various accelerated applications. The utility can be used to query the platform status, and manage the accelerated apps, as well as several other features.

Note that target output could be dp/rtsp/file. Follow for different functions supported here. We would be using file input here.

Getting video file : https://pixabay.com/videos/new-york-traffic-nyc-street-city-33003/

The smartcam file input only supports.h264/.h265 so, we would have to convert.mp4 file into.h264 for inference. We would be using ffmpeg for this process. Since I already have ubuntu installed in my Virtual box, I would be using that for conversion of stock mp4 footage. For transferring files between Ubuntu and Kria I am using WinSCP.

For installing ffmpeg in Ubuntu use:

$ sudo apt update
$ sudo apt install ffmpeg

validate ffmpeg installation

$ ffmpeg -version

use following command for file conversion

The encoded.h264 file can now be transferred from windows to kria using WinSCP.

1) find your board ip using

ifconfig

2) This ip is used to connect to the kria for easy file transfer

3) root file is the output file from the board

4) Following command is used to run inference in the board

VITIS AI

The Vitis AI development environment is a specialized development environment for accelerating AI inference. Vitis AI development environment supports deep learning frameworks like Tensorflow and Caffe, and APIs to prune, quantize, optimize, and compile pre-trained networks for AI inference.

Follow the following link for installing Vitis AI v2.0 here

The downloaded dpu image image would be named xilinx-kv260-dpu-v2021.2-v2.0.0.img and would be around 2.5 GB.

Python

The board can run python using jupyter labs. For running the python files in Jupyter :

The basic structure in Dynamic traffic light can be thought of -

1. Get real time image of each lane.

2. Scan and determine traffic density.

3. Input this data to the Time Allocation Algorithm.

4. The output will be the time slots for each lane would be given according to the algorithm.

SUMMARY

In conclusion I am impressed with the AI capabilities of Kria KV260 as well as ease of working with the FPGA board compared to other FPGA boards. The board requires little to no FPGA experience as well is able to run powerful AI Algorithms with the help of VITIS ai Libraries. Lot more is needed to be done to explore all these libraries and packages and further work is required from my side to complete this project in its full essence which would be coming in the future project submissions. From the time working with the board I am completely assured that the board would be extremely fruitful for the organizations currently working in the AI fields to quickly improve their AI hardware performance by easily implementing newer AI algorithms as soon as older ones gets outdated (such as implementing YOLO 3 instead of YOLO etc.). This would be much better solution compared to ASIC as well as GPU (due to large power consumption). Furthermore it was observed that model performance is better at complex scenes where a lot of objects and classes are detected at the same time compared to GPU's which would make it suitable for High Density Vision Applications.

OUTPUT