Perception on Xilinx Kria KV260 Vision AI Starter Kit

Introduction

Road accidents were the spark that ignited the early adoption of autonomous driving vehicles. But in recent times, the increasing demand from e-commerce sales has put a toll on the supply chain industry. The effect is predominantly evident on the last-mile delivery routes.

Outdoor Robot Use-case

As, last-mile routes are mostly low-speed scenarios such as university campuses, industrial parks, and areas with limited traffic - the idea is to build & deploy robot systems for these scenarios. To address common issues in these huge marked spaces, I propose to build a perception module for Outdoor Robotics. Other systems are robust and safe but not affordable for large-scale adoption. The more we enable robot systems to serve the human race, the easier it would be to cross the uncanny valley of automation and convince people to completely trust the technology for early adoption and gain traction in the marketplace. The only way forward is safe, robust, efficient, and affordable systems.

Proposed Solution

I plan to build an efficient perception pipeline using multiple sensor modalities to enable accurate pose estimation. The data from multiple GPS, IMU, and camera units will be fused together to estimate the ego-motion of the outdoor robot.

Kria system-on-module (SoM) from AMD-Xilinx has enough and more I/O with parallel data processing - would serve as the best candidate as a compute unit. (To know more about getting started with the board - click here )

In this demo, we use ORB SLAM 2 algorithm to map the environment while simultaneously localizing in it.

Due to the dynamic nature of the outdoor environment, where objects such as humans, vehicles, and other dynamic actors affect how efficiently the robot can localize in the generated map. So, a multi-modal approach is an interesting case that uses the IMU & GPS to correct the odometry of the robot using an EKF filter package from ROS - link here, which further increases the certainty in the robot pose against the camera alone.

Future Prospect

For a multi-view estimation, a more real-time mode is required and would be considered as KV260 provides FPGA core to be programmed using Vitis from Xilinx.