AI Cleanroom Security Guard

Background

In many cases we need to authenticate people in real life to ensure they are satisfying some kind of safety requirements. For example, a cleanroom requires people to wear face masks when operating. Automatically identifying such risks is useful and practical in smart factories, of which this project is capable.

Setup: Playground and Preparation

We will implement the project on Kria KR260 Robotics Starter Kit, which is a DPU-ready SoM solutions designed for robotic applications.

The official Ubuntu image does not carry Xilinx DPU driver, but it might be treated as a playground for quick pick-up of the board.

Playground: Deploy Ubuntu on TF Card

Go to https://ubuntu.com/download/amd and select Kria™K26 SOMs at the left tabs, download the latest Ubuntu image (as of this writing) adapted to **Kria KR260 Robotics Starter Kit**.

The official guide recommends to use Balena Etcher, but I failed to flash it. Besides, the Balena Etcher is bloated with regard to the application size. I finally managed to flash my TF card using xzcat and dd:

# xzcat iot-limerick-kria-classic-desktop-2204-20240304-165.img.xz | dd of=/dev/sdb bs=1M

In order to make full use of the TF card, resize the last partition when necessary:

# resize2fs /dev/sdb2

Playground: Camera Test

The image carries common camera drivers, so most cameras will plug and play.

You can use ffmpeg or vlc to test the availability of the local camera.

Setup: Host Development Kit

To launch Vitis™ Core Development Kit on the host, installers need to be fetched from Vitis (SW Developer) @ Downloads. If the progress of your self-extracting Web installer is horribly slow, like me, you can instead get the SFD (Single File Download) version, which behaves as an offline installer.

Setup: PetaLinux (in Docker)

Visit PetaLinux @ Downloads to get PetaLinux Installer & Kria KR260 Starter Kit BSP. Here I deployed PetaLinux in a Ubuntu docker.

$ docker create --name ubuntu -h ubuntu --tty -v $HOME/workspace:/workspace ubuntu:jammy
$ docker start ubuntu
$ docker exec -it ubuntu bash
root@ubuntu:/# apt install build-essential zlib1g-dev libtool-bin libncurses-dev bc python3 python-is-python3 xterm less rsync texinfo gcc-multilib lsb-release locales vim
root@ubuntu:/# sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

Drop the installer to the PetaLinux container via either copying into the shared volume or docker cp , and run the PetaLinux Installer.

Setup: DPU-enabled System Image

Get the DPU IP Reference Design (3.0) and patch the project builder Tcl script to generate a Vivado project.

Later when building PetaLinux, select necessary modules in kernel menuconfig, which may include DPU drivers and camera drivers.

Model

Face Mask Detection

We are using pt_face-mask-detection_512_512_0.67G_3.0 from model zoo. We first try it on local PC, to avoid configuring dependencies, we run an official container image xilinx/vitis-ai-pytorch-cpu, more variants can be checked here.

To run the model on the board, we first need to load the firmware to make DPU visible.

This model accepts a 512x512 (pixel = BGR) image as its input, so it's necessary to resize a user selected image of arbitrary size.

Testing with static images

A daemon will be deployed to capture from /dev/video0 continuously and run the model, send feedback to a backend.

Apps

Owners can check the alerts on phones, it will prompt the alert time and the corresponding captured image at that time.

Security Considerations

We make nftables rules to do network-level access control, so most of the probing attacks can be mitigated.

Future

We plan to integrate more biometrics-related authentication and alerts. Besides, offloading firewall policies to FPGA is still WIP.