Airborne Object Detection on reComputer RK3576

Introduction

Airborne object detection is a critical capability for drone-based surveillance, traffic monitoring, and aerial analytics. However, deploying deep learning models on edge devices presents unique challenges — limited compute, thermal constraints, and the need for real-time performance.

This project implements a complete airborne object detection system on the Seeed reComputer RK3576 edge AI platform, using a VisDrone-fine-tuned YOLOv8s model accelerated by the RK3576's integrated NPU (6 TOPS). The system detects 11 classes of objects from drone-view footage — pedestrians, vehicles, bicycles, and more — at real-time speed using Rockchip's RKNN toolkit.

The project covers the full pipeline: PC-side development and testing, model conversion (PyTorch → ONNX → RKNN), NPU deployment, and Docker-based web service packaging for the Seeed ai-lab platform.

Hardware Components

VisDrone Dataset Classes

The model is fine-tuned on the VisDrone2019-VID dataset, which contains drone-captured videos with diverse scenes, object scales, and occlusions. The 11 detection classes are:

System Architecture

Step 1: PC-Side Development & Testing

Before deploying to the edge device, we develop and test the detection pipeline on PC using PyTorch with the original YOLOv8s model.

Install dependencies:

bash
pip install ultralytics opencv-python numpy

Run PC-side detection:

bash
# Basic detection with ByteTrack tracking
python airborne_detection.py --model ../model/best.pt --video ../video/test1.mp4

# Enhanced version with trajectory, heatmap, ROI, counter
python airborne_detection_v2.py

The PC-side code supports real-time detection with bounding boxes, ByteTrack multi-object tracking, motion trajectory visualization, density heatmap generation, ROI intrusion detection, virtual counting line, and JSON export.

Step 2: Model Conversion

1 Export ONNX

bash
pip install onnx onnxruntime onnxslim

python -c "from ultralytics import YOLO; YOLO('model/best.pt').export(format='onnx', imgsz=640, simplify=True)"

This produces best.onnx (42.7 MB) with output shape (1, 15, 8400) — 4 box coordinates + 11 class scores per anchor.

2 Verify ONNX Model

bash
python verify_onnx.py

This script loads the ONNX model with onnxruntime, runs inference on a video frame, and visualizes the results to confirm the model works correctly before RKNN conversion.

3 Convert to RKNN(WSL2 Linux)

bash
# Install rknn-toolkit2 (Python 3.10)
pip3 install rknn-toolkit2

# Critical: downgrade onnx to 1.14.1
# rknn-toolkit2 2.3.2 uses onnx.mapping which was removed in onnx 1.15+
pip3 install onnx==1.14.1

Important: RKNN Toolkit2 only runs on x86 Linux. Use WSL2 or a Linux machine.

python
from rknn.api import RKNN

rknn = RKNN(verbose=True)

# Configure for RK3576 NPU
rknn.config(
    mean_values=[[0, 0, 0]],        # Mean subtraction (per channel)
    std_values=[[255, 255, 255]],   # Std normalization (per channel)
    target_platform='rk3576'        # Target: RK3576
)

# Load ONNX model
rknn.load_onnx(model='best.onnx')

# Build without quantization (FP16)
rknn.build(do_quantization=False)

# Export RKNN model
rknn.export_rknn('best.rknn')
rknn.release()

Step 3: Deploy on RK3576

1Install Runtime Library

bash
# Copy librknnrt.so to system library path
sudo cp lib/librknnrt.so /usr/lib/
sudo ldconfig

2 Install Python Dependencies

bash
pip install opencv-python numpy
pip install rknn-toolkit-lite2-packages/rknn_toolkit_lite2-2.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

3Run Detection

bash
# Basic detection
python airborne_detection_rknn.py --model model/best.rknn --video video/test1.mp4

# Headless mode (SSH without display)
python airborne_detection_rknn.py --no-display

# Filter vehicle classes only (car, van, truck, bus)
python airborne_detection_rknn.py --classes 3,4,5,8

# Adjust confidence threshold
python airborne_detection_rknn.py --conf 0.15

Step 4: Docker Web Service

Build Docker Image

bash
docker build -t airborne-detection-rk3576

Run Container

bash
# Video analysis mode (web UI)
docker run -d -p 8000:8000 airborne-detection-rk3576

# With camera input
docker run -d -p 8000:8000 --device /dev/video1 airborne-detection-rk3576 \
  python web_detection.py --model_path model/best.rknn --camera_id 1

Access web UI at http://<RK3576_IP>:8000. Features include:

• Real-time Detection: Live MJPEG stream with bounding boxes
• Video Analysis: Upload video, watch real-time processing, download results
• REST API: Single image prediction endpoint at /api/models/yolov8/predict

Technical Details: RKNN Inference Pipeline

The rknn_detector.py implements the complete NPU inference pipeline:

• Preprocessing: Letterbox resize (preserve aspect ratio, pad with black) → BGR→RGB → uint8 NHWC format (1, 640, 640, 3). The RKNN model handles mean/std normalization internally.
• NPU Inference: RKNNLite.inference() runs on the NPU with dual-core auto mode (NPU_CORE_0_1).
• Post-processing: The RKNN model outputs a single tensor (1, 15, 8400) with already-decoded box coordinates (cx, cy, w, h) + 11 class scores. The post-processing extracts boxes and scores, converts to xyxy format, applies confidence filtering, per-class NMS, and letterbox coordinate restoration.

Key implementation notes:

• Input format: NHWC (1, H, W, 3), not NCHW
• Input dtype: uint8, not float32 (mean/std built into model)
• init_runtime(): Use core_mask= parameter, not core=
• librknnrt.so: Must be placed in /usr/lib/ for direct execution

Results

The system successfully detects and classifies airborne objects across diverse drone-view scenarios including highways, urban streets, squares, and residential areas. The VisDrone-fine-tuned YOLOv8s model provides significantly better accuracy on drone footage compared to COCO-pretrained models, particularly for small objects and aerial perspectives.

Source Code

Complete source code, pre-converted RKNN model, runtime library, Docker configuration, and test videos are available on GitHub:

github.com/inteintegrity/rk3576-airborne-detection