Computer Vision: Tinkerboard+RPI Streaming

Working on different projects using the Raspberry Pi has been beneficial, ranging from DNS servers, PiHole security monitoring, and media hosting. However, it was not without its challenges. One of the limitations of the Pi is its computing power, as I discovered when installing opencv. This required me to restrict parallel jobs to 1 in order to compile successfully.

make install -J 1
ninja install -J 1

As a result, I explored other options such as the Azure-approved Tinker Board Edge R. A comparison of the specifications between the two devices is presented below. The Tinker Board Edge R boasts a more powerful processor, higher RAM capacity, superior multimedia capabilities, and a distinct power supply. On the other hand, the Raspberry Pi 3B+ is more budget-friendly and enjoys wider usage, thus featuring greater community support and a more extensive selection of compatible accessories and software.

Rather than choosing one over the other, I had the idea of combining both the Raspberry Pi and Tinker Board by using standoff pins to physically connect the two PCB boards together. The reason for not simply using the Tinker Board alone was due to the unavailability of a reliable 22 pin to 15 pin connector. Additionally, since there is minimal latency when one hop away, the Raspberry Pi can be used solely as a webcam without any issues. However, this opens up other possibilities, such as using it for photo/video archiving or creating a database for various projects, classes, and labels. In this setup, the Tinker Board can be utilized exclusively for training and validation purposes.

In this configuration, the Raspberry Pi is powered by the Tinker Board via jumper wires, since they share the same GPIO pin layout its mapping is 1 to 1 (Black{ground}, Red{5V}). An Ethernet cable is utilized to facilitate data communication between the two boards. The Tinker Board is connected to the internet via its wlan interface, which is linked to the Pi’s Ethernet interface. This connectivity is established and routed using dnsmasq and Iptables.

#enable ip forwarding on Tinkerboard Edge R
sudo sysctl -w net.ipv4.ip_forward=1

#install dns iptables
sudo apt-get install dnsmasq iptables

#disable systemc-resolved
sudo systemctl disable systemc-resolved

#iptables route definitions 
sudo iptables -t nat -A POSTROUTING -o wlp1s0 -J MASQUERADE
sudo iptables -A FORWARD -i wlp1s0 -o eth0 -m state RELATED, ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i eth0 -o wlp1s0 -j ACCEPT

SSH was used to access pi host steps as follows:

sudo nano etc/ssh/ssh_config

#ensure the following are uncommented 
hosts *
ForwardAgent yes
ForwardX11 yes
ForwardX11Trusted yes
Port 22

#Tinkerboard client
ssh -x <user>@<host>

The Raspberry Pi is equipped with the newly released Wide Angle V3 Camera Module, which depends on libcamera to interface with the camera module. It was installed following steps from the Official Webpage.

https://libcamera.org/

https://github.com/mesonbuild/meson.git
cd meson
sudo python3 -m setup.py install

https://github.com/ninja-build/ninja.git
cd ninja
./configure.py --bootstrap #binaries are now in main directory and can be moved 
sudo mv ninja /usr/local/bin

git clone https://git.libcamera.org/libcamera/libcamera.git
cd libcamera
meson setup build
ninja -C build install

The Camera can be detected using v4l2 or dmesg |grep i2c or imx

v4l2-ctl --list-devices

dmesg |grep i2c
dmesg |grep imx

#note that if device is not detected try the following steps.
1. Verify Ribbon cable is properly seated
2. sudo raspi-config -> interfaces -> enable i2c -> reboot
3. dmesg |grep i2c #should see message about bus device being setup

Camera test

libcamera-still or libcamera-still --qt-preview
libcamera-hello

Camera stream test

#Pi Host
libcamera-vid -t 0 --width 1920 --height 1080 --inline --listen -o tcp://0.0.0.0:8000

#Tinker Host
vlc tcp://<user>@<Host>:8000

It is theoretically possible to also use OpenCV to capture livestream via rtsp/tcp/udp as shown in the following c++ script.

Note: This has not been verified on my end. Update 03/15 This has been confirmed working and reflected the code to what was trialed same as before but used HTTP as opposed to RTSP.

Capture RTSP Stream from IP Camera using OpenCV | Lindevs

#include <iostream>
#include <opencv2/opencv.hpp>

using namespace cv;

int main()
{
    const std::string HTTP_URL = "http://10.0.0.23:4343/video";

    Mat frame;
    VideoCapture cap(HTTP_URL, CAP_FFMPEG);

    if (!cap.isOpened()) {
        std::cout << "Cannot open HTTP stream" << std::endl;

        return -1;
    }

    while (true) {
        cap >> frame;
        imshow("HTTP stream", frame);

        if (waitKey(1) == 27) {
            break;
        }
    }

    cap.release();
    destroyAllWindows();

    return 0;
}

OpenCV can be installed on Tinkerboard following steps on official page.

https://docs.opencv.org/4.7.0/d7/d9f/tutorial_linux_install.html

# Install minimal prerequisites (Ubuntu 18.04 as reference)
sudo apt update && sudo apt install -y cmake g++ wget unzip
# Download and unpack sources
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.x.zip
unzip opencv.zip
unzip opencv_contrib.zip
# Create build directory and switch into it
mkdir -p build && cd build
# Configure
cmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x
# Build
cmake --build .
# Install
sudo make install

References

Capture RTSP Stream from IP Camera using OpenCV | Lindevs