Rockchip (EASY EAI) RV1126B BSD (Blind Spot Detection)

BSD (Blind Spot Detection)

1. Introduction to BSD

BSD (Blind Spot Detection) algorithms for vehicles usually require multiple cameras to monitor the blind spots on both sides and the rear of the vehicle in real time. They feature high frame rates and high-precision recognition to adapt to various complex environments.

The performance of this BSD algorithm on industry-specific datasets is shown as follows:

2. Quick Start

If you are reading this document for the first time, please refer to《Getting Started Guide/Development Environment Setup/Easy-EAI Compilation Environment Preparation and Update》,and deploy the compilation environment in accordance with the relevant operations specified therein.

Run the run script on a PC with the Ubuntu system to access the EASY-EAI compilation environment, as detailed below.

cd ~/develop_environment
./run.sh

2.1 Source code download

Create a management directory to store the source code repository under the EASY-EAI compilation environment:

cd /opt
mkdir EASY-EAI-Toolkit
cd EASY-EAI-Toolkit

Clone the remote repository within the managed directory using the git tool

git clone https://github.com/EASY-EAI/EASY-EAI-Toolkit-1126B.git

Note:

* There may be lags caused by network issues here, please wait patiently.

* If you have to download from the GitHub webpage, you need to download the entire repository instead of only the directory corresponding to this example.

2.2 Model deployment

To run the algorithm demo, you need to download the BSD detection algorithm model first.

Baidu Netdisk link: https://pan.baidu.com/s/1zbZc9uEns90vf1yGlh854Q?pwd=1234 (Extraction code: 1234).

同时需要把下载的BSD算法模型复制粘贴到Release/目录：

2.3 Program compilation

Go to the corresponding routine directory and perform compilation operations. The specific commands are shown as follows:

cd EASY-EAI-Toolkit-1126B/Demos/algorithm-bsd/
./build.sh cpres

Note:

* Since the dependent libraries are deployed on the board, the /mnt mount must be maintained during cross-compilation.

* If the build.sh script carries the cpres parameter, all resources under the Release/ directory will be copied to the development board.

2.4 Program operation and effect

Access the backend of the board via serial port debuggingor SSH debugging, and navigate to the location where the sample program is deployed, as shown below:

cd /userdata/Demo/algorithm-bsd/

The commands for running the routine are shown as follows:

sudo ./test-bsd bsd_person.model test.jpg

Test images can be retrieved in the EASY-EAI compilation environment:

cp /mnt/userdata/Demo/algorithm-bsd/result.jpg .

The resulting image is shown as follows:

Detailed description of the API and API invocation (source code for this example); see the explanation below for full details.

3. BSD Detection API Instructions

3.1 Citation format

To facilitate customers to directly call our EASY EAI API library in local projects, the libraries to be linked and header files required in the project are listed here for users to add directly.

3.2 BSD detection initialization function

The prototype of the BSD detection initialization function is shown as follows.

int bsd_init(rknn_context *ctx, const char * path)

The specific introduction is as follows.

3.3 BSD detection operation function

The prototype of the BSD detection running function bsd_run is shown as follows.

int bsd_run(rknn_context ctx, cv::Mat input_image, detect_result_group_t*detect_result_group)

The specific introduction is as follows.

3.4 BSD detection release function

The prototype of the BSD detection release function is shown as follows.

int bsd_release(rknn_context ctx)

The detailed introduction is as follows.

4. BSD detection algorithm routine

The routine directory is Demos/algorithm-bsd/test-bsd.cpp, and the operation process is as follows.

The reference routine is shown as follows.

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include <sys/time.h>
#include"bsd.h"

using namespace cv;
using namespace std;

static Scalar colorArray[10]={
    Scalar(255, 0, 0, 255),
    Scalar(0, 255, 0, 255),
    Scalar(0,0,139,255),
    Scalar(0,100,0,255),
    Scalar(139,139,0,255),
    Scalar(209,206,0,255),
    Scalar(0,127,255,255),
    Scalar(139,61,72,255),
    Scalar(0,255,0,255),
    Scalar(255,0,0,255),
};

int plot_one_box(Mat src, int x1, int x2, int y1, int y2, char *label, char colour)
{
    int tl = round(0.002 * (src.rows + src.cols) / 2) + 1;
    rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), colorArray[(unsigned char)colour], 3);

    int tf = max(tl -1, 1);

    int base_line = 0;
    cv::Size t_size = getTextSize(label, FONT_HERSHEY_SIMPLEX, (float)tl/3, tf, &base_line);
    int x3 = x1 + t_size.width;
    int y3 = y1 - t_size.height - 3;

    rectangle(src, cv::Point(x1, y1), cv::Point(x3, y3), colorArray[(unsigned char)colour], -1);
    putText(src, label, cv::Point(x1, y1 - 2), FONT_HERSHEY_SIMPLEX, (float)tl/3, cv::Scalar(255, 255, 255, 255), tf, 8);
    return 0;
}

int main(int argc, char **argv)
{
	if (argc != 3)
    {
        printf("%s <model_path> <image_path>\n", argv[0]);
        return -1;
    }

    const char *model_path = argv[1];
    const char *image_path = argv[2];

	/* 参数初始化 */
	detect_result_group_t detect_result_group;

	/* 算法模型初始化 */
	rknn_context ctx;
	bsd_init(&ctx, model_path);

	/* 算法运行 */
	cv::Mat src;
	src = cv::imread(image_path, 1);

	struct timeval start;
	struct timeval end;
	float time_use=0;

	gettimeofday(&start,NULL); 

	bsd_run(ctx, src, &detect_result_group);

	gettimeofday(&end,NULL);
	time_use=(end.tv_sec-start.tv_sec)*1000000+(end.tv_usec-start.tv_usec);//微秒
	printf("time_use is %f\n",time_use/1000);

	/* 算法结果在图像中画出并保存 */
	// Draw Objects
	char text[256];
	for (int i = 0; i < detect_result_group.count; i++) 
	{

		detect_result_t* det_result = &(detect_result_group.results[i]);
		if( det_result->prop < 0.4)
		{
			continue;
		}

		sprintf(text, "%s %.1f%%", det_result->name, det_result->prop * 100);
		printf("%s @ (%d %d %d %d) %f\n", det_result->name, det_result->box.left, det_result->box.top,
			   det_result->box.right, det_result->box.bottom, det_result->prop);
		int x1 = det_result->box.left;
		int y1 = det_result->box.top;
		int x2 = det_result->box.right;
		int y2 = det_result->box.bottom;
		/*
		rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(255, 0, 0, 255), 3);
		putText(src, text, cv::Point(x1, y1 + 12), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
		*/
		plot_one_box(src, x1, x2, y1, y2, text, i%10);
	} 

	cv::imwrite("result.jpg", src);	


	/* 算法模型空间释放 */
	bsd_release(ctx);

	return 0;
}