Rockchip (EASY EAI) RV1126B Flame Detection

Flame detection

1. Introduction to Flame Detection

Flame detection is a type of object detection based on deep learning that detects and locates flames. Trained on large-scale flame data recognition, it works with cameras to real-time identify open flames in the monitored area, immediately trigger an alarm, prompt the monitoring room to check, and stop losses in a timely manner.

The performance of this flame detection algorithm on the dataset is shown below:

2. Quick Start

2.1 Development Environment Preparation

If you are reading this document for the first time, please refer to《Getting Started Guide/Development Environment Preparation/Easy-Eai Compilation Environment Preparation and Update》，and deploy the compilation environment according to the relevant operations.

Execute the run script in the PC-side Ubuntu system to enter the EASY-EAI compilation environment, as follows:

cd ~/develop_environment
./run.sh

2.2 Source Code Download

Create a management directory for storing the source code repository in the EASY-EAI compilation environment:

cd /opt
mkdir EASY-EAI-Toolkit
cd EASY-EAI-Toolkit

Clone the remote repository in the management directory using the Git tool:

git clone https://github.com/EASY-EAI/EASY-EAI-Toolkit-1126B.git

Notes:

• The process may get stuck due to network issues; please wait patiently.
• If you have to download from the GitHub webpage, download the entire repository instead of only the directory corresponding to this example.

2.3 Model Deployment

To run the algorithm Demo, you need to download the flame detection algorithm model first.

Baidu Netdisk link: https://pan.baidu.com/s/11hAoFKprLXvH98M_iQh6vQ?pwd=1234 (Extraction Code: 1234).

Meanwhile, copy and paste the downloaded flame detection algorithm model to the Release/ directory:

2.4 Routine Compilation

Enter the corresponding routine directory and execute the compilation command as follows:

cd EASY-EAI-Toolkit-1126B/Demos/algorithm-fire/
./build.sh cpres

Notes:

• Since the dependent libraries are deployed on the board, the /mnt mount must be maintained during the cross-compilation process.
• If the build.sh script is run with the cpres parameter, all resources in the Release/ directory will be copied to the development board.

2.5 Routine Execution and Effects

Enter the board background via serial port debugging or SSH debugging，and navigate to the routine deployment location:

cd /userdata/Demo/algorithm-fire/

Run the routine with the following command:

sudo ./test-fire_detect fire_detect.model test.jpg

Retrieve the test image in the EASY-EAI compilation environment:

cp /mnt/userdata/Demo/algorithm-fire/result.jpg .

The result image is shown below:

For detailed API descriptions and API calls (source code of this routine), see the instructions below.

3. Flame Detection API Description

3.1 Reference Method

To facilitate customers to directly call our EASY EAI API library in local projects, the libraries, header files, etc. that need to be linked in the project are listed here for easy addition by users.

3.2 Flame Detection Initialization Function

The prototype of the flame detection initialization function is as follows:

int fire_detect_init(rknn_context *ctx, const char * path)

Detailed description:

3.3 Flame Detection Running Function

The prototype of the flame detection running function fire_detect_run is as follows:

int fire_detect_run(rknn_context ctx, cv::Mat input_image, person_detect_result_group_t *detect_result_group)

Detailed description:

3.4 Flame Detection Release Function

The prototype of the flame detection release function is as follows:

int fire_detect_release(rknn_context ctx)

Detailed description:

4. Flame Detection Algorithm Routine

The routine directory is Demos/algorithm-fire/test-fire_detect.cpp, and the operation process is as follows.

The reference routine is shown below:

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include <sys/time.h>
#include"fire_detect.h"

using namespace cv;
using namespace std;

static Scalar colorArray[10]={
    Scalar(255, 0, 0, 255),
    Scalar(0, 255, 0, 255),
    Scalar(0,0,139,255),
    Scalar(0,100,0,255),
    Scalar(139,139,0,255),
    Scalar(209,206,0,255),
    Scalar(0,127,255,255),
    Scalar(139,61,72,255),
    Scalar(0,255,0,255),
    Scalar(255,0,0,255),
};

int plot_one_box(Mat src, int x1, int x2, int y1, int y2, char *label, char colour)
{
    int tl = round(0.002 * (src.rows + src.cols) / 2) + 1;
    rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), colorArray[(unsigned char)colour], 3);

    int tf = max(tl -1, 1);

    int base_line = 0;
    cv::Size t_size = getTextSize(label, FONT_HERSHEY_SIMPLEX, (float)tl/3, tf, &base_line);
    int x3 = x1 + t_size.width;
    int y3 = y1 - t_size.height - 3;

    rectangle(src, cv::Point(x1, y1), cv::Point(x3, y3), colorArray[(unsigned char)colour], -1);
    putText(src, label, cv::Point(x1, y1 - 2), FONT_HERSHEY_SIMPLEX, (float)tl/3, cv::Scalar(255, 255, 255, 255), tf, 8);
    return 0;
}

int main(int argc, char **argv)
{
	if (argc != 3)
    {
        printf("%s <model_path> <image_path>\n", argv[0]);
        return -1;
    }

    const char *model_path = argv[1];
    const char *image_path = argv[2];

	/* 参数初始化 */
	detect_result_group_t detect_result_group;

	/* 算法模型初始化 */
	rknn_context ctx;
	fire_detect_init(&ctx, model_path);

	/* 算法运行 */
	cv::Mat src;
	src = cv::imread(image_path, 1);

	struct timeval start;
	struct timeval end;
	float time_use=0;

	gettimeofday(&start,NULL); 

	fire_detect_run(ctx, src, &detect_result_group);

	gettimeofday(&end,NULL);
	time_use=(end.tv_sec-start.tv_sec)*1000000+(end.tv_usec-start.tv_usec);//微秒
	printf("time_use is %f\n",time_use/1000);

	/* 算法结果在图像中画出并保存 */
	// Draw Objects
	char text[256];
	for (int i = 0; i < detect_result_group.count; i++) 
	{

		detect_result_t* det_result = &(detect_result_group.results[i]);
		if( det_result->prop < 0.4)
		{
			continue;
		}

		sprintf(text, "%s %.1f%%", det_result->name, det_result->prop * 100);
		printf("%s @ (%d %d %d %d) %f\n", det_result->name, det_result->box.left, det_result->box.top,
			   det_result->box.right, det_result->box.bottom, det_result->prop);
		int x1 = det_result->box.left;
		int y1 = det_result->box.top;
		int x2 = det_result->box.right;
		int y2 = det_result->box.bottom;
		/*
		rectangle(src, cv::Point(x1, y1), cv::Point(x2, y2), cv::Scalar(255, 0, 0, 255), 3);
		putText(src, text, cv::Point(x1, y1 + 12), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
		*/
		plot_one_box(src, x1, x2, y1, y2, text, i%10);
	} 

	cv::imwrite("result.jpg", src);	


	/* 算法模型空间释放 */
	fire_detect_release(ctx);

	return 0;
}