Rockchip (EASY EAI) RV1126B Face Detection Usage

Face detection

1. Overview of Face Detection

Face detection is an indispensable early step for tasks such as face recognition, facial attribute classification, face editing, and face tracking, and its performance directly impacts the effectiveness of face recognition and other related tasks. Despite significant advancements in unconstrained face detection over the past few decades, accurate and efficient face detection in the wild remains an open challenge. This is attributed to factors including pose variations, facial expressions, scale differences, lighting conditions, image distortion, and facial occlusions. Unlike general object detection, face detection is characterized by small variations in aspect ratio but much larger variations in scale (ranging from a few pixels to thousands of pixels).

The performance of this face detection algorithm on datasets is shown below:

Runtime efficiency on the EASY-EAI-Nano-TB hardware board:

2. Quick Start

2.1 Development Environment Preparation

If you are reading this document for the first time, please refer to 《Getting Started Guide / Development Environment Preparation / Easy-Eai Compilation Environment Preparation and Update》，and follow the relevant operations to deploy the compilation environment.

Execute the run script in the Ubuntu system on the PC to enter the EASY-EAI compilation environment, as shown below:

cd ~/develop_environment
./run.sh

2.2 Source Code Download

Create a management directory for storing the source code repository in the EASY-EAI compilation environment:

cd /opt
mkdir EASY-EAI-Toolkit
cd EASY-EAI-Toolkit

Clone the remote repository into the management directory using the git tool:

git clone https://github.com/EASY-EAI/EASY-EAI-Toolkit-1126B.git

Notes:

• Network issues may cause delays here; please wait patiently.
• If you must download from the GitHub webpage, download the entire repository instead of only the directory corresponding to this example.

2.3 Model Deployment

To run the algorithm demo, you first need to download the face detection algorithm model.

Baidu Netdisk link: https://pan.baidu.com/s/1nGQCTpD_Bk4byxqILZSWwA?pwd=1234 (Extraction code: 1234).

Copy the downloaded face detection algorithm model to the Release/ directory:

2.4 Example Compilation

Navigate to the corresponding example directory and execute the compilation command:

cd EASY-EAI-Toolkit-1126B/Demos/algorithm-face_detect/
./build.sh cpres

Notes:

• Since dependent libraries are deployed on the board, the /mnt mount must be maintained during cross-compilation.
• If the build.sh script is run with the cpres parameter, all resources in the Release/ directory will be copied to the development board.

2.5 Example Execution and Results

Access the board's background viaserial port debuggingor SSH，and navigate to the deployment location of the example:

cd /userdata/Demo/algorithm-face_detect/

Run the example with the following command:

sudo ./test-face-detect test.jpg

Retrieve the test result image in the EASY-EAI compilation environment:

cp /mnt/userdata/Demo/algorithm-face_detect/result.jpg .

Result image is shown below:

For detailed API descriptions and API calls (source code of this example), refer to the instructions below.

3. Face Detection API Description

3.1 Reference Method

To facilitate customers directly calling our EASY EAI API library in local projects, the libraries and header files to be linked in the project are listed below for easy integration.

3.2 Face Detection Initialization Function

The prototype of the face detection initialization function is as follows:

int face_detect_init(rknn_context *ctx, const char *path)

Detailed description:

3.3 Face Detection Execution Function

The prototype of the face detection execution function face_detect_run is as follows:

int face_detect_run(rknn_context ctx, cv::Mat &input_image, std::vector<det> &result)

Detailed description:

3.4 Face Detection Release Function

The prototype of the face detection release function is as follows:

int face_detect_release(rknn_context ctx)

Detailed description:

4. Face Detection Algorithm Example

The example directory is Demos/algorithm-face_detect/test-face-detect.cpp, with the operation process as follows.

Reference example code:

#include <opencv2/opencv.hpp>
#include <stdio.h>
#include <sys/time.h>
#include "face_detect.h"

using namespace cv;


int main(int argc, char **argv)
{
	if( argc != 2)
	{
		printf("./test-face-detect xxx\n");
		return -1;
	}

	struct timeval start;
	struct timeval end;
	float time_use=0;

	rknn_context ctx;
	std::vector<det> result;

	Mat image;
	image = cv::imread(argv[1], 1);	

	face_detect_init(&ctx, "face_detect.model");

	gettimeofday(&start,NULL); 
	face_detect_run(ctx, image, result);

	gettimeofday(&end,NULL);
	time_use=(end.tv_sec-start.tv_sec)*1000000+(end.tv_usec-start.tv_usec);//微秒
	printf("time_use is %f\n",time_use/1000);

	printf("face num:%d\n", (int)result.size());

	for (int i = 0; i < (int)result.size(); i++)
	{
		int x = (int)(result[i].box.x);
		int y = (int)(result[i].box.y);
		int w = (int)(result[i].box.width);
		int h = (int)(result[i].box.height);
		rectangle(image, Rect(x, y, w, h), Scalar(0, 255, 0), 2, 8, 0);

		
		for (int j = 0; j < (int)result[i].landmarks.size(); ++j) 
		{
			cv::circle(image, cv::Point((int)result[i].landmarks[j].x, (int)result[i].landmarks[j].y), 2, cv::Scalar(225, 0, 225), 2, 8);
		}
	}

	imwrite("result.jpg", image);

	face_detect_release(ctx);

	return 0;
}