Product inspection with Renesas RZ/V2L and Edge Impulse

In this project I implemented a computer vision system for cast product inspection. I trained a YOLOv5 object detection model using Edge Impulse platform and deployed it to the Renesas RZ/V2L Evaluation Board Kit. A Google Coral camera first takes a top view image of a submersible pump impeller moving on a conveyor belt. An application running on the Renesas RZ/V2L then predicts if the impeller is good or defective.

The time per inference is around 63 milliseconds which gives around 16 frames per second (fps). This low latency during inference can be related to the fact that the Renesas RZ/V2L board is designed for vision AI applications and it offers a powerful hardware acceleration through its Dynamically Reconfigurable Processor (DRP) and multiply-accumulate unit (AI-MAC).You can find the public Edge Impulse project here: Submersible pump impeller defect detection. To add this project to your Edge Impulse projects, click “Clone this project” at the top of the window. Please up on the project's documentation on Edge Impulse's page to help you get started on your own creation.

Dataset preparation

I used the casting product image data for quality inspection dataset on Kaggle. This dataset contains top view gray-scaled and augmented images of submersible pump impellers. The dataset has folders for defective and non-defective images of submersible pump impellers.

In total, I had 1300 images: 1047 for training and 253 for testing.

With 1300 images it would be tiresome to draw bounding boxes for all the objects. Edge Impulse offers various AI-assisted labelling methods to automate this process. In my case, I chose to track object within frames and the bounding boxes as well as their labels were set automatically.

Training and building the model

In Edge Impulse, an Impulse is a machine learning pipeline that indicates the type of input data, extracts features from the data and finally a neural network that trains on the features from your data.

For the YOLOv5 model, I used an image width and height of 160 pixels and the "Resize mode" set to "Squash". The processing block was set to "Image" and the learning block set to "Object Detection (Images)".

Under "Image" in Impulse design, the color depth of the images is set to RGB and the next step was to extract features.

On the Feature explorer, we can see that the blue and orange data don't completely separate from each other. This can be related to the fact that both defective and good impellers have similar features like their disc-shaped property. We can see that there is a trend of some sort on the data; the blue data are grouped together on one side and the orange data grouped together on another side. What separates the good and defective impellers is the bad "features" like cracks, holes and protrusions.

In the features tab we can also see the on-device performance for generating features during the deployment. These estimated metrics are for the Renesas RZ/V2L(with DRP-AI accelerator). The Renesas RZ/V2L Evaluation Board Kit is supported by Edge Impulse. This board is designed for vision AI applications and it offers a powerful hardware acceleration through its Dynamically Reconfigurable Processor (DRP) and multiply-accumulate unit (AI-MAC).

Currently, all Edge Impulse models can run on the RZ/V2L CPU which is a dedicated Cortex A55. However, so that I can benefit from the DRP-AI hardware acceleration, I chose a YOLOV5 model. Note that on the training page you have to select the RZ/V2L (with DRP-AI accelerator) before starting the training in order to tell the studio that you are training the model for the RZ/V2L. This can be done on the top right in the training page or by changing target device in the Dashboard page.

After training the model on various parameters, I settled with 100 training cycles and a learning rate of 0.001. It is however advised to train a YOLOv5 model using more than 1500 photos per class and more than 10, 000 instances per class to produce a robust YOLOv5 model.

After the training process, I got a precision score of 96%. Precision is the number of True Positives divided by the number of True Positives plus the number of False Positives.

Model testing

After training a model, we need to do a test with the unseen(test) data. In my case, the model had an accuracy of 91%. This accuracy is a percent of all samples with a precision score above 98%.

I chose this as an acceptable performance and proceeded to deploy the model to the Renesas RZ/V2L board.

Deploying the model to a Renesas RZ/V2L

The Renesas RZ/V2L Evaluation Kit comes with the Renesas RZ/V2L board and a 5-megapixel Google Coral Camera. To setup the board, Edge Impulse has prepared a guide that shows how to prepare the Linux Image, install Edge Impulse for Linux and finally connecting the board to Edge Impulse Studio.

After the Renesas RZ/V2L board has been setup we can SSH into the board by an ethernet connection between a computer and the board, or the board and a router. To SSH into the RZ/V2L we can run the following terminal/Command prompt command on a computer:

ssh root@smarc-rzv2l

To run the model locally on the RZ/V2L we can run the command edge-impulse-linux-runner --clean which lets us log in to our Edge Impulse account and select the cloned public project. The model will be downloaded and inference will start automatically.

We can go to the provided URL(RZ/V2L IP address at port 4912) and we will see the feed being captured by the camera as well as the bounding boxes if present.

Result - a demo of using AI in an industry quality inspection

I first deployed my model as an .eim model and using the Edge Impulse Python SDK, I developed a Web Application using Flask that counts the number of good and defective submersible pump impellers. The taken images and the counts are then displayed on the Webpage in real time.

Afterwards I designed and 3D printed some PLA based parts that can be assembled to a mini conveyor belt. These parts can be downloaded on printables.com or thingiverse.com. The mini conveyor mechanism assembly is made up of corner brackets, rollers, roller support, M3 bolts and even a motor housing. I assembled the conveyor on a board platform made of chip plywood. I also designed and 3D printed a crate that is used to catch the parts as they are falling from the conveyor.

The conveyor mechanism is actuated by one N20 3mm DC 6V 200RPM Metal Gear motor. The motor housing has a chamber to put a circuit board for driving the motor. A cable management channel is also provided for the wires running from the motor driver circuit to the motor.

The motor driver circuit is based on this tutorial. It has one DC input and a DC output. I used a 12V AC/DC adapter to power the conveyor mechanism. The speed of the motor can be controlled by adjusting the potentiometer.

Afterwards, I also designed and 3D printed a support for the Renesas RZ/V2L Evaluation Board Kit components. This support is made up of two parts: a base support for the board and a part for increasing the height of the base support. The Google Coral camera can be mounted upright or facing downwards through the provided slots. This Renesas RZ/V2L EVK support is also available for downloaded on printables.com, thingiverse.com and Renesas' community forum.

Finally, I printed some 3cm wide submersible pump impeller images from the testing dataset. As the conveyor belt is moving, the Google Coral camera takes a picture of the printed images and the results are shown on the WebApp.

Here is a demo of the application classifying submersible pump impellers as they are moving on the conveyor belt.

The source code for the WebApp is available for download from this GitHub repository. It can be cloned on the Renesas RZ/V2L board and the installation steps are in the repository. You can run the Flask application using Python(after installing Flask) or run the binary executable built with PyInstaller for AARCH64 platform.