Hands-On Lab: How to Perform Automated Defect Detection Using Anomalib

This article shows how to use Anomalib with your own datasets, and walks through an exciting industrial use case featuring the Dobot robot.

Let’s Get Started

To understand how Anomalib works, we will navigate through a production line that inspects color cubes (Figure 1). Some of these cubes will have holes or defects that will need to be taken off the conveyor belt. Since these defects are not common in the production line, we will take a few images for our AI model.

Installation:

To install Anomalib using the source file, follow these steps:

1. Create an environment to run Anomalib + Dobot DLL using Python version 3.8

a. For Windows, use the following:

python -m venv anomalib_env
anomalib_env\Scripts\activate

b. For Ubuntu:

python3 -m venv anomalib_env
source anomalib_env/bin/activate

2. Install Anomalib using pip install:

m pip install anomalib[full]

3. Install Jupyter Lab or Jupyter Notebook through: https://jupyter.org/install

pip install notebook
pip install ipywidgets

4. Then connect your USB camera and verify it works using a simple camera application. Once it is verified, close the application.

Optional: If you have access to the Dobot, follow these steps:

1. Install Dobot requirements (refer to the Dobot documentation for more information).

2. Check all connections to the Dobot and verify it is working using Dobot Studio.

3. Install the vent accessory on the Dobot and verify it is working using Dobot Studio.

4. In the Dobot Studio (Figure 2), hit the "Home" button, and locate the:

a. Calibration coordinates: Initial position upper-left corner of cubes array.

b. Place coordinates: Position where the arm should leave the cubic over the conveyor belt.

c. Anomaly coordinates: Where you want to release the abnormal cube.

d. Find the instruction to calculate the coordinates in the Dobot documentation.

5. For running the notebooks using the robot, download the Dobot API and drivers files from here and add those to the notebooks/500_uses_cases/dobot in the Anomalib folder of this repo.

Figure 2: Dobot Studio interface.

Note: If you don’t have the robot, you can jump to another notebook like the 501a notebook, download the dataset from this link, and try the training and inferencing there.

Data Acquisition and Inferencing of the Notebook

Next, we need to create a folder with a normal dataset. For this example, we created a dataset of color cubes and added the abnormality with a black circle sticker, which will simulate a hole or defect on the box (Figure 3). For data acquisition and inferencing, we will use the 501b notebook.

Figure 3: Dataset used for the first round of training.

For acquiring data, be sure you set up this notebook with the acquisition variable True and define normal FOLDER for data without abnormalities and abnormal FOLDER for anomalous images. The dataset will be created directly in the Anomalib cloned folder, so we will see Anomalib/dataset/cubes folder there.

In case you don’t have the robot, you can modify the code to save images or use the downloaded dataset for training.

Inferencing:

For inferencing, the acquisition variable should be False, so in that case we won’t save any images. We will read the frame, run the inference using OpenVINO, and make decisions about where we will locate the cube: on the conveyor belt for normal pieces and out of the conveyor belt for abnormal cubes.

We need to identify the acquisition flag — True for acquisition mode and False for inferencing mode. In acquisition mode, be aware of the normal or abnormal FOLDER you want to create. For instance, in acquisition mode, the notebook will save every image in the anomalib/datasets/cubes/{FOLDER} for further training. In inferencing mode, the notebook won't save images; it will run the inference and show the results.

Training:

For training, we will use the 501a notebook. In this notebook, we will use PyTorch Lighting, and we will use "Padim" model for training. This model has several advantages: We don’t need a GPU, so with just the CPU we can perform the training process, and the training speed is high.

Now let’s deep dive into the training notebook! 🤿

Imports

In this section, we will explain the packages we are using for this example. We will also be calling the packages we need to use from the Anomalib library.

Configuration:

There are two ways to configure Anomalib modules—one using the config file, and another one using the API. The simplest way to see the functionality of the library is through the API. If you want to implement Anomalib in your production system, use the configuration file (YAML file) which is the core training and testing process where you will find the dataset, model, experiment, and callback management (Figure 4). You can find that file in the repository, in this path “anomalib/src/Anomalib/models/Padim/config.yaml," if you are using a different model find the proper name in the list to select the proper config file. For this specific example, I can also use this config file.

In the next sections, we will describe how to configure your training using the API.

Figure 4: The modules for training and validation.

Dataset Manager:

Via the API, we can modify the dataset module. We will prepare the dataset path, format, image size, batch size and task type. And then we load the data into the pipeline using:

i, data = next(enumerate(datamodule.val_dataloader()))

i, data = next(enumerate(datamodule.val_dataloader()))

Model Manager:

For our anomaly detection model, we are using Padim, but there are other Anomalib models you can also use, such as: CFlow, CS-Flow, DFKDE,DFM,DRAEM, FastFlow, Ganomaly Patchcore, Reverse Distillation, and STFPM. Also, we have set up the model manager using the API; Padim is imported using anomalib.models.

Callbacks Manager:

To train the model properly, we will have to add some other "nonessential" logic such as saving the weights, early stopping, normalizing the anomaly scores, and visualizing the input/output images. To achieve these, we use callbacks. Anomalib has its own callbacks and supports PyTorch Lightning's native callbacks. With this code, we will create the list of callbacks we want to execute during the training.

Training:

Now that we set up the datamodule, the model, and the callbacks, we can train the model. The final component to training the model is pytorch_lightning Trainer object, which handles the train, test, and predict pipeline. An example of trainer object in the notebook can be found here.

Validation:

For validation, we are using the OpenVINO inference. Previously, in the imports section, we imported OpenVINOInferencer from the anomalib.deploy module. Now we will use it to run the inference and check the results. First, we need to check that the OpenVINO model is in the result folder.

Prediction Result:

For performing the inference, we need to call predict method from OpenVINOinference where we can set up where the OpenVINO model and metadata of the model and also determine which device we want to use:

predictions = inferencer.predict(image=image)

predictions = inferencer.predict(image=image)

Predictions contain a variety of information related to the result: original image, prediction score, anomaly map, heat map image, prediction mask, and segmentation result (Figure 5). More information will be useful, depending on the task type you want to select.

Figure 5: Prediction result

At the end, our defect detection use case featuring the Dobot robot looks something like this (Figure 6):

Figure 6: Education robot running the inference of the Anomalib model.

Tricks and Tips for Using Your Own Dataset

Dataset Transformation:

If you want to improve the accuracy of your model, you could apply data transformations to your training pipeline. You should provide the path of the augmentations config file in the dataset.transform_config section of the config.yaml. That means you need to have one config.yaml file for the Anomalib setup and a separate albumentations_config.yaml file that will be addressed by the Anomalib config yaml file.

In this discussion thread, you can learn how to add data transformations to your actual training pipeline.

Robust Models:

Anomaly detection libraries aren’t magic and can fail when used on challenging datasets. But we have good news: You can try 13 different models and benchmark the results of each experiment. You can use the benchmarking entry point script for this, and the config file for benchmarking purposes. This will help you select the best model for your actual use case.

For more guides, please check these “How to Guides.”

What's Next

Check out our edge AI reference kits, where you can jump-start your AI solutions with how-to videos, step-by-step tutorials, and more. For defect detection using Anomalib, we provide a solutions overview, a technical how-to video, and a step-by-step tutorial to help you get started. And when you’re ready, you can clone the GitHub repository as a stand-alone project and begin building your very own defect detection solution.

If you are using the Dobot and want to see more on working with it through this notebook, please add your comments or questions to this post. If you come across any issues or errors with the Anomalib installation process, please submit your issue in our GitHub repository.

We look forward to seeing what else you come up with using the Anomalib library.

Have fun and share your results in our discussion channel! 😊

Notices and Disclaimers

Intel technologies may require enabled hardware, software, or service activation.

No product or component can be absolutely secure.

Your costs and results may vary.

Intel does not control or audit third-party data. You should consult other sources to evaluate accuracy. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

About the Authors

Paula Ramos, Intel AI Evangelist, America
Zhuo Wu, Intel AI Evangelist, China
Samet Akcay, Intel AI Research Engineer/Scientist

artificial intelligence

machine learning

robotics