Anomalib fastflow universal detectors (Category 1)

Abstract

This model was made to test out the capabilities of anomalibs pre-trained models and to improve them to be able to recognize images of any class in the MVTech dataset. For that I used a fast flow model with resnet 18 as a backbone model and 30 flow steps. Although I had some initial problems with the memory and time required to train the model the model itself worked perfectly, getting an almost perfect image F1 Score. One particularly interesting finding has been that data augmentation in the process of training the model wasn’t as necessary as other models and that the model itself is incredibly robust almost without any data augmentation.

Introduction

· Background: The problem this AI solves is early detection of error in production. That is useful because early prevention can lower the amount of waste and money created by production errors. This AI also solver the versatility problem that many AI models have. It uses very little resources compared to other computer vision models, which is ideal, especially for putting it in production machines that either have very few or use most of their computational power for the production.

· Challenge Description: This model was made for the track 1 challenge. Although data augmentation was an important factor in the challenge it has almost not been necessary for making a robust model as all test with the model and most variations achieve a hight F1 score without data augmentation during training.

Methodology

Model Design

· Approach: To make this model I started of using the anomalibs fast flow model. I started out with a sample notebook in their git-hub and modified all hyperparameters and some of the code to fit the needs of the model. The main modification was to train on all image classes instead of only one. That helped increase robustness and make the model work with every category.

· Architecture: The architecture uses the Resnet 18 backbone with 30 step fast flow. That modification improves robustness and boosts the F1 score.

· Training: The model was trained in a colab environment and initially had some memory problems as it exceeded the 12GB of ram in colab. And the training of the model was very slow as google colab only enabled it to work the CPU, taking 45 minutes to only train 6 epochs. Fortunately, the training stopped after that as the loss values weren’t changing anymore.

Dataset & Evaluation

· Dataset Utilization: To make this model I used the MVTech anomaly dataset provided in the anomalib library. The model was trained using the 1 class paradigm to identify the objects and the model was set to classification mode for easier use.

· Evaluation Criteria: The performance of the model was evaluated using anomalibs integrated test mode. The results of the test were very good, as the model archived high scores in all three evaluation criteria.

Results

· Performance Metrics: The results of the model in the metrics for the first category of this contest where the following:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ Test metric ┃ DataLoader 0 ┃

┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩

│ image_AUROC │ 1.0 │

│ image_F1Score │ 0.9919999837875366 │

│ pixel_AUROC │ 0.9780702590942383 │

└───────────────────────────┴───────────────────────────┘

· Comparison: Comparing this to the last model I submitted to this contest this model performs much better getting a 0.99 in the F1 score compared to the 0.89 F1 score of the other model, meaning that it has improved a 10% in respect to the other model.

Discussion

· Challenges & Solutions: The main challenge with this model was the power limitations of google colab meaning that it took a lot of time to train and that it crashed several times due to the amount of memory it used.

· Model Robustness & Adaptability: The model is very good at detecting small variations in the images and their importance meaning that it can be very robust, even without being trained on data augmentation, testing has proved that the model is very robust.

· Future Work: Future work for this model will probably be centred around better testing with augmented data to make the model even more robust to work with any kind of product type and image quality.

Conclusion

To conclude. This model is very robust, even if data augmentation want used for training the model robustness is good and it can correctly identify the area where the defect is in an image, also with a good score.