Smart Automated Shopping with AI, IoT and Embedded Cameras

Sponsored by Basler AG
16 days ago

Basler created a live demonstration of an automated retail check-out terminal equipped with AI software. The embedded vision system quickly identifies and classifies products in a shopping basket, and displays the pricing. This demo shows how combining AI and lean embedded vision systems can lead to innovative embedded applications that simplify our everyday life.

1. Overview

Embedded vision technology becomes smaller, more powerful and thus enables a wealth of new applications that rely on small size, low power consumption and low costs. One of the many areas where embedded technology can help to automate and simplify processes is retail.

Among the systems that will help to make the shopping experience easier are self-checkout systems. Today those systems usually use 2D barcode scanners to detect and record products on conveyor belts.

More recent systems use traditional object classification methods (using features like color or type) to ensure precise identification of product features. These methods are not robust when deployed in uncontrolled environments where different light and geometry conditions occur.

Those problems can be avoided when adding AI to the system: With latest AI technology it is possible to detect products without barcodes in a seamless environment and scale up the product portfolio easily on the fly.

The challenge when building such a system is to combine the best of two worlds: AI and lean embedded vision systems. Basler designed this proof of concept to show the benefits and possibilities of this new technology.

2. Solution

Basler created – with the support of their close partnership with NXP Semiconductors – a live demonstration of an automated retail checkout terminal equipped with AI software. The embedded vision system quickly identifies and classifies products in a shopping basket, and displays the pricing. Customers can select what to put in their baskets, and the trained neural network then detects the products based on a video stream – similarly to how face recognition works – and finally the total pricing is displayed.

2.1 Hardware

The AI retail solution is based on the new Basler Add-on Camera Kit and the NXP evaluation kit for the i.MX 8MQuad. This state-of-the-art edge processing system consists of:

The Basler Add-on Camera Kit (daA4200-30mci-IMX8-EVK), consisting of:

...and the NXP i.MX 8MQuad Evaluation Kit (MCIMX8M-EVK)

2.2 Software

The software for this solution consists of two major parts: the system software and the application software.

To make such an embedded system run smoothly, different elements of the system software must be connected to create a coherent system. Basler used its long experience in different software fields to connect software elements like BSPs, NXP’s Yocto kernel and to create a high-performing and lean embedded system.

The second part of the solution’s software is the application software. The software is based on a custom Convolutional Neural Network (CNN) by Irida labs. The model used is based on the latest deep learning and edge processing techniques, to provide fast and robust response in a seamless scenario. The training of the CNN takes place on the host side but the inference takes place on the edge.

2.3 Advantages of this solution

This solution offers a robust system with a lot of advantages.

  • This system is scalable. Products can be added easily to the system: a host-sided training allows for an easy deployment to the edge systems using IoT technology.
  • This robust system is optimized for vision, has high accuracy and a fast inference time.
  • Hardware is industry-proven and long-lasting.
  • Cost-efficient, lean design due to embedded and IoT technology.
  • Small form factor allows for different integration options.
  • Retail stores benefit from lower labor costs as well as significantly improved customer experience through instant checkouts, minimized queues and 100% checkout capacity at all times, even when the shop is open 24/ 7.

3. Conclusion

When creating this solution, one of Basler‘s crucial tasks was to bring all relevant partners together and to combine the different elements to create a perfect solution. With Embedded Vision Solutions by Basler, the customer has a single contact that takes care of all relevant coordination. In addition, Basler takes care of optimizing each system for vision requirements and making different components work together seamlessly.

With the high flexibility that is necessary to create such a system, Basler is the right partner to create complex embedded vision solutions.

Related articles
Related articles