STMicro Partners with Leopard Imaging on a Three-In-One Smart Vision Platform for NVIDIA's Jetson

STMicroelectronics has announced a partnership with Leopard Imaging to produce a "multimodal vision module" — delivering imaging, 3D scene mapping, and motion detection, with NVIDIA Holoscan Sensor Bridge support for connection to NVIDIA Jetson modules and the Isaac platform.

"Humanoid robotics is moving beyond research projects and demonstrations to deliver powerful new machines for a wide range of roles in manufacturing and automotive factories, logistics and warehousing, as well as retail and customer service," claims STMicro's Marco Angelici of the company's vision for the new vision module. "Our collaboration with Leopard Imaging brings market-leading ST sensors and actuators, seamlessly integrated into the NVIDIA robotics ecosystem, to accelerate the deployment of physical AI applications with human-like awareness."

"Access to ST sensors and actuators directly within the ecosystem has allowed us to standardize and streamline data acquisition and logging for humanoid robot vision across the HSB [Holoscan Sensor Bridge] interface," adds Leopard Imaging chief executive officer Bill Pu. "Robot builders can use our multi-sensing vision module with [NVIDIA] Isaac tools to accelerate learning and quickly bridge the 'sim-to-real' gap."

The compact module is designed around three key STMicro sensors in a single device: The ST VB1940 5.1-megapixel global-or-rolling shutter image sensor offers visible-light RGB and infrared night-vision imaging; the VL54L9CX direct time-of-flight (dToF) all-in-one lidar module delivers accurate 3D depth sensing at up to nine meters (around 30 feet) across almost 2,300 zones at a measurement rate of up to 100 frames per second; and the LSM6DSV16X six-axis inertial measurement unit (IMU) provides motion sensing, with an on-board in-house machine learning core (MLC) for sensor fusion and on-board model execution.

Unveiled at the GPU Technology Conference (GTC) this week, the sensor platform uses NVIDIA's Holoscan Sensor Bridge platform for connection to Jetson modules via Ethernet and to the Isaac robot development platform. Sample applications are included, the companies have said.

Additional information is available on the Leopard Imaging website; the company had not announced pricing and availability at the time of writing.