In remote vehicle operation, video feedback provides the operator with real-time spatial and environmental information. Stereoscopic video feedback can improve upon standard single camera systems by providing the operator with details about scale, depth and motion in a rapid and intuitive form. This supplementary information is especially important when operating in unfamiliar or hazardous environments.
This project was for a university mechatronic engineering assignment.
The scope of the design was to test the effects on vehicle control when using a 3D visual telepresence system to navigate surroundings. An effective design would deliver heightened interactivity, control, navigation and spatial awareness to the driver, whilst minimizing the negative effects of motion sickness.
A 3D first person vision system has been developed, which captures stereoscopic perspectives with a pair of small cameras and displays them to a user’s corresponding eyes via a head mounted display. The head mounted display simultaneously captures positional and rotational values of the user’s head, using this data to manipulate a pan-tilt unit on which the cameras are mounted. The effect is delivery of a minimal latency 3D perspective of the remote controlled vehicles surroundings, where the field of view can be mechanically adjusted by rotating the head in the desired direction (see first attached video).
- Video Capture and Communication (blue)
- Headtracker Data Communication (green)
- Pan-Tilt unit (PTU) with Controller (yellow)
An Arduino Uno served as a remote (vehicle mounted) platform for manipulating the PTU, via two digital servos embedded within a 3D printed enclosure. At a minimum the design required at least two Pulse Width Modulated signal outputs, an RS232 UART/USART Serial Controller, and additional I/O pins for additional sensors/actuators (such as dynamic stability). The Arduino was selected over a conventional microcontroller on the basis of:
- High level language with pre-compiled libraries,
- On-board power regulator, bootloader, etc
- Ease of interface to external devices, such as XBee wireless serial devices
- Processing speed and memory control not critical to minimize latency, when combined with already substantial video latency.
Various different hypothesis related to motion sickness, control, cognitive memory retention and depth perception were all tested from the prototype design, but when focusing on hardware performance, the Arduino Uno was a simple and reliable platform.
The main target criteria for a successful design was anticipated as;
- A wide enough field of view (>100 degrees)
- Adequate video resolution
- Accurate and responsive tracking
- A high refresh rate
- Low Latency
From these, the main issue observed was that to do with minimizing video latency, that is the time from moving your head to receiving the corresponding image to your eyes. The cameras, wireless video communication and display methods created a latency up to 133 milliseconds when stationary, with an additional 26 ms generated via the mechanical response of the head-tracking, serial comm's and mechanical adjustment of the PTU. This 26 ms of data processing/signal generation is insignificant when compared to that of the video latency.
In replicating this project, there are many issues regarding video equipment I would reconsider to increase image quality whilst minimizing latency. However, regarding the elements relating to the Arduino, I would consider a smaller packaged board (such as Arduino Pro Mini) and faster operating servo motors as the only limitation experienced was in the rate of PTU adjustment, not in speed of processing and I/O capacity.
Thank you, Arduino, for meeting my needs in what would otherwise be a daunting project.