ADAS vehicle application

Everyone should be aware of the lane when driving, ensure that the vehicles are within the lane limits, ensure smooth traffic, and minimize the chance of collisions with other vehicles in the nearby lane. This is a critical task for an automobile.

Challenges

The biggest challenge was the first time all members of the team used Xilinx hardware, specifically the Kv-260 SOM. Members need to spend a lot of time learning how to use the Kv-260.

Hardware

• Xilinx Kv-260 SOM
• Electronic Display
• Ethernet Camera
• Binocular Camera

Neural Architect

The network has four task modules and each task performs complementary cooperation: grid box regression, object detection, multi-label classification, and prediction of the vanishing point. This structure allows us to detect and classify the lane and road markings, and predict the vanishing region simultaneously in a single forward pass.

Lane Recognition and Offset Direction Monitoring

The data used in the lane recognition and traffic sign recognition tests is the Caltech lanes dataset. In this experimental design, the lane lines on the road were successfully identified. On the display of the final results, the results of lane line identification were marked with pink lines. In actual vehicle applications, the camera is placed in the very center of the vehicle. In the picture displayed by the results, the red vertical line is the position of the center of the camera picture, that is, the driving direction of the vehicle. At the same time, according to the result of the lane line detection just now, the center position of the lane line is marked and marked with a blue diamond on the screen. In practical applications, the next driving operation of the vehicle is prompted according to the relative position of the red vertical line and the blue diamond, that is, the relative position of the driving direction of the vehicle and the center of the lane. If the diamond is located to the left of the red vertical line, it means that the driving direction of the vehicle deviates from the left side of the road. At this time, the driving direction of the prompt should be to drive on the right. Similarly, if the blue diamond is located to the right of the red vertical line, it indicates that the driving direction is to drive on the left.

Binocular Camera

With the development of intelligent manufacturing, distance detection is very important for autonomous mobile robots and driverless vehicles. The common ranging methods are

Acoustic ranging, laser pulse ranging, infrared ranging, optical ranging and stereo ranging, etc. by measuring the distance between the emission and return of the emission source

The method of calculating distance by time difference is called active method. Acoustic, laser, infrared and other RF devices use this method. Although this kind of ranging is convenient and rapid, it performs poorly in the environment of reflection, noise, intersection and other problems, so the application of active ranging is limited.

Traffic sign detection

In assisted driving, traffic sign recognition is an important task. This project uses the Chinese traffic sign dataset TT100K jointly collected by Tsinghua University and Tencent as the training object, and uses the YOLOX target detection method to realize real-time traffic sign recognition. Traffic sign recognition also requires preprocessing operations such as noise removal and illumination improvement. The next step is to perform segmentation detection on the image, and segment the traffic sign part detection. After that, the key feature information in the image is extracted, and finally it is compared with the information of the data set.

The preprocessed image still contains a lot of information, and the traffic sign has only a small area in it. In order to reduce the amount of data processed and speed up the processing speed, the area of ​​the traffic sign is generally detected first, and then to determine the area in this area. The specific meaning of the traffic signs. Traffic signs have certain particularities in color and shape, and can be classified to a certain extent according to the following figure, so traffic signs are generally detected by these two features. Colors can be divided into RGB, HSV, HSI and other color spaces according to international standards, and quantitative analysis is carried out. Taking RGB space as an example, the colors are divided into three colors: red, green, and blue. The chromaticity coordinate range of blue, yellow and red can filter out irrelevant color information and quickly detect traffic signs.

It is obviously not enough to only detect the color. Due to the influence and interference of light and background color, it is also necessary to perform shape detection on the corresponding area on the basis of the color detection result. Traffic signs are characterized by clear edges and simple and easily recognizable shapes. These features are more obvious in the grayscale image after excluding the influence of color. Therefore, by using a certain edge detection operator to determine the grayscale step change in the image pixels, the shape and outline of the traffic sign can generally be more accurately detected. feature.

After the image detection is completed, basically only the key information of the traffic signs is left in the image. These information are simple and intuitive, but the computer still does not know the specific meaning of the information. At this time, it is necessary to further extract and compare these image features. Yes, in order to identify specific information. The key features of an image are the key factors for identifying specific information, and the quality of the features directly determines the accuracy of the identification. Generally speaking, these key features need to have several elements such as distinguishability, simplicity, and anti-interference. Differentiability means that the characteristics of different signs must have sufficient differences. Simplicity is the premise of ensuring distinguishability. Less data represents image features, which can ensure the speed and efficiency of detection, and the anti-interference degree, that is, image feature information, should be guaranteed to be as little as possible affected by noise, natural light and image distortion. In traffic sign recognition, color features, line change features, moment features, gray-scale histogram statistical features, etc. are generally extracted, and a feature library with a sufficient number of samples will be maintained in advance, including the image feature information of existing traffic signs.. At the time of recognition, the features of the collected images are extracted and compared with the conditions in the data set, and the actual meaning of the traffic sign can be judged.

Conclusion and Future Work

All in all, we have successfully designed a system that can achieve binocular ranging, traffic sign recognition and level road recognition, lane detection, prompting the driver's driving direction, and achieved good results.

References

https://github.com/SeokjuLee/VPGNet

https://github.com/csust7zhangjm/CCTSDB

https://github.com/mohamedadaly/caltech-lane-detection