Learning to Go with the Flow

For those of us that live in a large metropolitan area, there are few things more frustrating than traffic. On average, the typical American commuting to an urban center spends 42 hours per year stuck in traffic according to the Texas A&M Transportation Institute. That is a lot of wasted time, and the problem is only growing worse — that figure was 37 hours in 2000, and just 18 hours in 1982. Given this trajectory, we are on track to spend far more of our lives stuck in traffic jams in the future. Spending literally months of our lives over the course of a typical career fighting traffic is not good for one's physical or mental wellbeing and is also harmful economically. Any solution that can ease this burden would be well received by the many millions of commuters that want that time back.

One particularly infuriating source of traffic delays is poorly programmed traffic lights. When a red light stops the flow of traffic at an intersection where there is no cross traffic waiting to go by, there is clearly a problem that needs to be solved. This is the challenge that was taken on by a group of researchers at Aston University in the UK. They developed a system that uses computer vision and machine learning to adapt traffic signal patterns to minimize wait times for motorists.

The team's solution uses RGB cameras to capture images of an intersection, such that it can see vehicles near the intersection, as well as those approaching from a bit of a distance. These images are fed into a deep reinforcement learning model that was trained to minimize wait times at the intersection. Based on the state of traffic at any given time, the traffic lights adapt to optimize the flow of traffic. Over time, the system will continue to learn and improve its performance as it encounters more situations. Testing has shown that this new method significantly outperforms traditional methods of programming traffic lights.

To initially train the model, a photorealistic simulated environment was developed. The reinforcement learning model improved its capabilities by being rewarded for reducing vehicle wait times and penalized as wait times increased in the simulation. No hard coded logic was implemented in the algorithm, although the team notes that it would be possible to prioritize certain types of traffic, such as emergency vehicles. To ensure a robust algorithm was produced, a large amount of training was conducted, and under various weather conditions. In a real-world test, it was found that this method was capable to adapting to varying traffic conditions and streamlining the flow of vehicles through the intersection.

The researchers hope to begin larger-scale testing of their technique on real roads later this year. At present, it is only possible for each intersection to be independently controlled, which misses the importance of traffic flow over larger areas, so the team is currently evaluating a means to build a multi-reinforcement learning agent architecture, in which the agents can coordinate with one another to achieve optimal traffic flow over larger areas. I think it is safe to say we are all pulling for the success of this team — less time wasted in traffic is a win for us all.