Researchers at UC Berkeley are developing a tool that uses machine learning to manage traffic so that autonomous, semi-autonomous, and manned vehicles can more efficiently share the road. The first rendition of the tool, called Flow, is designed to solve real-world traffic problems, including easing the bottlenecks on the San Francisco-Oakland Bay Bridge.
Many traffic researchers use models based on manually derived algorithms to design controls, but it is thought that machine-learning-based controls will provide more tangible and, in some cases, unpredictable benefits such as lower energy consumption and novel traffic-management solutions that are out of reach of human calculations.
“Flow solves large-scale, multi-vehicle problems by using simulations that are much more efficient than what can be produced without the aid of artificial intelligence,” says Alexandre Bayen, director of the UC Berkeley Institute of Transportation Studies. “And we use a cloud-based, open-source system so the development community can continue to build on it.”
Berkeley engineers have developed a system that brings machine learning to traffic management, particularly where autonomous, semi-autonomous, and manned vehicles share the road. (Credit: metamorworks/iStock/Getty Images)
One assumption in the Flow tool is that automated cars will use data from nearby smart vehicles and infrastructure to manage traffic, effectively becoming mobile traffic-managers. For example, to prevent bottlenecking, an automated car could use its speed and position to control nearby vehicles as they merge. Or it could pace its speed to help prevent the random, human-caused slowdowns that increase travel time and frustrate drivers. Some researchers believe there are huge benefits that could be provided by only 4 or 5% of the vehicles on the road if they were properly programmed and networked.
Flow uses deep reinforcement learning, a facet of machine learning that continuously improves its decision-making by learning from each problem it solves, then advancing those solutions through their repetition. Many applications, including robotics and game theory, use deep reinforcement learning, but this is the first time it has been combined with traffic-simulation tools.
The tool is used for highly detailed scenarios or standard tasks engineers can use to solve common types of traffic challenges, such as bottlenecks and intersection control. The solutions become shared baselines, called benchmarks, and they are critical to making progress, researchers say.
“In the transportation community we can use these benchmarks to compare and compete, and most importantly, to reproduce results,” says Daniel Work, a Vanderbilt professor studying these issues. “The benchmarks also accelerate progress by removing the need for developers to create their own elaborate models. So engineers can stop recreating the same task, such as an on-ramp merge, over and over. Instead, we can focus on finding solutions to that task.”
Standardized benchmarks already exist in other deep reinforcement learning applications, including natural-language processing and robotics. But for traffic management, researchers were unable to find such standards, so they set out to create them. Earlier papers from Bayen’s lab presented Flow as a traffic-simulation platform, then established initial benchmarks using simple scenarios such as cars driving on a ring or figure-eight.
One task features a computer model of the San Francisco-Oakland Bay Bridge, a classic bottleneck where 15 toll lanes merge into five traffic lanes, and the goal is to maximize how many cars flow off the bridge. Details for the task include the traffic volume and vehicle position and speed in each lane, so anyone can come up with a solution. There’s also a standard for comparing solutions.
The study’s other benchmarks include an on-ramp merge and a model of a Manhattan grid of traffic lights.
Researchers plan to tackle increasingly complex scenarios, and they hope outside collaborators will do the same. The goal is for the system to manage traffic at the citywide scale, incorporating the many different ways people drive while capturing the benefits of the small but growing percentage of smart vehicles on the road.
Researchers also plan to study potential downsides and unintended consequences of this technological approach. For example, if traffic smoothing works too well, improved travel times could lure more drivers onto the road, undermining short-term gains as well as the larger goals of reducing greenhouse gas emissions and energy consumption.
“Understanding both the opportunities and the pitfalls of mixed autonomy traffic is an incremental step to the gradual and inevitable combination of autonomous vehicles in not-too-distant future,” says Bayen. “We should simulate what this future will look like and start working on strategies to improve it.”
