Divide and Conquer for Smarter AI

In the late 1970s, engineers at IBM gave a presentation containing the now-famous quote: “a computer can never be held accountable, therefore a computer must never make a management decision.” My, how the times have changed! Due largely to the rise of artificial intelligence (AI), what once seemed like sound advice is no longer being heeded. The decision-making potential of AI algorithms is simply too great to ignore. These intelligent algorithms are already powering robots, chatbots, and many more systems that rely on them for their ability to make decisions. And there are big plans to lean more heavily on these AI systems in the years ahead.

While the potential is huge for these rapidly advancing technologies, anyone that has worked with them might shudder just a bit at the thought of handing control over to them. They make more than their fair share of mistakes, and they tend to get tripped up pretty easily when presented with inputs that deviate even a small amount from the distribution of their training data. Entrusting these tools with autonomy in important applications does not sound like a recipe for success.

Researchers at MIT may have found at least part of the solution to these problems, however. They have developed a technique that allows them to train models to make better decisions. Not only that, but it also makes the training process far more efficient, slicing costs and model training times to boot.

The team’s work builds upon reinforcement learning, which is a broad classification of algorithms that teach machines skills via a process that is something like trial-and-error. Existing approaches have some problems, however. They can be designed to only carry out a single task, in which case many algorithms have to be laboriously developed and trained to carry out complex tasks, or a single algorithm can be trained on mountains of data so that it can do many things, but the accuracy of these models suffer and they tend to be brittle as well.

The new approach takes a middle ground between these options, selecting some subset of the total set of tasks to be handled by each model. Of course the choice of tasks to train each algorithm for cannot be random, rather they must naturally work together well. So to make these selections, the researchers developed an algorithm called Model-Based Transfer Learning (MBTL).

MBTL assesses how well each model would perform on a single task, then checks how that performance would change as additional tasks are added in. In this way, the algorithm can find the tasks that naturally group together the best, giving the smallest possible reduction in performance.

An experiment was conducted in a simulated environment to evaluate how well the system might work under real-world conditions. The traffic signals in a city were simulated, with the goal of deciding how best to control them for optimal traffic flow. MBTL decided which individual traffic signals could be grouped together for control by a single algorithm, with multiple algorithms controlling the entire network.

It was found that this new approach could arrive at approximately the same level of performance as existing reinforcement learning techniques, but was up to 50 times more efficient in getting there. This is because far less training data was required to arrive at that state. Because the efficiency is so much greater with this new approach, in theory the performance could be much better in the future. It would be practical to supply a model with far more training data, which would help it to perform with greater accuracy and under a more diverse set of conditions.

Looking ahead, the team is planning to apply their technique to even more complex problems. They also want to step outside of the computer simulations and prove the algorithm’s worth in real-world use cases.