Words of Wisdom

The recent artificial intelligence (AI) boom that has swept over the whole world was ignited primarily by advances in large language models (LLMs). These LLMs have shown extraordinary potential in a number of fields, ranging from natural language processing and text generation to translation and chatbots. Their ability to understand and generate human-like text has transformed how we think about interacting with technology. This advancement has not only impacted industries such as customer service, education, and healthcare but has also opened up new possibilities for creative applications like content creation and storytelling.

But as people have had more and more time to experiment with this technology, certain limitations have become apparent. Despite their uncanny ability to produce coherent and relevant text in response to a wide range of prompts, LLMs often struggle when complex reasoning skills are needed. These shortcomings become more obvious when the algorithms are leveraged for tasks like generating source code for software, understanding the real world, or asking for help with decision making.

A major reason that LLMs struggle in these scenarios is that they do not deeply understand the text that they are trained on. We can form connections between different data points to form abstractions — or high-level understandings of complex concepts — that give us the ability to understand the world in a much deeper way. LLMs cannot form these abstractions in the way that humans can, which leaves them at a huge disadvantage in complex reasoning tasks.

Several active research projects underway at MIT’s CSAIL are seeking to change this present paradigm. Three teams have been working on methods that enable LLMs to form abstractions, and therefore reason in complex ways, in different areas. In particular, these projects seek to give LLMs the ability to reason deeply about code generation, decision making, and robot task planning. While each system is unique, they all employ a neurosymbolic method which combines traditional AI algorithms with logical rules and structured representations of knowledge.

The first framework, LILO, focuses on code synthesis, compression, and documentation by using LLMs paired with a Stitch algorithm to identify common code structures and create useful abstractions. This process leads to simplified programs that can be utilized for complex tasks, such as manipulating Excel spreadsheets and drawing two-dimensional graphics.

The second framework, named Ada, is designed to help AI agents with sequential decision-making tasks in household and gaming environments. Ada trains on potential tasks and their natural language descriptions, proposing action plans from the dataset. Human operators select the best plans, which are then implemented for various tasks. The framework showed significant improvements in virtual environments like a kitchen simulator.

The third framework, Language-guided abstraction (LGA), is focused on aiding robots in understanding their environments and developing plans for specific tasks. LGA translates human-provided task descriptions into abstractions that guide a robot's actions. The framework has been successful in helping robots complete tasks such as picking up fruit and throwing drinks into a recycling bin, demonstrating its potential to improve robots' ability to navigate and perform tasks in unstructured environments.

The potential for these advancements to impact fields like software engineering, robotics, and planning is significant, paving the way for more human-like AI systems that can tackle complex tasks and environments with greater ease. As these technologies continue to evolve, they could radically change how we approach a range of applications, from household chores to industrial automation.