This "Speech-to-Reality" System Is the Closest You Can Get to a Working Star Trek Replicator
Chained artificial intelligence models turn spoken requests into physical objects in minutes.
Researchers from the Massachusetts Institute of Technology have developed a system that, they say, allows its users to "speak objects into existence" — turning spoken-word natural language commands into the robotic construction of physical objects.
"We’re connecting natural language processing, 3D generative AI [Artificial Intelligence], and robotic assembly," explains first author Alexander Htet Kyaw of the team's work. "These are rapidly advancing areas of research that haven't been brought together before in a way that you can actually make physical objects just from a simple speech prompt."
Dubbed a "speech-to-reality" system, the platform uses a speech recognition system to feed a spoken prompt into a large language model (LLM), which distills it down into tokens and finds the most statistically-likely continuation tokens. These are then fed into a generative AI system that produces a 3D model of the requested object as a digital mesh, which is then processed by a voxelization system to break down the mesh into standardized components for assembly.
The next step is for that list of components to be further processed to account for the physical constraints of the robotic assembly system — meaning no parts can be floating in free space, there's a limit to how big overhangs can be, and you can't build anything larger than the available working space — before a pair of robotic arms bring the object, or at least a skeletal representation of its archetype, to life.
"This project is an interface between humans, AI, and robots to co-create the world around us," Kyaw claims of the project. "Imagine a scenario where you say 'I want a chair,' and within five minutes a physical chair materializes in front of you." If that sounds like science fiction, then you're onto something: Kyaw lists Star Trek's Replicators and the self-assembling nanobots of Big Hero 6 as his inspiration.
"I want to increase access for people to make physical objects in a fast, accessible, and sustainable manner," Kyaw says. "I'm working toward a future where the very essence of matter is truly in your control. One where reality can be generated on demand."
The team's work has been published in the Proceedings of the ACM Symposium on Computational Fabrication 2025 (SCF '25) under open-access terms.