Boyuan Chen, Jane Nisselson/Columbia Engineering
Robot projected onscreen

Autonomous Robot Creates a Likeness of Itself

July 14, 2022
A robot that can model itself has important benefits, but the technology behind the self-awareness gives rise to concerns, too.

A Columbia Engineering team has created a robot that is able to learn a model of its entire body from scratch.

In a nutshell, the robot created a kinematic model of itself and used it to “plan motion, reach goals and avoid obstacles in a variety of situations,” noted the engineers in a press release. What’s more, the robotic arm was able to automatically recognize and compensate for damage to its body.

READ MORE: Welding Cobots are the New Hired Guns

The feat was achieved by placing the robotic arm inside a circle of five streaming video cameras. The robot was then able to observe itself through the cameras as it moved about and explored itself freely. The researchers noted that the robot contorted to learn how exactly its body moved in response to various motor commands.

The engineers reported that the robot was programmed to learn the relationship between its motor actions and the volume it occupied in its environment. The robot’s internal deep neural network took about three hours to complete the self-inspection and examination of its actions. The self-model was accurate to about 1% of its workspace, stated the press release.

“We were really curious to see how the robot imagined itself,” said Hod Lipson, professor of mechanical engineering and director of Columbia’s Creative Machines Lab, where the work was done. “But you can’t just peek into a neural network; it’s a black box.”

The researchers said they toiled with various visualization techniques before the self-image gradually emerged. “It was a sort of gently flickering cloud that appeared to engulf the robot’s three-dimensional body,” said Lipson. “As the robot moved, the flickering cloud gently followed it.”

READ MORE: Automate 2022: Machine Vision Solutions Point to the Future of Automation

Nurturing Self-reliant Autonomous Systems

The ability of robots to model themselves without being assisted by engineers has important implications. Aside from saving on labor, it provides prospects for self-regulated preventive and predictive maintenance. 

The engineering team said that self-reliance, such as the ability to keep up with its own wear-and-tear and the ability to compensate for damage, can have useful implications in given situations. For example, a factory robot can detect an anomaly and compensate or call for assistance.

“Self-modeling is a primitive form of self-awareness,” explained Lipson, who has worked over a decade on finding ways to give robots some form of self-awareness. “If a robot, animal or human has an accurate self-model, it can function better in the world, it can make better decisions and it has an evolutionary advantage.”

Giving robots full autonomy can be controversial and may give rise to ethical questions. Implementing the right kinds of technologies in the right situations, however, can be beneficial.

READ MORE: The Art of Robotics—and the Robotics of Art

Lipson added that the self-awareness demonstrated in this study is “trivial compared to that of humans, but you have to start somewhere. We have to go slowly and carefully, so we can reap the benefits while minimizing the risks.”

The authors—Boyuan Chen, Robert Kwiatkowski, Carl Vondrick and Hod Lipson—reported their work in Science Robotics.

A technical summary of this work can be seen below:

About the Author

Rehana Begg | Editor-in-Chief, Machine Design

As Machine Design’s content lead, Rehana Begg is tasked with elevating the voice of the design and multi-disciplinary engineer in the face of digital transformation and engineering innovation. Begg has more than 24 years of editorial experience and has spent the past decade in the trenches of industrial manufacturing, focusing on new technologies, manufacturing innovation and business. Her B2B career has taken her from corporate boardrooms to plant floors and underground mining stopes, covering everything from automation & IIoT, robotics, mechanical design and additive manufacturing to plant operations, maintenance, reliability and continuous improvement. Begg holds an MBA, a Master of Journalism degree, and a BA (Hons.) in Political Science. She is committed to lifelong learning and feeds her passion for innovation in publishing, transparent science and clear communication by attending relevant conferences and seminars/workshops. 

Follow Rehana Begg via the following social media handles:

X: @rehanabegg

LinkedIn: @rehanabegg and @MachineDesign

Sponsored Recommendations

Crisis averted: How our AI-powered services helped prevent a factory fire

July 10, 2024
Discover how Schneider Electric's services helped a food and beverage manufacturer avoid a factory fire with AI-powered analytics.

Pumps Push the Boundaries of Low Temperature Technology

June 14, 2024
As an integral part of cryotechnology, KNF pumps facilitate scientific advances in cryostats, allowing them to push temperature boundaries and approach absolute zero.

The entire spectrum of drive technology

June 5, 2024
Read exciting stories about all aspects of maxon drive technology in our magazine.


May 15, 2024
Production equipment is expensive and needs to be protected against input abnormalities such as voltage, current, frequency, and phase to stay online and in operation for the ...

Voice your opinion!

To join the conversation, and become an exclusive member of Machine Design, create an account today!