Autonomous Robot Creates a Likeness of Itself

July 14, 2022

A robot that can model itself has important benefits, but the technology behind the self-awareness gives rise to concerns, too.

Rehana Begg

A Columbia Engineering team has created a robot that is able to learn a model of its entire body from scratch.

In a nutshell, the robot created a kinematic model of itself and used it to “plan motion, reach goals and avoid obstacles in a variety of situations,” noted the engineers in a press release. What’s more, the robotic arm was able to automatically recognize and compensate for damage to its body.

The feat was achieved by placing the robotic arm inside a circle of five streaming video cameras. The robot was then able to observe itself through the cameras as it moved about and explored itself freely. The researchers noted that the robot contorted to learn how exactly its body moved in response to various motor commands.

The learning progress of a robot arm trying to acquire a full-body 3D visual self-model.

The engineers reported that the robot was programmed to learn the relationship between its motor actions and the volume it occupied in its environment. The robot’s internal deep neural network took about three hours to complete the self-inspection and examination of its actions. The self-model was accurate to about 1% of its workspace, stated the press release.

“We were really curious to see how the robot imagined itself,” said Hod Lipson, professor of mechanical engineering and director of Columbia’s Creative Machines Lab, where the work was done. “But you can’t just peek into a neural network; it’s a black box.”

The researchers said they toiled with various visualization techniques before the self-image gradually emerged. “It was a sort of gently flickering cloud that appeared to engulf the robot’s three-dimensional body,” said Lipson. “As the robot moved, the flickering cloud gently followed it.”

A robot can learn full-body morphology via visual self-modeling to adapt to multiple motion planning and control tasks.

Nurturing Self-reliant Autonomous Systems

The ability of robots to model themselves without being assisted by engineers has important implications. Aside from saving on labor, it provides prospects for self-regulated preventive and predictive maintenance.

The engineering team said that self-reliance, such as the ability to keep up with its own wear-and-tear and the ability to compensate for damage, can have useful implications in given situations. For example, a factory robot can detect an anomaly and compensate or call for assistance.

A robot arm learns to reach a target sphere while avoiding the cuboid obstacle using the learned visual self-model.

“Self-modeling is a primitive form of self-awareness,” explained Lipson, who has worked over a decade on finding ways to give robots some form of self-awareness. “If a robot, animal or human has an accurate self-model, it can function better in the world, it can make better decisions and it has an evolutionary advantage.”

Giving robots full autonomy can be controversial and may give rise to ethical questions. Implementing the right kinds of technologies in the right situations, however, can be beneficial.

Lipson added that the self-awareness demonstrated in this study is “trivial compared to that of humans, but you have to start somewhere. We have to go slowly and carefully, so we can reap the benefits while minimizing the risks.”

The authors—Boyuan Chen, Robert Kwiatkowski, Carl Vondrick and Hod Lipson—reported their work in Science Robotics.

A technical summary of this work can be seen below: