Remotely controlling a robotic arm is no easy task, but one group of scientists is finding success. By integrating video technology and control devices, a team from Georgia Tech and the Georgia Tech Research Institute (GTRI) is developing a new way of simplifying remote control of robotic devices.
The goal is to enhance an operator’s ability to perform precise tasks using a multi-jointed robotic device such as an articulated mechanical arm. The approach is proving to be easier and faster than previous methods, especially when an operator controls the robot while watching it move on a monitor. Called Uncalibrated Visual Servoing for Intuitive Human Guidance of Robots, the idea is a special take on an existing vision-guided control method called visual servoing (VS). By applying VS technology in new ways, researchers have constructed a robotic system that responds to human commands more directly and intuitively than older techniques.
For decades, articulated robots have performed tasks such as welding vehicle seams and assembling electronics. Users develop a software program that enables the device to cycle through required motions, using feedback from sensors built into the robot. Though effective, programming is time-consuming. The robot must be maneuvered joint by joint through every action required to complete a task, and the technology only works in a structured environment for which spatial relationships are constant.
In recent years, new techniques have helped operators freely guide remote robots through unstructured environments, to perform tasks such as bomb disposal. Operators control the device by direct observation or by using a robot-mounted camera that sends images of both the robot and its target. Manipulating a remote robot into place is slow and laborious when operators must depend on imprecise images provided by 2D video feedback.
“Essentially, the operator is trying to visualize and reconstruct a 3D scenario from flat 2D camera images,” explains research engineer Ai-Ping Hu.
To simplify control, the team turned to VS. Traditional visual servoing is calibrated — position information generated by a video camera is transformed into data meaningful to the robot. Using this data, the robot can adjust itself to stay in a correct spatial relationship with target objects.
“If a conveyor line has accidentally moved a few millimeters, a robot with calibrated VS can detect the movement using the video image and a fixed reference point, and then readjust to compensate,” says Hu.
The research team has now adapted VS technology in ways that facilitate human control of remote robots, taking advantage of both calibrated and uncalibrated techniques. A calibrated 3D “time of flight” camera is mounted at the end of a robotic arm. The camera features an active sensor that detects depth data, allowing it to send back 3D coordinates that pinpoint the end-effector’s spatial location. At the same time, the camera also supplies an uncalibrated 2D video image to the monitor, giving the operator a robot’s-eye view of the target. By watching this image in a monitor, the operator can visually guide the robot using a gamepad.
The VS technology then automatically actuates all the joints needed to complete the action indicated on the gamepad. In the background, the new system performs the computation needed to coordinate the monitor image, 3D camera information, robot spatial position, and gamepad commands. For more information, visit gatech.edu.