The task of manual bin picking is tedious and slow, and every time a human touches a part, both wage costs and the potential for injury increase. A primary goal of many manufacturing facilities is to minimize the need for workers to perform repetitive tasks and automate the process entirely. This approach allows manufacturers to make better use of skilled labor for more complex tasks and continue optimizing their operations.
Although picking randomly situated items from a bin is a simple task for humans, the job is actually quite complex to automate. Because robotic bin picking requires multiple forms of perception and decision making, the system must recognize the part position and orientation, understand the bin location, and calculate and optimize 3D trajectories to reach and extract the bin contents. Then, when movement commences, the robot must avoid obstacles, including the interior of the bin itself, extract the part without damaging it and place the part where it belongs.
For robots to achieve effective bin picking, three key technologies come into play:
- 3D robotic vision
- Machine learning capability
- Computer processing power sufficient to carry out the process quickly and reliably.
To understand the importance of 3D vision—which is critical to successful automated bin picking—it is necessary to understand the development and application of 2D robotic vision. Early pick-and-place systems recognized parts in X- and Y-planes by applying 2D vision technology. The robot controller used that information to train robots to repeat the tasks. Beginning in the 1990s, evolution of the 2D vision systems has made bin picking faster, less expensive and quicker to set up, and now includes advanced algorithms that simplify training the systems for specific applications.
2D vision is the most common industrial robotic vision system and is highly developed and relatively inexpensive. Its ability to locate an object and pick it up eliminates the need for an operator to handle an item and/or build a fixture to hold it. Conveyor part transfer does not require a holder to locate parts. Objects can be randomly oriented, and it is not necessary to organize them so the robot can detect them. A 2D system has few moving parts and consequently simplifies the operation of an automated cell.
However, simplicity of 2D vision systems results in some limitations. A 2D system uses X and Y offsets and perhaps the rotation angle of a flat surface to locate parts. Items must lay relatively flat, and lighting must produce sufficient contrast between the parts and background. Some part overlap can be tolerated, but objects should be spread out to some extent to ensure reliable performance.
With the addition of sensing capability in the third or Z-axis dimension, 3D vision systems use X, Y and Z positions to locate parts in many degrees of freedom. As a result of this capability, 3D vision makes bin picking of overlapping parts much easier to perform and can even discriminate between semi-structured items such as parts wrapped in plastic or paper protective holders. 3D Vision systems are ideal in harsh environments that human operators cannot tolerate, including freezers.
Setting up for Success
To ensure effective bin picking operations, optimal camera placement is essential for 3D robotic vision systems and depends on the parts and operations required. When a wide view is necessary, a well-placed fixed camera can see the entire workspace. The robot picks the part and the processor takes the time identifying the location of the next part after the robot returns from dealing with the part. A second option with the camera carried on the robot also views a wide area. The robot positions the camera and takes pictures to determine its precise location. The arrangement provides greater flexibility than the fixed camera setup, but cycle times are longer. A third, but less common, approach to camera placement uses a fixed-mount camera on the robot itself.
In addition to camera placement, a variety of other factors affect the ability of robotic vision systems to sustain effective recognition reliability. Consistent and bright light sources, for example, can be critical to bin-picking operations. This is especially true for 2D systems which benefit from using light sources mounted on the robot to overcome sensitivities to background clutter or slight variances in lighting.
In some situations, light may also need to be of a specific color or frequency, such as infrared lighting, to boost detection accuracy. The configuration of the object itself—including its size and complexity, as well as the size of the part and the size of the image produced by the camera—can also affect reliability of recognition.
When part recognition is consistent, a vision system provides multiple benefits in a bin picking operation. The system eliminates the need to organize the parts for the robot to see them, which allows for greater flexibility within the overall operation. In the absence of a vision system, additional tooling is required to hold the part in a specific place and manner.
Similarly, part transfer with a conveyor no longer requires a holder to position parts in a specific place and position. The vision system simplifies overall operation of an automated cell because it minimizes the number of moving parts and steps needed to complete pick-and-place operations.
In addition to selection of robotic vision system hardware components, calibration is necessary to correlate the camera image with the robot’s coordinate-directed motion system.
In 2D vision systems, a plate overlaid with a grid comprised of blocks of a known size is used to adjust image pixels. In hand-eye calibration, the camera sends information about the part location, and the robot learns to translate that into positioning information. Calibration basically correlates where the camera is and what it sees with the robot’s coordinate system.
Using Machine Learning
Combined with effective vision and calibration, machine learning plays a critical role in optimizing the task by supplementing the vision system’s ability to understand and manipulate organic part shapes—i.e., objects whose shapes vary from one instance to another. Machine learning helps reduce complexity in configuration, for example, dealing with variances in lighting.
Working with Roboception GmbH, a pioneer of 3D sensor technology, KUKA Robotics developed a cost-effective system that achieves accurate robotic bin picking. Together, the two companies perfected the system that successfully blends the physical (robot) with the digital (AI)/machine learning and allows users to quickly configure and deploy the automation through a user-friendly web GUI without the need for special skill sets.
In essence, the KUKA/Roboception system allows the robot to identify which parts it can pick from a bin and which are not oriented in a reachable position or are obscured by another part. For those parts that are accessible but not in a perfect pick position, the robot makes quick adjustments.
Additionally, KUKA’s open interfaces allow for the easy integration of third-party vision solutions, such as those from SICK and Keyence Corporation of America. The SICK PLB robot guidance system is designed for precise localization of parts stored in boxes or on pallets, while Keyence’s systems address common vision system challenges and provide image stability, quality and consistency needed for bin-picking applications.
The large amount of information about the production environment and the parts themselves requires considerable computing power to process effectively and at full speed. Bin picking relies on a robot controller with sufficient processing “muscle” to automate with low cycle times. Because every manufacturer wants to reduce cycle times, system refinement, along with greater processing power, is the focus of bin picking efficiency.
To operate at maximum speed and efficiency, a robotic bin picking system requires optimized vision and learning capabilities supported by considerable computing resources. The robot controller must have sufficient power to provide maximum accuracy and repeatability as well as the lowest possible cycle times. Fast and accurate bin picking relies on a well-designed system with optimized vision, learning and processing power.
Yarek Niedbala is the vice president of sales at KUKA Robotics Canada Ltd. in Toronto. He has been with KUKA for more than 23 years, and holds degrees in systems design engineering from the Universite de Technologie de Compiegne and the University of Waterloo.