Author : Louis Perreault M.Sc., P. Eng.
ADEPT Technology INC.

1. Introduction

Modern manufacturing processes need to be flexible because of frequent product designchanges and shorter production runs arising from the increasing need for customizedproducts. This has led to the extensive use of programmable robots. However robotsalone are blind and parts need to be presented at a predefined location. Fixtures to accomplish this are often complex and require frequent attention for line changeovers. Adding vision guidance to a Robot can allow parts to be presented unfixtured. This presentation describes performance, features and other considerations for adding visioncapability to industrial robots. The following features will be covered:
- Vision system calibration
- Hand-eye calibration
- Robustness
- Accuracy
- Speed
- Integration with robot controller
- Multi-model capacity

2. Vision calibration

Calibrating the vision system means that it will return results in real world units (mm,inches…) not pixels. It also allows the system to compensate for problems like nonsquarepixels (which is the case for most cameras), perspective distortion (the camera isnot perfectly perpendicular to the work surface) and lens distortion. Because of thesecorrections the system is much more robust, accurate and repeatable. The system will beable to give very accurate results independent from part orientation and cameraplacement. Furthermore, given that the vision system sees objects in their real size,models of objects are portable from one station to another even if the physical setup is notexactly the same. This greatly reduces the workload for deployment and maintenance ofmulti-station systems. In a good vision system, calibration is a very short and simpleoperation usually done by presenting a known target to the system.

3. Hand-Eye Calibration

This is an absolute requirement for vision-guided robots. The vision system and the robotwork in two different reference frames. For vision results to have any meaning to therobot, we must find a transform that will enable us to translate vision results into therobot reference frame. This transform can be very simple or more complex depending onthe setup and the accuracy needs (simplification can be made when we have coincidingaxes and/or origins, which can be attained through proper choice of vision calibrationparameters and target placement). It can range from simple X-Y offsets to a full 6 D.O.Ftransformation between the two frames. Hand-eye calibration usually takes more timeand is more complex than vision calibration as it involves moving an object with therobot to many different positions in the field of view. Having a few point pairs in thevision and robot frames, it is then possible to compute a transform. Care must be taken todo this properly otherwise if hand-eye calibration is not accurate, accuracy of visionsystem is useless.

4. Robustness

One of the main goals of vision guidance is to reduce cost. If the vision system is notrobust enough, all that is saved in fixtures will be spent in vision setup. The vision systemmust be able to work with ambient light and off the shelf cameras and lenses. It must alsobe able to tell when it cannot return a reliable result so that proper action can be taken atthe application level. Robustness also means that the vision system won’t be affected bychanges in lighting, part orientation, defects or dirt on the part, part occlusion, a clutteredbackground or noise in the image signal. A very important element is also that the systemmust not detect parts that are not there and must detect all present parts. This may soundsimple and is relatively easy in a lab environment but is very difficult to reach in realworldconditions.

5. Accuracy

Accuracy needs will vary depending on the application. Some very simple packagingapplications need very high speed but do not require high accuracy. On the other handprecise assembly will require very high accuracy. If the vision system is not accurateenough for the task, the robot may pick up the part improperly and may have an impropergrip and/or incorrect placement. For applications where the robot must work on the part(drilling holes, machining, …), features may be machined off tolerance leading to rejectsor low quality. Since high accuracy is not always needed and is often reached at the costof speed, a good system should have the flexibility to choose different levels of accuracy.

6. Speed

Speed requirements will also vary depending on the application. In some applications theoperation the robot must perform or some other operation before or after is very long andso vision recognition time is not critical. But in many applications where Robots are used,speed is critical and any time saved in recognition means higher throughput. This in turnsof course means higher productivity and better ROI. It is important that speed be attainedwithout impairing robustness. A system that is fast but returns wrong results is uselessand can even lead to equipment damage. With modern computers, recognition times canbe well below 100ms even for complex tasks and below 10ms for simpler tasks.

7. Integration with robot controller

This is a very important, often overlooked feature. If the robot controller and visionsystem are not made to work together, Programming can be difficult and cumbersomeand speed can decrease very significantly. Even if the vision system is very fast, if thetime to communicate the result to the robot is long, total cycle time will greatly suffer. Inpoorly integrated system, this overhead can often be longer than the actual recognitiontime. A robot is often part of a workcell with other machines. Having a highly portablevision system means that it is possible to choose the best device to run it, whether it is therobot controller, a PC or embedded in another machine.

8. Multi-model

Multi-model capacity is not required in all applications but can be very useful when thepart can have multiple stable poses or many parts need to be recognized (e.g. assembly ofmultiple parts or production of many products on the same line). It is usually (but notalways) possible to handle multiple part type applications even with vision system thatcannot handle more than one model at a time. However this leads to more complexprogramming, slower execution time and most importantly lower reliability. Thisbecomes critical if you need to recognize very similar part. Having a knowledge of allpossible parts, a multi-model system will be able to reliably disambiguate between thedifferent possibilities and identify the correct part. If the system is not multi-model theburden is on the application programmer and the task is often impossible to accomplish.A one-model system will recognize both parts as the same with maybe a slight variationin the return result reliability (quality factor). A good multi-model system will be able tospot the subtle difference between the parts and give the correct result.

9. Conclusion

In this paper we have seen that using a good vision system to guide a robot can lead tocost savings and a better reaction time to changes. However to reach these goals, thevision system must have some important features. The ability of the vision system toresponds to these needs will directly affect the success of your project.

Tech Materials (Free)

Robot Behaviors Exploring the T-Maze: Evolving Learning-Like Robot Behaviors using CTRNNs
Humanoid Robotics A Biochemical Subsystem for a Humanoid Robot
Industrial Automation Systems Applying Agents for Engineering of Industrial Automation Systems
Robot Team Cooperation A Descriptive Model of Robot Team and the Dynamic Evolution of Robot Team Cooperation
Kuka Robots For ONU ONU Robotics Technology Center of Excellence, powered by KUKA Robotics Corporation
Augmented reality Annotation System for Robotic Application
Modular Robots Self-Reconfiguration Planning Of Identical Modules
Autonomous robots A New Approach To Robotics
Robotic Mounting Flat Panel Displays With Robotic Mounting
Calibration of Industrial Robots A Photogrammetric Robot Calibration System Based On Off-The-Shelf Low Cost Hardware Components

More...

Amazon Books

	Creative Projects with LEGO Mindstorms by Benjamin Erwin Buy new: $20.64 / Used from: $13.00 A good place to start, especially for kids, with Lego Mindstorms
	A Practical Guide to Behavior-Based Robotics by Joe Jones Buy new: $20.67 / Used from: $15.13 Very good for programming not so much behavior as control. Language and controller agnostic