While the paper is focused on the recognition aspect, its connection with the monitoring aspect and recovery aspect are also briefly described below.
The maximum force on the Z axis (Fzmax
) during picking up a capillary is used to monitor if the robotic manipulation is successful or failed. In the normal operation, Fzmax
has the range from 5.43 N to 6.13 N. While in the failed operation, a spike of force is sensed. shows three typical failures of picking up capillaries and their manipulation steps (approaching to the picking position for a new capillary, leaving from the picking position). In the first failure class, the last capillary is not disposed successfully, and when the gripper moves to the picking position for a new capillary, the old capillary crashed onto the new one and the already occupied gripper cannot grasp the new capillary. The second and third failure classes are caused by position misalignments. In the second class (small misalignment), the capillary is squashed by the spring-loaded collet of the gripper [2
]. With the third class (large misalignment), the capillary is pressed by the outer solid body of the gripper.
Failure modes on manipulating capillaries (A: failure because of last capillary not disposed (class1); B: failure because of small position misalignment (class2); C: failure because of big position misalignment (class3)).
For the three failure classes identified below, Fzmax varies respectively from 23.15 N to 47.88 N (class 1), from 22.62 N to 26.51 N (class 2), and from 35.08 N to 42.16 N (class 3). Therefore, the mean value of the lowest Fzmax of failure classes (22.62 N) and the highest Fzmax of the normal operation (6.13 N) is set as a threshold to label the current manipulation successful or not. Since this threshold is significantly away from either normal or abnormal scenarios, the probability of “false positive” is negligible.
Each failure class has different forces and duration time of the crashing between capillaries or the capillary and the gripper. This information is helpful to label manipulation failures and take corresponding recovery actions. shows typical sampled force/torque data knowing the first failure class. Obviously, Fz and Tx show the most significant changes when manipulation failures happen because the service robot moves vertically while picking up a capillary, and bent tubes cause torques around a horizontal axis. The duration time is around one second, which is used as the sampling window.
Sampled raw force/torque data from the force/torque sensor in the Cartesian coordinates (force: Fx, Fy, Fz; torque: Tx, Ty, Tz).
illustrates the procedure of failure recognition. The offline classifier training is to search for optimized classifiers for a given training data set (force/torque data of repeated experimental failures). The online fault recognition is to make a classification decision with the trained classifiers when manipulation failures happen in the RABiT. All of raw sampling data are passed through a low-pass filter to reduce noises. Characteristic features, such as peak value, duration time, and signal energy, are calculated for the following recognition steps.
Recognition procedure of robotic manipulation failures.
After the failure diagnosis, different recovery schemes are employed. When the class 1 failure occurs, the RABiT must stop the movement of the robot immediately, and send an emergency alert to the operator. Then the operator manually removes the undisposed capillary, and restarts the robot. As to the class 2 failure, automated recovery is implemented because the current capillary is not destroyed. The RABiT identifies the arm length (ratio of torque and force) and then calculate the corresponding position offset in 3D Cartesian coordinates to negate the arm length. The service robot will adjust the picking position of the desired new capillary accordingly and try to pick it up again. As to the failure of class 3, a semi-automated recovery is implemented. Although the current capillary is destroyed, the position offset can be known through the automated analysis of force/torque information. A logical and efficient sequence is to pass over this bent capillary, move onto next one with a position adjustment to account for the large misalignment.