JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2016, Vol. 46 ›› Issue (5): 54-63.doi: 10.6040/j.issn.1672-3961.0.2016.262

Previous Articles     Next Articles

Hand-eye calibration and visual servo control for mobile manipulator

YANG Yuanhui, LI Guodong*, WU Chunfu, WANG Xiaolong, CAI Xiaowei   

  1. School of Mechanical and Electrical Engineering, Longyan University, Longyan 364012, Fujian, China
  • Received:2016-07-14 Online:2016-10-20 Published:2016-07-14

PDF (PC)

1139

Abstract: The eye-in-hand visual servo system integrated vision sensor into mobile manipulator was widely used for object grasping and delivering operation. However, there were some challenges, such as nonlinearity and complexity of the existing hand-eye calibration algorithm, together with the nonholonomic constraint of the mobile manipulator. To solve these problems, the mobile manipulator was proceeded as a generalized manipulator whose kinematic model was established, and the inverse kinematic analytical solutions were acquired. Then, a novel linear active vision based camera calibration method was proposed to determine the hand-eye relationship. Finally, a visual servo switching control law composed of eye-fixed approaching and look-then-doing open loop handling was designed to control the mobile manipulator to finish household objects handling operation. Simulation and experimental results showed that the linear active vision-based hand-eye calibration method could be utilized conveniently and precisely. Meanwhile, the designed control law could reduce the control difficulty brought by the nonholonomic constraint of the mobile manipulator.

Key words: hand-eye calibration, switch control, mobile manipulator, homography decomposition, active vision, visual servo

CLC Number: 

  • TP24.2
[1] EKVALL S, KRAGIC D, JENSFELT P. Object detection and mapping for service robot tasks[J]. Robotica, 2007, 25(02):175-187.
[2] SEVERINSON-EKLUNDH K, GREEN A, HUTTENRAUC-H H. Social and collaborative aspects of interaction with a service robot[J]. Robotics and Autonomous Systems, 2003, 42(3):223-234.
[3] CARRIKER W F, KHOSLA P K, KROGH B H. Path planning for mobile manipulators for multiple task execution[J]. IEEE Transactions on Robotics and Automation, 1991, 7(3): 403-408.
[4] HUANG Q, TANIE K, SUGANO S. Coordinated motion planning for a mobile manipulator considering stability and manipulation[J]. The International Journal of Robotics Research, 2000, 19(8): 732-742.
[5] KORAYEM M H, RAHIMI H N, NIKOOBIN A. Mathematical modeling and trajectory planning of mobile manipulators with flexible links and joints[J]. Applied Mathematical Modelling, 2012, 36(36): 3229-3244.
[6] GALICKI M. Real-time constrained trajectory generation of mobile manipulators[J]. Robotics and Autonomous Systems, 2016, 78: 49-62.
[7] VANNOY J, XIAO J. Real-time adaptive motion planning(RAMP)of mobile manipulators in dynamic environments with unforeseen changes[J]. IEEE Transactions on Robotics, 2008, 24(5): 1199-1212.
[8] KORAYEM M H, NAZEMIZADEH M, AZIMIRAD V. Optimal trajectory planning of wheeled mobile manipulators in cluttered environments using potential functions[J]. Scientia Iranica, 2011, 18(5):1138-1147.
[9] GALICKI M. An adaptive non-linear constraint control of mobile manipulators[J]. Mechanism and Machine Theory, 2015, 88:63-85.
[10] MAZUR A. Hybrid adaptive control laws solving a path following problem for non-holonomic mobile manipulators[J]. International Journal of Control, 2004, 77(15):1297-1306.
[11] LIN S, GOLDENBERG A A. Robust damping control of mobile manipulators[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2002, 32(1): 126-132.
[12] MAILAH M, PITOWARNO E, JAMALUDDIN H. Robust motion control for mobile manipulator using resolved acceleration and proportional-integral active force control[J]. International Journal of Advanced Robotic System, 2005, 2(2): 125-134.
[13] GALICKI M. Control of mobile manipulators in a task space[J]. IEEE Transactions on Automatic Control, 2012, 57(11): 2962-2967.
[14] HUTCHINSON S, HAGER G D, CORKE P I. A tutorial on visual servo control[J]. IEEE Transactions on Robotics and Automation, 1996, 12(5): 651-670.
[15] CHAUMETTE F, HUTCHINSON S. Visual servo control, part I: basic approaches[J]. IEEE Robotics and Automation Magazine, 2006, 13(4): 82-90.
[16] CHAUMETTE F, HUTCHINSON S. Visual servo control, part II: advanced approaches[J]. IEEE Robotics and Automation Magazine, 2007, 14(1): 109-118.
[17] WILSON W J, WILLIAMS H C C, BELL G S. Relative end-effector control using cartesian position based visual servoing[J]. IEEE Transactions on Robotics and Automation, 1996, 12(5): 684-696.
[18] CORKE P I, HUTCHINSON S A. A new partitioned approach to image-based visual servo control[J]. IEEE Transactions on Robotics and Automation, 2001, 17(4): 507-515.
[19] MALIS E, CHAUMETTE F, BOUDET S. 2 1/2 D visual servoing[J]. IEEE Transactions on Robotics and Automation, 1999, 15(2): 238-250.
[20] MA Y, KOSECKA J, SASTARY S S. Vision guided navigation for a non-holonomic mobile robot[J]. IEEE Transactions on Robotics and Automation, 1999, 15(3): 521-536.
[21] DIXON W E, DAWSON D M, ZERGEROGLU E, et al. Adaptive tracking control of a wheeled mobile robot via an uncalibrated camera system[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2001, 31(3): 341-352.
[22] AMARASINGHE D, MANN G K I, GOSINE R G. Vision-based hybrid control scheme for autonomous parking of a mobile robot[J]. Advanced Robotics, 2007, 21(8): 905-930.
[23] LUCA A D, ORIOLO G, GIORDANO P R. Image-based visual servoing schemes for nonholonomic mobile manipulators[J]. Robotica, 2007, 25(02): 131-145.
[24] MANSARD N, STASSE O, CHAUMETTE F, et al. Visually-guided grasping while walking on a humanoid robot[C] //IEEE International Conference on Robotics and Automation. Roma, Italy: IEEE Robotics and Automation Society, 2007: 3041-3047.
[25] WANG Y, LANG H, DE SILVA C W. A hybrid visual servo controller for robust grasping by wheeled mobile robots[J]. IEEE/ASME Transactions on Mechatronics, 2010, 15(5): 757-769.
[26] 杨唐文, 高立宁, 阮秋琦,等. 移动双臂机械手系统协调操作的视觉伺服技术[J]. 控制理论与应用, 2015, 32(1): 69-74. YANG Tangwen, GAO Lining, RUAN Qiuqi, et al. Visual servo technology for coordinated manipulation of a mobile dual-arm manipulator system[J]. Journal of Control Theory and Application, 2015, 32(1): 69-74.
[27] DENAVIT J, HARTENBERG R S. A kinematic notation for lower-pair mechanisms based on matrices[J]. Trans of The ASME Journal of Applied Mechanics, 1955, 22: 215-221.
[1] ZHAO Zijian, CHEN Zhaorui, LI Bingqing. A new method for hand-eye calibration based on non-minimization optimization [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(4): 28-33.
[2] YUAN Li, TIAN Guohui*, LI Guodong. Visual servo grasping of household objects for NAO robot [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(3): 57-63.
[3] DING Na-na, TIAN Guo-hui*, LI Guo-dong, ZHANG Qing-bin. Visual self-localization of biped robot based on artificial landmark [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(4): 51-56.
[4] LI Guo-dong, ZHAO Wei, TIAN Guo-hui*, XUE Ying-hua. A visual servoing algorithm based on rotation matrix decomposition [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2012, 42(1): 45-50.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[2] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[3] YUE Yuan-Zheng. Relaxation in glasses far from equilibrium[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 1 -20 .
[4] CHENG Daizhan, LI Zhiqiang. A survey on linearization of nonlinear systems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 26 -36 .
[5] WANG Yong, XIE Yudong. Gas control technology of largeflow pipe[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 70 -74 .
[6] LIU Xin 1, SONG Sili 1, WANG Xinhong 2. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 98 -100 .
[7] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 104 -107 .
[8] CHEN Huaxin, CHEN Shuanfa, WANG Binggang. The aging behavior and mechanism of base asphalts[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 125 -130 .
[9] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 131 -136 .
[10] LI Shijin, WANG Shengte, HUANG Leping. Change detection with remote sensing images based on forward-backward heterogenicity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 1 -9 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd