Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (5): 103-108.doi: 10.6040/j.issn.1672-3961.0.2018.228

• Control Science & Engineering • Previous Articles     Next Articles

Intelligent interaction method for power distribution robot based on Kinect

Mian ZHANG(),Ying HUANG,Haiyi MEI,Yu GUO*()   

  1. School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China
  • Received:2018-05-28 Online:2018-10-01 Published:2018-05-28
  • Contact: Yu GUO E-mail:zhangmian126@163.com;guoyu@njust.edu.cn
  • Supported by:
    江苏省重点研发计划资助项目(BE2017161);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1701049);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1701074);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1601242);江苏高校优势学科建设工程资助项目(AD20540)

RichHTML

8

PDF (PC)

589

Abstract:

In order to realize natural interaction between the operator and the live-line working robot, an intelligent man-machine interaction method based on the Kinect gesture recognition was proposed. Based on the Kinect's depth information and skeleton information, the operator's gesture was segmented, the geometric moment Hu moment was selected as the gesture feature, and the machine learning method of SVM was used to classify and identify the operator's gesture. The gestures were mapped to the motion of the robot, and the motion control of the robot was realized by the gesture. The experimental results verified the feasibility of the intelligent human-computer interaction method of the power distribution robot.

Key words: live-line working robot, intelligent human-computer interaction, machine learning, gesture segmentation, gesture recognition, motion control

CLC Number: 

  • TP319.56

Fig.1

Segmentation diagram of gesture"

Table 1

Statistic results of gesture recognition error rate based on SVM"

试验编号 手势1错误数 手势2错误数 手势3错误数 手势4错误数 手势5错误数 总错误数 错误率/%
1 2 3 0 3 0 8 1.6
2 2 3 1 4 1 11 2.2
3 1 4 0 3 0 8 1.6
4 0 3 0 3 0 6 1.2
5 1 3 0 3 1 8 1.6
6 1 2 0 3 0 6 1.2
7 2 1 0 2 0 5 1.0
8 1 2 1 1 1 6 1.2
9 0 2 0 2 0 4 0.8
10 1 3 1 1 1 7 1.4

Fig.2

Schematic diagram of human-computer interaction"

Table 2

Mapping relationship between static gesture and manipulator"

手势编号 静态手势 机械臂末端移动方向
1 机械臂末端向X方向运动
2 机械臂末端向-X方向运动
3 机械臂末端向Y方向运动
4 机械臂末端向-Y方向运动
5 停止

Fig.3

The experiment results of robot human-computer interaction control"

1 胡毅. 输配电线路带电作业技术的研究与发展[J]. 高电压技术, 2006, 32 (11): 1- 10.
doi: 10.3969/j.issn.1003-6520.2006.11.001
HU Yi . Research and development of live working technology of transmission and distribution line[J]. High Voltage Technology, 2006, 32 (11): 1- 10.
doi: 10.3969/j.issn.1003-6520.2006.11.001
2 胡毅, 王力农, 邹伟. 送配电带电作业的发展与标准制定[J]. 高电压技术, 2003, 29 (4): 6- 8.
doi: 10.3969/j.issn.1003-6520.2003.04.003
HU Yi , WANG Linong , ZOU Wei . Development and standard formulation of live working for distribution[J]. High Voltage Technology, 2003, 29 (4): 6- 8.
doi: 10.3969/j.issn.1003-6520.2003.04.003
3 高焕兵, 田国会. 面向配电系统的带电抢修作业机器人[J]. 山东大学学报(工学版), 2015, 45 (1): 45- 53.
GAO Huanbing , TIAN Guohui . Live-repair robot for power distribution system[J]. Shandong University Journal(Engineering Edition), 2015, 45 (1): 45- 53.
4 蔡自兴, 郭璠. 中国工业机器人发展的若干问题[J]. 机器人技术与应用, 2013, (3): 9- 12.
doi: 10.3969/j.issn.1004-6437.2013.03.003
CAI Zixing , GUO Fan . Some problems in the development of industrial robots in China[J]. Robot Technology and Applications, 2013, (3): 9- 12.
doi: 10.3969/j.issn.1004-6437.2013.03.003
5 戚晖,赵玉良,李健,等.高压带电作业机器人主从式液压机械臂系统: CN201210096179.6[P]. 2012-08-01.
6 李健,戚晖,李运厂.一种高压带电作业主从控制机器人作业平台: CN201210095860.9[P]. 2012-08-01.
7 刘金存.带电作业机器人主从控制的研究[D].济南:山东建筑大学, 2014.
LIU Jincun. Research on master-slave control of live working Robot[D]. Jinan: Shandong University of Architecture, 2014.
8 高焕兵.带电抢修作业机器人运动分析与控制方法研究[D].济南:山东大学, 2015.
GAO Huanbing. Research on motion analysis and control method of robot for live repair work[D]. Jinan: Shandong University, 2015.
9 周航.基于计算机视觉的手势识别系统研究[D].北京:北京交通大学, 2007.
ZHOU Hang. Research on gesture recognition system based on computer vision[D]. Beijing: Beijing Jiaotong University, 2007.
10 KANG J , SEO D , JUNG D . A Study on the control method of 3-dimensional space application using Kinect system[J]. International Journal of Computer Science and Network Security, 2011, 11 (9): 55- 59.
11 WONGPHATI M , OSAWA H , IMAI M . User-defined gestures for controlling primitive motions of an end effector[J]. Advanced Robotics, 2015, 29 (4): 225- 238.
12 林海波, 梅为林, 张毅, 等. 基于Kinect骨骼信息的机械臂体感交互系统的设计与实现[J]. 计算机应用与软件, 2013, 30 (2): 157- 160.
doi: 10.3969/j.issn.1000-386x.2013.02.040
LIN Haibo , MEI Weilin , ZHANG Yi , et al. Design and implementation of mechanical arm body sense interaction system based on Kinect skeleton information[J]. Computer Applications and Software, 2013, 30 (2): 157- 160.
doi: 10.3969/j.issn.1000-386x.2013.02.040
13 张莹莹, 郭星. 基于Kinect动态手势识别算法的研究与实现[J]. 计算机技术与发展, 2017, 27 (12): 11- 15.
ZHANG Yingying , GUO Xing . Research and implementation of the algorithm based on the Kinect dynamic gesture recognition[J]. Computer Technology and Development, 2017, 27 (11): 11- 15.
14 于泽升, 崔文华, 史添玮. 基于Kinect手势识别的应用与研究[J]. 计算机科学, 2016, (增刊2): 568- 571.
YU Zesheng , CUI Wenhua , SHI Tianwei . Application and research on the Kinect gesture recognition[J]. Computer Science, 2016, (Suppl.2): 568- 571.
15 DING J , CHANG C W . An adaptive hidden Markov model-based gesture recognition approach using Kinect to simplify large-scale video data processing for humanoid robot imitation[J]. Multimedia Tools and Applications, 2016, 75 (23): 15537- 15551.
doi: 10.1007/s11042-015-2505-9
16 余涛, 叶金永, 邵菲杰, 等. Kinect核心技术之骨架追踪技术[J]. 数字技术与应用, 2012, (10): 115.
YU Tao , YE Jinyong , SHAO Feijie , et al. Skeleton tracking technology for the Kinect core technology[J]. Digital Technology and Applications, 2012, (10): 115.
17 江冬梅.基于视觉的手势识别及其在人机交互中的应用[D].济南:山东大学, 2005.
JIANG Dongmei. Vision-based gesture recognition and its application in human-computer interaction[D]. Jinan: Shandong University, 2005.
18 王松林.基于Kinect的手势识别与机器人控制技术研究[D].北京:北京交通大学, 2014.
WANG Songlin. Research on gesture recognition and robot control technology based on Kinect[D]. Beijing: Beijing Jiaotong University, 2014.
19 罗元, 谢彧, 张毅. 基于Kinect传感器的智能轮椅控制系统的设计与实现[J]. 机器人, 2012, 34 (1): 110- 113, 119.
LUO Yuan , XIE Yu , ZHANG Yi . Design and implementation of intelligent wheelchair control system based on Kinect sensor[J]. Robot, 2012, 34 (1): 110- 113, 119.
20 王耀明. 几何矩及其几何矩不变量(英文)[J]. 上海电机学院学报, 2006, (2): 7- 10.
doi: 10.3969/j.issn.2095-0020.2006.02.003
WANG Yaoming . Geometric moment and its geometric moment invariants (English)[J]. Journal of Shanghai Institute of Electrical Engineering, 2006, (2): 7- 10.
doi: 10.3969/j.issn.2095-0020.2006.02.003
[1] LIU Yang, LIU Bo, WANG Feng. Optimization algorithm for big data mining based on parameter server framework [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 1-6.
[2] WEI Bo, ZHANG Wensheng, LI Yuanxiang, XIA Xuewen, LYU Jingqin. A sparse online learning algorithm for feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 22-27.
[3] ZHOU Wang, ZHANG Chenlin, WU Jianxin. Qualitative balanced clustering algorithm based on Hartigan-Wong and Lloyd [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(5): 37-44.
[4] MENG Lingheng, DING Shifei. Depth perceptual model based on the single image [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 37-43.
[5] LIU Jie, YANG Peng, LYU Wensheng, LIU Agudamu, LIU Junxiu. Prediction models of PM2.5 mass concentration based on meteorological factors [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(6): 76-83.
[6] ZHENG Yi, ZHU Chengzhang. A prediction method of atmospheric PM2.5 based on DBNs [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(6): 19-25.
[7] JI Shuai, ZHANG Chengrui, SUN Shuren, HU Tianliang. Motion control system based on real-time ethernet [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(5): 7-13.
[8] YU Jing1, TIAN Guohui1*, YIN Jianqin2. Dynamic hand gesture recognition algorithm based on depth information [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(3): 52-56.
[9] XIE Lin1, YIN Xi-yao2, LI Fan-zhang3, WU Jia3. A kind of inverse resolution learning expression [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(4): 46-50.
[10] HE Xue-ying1, 2, QIN Wei1, YIN Yi-long1 *, ZHAO Lian-zheng1，QIAO Hao3. Video-based fingerprint verification using machine learning [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(4): 29-33.
[11] LIANG Chun-lin1, PENG Ling-xi2*. An immune network based unsupervised classifier [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 82-86.
[12] GUO Mao-Zu, ZOU Quan, LI Wen-Bin, HAN Ying-Peng. Learning in bioinformatics [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 1-6.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] HE Dongzhi, ZHANG Jifeng, ZHAO Pengfei. Parallel implementing probabilistic spreading algorithm using MapReduce programming mode[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 22 -28 .
[2] HUANG Jinchao. A new method for muti-objects image segmentation based on faster region proposal networks[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(4): 20 -26 .
[3] TANG Qingshun, JIN Lu, LI Guodong, WU Chunfu. Robotic manipulators tracking control based on adaptive terminal sliding mode controller[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(5): 45 -53 .
[4] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 77 -81 .
[5] WANG Huan, ZHOU Zhongmei. An over sampling algorithm based on clustering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 134 -139 .
[6] YANG Ai-min1, ZHOU Yong-mei1, DENG He2, ZHOU Jian-feng3. Method of feature generation and selection for network traffic classification[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 1 -7 .
[7] YOU Ming-yu, CHEN Yan, LI Guo-zheng. Im-IG: A novel feature selection method for imbalanced problems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 123 -128 .
[8] WU Guo-yao1, MA Li-yong2. A method based on FFD B-spline registration of the iris image fusion[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 24 -27 .
[9] XIAO Qiao, PEI Jihong, WANG Lixia, GONG Zhicheng. Ship detection in remote sensing image based on the fuzzy fusion of multi-channel Gabor filtering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 29 -35 .
[10] MA Xiangming, SUN Xia, ZHANG Qiang. Construction and analysis on typical working cycle of wheel loader[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 82 -87 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd