Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (5): 103-108.doi: 10.6040/j.issn.1672-3961.0.2018.228

• Control Science & Engineering • Previous Articles     Next Articles

Intelligent interaction method for power distribution robot based on Kinect

Mian ZHANG(),Ying HUANG,Haiyi MEI,Yu GUO*()   

  1. School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China
  • Received:2018-05-28 Online:2018-10-01 Published:2018-05-28
  • Contact: Yu GUO E-mail:zhangmian126@163.com;guoyu@njust.edu.cn
  • Supported by:
    江苏省重点研发计划资助项目(BE2017161);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1701049);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1701074);国家电网公司科技项目资助资助项目(SGJSCZ00FZJS1601242);江苏高校优势学科建设工程资助项目(AD20540)

RichHTML

9

PDF (PC)

769

Abstract:

In order to realize natural interaction between the operator and the live-line working robot, an intelligent man-machine interaction method based on the Kinect gesture recognition was proposed. Based on the Kinect's depth information and skeleton information, the operator's gesture was segmented, the geometric moment Hu moment was selected as the gesture feature, and the machine learning method of SVM was used to classify and identify the operator's gesture. The gestures were mapped to the motion of the robot, and the motion control of the robot was realized by the gesture. The experimental results verified the feasibility of the intelligent human-computer interaction method of the power distribution robot.

Key words: live-line working robot, intelligent human-computer interaction, machine learning, gesture segmentation, gesture recognition, motion control

CLC Number: 

  • TP319.56

Fig.1

Segmentation diagram of gesture"

Table 1

Statistic results of gesture recognition error rate based on SVM"

试验编号 手势1错误数 手势2错误数 手势3错误数 手势4错误数 手势5错误数 总错误数 错误率/%
1 2 3 0 3 0 8 1.6
2 2 3 1 4 1 11 2.2
3 1 4 0 3 0 8 1.6
4 0 3 0 3 0 6 1.2
5 1 3 0 3 1 8 1.6
6 1 2 0 3 0 6 1.2
7 2 1 0 2 0 5 1.0
8 1 2 1 1 1 6 1.2
9 0 2 0 2 0 4 0.8
10 1 3 1 1 1 7 1.4

Fig.2

Schematic diagram of human-computer interaction"

Table 2

Mapping relationship between static gesture and manipulator"

手势编号 静态手势 机械臂末端移动方向
1 机械臂末端向X方向运动
2 机械臂末端向-X方向运动
3 机械臂末端向Y方向运动
4 机械臂末端向-Y方向运动
5 停止

Fig.3

The experiment results of robot human-computer interaction control"

1 胡毅. 输配电线路带电作业技术的研究与发展[J]. 高电压技术, 2006, 32 (11): 1- 10.
doi: 10.3969/j.issn.1003-6520.2006.11.001
HU Yi . Research and development of live working technology of transmission and distribution line[J]. High Voltage Technology, 2006, 32 (11): 1- 10.
doi: 10.3969/j.issn.1003-6520.2006.11.001
2 胡毅, 王力农, 邹伟. 送配电带电作业的发展与标准制定[J]. 高电压技术, 2003, 29 (4): 6- 8.
doi: 10.3969/j.issn.1003-6520.2003.04.003
HU Yi , WANG Linong , ZOU Wei . Development and standard formulation of live working for distribution[J]. High Voltage Technology, 2003, 29 (4): 6- 8.
doi: 10.3969/j.issn.1003-6520.2003.04.003
3 高焕兵, 田国会. 面向配电系统的带电抢修作业机器人[J]. 山东大学学报(工学版), 2015, 45 (1): 45- 53.
GAO Huanbing , TIAN Guohui . Live-repair robot for power distribution system[J]. Shandong University Journal(Engineering Edition), 2015, 45 (1): 45- 53.
4 蔡自兴, 郭璠. 中国工业机器人发展的若干问题[J]. 机器人技术与应用, 2013, (3): 9- 12.
doi: 10.3969/j.issn.1004-6437.2013.03.003
CAI Zixing , GUO Fan . Some problems in the development of industrial robots in China[J]. Robot Technology and Applications, 2013, (3): 9- 12.
doi: 10.3969/j.issn.1004-6437.2013.03.003
5 戚晖,赵玉良,李健,等.高压带电作业机器人主从式液压机械臂系统: CN201210096179.6[P]. 2012-08-01.
6 李健,戚晖,李运厂.一种高压带电作业主从控制机器人作业平台: CN201210095860.9[P]. 2012-08-01.
7 刘金存.带电作业机器人主从控制的研究[D].济南:山东建筑大学, 2014.
LIU Jincun. Research on master-slave control of live working Robot[D]. Jinan: Shandong University of Architecture, 2014.
8 高焕兵.带电抢修作业机器人运动分析与控制方法研究[D].济南:山东大学, 2015.
GAO Huanbing. Research on motion analysis and control method of robot for live repair work[D]. Jinan: Shandong University, 2015.
9 周航.基于计算机视觉的手势识别系统研究[D].北京:北京交通大学, 2007.
ZHOU Hang. Research on gesture recognition system based on computer vision[D]. Beijing: Beijing Jiaotong University, 2007.
10 KANG J , SEO D , JUNG D . A Study on the control method of 3-dimensional space application using Kinect system[J]. International Journal of Computer Science and Network Security, 2011, 11 (9): 55- 59.
11 WONGPHATI M , OSAWA H , IMAI M . User-defined gestures for controlling primitive motions of an end effector[J]. Advanced Robotics, 2015, 29 (4): 225- 238.
12 林海波, 梅为林, 张毅, 等. 基于Kinect骨骼信息的机械臂体感交互系统的设计与实现[J]. 计算机应用与软件, 2013, 30 (2): 157- 160.
doi: 10.3969/j.issn.1000-386x.2013.02.040
LIN Haibo , MEI Weilin , ZHANG Yi , et al. Design and implementation of mechanical arm body sense interaction system based on Kinect skeleton information[J]. Computer Applications and Software, 2013, 30 (2): 157- 160.
doi: 10.3969/j.issn.1000-386x.2013.02.040
13 张莹莹, 郭星. 基于Kinect动态手势识别算法的研究与实现[J]. 计算机技术与发展, 2017, 27 (12): 11- 15.
ZHANG Yingying , GUO Xing . Research and implementation of the algorithm based on the Kinect dynamic gesture recognition[J]. Computer Technology and Development, 2017, 27 (11): 11- 15.
14 于泽升, 崔文华, 史添玮. 基于Kinect手势识别的应用与研究[J]. 计算机科学, 2016, (增刊2): 568- 571.
YU Zesheng , CUI Wenhua , SHI Tianwei . Application and research on the Kinect gesture recognition[J]. Computer Science, 2016, (Suppl.2): 568- 571.
15 DING J , CHANG C W . An adaptive hidden Markov model-based gesture recognition approach using Kinect to simplify large-scale video data processing for humanoid robot imitation[J]. Multimedia Tools and Applications, 2016, 75 (23): 15537- 15551.
doi: 10.1007/s11042-015-2505-9
16 余涛, 叶金永, 邵菲杰, 等. Kinect核心技术之骨架追踪技术[J]. 数字技术与应用, 2012, (10): 115.
YU Tao , YE Jinyong , SHAO Feijie , et al. Skeleton tracking technology for the Kinect core technology[J]. Digital Technology and Applications, 2012, (10): 115.
17 江冬梅.基于视觉的手势识别及其在人机交互中的应用[D].济南:山东大学, 2005.
JIANG Dongmei. Vision-based gesture recognition and its application in human-computer interaction[D]. Jinan: Shandong University, 2005.
18 王松林.基于Kinect的手势识别与机器人控制技术研究[D].北京:北京交通大学, 2014.
WANG Songlin. Research on gesture recognition and robot control technology based on Kinect[D]. Beijing: Beijing Jiaotong University, 2014.
19 罗元, 谢彧, 张毅. 基于Kinect传感器的智能轮椅控制系统的设计与实现[J]. 机器人, 2012, 34 (1): 110- 113, 119.
LUO Yuan , XIE Yu , ZHANG Yi . Design and implementation of intelligent wheelchair control system based on Kinect sensor[J]. Robot, 2012, 34 (1): 110- 113, 119.
20 王耀明. 几何矩及其几何矩不变量(英文)[J]. 上海电机学院学报, 2006, (2): 7- 10.
doi: 10.3969/j.issn.2095-0020.2006.02.003
WANG Yaoming . Geometric moment and its geometric moment invariants (English)[J]. Journal of Shanghai Institute of Electrical Engineering, 2006, (2): 7- 10.
doi: 10.3969/j.issn.2095-0020.2006.02.003
[1] ZHU Ming, SHI Chenglong, LÜ Pan, LIU Xianrong, SUN Chi, CHEN Jiancheng, FAN Hongyun. Deformation prediction method and engineering application of deep foundation pit based on optimized LSTM method [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 141-148.
[2] Xiushan NIE,Yuling MA,Huiyan QIAO,Jie GUO,Chaoran CUI,Zhiyun YU,Xingbo LIU,Yilong YIN. Survey on student academic performance prediction from the perspective of task granularity [J]. Journal of Shandong University(Engineering Science), 2022, 52(2): 1-14.
[3] Gaoteng YUAN,Yihui LIU,Wei HUANG,Bing HU. MR image classification and recognition model of breast cancer based onGabor feature [J]. Journal of Shandong University(Engineering Science), 2020, 50(3): 15-23.
[4] Dapeng ZHANG,Yajun LIU,Wei ZHANG,Fen SHEN,Jiansheng YANG. Fake comment detection based on heterogeneous ensemble learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 1-9.
[5] Minghe GAO,Ying ZHANG,Rongrong ZHANG,Zihao HUANG,Linyan HUANG,Fanyu LI,Xin ZHANG,Yanhao WANG. Air quality prediction approach based on integrating forecasting dataset [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 91-99.
[6] Yutian LIU, Runjia SUN, Hongtao WANG, Xueping GU. Review on application of artificial intelligence in power system restoration [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 1-8.
[7] Tong LI,Ran MA,Honghe ZHENG,Ping AN,Xiangyu HU. An error sensitivity model based on video statistical features [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 116-121.
[8] Qijie ZOU,Haoyu LI,Rubo ZHANG,Tengda PEI,Yan LIU. Survey of human-robot interaction control for autonomous driving [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 23-33.
[9] LIU Yang, LIU Bo, WANG Feng. Optimization algorithm for big data mining based on parameter server framework [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 1-6.
[10] WEI Bo, ZHANG Wensheng, LI Yuanxiang, XIA Xuewen, LYU Jingqin. A sparse online learning algorithm for feature selection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 22-27.
[11] ZHOU Wang, ZHANG Chenlin, WU Jianxin. Qualitative balanced clustering algorithm based on Hartigan-Wong and Lloyd [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(5): 37-44.
[12] MENG Lingheng, DING Shifei. Depth perceptual model based on the single image [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 37-43.
[13] LIU Jie, YANG Peng, LYU Wensheng, LIU Agudamu, LIU Junxiu. Prediction models of PM2.5 mass concentration based on meteorological factors [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(6): 76-83.
[14] ZHENG Yi, ZHU Chengzhang. A prediction method of atmospheric PM2.5 based on DBNs [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(6): 19-25.
[15] JI Shuai, ZHANG Chengrui, SUN Shuren, HU Tianliang. Motion control system based on real-time ethernet [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(5): 7-13.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] ZHANG Yong-hua,WANG An-ling,LIU Fu-ping . The reflected phase angle of low frequent inhomogeneous[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 22 -25 .
[3] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[4] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[5] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[6] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[7] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[8] WANG Bo,WANG Ning-sheng . Automatic generation and combinatory optimization of disassembly sequence for mechanical-electric assembly[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 52 -57 .
[9] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[10] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd