Improved ORB-FLANN efficient matching method for coal gangue image
-
摘要:
针对煤矸石分拣机器人分拣煤矸石时，带式输送机输送带打滑、跑偏以及带速波动造成的目标煤矸石位姿变化，从而导致抓取失败或空抓漏抓等问题，提出了一种改进的ORB-FLANN (Oriented FAST and Rotated BRIEF-Fast Library for Approximate Nearest Neighbors)煤矸石识别图像与分拣图像高效匹配方法。提出改进ORB的特征点检测方法对煤矸石识别图像与分拣图像进行特征点检测，实现快速检测图像特征点；提出改进FLANN匹配算法对图像特征点进行匹配，实现煤矸石识别图像与分拣图像高效匹配。针对传统ORB方法对煤矸石图像特征检测时间长、重复率低问题，提出了改进ORB特征检测方法，提高了图像特征点检测速度和重复率；针对传统FLANN匹配方法对煤矸石图像匹配精确率低问题，提出了融合PROSAC算法的改进FLANN匹配方法，剔除错误特征匹配点对，提高了图像匹配的精确率。在自主研发的双机械臂桁架式煤矸石分拣机器人试验平台上应用文中方法、SURF特征匹配方法、HU不变矩匹配方法、SIFT特征匹配方法和ORB特征匹配方法分别进行了不同带速、尺度、旋转角度条件下的煤矸石匹配试验，结果表明：本方法的匹配率为98.2%，匹配时间为141 ms，具有匹配率高、实时性好以及鲁棒性强等特点，能够满足煤矸石识别图像与分拣图像高效精准匹配的要求。
Abstract:In order to solve the problem of grasping failure or missing grasping due to the change of target gangue position and posture caused by belt slip, deviation and belt speed fluctuation of belt conveyor when the gangue sorting robot sorts gangue, an improved ORB-FLANN efficient matching method of gangue recognition image and sorting image is proposed. An improved ORB feature point detection method is proposed to detect the feature points in the recognition image and sorting image of coal gangue, so as to realize fast detection of image feature points; An improved FLANN matching algorithm is proposed to match the image feature points to achieve efficient matching between the recognition image of coal gangue and the sorting image. Aiming at the problem of long time and low repetition rate of traditional ORB method for coal gangue image feature detection, an improved ORB feature detection method is proposed to improve the speed and repetition rate of image feature point detection; Aiming at the low accuracy of traditional FLANN matching method for coal gangue image matching, an improved FLANN matching method integrating PROSAC algorithm is proposed to eliminate the wrong feature matching point pairs and improve the accuracy of image matching. The method, SURF feature matching method, HU moment invariant matching method, SIFT feature matching method and ORB feature matching method are applied on the experimental platform of the double mechanical arm truss type gangue sorting robot independently developed by the team to carry out gangue matching experiments under different belt speeds, scales and rotation angles. The results show that the matching rate of the method in this paper is 98.2%, and the matching time is 141 ms. It has the characteristics of high matching rate, good real-time performance and strong robustness, It can meet the requirements of efficient and accurate matching of gangue recognition image and sorting image.
-
-
![]()
图 6 双机械臂煤矸石分拣机器人试验平台
Figure 6. Experimental platform of coal gangue sorting robot with double mechanical arm
表 1 不同带速下的匹配试验数据结果
Table 1 Results of matching experimental data at different band speeds
带速/
(m·s−1)匹配率/% 匹配时间/ms 文中方法 SURF
特征方法HU不变
矩方法SIFT
特征方法ORB
特征方法文中方法 SURF
特征方法HU不变
矩方法SIFT
特征方法ORB
特征方法0.5 98.8 95.5 92.5 94.9 96.1 145 347 645 1175 138 0.6 98.7 95.8 92.2 93.6 95.9 139 336 639 1159 136 0.7 98.5 93.8 91.6 93.5 95.4 132 323 628 1040 129 0.8 98.1 90.2 89.1 88.6 95.1 123 295 595 980 125 0.9 98.0 88.1 87.6 87.2 94.7 114 250 531 862 121 1.0 97.7 87.2 85.7 85.1 93.2 109 248 515 826 120 1.1 97.4 86.4 82.5 83.5 91.5 106 244 512 789 117 
下载: 导出CSV
表 2 不同尺度下的匹配试验数据结果
Table 2 Results of matching experimental data at different scales
尺度 匹配率/% 匹配时间/ms 文中方法 SURF
特征方法HU不变矩方法 SIFT
特征方法ORB
特征方法文中方法 SURF
特征方法HU不变矩方法 SIFT
特征方法ORB
特征方法0 98.8 95.5 92.5 94.9 96.1 145 347 645 1175 138 1σ 98.5 92.1 91.2 91.8 95.8 151 362 663 1185 157 2σ 98.3 90.9 89.8 89.5 94.2 154 398 685 1197 189 3σ 98.0 89.5 87.5 85.3 92.6 162 472 744 1203 212 4σ 98.0 86.2 84.3 84.8 91.3 173 531 803 1243 265
下载: 导出CSV
表 3 不同旋转角度下的匹配试验数据结果
Table 3 Results of matching experimental data at different rotation angles
旋转角度/（°） 匹配率/% 匹配时间/ms 文中方法 SURF
特征方法HU不变
矩方法SIFT
特征方法ORB
特征方法文中方法 SURF
特征方法HU不变
矩方法SIFT
特征方法ORB
特征方法−90 98.3 88.2 86.8 87.5 92.9 158 402 683 1327 372 −45 98.5 89.3 89.4 89.6 94.3 149 381 668 1268 259 0 98.5 93.8 91.6 93.5 96.1 132 323 628 1240 138 45 98.4 89.6 89.8 89.8 93.4 151 375 657 1278 263 90 98.3 88.5 87.9 87.9 92.6 156 396 679 1316 368
下载: 导出CSV
-
[1] 李 曼,段 雍,曹现刚,等. 煤矸分选机器人图像识别方法和系统[J]. 煤炭学报,2020,45(10):3636−3644. LI Man,DUAN Yong,CAO Xiangang,et al. Image identification method and system for coal and gangue s-orting robot[J]. Journal of China Coal Society,2020,45(10):3636−3644.
[2] 曹现刚,刘思颖,王 鹏,等. 面向煤矸分拣机器人的煤矸识别定位系统研究[J]. 煤炭科学技术,2022,50(1):237−246. CAO Xiangang,LIU Siying,WANG Peng,et al. Researchon coal g-angue recognition and positioning system for coal gangue sorti-ng robot[J]. Coal Science and Technology,2022,50(1):237−246.
[3] 李亚坤,马宏伟,王 鹏. 基于VGG_16网络的煤和矸石识别技术研究[J]. 煤炭技术,2022,41(9):156−159. LI Yakun,MA Hongwei,WANG Peng. Research on coal and ga-ngue recognition technology based on VGG_16 network[J]. Coal Technology,2022,41(9):156−159.
[4] WANG Peng,MA Hongwei,ZHANG Ye, et al. Trajectory planning for coal gangue sorting robot tracking fast-mass target under multiple constraints. [J]. Sensors,2023,23(9):4412.
[5] 王 鹏,曹现刚,马宏伟,等. 基于余弦定理-PID的煤矸石分拣机器人动态目标稳准抓取算法[J]. 煤炭学报,2020,45(12):4240−4247. WANG Peng,CAO Xiangang,MA Hongwei,et al. Dynamic target steady and accurate grasping algorithm of gangue sorting robot based on cosine theorem-PID[J]. Journal of China Coal Society,2020,45(12):4240−4247.
[6] 马宏伟,孙那新,张 烨,等. 煤矸石分拣机器人动态目标稳定抓取轨迹规划[J]. 工矿自动化,2022,48(4):20−30. MA Hongwei,SUN Naxin,ZHANG Ye,et al. Track planning of c-oal gangue sorting robot for dynamic targetstable grasping[J]. Jo-urnal of Mine Automation,2022,48(4):20−30.
[7] MA Hongwei,WEI Xiaorong,WANG Peng,et al. Multi-Arm Global Cooperative Coal Gangue Sorting Method Based on Improved Hungarian Algorithm[J]. Sensors,2022,22(20):7987.
[8] 曹现刚,吴旭东,王 鹏,等. 面向煤矸分拣机器人的多机械臂协同策略[J]. 煤炭学报,2019,44(S2):763-774. CAO Xiangang,WU Xudong,WANG Peng,et al. Collaborative str-ategy of multi-manipulator for coal-ganguesorting robot[J]. Journal of China Coal Society,44(S2):763-774.
[9] 曹现刚,乔欢乐,吴旭东,等. 考虑含矸率时变性的多臂协同策略优化方法[J]. 机械科学与技术,2023,42(11):1887−1894. CAO Xiangang,QIAO Huanle,WU Xudong,et al. Multi-arm coop-erative strategy optimization method considering time variabilityof gangue rate[J]. Mechanical Science and Technology for Aerospace Engineering,2023,42(11):1887−1894.
[10] 马宏伟,张 烨,王 鹏,等. 多机械臂煤矸石智能分拣机器人关键共性技术研究[J]. 煤炭科学技术,2023,51(1):427−436. MA Hongwei,ZHANG Ye,WANG Peng,et al. Research on key generic technology of multi-arm intelligent coal gangue sorting robot[J]. Coal Science and Technology,2023,51(1):427−436.
[11] 张 烨,马宏伟,王 鹏,等. 煤矸石智能分拣机器人研究进展与关键技术[J]. 工矿自动化,2022,48(12):42−48,56. ZHANG Ye,MA Hongwei,WANG Peng,et al. Research progress and key technologies of gangue sorting robot[J]. Journal of Mine Automation,2022,48(12):42−48,56.
[12] 程德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349−365. CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349−365.
[13] 汪 洋,金 勇,都之夏,等. 煤矿井下采煤工作面可视化安全监控系统设计[J]. 煤炭科学技术,2018,46(S1):171−175. WANG Yang,JIN Yong,DU Zhixia,et al. Design of under-shaft visual safety monitoring system for coal mining face in mine underground[J]. Coal Science and Technology,2018,46(S1):171−175.
[14] 程德强,寇旗旗,江 鹤,等. 全矿井智能视频分析关键技术综述[J]. 工矿自动化,2023,49(11):1−21. CHENG Deqiang,KOU Qiqi,JIANG He,et al. Overview of key t-echnologies for mine-wide intelligent video analysis[J]. Journal of Mine Automation,2023,49(11):1−21.
[15] 刘孝军,王 飞. 基于AI的煤矿视频智能分析技术[J]. 煤炭科学技术,2022,50(S2):260−264. LIU Xiaojun,WANG Fei. Application of video intelligent analysi-s technology in coal mine based on computer vision[J]. Coal S-cience And Technology,2022,50(S2):260−264.
[16] 程德强,徐进洋,寇旗旗,等. 融合残差信息轻量级网络的运煤皮带异物分类[J]. 煤炭学报,2022,47(3):1361−1369. CHENG Deqiang,XU Jinyang,KOU Qiqi, et al. Overview of key t-echnologies for mine-wide intelligent video analysis[J]. Journal of China Coal Society,2022,47(3):1361−1369.
[17] 孙 林,陈 圣,姚旭龙,等. 矿井智能监控目标识别的图像增强方法与应用[J/OL]. 煤炭学报,1−12[2023-12-20]. https://doi.org/10.13225/j.cnki.jccs.2023.0489. SUN Lin,CHEN Sheng,YAO Xulong,et al. Image enhancement methods and applications for target recognition in intelligent m-ine monitoring[J/OL]. Journal of China Coal Society,1−12[2023-12-20]. https://doi.org/10.13225/j.cnki.jccs.2023.0489.
[18] 刘嗣超,武鹏达,赵占杰,等. 交通监控视频图像语义分割及其拼接方法[J]. 测绘学报,2020,49(4):522−532. LIU Sichao,WU Pengda,ZHAO Zhanjie,et al. Image semantic segmentation and stitching method of trafficmonitoring video[J]. Acta Geodaetica et Cartographica Sinica,2020,49(4):522−532.
[19] 桂 辉,徐晓婷,李 博. 安防监控中图像拼接的配光问题研究[J]. 红外与激光工程,2018,47(8):378−385. GUI Hui,XU Xiaoting,LI Bo. Research on problems of light d-istribution of image splicing in security monitoring[J]. Infrared and Laser Engineering,2018,47(8):378−385.
[20] RYU Jeong-Tak,DONGHWOON Kwon. An Analysis of the Surveilla-nce image monitoring system using multi-image stitching[J]. International Journal of Imaging and Robotics,2017,17(3):31−40.
[21] 孙希延,刘 博,纪元法,等. 基于SIFT改进的无人机图像匹配算法[J]. 电光与控制,2023,30(5):34−38. SUN Xiyan,LIU Bo,JI Yuanfa,et al. Improved UAV image matc-hing algorithm based on SIFT[J]. Electronics Optics & Control,2023,30(5):34−38.
[22] 韩 宇,宗 群,邢 娜. 基于改进SIFT的无人机航拍图像快速匹配[J]. 南开大学学报(自然科学版),2019,52(1):5−9. HAN Yu,ZONG Qun,XING Na. Fast matching of UAV aerial image based on SIFT[J]. Acta ScientiarumNaturalium Universitatis Nankaiensis,2019,52(1):5−9.
[23] MOUSAVI Vahid,Varshosaz Masood,Remondino Fabio. Using Information Content to Select Keypoints for UAV Image Matching[J]. Remote Sensing,2021,13(7):1302−1302. doi: 10.3390/rs13071302
[24] 魏 玮,张芯月,朱 叶. 改进的SIFT结合余弦相似度的人脸匹配算法[J]. 计算机工程与应用,2020,56(6):207−212. WEI Wei,ZHANG Xinyue,ZHU Ye. Improved SIFT algorithm combined with cosine similarity for face matching[J]. Computer Engineering and Applications,2020,56(6):207−212.
[25] 师 硕,于 洋,杨志坚,等. 基于SURF和形状上下文的人脸匹配算法[J]. 计算机应用研究,2018,35(10):3197−3200. SHI Shuo,YU Yang,YANG Zhijian,et al. Face image matching algorithm based on SURF and shape context[J]. Application Re-search of Computers,2018,35(10):3197−3200.
[26] GOGAN Taylor,BEAUDRY Jennifer,OLDMEADOW Julian. Image variabi-lity and face matching[J]. Perception,2022,51(11):804−819.
[27] 张朝伟,周 焰,吴思励,等. 基于SIFT特征匹配的监控图像自动拼接[J]. 计算机应用,2008(1):191−194. ZHANG Chaowei,ZHOU Yan,WU Sili,et al. Automatic Mosaic of surveillance images based on SIFT featurematching[J]. Computer Applications,2008(1):191−194.
[28] LI X G. ,REN C,ZHANG T X, et al. UNMANNED AERIAL VEH-ICLE IMAGE MATCHING based on improved ransacalgorithm and surf algorithm[J]. ISPRS - Internatio-nal Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2020:XLII-3/W1067-70.
[29] 姜煜杰. 改进的LBP算法在人脸识别中的研究与应用[J]. 湖北师范大学学报(自然科学版),2023,43(2):51−59. JIANG Yujie. Research and application of improved LBP algor-ithm in face recognition[J]. Journal of Hubei University(Natural Science),2023,43(2):51−59.
[30] 钟鹏程,李 伟,刘敬华. 基于改进的ORB算法的工件图像识别方法[J]. 机床与液压,2020,48(21):12−16. ZHONG Pengcheng,LI Wei,LIU Jinghua. Workpiece image reco-gnition method based on improved ORB algorithm[J]. MachineTool & Hydraulics,2020,48(21):12−16.
[31] ZHANG Hua,ZHENG Guoxun,FU Haohai. Research on image feature point matching based on ORB and RANSAC Algorith-m[J]. Journal of Physics:Conference Series,2020,1651(1):012187.
[32] 刘 畅,党淑雯,陈 丽. 基于ORB-SLAM3的改进型特征匹配与稠密建图算法[J]. 计算机应用研究,2023,40(11):3443−3449. LIU Chang,DANG Shuwen,CHEN Wen. Improved feature match-ing and dense-mapping algorithm based on ORB-SLAM3[J]. Application Research of Computers,2023,40(11):3443−3449.
[33] 廖泓真,王 亮,孙宏伟,等. 一种改进的ORB特征匹配算法[J]. 北京航空航天大学学报,2021,47(10):2149−2154. LIAO Hongzhen,WANG Liang,SUN Hongwei,et al. An improvedORB feature matching algorithm[J]. Journal of Beijing University of Aeronautics and Astronautics,2021,47(10):2149−2154.
[34] 包家汉,孙德尚,黄建中,等. 基于自适应阈值的型钢精确角点FAST检测算法[J/OL]. 上海交通大学学报:1−25[2023-10-15]. https://doi.org/10.16183/j.cnki.jsjtu.2023.276. BAO Jiahan,SUN Deshang,HUANG Jianzhong,et al. FAST algo-rithm for accurate corner points detection of section steel based on adaptive threshold [J/OL]. Journal of Shanghai JiaotongUniversity:1−25[2023-10-15]. https://doi.org/10.16183/j.cnki.jsjtu.2023.276.
[35] 宋超群,许四祥,杨 宇,等. 基于改进FAST和BRIEF的双目视觉测量方法[J]. 激光与光电子学进展,2022,59(8):173−180. SONG Chaoqun,XU Sixiang,YANG Yu,et al. Binocular vision measurement method using improved FAST and BRIEF[J]. Las-er & Optoelectronics Progress,2022,59(8):173−180.
[36] 周莉莉,姜 枫. 基于FAST和BRIEF的图像匹配算法[J]. 计算机工程与设计,2015,36(5):1269−1273. ZHOU Lili,JIANG Feng. Image matching algorithm based on F-AST and BRIEF[J]. Computer Engineeringand Design,2015,36(5):1269−1273.
[37] 杨 雷,唐瑞尹,张 怡. HOG-FLANN在图像匹配ORB算法中的应用[J]. 机械设计与制造,2022(10):67−70. YANG Lei,TANG Ruiyin,ZHANG Yi. HOG-FLANN application in image matching ORB Algorithm[J]. Machinery Design & Manufacture,2022(10):67−70.
[38] 张志敏,李 彬,田联房,等. 基于SURF的改进FLANN匹配算法[J]. 计算机工程与设计,2022,43(4):941−948. ZHANG Zhimin,LI Bin,TIAN Lianfang,et al. Improved FLANN matching algorithm based on SURF[J]. Computer Engineering a-nd Design,2022,43(4):941−948.
[39] IAGO S,GHESN S,M. J B, et al. BEBLID:Boosted Efficient Bin-ary Local Image Descriptor[J]. Pattern Recognition Letters,2020,133: 366-372.
[40] 杜文康,雷 冬,杭宗庆,等. 基于SURF-PROSAC法的高铁桥梁位移测量技术研究[J]. 铁道科学与工程学报,2023,20(9):3579−3591. DU Wenkang,LEI Dong,HANG Zongqing,et al. Deformation me-asurement technology of high-speed railway bridge based on S-URF-PROSAC method[J]. Jou-rnal of Railway Science and Engineering,2023,20(9):3579−3591.
[41] 张 均,叶庆卫. 基于PSO的改进AdaBoost人脸检测算法[J]. 计算机应用,2020,40(S1):61−64. ZHANG Jun,YE Qingwei. Improved AdaBoost face detection al-gorithm based on particle swarm optimization[J]. Journal of Co-mputer Applications,2020,40(S1):61−64.
[42] K. M,C. S. A performance evaluation of local descriptors[J]. I-EEE Transactions on Pattern Analysis andMachine Intelligence,2005,27(10):1615−1630.
[43] 张慧玲,李 博,张文平,等. 基于PCA-KD-KNN方法的矿井突水水源判别分析研究[J]. 矿业研究与开发,2020,40(12):106−111. ZHANG Huiling,LI Bo,ZHANG Wenping, et al. Studyon discri-minant analysis of mine water inrush source based on PCA-KD-KNN Method[J]Mining Research and Development, 2020,40(12):106−111.
[44] MIKOLAJCZYK K,SCHMID C. Scale & Affine Invariant Interest Poi-nt Detectors[J]. International Journal of Computer Vision,2004,60(1):63−86.
[45] CHEN L,GUO L,CHENG D,et al. Structure-preserving and color-res-toring up-sampling for single low-light image[J]. IEEE Transa-ctions on Circuits and Systems for Video Technology,2022,32(8):1889−1902.
[46] CHENG D, CHEN L,LV C,GUO L,et al. Light-guided and cross-fusionu-net for anti-illumination image super-resolution[J]. IEEE Tra-nsactions on Circuits and Systems for Video Technology,2022,32(12):8436−8449. doi: 10.1109/TCSVT.2022.3194169
计量
- 文章访问数: 221
- HTML全文浏览量: 50
- PDF下载量: 76
