| Citation: |
LI Heping，HONG Shaoqiu，CHENG Jian，et al. Coalmine image matching by fusing multi-level feature enhancement and weighted grid statistics[J]. Coal Science and Technology，2024，52（11）：129−140. DOI: 10.12438/cst.2024-1224
|
Image feature extraction and matching is a critical technique for video and image stitching, visual localization and navigation in underground coal mines. However, underground tunnelling environmental factors, such as low light, uneven illumination, and repetitive textures, result in low-contrast images with indistinct texture information, making feature point extraction difficult and increasing the rate of mismatches. To address these issues, an coalmine image matching method based on multi-level feature fusion enhancement and weighted grid statistics is proposed. Firstly, multi-level feature detection network based on deformable convolution layers and stacked network is adopted for ensuring the rotational invariance of features. Secondly, both the extracted feature points and descriptors are encoded and projected into a high-dimensional space using a feature enhancement module, and the Transformer is then used to enhance the distinguishability of the features. Finally, to address the perceptual confusion problem caused by repetitive textures in underground scenes, a multi-stage matching optimization strategy based on weighted grids is used. This strategy can combine matching quality factors and motion smoothness constraints to filter the mismatches. A lot of experiments were conducted on an underground actual dataset, as well as on two public datasets including LOL and HPatches. The experimental results show that the proposed feature extraction and matching method has higher accuracy and robustness. Specifically, compared with the ORB, SIFT, ASLFeat and Superpoint algorithms, the proposed feature extraction method achieved average accuracy improvements of 33.07%, 69.78%, 17.65% and 33.52% respectively. Compared with the feature matching methods including FLANN, BF+KNN, BF+RANSAC and BF+GMS, the proposed feature matching method achieved average accuracy improvements of 19.66%, 23.26%, 4.16% and 18.46%.
| [1] |
王国法,庞义辉,任怀伟,等. 智慧矿山系统工程及关键技术研究与实践[J]. 煤炭学报,2024,49(1):181−202.
WANG Guofa,PANG Yihui,REN Huaiwei,et al. System engineering and key technologies research and practice of smart mine[J]. Journal of China Coal Society,2024,49(1):181−202.
|
| [2] |
孔庆群,吴福朝,樊 彬. 基于深度学习的图像匹配:方法、应用与挑战[J]. 计算机学报,2024,41(7):1485−1520.
KONG Qingqun,WU Fuchao,FAN Bin. Image Matching in Deep Learning Era:Methods,Applications and Challenges[J] Chinese Journal of Computers,2024,41(7):1485−1520.
|
| [3] |
程健,李昊,马昆,等. 矿井视觉计算体系架构与关键技术[J]. 煤炭科学技术,2023,51(9):202−218. doi: 10.12438/cst.2023-0152
CHENG Jian,LI Hao,MA Kun,et al. Architecture and key technologies of coalmine unde rground vision computing[J]. Coal Science and Technology,2023,51(9):202−218. doi: 10.12438/cst.2023-0152
|
| [4] |
肖照林,孙彤鑫,张晶瑞,等. 油画相似度的表面光场特征点分布鉴别[J]. 中国图象图形学报,2023,28(10):3123−3135. doi: 10.11834/jig.220774
XIAO Zhaolin,SUN Tongxing,ZHANGJingrui. Feature point distribution of the surface light field-measuredoil painting similarity identification[J]. Journal of Image and Graphics,2023,28(10):3123−3135. doi: 10.11834/jig.220774
|
| [5] |
RUBLEE E,RABAUD V,KONOLIGE K,et al. ORB:An efficient alternative to SIFT or SURF[C]// 2011 International Conference on Computer Vision. IEEE,Barcelona,Spain,2011:2564-2571.
|
| [6] |
向浩鸣,夏晓华,葛兆凯,等. 仿人眼双目图像特征点提取与匹配方法[J]. 哈尔滨工业大学学报,2024,56(4):92−100. doi: 10.11918/202305044
XIANG Haoming,XIA Xiaohua,GE Zhaokai,et al. Feature point extraction and matching method of humanoid-eye binocular images[J]. Journal of Harbin Institute of Technology,2024,56(4):92−100. doi: 10.11918/202305044
|
| [7] |
YI K M,TRULLS E,LEPETIT V,et al. Lift:Learned invariant feature transform[C]// Computer Vision–ECCV 2016:14th European Conference,Amsterdam,The Netherlands,October 11-14,2016,Proceedings,Part VI 14. Springer International Publishing,2016:467−483.
|
| [8] |
DETONE D,MALISIEWICZ T,RABINOVICH A. Superpoint:Self-supervised interest point detection and description[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition workshops,Salt Lake City,UT,USA,2018:224−236.
|
| [9] |
TIAN Y,FAN B,WU F. L2-net:Deep learning of discriminative patch descriptor in euclidean space[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,HI,USA,2017:661−669.
|
| [10] |
DUSMANU M,ROCCO I,PAJDLA T,et al. D2-net:A trainable cnn for joint description and detection of local features[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Long Beach,CA,USA,2019:8092−8101.
|
| [11] |
REVAUD J,DE SOUZA C,HUMENBERGER M,et al. R2d2:Reliable and repeatable detector and descriptor[C]// Proceedings of the Advances in Neural Information Processing Systems. Vancouver,Canada,2019:12414−12424.
|
| [12] |
LUO Z,ZHOU L,BAI X,et al. Aslfeat:Learning local features of accurate shape and localization[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle,WA,USA 2020:6589−6598.
|
| [13] |
XU S,CHEN S,XU R,et al. Local Feature Matching Using Deep Learning:A Survey[J]. Information Fusion,2024,107:1566−2535.
|
| [14] |
CHENG J,YAN P P,WANG R B,et al. A Vanishing Point Detection Method Based on Imparoved Ransac In Coalmine Tussel[C]// 26th World Mining Congress 2023.
|
| [15] |
黄丹丹,郝文豪,杨 阳. 基于误匹配剔除和地面约束的视觉SLAM算法[J]. 中国惯性技术学报,2024,32(2):163−169.
HUANG Dandan,HAO Wenhao,YANG Yang. Visual SLAM algorithm based on mismatch rejection and ground constraints[J]. Journal of Chinese Inertial Technology,2024,32(2):163−169.
|
| [16] |
XU C,YANG H,YAN S,et al. Multi-source image matching using improved ORB algorithm[C]// Third International Conference on Computer Vision and Pattern Analysis (ICCPA 2023). SPIE,2023,12754:572−576.
|
| [17] |
BIAN J W,LIN W Y,MATSUSHITA Y,et al. Gms:Grid-based motion statistics for fast,ultra-robust feature correspondence[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,USA,2017:4181−4190.
|
| [18] |
毛清华,王孟寒,胡鑫,等. 改进SURF-FLANN的综采工作面视频拼接特征提取与匹配算法[J/OL]. 煤炭学报. https://doi.org/10.13225/j.cnki.jccs.2023.1560.
MAO Qinghua,WANG Menghan,HU Xing,et al. Improved SURF-FLANN feature extraction and matching algorithm for video stitching of fully mechanical mining face[J/OL]. Journal of China Coal Society. https://doi.org/10.13225/j.cnki.jccs.2023.1560.
|
| [19] |
张艳博,韩鑫,梁鹏,等. 目标追踪算法在岩石表面变形测量中的实现与应用[J]. 岩石力学与工程学报,2023,42(3):661−671.
ZHANG Yanbo,HAN Xin,LIANG Peng,et al. Realization and application of target tracking algorithm in rock surface deformation monitoring[J]. Chinese Journal of Rock Mechanics and Engineering,2023,42(3):661−671.
|
| [20] |
程健,闫鹏鹏,郁华森,等. 基于有向线段误匹配剔除的煤矿巷道复杂场景图像拼接方法[J]. 煤炭科学技术,2022,50(9):179−191.
CHENG Jian,YAN Pengpeng,YU Huasen,et al. Image stitching method for the complicated scene of coalmine tunnel based on mismatched elimination with directed line segments[J]. Coal Science and Technology,2022,50(9):179−191.
|
| [21] |
马宏伟,周文剑,王 鹏,等. 改进的ORB-FLANN煤矸石图像高效匹配方法[J]. 煤炭科学技术,2024,52(1):288−296.
MA Hongwei,ZHOU Wenjian,WANG Peng,et al. Improved ORB-FLANN efficient matching method for coal gangue image[J]. Coal Science and Technology,2024,52(1):288−296.
|
| [22] |
WEI C,WANG W,YANG W,et al. Deep retinex decomposition for low-light enhancement[A]. Proceedings of the 29rd British Machine Vision Conference (BMVC)[C]// Newcastle:BMVC,2018:155−176.
|
| [23] |
BALNTAS V,LENC K VEDALDI A,et al. HPatches:A benchmark and evaluation of handcrafted and learned local descriptors[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,USA,2017:3852−3861.
|
| [24] |
WANG W,DAI J,CHEN Z,et al. Internimage:Exploring large-scale vision foundation models with deformable convolutions[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,Canada,2023:14408−14419.
|
| [25] |
WANG X,LIU Z,HU Y,et al. Featurebooster:Boosting feature descriptors with a lightweight neural network[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Vancouver,Canada 2023:7630−7639.
|
| [26] |
SARLIN P E,DETONE D,MALISIEWICZ T,et al. Superglue:Learning feature matching with graph neural networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle,WA,USA,2020:4938−4947.
|
| [27] |
ZHAI S,TALBOTT W,SRIVASTAVA N,et al. An attention free transformer[J]. arxiv preprint arxiv:2105. 14103,2021.
|
| [28] |
SHEN T,LUO Z,ZHOU L,et al. Matchable image retrieval by learning from surface reconstruction[C]// Asian Conference on Computer Vision. Cham:Springer International Publishing,Salt Lake City,UT,USA ,2018:415−431.
|
| [29] |
FISHER A,CANNIZZARO R,COCHRANE M,et al. ColMap:A memory-efficient occupancy grid mapping framework[J]. Robotics and Autonomous Systems,2021,142:103755. doi: 10.1016/j.robot.2021.103755
|
| [30] |
MIKOLAJCZYK K,SCHMID C. A performance evaluation of local descriptors[J]. IEEE transactions on pattern analysis and machine intelligence,2005,27(10):1615−1630. doi: 10.1109/TPAMI.2005.188
|
| [31] |
刘海桥,刘 萌,龚子超,等. 基于深度学习的图像匹配方法综述[J]. 航空学报,2024,45(3):28796−28796.
LIU Haiqiao,LIU Meng,GONG Zichao,et al. A review of image matching methods based on deep learning[J]. Acta Aeronautica et Astronautica Sinica,2024,45(3):28796−28796.
|
| [32] |
NOH H,ARAUJO A,SIM J,et al. Large-scale image retrieval with attentive deep local features[C]// Proceedings of the IEEE International Conference on Computer Vision,Venice,Italy,2017:3456−3465.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: