Coal gangue image recognition model based on CSPNet-YOLOv7 target detection algorithm
-
摘要:
煤矸识别技术是矿井智能化建设的关键技术之一，针对工作面低照度高粉尘环境造成的煤矸识别模型精度不高以及小目标煤矸难以识别的问题，提出一种基于CSPNet-YOLOv7目标检测算法的煤矸图像识别模型。采用跨阶段部分网络（Cross Stage Partial Network，CSPNet）改进YOLOv7模型的主干特征提取网络，优化梯度信息减少网络参数，同时采用递归特征金字塔（Recursive Feature Pyramid，RFP）和可切换卷积（Switchable Auto Convolution，SAC）替换颈部特征提取网络中简单的上下采样和普通卷积模块，并采用3次迁移训练进行不同宽度和深度的特征学习，增强网络的泛化能力。试验结果表明，CSPNet-YOLOv7模型的平均精度均值为97.53%，准确率为92.24%，召回率为97.91%，F1得分为0.95，模型的参数量为30.85×106，浮点运算次数为42.15×109，每秒传输帧数为24.37 f/s，与YOLOv7模型相比，平均精度均值提高了7.46%，参数量和浮点运算次数分别降低了17.23%和60.41%，相较于FasterRCNN-Resnet50、YOLOv3、YOLOv4、MobileNet V2 -YOLOv4、YOLOv4-VGG、YOLOv5s模型、CSPNet-YOLOv7模型对煤矸识别的平均精度均值最高，同时参数量和浮点运算次数较小，在识别精度和速度之间有着较好的平衡。最后，通过井下现场试验验证了CSPNet-YOLOv7模型，为煤矸精准识别提供了有效技术手段。
Abstract:The gangue recognition technology is one of the key technologies in the intelligent construction of mines. To address the problem of low accuracy of the gangue recognition model caused by low illumination and high dust environment at the working face and the difficulty of recognizing small target gangue, a coal gangue image recognition model based on CSPNet-YOLOv7 target detection algorithm is proposed. Cross Stage Partial Network (CSPNet) is used to improve the backbone feature extraction network of YOLOv7 model, optimize the gradient information to reduce the network parameters, while Recursive Feature Pyramid (RFP) and Switchable Auto Convolution (SAC) to replace the simple up and down sampling and normal convolution modules in the neck feature extraction network, and to enhance the generalization ability of the network by using three migration training for feature learning of different widths and depths. The experimental results show that the CSPNet-YOLOv7 model has an average accuracy mean of 97.53%, an accuracy rate of 92.24%, a recall rate of 97.91%, an F1 score of 0.95, a model parametric number of 30.85×106, a floating point operation count of 42.15×109, and a frame rate of 24.37 f/s transmitted per second, Compared to the YOLOv7 model, the average mean accuracy is improved by 7.46%, and the number of parameters and floating point operations are reduced by 17.23% and 60.41%, respectively, compared to the FasterRCNN-Resnet50, YOLOv3, YOLOv4, MobileNet V2 -YOLOv4, YOLOv4-VGG, YOLOv5s models. The CSPNet-YOLOv7 model has the highest average accuracy mean for coal gangue identification, while the number of parameters and floating point operations is small, which has a good balance between identification accuracy and speed. Finally, the CSPNet-YOLOv7 model is validated through downhole field tests, providing an effective technical means for accurate coal gangue identification.
-
-
![]()
图 1 Cross stage partial network结构图
Figure 1. Structural drawings of cross stage partial network
![]()
图 3 基于CSPNet架构的YOLOv7主干网络改进
Figure 3. YOLOv7 backbone network improvement diagram based on CSPNet architecture
![]()
图 8 CSPNet-YOLOv7和YOLOv7模型煤矸识别评价指标变化曲线
Figure 8. CSPNet-YOLOv7 and YOLOv7 model coal and gangue identification evaluation index change curve
![]()
图 10 不同模型对比试验的平均精度均值曲线
Figure 10. mAP curves for different model comparison experiments
![]()
图 11 不同模型的煤矸识别结果
Figure 11. Coal and gangue identification results from different models
![]()
图 12 工作面皮带输送机上煤矸石图像识别结果
Figure 12. Image recognition results for coal and gangue on belt conveyor at working face
表 1 CSPNet-YOLOv7和YOLOv7模型煤矸识别结果对比
Table 1 Comparison of CSPNet-YOLOv7 and YOLOv7 algorithm coal and gangue identification results
识别类别 CSPNet-YOLOv7(Confidence=0.5) YOLOv7(Confidence=0.5) 准确率/% 召回率 /% F1得分 平均精度/% 平均精度
均值/%准确率/% 召回率/% F1得分 平均精度/% 平均精度
均值/%煤 90.69 97.57 0.94 96.93 97.53 89.23 90.27 0.90 92.47 90.07 矸石 93.79 98.25 0.96 98.13 86.87 87.15 0.87 87.66 平均值 92.24 97.91 0.95 97.53 88.05 88.71 0.88 90.07 
下载: 导出CSV
表 2 基于YOLO v7模型不同改进策略的消融试验
Table 2 Ablation experiments based on different improvement strategies of YOLO v7 model
组别 模型 平均精度/% 平均精度均值/% 每秒传输帧数/(f·s−1) 参数量/106 煤 矸石 C1 YOLOv7 92.47 87.66 90.07 22.92 37.27 C2 YOLOv7+数据增强 93.56 88.92 91.24 22.92 37.27 C3 YOLOv7+数据增强+RFP-SAC 95.19 90.23 92.71 22.08 38.55 C4 CSPNet-YOLOv7+数据增强 93.64 95.78 94.71 23.73 29.64 C5 CSPNet-YOLOv7+数据增强+RFP-SAC 96.93 98.13 97.53 22.74 30.85
下载: 导出CSV
表 3 不同煤矸识别模型性能对比
Table 3 Performance comparison of different coal and gangue identification models
模型 参数量/106 浮点运算次数/109 每秒传输帧数/(f·s−1) 平均精度均值/% YOLOv3 61.53 65.60 25.93 87.75 FasterRCNN-Resnet50 28.42 120.54 23.66 85.69 MobileNet V2 -YOLOv4 39.06 39.99 20.39 88.94 YOLOv4-VGG 23.94 112.37 25.67 89.37 YOLOv4 64.36 60.53 21.33 90.02 YOLOv5s 47.06 114.48 14.28 92.13 YOLOv7 37.27 106.47 15.49 90.07 CSPNet-YOLOv7(ours) 30.85 42.15 24.37 97.53
下载: 导出CSV
-
[1] 葛世荣,王世博,管增伦,等. 数字孪生−应对智能化综采工作面技术挑战[J]. 工矿自动化,2022,48(7):1−12. GE Shirong,WANG Shibo,GUAN Zenglun,et al. Digital twin:meeting the technical challenges of intelligent fully mechanized working face[J]. Journal of Mine Automation,2022,48(7):1−12.
[2] 王国法. 煤矿智能化最新技术进展与问题探讨[J]. 煤炭科学技术,2022,50(1):1−27. WANG Guofa. New technological progress of coal mine intelligence and its problems[J]. Coal Science and Technology,2022,50(1):1−27.
[3] 王国法,杜毅博,任怀伟,等. 智能化煤矿顶层设计研究与实践[J]. 煤炭学报,2020,45(6):1909−1924. WANG Guofa,DU Yibo,REN Huaiwei,et al. Top level design and practice of smart coal mines[J]. Journal of China Coal Society,2020,45(6):1909−1924.
[4] 崔亚仲,任艳艳,白明亮. 神东矿区煤炭智能化建设实践[J]. 煤炭科学技术,2022,50(S1):218−226. CUI Yazhong,REN Yanyan,BAI Mingliang. Practice of intelligent construction of shendong coal mine[J]. Coal Science and Technology,2022,50(S1):218−226.
[5] 王玉涛. 煤矸石固废无害化处置与资源化综合利用现状与展望[J]. 煤田地质与勘探,2022,50(10):54−66. WANG Yutao. Status and prospect of harmless disposal and resource comprehensive utilization of solid waste of coal gangue[J]. Coal Geology & Exploration,2022,50(10):54−66.
[6] 李小萌,赵巧蓉. 基于图像处理的煤矸识别技术研究进展及方向[J]. 现代矿业,2021,37(11):200−203. LI Xiaomeng,ZHAO Qiaorong. Research progress and development trend of coal gangue identification technology based on image processing[J]. Modern Mining,2021,37(11):200−203.
[7] 田妍,田丰. 放顶煤开采过程煤矸识别技术发展现状及前景[J]. 煤炭工程,2018,50(10):142−145. TIAN Yan,TIAN Feng. Development status and prospect of coal gangue recognition technology in top-coal caving[J]. Coal Engineering,2018,50(10):142−145.
[8] 张强,张润鑫,刘峻铭,等. 煤矿智能化开采煤岩识别技术综述[J]. 煤炭科学技术,2022,50(2):1−26. ZHANG Qiang,ZHANG Runxin,LIU Junming,et al. Review on coal and rock identification technology for intelligent mining in coal mines[J]. Coal Science and Technology,2022,50(2):1−26.
[9] 王昕. 基于电磁波技术的煤岩识别方法研究[D]. 徐州:中国矿业大学,2017. WANG Xin. Study of coal rock identification method based on electromagnetic wave technology[D]. Xuzhou:China University of Mining and Technology,2017.
[10] 韦任. 基于煤岩反射光谱的识别技术研究[D]. 淮南:安徽理工大学,2021. WEI Ren. Research on recognition technology based on reflection spectrum of coal and rock[D]. Huainan:Anhui University of Science and Technology,2021.
[11] 李力,刘佳鹏,魏伟. 基于经验小波变换的煤岩界面识别超声回波处理方法研究[J]. 煤炭学报,2019,44(S1):370−377. LI Li,LIU Jiapeng,WEI Wei. Signal processing method on ultrasonic echo from coal-rock interface based on EWT[J]. Journal of China Coal Society,2019,44(S1):370−377.
[12] 王海舰. 煤岩界面多信息融合识别理论与实验研究[D]. 阜新:辽宁工程技术大学,2017. WANG Haijian. Theoretical and experimental study on coal-rock interface identification based on multi information fusion[D]. Fuxin:Liaoning Technical University,2017.
[13] 张勇,杨鹏,王亮,等. 基于经典卷积神经网络的智能煤矸识别[J]. 邵阳学院学报(自然科学版),2021,18(1):34−44. ZHANG Yong,YANG Peng,WANG Liang,et al. Intelligent coal gangue identification based on classical convolutional neural network[J]. Journal of Shaoyang University (Natural Science Edition),2021,18(1):34−44.
[14] WANG Mingxun,YANG Baolu,WANG Xin,et al. YOLO-T:Multitarget intelligent recognition method for X-ray images based on the YOLO and transformer Models[J]. Applied Sciences,2022,12(22):11848. doi: 10.3390/app122211848
[15] SAMOTHAI P,SANGUANSAT P,KHEAKSONG A,et al. The evaluation of bone fracture detection of YOLO series[A]. International Technical Conference on Circuits/Systems,Computers and Communications (ITC-CSCC)[C]. Phuket,Thailand,2022:1054-1057.
[16] 徐志强,吕子奇,王卫东,等. 煤矸智能分选的机器视觉识别方法与优化[J]. 煤炭学报,2020,45(6):2207−2216. XU Zhiqiang,LYU Ziqi,WANG Weidong,et al. Machine vision recognition method and optimization for intelligent separation of coal and gangue[J]. Journal of China Coal Society,2020,45(6):2207−2216.
[17] 雷世威,肖兴美,张明. 基于改进YOLOv3的煤矸识别方法研究[J]. 矿业安全与环保,2021,48(3):50−55. LEI Shiwei,XIAO Xingmei,ZHANG Ming. Research on coal and gangue identification method based on improved YOLOv3[J]. Mining Safety & Environmental Protection,2021,48(3):50−55.
[18] ADIBHATLA V A ,CHIH H C,HSU C C,et al. Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once[J]. Mathematical Biosciences and Engineering,2021,18(4):4411−4428.
[19] 张释如,黄综浏,张袁浩,等. 基于改进YOLOv5的煤矸识别研究[J]. 工矿自动化,2022,48(11):39−44. ZHANG Shiru,HUANG Zongliu,ZHANG Yuanhao,et al. Coal and gangue recognition research based on improved YOLOv5[J]. Journal of Mine Automation,2022,48(11):39−44.
[20] 宋卫虎,郎利影,张步勤,等. 基于改进U-net网络模型的煤矸识别方法[J]. 智能矿山,2021,2(3):63−68. SONG Weihu,LANG Liying,ZHANG Buqin,et al. A coal gangue identification method based on improved U-net network model[J]. Journal of Intelligent Mine,2021,2(3):63−68.
[21] 郭永存,张勇,李飞. 嵌入空洞卷积和批归一化模块的智能煤矸识别算法[J]. 矿业安全与环保,2022,49(3):45−50. GUO Yongcun,ZHANG Yong,LI Fei,et al. Intelligent coal and gangue identification algorithm embedded in dilated convolution and batch normalization module[J]. Mining Safety & Environmental Protection,2022,49(3):45−50.
[22] 雷孟宇,张旭辉,杨文娟,等. 煤矿掘进装备视觉位姿识别与控制研究现状与趋势[J]. 煤炭学报,2021,46(S2):1135−1148. LEI Mengyu,ZHANG Xuhui,YANG Wenjuan,et al. Current status and trend of research on visual pose detection and control of heading equipment in coal mines[J]. Journal of China Coal Society,2021,46(S2):1135−1148.
[23] 李晨鑫,张立亚. 煤矿井下网联式自动驾驶技术研究[J]. 工矿自动化,2022,48(6):49−55. LI Chenxin,ZHANG Liya. Research on the network connected automatic driving technology in underground coal mine[J]. Journal of Mine Automation,2022,48(6):49−55.
[24] WANG C Y,BOCHKOVSKIY A,LIAO H. YOLOv7:Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[A]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)[C],Vancouver,BC,Canada,2023:7464−7475.
[25] WANG C Y,LIAO H,WU Y H,et al. CSPNet:A new backbone that can enhance learning capability of CNN[A]. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)[C],Seattle,WA,USA,2020:1571−1580.
[26] QIAO S,CHEN L C,YUILLE A. DetectoRS:Detecting objects with recursive feature pyramid and switchable atrous convolution[A]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)[C],Nashville,TN,USA,2021:10208−10219.
[27] ZHOU Wujie,GUO Qinling,LEI Jingsheng,et al. IRFR-Net:Interactive recursive feature-reshaping network for detecting salient objects in RGB-D images[J]. IEEE Transactions on Neural Networks and Learning Systems,2021,11(3):2162−2388.
[28] LAI Weisheng,HUANG Jiabin,AHUJA Narendra,et al. Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,41(11):2599−2613. doi: 10.1109/TPAMI.2018.2865304
[29] PAN S J,QIANG Y. A Survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345−1359. doi: 10.1109/TKDE.2009.191
[30] SHIN H C,ROTH H R,GAO M,et al. Deep convolutional neural networks for computer-aided detection:CNN architectures,dataset characteristics and transfer learning[J]. IEEE Transactions on Medical Imaging,2016,35(5):1285−1298. doi: 10.1109/TMI.2016.2528162
计量
- 文章访问数: 119
- HTML全文浏览量: 9
- PDF下载量: 62
