| Citation: |
FAN Weiqiang，WANG Xuejin，ZHANG Yinghui，et al. Underground personnel detection and tracking using improved YOLOv7 and DeepSORT[J]. Coal Science and Technology，2024，52（S2）：343−355. DOI: 10.12438/cst.2023-1412
|
The coal industry is undergoing a transformation in the concept of intelligent mining with "safety, efficiency, intelligence, and green" as its core. Computer vision, as an emerging technology with high efficiency, intelligence, and low cost, has become an important highlight in the current construction of intelligent mines. The underground monitoring video is susceptible to interference factors such as artificial light source, dust and spray, which leads to the poor real-time performance, high missed & false detection rates and poor tracking accuracy of existing underground personnel monitoring methods using computer vision. Based on this, an improved YOLOv7 and DeepSORT underground personnel detection and tracking algorithm is proposed. First, in order to be able to extract more critical underground personnel image features and improve the model's adaptability in the complex scene of coal mine underground, the SimAM attention mechanism is incorporated into the Neck module of YOLOv7, and the improved YOLOv7 model is used to detect underground personnel targets. After that, in order to be able to further improve the tracking accuracy of personnel targets while reducing the number of model parameters and network complexity, the ShuffleNetV2 lightweight module is introduced into the feature extraction network of DeepSORT, and the improved DeepSORT model is used to encode and track downhole personnel targets. Finally, the experimental validation of the proposed algorithm is carried out on the established downhole personnel video image dataset and the public dataset, and the results show that: the average detection accuracy of the improved YOLOv7 model is improved by 3.9% compared with that of the YOLOv7 model; the multi-target tracking accuracy of the improved DeepSORT algorithm reaches 74.9%, the tracking accuracy reaches 82.3%, and the speed reaches 24 FPS. Compared with YOLOv7-DeepSORT algorithm, the number of network parameters of the improved algorithm is reduced by 36%, significantly improving the real-time performance of underground multi personnel target detection and tracking, and is expected to be deployed in the underground intelligent edge computing monitoring platform.
| [1] |
王秦生. 矿用安全头盔信息感知技术研究[D]. 西安:西安科技大学,2020.
WANG Qinsheng. Research on information perception technology for mine safety helmets[D]. Xi'an:Xi'an University of Science and Technology,2020.
|
| [2] |
孙继平. 智能矿山信息综合承载网与网络切片路由器[J]. 智能矿山,2023,4(1):14−17.
SUN Jiping. Intelligent mine information comprehensive carrying network and network slice router[J]. Intelligent Mining,2023,4(1):14−17.
|
| [3] |
李晓宇,陈伟,杨维,等. 基于超像素特征与SVM分类的人员安全帽分割方法[J]. 煤炭学报,2021,46(6):2009−2022.
LI Xiaoyu,CHEN Wei,YANG Wei,et al. Segmentation method for personnel safety helmet based on super-pixel features and SVM classification[J]. Journal of China Coal Society,2021,46(6):2009−2022.
|
| [4] |
CIAPARRONE G,SANCHEZ F L,TABIK S,et al. Deep learning in video multi-object tracking:a survey[J]. Neurocomputing,2020,381:61−88. doi: 10.1016/j.neucom.2019.11.023
|
| [5] |
刘艺,李蒙蒙,郑奇斌,等. 视频目标跟踪算法综述[J]. 计算机科学与探索,2022,16(7):1504−1515.
LIU Yi,LI Mengmeng,ZHENG Qibin,et al. Survey on video object tracking algorithms[J]. Computer Science and Exploration,2022,16(7):1504−1515.
|
| [6] |
孙继平,范伟强. MS-ADoG域结合ReNLU与VGG-16的矿井双波段图像融合算法[J]. 光子学报,2022,51(3):1−15.
SUN Jiping,FAN Weiqiang. Mine Dual-Band image fusion in MS-ADoG domain combined with ReNLU and VGG-16[J]. Journal of Photonics,2022,51(3):1−15.
|
| [7] |
孙继平,贾倪. 矿井视频图像中人员目标匹配与跟踪方法[J]. 中国矿业大学学报,2015,44(3):540−548.
SUN Jiping,JIA Ni. Human target matching and tracking method in coal mine video[J]. Journal of China University of Mining & Technology,2015,44(3):540−548.
|
| [8] |
陈伟,任鹏,田子建,等. 基于注意力机制的无监督矿井人员跟踪[J]. 煤炭学报,2021,46(S1):601−608.
CHEN Wei,REN Peng,TIAN Zijian,et al. Unsupervised mine personnel tracking based on attention mechanism[J]. Journal of China Coal Society,2021,46(S1):601−608.
|
| [9] |
LI X Y,WANG S,CHEN W,et al. An intelligent vision-based tracking method for underground human using infrared videos[J]. Symmetry,2022,14(8):1745. doi: 10.3390/sym14081745
|
| [10] |
张玉涛,张梦凡,史学强,等. 基于深度学习的井下运动目标跟踪算法研究[J]. 煤炭工程,2022,54(10):151−155.
ZHANG Yutao,ZHANG Mengfan,SHI Xueqiang,et al. Object tracking algorithm of moving objects in underground mine based on deep learning[J]. Coal Engineering,2022,54(10):151−155.
|
| [11] |
邵小强,李鑫,杨涛,等. 改进YOLOv5s和DeepSORT的井下人员检测及跟踪算法[J]. 煤炭科学技术,2023,51(10):291−301.
SHAO XiaoQiang,LI Xin,YANG Tao,et al. Underground personnel detection and tracking based on improved YOLOv5s and DeepSORT[J]. Coal Science and Technology,2023,51(10):291−301.
|
| [12] |
杨艺,徐建,郭瑜,等. 基于注意力机制的双重调制煤矿井下目标跟踪[J]. 安全与环境学报,2023,23(9):3277−3285.
YANG Yi,XU Jian,GUO Yu,et al. Research on dual modulation object tracking algorithm based on attention mechanism in coal mines[J]. Journal of Safety and Environment,2023,23(9):3277−3285.
|
| [13] |
WANG C Y,BOCHKOVSKIY A,LIAO H. YOLOv7:trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. (2022−07−02)[2022−10−26]. https://arxiv.org/pdf/2207.02696vl.pdf.
|
| [14] |
戚玲珑,高建瓴. 基于改进YOLOv7的小目标检测[J]. 计算机工程,2023,49(1):41−48.
QI Linlong,GAO Jianling. Small Object detection based on improved YOLOv7[J]. Computer Engineering,2023,49(1):41−48.
|
| [15] |
BOCHKOVSKIY A,WANG C Y,LIAO H,et al. YOLOv4:optimal speed and accuracy of object detection[J]. ArXiv:2004.10934,2020. https://doi.org/10.48550/arXiv.2004.10934
|
| [16] |
ZHANG X,ZENG H,GUO S,et al. Efficient long-range attention network for image super-resolution[C]//Computer Vision-ECCV 2022:17th European Conference,Switzerland,2022:649-667.
|
| [17] |
DING X,ZHANG X,MA N,et al. RepVGG:Making VGG-style ConvNet Again[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Rec(CVPR). Nashville,TN,USA:IEEE,2021:13728−13737.
|
| [18] |
WOJKE N,BEWLEY A,PAULUS D. Simple online and realtime tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing (ICIP),Beijing,China ,2017:3645−3649.
|
| [19] |
BEWLEY,GE Z,OTT L,et al. Simple online and realtime tracking[C] //2016 IEEE International Conference on Image Processing (ICIP),Phoenix,AZ,USA:IEEE,2016:3464-3468.
|
| [20] |
刘翌祥. 卡尔曼滤波在目标跟踪中的应用研究[J]. 电子世界,2021,43(18):61−62.
LIU Yixiang. Research on the Application of Kalman Filter in Target Tracking[J]. Electronics World,2021,43(18):61−62.
|
| [21] |
范伟强,刘毅. 基于自适应小波变换的煤矿降质图像模糊增强算法[J]. 煤炭学报,2020,45(12):4248−4260.
FAN Weiqiang,LIU Yi. Fuzzy enhancement algorithm of coal mine degradation image based on adaptive wavelet transform[J]. Journal of China Coal Society,2020,45(12):4248−4260.
|
| [22] |
YANG L X,ZHANG R Y,LI L,et al. SimAM:a simple,parameter-free attention module for convolutional neural networks[C] //Proceedings of the 38th International Conference on Machine Learning. NewYork:PMLR,2021:11863.
|
| [23] |
MA N N,ZHANG X Y,ZHENG H T,et al. ShuffleNetV2:Practical guidelines for efficient CNN architecture design[C] //Proceedings of the European conference on computer vision (ECCV). Munich,Germany:Springer,2018:116−131.
|
| [24] |
ZHANG X Y,ZHOU X Y,LIN M X,et al. ShuffleNet:an extremely efficient convolutional neural network for mobile devices[C] //Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE,2018. 6848−6856.
|
| [25] |
胡杰,熊宗权,徐文才,等. 基于轻量化卷积神经网络的车道线检测[J]. 激光与光电子学进展,2022,59(10):352−360.
HU Jie,XIONG Zongquan,XU Wencai,et al. Lane detection based on a lightweight convolutional neural network[J]. Laser & Optoelectronics Progress,2022,59(10):352−360.
|
| [26] |
LI W,ZHAO R,XIAO T,et al. Deepreid:deep filter pairing neural network for person re-identification[C]//Institute of Electrical and Electronics Engineers. IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos:IEEE,2014:152−159.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: