| Citation: |
CHEN Wei，REN Peng，AN Wenni，et al. Mine object detection based on space attention in coal mine edge intelligent surveillance images[J]. Coal Science and Technology，2024，52（S2）：201−210. DOI: 10.12438/cst.2022-2140
|
Intelligence is an inevitable trend for the safe production and rapid development of coal mines, and it is an important direction for the development of coal mine intelligence to build the framework of intelligent production system, cloud-edge computing synergy system, and fast and accurate object detection for key targets in coal mines. However, the existing deep learning-based object detection algorithms are large in size and high in computational complexity, making it difficult to deploy them to edge devices to provide low-latency data analysis services. The article proposes a coal mine focused object detection method based on the YOLO-v4L-EA algorithm, which deploys the lightweight YOLO-v4 incorporating the spatial attention mechanism to the edge computing devices of the mine IoT system, so as to provide users with focused object sensing services with high response speed. At the algorithmic level, to address the problem of uneven brightness and other defects in coal mine underground images that affect the effect of object detection, the article designs the pixel regularized spatial attention structure (PNSAM), which implements the spatial attention mechanism with batch regularization, which can assist the target detection model to strengthen the attention to important features and help the algorithm to perceive the task target from low-quality images; inspired by the MobileNet base inspired by the MobileNet base structure, the YOLO-v4 backbone network is lightweight and improved based on the depth-separable convolution, so that the overall model can be deployed in the mine edge computing devices; in order to reduce the gradient loss caused by the h-swish activation function, this paper tries to use the Mish activation function in the model, which achieves efficient deep feature extraction by virtue of its gradient-smoothing property. A mine object detection dataset is constructed based on coal mine video surveillance data for evaluating the practical application performance of the object detection network model. The article uses the NVIDIA Jetson TX2 edge computing platform as the experimental hardware equipment, and the comparison experiments show that the experimental results of the model on the public test dataset VOC2012 improve the mAP value by 13.39% relative to the YOLO-v4-Tiny model, which proves the validity and correctness of the algorithm; the mAP value of the model on the mine object detection dataset is 88.9%, indicating that the method can effectively realize the key object detection in coal mine underground.
| [1] |
李梅,杨帅伟,孙振明,等. 智慧矿山框架与发展前景研究[J]. 煤炭科学技术,2017,45(1):121−128,34.
LI Mei,YANG Shuaiwei,SUN Zhenming,et al. Research on framework and development prospect of intelligent mine[J]. Coal Science and Technology,2017,45(1):121−128,34.
|
| [2] |
吕鹏飞,何敏,陈晓晶,等. 智慧矿山发展与展望[J]. 工矿自动化,2018,44(9):84−88.
LYU Pengfei,He Min,Chen Xiaojing,et al. Development and prospect of intelligent mine[J]. Industry and Mine Automation,2018,44(9):84−88.
|
| [3] |
陈浮,于昊辰,卞正富,等. 碳中和愿景下煤炭行业发展的危机与应对[J]. 煤炭学报,2021,46(6):1808−1820.
ChEN Fu,YU Haochen,BIAN Zhengfu,et al. Development crisis and countermeasures of coal industry under the vision of carbon neutrality[J]. Journal of China Coal Society,2021,46(6):1808−1820.
|
| [4] |
丁恩杰,赵志凯. 煤矿物联网研究现状及发展趋势[J]. 工矿自动化,2015,41(5):1−5.
DING Enjie,ZHAO Zhikai. Research status and development trend of internet of things in coal mine[J]. Industry and Mine Automation,2015,41(5):1−5.
|
| [5] |
SADOOGHI I,MARTIN J H,Li T,et al. Understanding the performance and potential of cloud computing for scientific applications[J]. IEEE Transactions on Cloud Computing,2017,5(2):358−371. doi: 10.1109/TCC.2015.2404821
|
| [6] |
王国法,王虹,任怀伟,等. 智慧煤矿2025情景目标和发展路径[J]. 煤炭学报,2018,43(2):295−305.
WANG Guofa,WANG Hong,REN Huaiwei,et al. Scenario goal and development path of smart Coal Mine 2025[J]. Journal of China Coal Society,2018,43(2):295−305.
|
| [7] |
ZOU Z,SHI Z,GUO Y,et al. Object detection in 20 Years:A Survey[J]. Proceedings of the IEEE,2023,111(3):257−76.
|
| [8] |
GIRSHICK R,DONAHUE J,DARRELL T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation [C]. IEEE Computer Vision and Pattern Recognition,IEEE,2014:580−587.
|
| [9] |
刘晓阳,霍祎炜. 多尺度分区统一化LBP算子井下人员人脸识别方法[J]. 煤炭科学技术,2019,47(12):116−123.
LIU Xiaoyang,HUO Yiwei. Multi-scale partition unified LBP operator for downhole face recognition method[J]. Coal Science and Technology,2019,47(12):116−123.
|
| [10] |
温廷新,王贵通,孔祥博,等. 基于迁移学习与残差网络的矿工不安全行为识别[J]. 中国安全科学学报,2020,30(3):41−46.
WEN Tingxin,WANG Guitong,KONG Xiangbo,et al. Mining unsafe behavior identification based on transfer learning and residual network[J]. China Safety Science Journal,2020,30(3):41−46.
|
| [11] |
陈伟,任鹏,田子建,等. 基于注意力机制的无监督矿井人员跟踪[J]. 煤炭学报,2021,46(S1):601−608.
CHEN Wei,REN Peng,TIAN Zijian,et al. Unsupervised mine personnel tracking based on attentional mechanism[J]. Journal of China Coal Society,2021,46(S1):601−608.
|
| [12] |
魏力,云霄,程小舟,等. 井下复杂环境人员重识别研究[J]. 工矿自动化,2021,47(6):63−70.
WEI Li,YUN Xiao,CHENG Xiaozhou,et al. Re-identification of downhole personnel in complex environment[J]. Industry and Mine Automation,2021,47(6):63−70.
|
| [13] |
杨潇,陈伟,任鹏,等. 基于域适应的煤矿环境监控图像语义分割[J]. 煤炭学报,2021,46(10):3386−3396.
YANG Xiao,CHEN WEI,REN Peng,et al. Semantic segmentation of coal mine environmental monitoring image based on domain Adaptation[J]. Journal of China Coal Society,2021,46(10):3386−3396.
|
| [14] |
董锴文,孙彦景,陈岩,等. 矿山生产作业场景视频结构化分析关键技术[J]. 煤炭学报,2021,46(11):3724−3735.
DONG Kaiwen,SUN Yanjing,CHEN Yan,et al. Key technology of structured video analysis of mine production scene[J]. Journal of China Coal Society,2021,46(11):3724−3735.
|
| [15] |
LIU W,ANGUELOV D,ERHAN D,et al. Ssd:Single shot multibox detector [C]. European Conference on Computer Vision,2016:21-37.
|
| [16] |
REN S,HE K,GIRSHICK R,et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137−1149. doi: 10.1109/TPAMI.2016.2577031
|
| [17] |
REDMON J,FARHADI A. YOLOv3:An incremental improvement [J/OL]. arXiv:1804.02767,2018. DOI: 10.48550/arXiv.1804.02767.
|
| [18] |
李伟山,卫晨,王琳. 改进的Faster RCNN煤矿井下行人检测算法[J]. 计算机工程与应用,2019,55(4):200−207.
LI Weishan,WEI Chen,WANG Lin. Improved faster RCNN algorithm for pedestrian detection in coal mine[J]. Computer Engineering and Applications,2019,55(4):200−207.
|
| [19] |
谢斌红,袁帅,龚大立. 基于RDB-YOLOv4的煤矿井下有遮挡行人检测[J]. 计算机工程与应用,2022,58(5):200−207.
XIE Binhong,YUAN Shuai,GONG Dali. Detection of concealed pedestrians in coal mine based on RDB-YOLOV4[J]. Computer Engineering and Applications,2022,58(5):200−207.
|
| [20] |
BOCHKOVSKIY A,WANG C-Y,LIAO H-Y M. YOLOv4:Optimal speed and accuracy of object detection [J/OL]. arXiv:2004.10934,2020. DOI: 10.48550/arXiv.2004.10934.
|
| [21] |
SHI W,CAO J,ZHANG Q,et al. Edge computing:Vision and challenges[J]. IEEE Internet of Things Journal,2016,3(5):637−646. doi: 10.1109/JIOT.2016.2579198
|
| [22] |
HOWARD A,SANDLER M,CHU G,et al. Searching for mobilenetv3 [C]. IEEE/CVF International Conference on Computer Vision,2019:1314−1324.
|
| [23] |
HE K M,ZHANG X Y,REN S Q,et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904−1916. doi: 10.1109/TPAMI.2015.2389824
|
| [24] |
HOWARD A G,ZHU M,CHEN B,et al. Mobilenets:Efficient convolutional neural networks for mobile vision applications [J/OL]. arXiv:1704. 04861,2017. DOI: 10.48550/arXiv.1704.04861.
|
| [25] |
SANDLER M,HOWARD A,ZHU M,et al. Mobilenetv2:Inverted residuals and linear bottlenecks [C]. IEEE Conference on Computer Vision and Pattern Recognition,2018:4510−4520.
|
| [26] |
HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition [C]. IEEE Conference on Computer Vision and Pattern Recognition. IEEE,2016:770−778.
|
| [27] |
HU J,SHEN L,SUN G. Squeeze-and-excitation networks [C]. IEEE Conference on Computer Vision and Pattern Recognition,2018:7132−7141.
|
| [28] |
WOO S,PARK J,LEE J Y,et al. Cbam:Convolutional block attention module [C]. European Conference on Computer Vision,2018:3−19.
|
| [29] |
LIU Y,SHAO Z,TENG Y,et al. NAM:Normalization-based Attention Module [J/OL]. arXiv:2111.12419,2021. DOI:10.48550/arXiv. 2111.12419.
|
| [30] |
IOFFE S,SZEGEDY C. Batch normalization:Accelerating deep network training by reducing internal covariate shift [C]. International Conference on Machine Learning. JMLR,2015:448−456.
|
| [31] |
EVERINGHAM M,ESLAMI S,VAN GOOL L,et al. The pascal visual object classes challenge:A retrospective[J]. International Journal of Computer Vision,2015,111(1):98−136. doi: 10.1007/s11263-014-0733-5
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: