Advance Search
TIAN Zijian,YANG Kang,WU Jiaqi,et al. LMIENet enhanced object detection method for low light environment inunderground mines[J]. Coal Science and Technology,2024,52(5):222−235. DOI: 10.12438/cst.2023-0675
Citation: TIAN Zijian,YANG Kang,WU Jiaqi,et al. LMIENet enhanced object detection method for low light environment inunderground mines[J]. Coal Science and Technology,2024,52(5):222−235. DOI: 10.12438/cst.2023-0675

LMIENet enhanced object detection method for low light environment inunderground mines

Funds: 

National Natural Science Foundation of China (52074305, 52274160, 51874300)

More Information
  • Received Date: May 19, 2023
  • Accepted Date: May 19, 2023
  • Available Online: April 25, 2024
  • The underground working environment of coal mines is complex, with unfavorable factors such as low brightness of artificial light sources, high dust content, and high water vapor density. This leads to difficulties in extracting features and low accuracy of object recognition and positioning when existing object detection algorithms are applied to coal mines. An object detection algorithm for low illumination environments in coal mines is proposed, which consists of an low-light mine image enhancement module LMIENet and a object detection module. The image enhancement module is used to improve the image quality of the original image, restore various image information, and then use a target detection network to perform specific target detection on the enhanced image, effectively improving the accuracy of detection. In the image enhancement module, a lightweight enhancement parameter prediction network is designed with reference to the zero reference depth curve estimation algorithm, and the pixel level enhancement parameter matrix is calculated for image quality enhancement and brightness adjustment of low light images. The network implicitly measures the image enhancement effect through the designed non-reference loss function, and guides the network to conduct unsupervised learning, Enable the network to adaptively enhance the image quality of the original image without relying on paired datasets. In the object detection module, the YOLO v8n object detection model is adopted, which has a lightweight model size and high flexibility to avoid excessive overall model complexity; Using Focal EIoU Loss to improve regression loss, accelerate model convergence, and improve model detection accuracy. The experimental results show that compared with classic object detection algorithms such as Faster R–CNN, SSD, RetinaNet, etc., the proposed algorithm performs well on the self-made coal mine object detection dataset, and is effective in object detection in low light environments mAP@0.5 reaches 98.0%, mAP@0.5∶0.95 reaches 64.8%, and the running time of a single frame image in the experimental environment is only 11 ms, which is superior to other comparison methods. It is proven that the proposed algorithm can effectively achieve object detection in low illumination and complex environments in coal mines, with short time consumption and high computational efficiency.

  • [1]
    智 宁,毛善君,李 梅. 基于照度调整的矿井非均匀照度视频图像增强算法[J]. 煤炭学报,2017,42(8):2190−2197.

    ZHI Ning,MAO Shanjun,LI Mei. Video image enhancement al-gorithm of mine non-uniform illumination based on illumination adjustment[J]. Journal of China Coal Society,2017,42(8):2190−2197.
    [2]
    陈德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349−365.

    CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349−365.
    [3]
    KRIZHEVSKY A,SUTSKEVER I,GEOFFREY E H. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems,2012,25:1097−1105.
    [4]
    GIRSHICK R,DONAHUE J,DARRELL T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2014:580–587.
    [5]
    REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:Unified,real-time object detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:779–788.
    [6]
    LIU W,ANGUELOV D,ERHAN D,et al. Ssd:Single shot multibox detector[C]. European Conference on Computer Vision,Springer,2016:21–37.
    [7]
    LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[C]. Proceedings of the IEEE International Conference on Computer Vision,2017:2980–2988.
    [8]
    IGOR M,CHEN Y,LIN Y,et al. NOD:Taking a closer look at detection under extreme low-light conditions with night object detection dataset[C]. BMVC,2021.
    [9]
    龚 云,颉昕宇. 基于同态滤波方法的煤矿井下图像增强技术研究[J]. 煤炭科学技术,2023,51(3):241−250.

    GONG Yun,XIE Xinyu. Research on coal mine underground image recognition technology based on homomorphic filtering method[J]. Coal Science and Technology,2023,51(3):241−250.
    [10]
    苏 波,李 超,王 莉. 基于多权重融合策略的Retinex矿井图像增强算法[J]. 煤炭学报,2023,48,(S2):813−822.

    SU Bo,LI Chao,WANG Li. Mine image enhancement algorithm based on retinex using multi-weight fusion strategy[J]. Journal of China Coal Society,2023,48,(S2):813−822.
    [11]
    李 曼,杨茂林,刘长岳,等. 基于图像的煤矸分选中图像照度调节方法[J]. 煤炭学报,2021,46(S2):1149−1158.

    LI Man,YANG Maolin,LIU Changyue,et al. Illuminance adjustment method for image-based coal and gangue separation[J]. Journal of China Coal Society,2021,46(S2):1149−1158.
    [12]
    乔佳伟,贾运红. Retinex算法在煤矿井下图像增强的应用研究[J]. 煤炭技术,2022,41(3):193−195.

    QIAO Jiawei,JIA Yunhong. Research on Application of Retinex Algorithm in Image Enhancement in Coal Mine[J]. Coal Technology,2022,41(3):193−195.
    [13]
    WANG R,ZHANG Q,FU C,et al. Underexposed photo enhancement using deep illumination estimation[C]. IEEE Conference on Computer Vision and Pattern Recognition,2019:6849–6857.
    [14]
    KIM H,CHOI S,KIM C,KOH Y J. Representative Color Transform for Image Enhancement[C]. The IEEE/CVF International Conference on Computer Vision,2021.
    [15]
    ZHAO L,LU S,CHEN T,et al. Deep symmetric network for underexposed image enhancement with recurrent attentional learning[C]. The IEEE/CVF International Conference on Computer Vision,2021.
    [16]
    田子建,吴佳奇,张文琪,等. 基于Transformer和自适应特征融合的矿井低照度图像亮度提升和细节增强方法[J]. 煤炭科学技术,2024,52(1):29−310.

    TIAN Zijian,WU Jiaqi,ZHANG Wenqi,et al. An illuminance improvement and details enhancement method on coal mine low-light images based on Transformer and adaptive feature fusion[J]. Coal Science and Technology,2024,52(1):297−310.
    [17]
    KIM I S,JEONG Y,KIM S H,et al. Deep Learning based Effective surveillance system for low-illumination environments[C]. International Conference on Ubiquitous and Future Networks,2019.
    [18]
    SASAGAWA Y,NAGAHARA H. YOLO in the Dark – Domain adaptation method for merging multiple models[A]// European Conference on Computer Vision,2020.
    [19]
    QU Y,OU Y,XIONG R. Low illumination enhancement for object detection in self-driving[A]// IEEE International Conference on Robotics and Biomimetics,2019.
    [20]
    WANG W,PENG Y,CAO G,et al. Low-illumination image enhancement for night-time UAV pedestrian detection[J]. IEEE Transactions on Industrial Informatics,2021,17(8):5208−5217. doi: 10.1109/TII.2020.3026036
    [21]
    南柄飞,郭志杰,王 凯,等. 基于视觉显著性的煤矿井下关键目标对象实时感知研究[J]. 煤炭科学技术,2022,50(8):247–258.

    NAN Bingfei,GUO Zhijie,WANG Kai,et al. Study on real-time perception of target ROI in underground coal mines based on visual saliency[J]. Coal Science and Technology,2022,50(8):247–258.
    [22]
    杨 艺,付泽峰,高有进,等. 基于深度神经网络的综采工作面视频目标检测[J]. 工矿自动化,2022,48(8):33−42.

    YANG Yi,FU Zefeng,GAO Youjin,et al. Video object detection of the fully mechanized working face based on deep neural network[J]. Journal of Mine Automation,2022,48(8):33−42.
    [23]
    LI C,GUO C,GUO J,et al. Learning to enhance low-light image via zero-reference deep curve estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(8):4225−4238.
    [24]
    ZHANG Y,REN W,ZHANG Z,et al. Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing,2022,506:146−157.
    [25]
    LORE K G,AKINTAYO A,SARKAR S. L lnet:A deep autoencoderapproach to natural low-light image enhancement[J]. Pattern Recognition,2017,61:650−662. doi: 10.1016/j.patcog.2016.06.008
    [26]
    REN S,HE K,GIRSHICK R,et al. Faster R–CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137−1149. doi: 10.1109/TPAMI.2016.2577031
    [27]
    TIAN Z,SHEN C,CHEN H,et al. FCOS:Fully convolutional one-stage object detection[A]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV). IEEE,2020.
    [28]
    ZHOU X,WANG D,P Krhenbühl. Objects as points[EB/OL]. 2019,arXiv preprint arXiv:1904.07850.
    [29]
    REDMON J,FARHADI A. YOLOv3:an incremental improvement[EB/OL]. 2018,arXiv preprint arXiv:1804.02767.
    [30]
    LYU F,LU F,WU J,et al. MBLLEN:Low-light Image/Video Enhancement Using CNNs[C]. British Machine Vision Conference,2018.
    [31]
    GUO X,LI Y,LING H. Lime:Low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing,2016,26(2):982−993.
    [32]
    WEI C,WANG W,YANG W,et al. Deep retinex decomposition for low-light enhancement[EB/OL]. 2018,arXiv preprint arXiv:1808.04560.
    [33]
    MA L,MA T,LIU R,et al. TOWARD Fast,Flexible,and Robust Low-Light Image Enhancement[A]// IEEE Conference on Computer Vision and Pattern Recognition,2022.
  • Related Articles

    [1]CHEN Peng, CHEN Xuexi, LIU Yongjie, YANG Tao. Gas pressure affected to coal resistivity law and mechanism[J]. COAL SCIENCE AND TECHNOLOGY, 2018, (7).
    [2]Xu Jiang Tang Xupei Zhang Chaolin Peng Shoujian Feng Dan Liu Longrong, . Experiment study on physical simulation of gas drainage effect under the condition of different gas pressure[J]. COAL SCIENCE AND TECHNOLOGY, 2017, (5).
    [3]Zhang Yuzhu Lyu Guichun Liu Huihui, . Study on gas content critical value of outburst predicted index in outburst seam region[J]. COAL SCIENCE AND TECHNOLOGY, 2016, (11).
    [4]Guo Keju Gao Kun, . Gas pressure growth curve method applied to measurement of seam gas pressure[J]. COAL SCIENCE AND TECHNOLOGY, 2016, (11).
    [5]Wang Chaojie Jiang Chenglin Yang Dingding Yang Kai, . Analysis on borehole parameters affected to gas pressure measuring time[J]. COAL SCIENCE AND TECHNOLOGY, 2016, (7).
    [6]MOU Quan-bin. Study Status and Prospects for Regional Prediction Methods of Coal and Gas Outburst in China[J]. COAL SCIENCE AND TECHNOLOGY, 2014, (11).
    [7]HAO Nai TU Hui MAO Ling-tao LI Min, . Study on Experiment of Coal and Rock Adsorbed Gas Speciality Under Axial Load Role[J]. COAL SCIENCE AND TECHNOLOGY, 2014, (5).
    [8]Correction on Pressure Measured Results of Watery Seam Gas Based on Pressure Recovered Curve[J]. COAL SCIENCE AND TECHNOLOGY, 2013, (7).
    [9]The Critical Gas Pressure of Coal and Gas Outburst with Different Coal Structure Combination[J]. COAL SCIENCE AND TECHNOLOGY, 2012, (11).
    [10]Regional Prediction of Coal and Gas Outburst from Seam in Erdaoling Mining Area[J]. COAL SCIENCE AND TECHNOLOGY, 2011, (10).

Catalog

    Article views (273) PDF downloads (115) Cited by()
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return