YANG Zhifang,ZHANG Liya,HAO Bonan,et al. Conveyor belt deviation detection method based on dual flow network[J]. Coal Science and Technology,2023,51(S2):259−267
. DOI: 10.13199/j.cnki.cst.2023-0215Citation: |
YANG Zhifang,ZHANG Liya,HAO Bonan,et al. Conveyor belt deviation detection method based on dual flow network[J]. Coal Science and Technology,2023,51(S2):259−267 . DOI: 10.13199/j.cnki.cst.2023-0215 |
Among the traditional belt edge detection methods, the contact detection technology has high cost and the non-contact detection technology has low precision. With the development of artificial intelligence technology, although the method based on convolutional neural network can effectively improve the detection accuracy, but limited by the local operation characteristics of the convolutional operation itself, there are still problems such as insufficient perception of long-distance and global information, it is difficult to improve the accuracy of the belt edge detection. In order to solve the above problems, ① by combining the traditional convolutional neural network's ability to extract local features and the Transformer structure's ability to perceive global and long-distance information, a dual-flow transformer network (DFTNet) which integrates global and local information is proposed. The edge detection network model can better improve the belt edge detection accuracy and suppress the interference of belt image noise and background; ② By designing the CNN and Transformer feature fusion modules, a dual-flow encoder-decoder structure is formed. The clever design can better integrate the global context information, avoid the pre-training of the Transformer structure on large-scale data sets and be flexibly adjusted; ③ By Through the multi-scene conveyor belt pictures collected from the actual industrial scene, a belt conveyor belt dataset containing five different scenes, various angles and different positions is constructed. Through experimental verification, the DFTNet proposed in this paper has the best comprehensive performance with mIou 91.08%, ACC 99.48%, mPrecision 91.88% and mRecall 96.22%. which are 25.36%, 0.29%, 17.70% and 29.46% respectively compared to the pure convolutional neural network HRNet, and 29.5%, 0.32%, 24.77% and 34.13% respectively compared to FCN. At the same time, the frame rate of DFTNet processing images reaches 53.07 fps, which meets the real-time requirements in the industry and has great practical value.
[1] |
张 佳,尹君驰,王宏等. 输送带输煤采样技术研究现状及发展趋势[J]. 煤炭科学技术,2022,50(9):200−206.
ZHANG Jia,YIN Junchi,WANG Hong,et al. Research status and development trend of conveyor belt coal transport sampling technology[J]. Coal Science and Technology,2022,50(9):200−206.
|
[2] |
王海军, 王洪磊. 基于参数化对数图像处理模型的光照不均匀图像的边缘检测算法[J]. 煤炭科学技术,2022,50(12):225−239.
WANG Haijun, WANG Honglei. Status and prospect of intelligent key technologies of belt conveyorrStatus and prospect of intelligent key technologies of belt conveyorr[J]. Coal Science and Technology,2022,50(12):255−239.
|
[3] |
谭 恒,张红娟,靳宝全等. 基于机器视觉的煤矿带式输送机跑偏检测方法[J]. 煤炭技术,2021,40(5):152−156.
TAN Heng,ZHANG Hongjuan,JIN Baoquan,et al. Machine vision-based coal mines band-type conveyor running partial detection method[J]. Coal Technology,2021,40(5):152−156.
|
[4] |
徐 欢,李振璧,姜媛媛等. 基于OpenCV的输送带跑偏自动检测算法研究[J]. 工矿自动化,2014,40(9):48−52.
XU Huan,LI Zhenyu,JIANG Yuanyuan,et al. Research on automatic detection algorithm based on OpenCV-based conveyor belt running partial test algorithm[J]. Industry and Mining Automation,2014,40(9):48−52.
|
[5] |
韩涛,黄友锐,张立志等. 基于图像识别的带式输送机输煤量和跑偏检测方法[J]. 工矿自动化,2020,46(4):17−22.
HAN Tao,HUANG Yirui,ZHANG Lizhi,et al. The coal transmission volume and running test method based on image recognition [J]. Industry and Mining Automation,2014,40 (9):48−52.
|
[6] |
LIU Y,Wang Y,Zeng C,et al. Edge detection for conveyor belt based on the deep convolutional network[A]. Proceedings of 2018 Chinese Intelligent Systems Conference[C]. Springer,Singapore,2019:275−283.
|
[7] |
LONG J,SHELHAMER E,& DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE transactions on pattern analysis and machine intelligence,2016,39(4):474−482.
|
[8] |
CHEN L C,PAPANDREOU G,KOKKINOS I,et al. DeepLab:Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):834−848. doi: 10.1109/TPAMI.2017.2699184
|
[9] |
XIE S,TU Z. Holistically-nested edge detection[A]//Proceedings of the IEEE international conference on computer vision[C]. 2015:1395−1403.
|
[10] |
POMA X S,SAPPA A D. Improving edge detection in RGB images by adding NIR channel [A]//2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). IEEE,[C]. Las Palmas de Gran Canaria,Spain,2018:266−273.
|
[11] |
VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[A]//Advances in neural information processing systems[C]. Long Beach,CA,United states:MIT,2017:5998.
|
[12] |
DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al. An image is worth 16x16 words:Transformers for image recognition at scale[A]. International Conference on Learning Representations[C] Vienna:Springer,2021.
|
[13] |
TOUVRON H,CORD M,DOUZE M,et al. Training data-efficient image transformers & distillation through attention[A]//International Conference on Machine Learning. PMLR[C]. 2021:10347−10357.
|
[14] |
LIU Z,LIN Y,CAO Y,et al. Swin transformer:Hierarchical vision transformer using shifted windows[A]//Proceedings of the IEEE/CVF International Conference on Computer Vision[C]. Jeju,Korea:IEEE,2021:10012−10022.
|
[15] |
HU H,GU J,ZHANG Z,et al. Relation networks for object detection[A]//Proceedings of the IEEE conference on computer vision and pattern recognition[C]. 2018:3588−3597.
|
[16] |
HU H,ZHANG Z,XIE Z,et al. Local relation networks for image recognition[A]. Proceedings of the IEEE/CVF International Conference on Computer Vision[C] 2019:3464−3473.
|
[17] |
ZHANG. Z,SABUNCU M. Generalized cross entropy loss for training deep neural networks with noisy labels[J]. Advances in neural information processing systems,2018,31:2241−2252.
|
[18] |
RONNEBERGER O,FISCHER P,BROX T. U-net:Convolutional networks for biomedical image segmentation[A]. International Conference on Medical image computing and computer-assisted intervention[C] Springer,Cham,2015:234−241.
|
[19] |
BADRINARAYANAN. V,KENDALL A,CIPOLLA R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(12):2481−2495.
|
[20] |
ZHAO H,SHI J,QI X,et al. Pyramid scene parsing network[A]//Proceedings of the IEEE conference on computer vision and pattern recognition[C] Hawaii,2017:2881−2890.
|
[21] |
FU J,LIU J,TIAN H,et al. Dual attention network for scene segmentation [A]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition [C]. Tennessee,2019:3146−3154.
|
[22] |
YU C,GAO C,WANG J,et al. Bisenet v2:Bilateral network with guided aggregation for real-time semantic segmentation[J]. International Journal of Computer Vision,2021,129(11):3051−3068. doi: 10.1007/s11263-021-01515-2
|