ZOU Liang，CHEN Zhifeng，TAN Zhiyi，et al. Scratch detection and restoration of coal photomicrograph via deep neural network[J]. Coal Science and Technology，2023，51（S2）：275−284
. DOI: 10.13199/j.cnki.cst.2023-0058| Citation: |
ZOU Liang，CHEN Zhifeng，TAN Zhiyi，et al. Scratch detection and restoration of coal photomicrograph via deep neural network[J]. Coal Science and Technology，2023，51（S2）：275−284 . DOI: 10.13199/j.cnki.cst.2023-0058
|
Quantitative analysis of coal maceral is crucial for objectively evaluating the properties and quality of coal, thereby enabling its efficient utilization. However, due to irregularities in operation or broken tools, it is easy to cause the scratch of the polished grain mounts, which affects the further automatic analysis, and the re-making of the polished grain mounts will cause the waste of manpower and resources. In view of this, in order to repair scratch areas in coal photomicrographs, a deep learning-based photomicrographs scratch detection and repair strategy is proposed. Firstly, a dual-attention model combining spatial attention and channel attention is designed for scratch detection, which fully mines the semantic information of coal photomicrographs and integrates it with U-Net semantic segmentation network to improve the accuracy of scratch detection. Then, to address the issue of texture differences between the scratch-removed areas and surrounding areas in image repair algorithms based on patch matching, a coal photomicrographs restoration network combining contextual attention and generative adversarial learning is designed. The network adaptively fills the scratched area of coal with reasonable content and improves the quality of image restoration. Experimental results show that the average pixel accuracy and mean Intersection over Union for scratch detection reached 90.93% and 83.95%, respectively, while the peak signal-to-noise ratio and structural similarity (SSIM) of the repaired images reached 43.29 dB and 99.32%, respectively. Compared to traditional patch-matching-based algorithms, these represent improvements of 8.76 dB and 1.65%, respectively, verifying the effectiveness of the proposed scratch detection and repair method.
| [1] |
白向飞. 炼焦煤性质与分类及煤岩学应用[J]. 煤质技术,2021,36(6):6−12.
BAI Xiangfei. Properties and classification of coking coal and its application in coal petrology[J]. Coal Quality Technology,2021,36(6):6−12.
|
| [2] |
王 越,丁 华,武琳琳,等. 煤岩显微组分自动识别技术现状及关键问题分析[J]. 煤质技术,2019,34(1):1−4,15. doi: 10.3969/j.issn.1007-7677.2019.01.001
WANG Yue,DING Hua,WU Linlin,et al. Study on the technical route and key factors of automatic coal maceral determination[J]. Coal Quality Technology,2019,34(1):1−4,15. doi: 10.3969/j.issn.1007-7677.2019.01.001
|
| [3] |
王培珍,翟羽佳,王 慧,等. 基于曲波变换和压缩感知的煤岩惰 质组分类[J]. 中国矿业大学学报,2019,48(5):1119−1125.
WANG Peizhen,ZHAI Yujia,WANG Hui,et al. Classfication for inertinite of coal based on curvelet transform and compressed sensing[J]. Journal of China University of Mining & Technology,2019,48(5):1119−1125.
|
| [4] |
王培珍,殷子睆,王 高,等. 一种基于 PCA 与 RBF-SVM 的煤岩显微组分镜质组分类方法[J]. 煤炭学报,2017(4):977−984.
WANG Peizhen,YIN Ziwan,WANG Gao,et al. A classification method of vitrinite for coal macerals based on the PCA and RBF-SVM[J]. Journal of China Coal Society,2017(4):977−984.
|
| [5] |
SKIBA M,MłYNARCZUK M. Identification of macerals of the inertinite group using neural classifiers,based on selected textural features[J]. Archives of Mining Sciences,2018,63(4):827−837.
|
| [6] |
MLYNARCZUK M,SKIBA M. The application of artificial intelligence for the identification of the maceral groups and mineral components of coal[J]. Computers & Geosciences,2017,103:133−141.
|
| [7] |
WANG H,LEI M,CHEN Y,et al. Intelligent identification of maceral components of coal based on image segmentation and classification[J]. Applied Sciences,2019,9(16):3245. doi: 10.3390/app9163245
|
| [8] |
O’BRIEN G,JENKINS B,OFORI P,et al. Semi-automated petrographic assessment of coal by coal grain analysis[J]. Minerals engineering,2007,20(5):428−434. doi: 10.1016/j.mineng.2006.11.006
|
| [9] |
WANG H,LEI M,LI M,et al. Intelligent estimation of vitrinite reflectance of coal from photomicrographs based on machine learning[J]. Energies,2019,12(20):3855. doi: 10.3390/en12203855
|
| [10] |
朱宪坤,吕 青,王 芳,等. 基于改进 FMM 的煤岩表面划痕与孔洞处理[J]. 煤炭技术,2016(2):308−310.
ZHU Xiankun,LYU Qing,WANG Fang,et al. Process of scratches and holes on coal surface based on improved FMM algorithm[J]. Coal Technology,2016(2):308−310.
|
| [11] |
王洪栋. 基于机器学习的煤岩显微图像分析研究[D]. 徐州:中国矿业大学,2019.
WANG Hongdong. Research on photomicrograph analysis of coal based on machine learning[D]. Xuzhou:China University of Mining & Technology,2019.
|
| [12] |
李 瑶,冷思雨,雷 萌,等. 煤岩显微图像划痕检测与去除方法[J]. 工矿自动化,2021,47(5):95−100.
LI Yao,LENG Siyu,LEI Meng,et al. Scratch detection and removal method for coal microscopic images[J]. Industry and Mine Automation,2021,47(5):95−100.
|
| [13] |
曹承瑞,刘微容,史长宏,等. 多级注意力传播驱动的生成式图像修复方法[J]. 自动化学报,2022,48(5):1343−1352.
CAO Chengrui,LIU Weirong,SHI Changhong. Generative image inpainting with attention propagation[J]. Acta Automatica Sinica,2022,48(5):1343−1352.
|
| [14] |
张 东,唐向宏,张少鹏,等. 小波变换与纹理合成相结合的图像修复[J]. 中国图象图形学报,2015,20(7):882−894. doi: 10.11834/jig.20150704
ZHANG Dong,TANG Xianghong,ZHANG Shaopeng,et al. Image inpainting based on combination of wavelet transform and texture synthesis[J]. Journal of Image and Graphics,2015,20(7):882−894. doi: 10.11834/jig.20150704
|
| [15] |
JAM J,KENDRICK C,WALKER K,et al. A comprehensive review of past and present image inpainting methods[J]. Computer vision and image understanding,2021,203:103147. doi: 10.1016/j.cviu.2020.103147
|
| [16] |
YAO F. Damaged region filling by improved criminisi image inpainting algorithm for thangka[J]. Cluster Computing,2019,22(6):13683−13691.
|
| [17] |
HEDJAZI M A,GENC Y. Efficient texture-aware multi-GAN for image inpainting[J]. Knowledge-Based Systems,2021,217:106789. doi: 10.1016/j.knosys.2021.106789
|
| [18] |
CHEN Y,ZHANG H,LIU L,et al. Research on image inpainting algorithm of improved GAN based on two-discriminations networks[J]. Applied Intelligence,2021,51(6):3460−3474. doi: 10.1007/s10489-020-01971-2
|
| [19] |
YI Z,TANG Q,AZIZI S,et al. Contextual residual aggregation for ultra high-resolution image inpainting[A]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition[C]. Seattle,USA. 2020:7508−7517.
|
| [20] |
YU J,LIN Z,YANG J,et al. Generative image inpainting with contextual attention[A]. Proceedings of the IEEE conference on computer vision and pattern recognition[C]. Salt Lake City,2018:5505-5514.
|
| [21] |
LEI M,RAO Z,WANG H,et al. Maceral groups analysis of coal based on semantic segmentation of photomicrographs via the improved U-net[J]. Fuel,2021,294:120475. doi: 10.1016/j.fuel.2021.120475
|
| [22] |
IWASZENKO S,RÓG L. Application of deep learning in petrographic coal images segmentation[J]. Minerals,2021,11(11):1265. doi: 10.3390/min11111265
|
| [23] |
RONNEBERGER O,FISCHER P,BROX T. U-net:Convolutional networks for biomedical image segmentation[C]. International conference on medical image computing and computer-assisted intervention. Springer,Cham,2015:234-241.
|
| [24] |
NIU R,SUN X,TIAN Y,et al. Hybrid multiple attention network for semantic segmentation in aerial images[J]. IEEE Transactions on Geoscience and Remote Sensing,2021,60:1−18.
|
| [25] |
BARNES C,SHECHTMAN E,FINKELSTEIN A,et al. PatchMatch:A randomized correspondence algorithm for structural image editing[J]. ACM Trans. Graph.,2009,28(3):24.
|
| [26] |
SHIN Y G,SAGONG M C,YEO Y J,et al. Pepsi++:Fast and lightweight network for image inpainting[J]. IEEE transactions on neural networks and learning systems,2020,32(1):252−265.
|
| [27] |
MOLITOR D,NEEDELL D,WARD R. Bias of homotopic gradient descent for the hinge loss[J]. Applied Mathematics & Optimization,2021,84(1):621−647.
|
| [28] |
LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[A]Proceedings of the IEEE conference on computer vision and pattern recognitionC]. Boston,USA. 2015:3431−3440.
|
| [29] |
BADRINARAYANAN V,KENDALL A,CIPOLLA R. Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence,2017,39(12):2481−2495. doi: 10.1109/TPAMI.2016.2644615
|
| [30] |
ZHAO H,SHI J,QI X,et al. Pyramid scene parsing network[A]Proceedings of the IEEE conference on computer vision and pattern recognition[C]. Honolulu,USA. 2017:2881-2890.
|
| [31] |
ZOU L,QIAO J,YU X,et al. Intelligent proximate analysis of coal based on near infrared spectroscopy and multi-output deep learning[J]. IEEE Transactions on Artificial Intelligence,2023,DOI: 10.1109/TAI.2023.3296714.
|
| [32] |
YU J,LIN Z,YANG J,et al. Free-form image inpainting with gated convolution[A]. Proceedings of the IEEE/CVF international conference on computer vision[C]. Seoul,Korea. 2019:4471-4480.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: