volume | PIER Journals

Abstract

Dual-modality tomography integrates two different imaging technologies, allowing for the acquisition of more comprehensive sensing data. By combining information from both modalities, the accuracy of final imaging results is enhanced. However, due to the use of different physical sensitive field backgrounds by different measurement modalities, integrating information from different modalities with differing dimensions presents a challenge. To address this issue, a supervised DSCTFusion-ECA deep learning method is proposed. This method consists of four modules: initial imaging, feature extraction, feature fusion, and image reconstruction. In the feature extraction module, dense connections are utilized first to extract shallow cross-modal features, then two dual-branch feature extraction networks are utilized to separately capture modality-specific low-frequency global features and high-frequency local features for both modalities. The performance and robustness of multi-modality tomography can be effectively improved through the extraction of more comprehensive features. In the feature fusion module, Efficient Channel Attention is employed to capture channel dependencies and generate attention weights. The modal complementarity and the representation ability of key features have been enhanced, while avoiding information redundancy, thereby improving the discriminative power of the features. Simulation results show that the proposed network can fully extract and fuse features from EMT and UTT modalities, demonstrating strong robustness and generalization. Compared to the widely used U-Net network in tomography, DSCTFusion-ECA yields better reconstruction results.

1. Rahim, R. Abdul, M. H. Fazalul Rahiman, K. S. Chan, and S. W. Nawawi, "Non-invasive imaging of liquid/gas flow using ultrasonic transmission-mode tomography," Sensors and Actuators A: Physical, Vol. 135, No. 2, 337-345, 2007.

2. Peyton, A. J., M. S. Beck, A. R. Borges, J. E. De Oliveira, G. M. Lyon, Z. Z. Yu, M. W. Brown, and J. Ferrerra, "Development of electromagnetic tomography (EMT) for industrial applications. Part 1: Sensor design and instrumentation," 1st World Congress on Industrial Process Tomography, Vol. 3, 306-312, 1999.

3. Hayashi, Sanetomo, Shunsuke Yoshimoto, and Akio Yamamoto, "Noncontact 2D temperature imaging of metallic foils using electromagnetic tomography," IEEE Sensors Journal, Vol. 23, No. 16, 17942-17950, 2023.

4. Rahiman, Mohd Hafiz Fazalul, Ruzairi Abdul Rahim, Mohd Hezri Fazalul Rahiman, and Mazidah Tajjudin, "Ultrasonic transmission-mode tomography imaging for liquid/gas two-phase flow," IEEE Sensors Journal, Vol. 6, No. 6, 1706-1715, 2006.

5. Zhang, Ronghua, Hongying Fang, Qiong Zhang, Junyu Wang, Danqi Zhang, Junhua Cheng, and Wuliang Yin, "In situ damage monitoring of CFRPS by electromagnetic tomography with the compatible multitemplate supervised descent method," IEEE Transactions on Instrumentation and Measurement, Vol. 72, 1-12, 2023.

6. Zhang, Wei, Chao Tan, and Feng Dong, "Dual-modality tomography by ERT and UTT projection sorting algorithm," IEEE Sensors Journal, Vol. 20, No. 10, 5415-5423, 2020.

7. Liang, Guanghui, Ville Kolehmainen, Marko Vauhkonen, and Feng Dong, "Structural similarity driven joint reconstruction of conductivity and sound speed in EIT/UTT dual-modality tomography," Inverse Problems, Vol. 39, No. 10, 105010, 2023.

8. Steiner, Gerald, Manuchehr Soleimani, and Daniel Watzenig, "A bio-electromechanical imaging technique with combined electrical impedance and ultrasound tomography," Physiological Measurement, Vol. 29, No. 6, S63, 2008.

9. Jiang, Yandan, Manuchehr Soleimani, and Baoliang Wang, "Contactless electrical impedance and ultrasonic tomography: Correlation, comparison and complementarity study," Measurement Science and Technology, Vol. 30, No. 11, 114001, 2019.

10. Xu, Cong, Feng Dong, and Zhiqiang Zhang, "Dual-modality data acquisition system based on CPCI industrial computer," 2012 IEEE International Conference on Imaging Systems and Techniques Proceedings, 567-572, Manchester, UK, Jul. 2012.

11. Liang, Guanghui, Shangjie Ren, and Feng Dong, "Ultrasound guided electrical impedance tomography for 2D free-interface reconstruction," Measurement Science and Technology, Vol. 28, No. 7, 074003, 2017.

12. Zhang, Ronghua, Qi Wang, Huaxiang Wang, Mu Zhang, and Hongli Li, "Data fusion in dual-mode tomography for imaging oil-gas two-phase flow," Flow Measurement and Instrumentation, Vol. 37, 1-11, 2014.

13. Pusppanathan, Jaysuman, Ruzairi Abdul Rahim, Fatin Aliah Phang, Elmy Johana Mohamad, Nor Muzakkir Nor Ayob, Mohd Hafiz Fazalul Rahiman, and Chan Kok Seong, "Single-plane dual-modality tomography for multiphase flow imaging by integrating electrical capacitance and ultrasonic sensors," IEEE Sensors Journal, Vol. 17, No. 19, 6368-6377, 2017.

14. Yue, Shihong, Teresa Wu, Jian Pan, and Huaxiang Wang, "Fuzzy clustering based ET image fusion," Information Fusion, Vol. 14, No. 4, 487-497, 2013.

15. Malikov, Azamatjon Kakhramon ugli, Manuel Fernando Flores Cuenca, Beomjin Kim, Younho Cho, and Young H Kim, "Ultrasonic tomography imaging enhancement approach based on deep convolutional neural networks," Journal of Visualization, Vol. 26, No. 5, 1067-1083, 2023.

16. Shi, Qinzhen, Tianhua Zhou, Yifang Li, Chengcheng Liu, and Dean Ta, "Deep learning for TOF extraction in bone ultrasound tomography," IEEE Transactions on Computational Imaging, Vol. 8, 1063-1073, 2022.

17. Xiao, Jun, Ze Liu, Pengfei Zhao, Yong Li, and Jiwei Huo, "Deep learning image reconstruction simulation for electromagnetic tomography," IEEE Sensors Journal, Vol. 18, No. 8, 3290-3298, 2018.

18. Zhang, Wenbiao, Zexin Zhu, and Yijian Geng, "Simultaneous conductivity and permeability reconstructions for electromagnetic tomography using deep learning," IEEE Transactions on Instrumentation and Measurement, Vol. 72, 1-11, 2023.

19. Guo, Qi, Jiamin Ye, Chao Wang, and Xiao Liang, "An ill-conditioned optimization method and relaxation strategy of Landweber for EMT system based on TMR," IEEE Transactions on Instrumentation and Measurement, Vol. 70, 1-9, 2020.

20. Gan, Woon Siong, Acoustical Imaging: Techniques and Applications for Engineers, John Wiley & Sons, 2012.

21. Huang, Gao, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger, "Densely connected convolutional networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4700-4708, 2017.

22. Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," 31st Conference on Neural Information Processing Systems (NIPS 2017), Vol. 30, Long Beach, CA, USA, 2017.

23. Dinh, Laurent, Jascha Sohl-Dickstein, and Samy Bengio, "Density estimation using Real NVP," ArXiv Preprint ArXiv:1605.08803, 2016.

24. Zhou, Man, Xueyang Fu, Jie Huang, Feng Zhao, Aiping Liu, and Rujing Wang, "Effective pan-sharpening with transformer and invertible neural network," IEEE Transactions on Geoscience and Remote Sensing, Vol. 60, 1-14, 2021.

25. Wang, Qilong, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu, "ECA-Net: Efficient channel attention for deep convolutional neural networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11534-11542, 2020.

26. Lee, Chen-Yu, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu, "Deeply-supervised nets," Artificial Intelligence and Statistics, 562-570, 2015.

Mr. Jinxun Le

Prof. Ronghua Zhang

Dr. Wenying Fu

Dr. Shuqing Jia

Dr. Xuefeng Bai

Dr. Boyang Li