volume | PIER Journals

Abstract

In order to solve the problem that the recognition accuracy of human motion is not high when a single feature is used, a feature fusion human motion recognition method based on Frequency Modulated Continuous Wave (FMCW) radar is proposed. By preprocessing the FMCW radar echo data, the range and Doppler parameters of human motions are obtained, and the range-time feature map and Doppler-time feature map datasets are constructed. In order to fully extract and accurately identify the human motion features, the two features are fused, and then the two features maps and feature fusion spectrograms are put into the VGG16 network model based on transfer learning for identification and classification. Experimental results show that this method can effectively solve the problem of lack of information and recognition rate of single feature motion recognition, and the recognition accuracy is more than 1{\%} higher than that of the single feature recognition method.

1. Yang, L. M. and Z. H. Li, "Design of gesture recognition system towards human computer interaction," Industrial Control Computer, Vol. 33, No. 3, 18-20, Mar. 2020.

2. Zhang, Y. Y. and X. Guo, "Research and realization of dynamical gesture recognition algorithm based on kinect," Computer Technology and Development, Vol. 27, No. 12, 11-15, Aug. 2017.

3. Liu, Y., R. Y. Xie, Y. Feng, et al., "Survey on resident’s daily activity recognition in smart homes," Computer Engineering and Applications, Vol. 54, No. 7, 35-42, Jan. 2021.

4. Gao, X. W., Z. Shen, G. Y. Xu, et al., "Traffic anomaly detection based on multi-target tracking," Application Research of Computers, Vol. 38, No. 6, 1879-1883, Dec. 2021.

5. Tran, Du, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri, "A closer look at spatiotemporal convolutions for action recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6450-6459, Salt Lake City, USA, Jun. 2018.

6. Xiong, X., Y. Zheng, and S. Zhang, "Fall detection and human behavior recognition system based on long and short time memory networks and variants," Information Communications, No. 2, 65-67, Feb. 2020.

7. Sabokrou, Mohammad, Masoud Pourreza, Mohsen Fayyaz, Rahim Entezari, Mahmood Fathy, Jürgen Gall, and Ehsan Adeli, "AVID: Adversarial visual irregularity detection," 14th Asian Conference on Computer Vision, 488-505, Perth, Australia, 2018.

8. Liu, Tianliang, Qingwei Qiao, Junwei Wang, Xiubin Dai, and Jiebo Luo, "Human action recognition via spatio-temporal dual network flow and visual attention fusion," Journal of Electronics & Information Technology, Vol. 40, No. 10, 2395-2401, Aug. 2018.

9. Jiang, L. B., G. Y. Wei, and L. Che, "Human motion recognition by 77 GHz radar based on dictionary learning," Science Technology and Engineering, Vol. 20, No. 6, 2137-2324, Feb. 2020.

10. Li, Xinyu, Yuan He, and Xiaojun Jing, "A survey of deep learning-based human activity recognition in radar," Remote Sensing, Vol. 11, No. 9, 1068, 2019.

11. Lee, Jonghyeok, Sunghyun Hwang, Sungjin You, Woo-Jin Byun, and Jaehyun Park, "Joint angle, velocity, and range estimation using 2D MUSIC and successive interference cancellation in FMCW MIMO radar system," IEICE Transactions on Communications, Vol. 103, No. 3, 283-290, 2020.

12. Shrestha, Aman, Haobo Li, Julien Le Kernec, and Francesco Fioranelli, "Continuous human activity classification from FMCW radar with Bi-LSTM networks," IEEE Sensors Journal, Vol. 20, No. 22, 13607-13619, 2020.

13. Zhang, L. L., B. Liu, L. L. Qu, et al., "Human activity recognition with FMCW radar based on fusion feature convolutional neural network," Telecommunication Engineering, Vol. 62, No. 2, 147-154, Jul. 2022.

14. Wang, Yong, Jinjun Wu, Zengshan Tian, Mu Zhou, and Shasha Wang, "Gesture recognition with multi-dimensional parameter using FMCW radar," Journal of Electronics & Information Technology, Vol. 41, No. 4, 822-829, 2019.

15. Zhao, Yinan, Zihao Zhang, and Zhaolin Zhang, "Multi-angle data cube action recognition based on millimeter wave radar," 2020 Chinese Control and Decision Conference (CCDC), 749-753, Hefei, China, Aug. 2020.

16. Franceschini, Stefano, Michele Ambrosanio, Vito Pascazio, and Fabio Baselice, "Hand gesture signatures acquisition and processing by means of a novel ultrasound system," Bioengineering, Vol. 10, No. 1, 36, 2023.

17. Simonyan, Karen and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition," Computer Science, 2014.

18. Hashemi, Sajjad, Hojjat Emami, and Amin Babazadeh Sangar, "A new comparison framework to survey neural networks-based vehicle detection and classification approaches," International Journal of Communication Systems, Vol. 34, No. 14, e4928, 2021.

19. Ali, Mohamed Ashraf, Hossam E. Abd El Munim, Ahmed Hassan Yousef, and Sherif Hammad, "A deep learning approach for vehicle detection," 2018 13th International Conference on Computer Engineering and Systems (ICCES), 98-102, Egypt, Dec. 2018.

20. Qi, C., Y. Zuo, Z. Chen, and K. Chen, "Rice processing accuracy classification method based on improved VGG16 convolution neural network," Transactions of the Chinese Society of Agricultural Machinery, Vol. 52, No. 5, 301-307, Mar. 2021.

21. Zhuang, F. Z., P. Luo, Q. He, et al., "Survey on transfer learning research," Journal of Software, Vol. 26, No. 1, 26-39, Jul. 2015.

22. Liu, W. and W. Q. Ning, "Research and application of face mask wear recognition based on transfer learning," Journal of Jilin Normal University (Natural Science Edition), Vol. 44, No. 1, 96-103, Feb. 2023.

23. Zhou, K. and M. Jiang, "Research progress and prospect of small sample target recognition based on transfer learning," Aeronautical Science and Technology, Vol. 34, No. 2, 1-9, Feb. 2023.

Dr. Xiaoyu Luo

Professor Qiusheng Li