IGGCAS OpenIR  > 油气资源研究院重点实验室
Learning Spatio-Temporal Representations With a Dual-Stream 3-D Residual Network for Nondriving Activity Recognition
Yang, Lichao1; Shan, Xiaocai2; Lv, Chen3; Brighton, James1; Zhao, Yifan1
2022-07-01
Source PublicationIEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
ISSN0278-0046
Volume69Issue:7Pages:7405-7414
AbstractAccurate recognition of nondriving activity (NDA) is important for the design of intelligent human machine interface to achieve a smooth and safe control transition in the conditionally automated driving vehicle. However, some characteristics of such activities like limited-extent movement and similar background pose a challenge to the existing 3-D convolutional neural network based action recognition methods. In this article, we propose a dual-stream 3-D residual network, named DS3D residual network (ResNet), to enhance the learning of spatio-temporal representation and improve the activity recognition performance. Specifically, a parallel two-stream structure is introduced to focus on the learning of short-time spatial representation and small-region temporal representation. A two-feed driver behavior monitoring framework is further build to classify four types of NDAs and two types of driving behavior based on the driver's head and hand movement. A novel NDA dataset has been constructed for the evaluation, where the proposed DS3D ResNet achieves 83.35% average accuracy, at least 5% above three selected state-of-the-art methods. Furthermore, this study investigates the spatio-temporal features learned in the hidden layer through the saliency map, which explains the superiority of the proposed model on the selected NDAs.
KeywordThree-dimensional displays Vehicles Feature extraction Solid modeling Computational modeling Activity recognition Videos Action recognition automated driving nondriving related task
DOI10.1109/TIE.2021.3099254
Funding OrganizationCranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC
Language英语
Funding ProjectCranfield's EPSRC[EP/R511511/1]
Funding OrganizationCranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC ; Cranfield's EPSRC
WOS Research AreaAutomation & Control Systems ; Engineering ; Instruments & Instrumentation
WOS SubjectAutomation & Control Systems ; Engineering, Electrical & Electronic ; Instruments & Instrumentation
WOS IDWOS:000753527500089
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation statistics
Document Type期刊论文
Identifierhttp://ir.iggcas.ac.cn/handle/132A11/104927
Collection油气资源研究院重点实验室
Corresponding AuthorZhao, Yifan
Affiliation1.Cranfield Univ, Sch Aerosp Transport & Mfg, Cranfield MK43 0AL, Beds, England
2.Chinese Acad Sci, Inst Geol & Geophys, Beijing 100029, Peoples R China
3.Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore 639798, Singapore
Recommended Citation
GB/T 7714
Yang, Lichao,Shan, Xiaocai,Lv, Chen,et al. Learning Spatio-Temporal Representations With a Dual-Stream 3-D Residual Network for Nondriving Activity Recognition[J]. IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS,2022,69(7):7405-7414.
APA Yang, Lichao,Shan, Xiaocai,Lv, Chen,Brighton, James,&Zhao, Yifan.(2022).Learning Spatio-Temporal Representations With a Dual-Stream 3-D Residual Network for Nondriving Activity Recognition.IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS,69(7),7405-7414.
MLA Yang, Lichao,et al."Learning Spatio-Temporal Representations With a Dual-Stream 3-D Residual Network for Nondriving Activity Recognition".IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS 69.7(2022):7405-7414.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yang, Lichao]'s Articles
[Shan, Xiaocai]'s Articles
[Lv, Chen]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yang, Lichao]'s Articles
[Shan, Xiaocai]'s Articles
[Lv, Chen]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yang, Lichao]'s Articles
[Shan, Xiaocai]'s Articles
[Lv, Chen]'s Articles
Terms of Use
No data!
Social Bookmark/Share
Add to CiteULike Add to Connotea Add to Del.icio.us Add to Digg Add to Reddit Add to Technorati
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.