基于全局注意力机制和LSTM的连续手语识别算法

杨观赐, 韩海峰, 刘赛赛, 蒋亚汶, 李杨

包装工程(设计栏目) ›› 2022, Vol. 43 ›› Issue (8) : 28-34.

PDF(1594 KB)
PDF(1594 KB)
包装工程(设计栏目) ›› 2022, Vol. 43 ›› Issue (8) : 28-34. DOI: 10.19554/j.cnki.1001-3563.2022.08.004

基于全局注意力机制和LSTM的连续手语识别算法

  • 杨观赐, 韩海峰, 刘赛赛, 蒋亚汶, 李杨
作者信息 +

Continuous Sign Language Recognition Algorithm Based on Global Attention Mechanism and LSTM

  • YANG Guan-ci1, HAN Hai-fenga2, LIU Sai-sai3, JIANG Ya-wen3, LI Yang3
Author information +
文章历史 +

摘要

目的 为提高连续手语识别准确率,缓解听障人群与非听障人群的沟通障碍。方法 提出了基于全局注意力机制和LSTM的连续手语识别算法。通过帧间差分法对视频数据进行预处理,消除视频冗余帧,借助ResNet网络提取特征序列。通过注意力机制加权,获得全局手语状态特征,并利用LSTM进行时序分析,形成一种基于全局注意力机制和LSTM的连续手语识别算法,实现连续手语识别。结果 实验结果表明,该算法在中文连续手语数据集CSL上的平均识别率为90.08%,平均词错误率为41.2%,与5种算法相比,该方法在识别准确率与翻译性能上具有优势。结论 基于全局注意力机制和LSTM的连续手语识别算法实现了连续手语识别,并且具有较好的识别效果及翻译性能,对促进听障人群无障碍融入社会方面具有积极的意义。

Abstract

To improve the continuous sign language recognition accuracy and alleviate the communication barrier between hearing-impaired people and non hearing-impaired people, this paper proposed the continuous sign language recognition algorithm based on Global Attention Mechanism and LSTM (CSLR-GAML). The video data is preprocessed by applying inter-frame difference to eliminate redundant video frames, and then the feature sequences of the key frames are extracted by using ResNet. After that, the attention mechanism is used to update the network parameters, which is capable of obtain the global feature of sign language, and then the LSTM is employed to finish the timing sequence analysis. Finally, use the Chinese continuous sign language data set CSL to check algorithm performance. And the experimental results show that the average recognition accuracy of the proposed algorithm is 90.08%, and the average word error rate is 41.2%. Compared CSLR-GAML with other Five algorithms, the proposed CSLR-GAML has advantages in recognition accuracy and translation performance. The sontinuous sign language recognition algorithm based on Global Attention Mechanism and LSTM realizes continuous sign language recognition, and has good recognition effect and translation performance. It is of positive significance to promote the barrier free integration of hearing-impaired people into society.

关键词

手语识别;特征提取;全局注意力机制;LSTM

Key words

sign language recognition; feature extraction; global attention mechanism; LSTM

引用本文

导出引用1
杨观赐, 韩海峰, 刘赛赛, 蒋亚汶, 李杨. 基于全局注意力机制和LSTM的连续手语识别算法[J]. 包装工程. 2022, 43(8): 28-34 https://doi.org/10.19554/j.cnki.1001-3563.2022.08.004
YANG Guan-ci, HAN Hai-fenga, LIU Sai-sai, JIANG Ya-wen, LI Yang. Continuous Sign Language Recognition Algorithm Based on Global Attention Mechanism and LSTM[J]. Packaging Engineering. 2022, 43(8): 28-34 https://doi.org/10.19554/j.cnki.1001-3563.2022.08.004

基金

国家自然科学基金(62163007);贵州省科技计划项目(黔科合平台人才[2020]6007,黔科合支撑[2021]一般439,JXCX[2021]001)

PDF(1594 KB)

Accesses

Citation

Detail

段落导航
相关文章

/