Abstract: Sign language is an important communication tool for hearing impaired people,and accurate recognition of sign language can reduce the communication barrier between able-bodied and hearing impaired people.The performance of general deep learning recognition models is highly dependent on the collected data,which leads to poor cross-object generalization ability of the models.Therefore,this paper designs a sign language gesture recognition model with cross-object generalization capability through a transfer learning approach.Firstly,a feature extractor is used to fuse the surface electromyography (sEMG) signal and the inertial measurement unit (IMU) signal.Then,a domain adversarial training method is proposed,which can complete the adversarial training of the feature extractor and domain classifier by relying on the source domain data only,and realize the migration of feature extraction from the source domain to the target domain.Finally,domain-invariant features are used in the gesture classifier to achieve sign language gesture cross-object recognition,which improves the generalization ability of the model in this paper.Experiments show that the proposed model can improve the accuracy of sign language cross-object recognition to 85.1% on a dataset containing 200 sign language gestures with a total of 60 000 sign language samples.

Key words: Sign language gesture recognition, Feature fusion, Domain adversarial, Transfer learning, Feature transfer